Anagram challenges accept only one of several correct answers
anagram.grade() compares the response against the single word the challenge was generated from, so a solver who answers a different real English word made of exactly the same letters is marked wrong.
Given the letters of star, "rats" is a correct answer. The gate rejects it:
letters "tasr" -> answered "rats" -> failed / attempt budget exhausted
That is against the live service at icaptcha.gitlawb.com. I hit it twice more organically while driving a model through the ladder: it answered "palm" for lamp and "bared" for bread, lost both runs, and was right both times.
How much of the pool this affects
I checked all 70 words in WORDS against the hunspell en_US dictionary, with affix rules expanded so inflected forms like rats and notes are included. 31 of 70 words (44%) have at least one other dictionary word with the same letters.
It is worst where the gate is easiest, because short words collide more:
| word length |
ambiguous |
of |
| 3 |
6 |
10 |
| 4 |
7 |
10 |
| 5 |
4 |
10 |
| 6 |
8 |
10 |
| 7 |
2 |
10 |
| 8 |
1 |
10 |
| 9 |
3 |
10 |
Restricted to alternates a solver would plausibly produce (dropping acronyms and proper nouns like usn, cpu, iran), 25 words are affected:
cat: act dog: god map: amp, pam
door: odor lamp: palm leaf: flea
star: arts, rats, tars tree: rete
bread: bared, beard, debar cloud: could
stone: notes, onset, tones bridge: begird
castle: cleats flower: fowler, reflow
forest: fortes, foster, softer garden: danger, gander, ranged
orange: onager planet: platen silver: livers, sliver
kitchen: chetnik, thicken picture: cuprite
treasure: austerer orchestra: carthorse
telephone: phenetole wonderful: underflow
cloud / could and garden / danger are the ones I would expect to bite most often.
Why it matters here specifically
iCaptcha's premise is that a capable reasoner passes and a script does not. This inverts that on the affected levels: the solver who genuinely unscrambled the letters is the one who gets failed, at random, depending on which valid word they thought of first. Since one miss escalates difficulty and can end a run, it costs exactly the users the gate is built to admit.
Suggested fix
Make the pool unambiguous rather than teaching the grader to accept alternates.
Accepting alternates sounds simpler but it weakens the gate. I tried it first: once a second valid answer exists, a scramble that happens to be that answer is printed in the prompt, and the existing reshuffle guard only rejects a scramble equal to the source word. Echoing the prompt letters back then passes 5.9% of level-1 challenges, measured over 50,000 generated challenges, against 0.000% today. A grader that accepts more strings is the wrong direction for a service whose job is to say no.
Replacing the 31 ambiguous words with words that have no anagram keeps the challenge well posed: exactly one correct answer, no grader change, no list to maintain. There are plenty of unambiguous candidates at every length (sky, bird, beach, guitar, morning, airplane, waterfall all check out).
Happy to send a PR for that, or for whichever direction you prefer. The dictionary check I used is a short script and I can include it so the pool can be re-audited whenever words are added.
Anagram challenges accept only one of several correct answers
anagram.grade()compares the response against the single word the challenge was generated from, so a solver who answers a different real English word made of exactly the same letters is marked wrong.Given the letters of
star, "rats" is a correct answer. The gate rejects it:That is against the live service at
icaptcha.gitlawb.com. I hit it twice more organically while driving a model through the ladder: it answered "palm" forlampand "bared" forbread, lost both runs, and was right both times.How much of the pool this affects
I checked all 70 words in
WORDSagainst the hunspellen_USdictionary, with affix rules expanded so inflected forms likeratsandnotesare included. 31 of 70 words (44%) have at least one other dictionary word with the same letters.It is worst where the gate is easiest, because short words collide more:
Restricted to alternates a solver would plausibly produce (dropping acronyms and proper nouns like
usn,cpu,iran), 25 words are affected:cloud/couldandgarden/dangerare the ones I would expect to bite most often.Why it matters here specifically
iCaptcha's premise is that a capable reasoner passes and a script does not. This inverts that on the affected levels: the solver who genuinely unscrambled the letters is the one who gets failed, at random, depending on which valid word they thought of first. Since one miss escalates difficulty and can end a run, it costs exactly the users the gate is built to admit.
Suggested fix
Make the pool unambiguous rather than teaching the grader to accept alternates.
Accepting alternates sounds simpler but it weakens the gate. I tried it first: once a second valid answer exists, a scramble that happens to be that answer is printed in the prompt, and the existing reshuffle guard only rejects a scramble equal to the source word. Echoing the prompt letters back then passes 5.9% of level-1 challenges, measured over 50,000 generated challenges, against 0.000% today. A grader that accepts more strings is the wrong direction for a service whose job is to say no.
Replacing the 31 ambiguous words with words that have no anagram keeps the challenge well posed: exactly one correct answer, no grader change, no list to maintain. There are plenty of unambiguous candidates at every length (
sky,bird,beach,guitar,morning,airplane,waterfallall check out).Happy to send a PR for that, or for whichever direction you prefer. The dictionary check I used is a short script and I can include it so the pool can be re-audited whenever words are added.