Skip to content

A generated example never changes the learner's meaning; a broken sentence gets another - #902

Merged
mircealungu merged 6 commits into
masterfrom
generated-sentence-keeps-card
Oct 25, 2026
Merged

mircealungu merged 6 commits into
masterfrom
generated-sentence-keeps-card

Conversation

@mircealungu

@mircealungu mircealungu commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Builds on #900 (merged).

Problem. validate_and_fix checks the meaning in the word's preferred bookmark, and that is sometimes a generated example. A FIX there moved the learner's word to a different meaning, one that fits the example. Since 1 June, 658 fixes were judged on generated examples. That's 58% of the generated examples the validator judged, against 13% of learners' own sentences, and 395 of those fixes changed the word itself (e.g. mennesker → umanned from "Dronen er mennesker…"). The example was generated for the meaning, so when the two disagree it's the example that's wrong: "selv" = "even" must not become "selv om".

The same goes for a sentence the validator calls BROKEN_SENTENCE (#900). It says nothing about the word, which may fit any other sentence.

Change. In both cases (any non-VALID verdict in a generated example, or BROKEN_SENTENCE in any sentence), _try_another_sentence:

  • leaves the learner's word on its meaning;
  • if the sentence was a generated example, records it in the new example_sentence_misfit table;
  • judges the meaning again in another sentence: another sentence of the learner's own (a reader fragment too: it is the only place a wrong meaning can be corrected, and the fragment swap keeps it out of exercises), else the next unused example. When the learner's own sentence was the broken one, examples only. At most 2 sentences in all (worst case 6 validator calls per word).
  • With no sentence left, gives the word up. A meaning that only ever failed as FIX in examples is logged as [word_not_fit]; broken sentences as [broken_sentence]. The meaning itself is not touched.

Examples recorded as misfits are never offered again: the fragment swap (replace_fragment_with_example) and /alternative_sentences both skip them.

A FIX in the learner's own sentence still changes the meaning, as before, because that's the sentence they actually looked the word up in.

Deploy: run tools/migrations/26-10-25--add_example_sentence_misfit.sql before deploying. It creates a new table, so no existing table gets rebuilt, but the example queries read from it.

Tests: 6 new DB tests:

  • a FIX in an example keeps the meaning and drops the example;
  • with no sentence left, the word is given up on without touching the meaning;
  • a VALID in an example is kept;
  • a broken own sentence goes to an example, not to another own sentence, and the word stays in study;
  • a broken own sentence with no example gives up without touching the meaning;
  • after a bad example, a reader fragment of the learner's own is judged.

Full suite: 1261 passed, 1 failed. The failure is test_cloze_alternatives::test_a_translation_verdict_is_not_a_cloze_verdict, which passes on its own (3/3) and next to the new tests, and doesn't touch this code. It looks order- or fixture-dependent.

Not in this PR:

  • The 395 learners' words already moved to a different meaning by generated examples need reviewing and possibly restoring. That's a prod data job; I'll show the query and the batch before any write.
  • Exercises can still show other generated example bookmarks via the arrows, and those are never checked. Checking every sentence shown is the larger design.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

ArchLens detected architectural changes in the following views:
diff

@mircealungu mircealungu changed the title A generated example never changes the learner's card A generated example never changes the learner's meaning Oct 25, 2026
@mircealungu
mircealungu force-pushed the generated-sentence-keeps-card branch from e86b520 to 24f4f0c Compare October 25, 2026 02:20
@mircealungu mircealungu changed the title A generated example never changes the learner's meaning A generated example never changes the learner's meaning; a broken sentence gets another Oct 25, 2026
@mircealungu
mircealungu force-pushed the generated-sentence-keeps-card branch 6 times, most recently from e4b8b17 to c9e64cf Compare October 25, 2026 12:07
…tence gets another

validate_and_fix judges the meaning in the word's preferred bookmark, which
is sometimes a generated example. A FIX there moved the learner's word to a
meaning that fit the example: since 1 Jun 2026, 658 fixes were judged on
generated examples (58% of the examples judged, against 13% of learners'
own sentences) and 395 of them changed the word. The example was generated
for the meaning, so when the two disagree, the example is what is wrong
("selv" = "even" must not become "selv om").

A BROKEN_SENTENCE says nothing about the word either. So for both: the
meaning is judged again in another of the learner's own sentences, else in
the next unused example, at most 3 sentences in all; with none left the word
is given up on (a FIX in examples gives up as WORD_NOT_FIT). A generated
example that failed is recorded in example_sentence_misfit and not offered
again: the fragment swap and /alternative_sentences skip it.

Migration: tools/migrations/26-10-25--add_example_sentence_misfit.sql (a new
table; must run before this deploys, the example queries read it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mircealungu
mircealungu force-pushed the generated-sentence-keeps-card branch from c9e64cf to 362aa17 Compare October 25, 2026 12:10
@mircealungu
mircealungu changed the base branch from validator-card-fits-sentence to master October 25, 2026 12:19
mircealungu and others added 5 commits October 25, 2026 13:34
…ntence → examples

- MAX_SENTENCES_TRIED 3 → 2: the worst case per word is 6 validator
  calls, not 9.
- A reader fragment is context for the meaning, not for study: falling back
  to the learner's own sentences skips fragments.
- When the learner's own sentence is the broken one, the word goes to
  generated examples only, not to another of their look-ups. Since 1 Oct
  88% of validated words already had examples; the rest give up until a new
  look-up (generating them on the spot is roadmap item 6b).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… method

The reason is LLM text, which may hold 4-byte characters that utf8 rejects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Newer examples come from newer prompts and models, so they are the likeliest
to fit the meaning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… example link

- Examples are shared across learners, so two validations can mark the same
  one at once; the insert runs in a savepoint and a duplicate is ignored,
  instead of failing scheduling with an IntegrityError.
- /alternative_sentences builds on usable_examples_query instead of
  repeating the misfit and newest-first rule.
- The example behind a bookmark is its first example link:
  ExampleSentenceContext.find_by_bookmark uses .one(), which raises when a
  bookmark carries more than one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mircealungu
mircealungu merged commit ae592c2 into master Oct 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant