Repository navigation
CI: fail when a prompt's text changes but its version does not - #913
Merged
Merged
Conversation
Stored LLM outputs are stamped with the prompt version that wrote them, and stale ones are found by querying it. A prompt edited in place keeps its name, so the stamp then names two texts: that is how ~900 audio lessons could only be found by a regex over their scripts. tools/check_prompt_versions.py hashes the text of every versioned prompt (the -vN lesson files, the prompt modules with a PROMPT_VERSION, each EXAMPLE_GENERATION_PROMPTS entry) into prompt_versions.lock.json. The check fails on a changed hash under an unchanged version; --update adds new versions and refuses to rehash an existing one. Only string literals are hashed, so comments and docstrings do not count. Standard library only, so the workflow needs no install step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
ArchLens - No architecturally relevant changes to the existing views |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stored LLM outputs are stamped with the prompt version that wrote them (
AIGenerator), and stale ones are found by querying that version. A prompt edited in place keeps its name, so the stamp then names two different texts. That is how ~900 audio lessons could only be found by a regex over their scripts (May 2026).tools/check_prompt_versions.pyhashes the text of every versioned prompt intozeeguu/core/llm_services/prompts/prompt_versions.lock.json: the-vN.txtlesson files, the prompt modules with aPROMPT_VERSION, and eachEXAMPLE_GENERATION_PROMPTSentry.--updaterecords new versions and refuses to rehash an existing one..github/workflows/prompt-versions.ymlruns it on PRs and pushes to master. Standard library only, no install step.Not covered: unversioned prompts (article simplification, the old unversioned lesson files) and
assess_summary_v1, whose text lives insidesimplification_and_classification.py.Tested locally: an in-place edit of a lesson file, the cloze prompt, or the example
v4entry fails; a comment or docstring change passes; an edit with a version bump asks for--update.🤖 Generated with Claude Code