diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 1d39aed..d3d5ba1 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -17,7 +17,7 @@ { "name": "ccaf", "source": "./ccaf", - "description": "CCAF (Claude Certified Architect – Foundations) mock-exam readiness gate. /ccaf:mock-exam assembles a 60-question case-study-framed mock with machine-enforced domain weighting — every question generated fresh per attempt and independently verified — administers it 4 per screen with resumable progress, and scores it on the real 100–1000 band with a 720 pass line plus a per-domain breakdown." + "description": "CCAF (Claude Certified Architect – Foundations, exam code CCAR-F) readiness gate, aligned to exam guide v1.0. /ccaf:mock-exam assembles a 60-item case-study-framed mock of single-answer questions, with machine-enforced domain weighting, answer-position spread, and per-objective tagging — every item generated fresh per attempt and independently verified — administers it 4 per screen with resumable progress, and scores it on the real 100–1000 band with a 720 pass line plus per-domain and per-objective breakdowns. /ccaf:prepare teaches the 30 task statements turn by turn; /ccaf:practice drills chosen domains." }, { "name": "discovery", diff --git a/.gitignore b/.gitignore index 63d77d8..3a0094a 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,12 @@ .claude/bee-insights/ .claude/ccaf-exam.local.md .claude/ccaf-exam.local.answers.md +.claude/ccaf-practice.local.md +.claude/ccaf-practice.local.answers.md + +# Reference PDFs (e.g. the CCAF exam guide) are read while authoring the plugin's +# self-authored content but must never be redistributed with it. +ccaf/*.pdf + +# Editor workspace settings +.vscode/ diff --git a/ccaf/.claude-plugin/plugin.json b/ccaf/.claude-plugin/plugin.json index c2c0347..d2ff276 100644 --- a/ccaf/.claude-plugin/plugin.json +++ b/ccaf/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "ccaf", - "version": "0.1.1", - "description": "CCAF (Claude Certified Architect – Foundations) mock-exam readiness gate. Assembles a 60-question case-study-framed mock with machine-enforced domain weighting — every question generated fresh per attempt and independently verified — administers it 4 per screen with resumable progress, and scores it on the real 100–1000 band with a 720 pass line plus a per-domain breakdown. Run /ccaf:mock-exam.", + "version": "0.3.0", + "description": "CCAF (Claude Certified Architect – Foundations, exam code CCAR-F) readiness gate, aligned to exam guide v1.0. Assembles a 60-item case-study-framed mock of single-answer questions with machine-enforced domain weighting, answer-position spread, and per-objective tagging — every item generated fresh per attempt and independently verified — administers it 4 per screen with resumable progress, and scores it on the real 100–1000 band with a 720 pass line plus per-domain and per-objective breakdowns. Learn with /ccaf:prepare, drill with /ccaf:practice, gate with /ccaf:mock-exam.", "author": { "name": "Incubyte" }, diff --git a/ccaf/CLAUDE.md b/ccaf/CLAUDE.md index 6b8c094..cc23315 100644 --- a/ccaf/CLAUDE.md +++ b/ccaf/CLAUDE.md @@ -1,8 +1,13 @@ # CCAF: Claude Certified Architect – Foundations mock exam -A self-serve readiness gate for the CCAF certification. `/ccaf:mock-exam` administers a faithful -mock exam and reports a scaled /1000 score with the 720 pass line, so a candidate can check -readiness before booking the real (paid) exam. +A self-serve readiness gate for the CCAF certification (exam code `CCAR-F`). `/ccaf:mock-exam` +administers a faithful mock exam and reports a scaled /1000 score with the 720 pass line, so a +candidate can check readiness before booking the real (paid) exam. + +Aligned to exam guide **v1.0 (effective July 2026)**. The guide's published *facts* are encoded +here; none of its prose is. Every stem, option, explanation, scenario brief, task-statement +description, and exercise is self-authored for this plugin — keep it that way when editing, and do +not commit the guide PDF (it is gitignored). ## Layout @@ -13,9 +18,10 @@ readiness before booking the real (paid) exam. - `skills/ccaf-exam/SKILL.md` — the assemble → administer → score engine. - `skills/ccaf-practice/SKILL.md` — the domain-selection → assemble → administer → score engine for focused domain practice; uses a separate state file so it never conflicts with `/ccaf:mock-exam`. - `agents/ccaf-check-author.md` — mini-agent the tutor spawns to author one scenario check at a time. -- `data/ccaf-blueprint.md` — domains, weights, scenarios, the syllabus, scope lists, scoring. Shared curriculum for both commands. -- `data/ccaf-question-bank.md` — 12 self-authored reference questions; style/difficulty anchors only, never served in an exam (helper-enforced). -- `scripts/ccaf-exam.sh` — silent state helper (init / get / record / blanks / audit / score / clear); never use Write/Edit on the attempt files. `init` takes one payload (with keys) and splits it: questions file (write-once) + answers file (hot, ~60 lines) — so `record` rewrites only the tiny answers file and `get` output is key-free. Guards: `init` validates the payload — and, for 60-question exams, enforces the blueprint composition (domain quotas 16/11/12/12/9, 4 scenarios in contiguous sections each headed by its own `[[CASE:]]` brief — so a screen's brief always matches its questions — non-degenerate key spread) — and refuses to overwrite an in-progress attempt (unless `--force`); `record` takes one or more `--q/--answer` pairs atomically (one call per screen) and requires an in-progress attempt; `score` cross-validates the pair and requires `--partial` to score with unanswered questions. All writes serialize through a directory lock (stale locks are stolen), so mid-exam `record` calls run **in the background** while the next screen shows; the final screen records in the foreground and completion is verified before scoring. +- `data/ccaf-blueprint.md` — domains, weights, item composition, the 30 task statements (D1.1–D5.6), scenarios, case-study briefs, scope lists, scoring. Shared curriculum for all three commands, and the only authority for item content. +- `data/ccaf-question-bank.md` — 30 self-authored reference questions, one per task statement, each tagged with its `task:`; style/difficulty anchors only, never served in an exam (helper-enforced). All single-answer. Their key letters lean toward A — content is the reference, letters are noise. +- `data/ccaf-prep-guide.md` — study routes, four hands-on exercises, and certification logistics (fee, retakes, recertification). Read by the tutor; read by the exam skills only for post-result guidance, never for item content. +- `scripts/ccaf-exam.sh` — silent state helper (init / get / record / blanks / audit / score / clear); never use Write/Edit on the attempt files. `init` takes one payload (with keys) and splits it: questions file (write-once) + answers file (hot, ~60 lines) — so `record` rewrites only the tiny answers file and `get` output is key-free. Guards: `init` validates the payload — one `answer_key` letter A–D per item, and a `task:` tag that exists and belongs to the item's own domain — and, for 60-item exams, enforces the blueprint composition (domain quotas 16/11/12/12/9; 4 scenarios in contiguous sections each headed by its own `[[CASE:]]` brief — so a screen's brief always matches its items; a key spread within a sixth to a third per letter) — and refuses to overwrite an in-progress attempt (unless `--force`); `record` takes one or more `--q/--answer` pairs atomically (one call per screen), uppercases each answer so a lowercase free-text reply matches, and requires an in-progress attempt; `score` cross-validates the pair and requires `--partial` to score with unanswered items. All writes serialize through a directory lock (stale locks are stolen), so mid-exam `record` calls run **in the background** while the next screen shows; the final screen records in the foreground and completion is verified before scoring. - `scripts/tests/ccaf-exam.test.sh` — shell test harness for the data files + helper logic. ## Conventions @@ -27,5 +33,25 @@ readiness before booking the real (paid) exam. so the two modes never interfere. - Untimed, honor-system, fully offline. Self-serve: nothing is reported or persisted as history. The real exam's 120-minute budget is stated once up front for self-pacing; no time is ever captured. +- **Single-answer items only** — four options A–D, exactly one correct. This is a deliberate + divergence: the real exam also uses multiple-response items. The blueprint records the reasoning + and the consequence (a score here is, if anything, optimistic), and the README's fidelity table + states it. Never reintroduce a response count without changing all three. - Scoring: `scaled = 100 + 15 × correct` (linear over the real 100–1000 band); pass = 720 (≥ 42/60). + Results always show per-domain correct/total **and percent**, labelled diagnostic-only — pass/fail + is the total scaled score, as on the real criterion-referenced exam. +- Every item carries a `task:` tag (`D1.1`–`D5.6`) that `init` validates against its `domain:`, and + `score` aggregates misses by task statement so a result names the objectives to revisit. This goes + beyond what the real score report shows; the validation exists because a mistagged item would send + a candidate to study the wrong thing. Task-statement counts per domain: D1 7, D2 5, D3–D5 6 each. - The scaled score is an honest estimate, never presented as Anthropic's proprietary equating curve. +- Process creation can be slow on some machines (Windows + AV in particular), so hot paths in the + helper avoid gratuitous subprocesses — `normalize_answer` is pure bash, and each validation check + (`check_items`, `check_composition_questions`, `check_key_spread`) is a single `awk` pass rather + than a loop of `grep`s. **Keep it that way.** This is not micro-optimising: + an earlier grep-per-check version made the test suite slow enough to exhaust Cygwin's fork table + mid-run (`fork: Resource temporarily unavailable`), so the suite could not finish at all on + Windows. `record` runs once per exam screen and `validate_pair` runs on every score. +- Even now the suite can occasionally report spurious failures on Windows when a fork fails inside a + fixture's `init` (the symptom is a whole section failing with "not found" / empty field reads). + Re-run before investigating; if it reproduces, it is real. A clean run is `100 passed, 0 failed`. diff --git a/ccaf/README.md b/ccaf/README.md index d60d65a..c08705a 100644 --- a/ccaf/README.md +++ b/ccaf/README.md @@ -2,13 +2,32 @@ CCAF is a Claude Code plugin that helps you **prepare for** and **mock-test** against the **Claude Certified Architect – Foundations** exam — right in your terminal — so you get an honest readiness verdict before you book the real (paid) exam. -**Why this exists.** Incubyte is having everyone get CCAF-certified, with a simple rule: *practice first, and only sit the real exam once you can reliably score 720+.* Instead of every engineer hand-rolling a quiz from the exam-guide PDF, this plugin makes that learn-and-gate flow a couple of commands, consistent for the whole team. All content is self-authored or publicly corroborated — no Anthropic exam material is reproduced. +**Why this exists.** Incubyte is having everyone get CCAF-certified, with a simple rule: *practice first, and only sit the real exam once you can reliably score 720+.* Instead of every engineer hand-rolling a quiz from the exam-guide PDF, this plugin makes that learn-and-gate flow a couple of commands, consistent for the whole team. + +Aligned to the published exam guide **v1.0 (effective July 2026, exam code `CCAR-F`)**: the same 60 items, the same five weighted domains, and the same 30-objective index. One format divergence — this plugin serves single-answer items only, where the real exam also uses multiple-response items ([why](#how-it-mirrors-the-real-exam)). **Every question, scenario brief, task-statement description, and exercise in this plugin is self-authored** — the guide's published *facts* (weights, item counts, the objective index, backend tool names, policies) are reflected, but none of its prose and no exam item is reproduced. + +## The real exam at a glance + +| | | +| --- | --- | +| Exam code | `CCAR-F` | +| Items | 60 — multiple-choice **and** multiple-response (each item states how many responses to select). *This plugin serves single-answer only — see below.* | +| Structure | 4 scenarios drawn from a bank of 6 | +| Time limit | 120 minutes | +| Delivery | Proctored by Pearson VUE — online or at a test centre | +| Passing score | Scaled **720** on a 100–1000 range | +| Reporting | Pass/fail + scaled score + percent correct per domain | +| Fee | $125 USD per attempt | +| Validity | 12 months, renewable with a free non-proctored assessment | +| Retakes | 14 / 30 / 90-day waits after successive failures; max 4 attempts per rolling year | + +That retake ladder is the argument for this plugin: a failed attempt costs $125 *and* two weeks. Full logistics — booking, ID, accommodations, appeals, recertification — are in `data/ccaf-prep-guide.md`. **What you get — three commands, one loop:** - **`/ccaf:prepare`** — a conversational coach that teaches the syllabus *turn by turn*: one concept at a time, a knowledge check every turn, pace and difficulty adapting to you. The formative side — build readiness. - **`/ccaf:practice`** — focused domain practice: pick the domains you want to drill, choose 10, 20, or 30 questions, and get a per-domain bar-chart score. The targeted side — close specific gaps. -- **`/ccaf:mock-exam`** — a 60-question weighted mock that mirrors the real exam's structure, a scaled `/1000` score with the **720** pass line, and a per-domain breakdown. The summative side — test readiness. +- **`/ccaf:mock-exam`** — a 60-item weighted mock that mirrors the real exam's structure, a scaled `/1000` score with the **720** pass line, and a per-domain and per-objective breakdown. The summative side — test readiness. ``` /ccaf:prepare ──build readiness──► /ccaf:practice ──close gaps──► /ccaf:mock-exam @@ -23,17 +42,29 @@ CCAF is a Claude Code plugin that helps you **prepare for** and **mock-test** ag | Real exam rule | This mock | | --- | --- | -| 60 questions | ✅ 60 per attempt | -| 5 domains, weighted 27 / 18 / 20 / 20 / 15 | ✅ same distribution (D1=16, D2=11, D3=12, D4=12, D5=9) | +| 60 items | ✅ 60 per attempt | +| 5 domains, weighted 27 / 18 / 20 / 20 / 15 | ✅ same distribution (D1=16, D2=11, D3=12, D4=12, D5=9), machine-enforced | +| 30 task statements (D1.1–D5.6) | ✅ every item is written against one, and spread across them | | 4 of 6 scenarios, chosen at random | ✅ same | -| Questions organized around case studies | ✅ 4 case-study sections; the case brief stays visible on every screen | -| Single-select, 1 correct + 3 distractors | ✅ same | +| Items organized around case studies | ✅ 4 case-study sections; the case brief stays visible on every screen | +| Multiple-choice **and** multiple-response | ❌ **single-answer only** — every item has four options and one correct answer. Deliberate; see the note below | | No penalty for guessing | ✅ unanswered = incorrect | | Answers revisable before submit | ✅ ask to change any earlier answer mid-exam | | Scaled 100–1000, pass = 720 | ✅ `scaled = 100 + 15 × correct`; pass at ≥ 42/60 | -| 120-minute time limit | ❌ untimed by design — it shows the 120-min / ~2-min-per-question budget up front so you can self-pace | +| Percent correct per domain on the report | ✅ shown, and labelled diagnostic-only — pass/fail is the total scaled score | +| — | ➕ **beyond the real report:** per-task-statement misses, so the result names the exact objectives to revisit, not just the weak domain | +| — | ➕ **beyond the real report:** answer-position spread is machine-enforced, so no letter is over-represented | +| 120-minute time limit | ❌ untimed by design — it shows the 120-min / ~2-min-per-item budget up front so you can self-pace | + +Three things it does **not** replicate, and why. + +**Multiple-response items.** The guide gives the item format as multiple-choice *and* multiple-response, with each item stating how many responses to select. This mock serves **single-answer items only** — a deliberate choice for this plugin, not an oversight. Two consequences worth holding onto: you will not rehearse a format that is scored all-or-nothing (one right and one wrong scores the same as zero right), and because single-answer items are the easier format, **a score here is if anything optimistic**. Treat 720 as the floor of readiness rather than a comfortable margin. `data/ccaf-prep-guide.md` explains how to train the underlying habit — decide the status of every option instead of stopping at the first strong one — without the format itself. + +**The scaled-scoring curve.** Anthropic's is proprietary and unpublished, so the score here is a transparent linear *estimate*, clearly labelled as such. + +**The 120-minute clock.** Deliberately absent — the mock is resumable and honor-system. Time yourself if you want realistic conditions. -Two things it does **not** replicate, on purpose: Anthropic's proprietary scaled-scoring curve (impossible — the score here is a transparent, linear *estimate*, clearly labelled) and the 120-minute clock (deliberate — the mock is resumable and honor-system; time yourself if you want realistic conditions). Treat 720+ as a readiness signal, not a guarantee. +Treat 720+ as a readiness signal, not a guarantee. ## The flow @@ -41,20 +72,22 @@ Two things it does **not** replicate, on purpose: Anthropic's proprietary scaled /ccaf:mock-exam | v - [ ASSEMBLE ] Pick 4 of 6 case studies; generate all 60 questions fresh + [ ASSEMBLE ] Pick 4 of 6 case studies; generate all 60 items fresh | (anchored to the reference bank, each independently verified, | A–D shuffled); group into 4 case-study sections; freeze the - | exam to ~/.claude/ccaf-exam.local.md. The 16/11/12/12/9 - | domain split is machine-enforced at write time (a - | mis-weighted exam is refused), and the composition is shown - | to you up front. + | exam to ~/.claude/ccaf-exam.local.md. The 16/11/12/12/9 domain + | split, the answer-position spread, and each item's task-statement + | tag are machine-enforced at write time (a mis-weighted exam, a + | skewed key spread, or a tag that does not belong to its own domain + | is refused), and the composition is shown to you up front. v - [ ADMINISTER ] 4 questions per screen, case brief always visible. Each screen - | saves atomically in the background while the next one shows — - | no save-wait between screens. Quit any time; re-run to resume. + [ ADMINISTER ] 4 items per screen, case brief always visible. Each screen saves + | atomically in the background while the next one shows — no + | save-wait between screens. Quit any time; re-run to resume. v - [ SCORE ] Scaled /1000, PASS/FAIL at 720, per-domain breakdown, and an - honest "this is an estimate" disclaimer. + [ SCORE ] Scaled /1000, PASS/FAIL at 720, per-domain percent breakdown, + the objectives you missed, and an honest "this is an + estimate" disclaimer. ``` ## Install @@ -94,7 +127,7 @@ Untimed and conversational. It's **stateless** — Claude Code's native session /ccaf:practice ``` -Select one or more domains to focus on, then choose how many questions you want (10, 20, or 30). Questions are drawn proportionally from the selected domains using the real blueprint weights. At the end you get a per-domain bar chart — no overall score or PASS/FAIL verdict — and a targeted recommendation for any domain that needs work. +Select one or more domains to focus on, then choose how many questions you want (10, 20, or 30). Items are drawn proportionally from the selected domains using the real blueprint weights. At the end you get a per-domain bar chart with percentages, plus the objectives you missed, — no overall score or PASS/FAIL verdict, since a partial session isn't weighted like a real form — and a targeted recommendation for any domain that needs work. ```bash /ccaf:practice fresh @@ -112,7 +145,7 @@ Discard any in-progress or completed practice attempt and start a new domain sel /ccaf:mock-exam ``` -Answer the questions four to a screen. When you finish, you get your scaled score, a PASS/FAIL at 720, and a domain-by-domain breakdown so you know where you're weak. A FAIL points you back to `/ccaf:prepare ` for targeted practice. +Answer the items four to a screen — each has four options and exactly one correct answer. When you finish, you get your scaled score, a PASS/FAIL at 720, a domain-by-domain percent breakdown, and the specific task statements you missed. A FAIL points you back to `/ccaf:prepare ` for targeted practice. ```bash /ccaf:mock-exam fresh @@ -126,10 +159,12 @@ Discard any in-progress or completed attempt and assemble a brand-new exam. ## How scoring works -- Raw `correct` = questions answered correctly (unanswered count as incorrect). +- Raw `correct` = items answered correctly (unanswered count as incorrect). - `scaled = 100 + 15 × correct` — a linear mapping over the real 100–1000 band (equivalently `100 + round(correct ÷ 60 × 900)`; since `900 ÷ 60 = 15`, no rounding is needed). - **Pass** iff `scaled ≥ 720`, i.e. **≥ 42 of 60** correct. -- A per-domain breakdown (correct / total per D1–D5) accompanies every result. +- A per-domain breakdown (correct / total **and percent** per D1–D5) accompanies every result, mirroring the real score report. Like the real exam, those percentages are diagnostic only — pass/fail is decided by the total scaled score. +- Every item is tagged with the task statement it tests, so the result also names **which objectives** you missed (`D5.2 Escalation and ambiguity resolution — 0/2`) rather than only which domain. The real score report doesn't do this; it's the most actionable thing a mock can give you, since it converts a weak domain into a short reading list. Tags are validated at write time — a tag must exist and belong to its own domain — so the report can't send you to study the wrong objective. +- The real exam is **criterion-referenced**: you're measured against a fixed standard set by a formal standard-setting study, not graded against other candidates. 720 is a fixed bar. ## What's inside @@ -151,8 +186,9 @@ ccaf/ │ └── ccaf-practice/ │ └── SKILL.md # practice engine: domain-select → assemble → administer → score (internal) ├── data/ -│ ├── ccaf-blueprint.md # public exam mechanics + self-authored syllabus, scenarios, scoring -│ └── ccaf-question-bank.md # 12 self-authored reference questions (anchors only — never served) +│ ├── ccaf-blueprint.md # exam mechanics + item composition + self-authored 30-task-statement syllabus, scenarios, scoring +│ ├── ccaf-question-bank.md # 30 self-authored single-answer reference questions, one per task statement (anchors only — never served) +│ └── ccaf-prep-guide.md # study routes, 4 hands-on exercises, certification logistics ├── scripts/ │ ├── ccaf-exam.sh # silent state helper (init / get / record / score / clear) │ └── tests/ @@ -164,9 +200,10 @@ ccaf/ ## Notes -- **Question sourcing.** Every question in every attempt is **generated fresh** from the blueprint syllabus and passes an independent verifier (re-solve cold, plausible distractors, shuffled positions) before being served. The 12 self-authored questions in the bank are style/difficulty anchors only — they never appear in an exam (machine-enforced), because the bank ships in this repo with answers, and re-serving readable questions would inflate your readiness signal. -- **How `prepare` teaches.** The tutor reads the same blueprint as its curriculum, teaches one task statement per turn, and verifies by retrieval. Its apply-to-scenario checks are authored on demand by a small `ccaf-check-author` subagent (built from the syllabus anti-patterns), keeping the main teaching thread lean. Nothing is written to disk. -- **Roadmap.** v1 generates everything per attempt anchored to a small reference bank; growing a larger verified anchor bank and verifying the full lifecycle end-to-end are the next steps (tracked in `docs/specs/ccaf-mock-exam.md`). Per-domain focused practice is now available via `/ccaf:practice`. +- **Question sourcing.** Every item in every attempt is **generated fresh** from the blueprint syllabus and passes an independent verifier (re-solve cold, exactly one defensible option, plausible distractors, shuffled positions) before being served. The 30 self-authored questions in the bank are style/difficulty anchors only — they never appear in an exam (machine-enforced), because the bank ships in this repo with answers, and re-serving readable questions would inflate your readiness signal. +- **How `prepare` teaches.** The tutor reads the blueprint as its curriculum, teaches one of the 30 task statements per turn, and verifies by retrieval. Its apply-to-scenario checks are authored on demand by a small `ccaf-check-author` subagent (built from the syllabus anti-patterns), keeping the main teaching thread lean. At domain boundaries it can assign one of the prep guide's four hands-on exercises instead of another quiz. Nothing is written to disk. +- **Roadmap.** Everything is generated per attempt against a 30-question anchor bank, one per task statement. Next: end-to-end lifecycle verification — nobody has sat a full 60-item mock yet. If the single-answer decision is ever revisited, the multiple-response machinery is in this repo's history. Tracked in `docs/specs/ccaf-mock-exam.md`. +- **Provenance.** The plugin encodes the exam guide's published *facts* (item count, weights, the objective index, the scenarios' backend tool names and targets, and program policies) and nothing else from it: every stem, option, explanation, scenario brief, task-statement description, and exercise is written for this plugin. The guide PDF itself is deliberately not committed. ## License diff --git a/ccaf/agents/ccaf-check-author.md b/ccaf/agents/ccaf-check-author.md index aa15b6e..b1c899b 100644 --- a/ccaf/agents/ccaf-check-author.md +++ b/ccaf/agents/ccaf-check-author.md @@ -1,11 +1,11 @@ --- name: ccaf-check-author description: >- - Authors ONE fresh single-select knowledge-check question for a given CCAF task - statement and difficulty, then returns it with a compact answer key. Spawned by - the ccaf-tutor skill (the /ccaf:prepare engine) for apply-to-scenario checks so the - main teaching thread stays lean — it never sees the authoring rationale, only the - question. Not user-invokable directly. + Authors ONE fresh single-answer knowledge-check question for a given CCAF task + statement and difficulty, then returns it with a compact answer key. Spawned by the ccaf-tutor + skill (the /ccaf:prepare engine) for apply-to-scenario checks so the main teaching + thread stays lean — it never sees the authoring rationale, only the question. Not + user-invokable directly. Context: The tutor just taught D1.4 (programmatic enforcement vs prompt-based ordering) and wants a scenario check. @@ -24,6 +24,7 @@ description: >- Difficulty is expressed by making distractors closer and the tradeoff finer, not by adding out-of-scope trivia. The agent stays strictly in-scope per the blueprint. + model: inherit color: cyan tools: ["Read", "Glob"] @@ -41,28 +42,32 @@ question plus a key the tutor can grade against. **Authority (read it; do not invent content):** - `${CLAUDE_PLUGIN_ROOT}/data/ccaf-blueprint.md` — the syllabus. Find the task statement - (e.g. `D1.4`), read its Knowledge-of / Skills-in bullets, and read the **anti-patterns** - flagged there. The blueprint's in-scope / out-of-scope lists are hard boundaries. -- `${CLAUDE_PLUGIN_ROOT}/data/ccaf-question-bank.md` — the 12 self-authored seed questions. - Use them as **style and difficulty anchors only**. Never reproduce one verbatim; author fresh. + (e.g. `D1.4`) in the 30-task-statement section, read what it says a candidate must be able to do + and the exact identifiers it names, and read its domain's **common mistakes** list — that is your + distractor source. The blueprint's in-scope / out-of-scope lists are hard boundaries. +- `${CLAUDE_PLUGIN_ROOT}/data/ccaf-question-bank.md` — the 30 self-authored reference questions, + one per task statement, each tagged with the `task:` it covers. Use them as + **style, difficulty, and format anchors only**. Your task statement always has exactly one anchor — + read it first, then author something different that tests the same objective. Never reproduce one + verbatim. **Authoring process:** 1. Locate the requested task statement in the blueprint and extract the concept it tests plus - its flagged anti-patterns. + its domain's flagged anti-patterns. 2. Frame ONE realistic production scenario in the requested scenario context. Keep the stem to a few sentences — a concrete situation a working architect would hit. -3. Write exactly four single-select options (A–D): one clearly-correct answer and three - plausible distractors **built from the syllabus anti-patterns** for that task statement - (the wrong answers a partial-knowledge candidate would actually pick). +3. Write exactly four options (A–D): one clearly correct, and three plausible distractors **built + from the syllabus anti-patterns** for that task statement (the wrong answers a partial-knowledge + candidate would actually pick). 4. Stay strictly in-scope. Never test an out-of-scope topic. Test judgment, not trivia or API-parameter memorization. 5. **Calibrate difficulty** by how close the distractors sit to the correct answer and how fine the tradeoff is — NOT by adding obscure facts: - - `easy` — one obviously-right option; distractors are clearly weaker. + - `easy` — the right option is obvious; distractors are clearly weaker. - `medium` — distractors are reasonable-sounding; the learner must apply the concept. - - `hard` — all four are defensible on a quick read; only the correct one survives the - tradeoff the task statement turns on. -6. Shuffle the correct option to a varied A–D position (don't default to A). + - `hard` — all four are defensible on a quick read; only the correct one survives the tradeoff + the task statement turns on. +6. Shuffle the correct option to a varied A–D position — do not default to A. **Output format — return EXACTLY this, nothing before or after:** @@ -75,19 +80,19 @@ B)