fix: harden forge command prompt against DCP context compression - #55
Conversation
643e740 to
b389f4b
Compare
b389f4b to
8cccfa5
Compare
- Fix keyword casing: Always -> ALWAYS (RFC 2119 consistency) - Add TestForgeMD_StructuralHardening with 7 subtests covering all spec scenarios from forge-prompt-structure.md - Fix intent drift in proposal.md: document skip_review prohibition as a new capability instead of claiming 'None' - File issue unbound-force#63 for code-level skip_review enforcement - Add .uf/feedback/ to .gitignore
yvonnedevlinrh
left a comment
There was a problem hiding this comment.
Reviewed using /uf.review-pr 55 with a custom-built UF binary (latest changes merged into main) after running /uf.init to scaffold the agent kit.
MEDIUM
A1: Inconsistent casing between Critical Invariants and Rules
Critical Invariants (line 15) reads ALWAYS create a forge, even for small tasks. while Rules (line 46) reads Always create a forge, even for small tasks. Per the PR's own FR-004: "MUST use explicit RFC 2119 language." The Rules instance predates this PR but the inconsistency was introduced by adding the ALWAYS variant in Critical Invariants without updating Rules to match.
C5: No Review Council APPROVE on revised code
Per AGENTS.md: "there must be minimal to no code changes between the council's APPROVE verdict and the PR submission." The original council review returned CHANGES_REQUESTED. The subsequent revisions added 169 lines of new test code and modified forge.md. No follow-up APPROVE verdict appears in the review history — only COMMENTED reviews with empty bodies. A fresh council APPROVE on the revised code is required before merge.
LOW
A3: No Closes #47 keyword
Issue #47 (all four fragilities addressed by this PR) has no Closes #47 / Fixes #47 keyword in the PR body or commits. It won't auto-close on merge.
Manual Testing SummaryTested by checking out Tests PerformedTest 1 — Small task via
Test 2 — Larger task via
Test 3 — Post-compression context probe: After compression triggered in test 2, asked the agent to describe the forge rules and
Findings
ConclusionThe structural changes and automated tests are solid. The prompt hardening addresses a real risk — during long forge sessions, the agent relies on what survives in compressed context, not the original file. However, the hardening may be necessary but not sufficient. In both manual tests, agents skipped the forge workflow entirely regardless of prompt language, suggesting tool-level enforcement (e.g., requiring |
|
Addressed all feedback from @yvonnedevlinrh's review: A1 [MEDIUM]: Applied — normalized A2/E [LOW]: Applied — updated proposal.md: Impact/Testing section now references A3 [LOW]: Applied — added T1/F [LOW]: Applied — fixed comment to say 'same-level heading' which matches the implementation. T2/G [LOW]: Applied — changed Step7OrderingConstraint from OR to AND logic. Test now independently asserts both C5: Acknowledged — will run review council on the revised code before requesting re-review. Re: manual testing observation — agreed that tool-level enforcement is the right next step. That's tracked in #63 for All changes in commit e2dece1. |
Review Council Report — PR #55Branch: Council Members
Consolidated Verdict: REQUEST CHANGESThe prompt restructuring is well-executed and all 4 fragilities from #47 are addressed. The spec artifacts are exemplary. However, the structural hardening tests have a systematic gap: they verify that standalone sections were removed but do not verify that content was actually inlined at the correct location. For a restructuring PR, this is the core regression risk — content can be silently dropped while tests pass. Blocking Findings (must fix)B1 [HIGH] Tests verify absence without verifying presence of inlined contentSource: Testing The
Fix: Add 3 subtests — one per inlined section — verifying the content is present at its new location. B2 [HIGH] No test for MCP tool reference completenessSource: Testing Task 2.3 requires verifying all MCP tool references are preserved after restructuring. This is the strongest regression guard for a restructuring change. Currently not tested. Fix: Add a subtest listing all expected tool references and verifying each appears in forge.md. B3 [MEDIUM] RedundantReviewConstraint allows 2-location coverage; spec requires 3Source: Testing The test requires Non-blocking Findings (address or acknowledge)N1 [HIGH] Worker prompt contradicts new Critical InvariantSource: Adversary, Guard, Scribe
Action: File a tracking issue for worker prompt hardening. Reference from design.md. N2 [HIGH]
|
| File | Change | Focus |
|---|---|---|
.gitignore |
Add .uf/feedback/ |
standard |
internal/agentkit/agentkit_test.go |
7 structural subtests for forge.md invariants | test-quality |
internal/agentkit/content/commands/forge.md |
Restructure: Critical Invariants → Workflow → Rules | documentation |
openspec/changes/forge-dcp-hardening/* (5) |
Full OpenSpec change artifacts | documentation |
Review Council — 5 Divisor agents, consolidated by coordinator. Pre-flight PASS, Gaze Q1 Safe.
Review Council — Re-review Verdict: APPROVEAll blocking findings from the previous council have been resolved in commit Blocking Findings Resolution
Divisor Results (Re-review)
Pre-flight: PASS
Non-blocking items (tracked separately)
Verdict: All 5 divisors APPROVE. Ready for merge. |
- Fix keyword casing: Always -> ALWAYS (RFC 2119 consistency) - Add TestForgeMD_StructuralHardening with 7 subtests covering all spec scenarios from forge-prompt-structure.md - Fix intent drift in proposal.md: document skip_review prohibition as a new capability instead of claiming 'None' - File issue unbound-force#63 for code-level skip_review enforcement - Add .uf/feedback/ to .gitignore
3d491e4 to
2ae8c1d
Compare
sonupreetam
left a comment
There was a problem hiding this comment.
Review Council (6 Divisor agents)
Verdict: REQUEST CHANGES — two inline findings below, plus mechanical blockers.
Findings
MEDIUM — skip_review:true naming risk: Explicitly prohibiting skip_review: true in the prompt teaches agents that this bypass parameter exists. Consider rephrasing as a positive constraint or filing a follow-up for tool-level enforcement.
MEDIUM — Inconsistent RFC 2119 casing: The Rules line "Coordinator orchestrates, workers execute" has no RFC 2119 keyword, while the Critical Invariants version states "workers MUST NOT call forge_complete". Consider adding a MUST keyword for consistency with FR-004.
LOW — Missing Closes #47: PR description should include Closes #47 for auto-closing the tracked issue on merge.
Mechanical blockers
- Needs rebase onto main (PR #54 merged, test file has shifted)
- GitHub review status from @yvonnedevlinrh still shows REQUEST_CHANGES on older commit
Merge order recommendation
Merge 2nd (after #54 which is already merged). Rebase before merge.
|
Addressed all findings from sonupreetam's 2nd review (commit 634c791): MEDIUM — skip_review naming risk: Rephrased as positive constraint. Line now reads: "The review gate is mandatory — MEDIUM — Inconsistent RFC 2119 casing: Added LOW — Missing Closes #47: Already present at bottom of PR body (may have been added after review). Verified still present. Rebase: Branch is already up to date with upstream/main (merge base is Pre-flight: |
yvonnedevlinrh
left a comment
There was a problem hiding this comment.
LGTM, only findings are nitpicky items.
Restructure forge.md to prevent the review-before-complete quality gate from being lost under context compression: - Add Critical Invariants section at top of file - Embed ordering constraint directly in step 7 text - Move review rule to 1st position in Rules section - Inline strategy selection into decompose step - Inline error recovery into monitoring step - Merge Completion section into step 8 sub-items - Add skip_review prohibition to invariants All original content preserved (17 MCP tool references verified). No behavioral changes to the forge workflow. Closes unbound-force#47 Assisted-by: claude-opus-4-6 Generated with AI assistance (claude-opus-4-6)
- Fix keyword casing: Always -> ALWAYS (RFC 2119 consistency) - Add TestForgeMD_StructuralHardening with 7 subtests covering all spec scenarios from forge-prompt-structure.md - Fix intent drift in proposal.md: document skip_review prohibition as a new capability instead of claiming 'None' - File issue unbound-force#63 for code-level skip_review enforcement - Add .uf/feedback/ to .gitignore
…recision
- A1: Normalize 'Always' to 'ALWAYS' in Rules section (forge.md:46)
- E/A2: Update proposal.md testing section to reflect TestForgeMD_StructuralHardening
- F/T1: Fix comment to match implementation ('same-level' not 'same/higher level')
- G/T2: Strengthen Step7OrderingConstraint to require MUST AND ordering signal
- A3: Add Closes unbound-force#47 to PR body
- B1: Add subtests verifying inlined content at correct steps: StrategyInlinedInStep3, ErrorRecoveryInlinedInStep6, CompletionInlinedInStep8 - B2: Add AllToolReferencesPresent subtest verifying all 17 MCP tool references are present in forge.md - B3: Fix RedundantReviewConstraint to require AND logic — all 3 locations (Critical Invariants, Workflow, Rules) must contain the review-before-complete constraint
… 2119 keyword to Rules Address sonupreetam's 2nd review findings: - MEDIUM: Rephrase skip_review prohibition to avoid teaching agents the bypass parameter name. Now uses positive constraint language: 'The review gate is mandatory — forge_complete MUST NOT be called until forge_review has passed for every worker. NEVER bypass the review gate.' - MEDIUM: Add 'workers MUST NOT call forge_complete' to the Rules section coordinator/worker rule for RFC 2119 consistency with Critical Invariants. - Update test: Rename SkipReviewProhibition → ReviewGateMandatory, verify positive constraint language (MUST NOT, NEVER, 'review gate'). - Update proposal.md: Reflect new capability description.
634c791 to
e8dbf37
Compare
- Fix keyword casing: Always -> ALWAYS (RFC 2119 consistency) - Add TestForgeMD_StructuralHardening with 7 subtests covering all spec scenarios from forge-prompt-structure.md - Fix intent drift in proposal.md: document skip_review prohibition as a new capability instead of claiming 'None' - File issue #63 for code-level skip_review enforcement - Add .uf/feedback/ to .gitignore
sonupreetam
left a comment
There was a problem hiding this comment.
Review Council — Re-review
Both MEDIUM findings from the prior council review are addressed:
-
skip_review naming — Rephrased as positive constraint ("The review gate is mandatory"). No longer names the bypass parameter. Test renamed to
ReviewGateMandatorywith updated assertions. Proposal updated. -
RFC 2119 casing — Rules section now uses
MUST NOTkeyword ("workers MUST NOT callforge_complete"), consistent with Critical Invariants.
Remaining mechanical blockers
- Needs rebase onto main (PR #54 merged)
Closes #47still missing from PR description
Verdict: APPROVE (pending rebase)
Summary
Restructures the
/forgecommand prompt (internal/agentkit/content/commands/forge.md) to survive DCP context compression, fixing the quality gate vulnerability where the review-before-complete ordering constraint was lost under compression.Addresses all four fragilities identified in #47:
Additionally adds a mandatory review gate constraint to the Critical Invariants section using positive constraint language (avoids naming bypass parameters).
How to Test
How to Demo
Read
internal/agentkit/content/commands/forge.mdand verify the restructured prompt:Key Files Changed
internal/agentkit/content/commands/forge.mdopenspec/changes/forge-dcp-hardening/proposal.mdopenspec/changes/forge-dcp-hardening/design.mdopenspec/changes/forge-dcp-hardening/specs/forge-prompt-structure.mdopenspec/changes/forge-dcp-hardening/tasks.mdThis PR was generated by /uf.finale (AI-assisted).
Closes #47