Skip to content

[codex] add std mutation testing - #68

Merged
greenthree merged 2 commits into
mainfrom
codex/mutation-testing
Aug 9, 2026
Merged

[codex] add std mutation testing#68
greenthree merged 2 commits into
mainfrom
codex/mutation-testing

Conversation

@greenthree

Copy link
Copy Markdown
Owner

What changed

  • Add probhub mutation and probhub mutate for conservative C++ standard-solution mutation testing.
  • Add comparison-boundary, boolean-negation, and integer-boundary operators with masked comments/literals, stable IDs, plan hashes, and bounded evidence.
  • Run each mutant from an isolated snapshot through the existing Validator/Judge/process control; formal sources, PDFs, ZIPs, metadata, and Manifests are never modified.
  • Integrate mutation state and kill/survivor summaries into reports, Skill guidance, CLI reference, README, CHANGELOG, and a committed mutation Fixture.
  • Preserve old evidence on cancellation, timeout, infrastructure failure, input changes, evidence-size overflow, lock contention, or publish failure.
  • Keep accepted sources at their original relative path in snapshots and forward the remaining overall deadline to each Judge invocation.

Why

Mutation testing supplements expected-matrix, independent-solution, and stress evidence by exposing boundary changes that current data does not distinguish. It is intentionally not a proof of correctness and is not a build hard gate.

Validation

  • npm run check: 678 passed, 30 skipped
  • npm run pack:check: probhub=220, probhub-skill=5
  • Skill quick validation passed
  • python -m py_compile probhub\\mutation.py probhub\\cli.py probhub\\reporting.py passed
  • python -m unittest tests.test_mutation tests.test_report tests.test_workspace_fixtures.FixtureCatalogTests -v: 23 passed
  • Current Core against phtest: all nine problems linted successfully; phtest remains clean and unpushed
  • git diff --check passed

@greenthree
greenthree marked this pull request as ready for review August 9, 2026 06:32
@greenthree
greenthree merged commit 09197cc into main Aug 9, 2026
21 of 22 checks passed
@greenthree
greenthree deleted the codex/mutation-testing branch August 9, 2026 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant