Skip to content

Add installable validation skill and manual eval suite - #128

Open
strider2038 wants to merge 10 commits into
mainfrom
feat/validation-usage-skill
Open

strider2038 wants to merge 10 commits into
mainfrom
feat/validation-usage-skill

Conversation

@strider2038

@strider2038 strider2038 commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Adds the installable muonsoft-validation skill to help coding agents compose Go entity/DTO validation with correct accumulation, nested paths, optional values, dependency checks, and error handling. The README documents installation, and the skill includes portable executable examples.

The manual OpenCode evaluator compares the same tasks with and without the skill in isolated workspaces. It retains seven main scenarios and three transfer scenarios, checks submissions against v0.19.0 and the current library snapshot, and records correctness, paired token usage, completion evidence, and reproducibility hashes. Run artifacts stay outside the checkout; compact runs support detached reporting and continuation. Repository-local review instructions keep code review separate from test scores.

Validation:

  • bash scripts/test-all.sh, including lint and race tests.
  • All 41 deterministic runner tests.
  • Main and transfer material checks: executable skill examples, passing references, failing starters, and behavioral mutations on both library versions.
  • Public and repository-local skill validation.

The final model evaluation will run against the cleaned branch before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant