Skip to content

docs: update ci:eval:lint:vally command references for grader-lineage check #2944

Description

@github-actions

Component

Documentation

Bug Description

commit 56c30bc (#2934) changed the ci:eval:lint:vally npm script in package.json from:

pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/

to:

pwsh -NoProfile -File scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf && npm run lint:eval-grader-lineage && vally lint --eval-spec evals/

adding a new lint:eval-grader-lineage step (Build-GraderLineageMap.ps1 -Check) in between. Two documentation pages still show the pre-change command composition and do not mention the new grader-lineage check or script:

  • docs/architecture/workflows.md (script/workflow table row for ci:eval:lint:vally)
  • docs/contributing/evals-ci.md (script/workflow table and the prose describing ci:eval:lint:vally runs vally lint --eval-spec evals/ ...)

Expected Behavior

Both pages should describe the current three-step composition of ci:eval:lint:vally, including the lint:eval-grader-lineage step and a brief mention of what scripts/evals/Build-GraderLineageMap.ps1 -Check validates (grader lineage mapping), so the documented command matches package.json.

Steps to Reproduce

  1. See package.json change in commit 56c30bc / PR feat(accessibility): strengthen NVDA evidence and validation #2934.
  2. Compare against docs/architecture/workflows.md line ~445 and docs/contributing/evals-ci.md line ~280, which still describe only the Build-AgentBehaviorSpec.ps1 -WhatIf && vally lint --eval-spec evals/ composition.

Additional Context

Related files:

  • package.json (ci:eval:lint:vally, lint:eval-grader-lineage scripts)
  • scripts/evals/Build-GraderLineageMap.ps1
  • docs/architecture/workflows.md
  • docs/contributing/evals-ci.md

Generated by Documentation Update Check · auto · 84.6 AIC · ⌖ 1.73 AIC · ⊞ 9.5K ·

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readydocumentationImprovements or additions to documentationneeds-triageRequires triage and prioritization

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions