Skip to content

Treat superseded Smart CI plans as non-red in shadow - #2415

Merged
Chris0Jeky merged 4 commits into
mainfrom
issue-2327/smart-ci-cancelled-plan
Sep 3, 2026
Merged

Treat superseded Smart CI plans as non-red in shadow#2415
Chris0Jeky merged 4 commits into
mainfrom
issue-2327/smart-ci-cancelled-plan

Conversation

@Chris0Jeky

Copy link
Copy Markdown
Owner

Summary

  • Treat a trusted cancelled Plan job as non-red only while Smart CI is in shadow mode.
  • Keep the receipt explicit: wouldFail: true, plan-job-cancelled, and an observation-exclusion note.
  • Preserve fail-closed behavior in enforce mode and for failure, timed_out, skipped, invalid, or mismatched plans.
  • Do not change workflow topology, policy, lane selection, runner choice, branch protection, or execution mode.

Advances #2327

Failure model

The shadow workflow always instantiates Smart CI / Required Gate and passes the trusted needs.plan.result. A concurrency-superseded Plan job previously became plan-job-failed, which made the stable gate context red even though the replacement run was the useful observation. The new result is green only in shadow mode and remains deliberately unusable as enforcement or recall evidence.

Verification

  • Failing first: focused cancellation regression failed before the implementation.
  • node --test --test-name-pattern='superseded cancelled plan|gate fails closed on missing' scripts/ci/smart-ci/plan.test.mjs: 2 passed.
  • node --test scripts/ci/smart-ci/*.test.mjs: 86 passed.
  • CLI missing-receipt probe through evaluate-gate.mjs: exit 0, ok=true, wouldFail=true, plan-job-cancelled, null head identity.
  • node scripts/check-github-ops-governance.mjs: passed.
  • node scripts/check-docs-governance.mjs: passed.
  • git diff --check: passed.
  • Portable Actionlint bootstrap contract: 5 passed.
  • Broader node --test scripts/ci/*.test.mjs: 262 passed; the two installed-binary tests were not run locally because ACTIONLINT_BIN, ACTIONLINT_SHELLCHECK_BIN, and ACTIONLINT_PYFLAKES_BIN are not installed/configured on this machine. Exact-head hosted CI supplies those tools.

Residual

This does not enable Smart CI selection or satisfy #2327's observation, landed-verifier, branch-current, consolidation, or maintainer-settings acceptance. The separate moving-base observation residual remains out of scope.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-03T04:11:31.868640Z acee74c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e30f067949

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/smart-ci/lib/plan.mjs
@Chris0Jeky

Copy link
Copy Markdown
Owner Author

Independent review disposition

Reviewed head: e30f0679495cb78de1108c4280b4177bfb9a16c4
Fix head: 80f3fbfc632b61dfa4ee6a4367160abf9ff49e54

  • HIGH, resolved: plan-job-cancelled was absent from ci-run.v1 and made the emitted receipt schema-invalid. The enum now admits the code, and the cancellation regression asserts that every emitted failure code is allowed by the receipt schema. The regression failed before the schema fix and passes after it.
  • MEDIUM, declined as nonblocking hardening: the implementation already leaves independent invalid-plan and SHA-mismatch planner failures red, which the reviewer directly probed. A combined future-regression fixture is not needed to close a current correctness defect in this bounded fix round.
  • LOW, declined: the evaluator header describes plan-job-failed behavior; cancellation now has a distinct plan-job-cancelled code. No executable instruction is wrong.
  • Recall fixture, no change: recall already excludes a non-success Smart CI workflow run. This slice does not relax that collector.

Scoped reproof at the fix head:

  • focused gate tests: 2/2 passed
  • full Smart CI tests: 86/86 passed
  • GitHub operations governance: passed
  • docs governance: passed
  • git diff --check: passed

The fix changes only the declared receipt schema and its regression, not gate logic or the risk boundary. Exact-head hosted checks restarted after the push.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d86464d40

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/smart-ci/lib/plan.mjs
@Chris0Jeky

Copy link
Copy Markdown
Owner Author

Current-base reconciliation

origin/main advanced to a32b4818df04fffeab348ed63012fafd6bd18631 through PR #2410, so the branch was merged forward instead of relying on the stale-base checks. New exact head: 6d86464d40bf6f3cffb15168269a38921604856d.

The base change touches only backend archive/proposal files and has no path overlap with this Smart CI/schema diff. Current-base proof:

  • full Smart CI tests: 86/86 passed
  • GitHub operations governance: passed
  • docs governance: passed
  • current-base git diff --check: passed
  • fresh interaction review: no CRITICAL/HIGH findings; effective PR diff remains the three declared Smart CI/schema files

Hosted exact-head checks restarted. The prior full required run was correctly cancelled by this superseding push and is not being used as merge evidence.

@Chris0Jeky

Copy link
Copy Markdown
Owner Author

Final current-base review at acee74cef73e848a288a91ac81042aa851135882 against 468d76dc8e7a7859324c40e6369c3caae57fe647: no CRITICAL/HIGH correctness, security, data-loss, or base-interaction finding.

Reviewed effective diff: ci/schemas/ci-run.v1.schema.json, scripts/ci/smart-ci/lib/plan.mjs, scripts/ci/smart-ci/plan.test.mjs. Local proof: Smart CI 86/86, focused plan tests 41/41, schema parse and git diff --check pass. Shadow cancellation remains explicit wouldFail=true; enforce cancellation and invalid cancelled plans remain red.

Known residual: cancellation provenance cannot distinguish manual from concurrency cancellation. It is classified MEDIUM/non-blocking while the policy is shadow-only and is retained under #2327 as a pre-registration requirement. Hosted exact-head CI is still running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant