Promotion: develop → main (pinned b1d5ab98 — the fixes that make a v* tag safe) - #594
Merged
Conversation
…nd the gate called that a pass (#590) fix(mcp,release): --version starts the server instead of answering, and the gate called that a pass FOUR DEFECTS. THE FIRST WAS VISIBLE IN THE RELEASE LOG; EACH OF THE OTHERS WAS FOUND BY TESTING THE FIX FOR THE ONE BEFORE IT. 1. `evolith-mcp --version` boots the MCP server. parseArgs read the command as `args.find((a) => !a.startsWith('-')) ?? 'serve'`. Every flag spelling starts with `-`, so it fell through to 'serve'. Measured against the PUBLISHED 1.3.2: stdin closed -> exit 0, stdout EMPTY (the stdio transport takes EOF and leaves) stdin open -> never returns (killed at 10s, printed nothing) The second is what a terminal, a doctor script or a CI probe does, and asking a tool its version is the first thing anyone does after installing it. `evolith-mcp version` — the positional — worked the whole time, which is why nothing noticed. `--version`, `-v`, `-V`, `--help` and `-h` are now commands by intent, and `--help` prints the USAGE string that already existed and had no caller. 2. A flag's VALUE was read as the command. Found because a test written for defect 1 failed for an unrelated reason: `evolith-mcp --transport http` resolved command to 'http' and exited 1 with `Unknown command: http`, since `http` is the first token not starting with `-`. Argv is now classified in one pass that skips the value each value-taking flag consumes, and both answers are read off that. This also means `--api-key -v` keeps `-v` as the key instead of printing a version. 3. The smoke gate's boot check asserted only the exit status. const res = run(process.execPath, [bin, '--version'], { cwd: treeDir }); if (res.status !== 0) fail([...]); console.log(`✓ the installed binary boots: --version prints ${String(res.stdout).trim()}`); It PRINTED stdout it never inspected. Release run 31986300098 logged, verbatim: ✓ the installed binary boots: --version prints An empty observable, reported as a pass — while defect 1 was already on the registry. Now: a 30s timeout, so a binary that hangs fails instead of hanging CI; a non-empty assertion; and the printed version must CONTAIN the version its own manifest declares, which is what keeps it from passing on any non-empty string. Exercised against the real script with four fixtures: answers correctly -> exit 0 ✓ answers --version: 1.3.0 (matches manifest 1.3.0) prints nothing -> exit 1 `--version` exited 0 and printed NOTHING on stdout wrong version -> exit 1 printed "9.9.9", which does not contain 1.3.0 hangs -> exit 1 did not answer within 30s (30s elapsed) The second fixture is exactly the case the release passed. 4. In `--tree` mode the gate decided whether to boot from the WRONG TREE. Found because fixtures 2-4 above passed when they should have failed. `boot` is computed from `pkgRoot/dist/main.js` — this workspace's local build — and then handed to a verifyTree call that inspects a completely different directory. On a checkout that has not been built locally, `boot` is false, the boot check never runs, and the guard prints a pass anyway. Whether the strongest check runs depended on unrelated local state. In `--tree` mode it is now decided from the tree under test. WHAT IS AND IS NOT FIXED HERE The published `@beyondnet/evolith-mcp@1.3.2` still has defects 1 and 2 — npm versions are immutable and this does not republish. It ships on the next version bump. Defects 3 and 4 are in the gate and take effect on the next release run. Related and NOT addressed: the anti-vacuous specifier scan compares against the WORKSPACE manifest rather than the tree's, which is why a `--tree` fixture must declare this repo's four siblings to be accepted at all. Same class as defect 4, different call site. Verification: mcp-server suite 601 tests / 65 suites green. The 11 new parseArgs cases fail 10-to-8 against the shipped parser and pass 18/18 against this one. Guards 43, 03 and 01 exit 0. CI: 32 SUCCESS, 2 SKIPPED, 0 failing.
…actly as designed (#591) The canary's gate assertion was exempted on one symptom: the published MCP server shipped no ruleset corpus (split out as GT-705), so it could not produce a verdict to assert. The exemption was keyed on `installedPackageShipsNoCorpus()` -- a property of the INSTALLED TARBALL, not a version number and not a date. GT-705 shipped as `mcp@1.3.2`. Run 31987205590 against the registry, cli@1.3.1 + mcp@1.3.2, from a throwaway npm prefix with no repository on the resolution path: ✓ the documented `evolith` command is published — bin/evolith present ✓ `init` completes on a clean directory — evolith.yaml written ✓ `validate --format json` returns an ADR-0073 envelope carrying a verdict — failed, 41/159 rules checked ✓ the published MCP server answers a tools/call with a REAL GATE VERDICT — verdict asserted by gate-verdict.assert.js The log contains ZERO occurrences of "exempt". Nobody had to remember to delete anything, and that is the whole point: a blanket skip would still be green today, with the defect fixed by accident and the assertion never having run. This is what the row meant by keeping the assertion and exempting only the exact symptom. All five criteria now MET, including the two that measurement had to correct first -- `--help` is a weak oracle (every published version answers `--version` with exit 0, including the one GT-625 recorded as broken), and the row's named red fixture `cli@1.2.0` is not broken at all, so falsifiability was proven against `cli@1.1.0`. Board: 672/703 done, 2 in progress. Closure record added with its measurements. Verified: 08 / 41 / 57 / 66 / 04 / 01 green, 46 at a fixed point. Signed-off-by: aarroyo <beyondnet.peru@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…s read outputs they never declared (#592) fix(ci): publish-npm asks the registry before publishing, and two jobs read outputs they never declared Closes #569. 1. A HARMLESS RE-RUN MANUFACTURED A BUG REPORT. `publish-npm` ran a bare `npm publish --provenance --access public` with no check that the version was absent. Any re-run against an already-published version — a retry after an unrelated flake, a workflow_dispatch, a re-pushed tag — died on "You cannot publish over the previously published versions". That failure reached `failure-notification`, which files a "Release Pipeline Failed" issue. Three accumulated that way (#492, #552, #553) before anyone read one, on a tracker that is among the first things a visitor sees. The publish is now gated on an explicit question asked BEFORE it: if [ "$(npm view "$NAME@$VERSION" version 2>/dev/null || true)" = "$VERSION" ]; then published=true -> ::notice:: already on the registry, skipping. Not a failure. Idempotence belongs before the publish, as a question with an answer, not after it as a swallowed error. `continue-on-error` or `|| true` ON THE PUBLISH would make a genuine failure indistinguishable from this one — the same defect wearing a different hat, and the issue says so explicitly. The `|| true` here is on a QUERY, where a non-zero exit means "not found", and the answer is then compared rather than ignored. Exercised against the live registry, both directions, under `set -euo pipefail`: @beyondnet/evolith-cli@1.3.0 -> published=true (on npm) @beyondnet/evolith-cli@9.9.9 -> published=false (never published) @beyondnet/evolith-mcp@1.3.2 -> published=true (published tonight) @beyondnet/does-not-exist@1.0.0 -> published=false (no such package) The two false cases are the ones that matter: they prove the query does not abort the step. 2. AN ECHO IS NOT A CONFIRMATION. The old final step was: echo "✅ Published @beyondnet/evolith-cli@${{ needs.release-gate.outputs.version }} to NPM" It reported success with no evidence — and printed it with an EMPTY version, because of defect 3. Replaced with a registry poll (5 attempts, 10s apart) that fails the job when npm does not hold what was just published, matching what npm-release.yml already does. 3. TWO JOBS READ `needs.<job>` FOR A JOB THEY DID NOT DECLARE. `needs.<job>` resolves only for jobs listed in that job's `needs`. `publish-npm` referenced `needs.release-gate.outputs.version` with `needs: build-and-test`, so the version was the empty string. Sweeping for the CLASS rather than the instance found a second, worse one in the same file: `upload-assets` interpolates `needs.release-gate.outputs.tag_name` into the GitHub Release's `tag_name` AND `name`, with `needs: [package-binaries, smoke-test, smoke-test-functional]`. That publishes a Release named "Release " against an empty tag. Both declared now. A sweep over every workflow in the repository reports zero dangling `needs.<job>` references remaining. NOT DONE HERE, and worth saying: nothing enforces this. The sweep above is a throwaway script in a commit message, not a guard — the 90-day freeze holds — so the next dangling reference will be found the same way this one was, by someone looking. CI: 30 SUCCESS, 2 SKIPPED, 0 failing.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pinned to
b1d5ab98. Three commits, all on the release path.b1d5ab98(#592)publish-npmpublished without asking the registry, so a re-run manufactured a "Release Pipeline Failed" issue — three of them (#492, #552, #553). Also: two jobs readneeds.<job>for a job they never declared. Closes #56938eeea27(#591)4334d344(#590)evolith-mcp --versionstarted the MCP server instead of answering, and the install gate reported that as a passWhy this one unblocks the thing that has been stuck all night
publish-mcp-registryonly runs onpush+refs/tags/v*. So the MCP registry — still{"servers":[],"metadata":{"count":0}}— cannot open without a tag, and a tag could not bepushed safely because
sdk-cli-release.ymlwould try to republish an already-published CLI,fail, and file the very noise issue #569 describes. With #592 on
main, that is no longer true.Two further defects found in the same job while fixing it, both from reading
needs.<job>foran undeclared job:
publish-npminterpolated an empty version into its confirmation message.upload-assetsinterpolatesneeds.release-gate.outputs.tag_nameinto the GitHub Release'stag_nameandname— it would have published a Release called"Release "against anempty tag. That one matters here specifically, because the next tag is the first to reach it.
A sweep across every workflow reports zero dangling
needs.<job>references remaining.State of the registry at this pin
Every package is level — repo version equals registry version for all five (
cli1.3.1,mcp1.3.2,core-domain1.3.1,infra-providers1.2.1,contracts1.2.0). A tag from herepublishes nothing new; what it does is create the GitHub Release and run the MCP registry
publish.
Noted and not decided here: the
--versionfix in4334d344is in the repo and not in thepublished
evolith-mcp@1.3.2, so a tag would register the MCP listing against a version whose--versionhangs. It does not affect MCP protocol use, which never invokes the flag.🤖 Generated with Claude Code