Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 64 additions & 4 deletions .github/workflows/sdk-cli-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,11 @@ jobs:
# GATE 3: Publish to NPM
# ============================================
publish-npm:
needs: build-and-test
# `release-gate` was referenced by this job (needs.release-gate.outputs.version) without
# being declared here, and `needs.<job>` only resolves for jobs listed in `needs`. So the
# version interpolated to the EMPTY STRING and the confirmation step announced
# "Published @beyondnet/evolith-cli@ to NPM".
needs: [release-gate, build-and-test]
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -211,15 +215,67 @@ jobs:
working-directory: ${{ env.CLI_DIR }}
run: npm run build

# ASK THE REGISTRY BEFORE PUBLISHING (#569).
#
# This job used to run a bare `npm publish`. Any re-run against an already-published
# version — a retry after an unrelated flake, a workflow_dispatch, a re-pushed tag —
# died on "You cannot publish over the previously published versions", and that failure
# reached `failure-notification`, which files a "Release Pipeline Failed" issue. Three
# of those accumulated (#492, #552, #553) before anyone read one. A harmless re-run
# manufactured a bug report.
#
# Idempotence belongs BEFORE the publish, as a question with an answer — not after it,
# as a swallowed error. `continue-on-error` or `|| true` on the publish itself would
# make a genuine failure indistinguishable from this one, which is the whole defect
# wearing a different hat. The `|| true` below is on a QUERY, where a non-zero exit
# simply means "not found", and its answer is then compared explicitly.
#
# npm-release.yml has solved this since `plan-npm-release.mjs`; this workflow predates
# it and never adopted it.
- name: Is this version already on the registry?
id: plan
working-directory: ${{ env.CLI_DIR }}
run: |
set -euo pipefail
NAME=$(node -p "require('./package.json').name")
VERSION=$(node -p "require('./package.json').version")
{
echo "name=$NAME"
echo "version=$VERSION"
} >> "$GITHUB_OUTPUT"
if [ "$(npm view "$NAME@$VERSION" version 2>/dev/null || true)" = "$VERSION" ]; then
echo "published=true" >> "$GITHUB_OUTPUT"
echo "::notice::$NAME@$VERSION is already on the registry. Skipping the publish — this is not a failure. Bump the version to ship a change."
else
echo "published=false" >> "$GITHUB_OUTPUT"
echo "$NAME@$VERSION is not on the registry — publishing."
fi

- name: Publish to NPM
if: steps.plan.outputs.published != 'true'
working-directory: ${{ env.CLI_DIR }}
run: npm publish --provenance --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

- name: Confirm Publication
# An echo is not a confirmation. The previous version of this step printed a success
# message it had no evidence for — and printed it with an empty version, at that.
# Verify against the registry, which is the only authority on what was published.
- name: Confirm the registry holds it
run: |
echo "✅ Published @beyondnet/evolith-cli@${{ needs.release-gate.outputs.version }} to NPM"
set -euo pipefail
NAME='${{ steps.plan.outputs.name }}'
VERSION='${{ steps.plan.outputs.version }}'
for attempt in 1 2 3 4 5; do
if [ "$(npm view "$NAME@$VERSION" version 2>/dev/null || true)" = "$VERSION" ]; then
echo "confirmed on registry: $NAME@$VERSION"
exit 0
fi
echo "not visible yet (attempt $attempt/5), waiting"
sleep 10
done
echo "::error::$NAME@$VERSION is NOT on the registry after this job ran. The publish reported success and the registry disagrees."
exit 1

# ============================================
# GATE 4: Package Binaries
Expand Down Expand Up @@ -361,7 +417,11 @@ jobs:
# GATE 6: Upload Release Assets
# ============================================
upload-assets:
needs: [package-binaries, smoke-test, smoke-test-functional]
# Same dangling reference as publish-npm had, and this one is worse: the step below
# interpolates needs.release-gate.outputs.tag_name into the GitHub Release's `tag_name`
# and `name`, so without release-gate declared here it publishes a Release named
# "Release " against an empty tag.
needs: [release-gate, package-binaries, smoke-test, smoke-test-functional]
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10081,6 +10081,25 @@
"dependencyDisposition": "none",
"dependencyRationale": "No dependency added or changed, and none could be: the change deletes a dependency rather than adding one. The test previously depended on git AND on the remote-tracking ref origin/develop being present and pointing at the pre-fix artifact — an external, mutable input. It now reads a file committed in this repository, using only node:fs, which the file already imported. The fixture itself is inert data with no imports. Nothing is imported by relative path from domain code: this is a harness test reading a harness fixture, and the corpus JSON under src/rulesets is untouched."
},
{
"id": "GT-671",
"closedAt": "2026-08-16",
"closureCommit": "213cc718",
"dependencyDisposition": "none",
"evidence": [
".github/workflows/published-canary.yml",
".harness/scripts/ci/published-artifact-canary.mjs",
"src/sdk/cli/examples/gate-verdict.assert.js"
],
"validationCommands": [
"WHAT THE ROW CLAIMED AND WHAT MEASUREMENT NARROWED: 'nothing re-verifies the published artifact' is false as written -- one check existed, `evolith-cli --help` at release time (sdk-cli-release.yml:328-332). It is a weak oracle, MEASURED: every published version answers `--version` with exit 0, INCLUDING the one GT-625 recorded as broken. A check that cannot tell a working artifact from a broken one is not a check, so the row was scoped to what that check cannot see.",
"THE CANARY MEASURES THE REGISTRY, NOT THE TREE. Packages install into a throwaway npm prefix and are driven from a temp directory; the checkout carries only the assertion and the script, deliberately off the resolution path -- GT-625 shipped an uninstallable CLI precisely because the workspace symlink hid it from every suite.",
"PROVEN FALSIFIABLE WITH A FIXTURE THAT WAS MEASURED, NOT ASSUMED. The row named cli@1.2.0 as the red fixture; measured, that version installs, runs `init` and returns a valid envelope. The real red fixture is cli@1.1.0, on three independent counts: no `evolith` bin (only `evolith-cli`), `init --name` writes a subdirectory so no evolith.yaml appears where the canary looks, and `validate --format json` truncates its own envelope through a pipe at 65,386 bytes of a 163,622-byte document. Green against latest, red against 1.1.0, BOTH OBSERVED.",
"AC3 WAS THE LAST ONE OPEN AND EXPIRED BY ITSELF. Its exemption was keyed on `installedPackageShipsNoCorpus()` -- a property of the INSTALLED TARBALL, not a version number and not a date. GT-705 shipped as mcp@1.3.2; run 31987205590 (published-canary.yml, cli@1.3.1 + mcp@1.3.2) reports 'the published MCP server answers a tools/call with a real gate verdict -- verdict asserted by gate-verdict.assert.js', and its log contains ZERO occurrences of 'exempt'. Nobody had to remember to delete anything. A blanket skip would still be green today with the defect fixed by accident.",
"SAME RUN, SAME INSTALL: `validate --format json` returns an ADR-0073 envelope with verdict `failed`, 41/159 rules checked -- a real denominator, so an install shipping no rulesets fails even when it exits 0.",
"node .harness/scripts/ci/published-artifact-canary.mjs --version 1.3.1 --mcp-version 1.3.2"
]
},
{
"id": "GT-677",
"closedAt": "2026-08-14",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8941,10 +8941,10 @@ La declaración tiene un hueco — un pack que no declara — y el directorio lo
- **Criterios de aceptación:**
- [x] Un workflow programado instala el CLI desde el registro en un entorno limpio sin checkout del repositorio en la ruta de resolución, y ejecuta `init` y luego `validate`. **CUMPLIDO** — `.github/workflows/published-canary.yml`, a diario a las 06:30 UTC más `workflow_dispatch` con versión como entrada. El checkout solo aporta la aserción y el script; los paquetes se instalan en un prefijo desechable y se ejecutan desde un directorio temporal, que es justo el punto: `GT-625` publicó un CLI no instalable porque el symlink del workspace lo ocultaba a todas las suites.
- [x] La aserción es un envelope ADR-0073 real con veredicto, no un código de salida cero. **CUMPLIDO** — el canario exige `data.status ∈ {passed, failed, warning}` Y `rulesTotal > 0`, así que una instalación sin rulesets falla aunque salga con 0. Un exit distinto de cero se trata como veredicto, no como fallo: un satélite con hallazgos bloqueantes sale con 2 por diseño.
- [ ] La misma corrida ejercita el servidor MCP publicado por stdio y asegura un veredicto de gate real, reutilizando `gate-verdict.assert.js`. **NO CUMPLIDO, y por eso esta fila sigue abierta.** El canario sí lo ejercita por stdio y sí llama a ese oráculo, pero el paquete publicado no puede producir un veredicto: `@beyondnet/evolith-mcp@1.2.2` declara `files: ["dist/", "README.md", "LICENSE"]` y **no incluye corpus de rulesets**, así que `evolith-gate-evaluate` responde `RULESET_NOT_FOUND` y `evolith-validate` «no pudo localizar el corpus», mientras `evolith-metrics` funciona. Separado como [`GT-705`](./gap-reference-catalog.es.md#gt-705). La aserción se CONSERVA y solo se exime ante ese síntoma exacto, así que cualquier otro fallo sigue tumbando la corrida y la aserción pasa a sostener el peso el día que `GT-705` se entregue — un salto en bloque habría sido el gemelo permanentemente-verde del workflow permanentemente-rojo de `GT-635`.
- [x] La misma corrida ejercita el servidor MCP publicado por stdio y asegura un veredicto de gate real, reutilizando `gate-verdict.assert.js`. **CUMPLIDO 2026-08-16, y la exención caducó sola en vez de borrarse.** `@beyondnet/evolith-mcp@1.3.2` — la build que lleva [`GT-705`](./gap-reference-catalog.es.md#gt-705) — llegó al registry, y la corrida del canario contra ella (`published-canary.yml`, run `31987205590`, `cli@1.3.1` + `mcp@1.3.2`) reporta **«el servidor MCP publicado responde un tools/call con un veredicto de gate real — veredicto asegurado por gate-verdict.assert.js»**. La exención estaba anclada a `installedPackageShipsNoCorpus()`, una propiedad del TARBALL INSTALADO y no un número de versión ni una fecha, así que incluir el corpus dejó de casarla sin nada que recordar: el log de la corrida contiene **cero** apariciones de «exempt». Esa es la diferencia entre una exención y un salto en bloque — el segundo habría sido el gemelo permanentemente-verde del workflow permanentemente-rojo de `GT-635`, y hoy seguiría verde con el defecto arreglado por accidente. Misma corrida, misma instalación: `validate --format json` devuelve `failed, 41/159 reglas comprobadas` desde un prefijo npm desechable sin repositorio en la ruta de resolución.
- [x] Un canario en rojo aparece en algún sitio que una persona lee. **CUMPLIDO** — una corrida programada fallida abre una issue `published-canary`, o comenta en la abierta en vez de duplicarla, y una corrida verde comenta y la cierra. Un hilo que nadie cierra es un hilo que nadie cree.
- [x] Falsabilidad probada, OBSERVADA en rojo. **CUMPLIDO, con un fixture medido y no supuesto.** El criterio nombraba `cli@1.2.0`; medido, esa versión instala, hace `init` y devuelve un envelope válido por este camino, así que no es el fixture roto que la fila creía. `cli@1.1.0` sí lo es, por tres motivos independientes: no publica el binario `evolith` (solo `evolith-cli`, con lo que toda invocación documentada falla), su `init --name` escribe un subdirectorio y no aparece `evolith.yaml` donde el canario mira, y `validate --format json` **trunca su propio envelope por tubería** — 65 386 bytes de un documento que a fichero ocupa 163 622, mientras `1.2.2` escribe 69 465 por la misma tubería y parsea. Verde contra `latest`, rojo contra `1.1.0`, ambos observados.
- **Estado:** `EN-PROGRESO`
- **Estado:** `COMPLETADO`

#### GT-672

Expand Down
4 changes: 2 additions & 2 deletions reference/core/control-center/gaps/gap-reference-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -9035,10 +9035,10 @@ The declaration has one hole — a pack that does not declare — and the direct
- **Acceptance criteria:**
- [x] A scheduled workflow installs the CLI from the registry in a clean environment with no repository checkout on the resolution path, and runs `init` then `validate`. **MET** — `.github/workflows/published-canary.yml`, daily at 06:30 UTC plus `workflow_dispatch` with a version input. The checkout carries only the assertion and the script; the packages are installed into a throwaway prefix and driven from a temp directory, which is the point — `GT-625` shipped an uninstallable CLI because the workspace symlink hid it from every suite.
- [x] The assertion is a real ADR-0073 envelope with a verdict, not a zero exit code. **MET** — the canary requires `data.status ∈ {passed, failed, warning}` AND `rulesTotal > 0`, so an install that shipped no rulesets fails even when it exits 0. A non-zero exit is treated as a verdict, not a failure: a satellite with blocking findings exits 2 by design.
- [ ] The same run exercises the published MCP server over stdio and asserts a real gate verdict, reusing `gate-verdict.assert.js`. **NOT MET, and it is why this row stays open.** The canary does exercise it over stdio and does call that oracle, but the published package cannot produce a verdict: `@beyondnet/evolith-mcp@1.2.2` declares `files: ["dist/", "README.md", "LICENSE"]` and ships **no ruleset corpus**, so `evolith-gate-evaluate` answers `RULESET_NOT_FOUND` and `evolith-validate` "could not locate the Evolith ruleset corpus", while `evolith-metrics` works. Split out as [`GT-705`](./gap-reference-catalog.md#gt-705). The assertion is KEPT and exempted only on that exact symptom, so any other failure still fails the run and the assertion becomes load-bearing the day `GT-705` ships — a blanket skip would have been the permanently-green twin of `GT-635`'s permanently-red workflow.
- [x] The same run exercises the published MCP server over stdio and asserts a real gate verdict, reusing `gate-verdict.assert.js`. **MET 2026-08-16, and the exemption expired by itself rather than being deleted.** `@beyondnet/evolith-mcp@1.3.2` — the build carrying [`GT-705`](./gap-reference-catalog.md#gt-705) — reached the registry, and the canary run against it (`published-canary.yml`, run `31987205590`, `cli@1.3.1` + `mcp@1.3.2`) reports **"the published MCP server answers a tools/call with a real gate verdict — verdict asserted by gate-verdict.assert.js"**. The exemption was keyed on `installedPackageShipsNoCorpus()`, a property of the INSTALLED tarball rather than a version number or a date, so shipping the corpus stopped it matching with nothing to remember: the run log contains **zero** occurrences of "exempt". That is the difference between an exemption and a blanket skip — the latter would have been the permanently-green twin of `GT-635`'s permanently-red workflow, and would still be green today with the defect fixed by accident. Same run, same install: `validate --format json` returns `failed, 41/159 rules checked` from a throwaway npm prefix with no repository on the resolution path.
- [x] A red canary surfaces somewhere a human reads. **MET** — a failing scheduled run opens a `published-canary` issue, or comments on the open one rather than duplicating it, and a green run comments and closes it. A thread nobody closes is a thread nobody believes.
- [x] Proven falsifiable, OBSERVED red. **MET, with a fixture that was measured rather than assumed.** The criterion named `cli@1.2.0`; measured, that version installs, runs `init` and returns a valid envelope on this path, so it is not the broken fixture the row believed. `cli@1.1.0` is, on three independent counts: it publishes no `evolith` binary (only `evolith-cli`, so every documented invocation fails), its `init --name` writes a subdirectory so no `evolith.yaml` appears where the canary looks, and `validate --format json` **truncates its own envelope through a pipe** — 65 386 bytes of a document that is 163 622 to a file, while `1.2.2` writes 69 465 through the same pipe and parses. Green against `latest`, red against `1.1.0`, both observed.
- **Status:** `IN-PROGRESS`
- **Status:** `DONE`

#### GT-672

Expand Down
Loading
Loading