ci: fail the monthly release fast on a bad release token - #7712
Conversation
The release job authenticates with BACKEND_RELEASE_TOKEN, a PAT that has to be rotated by hand. When it expires, the first thing that touches it is actions/checkout, which reports the 401 as fatal: could not read Username for 'https://github.com': terminal prompts disabled after three retries. That names neither the token nor the expiry, so the failure reads like a checkout or network problem. Check the token against the GitHub API before the checkout and report what actually went wrong: missing secret, or a rejected one with the HTTP status that distinguishes an invalid token from an under-scoped one. The step also warns on the run when the token is within 45 days of expiring, which is the signal to rotate it before a release fails.
There was a problem hiding this comment.
Pull request overview
This PR improves the monthly release workflow’s failure mode when BACKEND_RELEASE_TOKEN is missing/invalid by validating the token via the GitHub API before actions/checkout, and documents the release credential requirements in the Python development guide.
Changes:
- Add a pre-check step in
release-monthly.ymlthat calls the GitHub API usingBACKEND_RELEASE_TOKENand surfaces clearer errors (and expiry warnings). - Document why
BACKEND_RELEASE_TOKENis required (vsGITHUB_TOKEN) and how the monthly release is triggered.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
.github/workflows/release-monthly.yml |
Adds a “Verify release token” step before checkout to fail fast on missing/invalid tokens and warn on impending expiry. |
docs/python/development.md |
Documents release credential rationale and the monthly release trigger/wait behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| exit 1 | ||
| fi | ||
|
|
||
| EXPIRY="$(tr -d '\r' < "$HEADERS" | tr 'A-Z' 'a-z' | sed -n 's/^github-authentication-token-expiration: *//p')" |
There was a problem hiding this comment.
Taken in 2cddf98, though the stated breakage does not reproduce. GNU date parses the lowercased form fine, date -u -d "2026-12-31t00:00:00z" returns 1798675200 on coreutils 9.11, which is what the runner has.
The change is still worth making, since relying on that leniency to protect a value we deliberately mangled is not a property worth keeping. The header name is now matched case insensitively with awk and the value is passed through untouched, verified against both header casings and both timestamp formats. The related hole was that an unparseable expiry disabled the warning in silence, which would quietly undo the point of the step, so that now emits a warning naming the value it could not read.
| RELEASE_TOKEN: ${{ secrets.BACKEND_RELEASE_TOKEN }} | ||
| run: | | ||
| if [ -z "$RELEASE_TOKEN" ]; then | ||
| echo "::error title=Release token missing::BACKEND_RELEASE_TOKEN is not available to this workflow. Store a token with contents:write and workflow permissions on $GITHUB_REPOSITORY as an organization or repository secret." |
There was a problem hiding this comment.
Fixed in 2cddf98. The two vocabularies were mixed, and workflow was wrong on top of that: the release commit only touches package.json and package-lock.json, so it never needs the workflow scope. Good catch on the Actions API, the wait-for-CI step calls gh run list and that read was undocumented.
Both messages and the docs now say Contents: read and write plus Actions: read, with the classic PAT repo scope named once as the equivalent.
| if [ "$STATUS" != "200" ]; then | ||
| echo "::error title=Release token rejected::BACKEND_RELEASE_TOKEN was rejected by the GitHub API (HTTP $STATUS); it has most likely expired or been revoked. Rotate the secret with a token that has contents:write and workflow permissions on $GITHUB_REPOSITORY, then re-run this workflow." | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
Fixed in 2cddf98. Collapsing these was the worst part of the original, because it points at the secret when the secret is fine. Now a case on the status splits them: 000 says the API was unreachable and that this is a runner fault rather than evidence about the token, 401 says the value is not a credential GitHub recognizes and names the three ways that happens including stray whitespace in the secret, and 403 says the token is real and its grant is too narrow. Unexpected codes report the status rather than guessing.
| builds the release artifacts. The token needs `contents:write` and `workflow` | ||
| permissions on the repository. |
There was a problem hiding this comment.
Fixed in 2cddf98, same wording as the workflow. The section now also says what each permission is for, Contents for pushing the bump and the tag, Actions for polling the CI run the job waits on, since that is the context someone needs when deciding what to grant a replacement token.
Three states looked alike in the first version: an unreachable API reported the token as rejected, and a 401 and a 403 shared one message that guessed at expiry. Split them, since the remedy differs in each case. An unreachable API is a runner fault and no reason to touch the secret, a 401 means the value is not a credential at all, and a 403 means the grant is too narrow. State the permissions the way GitHub does, Contents and Actions, with the classic PAT scope named once rather than mixing the two vocabularies. Actions read is what the wait-for-CI step needs; the workflow scope is not, since the release commit only touches package.json and package-lock.json. Match the expiry header case insensitively by name instead of lowercasing the whole response, so the timestamp reaches date untouched, and warn when the expiry cannot be parsed rather than dropping the check in silence.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7712 +/- ##
==========================================
- Coverage 94.81% 91.92% -2.90%
==========================================
Files 526 526
Lines 42085 42085
==========================================
- Hits 39902 38685 -1217
- Misses 2183 3400 +1217 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Issue
No separate issue. The September release run failed on this: https://github.com/Project-OSRM/osrm-backend/actions/runs/33485931388
BACKEND_RELEASE_TOKENhad expired, and the first step that touched it wasactions/checkout, which surfaced the 401 asafter three retries. That message names neither the token nor the expiry, so it reads like a checkout or a network problem. Working out that it was the token, and that the replacement had also been stored wrong, took several attempts against a 40 second feedback loop.
This adds a
Verify release tokenstep ahead of the checkout. It calls the GitHub API with the token and reports what actually went wrong:It also reads
github-authentication-token-expirationoff the response and raises a run warning when the token is within 45 days of expiring, so the rotation happens before a release fails rather than after.Since the token is a hand-rotated PAT, this will recur. A GitHub App installation token via
actions/create-github-app-tokenwould remove the expiry entirely. Out of scope here.Verified the step locally against a missing token, an invalid token, and a valid one, and checked the expiry parsing and the day arithmetic against a synthetic header.
Was this change primarily generated using an AI tool?
Claude Code, Claude Opus 5 🤖
Tasklist
Requirements / Relations
None.