Skip to content

chore: migrate gsutil to gcloud storage - #1220

Open
SanabriaRusso wants to merge 2 commits into
mainfrom
chore/gsutil-to-gcloud-storage
Open

chore: migrate gsutil to gcloud storage#1220
SanabriaRusso wants to merge 2 commits into
mainfrom
chore/gsutil-to-gcloud-storage

Conversation

@SanabriaRusso

@SanabriaRusso SanabriaRusso commented Aug 27, 2026

Copy link
Copy Markdown
Member

Why

Google is retiring gsutil in March 2027, and it is already being dropped from new Cloud SDK bundles. Readers who follow these pages after that date will hit command not found. This PR migrates the documented commands to gcloud storage, which ships in every current Cloud SDK.

What changed

Commands — all 11 call sites were a bare gsutil cp or gsutil ls, so the swap is exact. There is no -m, -q, -o, -h, -l, -d, cat, stat, du or rsync anywhere in these pages, which means no flag translation, no output-format differences, and no exit-code semantics to audit. gsutil <verb> <args>gcloud storage <verb> <args>, arguments untouched.

  • docs/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites.mdx (4 × cp)
  • docs/network-upgrades/berkeley/archive-migration/archive-migration-installation.mdx (2 × cp)
  • docs/network-upgrades/berkeley/archive-migration/debian-example.mdx (3 × cp)
  • docs/network-upgrades/mesa/mesa-trail.mdx (2 × ls) — the live Mesa Trail page. Both are bare gsutil ls with no flags: nothing to carry over, and notably no -d, which has no gcloud storage equivalent and would have needed a rewrite rather than a swap. The adjacent comment on the HTTPS alternative is updated from "no gsutil required" to "no Google Cloud CLI required" so the contrast it draws still matches the command directly above it.

Prose — the requirement bullet gsutil application from Google Cloud Suite in version 5 or later now names the Google Cloud CLI. The "version 5 or later" qualifier is dropped deliberately: it is a gsutil version number with no gcloud analogue, so carrying it over would be meaningless.

  • docs/network-upgrades/mesa/archive-upgrade.mdx (the live upcoming-upgrade page)
  • docs/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites.mdx
  • docs/network-upgrades/berkeley/archive-migration/archive-migration-installation.mdx (install lead-in, the cURL/gcloud list label)

HTTPS alternative surfaced (additive). These buckets are already publicly readable over https://storage.googleapis.com/<bucket>/<object> — a path that is completely immune to the deprecation and needs no Cloud SDK at all. docker-example.mdx and mesa-trail.mdx already used it. It is now offered alongside the migrated commands on the two pages that previously only had a GCS-CLI path:

  • archive-migration-prerequisites.mdx — a wget https://storage.googleapis.com/... equivalent next to each of the two single-object bundle downloads.
  • debian-example.mdx — one line noting that every gcloud storage cp gs://<bucket>/<object> . in the copy-paste block can be replaced with wget https://storage.googleapis.com/<bucket>/<object>.

Reviewer note: the HTTPS URLs added here were constructed by mechanically mapping the already-documented gs://<bucket>/<object> paths onto the same pattern used by the existing reference at docker-example.mdx:45. They were not individually fetched against live storage, so please spot-check them rather than assuming they were verified.

No gcloud storage command was removed, no page was restructured, no headings were rewritten. The two wildcard downloads (devnet-*.json, mainnet-*.json) intentionally get no HTTPS equivalent, because HTTPS object GETs cannot expand a glob — offering one would be wrong.

static/llms-full.txt regenerated via node scripts/generate-llms-txt.mjs, as the check-llms-txt workflow requires. Its diff is exactly the mirror of the docs/ diff and nothing else.

Deliberately NOT changed

Several prose mentions of gsutil remain. Rewriting any of them would state something factually false:

  1. archive-migration-prerequisites.mdx:30 — "You can use any gsutil-compatible alternative to Google Cloud or a gsutil wrapper program."
  2. archive-migration-prerequisites.mdx:37 — "The berkeley-migration app uses the gsutil app to download blocks."
    Lines 30 and 37 describe the berkeley-migration app's own internals, not a command the reader types. The OCaml app still shells out to gsutil; rewriting these would assert a code change that has not happened. That migration is tracked separately against MinaProtocol/mina.
  3. migrating-archive-database-to-berkeley.mdx:515 — "you can use any gsutil-compatible storage backend (for example, S3)."
    "gsutil-compatible" here is a term of art meaning boto/S3 interoperability. gcloud storage is GCS-only and has no boto or S3 interop, so swapping the word would silently delete a documented S3 escape hatch for operators not on GCP.
  4. docs/network-upgrades/mesa/verify-the-release.mdx:85,95,188,219 — these four describe what the release-verification script itself requires and quote a verbatim error string it emits (Error: gsutil is required when PRECOMPUTED_FORK_BLOCK is nonexistent path). Same category as items 1–2: the change belongs with the upstream script, and documenting it as already done would be false.

Also untouched, for the record: the apt install blocks in installation.mdx and debian-example.mdx — they install the Cloud SDK, which still ships gcloud. Nothing to fix.

Verification

  • npm ci && npm run buildpasses (exit 0, "Generated static files in build"). This was the main risk: archive-migration-installation.mdx contains an inline code span holding {network}, which MDX v3 would parse as a JS expression if it ever fell outside backticks. It stays inside backticks and the build confirms it.
    • The only build warnings are four pre-existing broken anchors on /network-upgrades/mesa pointing at glossary#.... They are present on main and unrelated to this change.
  • The load-bearing list indentation in archive-migration-installation.mdx (4 spaces on the list item, 7 on the nested fence) is preserved byte-for-byte, as is the fence structure in mesa-trail.mdx.
  • grep -rn gsutil docs/ afterwards returns only the intentional lines listed above.
  • Existing typo repostory in installation.mdx left in place — out of scope for this PR.

Google is retiring gsutil in March 2027; it is already excluded from new
Cloud SDK installs. Every gsutil invocation in these docs is a bare
`gsutil cp`, so the swap to `gcloud storage cp` is exact -- no flags to
translate, no output parsing to audit.

Also surfaces the HTTPS download path (https://storage.googleapis.com/...)
next to the migrated commands, so a reader with no Cloud SDK installed has
an option that is immune to the deprecation entirely.

Three prose mentions of gsutil are deliberately left as-is because
rewriting them would be factually wrong -- see the PR description.

static/llms-full.txt is regenerated (scripts/generate-llms-txt.mjs) to keep
the check-llms-txt workflow green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs2 Ready Ready Preview Aug 27, 2026 6:13am

Request Review

mesa-trail.mdx landed on main after this change was first scoped. Its two
`gsutil ls` commands are commands a reader actually runs on the live
upcoming-upgrade page, so they stop working in March 2027 like the rest.

Both are bare `gsutil ls` with no flags -- no -l to carry over, no -d
(which has no `gcloud storage` equivalent and would have needed a rewrite
rather than a swap), no -m to drop. Arguments are byte-identical.

Also updates the adjacent comment on the HTTPS alternative from
"no gsutil required" to "no Google Cloud CLI required", so the contrast it
draws still matches the command directly above it.

static/llms-full.txt regenerated to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant