feat: fast_llvm_repo

[AlexanderLanin]

No description provided.

[Copilot]

Pull request overview

Adds an optional Bazel repository rule for downloading and parallel-extracting pinned Linux LLVM distributions.

Changes:

• Adds fast_llvm_repo with architecture/version selection and checksum verification.
• Exposes the rule through the rules package.
• Documents usage, repository layout, and trade-offs.

Critical finding: Label() cannot resolve @toolchains_llvm from this module because it is not a direct dependency, causing the documented setup to fail.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Summary
rules/fast_llvm_repo.bzl	Implements LLVM download and extraction; contains the critical label-resolution issue.
rules/BUILD	Exports the repository rule.
README.md	Adds documentation links and feature summary.
docs/repository_layout.md	Documents the new rule location.
docs/fast_llvm_repo.md	Describes setup, behavior, and trade-offs.

Suppressed comments (2)

docs/fast_llvm_repo.md:61

• This usage block never registers the generated toolchain, so a consumer following it will not select @llvm_toolchain//:all unless it has an unrelated registration elsewhere. Add register_toolchains("@llvm_toolchain//:all") after use_repo so the documented setup is complete.

use_repo(llvm, "llvm_toolchain")

rules/fast_llvm_repo.bzl:138

• xz -T only provides multithreaded decompression with XZ Utils 5.4 or newer. On the older xz versions still common on Linux, this command fails rather than extracting, while the presence check above accepts them. Check for a sufficiently recent version before downloading (or fall back to xz -dc) so the documented Linux support does not turn into a late repository-fetch failure.

            "set -e; \"$1\" -T0 -dc \"$2\" | \"$3\" -xf - --strip-components=1 -C \"$4\"",

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[AlexanderLanin]

alternative to faster extraction is to add bazel external cache

@dcalavrezo-qorix knows more :-)

[dcalavrezo-qorix]

We enabled external-cache: true (from the cache-optimized setup-bazel used by eclipse-score/cicd-actions/setup-bazel-cache) in the two CI workflows of one of our repos that fetch LLVM toolchains — a clang-tidy job pulling LLVM 19.1.1 and an LLVM-coverage job pulling 22.1.7 plus 19.1.1.

The problem: repository-cache: true only caches the downloaded archives. The extracted trees (~5 GB per LLVM distribution) live in Bazel's output base and are lost between runs, so every run re-extracted them from .xz — single-threaded, roughly 3 minutes per distribution. In the coverage job that was a silent ~4-minute gap during the analysis phase, and the extraction was also what pushed a stock runner into disk exhaustion.

The result: warm runs restore the extracted toolchains instead of unpacking them. Our coverage job went from ~11 minutes to ~3 minutes on warm runs, clang-tidy to ~2 minutes. Cold runs (the first push to main after a MODULE.bazel change) still pay the extraction once, since entries are keyed on MODULE.bazel and saved only on main.

One prerequisite worth knowing: every MODULE.bazel change writes a fresh generation of cache entries and the old ones get LRU-evicted, so this churns cache space. Our repo has the 50 GB Actions cache size, which absorbs it comfortably; under the default 10 GB limit the churn would likely evict more than it helps.