Repository navigation
buck2: name actions in RE request metadata, and record the data a BEP stream needs - #1598
jun-mercury wants to merge 2 commits into
Conversation
|
Hi @jun-mercury! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
a619447 to
097a383
Compare
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
|
@8Keep has imported this pull request. If you are a Meta employee, you can view this in D130203326. (Because this pull request was imported automatically, there will not be any future comments.) |
…crates Three new crates hold the generated Rust for the protocols an OSS build event service sink needs: - bes_grpc_proto: google.devtools.build.v1, the Build Event Service. - bazel_bep_proto: Bazel's build_event_stream and the messages it imports. - google_grpc_proto: google.api, google.rpc, google.longrunning and google.bytestream, with the serde helpers for Any. The Google protos move out of re_grpc_proto into google_grpc_proto, so the RE and BES crates share one copy. re_grpc_proto re-exports `google`, so code that names `re_grpc_proto::google::...` is unchanged. Nothing uses the new crates yet. Assisted-by: Claude:claude-opus-5-5
… stream needs Two changes to core crates that the OSS build event service sink builds on. Each is useful without it. RE requests carry the action's identity. ReActionIdentity gains the action id (the action digest), mnemonic (the action category), target id and configuration id, and the OSS client puts them in RequestMetadata. This covers executions, action cache lookups and writes, and uploads. Callers without an identity pass None. Servers such as BuildBuddy then show these per action. Events carry what a Bazel Build Event Protocol stream reports and Buck2's events left out: - an action's outputs (path, digest, size, kind) and output symlinks; - a test suite's labels and a test's timeout; - a materialization's URL; - `buck2 run`'s final command line (RunExecRequest); - fetches made by git-backed external cells (ExternalResourceFetch); - the repository root and working directory in the invocation metadata. Existing consumers see only new fields and new event variants. Assisted-by: Claude:claude-opus-5-5
097a383 to
3a7ba48
Compare
Summary
Two changes to core crates that the BES sink builds on. Each is useful on its own.
RE requests carry the action's identity.
ReActionIdentitygains the action id (the action digest), mnemonic (the action category), target id and configuration id. The OSS RE client puts them inRequestMetadatafor executions, action cache lookups and writes, and uploads. Callers without an identity passNone. Servers such as BuildBuddy then show these per action.Events carry what a Bazel BEP stream reports. These are things Buck2's events didn't carry before:
buck2 run's final command line (RunExecRequest);ExternalResourceFetch, field 71);Existing consumers see only new fields and new event variants.
This is part 2 of a stack of five that adds an OSS Build Event Service (BES) sink:
[bes]is configured, and compiled only in OSS builds. buck2_events: publish to a Bazel Build Event Service in OSS builds #1599buck2 kill --bes-drain-timeoutandbuck2 debug flush-bes. buck2: letbuck2 killwait for the BES server to acknowledge the daemon's events #1600Each PR targets main and contains the commits of the PRs before it. The commit this PR adds is the last one, 097a383. Each commit builds on its own.
Test plan
buck2 build //:buck2with a cargo-built buck2, the Build and Examples workflow's bootstrap step, passes on Linux, macOS and Windows with this stack and prelude: find libpython where each platform's sysconfig puts it #1606 to .github: get the Build and Examples workflow green on Linux, macOS and Windows #1609: https://github.com/MercuryTechnologies/buck2/actions/runs/42028461908. It caught thatbuck2_eventslistedprostandserde_jsononly for internal builds, which the BES sink uses in OSS builds, so buck2_events: publish to a Bazel Build Event Service in OSS builds #1599 now lists them for every build.cargo check --workspace --testspasses at each of the four commits.nightly-2026-08-15toolchain fromrust-toolchain.toml. Therust-overlaythatflake.lockpins predates that nightly, sonix developfails on main. I used a dev shell with a newerrust-overlay, and that change is not part of this stack.cargo test --libpasses for these crates: 841 tests, of which the BES subset (buck2_eventsandbuck2_re_configuration, filtered onsink::andbes) is 185.local_resource_registry::tests::protocol_release_uses_channel_not_pid_killfails on our NixOS test host because it spawns/bin/sleep, which that host lacks, and this stack leaves its file untouched.buck2 kill --bes-drain-timeoutandbuck2 debug flush-besran cleanly.Posted by Claude Code on behalf of Jun Matsushita.