Skip to content

remote_execution: add the Build Event Service and Bazel BEP protocol crates - #1597

Open
jun-mercury wants to merge 1 commit into
meta:mainfrom
MercuryTechnologies:bes-upstream/1-protos
Open

jun-mercury wants to merge 1 commit into
meta:mainfrom
MercuryTechnologies:bes-upstream/1-protos

Conversation

@jun-mercury

@jun-mercury jun-mercury commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds three crates of generated Rust for the protocols a Build Event Service sink needs:

  • bes_grpc_proto: google.devtools.build.v1, the Build Event Service.
  • bazel_bep_proto: Bazel's build_event_stream.proto and the messages it imports, vendored unchanged.
  • google_grpc_proto: google.api, google.rpc, google.longrunning and google.bytestream, with the serde helpers for Any.

The Google protos move out of re_grpc_proto into google_grpc_proto, so the RE and BES crates share one copy. re_grpc_proto re-exports google, so code that uses re_grpc_proto::google::… doesn't change.

The BES sink later in the stack is the first code to use these crates.

This is part 1 of a stack of five that adds an OSS Build Event Service (BES) sink:

  1. Protocol crates: bes_grpc_proto, bazel_bep_proto and a shared google_grpc_proto. (this PR)
  2. Core hooks: action identity in RE request metadata, and the event fields a BEP stream needs. buck2: name actions in RE request metadata, and record the data a BEP stream needs #1598
  3. The BES sink in buck2_events, off unless [bes] is configured, and compiled only in OSS builds. buck2_events: publish to a Bazel Build Event Service in OSS builds #1599
  4. buck2 kill --bes-drain-timeout and buck2 debug flush-bes. buck2: let buck2 kill wait for the BES server to acknowledge the daemon's events #1600
  5. A CI check of what buck2 sends a Build Event Service, in the Build and Examples workflow. examples/bes: check what buck2 sends a Build Event Service, in CI #1616

Each PR targets main and contains the commits of the PRs before it. The commit this PR adds is the last one, e290185. Each commit builds on its own.

Test plan

  • buck2 build //:buck2 with a cargo-built buck2, the Build and Examples workflow's bootstrap step, passes on Linux, macOS and Windows with this stack and prelude: find libpython where each platform's sysconfig puts it #1606 to .github: get the Build and Examples workflow green on Linux, macOS and Windows #1609: https://github.com/MercuryTechnologies/buck2/actions/runs/42028461908. It caught that buck2_events listed prost and serde_json only for internal builds, which the BES sink uses in OSS builds, so buck2_events: publish to a Bazel Build Event Service in OSS builds #1599 now lists them for every build.
  • cargo check --workspace --tests passes at each of the four commits.
    • I used the nightly-2026-08-15 toolchain from rust-toolchain.toml. The rust-overlay that flake.lock pins predates that nightly, so nix develop fails on main. I used a dev shell with a newer rust-overlay, and that change is not part of this stack.
  • At the top of the stack, cargo test --lib passes for these crates: 841 tests, of which the BES subset (buck2_events and buck2_re_configuration, filtered on sink:: and bes) is 185.
    • buck2_events, buck2_client_ctx, buck2_server
    • buck2_execute, buck2_execute_impl, buck2_build_api
    • buck2_re_configuration, buck2_external_cells, remote_execution
    • buck2_cli_proto, buck2_data
  • In buck2_test, 57 of 58 tests pass. local_resource_registry::tests::protocol_release_uses_channel_not_pid_kill fails on our NixOS test host because it spawns /bin/sleep, which that host lacks, and this stack leaves its file untouched.
  • We carry this sink in our fork, and it reports every build in our build lab to BuildBuddy. The stack is based on main at c7e71ec. A buck2 built from the head of the stack ran a scratch project against our BuildBuddy with remote execution on our workers. Builds, a failing build and a test run reported SUCCESS, BUILD_FAILURE and TESTS_FAILED. BuildBuddy recorded the test results in its test grid, and each remote execution carried its target and mnemonic. buck2 kill --bes-drain-timeout and buck2 debug flush-bes ran cleanly.

Posted by Claude Code on behalf of Jun Matsushita.

@meta-cla

meta-cla Bot commented Nov 4, 2026

Copy link
Copy Markdown

Hi @jun-mercury!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla

meta-cla Bot commented Nov 6, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@meta-codesync

meta-codesync Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

@8Keep has imported this pull request. If you are a Meta employee, you can view this in D130203209. (Because this pull request was imported automatically, there will not be any future comments.)

…crates

Three new crates hold the generated Rust for the protocols an OSS build event
service sink needs:

- bes_grpc_proto: google.devtools.build.v1, the Build Event Service.
- bazel_bep_proto: Bazel's build_event_stream and the messages it imports.
- google_grpc_proto: google.api, google.rpc, google.longrunning and
  google.bytestream, with the serde helpers for Any.

The Google protos move out of re_grpc_proto into google_grpc_proto, so the RE
and BES crates share one copy. re_grpc_proto re-exports `google`, so code
that names `re_grpc_proto::google::...` is unchanged.

Nothing uses the new crates yet.

Assisted-by: Claude:claude-opus-5-5
@jun-mercury
jun-mercury force-pushed the bes-upstream/1-protos branch from e290185 to a799733 Compare November 11, 2026 02:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant