Skip to content

examples/bes: check what buck2 sends a Build Event Service, in CI - #1616

Open
jun-mercury wants to merge 5 commits into
meta:mainfrom
MercuryTechnologies:bes-upstream/5-bes-ci
Open

jun-mercury wants to merge 5 commits into
meta:mainfrom
MercuryTechnologies:bes-upstream/5-bes-ci

Conversation

@jun-mercury

Copy link
Copy Markdown
Contributor

Summary

examples/bes/check.py runs buck2 against fake_bes_server.py, a Build Event Service that records every event it receives, and checks four commands:

  • A build that succeeds streams a complete invocation, from started to build_finished and the end of the stream.
  • A build that fails reports its exit code in build_finished.
  • A test run reports the status of each test.
  • buck2 kill --bes-drain-timeout delivers the events of a stream the daemon had not yet sent when its command ended. The server for this case reads slowly and acknowledges only at the end of a stream, as BuildBuddy does. Without the drain, the kill loses the last events of the stream, build_finished among them.

The Linux job of the Build and Examples workflow runs it on the release binary, beside the NativeLink example, as .github/actions/build_example_bes. It needs no secret and no outside service. The Python stubs come from the protos under remote_execution/oss, generated with a pinned grpcio-tools.

The e2e tests under tests/ would be the usual home, but open source does not run them, and builds with fbcode_build set compile the BES sink out.

Stack

  1. remote_execution: add the Build Event Service and Bazel BEP protocol crates #1597 remote_execution: add the Build Event Service and Bazel BEP protocol crates
  2. buck2: name actions in RE request metadata, and record the data a BEP stream needs #1598 buck2: name actions in RE request metadata, and record the data a BEP stream needs
  3. buck2_events: publish to a Bazel Build Event Service in OSS builds #1599 buck2_events: publish to a Bazel Build Event Service in OSS builds
  4. buck2: let buck2 kill wait for the BES server to acknowledge the daemon's events #1600 buck2: let buck2 kill wait for the BES server to acknowledge the daemon's events
  5. This PR

Each PR targets main and contains the commits of the PRs before it. The commit this PR adds is the last one, 618c02c. The workflow it adds to goes green only with #1606, #1607, #1608 and #1609, which fix the rest of the Build and Examples workflow.

Test plan

Posted by Claude Code on behalf of Jun Matsushita.

@meta-codesync

meta-codesync Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D130202572. (Because this pull request was imported automatically, there will not be any future comments.)

@jun-mercury
jun-mercury force-pushed the bes-upstream/5-bes-ci branch from 618c02c to 32ab23a Compare November 11, 2026 02:54
…crates

Three new crates hold the generated Rust for the protocols an OSS build event
service sink needs:

- bes_grpc_proto: google.devtools.build.v1, the Build Event Service.
- bazel_bep_proto: Bazel's build_event_stream and the messages it imports.
- google_grpc_proto: google.api, google.rpc, google.longrunning and
  google.bytestream, with the serde helpers for Any.

The Google protos move out of re_grpc_proto into google_grpc_proto, so the RE
and BES crates share one copy. re_grpc_proto re-exports `google`, so code
that names `re_grpc_proto::google::...` is unchanged.

Nothing uses the new crates yet.

Assisted-by: Claude:claude-opus-5-5
… stream needs

Two changes to core crates that the OSS build event service sink builds on.
Each is useful without it.

RE requests carry the action's identity. ReActionIdentity gains the action
id (the action digest), mnemonic (the action category), target id and
configuration id, and the OSS client puts them in RequestMetadata. This
covers executions, action cache lookups and writes, and uploads. Callers
without an identity pass None. Servers such as BuildBuddy then show these
per action.

Events carry what a Bazel Build Event Protocol stream reports and Buck2's
events left out:
- an action's outputs (path, digest, size, kind) and output symlinks;
- a test suite's labels and a test's timeout;
- a materialization's URL;
- `buck2 run`'s final command line (RunExecRequest);
- fetches made by git-backed external cells (ExternalResourceFetch);
- the repository root and working directory in the invocation metadata.

Existing consumers see only new fields and new event variants.

Assisted-by: Claude:claude-opus-5-5
An OSS build can stream its events to a server that speaks the Bazel Build
Event Service protocol, such as BuildBuddy. That is the role Scribe plays in
Meta's build. It is off unless [bes] is configured. The new sink code is
compiled only under cfg(not(fbcode_build)), and the shared code in
sink/scribe.rs and sink/remote.rs keeps the Scribe path under
cfg(fbcode_build).

- bazel_converter translates Buck2 events to Bazel BEP: the invocation and
  command line, targets, actions, tests, artifacts and the build graph.
- bes_client streams them over PublishBuildToolEventStream. It retries from a
  bounded replay buffer that spills to disk, and writes inline files to the
  CAS off the event path. A refused stream is not redialled, and a stalled
  connection does not block a command.
- The [bes] config takes an endpoint or the RE client's connection, plus
  headers with $VAR expansion. An unset variable turns BES off. It also
  takes a results URL, and TLS settings as the RE client has them.
- The console prints the server's results URL.
- The daemon finishes its streams before it exits.
- `buck2 kill` finishes the streams before it starts the shutdown, after
  the kill's timeout is validated. While that drain runs (up to 5 s), the
  daemon still accepts new commands, because 9908473 made
  `DaemonShutdown::start_shutdown` the only place that marks the daemon as
  shutting down, and the drain has to finish before the shutdown starts.
- Snapshots report where the sink's worker spends its time.

docs/users/build_observability/event_system.md describes the configuration.

Assisted-by: Claude:claude-opus-5-5
…emon's events

`buck2 kill --bes-drain-timeout <duration>` asks the daemon to deliver what
its event sink still holds. It then waits, up to the timeout, until the
server has acknowledged it, and only then exits. The result names any
invocation still unacknowledged.

`buck2 debug flush-bes` does the same for finished commands without stopping
the daemon.

The daemon drains once, for the whole process, before any tenant is evicted,
so a kill under daemon sharing drains the shared sink before the evictions
start. The client waits for the drain on top of its usual bounds. Evicting
one tenant (`kill_unidaemon_tenant`, the tenant proxy) does not drain.

Without the flag a kill behaves as before. A CI runner can use it so a job's
last build reaches the server before the runner goes away.

Assisted-by: Claude:claude-opus-5-5
`examples/bes/check.py` runs buck2 against `fake_bes_server.py`, a Build
Event Service that records every event it receives. It checks four commands:

- A build that succeeds streams a complete invocation, from `started` to
  `build_finished` and the end of the stream.
- A build that fails reports its exit code in `build_finished`.
- A test run reports the status of each test.
- `buck2 kill --bes-drain-timeout` delivers a stream that the daemon still
  held when its command ended. The server for this case reads slowly and
  acknowledges only at the end of a stream, as BuildBuddy does. Without the
  drain, the kill loses the last events of the stream, `build_finished`
  among them.

The Linux job of the Build and Examples workflow runs it on the release
binary, beside the NativeLink example. It needs no secret and no outside
service. The Python stubs come from the protos under `remote_execution/oss`,
generated with a pinned `grpcio-tools`.

The e2e tests under `tests/` would be the usual home, but open source does
not run them, and builds with `fbcode_build` set compile the BES sink out.

Assisted-by: Claude:claude-opus-5-5
@jun-mercury
jun-mercury force-pushed the bes-upstream/5-bes-ci branch from 32ab23a to 40a8a64 Compare November 17, 2026 01:26

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant