bridge-cpp: host callables - std::function crosses the boundary - #4089
Merged
Conversation
Bridge-week step 12, re-landed from -full with the wire layer rewritten
against protobuf (the original predates the pb migration). A callable
argument registers a type-erased dispatcher in the process-global
host_value_registry and rides as InboundValue.handle{HOST_VALUE_CALLABLE};
the engine's dispatch callback runs it fire-and-return on a detached
thread, decoding BamlToHostCall args (required by declared order,
supplied optionals by name into arg<T> slots) and completing via
complete_host_call on every exit path.
Host exceptions cross back in two shapes (python parity):
baml::host_throw<T> as the typed BAML class value (structurally caught
by BAML's catch), and any other exception as baml.errors.HostCallable
whose _handle rehydrates the ORIGINAL exception object by identity in
throw_from_result - custom types and fields survive the round trip. A
baml::error rethrown through a callable transcodes its payload
outbound->inbound so class identity survives.
Emitter: top-level Ty::Function params become
std::function<Ret(Slots...)> (optional callable params as arg<T> slots)
encoded via encode_callable with the declared wire names; nested
callables stay unsupported. 17 engine round-trip cases ported.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
⏭️ Performance benchmarks were skippedPerf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to To run them on this PR, do any of the following, then push a commit (or re-run CI):
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe C++ bridge and SDK generator now support BAML host-callable parameters, asynchronous dispatch, optional arguments, typed host throws, native exception rehydration, and expanded end-to-end coverage. ChangesHost callable bridge
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant GeneratedSDK
participant BAMLRuntime
participant DispatchTrampoline
participant HostCallable
participant Codec
GeneratedSDK->>Codec: encode_callable
Codec->>BAMLRuntime: send callable handle
BAMLRuntime->>DispatchTrampoline: dispatch call and arguments
DispatchTrampoline->>HostCallable: decode and invoke
HostCallable-->>DispatchTrampoline: return value or exception
DispatchTrampoline-->>BAMLRuntime: complete host call
BAMLRuntime-->>Codec: decode result or rethrow stored exception
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
baml_language/sdks/cpp/bridge_cpp/include/baml/detail/host_value.h (1)
508-518: 🚀 Performance & Scalability | 🔵 TrivialThread-per-invocation may not scale for high-frequency callbacks.
Each engine→host call spawns and detaches a fresh
std::thread. For tight loops (e.g.call_repeatedlywith large N) this is a per-call thread-creation cost and unbounded concurrent thread count. If this path ever sees high call volume, consider a bounded worker pool / dispatch queue rather than a raw thread per call. Not blocking given the fire-and-return contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@baml_language/sdks/cpp/bridge_cpp/include/baml/detail/host_value.h` around lines 508 - 518, The fire-and-return path currently creates and detaches one std::thread per engine→host call, which can cause excessive creation overhead and unbounded concurrency. Replace the per-invocation thread in the dispatcher block with a bounded worker pool or dispatch queue that preserves asynchronous execution and routes dispatcher exceptions through bridge_failure.baml_language/sdks/cpp/sdkgen_cpp/src/lib.rs (1)
1085-1130: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a focused unit test for
translate_callable_ty— no test covers this helper today; a small case with required + optional callable params, unnamed-name defaults, andvoidreturn would lock in the emittedstd::function<...>string andwire_names.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@baml_language/sdks/cpp/sdkgen_cpp/src/lib.rs` around lines 1085 - 1130, The helper translate_callable_ty lacks focused coverage for mixed callable parameter modes and void returns. Add a unit test that exercises required and optional parameters, verifies unnamed parameters produce empty wire_names entries, and asserts both the emitted std::function type string and returned wire_names.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@baml_language/sdks/cpp/bridge_cpp/include/baml/detail/host_value.h`:
- Around line 480-519: Update baml_cpp_host_dispatch_trampoline so all bridge
setup and dispatch operations, including bytes.assign, find_dispatcher, and
std::thread construction, are enclosed in a try/catch that cannot let exceptions
cross the C ABI. Route any caught setup failure through bridge_failure to
complete the call_id, while preserving the existing missing-dispatcher handling
and worker-thread catch for dispatch failures.
---
Nitpick comments:
In `@baml_language/sdks/cpp/bridge_cpp/include/baml/detail/host_value.h`:
- Around line 508-518: The fire-and-return path currently creates and detaches
one std::thread per engine→host call, which can cause excessive creation
overhead and unbounded concurrency. Replace the per-invocation thread in the
dispatcher block with a bounded worker pool or dispatch queue that preserves
asynchronous execution and routes dispatcher exceptions through bridge_failure.
In `@baml_language/sdks/cpp/sdkgen_cpp/src/lib.rs`:
- Around line 1085-1130: The helper translate_callable_ty lacks focused coverage
for mixed callable parameter modes and void returns. Add a unit test that
exercises required and optional parameters, verifies unnamed parameters produce
empty wire_names entries, and asserts both the emitted std::function type string
and returned wire_names.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 4b1aba11-31c0-480b-b325-9cbf0c9223db
📒 Files selected for processing (6)
baml_language/sdk_tests/crates/cpp/function_calls/customizable/tests/test_host_callables.ccbaml_language/sdks/cpp/bridge_cpp/include/baml/baml.hbaml_language/sdks/cpp/bridge_cpp/include/baml/codec.hbaml_language/sdks/cpp/bridge_cpp/include/baml/detail/host_value.hbaml_language/sdks/cpp/bridge_cpp/include/baml/errors.hbaml_language/sdks/cpp/sdkgen_cpp/src/lib.rs
Binary size checks failed❌ 4 violations · ✅ 3 passed
Details & how to fixViolations:
Add/update baselines:
[artifacts.baml-cli]
file_bytes = 19578112
stripped_bytes = 19578160
gzip_bytes = 9340868
[artifacts.bridge_wasm]
file_bytes = 16160508
gzip_bytes = 4402886
[artifacts.baml-cli]
file_bytes = 21103104
stripped_bytes = 21103104
gzip_bytes = 9548436
[artifacts.baml-cli]
file_bytes = 25295952
stripped_bytes = 25295944
gzip_bytes = 10740130Generated by |
… the C ABI Review finding: bytes.assign (bad_alloc), the registry lock, and the std::thread constructor (system_error under thread exhaustion) all ran outside a guard in the extern-C dispatch trampoline - an escape is UB/terminate and the in-flight call never completes. Setup faults now route through bridge_failure so the engine unblocks, with an outermost swallow for the pathological case where the failure path itself throws.
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bridge-week step 12. Re-landed from
-fullwith the wire layer rewritten against protobuf.What
host_value_registryand rides asInboundValue.handle{HOST_VALUE_CALLABLE}. The engine's dispatch callback runs it fire-and-return on a detached thread, decodingBamlToHostCallargs (required by declared order, supplied optionals by name intoarg<T>slots) and completing viacomplete_host_callon every exit path.MyDomainError.code == 42after the round trip).baml::host_throw<ValidationError>crosses as the real typed BAML class, structurally caught by BAML'scatch (e: ValidationError). Abaml::errorrethrown through a callable transcodes its payload so class identity survives.Ty::Functionparams becomestd::function<Ret(Slots...)>with optional callable params asarg<T>slots;encode_callablecarries the declared wire names for by-name optional dispatch. Nested callables stay unsupported.register_host_dispatch_callback/register_host_release_callback/complete_host_callwere already in the v1 ABI table.Deviations from python (documented in the test header)
std::functionis the sync path (blocking its dispatch thread is fine — dispatches are concurrent).Tests
test_host_callables.cc: 17 engine round-trip cases — simple/multi-arg/int/class-value callables, capturing lambdas, distinct registry keys, repeated invocation (incl. zero-N), native-exception identity round trips (std::runtime_error, std::out_of_range, custom type with fields),host_throwcaught in BAML and propagating back typed, BAML-side catch of HostCallable with demangled class_name, and the optional-args-in-callables matrix (all-unset host defaults / by-name partial / all-set).All 12
sdk_test_cppfixture jobs pass locally. Parity workflow to follow.Summary by CodeRabbit