Skip to content

feat: bind native host models with scoped admission and lifecycle - #696

Merged
danny-avila merged 7 commits into
mainfrom
danny-avila/host-model-binding
Nov 17, 2026
Merged

danny-avila merged 7 commits into
mainfrom
danny-avila/host-model-binding

Conversation

@danny-avila

@danny-avila danny-avila commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Remote SDK graphs need an authenticated host binding behind HostChatModel so every integration does not reimplement model authorization and lifetime ownership.

createHostModelService binds an already configured native model to a fixed execution identity, model reference and private snapshot of approved function tools. It validates every call, snapshots native messages (including binary metadata and tool artifacts) before asynchronous dispatch, keeps host configuration/callbacks authoritative and isolated from foreign ambient LangChain configuration, translates tool choice using the SDK provider registry, and combines host cancellation with invocation cancellation.

Required limits bound request/response bytes and duration. A reusable HostModelAdmission scope caps concurrent calls and aggregate request bytes across bindings, rejects full capacity before serializing another request, and retains ownership while cancelled/timed-out provider work or cleanup remains outstanding. Streaming is pull-based with no prefetch; idle-stream deadlines trigger cleanup even while the consumer is paused.

A failure-injection test exposed that LangChain's ReadableStream wrapper can return from cancellation before generator cleanup drains. The binding uses the same native _streamIterator/callback path without that wrapper and retains capacity through actual read and cleanup promises. Fully consumed streams release capacity immediately. Early stream termination closes the native callback run once after cleanup, preserving observed identity/tags and reported usage; missing usage is explicitly incomplete. Usage is captured independently of released output, so a final usage-bearing chunk rejected by response limits still reaches host accounting callbacks. The callback-composition helper is shared with existing SDK preemption, and admission also covers awaited terminal callbacks.

CI also covers PRs targeting the established danny-avila/** branches and base-branch edits, allowing dependent slices to be validated before their parents merge. Ignored body/title edits use an isolated concurrency group so they cannot cancel real validation.

This completes the SDK host binding. LibreChat transport authentication, bounded event transport/recovery, builder integration, protection/billing wiring and product/UI acceptance remain separate integrations; this PR does not enable protected remote saved agents. Admission is process-local and cannot account for hidden work a custom model conceals after its own promise/iterator settles. Non-function tools, dynamic schemas and child/fallback authorization remain documented gaps.

Validation: 57 focused native-binding tests, including a real SDK graph; 413 harness/tracing/preemption tests; Bun and TypeScript checks; scoped ESLint; CJS/ESM/declaration build; and circular-dependency checks. No package version change. Builds on merged #695; SDK release remains a separate commit/PR.

Live Langfuse observation-tree verification was not run because this checkout has no configured Langfuse credentials; local tracing and preemption regressions passed.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head 1d3f51a against its stacked base (#695). Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T01:34:37.139528Z 2bcc378 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head eb1ca65 against its stacked base (#695). Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

The follow-up extends the CI target-branch filter to the established danny-avila/** branches; previously stacked PRs received no CI until their parent merged. Runtime code is unchanged.

@danny-avila
danny-avila force-pushed the danny-avila/host-model-binding branch from eb1ca65 to aa5f3c0 Compare November 17, 2026 00:59
@danny-avila
danny-avila changed the base branch from danny-avila/host-model-service to main November 17, 2026 00:59

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head aa5f3c0 against main. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

Parent #695 is merged. This branch is rebased and retargeted to main; the complete file tree is unchanged from eb1ca65 (verified with git diff).

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head 923cfb5 against main. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

This follow-up enables CI on base-branch edits as well as pushes; retargeting a stacked PR previously did not start a run on its new merge result. Body edits are excluded from the extra trigger. YAML validation passes; runtime code is unchanged.

@danny-avila danny-avila mentioned this pull request Nov 17, 2026

danny-avila commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Local performance check on built CJS head 8342fc2 (including native snapshots, callback finalization and host-context isolation): deterministic 512-character response in 16 chunks; 10 warmups and 100 interleaved direct/hosted samples per history size; 256 characters per input message. No provider network or UI.

History Direct / hosted first-chunk p50 Direct / hosted completion p50
1 messages 1.48 / 2.12 ms 22.44 / 24.83 ms
32 messages 1.48 / 2.64 ms 22.43 / 25.22 ms
256 messages 1.48 / 4.84 ms 22.68 / 27.23 ms

Admission returned to zero after all samples. This measures local facade/binding overhead for this fixture; it is not evidence of cross-process or Agents UI performance parity. Those acceptance tests belong to the LibreChat transport integration.

Source-based invariant audit covered identity/reference/tool validation before dispatch, immutable native message snapshots, shared admission across bindings, cancellation/read/cleanup settlement, ambient configuration isolation, native terminal callbacks including malformed partial aggregation, and safe errors without retries. Existing preemption reuses the extracted callback helper and its regressions pass. Codegraph was unavailable (connector authentication); no graph-verification claim. The host binding is an in-process contract, with wire codecs, recovery and capability negotiation explicitly left to the transport layer.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 923cfb5479

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml Resolved

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head bd977d6 against main. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

Fixed the valid CI concurrency finding in f62f503: ignored body/title edits use a run-specific group and cannot displace real validation. Also fixed a source-audit finding: native message snapshots now preserve binary/structured metadata and tool artifacts instead of JSON coercion; host-local byte accounting uses V8 serialization, not a transport wire format. 370 harness/tracing tests, both typecheckers, scoped lint, build and cycles pass.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd977d64e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/harness/modelService.ts Resolved

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head 8342fc2. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

The callback lifecycle fix reuses native callback composition, closes early returns/timeouts/overflows once after actual provider cleanup, preserves observed run identity/tags and reported usage, and isolates host operations from foreign ambient LangChain configuration. Source-based lifecycle audit also covered aggregation failures, normal/error completion, and admission through awaited callbacks. 410 focused harness, tracing and preemption tests pass; Bun/TypeScript, scoped lint, production build and runtime circular-dependency checks pass.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8342fc29cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/harness/modelService.ts Resolved

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head 2bcc378. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

The valid overflow-usage finding is fixed: host usage is accumulated using native mergeUsageMetadata independently of released content and before byte checks. Synthetic completion preserves usage/cached/reasoning details and llmOutput.tokenUsage even when the final chunk is rejected. Tests cover cumulative overflow, a single oversized tail, split usage without double counting, native completion/errors and early cleanup. 413 harness/tracing/preemption tests pass, plus both typecheckers, scoped lint, production build and runtime-cycle checks.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 2bcc37899a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila merged commit 49c0001 into main Nov 17, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant