Skip to content

🛟 fix: Handle Gemini Heartbeats and Upstream Cancellation - #701

Merged
danny-avila merged 5 commits into
mainfrom
lia/google-stream-transport
Nov 17, 2026
Merged

danny-avila merged 5 commits into
mainfrom
lia/google-stream-transport

Conversation

@lia-by-librechat

@lia-by-librechat lia-by-librechat Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Google reasoning streams can freeze on SSE keepalives. Stop also leaves the archived SDK's upstream request and aggregate-response branch running.

Reconcile the current LangChainJS GenerateContentApiHandler HTTP/SSE flow with LibreChat's custom Google class:

  • Send abortable HTTP requests and parse SSE with eventsource-parser, without an SDK compatibility shim or handwritten SSE grammar.
  • Retain proxy URLs, headers, API versions, Gemini signed replay, tools, media, response metadata and usage accounting.
  • Keep system instructions request-local and cached-content requests on the same transport.
  • Abort/cancel on stream exit, including smoothing read-ahead. Preserve the original protection-marked signal for lazy protected streaming.
Google class → abortable fetch → SSE parser → existing Gemini converters → smoother

Deliberate differences from upstream: malformed/truncated JSON and provider errors fail rather than becoming empty chunks; stream exit cancels the reader and request. No gateway-specific [DONE] handling or broader LangChain migration.

Related to LibreChat-AI/LibreChat#18153.

Verification

Real local HTTP regressions cover heartbeats, fragmented UTF-8/CRLF, abort before headers and during idle streaming, early exit with smoothing on/off, protected budget rejection, cache/event paths and exact signed replay/metadata. Existing Google, smoothing and protection boundary tests remain.

Also checked: TypeScript, scoped lint/imports, package build, CJS/ESM public API smoke, circular dependencies, dependency audit and tracing regressions.

Not included: live Gemini/gateway verification, package publication or LibreChat's dependency update.

Fork Provenance

The fork began with custom header forwarding in June 2025. That alone no longer justifies a fork: current LangChain Google classes accept custom headers.

Local contract Why it exists
Native signed-part replay #679: preserve part order/signatures through restored history and Graph legacy projection; refuse stale replay after edits.
Model-specific prefill/media handling #323, #499, #620: prevent documented trailing-model-turn 400s and false local multimodal rejection.
Mixed built-in/function tool context #221: retain server-side tool parts/order and the explicit caller option.
Eager tool seals #237: mark complete Gemini calls for LibreChat's eager execution path rather than waiting for ToolNode.
Adaptive smoothing/protected laziness #389, #593: paced callbacks with bounded read-ahead; protection bypasses splitting/read-ahead to retain its release and byte-budget contract.
Usage shape #8, #679: emit cumulative stream usage once and include thinking tokens in output accounting.

Current upstream ChatGoogle already handles headers, thinking tokens and native thought signatures. Those capabilities are not unique fork requirements; LibreChat's persisted replay format, event metadata and caller contracts still need reconciliation. The old public completionWithRetry inputs/helpers remain compatibility obligations while inheriting the old class.

This PR follows upstream's HTTP/SSE approach. Its strict malformed/provider-error handling and explicit reader/request cancellation are deliberate, covered differences, not a new Gemini feature surface.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review head: c25a527c0f261560e95840df9e00cc93d3503e83. Replaces the Google HTTP transport with @google/genai, forwards upstream cancellation, and adds real HTTP heartbeat/cleanup regressions. Keeps the existing Gemini wire conversion and LangChain surface.

@lia-by-librechat

lia-by-librechat Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Review head: ece47f4e013e5fd99118c6108575a44c75eb71f9. Includes upstream abort on smoothed consumer exit (41c3e51bd) and cached system-instruction shorthand compatibility (ece47f4e0). Real HTTP regressions cover smoothing enabled/disabled and supported cached instruction shapes. Focused checks and CI are running for this head.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review head: 5a7a5e9262382f3c5627c16282c5a979b029d42d. Reconciles current LangChainJS GenerateContentApiHandler HTTP/SSE handling with the custom Google class. Keeps strict provider errors, raw Gemini replay/metadata and marked protection signals. Local validation: 386 Google/smoothing/protection tests, 211 tracing tests, TypeScript, scoped lint/imports, package build and CJS/ESM HTTP smoke pass. Supersedes the SDK-bridge approach in earlier heads.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review head: d22baca0ca4471bda4de66cd15964c392791a551. Preserves the inherited public completionWithRetry shorthand inputs and response helpers, with real HTTP regressions and comparison against the archived client's helper output. Prior cleanup, protection and signed-replay fixes remain. Focused validation and CI are running for this head.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Verified head: d22baca0ca4471bda4de66cd15964c392791a551.

Independent review: complete, no findings. Previous ledger: P1 smoothed-stream cleanup fixed in 41c3e51bd; P2 cached instruction shorthand fixed in ece47f4e0; P2 protection signal identity fixed in 5a7a5e926; P2 public completion shorthand/helpers fixed in d22baca0c. None rejected.

Local: 397 Google/smoothing/protection regressions and 211 tracing tests passed. TypeScript, scoped lint/import order, CJS/ESM build and HTTP smoke, circular dependencies and dependency audit passed. CI: all 13 validation jobs passed; the body-edit-only workflow was skipped.

The independent reviewer inspected frozen source and did not run dependency-based tests. Live Gemini/gateway tests were not run locally. No publication or LibreChat dependency update. Fork provenance and upstream reference are in the description.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head, final review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T12:39:11.619852Z d22baca Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: d22baca0ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila merged commit 553585b into main Nov 17, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants