Skip to content

Fix SSE streaming through the internal cross-node proxy - #623

Merged
digitaldan merged 1 commit into
openhab:mainfrom
digitaldan:fix/internal-proxy-sse-streaming
Jul 4, 2026
Merged

digitaldan merged 1 commit into
openhab:mainfrom
digitaldan:fix/internal-proxy-sse-streaming

Conversation

@digitaldan

Copy link
Copy Markdown
Contributor

Problem

Server-Sent Events (SSE) proxied through the cloud — e.g. GET /rest/events/states and /rest/events — close after the first event instead of staying open.

The cloud runs multiple node processes behind nginx, which routes proxied requests to the node holding the openHAB Socket.IO connection via the CloudServer affinity cookie. A client without that cookie (any first request, a REST/CLI client, etc.) is round-robined to an arbitrary node. When that node isn't the one holding the connection, ensureServer re-proxies the request internally to the correct node via proxyToServer().

That internal http.request was created with timeout: 5000. Node applies the timeout option as a socket idle timeout for the life of the connection, not just connection setup. So a long-lived streaming response that goes quiet between events (e.g. an unsubscribed states stream between heartbeats) tripped the 5s idle timeout, the internal request was destroyed, and the whole stream collapsed — the client saw the connection drop shortly after the first event.

Fix

All in proxyToServer (src/routes/middleware.ts):

  1. Clear the idle timeout once the upstream response begins streaming (proxyReq.setTimeout(0) in the response callback). The timeout now only guards connection setup / time-to-first-byte, so a dead or unreachable target node still fails fast, but a healthy streaming response is no longer torn down between events.
  2. Bump the timeout 5s → 10s — the 5s connect/first-byte budget was aggressive for a busy target node.
  3. Destroy the internal proxy request when the client disconnects (res.on('close')). .pipe() does not propagate the client-side close to the source request, so with the idle timeout gone this is what reaps the upstream connection when the client goes away — making teardown deterministic and immediate rather than relying on any idle timeout.

Testing

  • npm run typecheck — clean
  • Existing ensureServer unit tests pass (two mock res fixtures updated to include an on stub, since the real Express Response is an EventEmitter).
  • Verified against a live deployment: with the fix, GET /rest/events/states proxied through a node that internally re-proxies to the connection-holding node stays open across heartbeat gaps; without it the stream dropped ~5s after the last event.

Proxied requests that land on a cloud node not holding the openHAB
Socket.IO connection are re-proxied internally to the correct node via
proxyToServer(). That internal http.request used a 5s `timeout`, which
Node applies as a socket idle timeout for the life of the connection —
so long-lived streaming responses (the REST SSE endpoints /rest/events
and /rest/events/states) were torn down after the first quiet gap
between events, collapsing the whole stream.

- Clear the idle timeout once the upstream response begins streaming, so
  SSE responses that sit idle between events are not destroyed. The
  timeout now only guards connection setup / time-to-first-byte, and is
  bumped 5s -> 10s to be less aggressive on a slow target node.
- Destroy the internal proxy request when the client disconnects
  (res 'close'), since .pipe() does not propagate the client-side close
  to the source request. This reaps the upstream connection immediately
  and removes any reliance on an idle timeout for teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Dan Cunningham <dan@digitaldan.com>
@digitaldan
digitaldan merged commit 65b25fa into openhab:main Jul 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant