Skip to content

Firestore emulator intermittently withholds a new client's first listen results for ~5 s while idle (2-vCPU Linux runner) #11163

Description

@mythos-tech

Summary

On a GitHub-hosted 2-vCPU Linux runner, a web client that opens a new
Firestore channel and registers its listen targets intermittently receives
no listen results for ~5–6 s, then receives them all at once. During the
wait the emulator JVM and the whole machine are idle (≈ 98–99 % CPU idle,
no GC pause over 28 ms, no JVM safepoint for 4.6 s), and the browser has a
backchannel request open to the emulator. It never reproduces on a fast
developer workstation (Windows, same versions). Roughly 2 in 10 full e2e
runs hit it on firebase-tools 15.31.0; roughly 4 in 10 on 15.30.2 (same
emulator jar).

Versions

Component Version
firebase-tools 15.31.0 (also seen on 15.30.2)
Firestore emulator 1.22.0 (the jar both firebase-tools versions ship)
Firebase web SDK 11.10.0 (firebase npm)
Transport experimentalForceLongPolling: true, useFetchStreams: false (also seen with long polling NOT forced — streaming)
Browser Chromium (Playwright 1.61.1, chromium-1228)
JDK Temurin 21
Runner GitHub ubuntu-latest (private repository → 2 vCPU)

What the application does

A React SPA. Each page subscribes 5–15 onSnapshot listeners (simple
where tenantId == X queries, a few with orderBy + limit 500, one
orderBy on a field to collect corrections). Every page load opens a new
channel session; client-side navigation adds and removes targets on the same
session. Collections are small (largest: 3 929 telemetry docs; every query
shape a page issues answers in ≤ 0.4 s against the same emulator when timed
directly over REST).

What happens

  1. The page loads; the SDK opens a channel (POST .../Listen/channel?…RID=…)
    and posts its targets.
  2. Intermittently, the backchannel GET .../Listen/channel?…TYPE=xmlhttp
    is held open by the emulator for 5 488 ms (one run) / 5 902 ms
    (another) with no data, although targets are registered.
  3. Then every pending listener resolves at once.

Because the SDK raises snapshot events only once the server reports the batch
consistent, every listener registered in that batch waits — a 1-document
tanks query stalls alongside the rest.

Evidence (all from failing CI runs; artifacts retained 14 days)

  • Chrome net log of the failing browser (run 35981785621, attempt 4):
    975 backchannel requests in the whole e2e run; exactly one held longer
    than 3 s (5 488 ms) — the stall. In that instance the emulator had closed
    the previous backchannel mid-response (net_error -100 ERR_CONNECTION_CLOSED)
    1.5 s earlier; six other such closes in the run were harmless.
  • The same shape without long polling (streaming; run 35985336810,
    attempt 6): the stream held open 5 902 ms with no data after the page posted
    its targets, no connection close anywhere near it.
  • No browser-side queueing: zero SOCKET_POOL_STALLED_* events in the net
    log — it is not Chrome's per-host connection limit.
  • Emulator connection log (firestore-debug.log): the browser opens no new
    connection during the stall window (e.g. 09:12:17 → 09:12:25, 09:29:03 →
    09:29:08), after ~13–24 in the two seconds before — i.e. the client has its
    requests outstanding and is waiting.
  • Not GC, not CPU (run 35990275614, attempt 4): -Xlog:gc*,safepoint for
    the emulator JVM — longest pause in the whole run 28 ms; vmstat -t 1 — the
    runner was 98–99 % idle from 11:27:09 to 11:27:12, mid-stall; the JVM
    logged no safepoint for 4.6 s.
  • Named pending reads (the application's stall gate): e.g.
    "SILENT STALL on /claim/2026-06 … Reads still waiting: claimPeriods,
    deliveries corrections, deliveries [period], dispensingEvents corrections,
    … workSessions [period]"
    (13 reads, the first page load of the suite,
    run 36019271182); and after a client-side navigation, only the two reads
    added in that navigation's batch (serviceRecords corrections,
    serviceRecords
    , run 36019327566).

Artifacts (per failing attempt): Playwright trace.zip, error context,
chrome-netlog-<pid>.json, firestore-debug.log, emulator-gc-<pid>.log,
vmstat.log. The two most recent failures, on firebase-tools 15.31.0:

The repository is private, so these links will not open for you. They are
kept until 2026-10-08, and I can attach any of the files here on request.
The data in them is synthetic seed data.

Ruled out

The dev server (reproduces on a production bundle served by vite preview);
a slow query (all ≤ 0.4 s); the long-polling transport specifically (also
streaming); Chrome's connection limit; JVM GC; CPU starvation.

Expected

Listen results for newly registered targets are delivered as soon as the
query has run (≤ 0.4 s here), not after an idle ~5 s.

Reproduction (as far as we have it)

Not yet reduced to a minimal case. It needs the slower runner: on a fast
workstation 25 consecutive fresh page loads resolved all listeners together
in a median of 482 ms (max 1 568 ms), never near 5 s. The full suite that
reproduces it is private; a reduced repro (one page, ~13 listeners,
reload in a loop on a 2-vCPU Linux VM) is the next thing I will build, and
I will add it here.

Mitigations in the application meanwhile

None that change the emulator. The e2e gate has no retries and no
inflated timeouts
by policy; instead a stall gate fails any page still
"Loading…" after 4 s (below the 5 s assertion timeout) with zero page errors,
naming every listener still waiting — so each occurrence is identified, not
mistaken for an application defect.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions