Skip to content

fix(remote): arm the statistics floor on delivery, not on the attempt - #228

Open
veksen wants to merge 1 commit into
mainfrom
fix-stats-push-delivery
Open

fix(remote): arm the statistics floor on delivery, not on the attempt#228
veksen wants to merge 1 commit into
mainfrom
fix-stats-push-delivery

Conversation

@veksen

@veksen veksen commented Aug 12, 2026

Copy link
Copy Markdown
Member

Rebased onto main now that #229 has merged. The diff is the three files below.

Goal

A project's production statistics should refresh at least daily, so CI costs a run against numbers that match production. The site project has gone five days without a capture, and a docs-only PR failed CI with nine cost regressions its diff did not cause.

This PR stops a lost push from being recorded as delivered. It is one of several things wrong on that path and it is not the one that silenced captures; see the note at the bottom.

What

Before: a statistics dump that never reached the server was recorded as if it had. The next attempt came 24 hours later, and so did the one after that.

After: the daily floor advances only when the server accepts the dump. A push lost to a dropped socket takes the existing 15-minute backoff and is retried.

How

Read src/remote/remote.ts first. applyStatistics set lastStatsPushAt and then emitted statsApplied; the listener in api-client.ts pushes over a fire-and-forget pushStats whose rejection was only logged. Arming on the attempt made a dropped push indistinguishable from a stored one. The field's own comment already claimed it "only advances on success", so this makes the code match what it said.

markStatsPushed and markStatsPushFailed are the new seam. Only the api-client knows whether the server took the dump, so it reports the outcome back rather than the emit assuming it.

refreshStatsIfStale no longer clears retryStatsAfter after a successful dump. The push it starts settles a microtask later, so clearing it there wiped the backoff that a failed push had just set, and the next poll re-dumped immediately. The regression test caught this on its first run.

The drift baseline still moves at apply time. It records what this analyzer dumped rather than what the server stored, and rolling it back on a failed push would re-trigger Size Drift on every poll.

Tests

src/remote/stats-push-delivery.test.ts drives the real wiring: a fake pushStats through hookUpApiReporter against a real Remote.

The load-bearing case calls refreshStatsIfStale three times and counts dumps — once when the floor is due, once immediately after the lost push, once after the backoff lapses. Asserting on the floor field alone would stay green while the analyzer sat silent for a day, which is the actual symptom.

Verified as a regression test by restoring the old arming line: three of the five cases fail, including "expected dumpSourceStats to be called 2 times, but got 1".

Full suite passes. npm run typecheck and npm run build are clean.

What this does not fix

Captures stopped because Size Drift went quiet, not because a push was lost. The 2026-08-07 08:38 capture recorded project_queries at 6,186 rows, the high side of an estimate that swings because the table is bloated. Drift measures movement against the baseline as denominator with a 0.5 ratio, and from 6,186 the low side is a 42% drop. Computed against live pg_class, the largest mover is 41.6% and the verdict is not drifted.

That leaves the daily floor as the only trigger, and analyzer logs from 2026-08-11 show it never fired on a container that had been up for four days. Why is still open.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor — 6 successful checks

 Cost regression — No query went up more than 5%
 Untested data access — No changed data-access file without a test
 New query — No new queries
 New query with index recommendation — No new query ships an index recommendation
 Schema drift — No schema changes
 High-value nudge — No index or rewrite past the threshold


More details via MCP → get_ci_run({ runId: "019ff8d6-466b-79f4-b9a5-5fcd3e148288" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.

@veksen
veksen changed the base branch from main to feat-relay-observability August 12, 2026 19:54
@veksen
veksen force-pushed the fix-stats-push-delivery branch from 366c8bd to 837d78e Compare August 12, 2026 19:55
Base automatically changed from feat-relay-observability to main August 12, 2026 20:05
@veksen
veksen force-pushed the fix-stats-push-delivery branch from 837d78e to e21ae31 Compare August 12, 2026 20:05
A project went five days without a production-statistics capture while its
analyzer was connected the whole time.

applyStatistics set lastStatsPushAt and then emitted statsApplied, and the
listener pushes over a fire-and-forget pushStats whose rejection was only
logged. A dump lost to a dropped socket was therefore indistinguishable from a
stored one: the floor was armed regardless, and the next attempt was a day away.
The field's own comment already claimed it "only advances on success".

The floor is now armed by markStatsPushed, which the api-client calls when the
server accepts the dump. A failed push calls markStatsPushFailed, which takes
the existing 15-minute backoff rather than leaving the floor past due; left past
due, a broken connection would earn a fresh source dump on every 60s schema
poll.

refreshStatsIfStale no longer clears retryStatsAfter after a successful dump.
The push it starts settles a microtask later, so clearing there wiped the
backoff a failed push had just set.

The drift baseline still moves at apply time. It records what this analyzer
dumped rather than what the server stored, and rolling it back would re-trigger
Size Drift on every poll.

The regression test drives refreshStatsIfStale twice and counts the dumps, since
asserting on the floor field alone stays green while the analyzer sits silent.
Restoring the old arming line fails three of its five cases.

Co-Authored-By: Claude <noreply@anthropic.com>
@veksen
veksen force-pushed the fix-stats-push-delivery branch from e21ae31 to e8b1515 Compare August 13, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant