Problem
When an upload fails, requeueInflight() appends the old inflight bytes directly to the producer replacement queue. If that replacement ends in a partial JSON record or a complete record without its trailing newline, the first recovered row is concatenated to it. The next drain treats the combined line as malformed and loses a previously durable row.
Reproduced against main ce30de7ca94115cb73fa991538c6d2595ac2622a with synthetic data only.
Focused fix and ownership
Inspect and append through the same a+ descriptor and insert a separator only when the nonempty replacement queue is unterminated. Reuse the existing newline helper; close the descriptor in finally and remove inflight only after append succeeds. Empty and already terminated queues do not gain an extra byte.
I have prepared the implementation and checked the existing issue/PR inventory for overlapping work.
Reproduction and validation
The five-case negative-control matrix has three failing cases and two passing controls on unchanged source. With the patch, all five pass. Tests exercise failed uploads with partial/complete/terminated replacement tails, stale inflight recovery, and absent replacement queues, then verify row identities and no second-flush replay. The Linux surrounding run passes 48 tests; independent macOS review passes 43 queue and append-atomicity tests.
The exact proposed source commit 728aa7de466e51ba05174eb5ec7e92576456a48b passes the full Node 22/Linux suite with coverage: 5854 tests passed, zero failed, as well as typecheck/build and repository quality gates. Evidence: https://github.com/DivyamTalwar/hivemind/actions/runs/35659098663
Scope and limits
This is a record-boundary fix, not an outage-isolation, queue-capacity, cross-workspace migration, or distributed-lock redesign. Crash-after-append-before-removal and disk-failure semantics are unchanged. Existing open PRs #61 and #65 were checked at their actual head source: both still perform the unseparated append and do not include this fix.
Problem
When an upload fails, requeueInflight() appends the old inflight bytes directly to the producer replacement queue. If that replacement ends in a partial JSON record or a complete record without its trailing newline, the first recovered row is concatenated to it. The next drain treats the combined line as malformed and loses a previously durable row.
Reproduced against main
ce30de7ca94115cb73fa991538c6d2595ac2622awith synthetic data only.Focused fix and ownership
Inspect and append through the same a+ descriptor and insert a separator only when the nonempty replacement queue is unterminated. Reuse the existing newline helper; close the descriptor in finally and remove inflight only after append succeeds. Empty and already terminated queues do not gain an extra byte.
I have prepared the implementation and checked the existing issue/PR inventory for overlapping work.
Reproduction and validation
The five-case negative-control matrix has three failing cases and two passing controls on unchanged source. With the patch, all five pass. Tests exercise failed uploads with partial/complete/terminated replacement tails, stale inflight recovery, and absent replacement queues, then verify row identities and no second-flush replay. The Linux surrounding run passes 48 tests; independent macOS review passes 43 queue and append-atomicity tests.
The exact proposed source commit
728aa7de466e51ba05174eb5ec7e92576456a48bpasses the full Node 22/Linux suite with coverage: 5854 tests passed, zero failed, as well as typecheck/build and repository quality gates. Evidence: https://github.com/DivyamTalwar/hivemind/actions/runs/35659098663Scope and limits
This is a record-boundary fix, not an outage-isolation, queue-capacity, cross-workspace migration, or distributed-lock redesign. Crash-after-append-before-removal and disk-failure semantics are unchanged. Existing open PRs #61 and #65 were checked at their actual head source: both still perform the unseparated append and do not include this fix.