fix(bdk_electrum_streaming)!: Give the chain and anchors one owner

[evanlinjin]

Fixes #12. Fixes #19.

A reorg that moves a transaction into a different block of the same height leaves its Electrum script status untouched — the status is a hash over txid-height pairs — so the server never notifies the script hash. The anchor already delivered goes on pointing at a block that is no longer in the chain, nothing ever asks again, and the transaction stops being canonical for good.

Repair has to be driven from the chain tip, which is the one thing that does report the eviction. Doing that showed the split was in the wrong place. Anchors lived in SpkJob, so every script resolved them against a tip none of them could move — hence the resolve-the-whole-set-each-pass rule — and the chain layer had to synthesise script notifications to drive its own reorg repair. When one layer fabricates the other's events to do its own work, the boundary is wrong.

So ConfirmationJob replaces ChainJob and owns both the chain and the anchors, leaving SpkJob the history, the transactions in it and their prevouts. Header fetches batch into blockchain.block.headers instead of one blockchain.block.header per anchor.

State stages a single Update that both job kinds write into and hands it over once both are finished, so a transaction is never published ahead of the chain that anchors it. ConfirmationJob finishing is not enough on its own: it keeps offering the update until the last script is done with it. It is held back only until every script has its history, not until the scripts finish: the job works from the heights those histories name, so a script still downloading its own transactions has already told it every block it needs. Waiting for the downloads serialised the header and proof fetches behind them for nothing. Anchor scope comes from the statuses Subscriptions holds rather than from whichever jobs happen to be live, so a reorg re-verifies every script and not just the last one to notify.

Cache splits by who can reconstruct it. Subscriptions — per-script status and history — is persisted, because a status is a hash Electrum computes over the history it stands for that no wallet stores, and the server reports a history as it stands now, never again mentioning a transaction it dropped. TxCache — transactions, anchors, and which transactions paid a script — is not, because all of it is in the caller's wallet already; seeding it from wallet data is a caller's job, and a TxCache that starts empty is correct but re-downloads everything. response::Tx is Deserialize-only upstream, so histories round-trip through a private mirror until bitcoindevkit/electrum_streaming_client#20 lands.

Two ways to settle on a chain the server has left are fixed too, and neither subsumes the other (#19). ReqCoord::forget_job drops the superseded job's in-flight requests, so the replacement's identical request::Headers is not deduplicated against an answer already on its way from the abandoned chain. And a header batch has to link up to the tip that was actually announced, so a batch answered from a chain we were never told about is abandoned rather than adopted — the request there is current, only the answer is not. The notification carries its own header, so that header is never re-fetched and instead sits at the top of the run the batch must join onto; the linkage walk is what makes a mismatch visible.

Keying histories by status brought two faults of its own, both fixed here. A self-transfer with change pays two scripts the same transactions, so they share one status; dropping the outgoing status's history whenever one script moved on took it out from under the other, which then has no history and no notification coming, since its own status has not changed. The history is dropped only once no script answers to it. And a job waits for the history its notification's status stands for, so a server answering with a different one — it reorged since notifying, or the script's transactions dropped out entirely — left the job unable to find its own and asking again on every answer, unbounded and without backoff. SpkJob::awaiting_history compares the two, and the job is dropped rather than re-asked: the status the server actually holds is already on its way.

Breaking changes (→ 0.6.0)

• ChainJob and ChainJobOutcome are gone, replaced by ConfirmationJob, ConfirmationStage and ConfirmationProgress. JobId::Chain is JobId::Confirmation, and chain_job.rs becomes confirmation_job.rs.
• Named for what it owns rather than what it produces: it settles where transactions sit in the chain, and only two of an Update's three parts come from it — chain_update and tx_update.anchors. The rest are the scripts', and State assembles them.
• State::advance is State::poll. SpkJob::poll takes &mut self and returns anyhow::Result<SpkProgress>; SpkJobStage and TxsJobStage are replaced by SpkStage with an explicit Done. Additive: SpkJob::awaiting_history.
• SpkJob::poll is fallible so that a server answering blockchain.transaction.get with a transaction whose outputs cannot reach an outpoint we know is spent brings the connection down rather than panicking the state thread. That unimplemented! is on main; it and the txid check below are the two fixes here to code this PR does not introduce.
• ConfirmationStage gains Idle, so exhaustive patterns over it no longer compile. Done means the job resolved everything and still owes the caller an update, which it keeps offering until taken; Idle means nothing is owed — the update was taken, or the job was abandoned. One stage for both handed the same update over twice, and handed one over for a job that resolved nothing.
• ConfirmationJob::is_done is now true only for Done, and ConfirmationJob::set_idle is added. ConfirmationJob::reset is removed — nothing called it.
• ConfirmationJob's setters are set_tip, set_statuses and resolve_blocks — the first two no-op on an unchanged value, and set_statuses is called with the same set on every poll, so set_ says what they do.
• SpkHistories is Subscriptions, keyed by status rather than script hash. remove/insert/get/status are remove_spk/insert_spk/spk_history/spk_status. Its height index (spk_hashes_at_heights, prune, HEIGHT_INDEX_HORIZON) is gone — nothing read it once repair stopped being driven through script jobs.
• Cache::{txs, anchors, spk_txids} are now Cache::tx_cache::{…}, and a serialized Cache no longer carries them.
• Cache::failed_anchors is removed, and nothing replaces it. A proof the server refuses or contradicts cancels the job rather than being recorded against our block — an error proves nothing about that block either way. A script notification now builds a ConfirmationJob as well as a tip does, so a disagreement below the reorg window recovers without waiting on a block. The trade is deliberate: one pair the server will not prove now withholds the whole update rather than publishing the rest without that anchor.
• JobRequest::GetHeader is removed; nothing enqueues a single-header request any more.
• ReqCoord::pop returns Option<PoppedRequest>; ReqCoord::clear is removed. Additive: bump_chain_generation, forget_job.
• Two paths in the merkle-proof handler — a proof for a height not in our chain, and a block we hold with no header beside it — carry a debug_assert!. Both are reachable only if our own bookkeeping broke, so a debug build now panics where it used to log and continue. Release behaviour is unchanged.
• A blockchain.transaction.get response whose transaction does not hash to the requested txid now errors the connection instead of being cached under the wrong id. Behavioural, not a signature change, but a server that was quietly getting away with it no longer will. Predates this branch.

New dependency serde, already in the tree via serde_json.

Testing

cargo test — 24 deterministic (tests/state.rs), 5 live (tests/env.rs), 8 unit. Each fix was reverted in turn to confirm a test catches it.

Two exceptions, stated rather than glossed: ConfirmationStage::Done restoring itself after poll's mem::take, and the consistency guard parking in Idle rather than Done, are both reasoned rather than demonstrated. Reverting either produces redundant work rather than a wrong update, so no test catches them.

Every one of the seven commits passes cargo fmt --check, cargo check, cargo clippy --all-targets and the full suite on its own.

reorg_to_same_height_block_refetches_anchor_live confirms a tracked transaction, then re-mines it into a different block at the same height. Fails on main, passes here.

a_reorg_reanchors_every_script_not_just_the_last_to_notify is the one the single-script test harness was structurally blind to: the server now serves any number of scripts, matching a transaction to a script by the outputs it pays.

a_history_that_cannot_match_the_job_is_not_re_asked is capped at 50 requests so the unbounded re-ask fails the test rather than hanging it.

a_transaction_that_is_not_the_one_asked_for_is_rejected answers a blockchain.transaction.get with a different, perfectly valid transaction and requires the connection to come down.

a_merkle_error_below_the_reorg_window_is_recovered_by_a_script_notification refuses a proof at a height nothing will rewrite, so the tip never moves and the script notification is the whole of the recovery. It fails if a refused proof drops the job without something able to rebuild one.

confirmation_job_runs_ahead_but_the_update_waits_for_the_scripts pins both halves of the gate: it answers a script's history but withholds its transaction, asserts the job reaches proof fetching anyway, then answers the proof and asserts nothing is published — and only once the transaction lands does one update arrive carrying both it and its anchor. Re-announcing the same tip afterwards must produce nothing.

Known, not fixed here

• One script stuck on its history withholds every other script's update. Inherent to staging a single Update; the trade this PR makes deliberately. Scripts stuck downloading transactions no longer block the confirmation job itself.
• A single (txid, height) the server will not prove withholds the whole update, not just that anchor. The job is cancelled and rebuilt from the next notification rather than carrying a record of the refusal, so a server that refuses one proof indefinitely delivers nothing at all. Judged the server's fault to fix, and preferable to reporting a mined transaction as unconfirmed.
• TxCache has no seeding constructor, so a reconnect refetches everything.

[LLFourn]

Thanks for this — the diagnosis in #12 is right, the chain-driven design is the right
answer to it, and the write-up made the review much faster than it would otherwise have
been. I reproduced the bug against a live electrs and confirmed this branch fixes it.

Requesting changes. The first two are defects in the new code, both on the path this PR
exists to fix, and I have a failing test for each; the third is the modelling issue they
share, which I think is the one worth acting on first:

1. state.rs:256 — the stale-response guard covers the successful merkle answer but
   not the error answer. A server answers from the chain it had when it received the
   request; if a reorg has landed since, that error is about a block we have left
   behind. give_up_on_anchor blames it on the replacement block and writes two
   irreversible records — a permanent failed_anchors verdict, and the deletion of the
   txid from txids_by_height, which is the only record that would let the eviction
   path ask again. Since advance_anchors treats a failed anchor as resolved by
   omission, the job reports success with the anchor silently missing and the
   transaction stops being canonical for good. Three-line fix.

2. anchor_job.rs:55 — the refetch depends on the server answering in request order.
   A same-height reorg is applied by ChainJob's short-circuit without ever caching the
   replacement header, so the job asks for the header and the proof together. Proof
   first ⇒ cancel_jobs ⇒ anchor_job = None, unrecoverable; the header arriving
   second returns Ok(None) without advancing anything, so it cannot rescue it either.
   The protocol carries request ids precisely because ordering is not promised.

Underneath both: failed_anchors is a monotonic negative cache, which is only sound
for negatives that were proved. Right now it has two writers — a merkle root mismatch
(a proof) and any JSON-RPC error on get_merkle (not one). Keeping the connection up on
an error is right and well argued; recording a permanent disproof from it is the part I
would drop. Enforcing "only a verified proof writes failed_anchors" would make defect 1
unwriteable rather than fixed.

3. state.rs:468 — failed_anchors is a monotonic negative cache, and one of its two
   writers has not proved anything. A merkle root mismatch is a proof; a JSON-RPC error
   is not, and the predicate is the method name alone. So a rate limit, a syncing index or
   a daemon hiccup each permanently unconfirm a transaction. Worth stressing that the fix
   in (1) does not cover this — that guard closes the stale-chain case, this is a fault
   on the current chain. I ran the live unconfirming reorg against electrs to see what
   real classification would have to work with: the payload is a bare JSON string,
   "tx not found or is unconfirmed", which conflates the fault with the benign case. So
   I'd drop the durable write rather than try to classify it, and keep a classifier only
   as an optimisation.

The rest is smaller: cancel_jobs(JobId::Anchor) being terminal for the one job nothing
rebuilds; the GetHeader generation guard passing all 5 tests when deleted; the
breaking-change list missing SpkJobStage; txids_by_height retaining far more than the
~21 heights that can ever be read, which I'd like bounded. Details inline.

On testing — tests/env.rs runs fine here (cargo test --test env, 3 passed, 20s), so
the live path was never exercised in either direction. I have written two live tests:
one reproduces #12 end to end (fails on main, passes here), the other pins the
connection-survival fix (fails at 388d60a, passes here). Happy to hand both over along
with the fixes as a patch.