SH compact, header anti-DoS, and IBD stamp/pin rewind

[rearden-grok]

Do not merge until required checks are green.

Why

P1 leftover from the logical-error inventory (not covered by #443 / #444 / #445):

1. SH compact hole. compact_sealed_ovf took L0, dropped it, then installed L1 on a second mutex. locate_head could miss overflow keys. Occupancy walk never visited L1.
2. SH tip append at live_count == 0. Crash mid-finish leaves occupied head slots. Append skipped locate_head and dual-homed ingest over sealed main.
3. Stale most-work after equal-length reorg. disconnect_to did not truncate chain_work_prefix. When tip height came back the same, ensure_chain_work_prefix saw len == want and kept the losing branch's cumulative work.
4. Header persist/rank without POW. ensure_header only required a parent row. Pending ranking used claimed nBits, so an unmined compact target could look like more work and force getdata. Confirm still rejected the body; this is index/DoS.
5. Stamp/pin leftover identity. Stamp fail requeued the tail with block=None (BQ already taken) and continued the same lookup wave after clear_all. Pin fail did not clear_all. Next stamp hit parent create_fk unresolved.

What

• Compact: install L1, then drop L0. for_each_live_create walks L1. Tip append probes when heads are occupied even if live_count == 0.
• disconnect_to truncates chain_work_prefix to keep_height+1.
• ensure_header runs validate_header when the parent is on the best chain; otherwise claimed POW vs pow_limit. Pending ranking and getdata skip headers whose hash does not meet claimed nBits.
• Stamp/pin fail: requeue the tail with bodies, clear_all in-flight identity, bump the feed epoch so in-channel loadq is stale.

Test plan

• cargo test -p rbitcoin-store compact_merges_two_sealed_global_ovf_files
• cargo test -p rbitcoin-store append_after_zero_live_count_keeps_sealed_home
• cargo test -p rbitcoin-store --lib scripthash::
• cargo test -p rbitcoin-net --lib chain::tests
• cargo test -p rbitcoin-net --lib ibd::confirm::tests
• cargo test -p rbitcoin-net --lib ibd::events
• cargo test -p rbitcoin-net --lib ibd::reorg
• required GitHub Actions green

Out of scope

• P0 Reject same-block coinbase spends and rewind IBD lookup_taken_hi #443 (maturity + lookup_taken_hi).
• store: close Q-57 leftovers (fail-closed flush, fuse8, spender, seal, runs GC) #444 / Q-57 (fuse8, sidecar fsync, spender hop cap).
• mempool: close Q-58 persist order and worst-chunk eviction #445 / Q-58 (mempool persist order, eviction).
• P2 overlay (Electrum store: drop FdOnly ceremony and leftover ghost surface #28, gettxout deps(actions): bump the actions group with 2 updates #6/ibd: prune in-flight by confirmed tip #35, Esplora perf: BQ-ahead TipOnly head_fk; load stamps from hits #29/docs: bot fetch/push via HTTPS URL, leave origin on SSH #30, maxfeerate already elsewhere).

Made with Cursor

[rearden-grok]

auth probe

[rearden-grok]

Review

SH compact publish order, live_count==0 head probe, chain_work_prefix truncate, and claimed-nBits ranking all look right for the bugs they name. Required checks are green.

The stamp/pin leftover-identity slice does not restore the only production intake. I would not merge that part as written.

Must fix: stamp/pin rewind never returns wire to BQ

ConfirmFeed is readiness only. Lookup selects block_queue_unresolved_heights, then take_wave_items_for_load dequeues BQ before stamp/pin. load_fail_rewind_wave then:

1. clear_all (needed)
2. feed.clear() (epoch bump drops later loadq chunks of the same wave without requeueing their wire)
3. requeue_wire(tail) into feed.ready with cloned Blocks

Nothing in spawn_confirm_engine reads feed.ready wire. Lookup will not see those heights until BQ is filled again. load_fail_rewind_keeps_tail_wire_and_clears_lookup only inspects the feed mutex, so it cannot catch this.

Recovery today is a different, untested path: reject rewinds lookup_taken_hi, densify demote_zombie_pending_for_fetch (pending + no matching BQ), re-getdata. That can unstick IBD, but it is not “keep tail wire because BQ is already taken,” it re-downloads the wave, and the cloned tail sits in ready unused.

Either:

• re-offer the tail (and stale-dropped loadq chunks) through block_queue_offer / promote, then requeue_hashes, or
• drop the clones, document re-getdata as the recovery, and add an engine-level test that stamp/pin fail leaves those heights claimable again (BQ or densify), not merely feed.ready.

(*w.block).clone() plus requeue_wire’s inner clone is a full Block copy off the Arc for no consumer.

Other

• pending_path_claimed_pow_ok fails open when the next hash is missing from pending (return true). work_of_header_path fails closed (?). Needed because header_path_meets_minwork is a no-op once minwork is unset or already met — keep this gate, but a hole should not skip the POW walk.
• shorter_higher_work_fork_is_not_hopeless now asserts the attack (claimed mainnet nBits, nonce 0). The name and the old “real shorter higher-work still getdata” contract are gone. Rename; the legitimate case is still untested (regtest grind at a harder compact, or easier ChainParams).
• disconnect_truncates_chain_work_prefix_to_keep_height only checks len after rewind to 0. The equal-height bug was stale work at len == want. Truncate is the right mechanism; pin chain_work() after disconnect-to-fork + connect of a same-height winner.
• CodeQL “hard-coded nonce” on ensure_header_rejects_claimed_hard_bits_without_pow is a test literal. Do not shuffle it; dismiss the alert.
• Nit: put_create_batch_append // Crash mid-finish… restates the new condition.

Looks correct

• Compact: install L1, then take L0 — locate_head can see both, never neither. Occupancy walk visiting L1 matches 421 in compact_merges_two_sealed_global_ovf_files.
• Tip append entry_count() > 0 || !head_is_empty() is the crash-mid-finish hole; head_is_empty() only runs when live_count is already 0.
• ensure_header_fk: validate_header when height_of_hash hits (confirmed parent); else claimed POW vs pow_limit. Ranking/work_of_header_path no longer use nBits without validate_pow.
• Pin fail now clear_all + continue after sleep (no sync_body_hwm on a discarded wave). That matches stamp fail.