wallet: implement the BDK wallet improvement plan - #179
Merged
Conversation
The initial full scan only requested external SPKs, so a wallet restored from seed did not find its change outputs. Scan the internal keychain as well, raise the stop gap from 10 to 50, and make five esplora requests in parallel instead of one.
The sync path returned early when the wallet tip equaled the chain height, so unconfirmed transactions stayed invisible until the next block. Always run the sync; the timer in ddk.rs bounds the cost.
Seed each incremental sync with the txids we expect under our SPKs. Esplora reports the ones that no longer show up, BDK stamps them evicted_at, and replaced or evicted transactions drop out of the canonical view. Without this a stuck pending balance never recovered when its transaction left the mempool.
The incremental branch wrote the wallet's current derivation indices into the update as last-active indices. A sync must not set them; only a full scan learns which indices are active. Use the BDK conversions for both branches and make incremental syncs use the same request parallelism as full scans.
The lock file already resolved 3.1.0; raise the manifest floor so fresh builds get the add_foreign_utxo non-witness validation fix and the Utxo::txout panic fix for foreign UTXOs, both on our splice code paths.
Each sign_psbt_input call cloned the full PSBT and signed every input, and the manager calls it in a loop over the funding inputs. Sign the PSBT in place on the first call and skip inputs that are already finalized, so a funding transaction costs one signing pass.
Move coin selection into the wallet actor. Selection skips locked outpoints, and when the manager asks for locked UTXOs the selected outpoints are locked through BDK and persisted. Two concurrent offers can no longer fund themselves with the same coins, and the locks survive a restart.
unreserve_utxos was a no-op, so coins locked for a failed offer or a rejected contract stayed unspendable. Unlock them through BDK and persist the change. The manager trait method is synchronous, so it fires the unlock command without waiting; unlock_outpoints on the wallet API waits for persistence.
The postgres backend decomposes the BDK changeset into tables and silently dropped the locked_outpoints field, so a UTXO lock did not survive a restart. Add a locked_outpoints table, upsert with the merge semantics of the BDK changeset, and read it back on initialize. Sled and memory serialize the whole changeset and already round-trip the locks; a sled test now proves it.
A signed funding transaction can stay unbroadcast while the counterparty finishes the protocol, and its inputs were still selectable. Lock the wallet-owned inputs of a PSBT when the wallet signs it, release the lock when a confirmed transaction spends the outpoint, and expose locked_outpoints() so consumers can inspect the reserved coins. Unconfirmed spends keep their locks: an evicted transaction returns the coin to the spendable set.
The reported balance treated locked coins as available. Report spendable (unspent minus locked) and reserved (locked unspent) from the same wallet snapshot, surface both on crate::Balance, and return the richer WalletBalance from the wallet API.
The 2-of-2 funding output of a contract is invisible to the BDK wallet. Track it beside the wallet: an SpkTxOutIndex keyed by contract id over its own TxGraph, sharing the wallet's local chain view. The tracker reports funding confirmation, spend of the funding output, and a chain-truth balance, and stages a serde changeset for persistence.
Each wallet sync now registers the funding script of every contract in storage and runs a second, targeted sync request over the funding SPKs and outpoints. Esplora resolves outpoint spend status, so one round trip learns both confirmation of the funding transaction and any spend of the funding output (CET, refund, or counterparty close).
The contract balance came from collateral math over confirmed contracts. Report contract_confirmed and contract_pending instead, computed from the tracked funding outpoints against the wallet's local chain, and keep PnL from contract state. crate::Balance replaces the contract field with the two chain-truth fields.
Load the tracker changeset at wallet startup and persist staged changes after each sync through two new Storage trait methods. Memory merges in RAM, sled stores merged JSON under its own key, and postgres keeps one merged JSONB document per wallet. The trait methods default to a no-op so external backends keep compiling; without persistence the tracker rebuilds from contract storage and a chain sync. Funding script registrations serialize as a list of pairs, because a JSON map cannot key on a byte array.
contract_utxos() lists the tracked funding outputs with their chain state (confirmation and spending transaction), so a consumer such as ddk-node or an FFI binding can show locked collateral per contract and observe a counterparty close. The end-to-end contract test now asserts the tracker sees the confirmed funding output on both sides and its spend after the CET broadcasts.
Apply sync updates with apply_update_events and forward the BDK wallet events (TxConfirmed, TxUnconfirmed, TxReplaced, TxDropped, ChainTipChanged) on a broadcast channel. A transaction event triggers the manager's periodic check right away instead of waiting for the next timer tick, and consumers get reorg-aware confirmation notifications through subscribe_events.
The wallet's fee estimator returned constants and the esplora client returned one sat/kw. Each sync now fetches esplora's fee estimates into a lock-free cache keyed by confirmation target; the constants stay as floors and as the fallback while no estimate has arrived. The wallet delegates to the client's cache, every confirmation target has an entry (the old map panicked on MaximumFeeEstimate and OutputSpendingFee), and a failed fetch never fails a sync.
SendToAddress and SendAll shared the same build-sign-broadcast body. One helper now builds the transaction for either spend shape, signs, broadcasts, and also persists the wallet so the change index a build reveals survives a restart (both paths previously dropped it).
Labels for transactions, addresses, and outputs use the bip329 crate, keyed by record type plus reference so an input and an output record on the same outpoint coexist. Memory, sled, and postgres implement load, upsert-by-reference, and delete; the trait methods default to an empty store. DlcDevKit exposes set_label, labels, and delete_label.
Each sync labels the funding transaction and funding outpoint of a newly tracked contract with the contract id, and the broadcast CET of a closed contract with the attested outcome. DlcDevKit exports and imports the labels as BIP-329 JSONL. The end-to-end test asserts the labels and a JSONL round trip after settlement.
send_with_coin_control takes caller-selected UTXOs (spent exclusively via manually-selected-only), an unspendable exclusion list, and a confirmation floor. send_to_address and send_all stay as thin wrappers over the same command.
bump_fee builds an RBF replacement of an unconfirmed wallet transaction through Wallet::build_fee_bump (RBF is on by default in BDK 3.x), signs, broadcasts, and persists it. The send and bump paths share one sign-and-broadcast helper.
The 25 000 sat minimum change size of coin selection was a hardcoded constant. WalletConfig carries it, new_with_config takes it, and the DDK builder exposes set_min_change_size; the existing constructor and default stay unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements all five phases of
docs/wallet-improvement-plan.md, one commit per plan item: sync correctness fixes, persistent UTXO reservation with BDK outpoint locking, chain-truth tracking of contract funding outputs, wallet events with live fee estimation, and BIP-329 labels with coin control.Stacked on
postgres-storage-fixes(#178); merge after it.Changes
Phase 1 — correctness
bdk_walletmanifest floor raised to 3.1.0Phase 2 — UTXO reservation
unreserve_utxosunlocks through BDK instead of being a no-oplocked_outpoints(new migration); sled and memory already round-trip the fieldcrate::Balancereportsspendable(unspent minus locked) andreserved(locked)Phase 3 — contract UTXO tracker
ContractUtxoTracker: anSpkTxOutIndex<ContractId>over its ownTxGraph, sharing the wallet's local chain viewcontract_confirmed/contract_pendingcome fromTxGraph::balance()over the tracked outpoints instead of collateral math; PnL stays from contract stateStoragetrait in memory, sled, and postgres (new migration), with no-op defaults for external backendscontract_utxos()on the wallet and DDK API lists locked collateral per contractPhase 4 — events and fees
apply_update_events;WalletEvents forward on a broadcast channel and transaction events trigger the manager's periodic check immediatelyConfirmationTargetnow has an entry (the old map panicked onMaximumFeeEstimateandOutputSpendingFee)SendToAddress/SendAllshare one build-sign-broadcast helper that also persists the revealed change indexPhase 5 — labels and coin control
bip329crate, stored through theStoragetrait in all three backends (new migration), keyed by record type plus referencesend_with_coin_control: caller-selected UTXOs, an unspendable list, and a confirmation floorbump_feereplaces a stuck unconfirmed send via RBFWalletConfig/builder optionTesting
cargo test -p ddk --features "sled,postgres" --lib: 83 tests pass, including new tests for restore-from-seed, mempool visibility, RBF eviction, concurrent selection, lock restart survival, sign-time locking, balance split, tracker unit tests, storage round trips, fee cache, events, labels, coin control, and fee bumpingcargo test -p ddk --test enumeration -- --ignored(now also asserts tracker state, chain-truth balance, and auto-labels through settlement) andcargo test -p ddk --test short_call -- --ignoredcargo clippy -p ddk --all-featuresshows only pre-existing warnings in untouched oracle files