WIP: pplns pool_mode — pooled mining without operator variance - #61
Merged
Conversation
Groundwork for #48. No PPLNS accounting yet — this is the mode plumbing it needs, and one refactor that has to happen first. `pps_enabled` was doing two unrelated jobs. It decided whether the coinbase pays the pool or the miner, and it decided whether a stratum username is a Thunder address or a Bitcoin one. Those happened to move together across the only two modes that existed, so one flag was enough: mode coinbase pays username solo the miner bitcoin pps-classic the pool thunder pplns-btc is the combination that breaks it. It pools the reward like PPS, so the coinbase pays the pool, and it pays out on L1 like solo, so the username is a Bitcoin address. No single flag expresses that, so it is now two: coinbase_pays_pool and username_is_thunder. A third came out of the same split. The accrual gate suspends crediting when network difficulty is below what PPS can safely price a share at, and it was reading pps_enabled. Rewriting it to read the gate pointer instead looked equivalent — main.c installs that pointer unconditionally, so it would have gated solo and both PPLNS modes, refusing miners from modes that never accrued anything to suspend. test_solo_is_never_gated caught it, which is exactly the defensive property it was written for. So the gate keys on pps_accrues, true only for pps-classic: it is the only mode that prices a share when it arrives and can therefore misprice one. PPLNS values a share in hindsight, out of a block actually found, so there is nothing to gate. The two pplns values are one knob rather than a mode plus a rail knob, per the decision on the issue: an operator runs Thunder or L1, never both, because the rail decides what a username is. One value makes the inconsistent configuration unrepresentable instead of merely rejected. `pool_mode = pplns` on its own is refused with a message naming the two real values, checked before the generic catch-all since it is the likely typo. Window size is pplns_window_diff_multiple, a multiple of current network difficulty rather than an absolute share count, so it self-scales across retargets — an absolute window silently changes meaning ~4x at each of the forknet retargets. Required > 0, and warned below 1.0, where a block pays out across less work than it took to find and rewards hopping. Still to come: the distribution step itself (read the window on a block maturing, credit pps_credits pro rata) and the L1 payout rail in the payout worker, which mirrors ThunderClient's small surface — balance, batch send, confirmation — against bitcoind's already-generic rpc_call. 379 stratum assertions (was 374), clean under ASan/UBSan, and proxy.conf.example still loads with no unknown key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The distribution step from #48, and the last piece that is rail-agnostic — pplns-thunder and pplns-btc both land here and differ only in what pays out afterwards. On every confirmation pass, any block that is confirmed, 100 deep, and not yet distributed is split across the shares that produced it: walk backwards from the block's own share accumulating difficulty until the window is full, then credit each worker its proportion of (reward + fees) net of fee_bps. Three decisions worth stating, because each has a wrong answer that looks fine. Maturity is 100 confirmations, not "confirmed". A coinbase output is unspendable until then, so crediting at confirmation creates a balance the pool genuinely cannot fund — the reserve requirement PPLNS exists to remove, reintroduced by accident. Waiting also deletes the orphan question rather than answering it: crediting is additive and there is no negative share, so a credit from a block that later turns out not to be ours cannot be taken back, and the reversal path that would otherwise have to exist simply never does. The window is snapshotted onto the block row when it is found, not recomputed when it is paid. Those moments are ~100 blocks apart and the chain can retarget in between; recomputing would pay a block out across a window its own miners never worked under, and would make the same block distribute differently depending on when the pass happened to run. Stored, the split is reproducible from the row alone. Transaction fees are included. Unlike pure PPS, PPLNS shares what the block actually earned — blocks_found already had reward_sats and fee_sats as separate columns, so this is summing two numbers that were already there. Two smaller ones. The share that crosses the window boundary is counted whole rather than split: the window is a rule for choosing which work gets paid, not a claim that exactly N difficulty was performed. And a pool younger than its own window pays the full reward across whatever work exists rather than scaling down — scaling down is arithmetically tidier but leaves a remainder with nowhere honest to go, since it is the miners' block and no third party has a claim on the difference. pplns_distributed is an exactly-once latch, and one transaction per block. Crediting being additive means a partial or repeated distribution is the one failure that cannot be fixed by running again, and that leaves no trace in the amounts themselves. Tested against a window with older work deliberately sitting behind it: carol mines 1000 difficulty outside the window and is paid nothing, which is the behaviour a naive "sum all shares" query gets wrong. Verified by mutation — removing the window bound and removing the maturity gate each fail the suite. Next: the payout rails. pplns-thunder needs none, the existing worker drains pps_credits already. pplns-btc needs an L1 client mirroring ThunderClient's surface. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The last rail for #48, and much smaller than it was going to be. The plan before this was for the pool to track its own coinbase outpoints, serialise a BIP174 PSBT, and hand it to the operator to sign offline — several hundred lines of transaction construction whose bugs would be silent and expensive, in a binary that had never held a key or built a transaction. None of it needs to exist. bip300301_enforcer already ships a wallet, and its WalletService has SendTransaction: a destinations map, a fee rate, and it selects the inputs, signs and broadcasts itself. So pplns-btc is a client class of about a hundred lines and no new dependency. INSTALL.md had already been recommending an enforcer-owned pool_btc_address, which is exactly the arrangement this needs. The client mirrors ThunderClient's interface rather than inventing one — balance, transferBatchDetailed, getTransaction, walletUtxos, and mine() as a no-op because Bitcoin blocks arrive without being asked, where Thunder only advances when a mainchain block commits to it. The payout loop therefore never branches on which rail it is driving, and everything that makes a payout safe is written once and shared: the write-ahead payouts_in_flight row, one transaction per batch, and crediting paid_sats only on confirmation. Three things in it are load-bearing rather than defensive. destinations is keyed by address, and two rigs can authorize with the same payout address. Sending the list unmerged lets one entry overwrite the other, paying that miner once for two debts while the ledger marks both settled — a shortfall that balances perfectly on the pool's side and is visible only to the miner. The client sums by address first. Verified by mutation: replacing the sum with an assignment fails the suite. An unreachable enforcer reports unknown, never confirmed and never evicted. payout.js turns unknown into "block and ask a human", because "the node forgot it" and "it confirmed a while ago" look identical from here and guessing either way pays twice. The fee is a rate, not an amount. The enforcer selects the inputs, so it is the only party that knows the size of the transaction the fee applies to — there is no local estimator to drift out of date. PAYOUT_RAIL selects the rail and decides which of the two disjoint sets of environment variables is required, so a correctly configured L1 pool is not refused for lacking THUNDER_RPC_URL. The proxy logs what pplns-btc needs at startup — enforcer with --enable-wallet, pool_btc_address from that wallet, worker with PAYOUT_RAIL=btc — because otherwise the first sign of a misconfiguration is a payout failing 100 blocks after the block was found. 77 payout assertions (was 67), dashboard 135, C suites unchanged and clean under ASan/UBSan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
We run a coinbase-direct PPLNS pool on the ECX alpha network and have been paying miners
So for the The costs are real and worth stating plainly, because they are what the design has to
None of that changes the window maths, the maturity gate, or the flag split in your first If it is useful, I am happy to write it up as a PR against this branch or as a follow-up |
main moved 26 commits under this branch, including several in exactly the
files it touches: the per-connection coinbase use-after-free, the listener
teardown fixes, the vardiff floor correction, taproot payout addresses,
IPv6 listeners, and mining.suggest_difficulty. The branch had gone
CONFLICTING on GitHub, so nothing could be tested against current main.
Two conflicts, both in test files where each side appended its own tests to
the same place. Both sides kept:
test_store.c main's test_open_upgrades_a_pre_status_database alongside
the two pplns distribution tests
test_stratum.c main's failed-bind / suggest_difficulty tests alongside
the PPLNS mode block
The default diff algorithm interleaved the test_stratum.c hunks by matching
on shared boilerplate (subscribe, then handle_message) rather than on whole
function bodies, producing a conflict that spanned two unrelated tests. The
patience algorithm aligns them per-function and reduces it to one clean
either/or, which is what this resolution is.
One thing the merge broke that neither side broke alone: test_store.c calls
fresh_db_path() 17 times now, and g_db_paths held 16. Each side stayed under
the limit on its own. Overrunning it wrote past the array and the next
store_open() failed with nothing to say for itself. The array is larger and
the overrun is now an assert that names the cause.
C 416 stratum + store/coinbase/broadcast/thunder/config, payout 77,
dashboard 135. All green.
Neither of these is pplns work and neither is new here -- both reproduce on pristine main. They are in this branch because without them the branch cannot be tested on a Mac at all, and the pplns tests are the thing that needed running. Two separate problems. The suites did not compile. -D_POSIX_C_SOURCE=200809L is there to make clock_gettime and friends visible on glibc, but on Darwin the same macro works in the opposite direction: asking for a strict POSIX namespace hides everything BSD, and INADDR_LOOPBACK and MSG_DONTWAIT are BSD, not POSIX. 19 errors, all of them "use of undeclared identifier". _DARWIN_C_SOURCE puts them back, in the Darwin-only branch of the platform conditional, so it changes nothing on Linux. Once they compiled they hung, at the very first test, forever. The listener teardown breaks a thread out of accept() by calling shutdown() on the listening socket. That is a Linux behaviour -- there, shutdown() of a listening socket wakes a blocked accept(). On macOS and the BSDs it returns ENOTCONN and the accept() stays blocked, so the pthread_join right after it never returns. Every stratum test that starts a server hangs on the way out; CI is Ubuntu, so nothing ever noticed. The listener now waits in poll() with a 200ms timeout and re-tests the stop flag itself, rather than depending on the platform's shutdown() semantics to deliver the wakeup. The shutdown() call stays: where it works it still wakes the thread immediately and the timeout only bounds the worst case. A waiting connection wakes poll() straight away, so this is shutdown latency, not connection latency. 416 stratum assertions, the full make test, and make asan all pass on macOS now; the change is a no-op on Linux beyond the poll call itself.
blocks_found gained pplns_window_diff and pplns_distributed in store.c -- in the CREATE TABLE, in MIGRATIONS_SQL, and in an index -- and schema.sql was never updated to match. schema.sql is not documentation. scripts/deploy-to-server.sh seeds data/shares.db from it on every deploy that finds no database, tests/test_payout_regtest.sh builds the payout database from it, the dashboard's test fixtures are built from it, and INSTALL.md and README.md both tell operators to initialise from it. A pool deployed that way got a blocks_found without the two columns; the proxy's migrations repaired it on next start, so the damage was bounded, but anything reading that database before then -- or never opening it through store.c at all, which is exactly what the payout regtest does -- saw the old shape. The columns and the index are now in schema.sql, and the two paths agree exactly: same tables, same columns, checked rather than asserted by hand. Pinning that with a test, because this is the second time the two have drifted and neither drift was visible in a passing suite. The test builds one database from schema.sql and one from store_open(), then compares the column set of every table. Column sets rather than DDL text, so it does not fail on formatting or on constraints the two express differently, only on the thing that actually breaks: a column on one side and not the other. It refuses to skip when it cannot find schema.sql -- a parity check that quietly does nothing is how the drift got here. Verified by mutation both ways: dropping a column from schema.sql fails the test with a printed diff of the two column sets, and restoring it passes. Also covering the pplns distributor's second iteration, which nothing exercised. Every existing pplns test settles exactly one block per call, so the loop body had only ever run once. It opens a transaction, UPDATEs blocks_found, and commits while the outer SELECT over that same table is still stepping -- if committing mid-iteration were refused, or marking a row changed what the open cursor still returned, the second block would be skipped or paid twice. Two matured blocks in one pass now assert both get exactly their own window. C suites all green, payout 77, dashboard 135.
The identity strip knew two modes. pps-classic got its own description and everything else fell through to solo's: "Each block's coinbase pays the miner who found it, directly. No share credit accrues between blocks." Both halves of that are false under pplns. The coinbase pays the pool wallet, exactly as in PPS, and a matured block is split across the shares that produced it into the same pps_credits table PPS uses. A miner reading the strip on a pplns pool was told the pool owed them nothing, on the one page whose whole purpose is stating what a stratum URL cannot show. Both pplns modes now describe what actually happens -- nothing credited when a share arrives, blocks split across their window once matured 100 deep -- and name the rail the balance is finally paid over, which is the fact a miner most needs because it is also what their username has to be. poolMeta().accrues was PPS-only for the same reason. It means "does a balance build up between payouts", which is true of pps-classic and of both pplns modes and false only of solo, whose coinbase pays the finder directly. It is deliberately not "is there a rate" -- PPS prices a share when it arrives and pplns values it in hindsight out of a block actually found, so only PPS leaves rate_used on the row. rate_source and rate_sats_per_diff stay the PPS-only facts. Not touched here: the audit page's rate framing is still PPS-shaped. It does not misreport a pplns pool -- every share has rate_used = 0, so the verification counts nothing, reports ok and claims full coverage -- but "rate" and per-share re-derivation are the wrong questions to be asking of a mode that prices in hindsight. That is a bigger change than a label. Both new tests fail against the previous template and pass against this one. Dashboard 138 passing, payout 77, C suites unchanged.
The pplns e2e is the thing this branch was missing: unit tests prove
store_pplns_distribute splits a window correctly given a database somebody
hand-built to contain a matured block, and nothing proved a running pool
ever reaches that state. It does not, and finding out why turned up a bug
that has nothing to do with pplns.
reconcile_blocks was guarded by
if (t->height - 1 != s->last_height) reconcile_blocks(s, t->height - 1);
and last_height holds the TEMPLATE height, which is the tip plus one. So
the condition asks whether the new tip differs from the previous tip PLUS
ONE, which on an ordinary one-block advance is false. The pass ran only
when the tip jumped two or more blocks between polls.
That is the single most common event on any chain. Blocks sat at 'pending'
with checked_via unset, never confirmed, never counted deeper -- and pplns
distribution hangs off the end of that same pass, so a pplns pool credited
nobody, ever. Measured before the fix: 51 consecutive single-block tip
advances, 51 jobs rebuilt, zero reconcile passes.
It now keys on new_tip, which is computed a few lines above from both the
height and the previous hash. That is the correct notion, and it catches
one more case a height comparison of any kind cannot see: a reorg that
replaces the tip at the same height.
Pre-existing, from f8a11e4; nothing in this branch touched that line. It
surfaced here because pplns is the first feature whose visible output
depends on the pass running.
The test itself covers both rails, because pool_mode decides two things at
once and only one of them is the rail: pplns-thunder and pplns-btc pool the
reward identically and share every line of the distribution path, differing
only in what a stratum username IS. Each authorizes with its own username
shape, so a regression in one rail's validation cannot hide behind the
other passing.
It asserts the maturity gate as an absence before asserting distribution as
a presence. A confirmed block only 11 deep must credit nobody -- crediting
a coinbase before it is spendable creates a balance the pool cannot fund,
which is the reserve requirement pplns exists to remove. Asserting only the
end state would pass just as well against a distributor with no maturity
check at all.
Both rails, on a real enforcer template: window snapshotted at find time
(9.313e-10, twice the regtest difficulty), nothing credited at 11 deep,
distributed at 111 deep, 4950000000 sats credited of a 5000000000 gross --
reward plus fees, net of the 1% operator fee, to the exact sat -- and the
balance unmoved after five further tips, which is the exactly-once latch.
The nudge loops mine one block at a time on purpose. That is precisely the
case the old condition missed, so the test would have caught this bug by
construction rather than by luck.
Regression-checked: the pps-classic e2e and the payout e2e both still pass,
the latter also exercising the schema.sql change from the previous commit.
C suites all green.
CI runs it after the existing e2e; the job timeout goes to 35 minutes
because this one mines a chain to maturity twice over.
The second half of the same bug. reconcile_blocks settles block statuses by
one of two mechanisms -- the node's own getblockhash where the backend
serves it, and the observed chain of template tips where it does not -- and
PPLNS distribution ran at the end, after both. Except it did not:
if (atomic_load(&s->gbh_state) > 0) return; /* before everything below */
gbh_state latches to 1 on the first successful getblockhash and stays there
for the life of the process, so on any backend that serves the call, every
subsequent pass returned before reaching the distributor. A pool on an
ordinary bitcoind confirmed its blocks, counted them past 100 deep, and
credited nobody, ever.
Nothing about that looks wrong from the outside. The rows carry a window, a
status and the depth; only pps_credits stays empty. It is the same failure
as the previous commit's, reached down the other branch, and it is worse
because the previous one at least stalled confirmations too.
The early return existed for a real reason: getblockhash is the node's own
answer, so where there is one it wins outright and the weaker templates
fallback must not run and overwrite its verdict and its checked_via. That
reason applies to the fallback, not to the distribution. So it is now a
flag that skips the fallback, and the distribution sits on the function's
single exit path, reached however the statuses were settled. The transient
getblockhash failure does the same: it still leaves the rows alone, but no
longer stops paying out rows an earlier pass already settled.
The e2e could not have caught this. It runs against the enforcer, which
serves no getblockhash, so the branch was never taken -- which is exactly
how the bug survived the last commit. It now runs a third scenario with the
proxy pointed at plain bitcoind, where getblockhash is served and the
preferred path is the one under test.
Verified by mutation: against the previous code that scenario fails with
"block is 100 deep and still undistributed", confirmations=100
distributed=0; with the fix it distributes 2475000000 sats of a 2500000000
gross, exact to the sat. Both enforcer scenarios still pass, as do the
pps-classic and payout e2e suites and the C suites.
⚠️ left in the source above the distribution: it must stay on the single
exit path, and adding a return above it silently stops a pplns pool paying.
The rail had unit tests against a stubbed client and nothing else. Pointing
it at a real bip300301_enforcer wallet for the first time found three bugs,
each of which on its own made pplns-btc unable to pay anyone.
1. run-once.mjs ignored PAYOUT_RAIL.
index.js selects the client from cfg.rail; the one-shot entrypoint
constructed a ThunderClient unconditionally. On an L1 pool that is a
Thunder client with cfg.rpcUrl === null, so the tick found nobody to pay
and exited 0 -- a clean run, no payments, no error. It now makes the same
choice index.js does.
2. balance() returned the wrong shape, so every tick refused to pay.
payout.js reads `BigInt(bal.available_sats ?? bal.total_sats ?? 0)` --
ThunderClient's shape, and the reason the payout loop can drive either
rail without branching. EnforcerWalletClient returned a bare BigInt, on
which both fields are undefined, so the reserve gate saw zero and stopped
every tick with "reserve short — available=0" against a wallet holding
250 BTC. The unit test asserted `await c.balance() === 500000n`, which
passes against exactly that bug; it now asserts the way the caller reads
it. While here: confirmedSats arrives as a decimal string and went
through Number(), which rounds above 2^53 -- about 90,000 BTC. Parsed as
a BigInt.
3. Settlement credited miners on broadcast, not on confirmation.
Two independent causes, both of which had to be fixed:
getTransaction keyed on `confirmations`/`confirmationHeight`. The
enforcer reports a confirmationInfo submessage instead -- and emits it
for mempool transactions too, carrying only a timestamp ("when we saw
it"). A mined one additionally carries height and blockHash. So the field
read never matched anything, and once it was made to match, presence
alone reported every broadcast as confirmed. It now keys on
height/blockHash.
walletUtxos counted unconfirmed outputs. payout.js treats a wallet output
from the batch's txid as proof the batch settled, and the enforcer
applies a transaction to its wallet the moment it broadcasts -- so the
change output of an unmined payout appeared immediately. Confirmed
outputs only; unconfirmedLastSeen is present exactly while unmined.
Either one alone moved paid_sats and wrote the ledger row while the
transaction was still in the mempool. That is what "paid means mined, not
sent" forbids: crediting is additive and there is no negative credit, so
a dropped transaction leaves the debt marked settled and the miner never
paid.
tests/test_pplns_btc_payout_regtest.sh walks the sequence against a real
wallet-enabled enforcer, no Thunder in the stack at all: a tick broadcasts
and credits nobody; the next tick blocks on the unconfirmed txid rather
than re-broadcasting into a double spend; one L1 block later a tick
settles, paid_sats moves exactly once and the in-flight row clears; a
further tick pays nothing more.
The assertion that matters is the last one, and it reads neither the payout
worker's database nor the wallet that sent the money: 250000 sats unspent
at the miner's own address, straight out of the chain's UTXO set. Every
other check above would still pass if the ledger were being written with no
payment behind it.
Three unit tests pin the enforcer's real response shapes so the next
refactor cannot quietly reintroduce any of this.
payout 83, dashboard 138, and the Thunder payout e2e still passes.
…miners
The single-recipient run could not reach either of the two things a real
pool does on every payout: send to many addresses at once, and handle two
rigs that authorized with the same one. Extending it to three workers, two
of them sharing an address, found a fourth bug in the rail.
payout.js decides whether it may batch across addresses by LEARNING what a
previous transfer turned out to do, and deliberately treats a node it has
not yet proven as one that cannot -- for Thunder the answer genuinely
varies, and guessing wrong spends somebody else's balance. The enforcer
client returns no broadcastByNode, so it never proved anything, and the
first tick of every process paid one address and deferred the rest:
payout: 2 due ... (this node cannot batch across addresses, so paying
bcrt1qyg3z... now; 1 more address(es) follow on later ticks)
There is nothing to learn. WalletService/SendTransaction takes a
destinations map -- paying many addresses at once is the shape of the call.
And run-once.mjs is one process per tick, so under cron every tick is a
first tick: one address per run, a separate fee each, and a pool with fifty
miner addresses taking fifty daily ticks to pay everyone once.
The client now declares batchesAcrossAddresses and payout.js believes a
client that knows, while still making a node whose behaviour has to be
discovered prove it. Thunder's path is untouched: it has no such property,
so it learns exactly as before, and its e2e still passes.
The test now asserts what only a multi-recipient batch can show:
- all three due workers leave in ONE transaction (three in-flight rows,
one distinct txid) rather than one transaction and one fee each
- the transaction pays the shared address ONCE, carrying 300000 -- the
sum of both rigs' debts, which is neither operand, so no assertion can
pass by coincidence. Two outputs would mean it was written twice; the
wrong value would mean one entry overwrote the other, paying that miner
once for two debts while the ledger marked both settled
- each rig is still credited its OWN debt (180000 and 120000) even though
a single output covered both; crediting the merged amount to either
would leave the other owed forever
- the addresses really hold 250000 and 300000 in the chain's UTXO set,
read from neither the payout worker's database nor the wallet that sent
the money
Read off the wire before the block, so the output shape is checked against
the transaction itself rather than against anything either side recorded.
payout 85, dashboard 138, Thunder payout e2e still green.
The trim comment claimed "nothing but the dashboard reads this table, so a dropped row costs visibility and nothing else". That stopped being true when block reconciliation started reading it. On a backend serving no getblockhash -- which is every enforcer, and so the production configuration -- store_reconcile_blocks_from_templates() confirms a block by finding the template at height+1 whose prev_hash is that block. Trim that row and the block stops being confirmable: its confirmations freeze where they were, and under pplns a block frozen short of maturity is never distributed and its miners are never paid. No code change. The default retention is 30 days against a ~17-hour maturity, so the margin is wide -- but it is a margin, not an absence of coupling, and the comment was actively telling the next person that lowering templates_retention_days costs only dashboard history.
An operator who sets ENFORCER_WALLET_PASSPHRASE against a wallet that is not
encrypted had every payout tick fail, and nobody was paid.
A wallet that is not encrypted is already unlocked, and the enforcer says so
with HTTP 409 already_exists / "enforcer wallet already unlocked".
enforcerRpc turns any non-2xx into a throw, and ensureUnlocked() is awaited
un-wrapped at the top of transferBatchDetailed, so that reply came straight
out of the transfer. It is not a failure: the wallet can sign, which is the
only thing the call is for.
Not an exotic misconfiguration. --wallet-auto-create makes an UNENCRYPTED
wallet, and that is how INSTALL.md and the regtest scripts create one, so
this is the default wallet plus a passphrase set defensively -- or set for a
wallet that was later decrypted.
A wrong passphrase still throws. That one really does leave the wallet
unable to sign, and swallowing it would turn a typo into payouts that stop
with no reason given. Both directions are tested, and the tolerant one is
mutation-checked: making ensureUnlocked rethrow unconditionally fails it.
enforcerRpc now keeps `code` and `status` on the error it throws, so callers
can tell one failure from another without matching on prose upstream is free
to reword. ensureUnlocked still falls back to a message match as well.
What this does NOT cover, now written where the next reader will find it: the
LOCKED path. A regtest enforcer cannot be made to hold an encrypted wallet at
all --
--wallet-auto-create creates an unencrypted wallet, and CreateWallet then
refuses ("a wallet seed already exists")
wallet on, uncreated the enforcer will not start: --enable-mempool is
mandatory and its sync task refuses an uninitialized
wallet
--walletless WalletService/CreateWallet is not served at all
-- so UnlockWallet's request shape cannot be verified here. An unencrypted
wallet answers "already unlocked" before reading the body, so a probe with a
deliberately bogus field name gets the same reply as a correct one: no
regtest call can tell them apart. The field name matches CreateWallet's,
which does take `password`, so it is probably right, and the comment says
"probably" rather than letting it look covered. If it is wrong, the failure
lands at the unlock rather than as a mispayment.
payout 87, dashboard 138, both payout e2e suites still pass.
`pplns` appeared in exactly one file in the repo — payout/README.md — while
README.md, docs/simplepool.html and INSTALL.md all still told the reader
simplepool has two modes. INSTALL.md is the one that mattered most: it is
where an operator chooses pool_mode, so a pplns pool could not be set up by
following the documentation at all.
The other five docs that mention pps-classic were left alone deliberately.
None of them enumerates the modes; their mentions are mode-specific
statements that are still true.
README.md — a four-row table of the two things pool_mode actually decides
(whether the coinbase pays the miner or the pool, and what a username is),
then the pplns entry: nothing credited on arrival, a block split across its
window at 100 confirmations, fees included, window as a multiple of network
difficulty snapshotted at find time. The framing throughout is who carries
the variance, because that is the whole reason the mode exists — PPS needs a
reserve measured in block rewards and ruins an operator who cannot fund it;
pplns never owes more than it has just been paid.
docs/simplepool.html — two more mode cards, a third palette colour for them,
the comparison table extended to four columns with the rows that actually
separate the modes (when a balance moves, whether work that found nothing is
paid, whether fees are shared, whether a reserve is needed), and a note on
why a fourth mode exists at all.
INSTALL.md — the mode list, a PPLNS config section, and the three things
pplns-btc needs that no other mode does: an enforcer with --enable-wallet,
pool_btc_address from that wallet, PAYOUT_RAIL=btc on the worker. Part F is
no longer "PPS modes only" and now carries the rail table; Part C says
Thunder is not needed for pplns-btc.
Two corrections found while writing:
- INSTALL.md's pps-classic example set `pps_sats_per_diff = 1000`, which
README.md tells you to leave unset and for good reason: a pinned rate
silently bypasses fee_bps and cannot follow a retarget. It is an escape
hatch, not a field to fill in. Removed, with the reason stated.
- the removed drivechain-coinbase mode was described as "a third mode" in
two places. It is now the fifth.
Every config error quoted in the troubleshooting section was checked against
the binary rather than transcribed from the source, and matches verbatim.
The worker page answers "why is this number what it is?" by re-deriving the
balance from shares.credited_sats — right for PPS, which prices a share the
moment it arrives and stores that price on the row.
PPLNS prices a share in hindsight, out of a block actually found, so every
share carries credited_sats = 0. The sum came to zero against a real
balance, and every miner on a pplns pool was shown
⚠ Off by 4,950,000,000 sats (49.50000000 BTC)
... Ask the operator to confirm the rate history.
on the one page whose entire purpose is being checkable. Not a display nit:
a false accusation that the pool is short by the miner's whole balance, with
an instruction to go and challenge the operator about it.
Silencing the warning would have been the wrong fix — it would leave the
mode with no audit at all, on a project whose stated goal is that a miner
can verify what they are owed without trusting the pool. So pplns gets the
derivation that actually produced the number: for each matured, distributed
block, the window it was split across and this worker's proportion of it,
recomputed from the raw shares and blocks_found rows rather than read back
from anything the payout path wrote. The window SQL mirrors
store_pplns_distribute() exactly, including the comparison against the
running total EXCLUDING the current row, which is what takes the share
crossing the boundary whole instead of splitting it.
The page now shows one row per block — reward plus fees, the amount left
after the operator fee, this worker's difficulty against the window's, the
resulting percentage and the sats — and compares the sum against the stored
balance. Where PPS says "rate", pplns says why there isn't one: nothing is
priced in advance, so the pool can never owe more than a block it has
actually been paid for.
One assumption, stated in the UI rather than hidden: fee_bps is read as it
stands now, because nothing records it per block. An operator who has
changed the fee will see older blocks fail to reproduce, and the page says
that is the likely cause rather than crying discrepancy. The same note
covers the case where the block list is truncated.
Verified by mutation, both halves. Removing the window bound fails "work
done before the window is worth nothing" — carol mines 1000 difficulty
outside a 50-difficulty window and must be paid nothing, which is exactly
what a naive sum-every-share query gets wrong. Reverting the page to the
per-share comparison fails three others, including the one asserting the
miner is not told to challenge the operator.
The PPS path is untouched and still asserted: a pps-classic pool gets the
same per-share derivation it always had.
dashboard 144 passing, payout 87.
LLVM source-based coverage over the C unit suites: builds each one instrumented, runs it, merges the profiles and prints a per-file report. Two exclusions, both so the number means something. Vendored cJSON is upstream code and would move the headline without saying anything about this project's tests. System and Homebrew headers are excluded for a sharper reason: hiredis and curl expand a great deal of inline code into the translation units that include them, and counting it put a third of the "missed regions" in files this project barely touches — bitcoind.c reads 8% branch coverage almost entirely because of curl's typecheck macros, while its line coverage is 63%. The comment in the target says what the number does NOT mean, because that is the part a coverage figure gets wrong on its own: it measures `make test` only. The three regtest e2e suites drive the real binary and cover a great deal that never appears here — the tip watcher, the confirmation pass, the distributor, both payout rails — so a low figure for main.c means "no unit suite links it", not "untested". main.c is in fact where the two worst bugs of this branch lived, and both were caught by an e2e rather than by a unit test, which is the same point from the other direction.
The two most expensive bugs on this branch lived in reconcile_blocks, and
neither could be caught by a unit test: it was inside main.c, which has
main(), so nothing in it can be linked into a test binary. Both were found
by a three-minute regtest run because there was no faster way to find them.
reconcile.c is that code lifted out unchanged, with the one call that needs
a network — the block hash lookup — behind a function pointer. Everything
else it touches is the store, which a test can hand it directly. main.c
keeps a six-line adapter and the same behaviour.
tests/test_reconcile.c now asserts, in under a second, what previously took
a regtest stack:
- distribution runs on the getblockhash path. THE regression: it used to
sit behind an early return taken whenever the backend served that call,
and since the state latches for the life of the process, a pool on an
ordinary bitcoind confirmed its blocks, counted them past maturity and
never distributed one. Only pps_credits stayed empty.
- distribution runs on the templates path too — the one every enforcer
takes, and the only one the regtest e2e can reach, which is exactly why
testing against it alone left the other branch broken.
- it survives the latch, which is the real shape of the bug: the FIRST
pass behaved correctly and every pass after it did not.
- a transient lookup failure leaves statuses alone but still pays out rows
an earlier pass settled, so a backend failing this one call cannot
silently stop crediting anyone.
- non-pplns modes never distribute; a reorged candidate is orphaned and
not paid; the operator fee comes off the top.
Verified by mutation: restoring the original early return fails 7 checks
across 3 tests, naming them.
Two things the writing turned up, both in the tests rather than the code:
- the first fixtures seeded only a MATURED block and asserted the node
path had run. It had not: a block past BLOCK_FINAL_DEPTH is deliberately
no longer a candidate, so the lookup was never called and the pass
quietly took the templates branch instead. A test meaning to exercise
one branch was exercising the other. The fixtures now seed a shallow
candidate alongside, and the helper says why.
- each test printed its own "ok" at the end, so a test whose CHECKs had
failed still announced success — the count at the bottom disagreed, but
the line a reader scans said ok. The runner decides now.
reconcile.c reports 94.59% line coverage, on code no unit suite could reach
at all yesterday. make coverage builds and includes it.
Four tests covered the comment and quoting rules. Nothing covered the
validation, the listener lines, or the log level — so `make coverage` read
33% line and 67% of functions never ran at all.
The validation matters most, because until now the only thing checking those
messages was a person running the binary by hand. Every error string
asserted here is also quoted in INSTALL.md's troubleshooting section, so a
reworded message that leaves the docs behind fails here first:
- bare `pplns` is refused ahead of the generic catch-all, and the message
names both real values
- an unknown mode names all four
- both pplns rails require pool_btc_address
- each mode sets the two flags pool_mode used to conflate, pplns-btc being
the combination no single "is this PPS" flag could express
- a non-positive window is refused; a window below 1.0 warns and loads,
because paying a block across less work than it took to find is a choice
an operator is allowed to make badly
- the window defaults to 2.0
Listener lines had no coverage at all, and they are how rented hashrate is
served its own difficulty. A marketplace measures what the port advertises
and cancels an order that comes in under it, so a line that parses wrongly
is a delisting rather than a cosmetic problem. Covered: a line becomes a
port policy, several keep their own, a field that is not key=value is
refused by name, and a listener without a port is refused.
One test was wrong and the code was right. An unparseable log_level warns
and keeps the default instead of refusing, which I had asserted as an error.
It is the better behaviour — a typo in a cosmetic setting should not stop a
pool accepting work — so the test now pins that, with the reason, rather
than the code being changed to match a test.
config.c: 33.14% -> 74.49% line, 66.67% -> 100% of functions. Whole-project
line coverage 75.6% -> 79.5%.
strcasecmp is in <strings.h> on glibc and in <string.h> on macOS, so the extracted file compiled clean locally and broke the Linux build under -Werror=implicit-function-declaration. main.c, which this code came out of, includes both — the include went missing in the move rather than being wrong. The mirror image of the Darwin feature-test problem earlier in this branch, and the same lesson: this project is built on two platforms and only one of them is CI.
Contributor
|
Congratulations on getting this in. I commented on the draft a few weeks back offering a third rail, and I want to restate it now
We have been running that in production on the ECX alpha network since 19 August. Some numbers
One thing that should make this cheap: our PPLNS window is already sized as a multiple of network What it costs, stated up frontI would rather you hear the drawbacks from me than find them in review.
None of that is a reason to prefer it. The reason to prefer it is that the pool never holds miner The offerNo pull request unless you want one — this is your design and I am not going to push code at it If it is not interesting, that is a completely fine answer and I will stop raising it. Either way |
Collaborator
Author
|
Hi @Wired4ncer, I'm working at #76 right now, maybe you can take a look and check if it matches what you have in mind. Thank you for your message. |
rsantacroce
added a commit
that referenced
this pull request
Sep 8, 2026
Wired4ncer, who has run a coinbase-direct PPLNS pool on ECX alpha since 2026-08-19, gave numbers on #61 that show the cap I built measures the wrong thing: - up to 16 miners paid per block - whole coinbases of 721-817 bytes - and the binding term is NOT the payouts. The same 16 payouts cost 817 bytes against four drivechain OP_RETURNs and 769 against three. A cap counted in outputs cannot express that, because the commitments are not outputs it counts. Worse, the number was badly wrong in the same direction: 200 outputs at ~31 bytes each is over 6000 bytes of payouts alone, roughly eight times what a marketplace is observed to accept. A pool trusting that default would have had jobs refused. So the limit is now the whole serialized coinbase in bytes. Everything that is not a payout is charged first — the transaction envelope, the scriptSig, the operator output, and the commitment OP_RETURNs the template already carries — and the payouts get what is left. The commitments are simply part of what has been spent, which is exactly the relationship the count could not see. A test pins it as a relationship rather than against someone else's absolute numbers, which would be asserting on another implementation: the same window and the same budget, built from an enforcer template versus from scratch, pays strictly fewer miners from the template. Measured at a 300-byte budget: 5 against 6. A second test builds across a range of budgets and asserts the bytes on the wire never exceed the number, because a budget is only worth having if it is true of the transaction and not just of the accounting. Two details the byte accounting forced, both of which a per-output figure would have got wrong: Cost is computed per address, after resolving it. A P2TR payout is 43 bytes against a P2WPKH one's 31, so budgeting at a flat rate lets more through than fits. A payee that does not fit does not stop the loop. A later, cheaper address may still fit, and stopping early would carry money that could have been paid. coinbase_max_bytes is configurable, as he asked, because the number belongs to whichever marketplace an operator sells to rather than to us. Default 1000, documented in proxy.conf.example with where the figure comes from. A value too small to hold a coinbase and one payout is refused: that is a pool that cannot run, not one that runs badly. The remaining item from his scope is the carry-forward ledger. carry_sats is computed and reported but still nothing consumes it — today the dust a miner is owed rides on the operator output and is not recorded as owed to anyone. He reports 62 of 100 addresses sitting below the floor, so it is not a corner.
rsantacroce
added a commit
that referenced
this pull request
Sep 8, 2026
Until now carry_sats was computed, reported, and then forgotten. A claim too small for a coinbase output rode on the operator output — so the operator was holding it — and nothing anywhere said whose it was. That is not a small custodial balance, it is money kept quietly. Wired4ncer names this as the mode's first cost on #61 and reports 62 of 100 addresses currently sitting below the floor, so it is the common case rather than an edge. The ledger is now written per worker, into the same pps_credits table the other rails use, so a carried claim shows up wherever a miner's balance already shows up. Three decisions in that: Only the UNPAID part is recorded. A claim the coinbase paid is settled on chain and has no business in a ledger of what is owed; a partly paid claim carries only its remainder. Recording the full claim would have the pool owing money it had already paid, and a zero row would put every settled miner into a ledger of debts the pool does not have. It is written when a block is FOUND, which is the only moment the information exists. The coinbase is decided when the template is built, but almost no template becomes a block, so nothing can be written earlier without recording debts for blocks that were never mined. The outcome is recomputed at that moment rather than remembered. The split is deterministic from the job and the config, so running the same builder again reproduces exactly what was rendered, and a found block is rare enough that the cost is irrelevant. The alternative — carrying the result along the submit path — would mean the ledger and the coinbase could disagree. Worker ids now ride on the job beside the payees, because an address is not enough: two rigs can share one and the ledger is per worker. Four store tests, mutation-verified: recording the full claim instead of the remainder fails them. Two things worth stating plainly. The e2e does NOT cover carry with anything in it. Carry needs a window where some claims fit the coinbase and others do not, which needs several miners of very different sizes, and the harness drives one cpuminer. Squeezing the byte budget instead does not produce it either: when nothing fits, the builder refuses, no coinbase is rendered, and no block is found. I wrote that stage, watched it print "carried: 0 sats" and pass regardless, and removed it — a stage that cannot fail is worse than an absent one. The gap is now stated in the file rather than papered over. And carry is recorded but not yet CLEARED: a carried claim does not currently raise the miner's share of a later block. Clearing it has to come out of the operator's fee, since the money is already on chain in the operator's output, and that is a design question rather than an oversight — it is the next piece.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — not for merge. Targeting a release ~3 weeks out. Closes #48 when it lands.
Rebuilt onto current
main(mergeba7cda5), so it now carries the stratum lifetime fixes, the vardiff floor correction, taproot payouts, IPv6 andmining.suggest_difficulty.Why
solopays each miner from their own coinbase and pools nothing.pps-classicpools properly but moves all variance onto the operator, who needs a reserve measured in block rewards to absorb it. There was no pooled mode where the miners carry the variance — which is the mode most small pools actually want, and the only alternative to a PPS pool that cannot fund its reserve (see #47).Under PPLNS the pool never owes more than it has just been paid. There is no reserve to size and operator ruin is not a failure mode.
What's here
071bf7fpool_modewas overloading; add the modes and the window knob02e19338fc6ce1ba7cda5mainad2256bdc62e38schema.sqlandstore.cdescribing the same database27f8b948f95232The flag split
pps_enabledwas doing two unrelated jobs — whether the coinbase pays the pool or the miner, and whether a username is a Thunder or Bitcoin address. They moved together across the only two modes that existed:solopps-classicpplns-thunderpplns-btcpplns-btcis the combination that breaks a single flag. A third came out of the same split: the accrual gate now keys onpps_accruesrather than the gate pointer —main.cinstalls that pointer for every mode, so keying on it would have suspended solo and both PPLNS modes, refusing miners from modes that never accrued anything.test_solo_is_never_gatedcaught it.Distribution
On each confirmation pass: confirmed, 100 deep, not yet distributed → walk back from the block's own share accumulating difficulty until the window fills → credit each worker its proportion of
(reward + fees)net offee_bps.The L1 rail
An earlier design had the pool tracking its own coinbase outpoints and serialising a BIP174 PSBT for offline signing. It doesn't need to exist:
bip300301_enforceralready ships a wallet, andWalletService/SendTransactiontakes a destinations map and a fee rate and does the selection, signing and broadcasting itself.The client mirrors
ThunderClient's interface, so the payout loop never branches on which rail it drives — the write-ahead row, one transaction per batch, and credit-on-confirmation are written once and shared.One bug this turned up that has nothing to do with pplns
reconcile_blockswas gated onand
last_heightholds the template height, which is the tip plus one. So the condition asked whether the new tip differed from the previous tip plus one, which on an ordinary one-block advance is false. The pass ran only when the tip jumped two or more blocks between polls.Blocks therefore sat at
pendingwithchecked_viaunset, never confirmed and never counted deeper — and PPLNS distribution hangs off the end of that same pass, so a PPLNS pool credited nobody, ever. Measured before the fix: 51 consecutive single-block tip advances, 51 jobs rebuilt, zero reconcile passes.It now keys on
new_tip, computed a few lines above from both the height and the previous hash — which also catches a reorg replacing the tip at the same height, something no height comparison can see. Pre-existing, fromf8a11e4; nothing in this branch touched that line. It surfaced here because PPLNS is the first feature whose visible output depends on the pass running.Design decisions already settled
pool_moderather than a mode plus a rail knob, so the inconsistent configuration is unrepresentable rather than merely rejectedpplns-btcrequires the enforcer running with--enable-wallet; the proxy says so at startupVerification
tests/test_pplns_regtest.shdrives both rails against a real enforcer template, and CI runs it after the pps-classic e2e. Per rail: window snapshotted at find time (9.313e-10, twice the regtest difficulty), nothing credited at 11 deep, distributed at 111 deep,4950000000sats credited of a5000000000gross — reward plus fees net of the 1% operator fee, to the exact sat — and the balance unmoved after five further tips, which is the exactly-once latch.The maturity gate is asserted as an absence before distribution is asserted as a presence: asserting only the end state would pass just as well against a distributor with no maturity check at all. The nudge loops mine one block at a time on purpose — precisely the case the old reconcile condition missed, so the test catches that bug by construction.
Unit: C 416 stratum + share 166 + bitcoind 63 + store/coinbase/broadcast/thunder/config, clean under ASan/UBSan; payout 77; dashboard 138. Both other e2e suites (pps-classic, payout) still pass.
Still to do before this is mergeable
pplns-thunderexercised end to endregtest e2e for both rails, against a real enforcerdocs—README.md,docs/simplepool.htmlandINSTALL.mdnow cover all four modesthe audit page— it turned out to actively misreport: with every share atcredited_sats = 0the per-share re-derivation summed to zero against a real balance, so every miner was shown "⚠ Off by <their entire balance>" and told to "ask the operator to confirm the rate history". PPLNS now gets the derivation that produced the number — per matured block, the window it was split across and this worker's proportion of it, recomputed from rawshares+blocks_foundUnlockWallet's locked branch is unverifiable in regtest (an encrypted enforcer wallet cannot be created there) — documented as such at the call site rather than assumed working