Repository navigation
[Security] An applied LeaseMajority win is never re-checked, so a winner that stalls past the TTL afterwards survives beside the side that took the lease #1660
Description
Component
src/cluster/downing/LeaseMajority.ts, src/cluster/Cluster.ts (evaluateDowning / applyDowningDecision)
Severity (assessment)
High. CWE-667 (improper locking) — both partition sides survive, the same end state as #937.
Problem
LeaseMajority consults the lease only while it is deciding. #1658 makes it re-validate a cached win before returning it again, but once a won decision has been applied, nothing ties the winner's survival to still holding the lease:
Cluster.applyDowningDecision(Cluster.ts:2696) downs and tombstones the other side.- The winner's view then has no unreachable members, so
LeaseMajority.decide()resets (LeaseMajority.ts:184) and never looks at the lease again for this partition.
Exploit walkthrough
No attacker needed — a GC pause or a starved container is enough.
- 2/2 split. Side A wins the lease;
decide()returns "down B" and Cluster applies it. B's members are tombstoned in A's view. - A stalls for longer than the lease TTL.
- B's side arbitrates after A's lease lapsed — its stability window opened later, or its first acquire timed out and was retried — and wins the lapsed lease. B downs A's side.
- Each side has downed the other, and both keep running as separate clusters.
#1658's re-validation cannot help: the decision was applied before the stall, so there is no cached decision left to re-validate.
Affected files
src/cluster/downing/LeaseMajority.tssrc/cluster/Cluster.ts
Fix sketch
Survival after a lease-arbitrated split has to stay conditional on keeping the lease, at least until the losing side is known to be gone:
- After applying a lease-won decision, keep watching the lease (
onLost, orcheckAlive()turningfalse). If it is lost while the partition's aftermath is still open — e.g. withindown-removal-margin, or until the downed members are confirmed removed — the winning side downs itself. Failing closed stops the cluster rather than forking it. - This needs a hook from the downing provider into
Cluster(lease loss → self-down), not aLeaseMajority-only change.
Acceptance criteria
- Deterministic test (two real
InMemoryLeases on aManualScheduler, like the [Security]Lease.checkAlive()returns a cached boolean instead of comparing againstexpiresAt, and has no callers, so two nodes can both believe they hold the lease after an event-loop stall #937 split test inDowningStrategies.test.ts): A wins and applies its decision, then stalls past the TTL; B wins the lapsed lease. A must not keep running as a separate cluster. - The multi-node
LeaseMajoritytest still passes. - Docs (
cluster/downing-strategies.mdx, EN + DE) describe the post-decision guarantee.
Context
Found during the independent verification and the PR review of #1658 (fix for #937), and listed there as deliberately out of scope. Related: #937, #142, #839.
Disclosure status
Already public (found in an open PR's review).
Activity
pathosDev commentedon Oct 7, 2026
Verified against the code (
Cluster.evaluateDowning/applyDowningDecisionandLeaseMajority.decide()on the #1658 head). The mechanism is real, with two corrections to how the issue frames it.What holds.
applyDowningDecisiontombstones the other side.decide()resets and nothing consults the lease for that partition again.onLostthat renewal fires (since fix(coordination): make Lease.checkAlive() honour the TTL, re-validate LeaseMajority wins (#937) #1658) has no listener on the downing side.Narrower than "high" suggests. In the ordinary case the loser is safe: a lost arbitration returns
downSelfSet, so B downs itself. The failure needs B's first effective arbitration to land after A's stall outlasted the TTL. That means B's stability window opened later than A's, or B's first acquire was abandoned at its deadline and retried. Both are possible, but the window is the gap between A applying its decision and B arbitrating, not the whole lifetime of a partition as in #937. I'd rate this severity: medium rather than high, and I'm relabelling it. It staysproduction-goal.The fix sketch names a knob that does not exist. There is no
down-removal-margininsrc/orreference.conf. Bounding the post-decision watch would need a new key, with itsConfigKeysentry, a reader, an options field, docs (EN + DE) and thereference-conf.mdxmirror. An alternative that needs no new key is to bound the watch by the tombstone lifetime of the downed members, or by "until the lease is released". Either way, the self-down hook from the downing provider intoClusterthat the issue describes is the right seam.Everything else (affected files, acceptance criteria, relation to #937/#142/#839) is accurate. This becomes actionable once #1658 lands.