Skip to content

02: Update RR correction and rhythm screening without replaying settled beats - #104

Draft
mikenrafter wants to merge 22 commits into
OpenStrap:mainfrom
mikenrafter:split/area-2-rr-rhythm
Draft

mikenrafter wants to merge 22 commits into
OpenStrap:mainfrom
mikenrafter:split/area-2-rr-rhythm

Conversation

@mikenrafter

@mikenrafter mikenrafter commented Nov 3, 2026 •

Copy link
Copy Markdown

Review updates

Inherited the review fixes from #102 and #103, including d059e6e. Added explicit cap-gap stream/restart coverage and rejected cap-only labels in uncapped checkpoints. Frozen pre-cap oracle comparisons explicitly use uncapped APIs; capped-label assertions were strengthened. Dart 3.11.4 analysis clean; full initial fix pass 1,564 passed, 6 existing capture skips. All 60 current output-change cases pass after the follow-ups. This cumulative draft will be rebased and marked ready after its predecessors merge.

Depends on 01 / #103.

Depends on 00 / #102.

This is draft 02 of the split of #99. It targets main, so its diff is cumulative and includes the preceding branches. Review this area in its final commit; merge the preceding PRs first, then rebase this branch before merging.

Correcting RR and screening rhythm again at every sync repeats work on beats whose result has already settled. This adds an ordered corrector and rhythm-screen state that preserve settled evidence and evaluate the unfinished tail separately.

What changes

  • Add production RrCorrector, settled/tail snapshots and versioned checkpoints.
  • Add gap-aware IrregularScreenState, shared window rules and structured diagnostics.
  • Add clipped sliding medians and the correction/rhythm oracle and property tests.
  • Keep the separate research corrector and its benchmark driver beside its own tests and tools.

Lipponen and Tarvainen 2019 and Peltola 2012 support RR classification/correction. The artifact-run cap and irregular-rhythm classifier remain explicitly internal ESTIMATE rules, not published diagnostic estimators. Gap-spanning pairs are newly excluded from the streaming screen to agree with the parent's gap-aware batch path. Stream SD1/SD2, pNN and votes can therefore change. Older screen checkpoints without previous-beat timestamps are refused. Consumers must account for this output change in Edge's kAlgoVersion when adopting it. Missing evidence stays null.

Algebraic breakdown

Structure State and identity Combine; lift; finish Laws and equality
Production RrCorrector, _CappedState Ordered raw context, class/normal-anchor context, settled counters, active segment/cap state; a fresh corrector No arbitrary summary combine. Lift fold(rr, tsMs) emits immutable RrSettled; snapshot() corrects the provisional suffix to produce RrSnapshot. Ordered transducer with checkpoint, not a monoid. Uncapped artifact runs can need unbounded context. RR oracle, artifact-cap and corrector tests check batch NN, source times/classes/counts, awkward seams, bounded capped state and restarts. rr_irregular_laws_test.dart checks chunk invariance, empty fold, snapshot purity, malformed checkpoints, refusal atomicity and conservation. NN/classes/counts exact; floating values use that file's NaN-aware comparison.
RrSettled, RrSnapshot Emitted sequences and accounting fields; empty output Sequence concatenation is ordered and associative for emitted settled data; snapshot tail must be replaced, not appended twice. Lift from corrector state; finish settled plus current tail. The snapshot object itself has no merge API. Same tests check normal + corrected + dropped == raw and NN == raw - dropped, including provisional contributions.
Research tool/incremental/rr_stream.dart corrector _Sorted threshold windows, raw tail, class and timing state; fresh object Ordered fold and provisional snapshot; no combine. JSON restores the prefix state. Research oracle tests arrive with the HRV/respiration area that also consumes this prototype. Production correctness is checked independently here. No generic associative merge claim.
IrregularScreenState Level/difference centered moments, threshold counters, previous beat/time, closed window votes, active window; zeros/no previous beat No combine. Lift corrected NN/timestamps with adjacency checks; finish evaluate or diagnostics including a temporary open-window vote. Ordered fold, not a commutative monoid. Stream, diagnostics and RR/rhythm property tests check chunk/restart invariance, exact count/vote/absence evidence and nonmutating evaluation. SD1/SD2/ratio parity in the property file uses 1e-9*max(1,abs(expected)), confidence 1e-12*max(1,abs(expected)), with the explicit near-zero SD2 qualification; decoder and wire metadata are exact.
irregularWindowConfigOk, irregularWindowVerdict Complete window and adjacency mask; empty window has no evidence Batch finish rule with no state combine. Lift eligible successive differences; finish SD1/SD2 and pNN threshold vote, null for thin evidence. Config refusal, independent window partition checks, time holes and minimum successive-pair gates in diagnostics/screen laws. Threshold decisions and integer counts are exact on tested inputs.
Diagnostics structures: RrCleaningCounts, IrregularThresholds, IrregularWindowCounts, IrregularDiagnostics, IrregularScreenResult, abstention/open-window enums Evidence records with nullable unavailable fields; no synthetic zero for an absent denominator Serialization and projection, no merge API. Lift caller cleaning counts plus screen evidence; finish versioned diagnostics JSON. Not monoids. Diagnostics and empty-input tests check exact wire round trips, invalid reason/version/map refusal and absent-versus-zero rules.
slidingMedians Sorted active clipped window and two indices; empty input gives empty output Ordered insertion/removal transition, no chunk combine. Lift incoming/outgoing values; finish percentile-50 at each center. Ordered window fold, not a monoid. sliding_median_test.dart compares to independently clipped/sorted windows, including duplicates/NaN/signed zero. Same type-7 interpolation; no relaxed approximation.
Benchmark/synthetic/driver utilities Deterministic seeded inputs and emitted steps Generate or drive folds and measure work; no scientific state-combine contract. Tests cover production and prototype streams separately. Benchmarks are measurement programs, not law tests.

A stream's chunk invariance says repeated append transitions equal one append over the concatenated input. It does not establish a combine operation. Associativity and commutativity are therefore not claimed for corrector or screen states.

Where the performance gains are

Correction reuses settled work and recalculates its unresolved tail. With the configured cap and fixed correction window, append arithmetic depends on new beats plus context; uncapped long runs remain an exception. The screen updates moments and counters for new NN, then evaluates its current window, instead of rescanning all earlier windows. Sliding medians avoid sorting and allocating a new window at every position; insertion/removal still shifts a list and costs O(window length).

No benchmark was rerun. Historical JIT results retained in docs/COMBINED_BUMP_STATUS.md, arriving in area 6, cite tool/bench_night_nn.dart, also arriving there: seed 1, synthetic 24-hour wake/sleep/wake layout, 32 night prefixes at 15-minute intervals. Streaming RR took 61.78 ms total versus 547.87 ms repeated batch. One cold whole-day fold plus snapshot took 210.42 ms versus 174.75 ms batch. These numbers show the repeated-pass benefit and the one-shot cost; they predate later integration changes and are not current split measurements. This branch's tool/incremental/bench_rr.dart measures a synthetic real-shaped 23-hour day; no timing from that input is claimed here. Screen and sliding-median gains are not separately measured.

Tests

Dart 3.11.4, resolved matcher 0.12.20. dart analyze: no issues. Full timeout 3000 nice -n 10 dart test -j 2 --reporter json: 1,548 passed, 6 skipped, 0 failed. Counts include the inherited suite. Elapsed test time 68.697 seconds.

Logs: areas-logs/area-2.analyze.log, areas-logs/area-2.test.jsonl, exit status files and start/finish timestamps. Skips are optional external fixtures/corpora. No assertion was removed and no tolerance was widened. Dart 3.5.0 itself was not run; final test blobs retain the existing NaN-aware matcher fix.

Suggested review order

One commit: 0725b6932e4e RR correction and rhythm screening replay settled beats on each sync. Review the state/empty/combine or append transitions first, the finish/publication rules next, then the area reference/law/checkpoint tests and tools.

Change size and stacking

Logic under lib/: +1,836/-110, 1,946 changed lines. Tests, tools and docs: +7,380/-0, 7,380 changed lines. Other root documentation: +0/-0.

Stacks on split/area-1-framework at b5903fd9a9ee. This is branch 2 of 6, one compiled and fully tested commit per branch. Apply the stack in numerical order; the final branch must match origin/split/aggregators exactly.

Questions for the reviewer

  • Are the cap's explicit ESTIMATE status and the uncapped-context limitation sufficient for the intended caller?
  • Is rejecting the old screen checkpoint preferable to rebuilding its missing timestamp context from raw data at the application boundary?

Lipponen & Tarvainen (2019), Peltola (2012). The 32-beat cap is an internal ESTIMATE rule, not a published threshold. Include synthetic input recipes and old/new measurements.
Lomb (1976), Scargle (1982), Press & Rybicki (1989). Shift the spectral origin to the first sample before forming trigonometric arguments; pin the unrounded power on synthetic epoch seconds.
Preserve Task Force (1996) and Welch (1967) formulas for finite inputs. Timestamp validity is an implementation guard, preventing endless segment walks and invalid integer conversions.
Nelson et al. (1979), Methods for cosinor-rhythmometry; Welford (1962) centered co-moments. A relative determinant gate replaces the absolute pivot test; preserve upstream significance policy.
Morton, Fitz-Clarke & Banister (1990); Neumaier (1974) compensated summation. Preserve weighting and anchors; pin changed last digits.
Banister et al. (1975); Morton et al. (1990); Neumaier (1974). Preserve EWMA formulas and bounded-prefix priming while matching the arithmetic of the later incremental path.
Karvonen (1957) supports reserve zones only. Duration credit is an internal ESTIMATE rule with low confidence; preserve cadence validation, median capping and tail credit.
Witting (1990); van Someren (1999); Welford (1962). Preserve batch profile sums and first-winner ties; centered variance changes last digits in IS and IV.
Task Force (1996); Welford (1962). Port centered NN, lag and bin moments into batch code and retain shared spectral jitter policy. Rejection and breathing-line exemptions remain internal ESTIMATE rules with low confidence.
Record synthetic old/new observations and freeze twelve batch-only cases before aggregator adoption. Pin unchanged batch timestamp-gap exclusion and circular near-tie selection as controls. Methods and internal-policy limits are documented with the results.
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@mikenrafter

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant