Skip to content

05: Update circadian and training metrics without replaying history - #107

Draft
mikenrafter wants to merge 16 commits into
OpenStrap:mainfrom
mikenrafter:split/area-5-circadian-load
Draft

mikenrafter wants to merge 16 commits into
OpenStrap:mainfrom
mikenrafter:split/area-5-circadian-load

Conversation

@mikenrafter

Copy link
Copy Markdown
Contributor

Depends on 04 / #106.

Depends on 03 / #105.

Depends on 02 / #104.

Depends on 01 / #103.

This is part 05 of the split of #99. It targets main, so its diff also contains the earlier parts until they merge. Its own changes are the commits listed under "Suggested review order". After each earlier part merges, I'll merge main into this branch so the diff shrinks to this part alone.

Circadian and training calculations replay long histories even when their summaries can be updated from new slices. This composes the existing batch arithmetic from per-period statistics, boundary windows, suffixes and affine recurrences.

What changes

  • Add cosinor, nonparametric circadian and Phillips SRI accumulators and convert their batch wrappers.
  • Add Banister TRIMP, CTL/ATL/TSB, time-in-zone and positional lnRMSSD readiness aggregators.
  • Retain date-aware readiness and malformed-mask SRI batch fallbacks.
  • Add each algorithm's reference, conversion, law and checkpoint tests with the area.

The algorithms retain their source citations: Halberg and Nelson 1979 cosinor rhythmometry, Phillips et al. 2017 SRI, Witting 1990/van Someren 1999 nonparametric IS/IV/RA, Banister 1991/Morton et al. 1990 training load and Plews et al. 2013 lnRMSSD readiness. The precise method references remain beside the arithmetic and in ALGORITHMS.md in the final area. Internal cadence/duration-credit policies remain ESTIMATE. This branch reuses the parent's published-output policy; changing sum order can still alter floating rounding, near-singular classification and near-tied L5/M10 starts. SRI and positional readiness comparisons are exact; calendar filtering remains in the batch wrapper. Adoption retains the parent's Edge kAlgoVersion requirement.

Algebraic breakdown

All aggregator instances require fixed period/bin origin, HR anchors, zone set, session/history parameters and other configuration. The common test set covers identities, association, splits, singleton/sliced lifts, pure finish and supported JSON continuation through finished observations. Approximate numeric monoids are mathematical summaries whose floating implementations satisfy fixture tolerances, not exact associative arithmetic on doubles.

Algorithm/structure State and empty Combine; lift; finish Laws and tested equality
TimedValues, _CosinorState, _CosinorStats, cosinorAggregator Count, centered x/y means/co-moments for cosine/sine basis and response; zero count/zeros Merge centered regression statistics; lift finite time/value rows at fixed period; finish 3-coefficient fit and mesor/amplitude/phase/R2/p-value gates. Approximate commutative summary for the accepted rows. cosinor_aggregator_test.dart checks conversion, common laws, ill conditioning, offsets and restarts. Mesor/amplitude relative 1e-9, scale-sensitive absolute floors 1e-12*max(1,abs(mesor)); circular phase 1e-9 with documented floor, hours relative to period 1e-9; R2, adjusted R2, p/confidence absolute 1e-7. Vanishing amplitude omits meaningless phase comparisons. Discrete metadata exact on fixtures. Generic harness does not separately test commutativity.
EpochSeries, _CircadianStats, _CircadianStatsAggregator Count, shifted/origin-aware global sums and squared statistics; zero count/zeros Merge global statistics; lift activity epochs; finish global variance components. Approximate commutative core. Part of circadian conversion/law/checkpoint tests, with the full result tolerances below.
_SquaredDiffs, _RawSum, circadianNpAggregator, circadian window helper Product of count/global stats, per-clock-epoch counts/sums and Lookback(2) successive squared differences; child identities Merge by epoch-of-day and repair adjacent seam; lift activity slices; finish IS, IV, mean daily profile, circular L5/M10, RA and starts. Ordered monoid overall because successive differences depend on adjacency. circadian_np_aggregator_test.dart: integer input numeric relative 1e-12 with exact start epochs; real input numeric relative 1e-9, start epochs deliberately not compared near ties. Common laws and malformed checkpoint tests. fieldsEqual defaults absolute tolerance to zero unless supplied.
CircularWindow, bestCircularWindow One completed daily profile and best mean/start; no accumulated input identity Batch circular search of all fixed-width windows with first-winner ties. Lift computes a window mean; finish selects max/min. No public combine or monoid contract; changing profile bins can change the winner globally. Circadian conversion cases preserve exact integer-data starts and qualify real-data ties as above. Numeric profile/result tolerances are those of the enclosing circadian tests. No independent associative-state law.
Epochs, _PairCounts, _PairCountsAggregator, _GroupedPairWindows, sriAggregator Sleep/valid byte sequences with absolute first index; per-day-pair agreements/valid counts and bounded day-offset edge context; empty child states Lookback(epochsPerDay+1) lifts valid day-separated pairs, grouping per day-pair; combine counts plus missing seam pairs; finish Phillips SRI and coverage gates. Ordered monoid under fixed epochs/day and valid layout. sri_aggregator_test.dart: exact all observable fields, common laws, index/join semantics and JSON continuation. Malformed-mask/negative-parameter public behavior remains in batch fallbacks rather than forced into the aggregator domain.
banisterTrimpAggregator Product of row count and compensated transformed sum; zero count/sum Add counts/sums; lift sex/HR-reserve-weighted finite HR contributions under fixed RHR/HRmax; finish value and absence/confidence. Approximate commutative monoid. trimp_aggregator_test.dart: numeric relative 1e-12, metadata exact, common laws and checkpoints. Compensated sum ordering can alter the last bits.
ctlAtlTsbAggregator Count plus first-prime rows and two affine EWMA summaries for the suffix; empty prefix and identity maps Compose chronologically, moving overflow out of prime prefix; lift daily loads; finish seed mean, chronic/acute recurrence and balance. Ordered noncommutative monoid through function composition. ctl_atl_aggregator_test.dart: common laws, seed/refusal cases, checkpoints; CTL/ATL/TSB 1e-9 absolute and relative, confidence/metadata exact.
HrStream, _PairHistograms, _ZoneHistograms, _TimeInZoneState, _TimeInZoneAggregator Exact gap/cadence histograms, per-zone gap weights, first/last context and provisional tail HR; empty histograms/no tail Add histograms plus the adjacent seam under Lookback(2); lift timestamp/HR rows; finish global median cadence, validation, capped zone durations and tail credit. Ordered monoid; histogram interiors commute, boundary/tail selection does not. time_in_zone_aggregator_test.dart: exact presence/cadence policy/count evidence, seconds relative 1e-12 plus absolute 1e-9, common laws, checkpoint and entry-count bounds. State size follows distinct gaps, which can grow with irregular timestamps.
readinessLnRmssdAggregator, suffix scorer Count and last windowDays+1 raw nights; zero count/empty suffix Count addition and suffix-of-concatenation; lift nightly values; finish identical suffix scoring, baseline evidence and need-baseline note. Ordered monoid for positional history. Date-aware filtering is outside its domain. readiness_lnrmssd_aggregator_test.dart: exact flattened metric value, flags, confidence, notes and provenance, common laws/checkpoints and bounded suffix cases. Negative window settings stay with the existing batch behavior.

Typed column batches concatenate in order with zero rows as identity, and their codecs preserve indices/nullable masks. These containers are sequence monoids; their finished metric summaries have the qualifications above. No separate new physiological estimator is introduced by the representation change.

Where the speedup comes from

Day and circadian calculations rebuild existing history, so this PR retains their summaries; the measured night has no corresponding calls. This PR: 0.000 of the 109.065 s saved per night (0.00%). Overall after the full stack: 13.98×.

Workload / calculation #102 batch s Retained s This PR saves s
day / TRIMP 0.000598 0.000041 0.000558
day / Time in zones 0.752842 0.006541 0.746301
history / Cosinor 0.024690 0.000338 0.024352
history / Circadian 0.179642 0.198695 -0.019053
history / SRI 0.002381 0.000261 0.002120
history / CTL ATL TSB 0.000199 0.000032 0.000167
history / Readiness 0.000262 0.000161 0.000101
day / guarded strain remains batch — — 0.000000
night / no calls in this workload — — 0.000000

See the sync-work explanation in #103 for the timeline, overall table and Edge adoption requirement; these are AOT medians from at least five full schedules on generated data, with this PR credited only beyond #102 and its inherited gains included in the overall saving.

How each structure saves work

Cosinor merges a fixed-size fit in O(1) and finishes a fixed-size solve, replacing a full residual/history pass. TRIMP combines O(1) summaries. CTL/ATL combines affine maps in O(1) after managing its fixed priming prefix. Readiness retains only a bounded suffix and scores that suffix. Circadian accumulation touches new activity and its seam, while finish scans the daily profile. SRI still keeps day-offset context and per-day-pair evidence, so tiny slices cost O(epochsPerDay) copying. Zone state merge/finish depends on the number of distinct gaps, rather than constant space for arbitrary timestamps. Date-aware readiness and exceptional SRI inputs retain batch work.

Fresh timings are in "Where the speedup comes from" above. Older numbers in the docs predate this split.

Merging the review fixes

This branch was written before the review fixes on #102. Merging them in (ab91821) conflicted in cosinor.dart, load_trimp.dart and circadian_np.dart, because this part moves those calculations into aggregators. The merge keeps the aggregators and moves each refusal from #102 into them:

  • The cosinor aggregator refuses invalid or non-finite periods, including subnormal ones, and refuses non-finite moments before its determinant check.
  • The training-load aggregator refuses a non-finite daily TRIMP before the minimum-history check, so an unknown day can't pass as a rest day. An observed zero still decays CTL and ATL.
  • The circadian variance stays centered and mergeable. A NaN stops later variance updates, matching the early stop in Abdul's change.

One output-adjacent change: the CTL/ATL checkpoint now stores finite days alongside total days. Older checkpoints can't show that an unknown load day was kept, so they are refused and the history is replayed.

The merge adds 9 tests in test/aggregate/review_input_refusals_test.dart. Each covers one slice, successive slices, merged states and JSON-restored states. Five failed before the guards. Disabling each of the three guards in turn made its test fail again.

Tests

Full suite on this branch with Dart 3.13.0: 2,369 passed, 7 skipped, 0 failed. dart analyze (Dart 3.11.4) reports no issues. Counts include the earlier parts' tests. The 7 skips are existing checks that need data outside the repo: six need a local whoop_hist.jsonl capture, and one wall-clock agreement test needs the DREAMT corpus.

Suggested review order

  1. f24c21f Circadian and training metrics replay unchanged history. Read the state and append transitions first, then the finish rules, then the law, checkpoint and reference tests.
  2. ab91821 merges part 04 and the review fixes, resolved as described above.

Change size

This part's own changes, measured against part 04: lib/ +1,824/-414 in 15 files; tests, tools and docs +3,310 in 9 files.

Questions for the reviewer

  • Is it worth keeping exact gap distributions for timestamp-weighted zones, given that they grow with the number of distinct gaps?
  • Are the real-data tie rule for circadian start times, and readiness's positional versus calendar distinction, acceptable public contracts?

@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @mikenrafter, this pull request is too large to review

It changes 34,637 lines. Sourcery can review pull requests with up to 20,000 changed lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants