Repository navigation
05: Update circadian and training metrics without replaying history - #107
Draft
mikenrafter wants to merge 16 commits into
Draft
mikenrafter wants to merge 16 commits into
mikenrafter wants to merge 16 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
Sorry @mikenrafter, this pull request is too large to review
It changes 34,637 lines. Sourcery can review pull requests with up to 20,000 changed lines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on 04 / #106.
Depends on 03 / #105.
Depends on 02 / #104.
Depends on 01 / #103.
This is part 05 of the split of #99. It targets
main, so its diff also contains the earlier parts until they merge. Its own changes are the commits listed under "Suggested review order". After each earlier part merges, I'll mergemaininto this branch so the diff shrinks to this part alone.Circadian and training calculations replay long histories even when their summaries can be updated from new slices. This composes the existing batch arithmetic from per-period statistics, boundary windows, suffixes and affine recurrences.
What changes
The algorithms retain their source citations: Halberg and Nelson 1979 cosinor rhythmometry, Phillips et al. 2017 SRI, Witting 1990/van Someren 1999 nonparametric IS/IV/RA, Banister 1991/Morton et al. 1990 training load and Plews et al. 2013 lnRMSSD readiness. The precise method references remain beside the arithmetic and in
ALGORITHMS.mdin the final area. Internal cadence/duration-credit policies remain ESTIMATE. This branch reuses the parent's published-output policy; changing sum order can still alter floating rounding, near-singular classification and near-tied L5/M10 starts. SRI and positional readiness comparisons are exact; calendar filtering remains in the batch wrapper. Adoption retains the parent's EdgekAlgoVersionrequirement.Algebraic breakdown
All aggregator instances require fixed period/bin origin, HR anchors, zone set, session/history parameters and other configuration. The common test set covers identities, association, splits, singleton/sliced lifts, pure finish and supported JSON continuation through finished observations. Approximate numeric monoids are mathematical summaries whose floating implementations satisfy fixture tolerances, not exact associative arithmetic on doubles.
TimedValues,_CosinorState,_CosinorStats,cosinorAggregatorcosinor_aggregator_test.dartchecks conversion, common laws, ill conditioning, offsets and restarts. Mesor/amplitude relative1e-9, scale-sensitive absolute floors1e-12*max(1,abs(mesor)); circular phase1e-9with documented floor, hours relative to period1e-9; R2, adjusted R2, p/confidence absolute1e-7. Vanishing amplitude omits meaningless phase comparisons. Discrete metadata exact on fixtures. Generic harness does not separately test commutativity.EpochSeries,_CircadianStats,_CircadianStatsAggregator_SquaredDiffs,_RawSum,circadianNpAggregator, circadian window helperLookback(2)successive squared differences; child identitiescircadian_np_aggregator_test.dart: integer input numeric relative1e-12with exact start epochs; real input numeric relative1e-9, start epochs deliberately not compared near ties. Common laws and malformed checkpoint tests.fieldsEqualdefaults absolute tolerance to zero unless supplied.CircularWindow,bestCircularWindowEpochs,_PairCounts,_PairCountsAggregator,_GroupedPairWindows,sriAggregatorLookback(epochsPerDay+1)lifts valid day-separated pairs, grouping per day-pair; combine counts plus missing seam pairs; finish Phillips SRI and coverage gates. Ordered monoid under fixed epochs/day and valid layout.sri_aggregator_test.dart: exact all observable fields, common laws, index/join semantics and JSON continuation. Malformed-mask/negative-parameter public behavior remains in batch fallbacks rather than forced into the aggregator domain.banisterTrimpAggregatortrimp_aggregator_test.dart: numeric relative1e-12, metadata exact, common laws and checkpoints. Compensated sum ordering can alter the last bits.ctlAtlTsbAggregatorctl_atl_aggregator_test.dart: common laws, seed/refusal cases, checkpoints; CTL/ATL/TSB1e-9absolute and relative, confidence/metadata exact.HrStream,_PairHistograms,_ZoneHistograms,_TimeInZoneState,_TimeInZoneAggregatorLookback(2); lift timestamp/HR rows; finish global median cadence, validation, capped zone durations and tail credit. Ordered monoid; histogram interiors commute, boundary/tail selection does not.time_in_zone_aggregator_test.dart: exact presence/cadence policy/count evidence, seconds relative1e-12plus absolute1e-9, common laws, checkpoint and entry-count bounds. State size follows distinct gaps, which can grow with irregular timestamps.readinessLnRmssdAggregator, suffix scorerwindowDays+1raw nights; zero count/empty suffixreadiness_lnrmssd_aggregator_test.dart: exact flattened metric value, flags, confidence, notes and provenance, common laws/checkpoints and bounded suffix cases. Negative window settings stay with the existing batch behavior.Typed column batches concatenate in order with zero rows as identity, and their codecs preserve indices/nullable masks. These containers are sequence monoids; their finished metric summaries have the qualifications above. No separate new physiological estimator is introduced by the representation change.
Where the speedup comes from
Day and circadian calculations rebuild existing history, so this PR retains their summaries; the measured night has no corresponding calls. This PR: 0.000 of the 109.065 s saved per night (0.00%). Overall after the full stack: 13.98×.
See the sync-work explanation in #103 for the timeline, overall table and Edge adoption requirement; these are AOT medians from at least five full schedules on generated data, with this PR credited only beyond #102 and its inherited gains included in the overall saving.
How each structure saves work
Cosinor merges a fixed-size fit in O(1) and finishes a fixed-size solve, replacing a full residual/history pass. TRIMP combines O(1) summaries. CTL/ATL combines affine maps in O(1) after managing its fixed priming prefix. Readiness retains only a bounded suffix and scores that suffix. Circadian accumulation touches new activity and its seam, while finish scans the daily profile. SRI still keeps day-offset context and per-day-pair evidence, so tiny slices cost O(epochsPerDay) copying. Zone state merge/finish depends on the number of distinct gaps, rather than constant space for arbitrary timestamps. Date-aware readiness and exceptional SRI inputs retain batch work.
Fresh timings are in "Where the speedup comes from" above. Older numbers in the docs predate this split.
Merging the review fixes
This branch was written before the review fixes on #102. Merging them in (
ab91821) conflicted incosinor.dart,load_trimp.dartandcircadian_np.dart, because this part moves those calculations into aggregators. The merge keeps the aggregators and moves each refusal from #102 into them:One output-adjacent change: the CTL/ATL checkpoint now stores finite days alongside total days. Older checkpoints can't show that an unknown load day was kept, so they are refused and the history is replayed.
The merge adds 9 tests in
test/aggregate/review_input_refusals_test.dart. Each covers one slice, successive slices, merged states and JSON-restored states. Five failed before the guards. Disabling each of the three guards in turn made its test fail again.Tests
Full suite on this branch with Dart 3.13.0: 2,369 passed, 7 skipped, 0 failed.
dart analyze(Dart 3.11.4) reports no issues. Counts include the earlier parts' tests. The 7 skips are existing checks that need data outside the repo: six need a localwhoop_hist.jsonlcapture, and one wall-clock agreement test needs the DREAMT corpus.Suggested review order
f24c21fCircadian and training metrics replay unchanged history. Read the state and append transitions first, then the finish rules, then the law, checkpoint and reference tests.ab91821merges part 04 and the review fixes, resolved as described above.Change size
This part's own changes, measured against part 04:
lib/+1,824/-414 in 15 files; tests, tools and docs +3,310 in 9 files.Questions for the reviewer