feat(insights): habit↔Daily Score correlation engine (#221) - #224
feat(insights): habit↔Daily Score correlation engine (#221)#224Eliaaazzz wants to merge 2 commits into
Conversation
Implements feature 2 of 3 from issue #221 — adapts Whoop's 2026 Behavior Insights pattern to AuraFitness's nutrition lane. Each insight answers "is this habit actually helping me?" with statistical backing. Backend (Spring Boot, Java 21): - V53 migration: user_behavior_days, behavior_insights - Shared BehaviorPredicateRegistry (8 predicates) — single source of truth for "did the user X today?", reused by feature #222 (Challenges) - DailyScoreCalculator: 0..100 from a day's meal logs (4 weighted components, transparent algorithm) - WelchTTest: Welch two-sample t + Welch-Satterthwaite df + Cohen's d via Apache Commons Math (org.apache.commons:commons-math3:3.6.1) - BehaviorDeriver: 90-day backfill + per-day delta derivation - InsightStatsService: thresholds (p<0.10 AND |d|>=0.30), confidence tiers (high <0.01, med <0.05, low otherwise), tier gating (free top-3 positive only; pro all positive + negative) - InsightScheduler: nightly 03:00 UTC; idempotent per-user incremental derive + recompute, trims rows >90 days - SubscriptionService: stub returning FREE for everyone, env override for QA — to be wired to RevenueCat - BehaviorInsightController: list / cold-start / pin / unpin / dismiss / detail (90-day calendar + score split) / recompute / backfill - 18 Mockito unit tests across WelchTTest, DefaultBehaviorPredicates, DailyScoreCalculator, InsightStatsService tier filter Frontend (React Native + Expo, TypeScript): - InsightsScreen: cold-start onboarding card under 30 days, otherwise scrollable list sorted by absolute delta (pinned first), pull-to-refresh, manual recompute button - InsightCard: ±delta with up/down arrow, server-formatted sentence, ConfidenceChip (3 tiers), pin toggle, mandatory AI disclaimer footer per CLAUDE.md - ConfidenceChip, InsightsOnboardingCard with 30-day progress bar - Optimistic pin toggle via React Query onMutate / onError revert - 5 Jest tests covering sentence rendering, sign formatting, confidence, AI disclaimer, onPress wiring Routes registered in AppNavigator (`navigation.navigate('Insights')`); nav entry from Profile is a small follow-up. @EnableScheduling enabled. Compiles cleanly under tsc --noEmit on all new files. Closes #221
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
aurafit | 9104e25 | May 06 2026, 02:12 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
aurafitness | 9104e25 | May 06 2026, 02:12 PM |
Deploying aurafitness-2 with
|
| Latest commit: |
9104e25
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6f2c1e9f.aurafitness-2.pages.dev |
| Branch Preview URL: | https://feat-behavior-insights.aurafitness-2.pages.dev |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the Behavior Insights feature, which uses Welch's t-test to correlate user habits with a calculated Daily Score. The implementation includes backend services for data derivation, statistical analysis, and scheduling, as well as a React Native frontend for displaying insights. Key feedback focuses on improving backend scalability by avoiding full table loads and N+1 queries, ensuring statistical accuracy by excluding incomplete days and empty logs from calculations, and addressing timezone discrepancies in habit predicates.
| LocalDate yesterday = LocalDate.now(ZoneOffset.UTC).minusDays(1); | ||
| LocalDate trimAfter = yesterday.minusDays(90); | ||
|
|
||
| List<User> users = userRepository.findAll(); |
There was a problem hiding this comment.
Using userRepository.findAll() is a significant scalability risk. As the user base grows, loading all users into memory will lead to OutOfMemoryError and excessive processing times. Consider using a Stream (with @Transactional(readOnly = true)) or pagination to process users in batches.
| */ | ||
| @Transactional | ||
| public int recomputeForUser(UUID userId) { | ||
| LocalDate to = LocalDate.now(ZoneOffset.UTC); |
There was a problem hiding this comment.
The recomputeForUser method uses LocalDate.now(ZoneOffset.UTC) as the end of the window. This includes the current day, which is likely incomplete. Including partial data for 'today' will skew the statistical results (e.g., a low daily score because the day isn't over yet). It is recommended to only compute insights on completed days by setting the end date to yesterday.
| LocalDate to = LocalDate.now(ZoneOffset.UTC); | |
| LocalDate to = LocalDate.now(ZoneOffset.UTC).minusDays(1); |
| LocalDate cursor = from; | ||
| while (!cursor.isAfter(to)) { | ||
| List<MealLog> dayMeals = grouped.getOrDefault(cursor, List.of()); | ||
| Short score = (short) scoreCalculator.calculate(dayMeals); |
There was a problem hiding this comment.
If dayMeals is empty (i.e., the user didn't log anything that day), scoreCalculator.calculate returns 0. Storing a 0 score for empty days will heavily skew the correlation analysis, as 'no-habit' days will appear to have a much lower mean score than they actually do, leading to false positive insights. Empty days should have a null score so they are ignored by the stats engine.
| Short score = (short) scoreCalculator.calculate(dayMeals); | |
| Short score = dayMeals.isEmpty() ? null : (short) scoreCalculator.calculate(dayMeals); |
| try { | ||
| deriver.deriveRange(user.getId(), yesterday, yesterday); | ||
| // Trim everything older than the 90-day window so the table doesn't grow unbounded. | ||
| dayRepository.deleteByUserIdAndDayBetween(user.getId(), LocalDate.of(2000, 1, 1), trimAfter); |
There was a problem hiding this comment.
Performing a delete operation per user inside the loop is inefficient. This results in N delete queries every night. It is better to perform a single bulk delete operation outside the loop to remove all records older than the 90-day window for all users at once.
| dayRepository.deleteByUserIdAndDayBetween(user.getId(), LocalDate.of(2000, 1, 1), trimAfter); | |
| // Move this outside the loop and use a bulk delete method in the repository | |
| dayRepository.deleteByDayBefore(trimAfter); |
| for (String key : registry.all().stream().map(p -> p.key()).toList()) { | ||
| List<UserBehaviorDay> series = byKey.getOrDefault(key, List.of()); | ||
| List<Double> yes = new ArrayList<>(); | ||
| List<Double> no = new ArrayList<>(); | ||
| for (UserBehaviorDay r : series) { | ||
| if (r.getDailyScore() == null) continue; | ||
| if (r.isObserved()) yes.add((double) r.getDailyScore()); | ||
| else no.add((double) r.getDailyScore()); | ||
| } | ||
| if (yes.size() < MIN_PER_BUCKET || no.size() < MIN_PER_BUCKET) { | ||
| // Not enough data yet — drop any stale insight for this behavior | ||
| insightRepository.findByUserIdAndBehaviorKey(userId, key) | ||
| .ifPresent(insightRepository::delete); | ||
| continue; | ||
| } | ||
|
|
||
| WelchTTest.Result result = WelchTTest.run(toArray(yes), toArray(no)); | ||
| double delta = result.meanA() - result.meanB(); | ||
| double absD = Math.abs(result.cohensD()); | ||
|
|
||
| if (result.pValue() > P_VALUE_MAX || absD < COHENS_D_MIN) { | ||
| insightRepository.findByUserIdAndBehaviorKey(userId, key) | ||
| .ifPresent(insightRepository::delete); | ||
| continue; | ||
| } | ||
|
|
||
| String confidence = result.pValue() < P_HIGH ? "high" | ||
| : result.pValue() < P_MED ? "med" | ||
| : "low"; | ||
|
|
||
| Optional<BehaviorInsight> existing = insightRepository.findByUserIdAndBehaviorKey(userId, key); | ||
| BehaviorInsight insight = existing.orElseGet(() -> BehaviorInsight.builder() | ||
| .userId(userId).behaviorKey(key).build()); | ||
| insight.setDeltaScore(BigDecimal.valueOf(delta).setScale(2, RoundingMode.HALF_UP)); | ||
| insight.setCohensD(BigDecimal.valueOf(result.cohensD()).setScale(3, RoundingMode.HALF_UP)); | ||
| insight.setPValue(BigDecimal.valueOf(result.pValue()).setScale(4, RoundingMode.HALF_UP)); | ||
| insight.setSampleYes(yes.size()); | ||
| insight.setSampleNo(no.size()); | ||
| insight.setConfidence(confidence); | ||
| insight.setComputedAt(OffsetDateTime.now(ZoneOffset.UTC)); | ||
| insightRepository.save(insight); | ||
| written++; | ||
| } |
There was a problem hiding this comment.
This loop introduces an N+1 query pattern by calling insightRepository.findByUserIdAndBehaviorKey multiple times (lines 134, 144, 153) for each predicate. To improve performance, fetch all existing insights for the user into a Map<String, BehaviorInsight> before entering the loop.
| return meals.stream().anyMatch(m -> m.getConsumedAt() != null | ||
| && m.getConsumedAt().getHour() < BREAKFAST_BEFORE_HOUR); |
There was a problem hiding this comment.
The BreakfastLogged predicate checks the hour of consumedAt against a UTC constant. Since the deriver buckets by UTC, this logic will be incorrect for any user not in the UTC timezone (e.g., 8 AM in PST is 4 PM UTC). While the PR notes mention timezone bucketing is a follow-up, this significantly impacts the correctness of the 'Logged breakfast' and 'Ate after 9pm' insights for v1.
Deploying aurafitness with
|
| Latest commit: |
9104e25
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://083d05f4.aurafitness.pages.dev |
| Branch Preview URL: | https://feat-behavior-insights.aurafitness.pages.dev |
Deploying aurafitness-1 with
|
| Latest commit: |
9104e25
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://93502fd9.aurafitness-1.pages.dev |
| Branch Preview URL: | https://feat-behavior-insights.aurafitness-1.pages.dev |
Six gemini-code-assist findings — 3 high, 3 medium — all addressed: 1. [HIGH] InsightScheduler.runNightly() loaded the entire users table. Replaced findAll() with a Pageable batch loop (200/page) so memory stays bounded as the user base grows. 2. [HIGH] InsightStatsService.recomputeForUser() included today (incomplete day) in the window, biasing means downward. Window now ends at yesterday. 3. [HIGH] BehaviorDeriver wrote a 0-score row for every empty day, which collapsed the no-bucket mean toward 0 and produced false positives. Empty days now skip persistence entirely; cold-start distinct-day counts stay honest as a side benefit. 4. [MED] Per-user delete inside the nightly loop turned trim into N SQL statements. Replaced with a single bulk JPQL delete keyed on the cutoff, executed once outside the loop. 5. [MED] N+1 lookup pattern in recomputeForUser (findByUserIdAndBehaviorKey per predicate × delete-or-update). Fetch all of the user's existing insights once into a Map, look up O(1), and batch the delete via deleteAllByIdInBatch. 6. [MED] Time-of-day predicates (breakfast_logged, late_eating) checked the hour on the raw UTC OffsetDateTime, so a PST user at 8am local was counted as a 16:00 UTC log and never qualified as "breakfast". Plumbed ZoneId through BehaviorPredicate.evaluate, BehaviorDeriver.deriveRange, and the controller backfill endpoint (X-User-Timezone header). Scheduler uses a configurable default zone with a TODO for per-user resolution. New test: - DefaultBehaviorPredicatesTest.breakfastLogged_respectsCallerZone_pstUserAt8amLocal asserts that 16:00 UTC is "breakfast" in America/Los_Angeles but not in UTC. All 23 behavior tests still pass locally (4 suites, 0 failures).
|
All six review comments addressed in 9104e25. Mapping each finding to the fix:
New test: Local test run: all 4 behavior test suites pass (23 tests, 0 failures). Thanks for the review! |
1 participant
Summary
Implements Feature 2 of 3 from issue #221 — Behavior Insights. Adapts Whoop's 2026 Behavior Insights pattern to AuraFitness's nutrition lane. Each insight tells the user which of their habits actually moves their Daily Score — with statistical backing, not vibes.
This is the strongest T2D-positioning lever in the 3-feature bundle: showing "on days you logged breakfast, your Daily Score is +6.2 higher" is directly actionable for metabolic-health users that MyFitnessPal/Noom underserve.
Closes #221.
What's in this PR
Backend (Spring Boot, Java 21)
V53__create_behavior_insights.sql—user_behavior_days(composite PK) +behavior_insightsBehaviorPredicateRegistry— shared with feature feat: Nutrition Challenges — time-bound goals, rest days, badges, AI personalized (Strava Challenges pattern) #222 (Challenges) for one source of truth on "did the user X today?". 8 default predicates (breakfast_logged,late_eating,meal_count_3_or_more,protein_goal_hit,protein_high,calories_in_range,varied_meal_types,protein_per_meal_decent). Fiber/sugar/processed predicates from the issue spec are deferred until those nutrient fields land onmeal_log.DailyScoreCalculator— transparent 0..100 score frommeal_log(meals 30 + protein 30 + calories 30 + variety 10). Documented circularity caveat: predicates and score draw from the same signal; profile-aware targets are a follow-up.WelchTTest— Welch two-sample t + Welch–Satterthwaite df + Cohen's d, p-values via Apache Commons Math (commons-math3:3.6.1). Matchesscipy.stats.ttest_ind(equal_var=False)to ~1e-3 on hand-computed fixtures.BehaviorDeriver— 90-day backfill + per-day derivation, idempotent (delete-then-rewrite the slice).InsightStatsService— thresholdsp<0.10 AND |d|>=0.30, confidence tiers (high <0.01,med <0.05,low), pin / dismiss with 14-day cooldown, freshness window (insights >7d old hidden), tier gating (Free: top 3 positive;Pro/Premium: all sorted by |Δ|, pinned first).InsightScheduler—@EnableScheduling+ nightly 03:00 UTC; per-user incremental derive (yesterday only) + recompute + trim rows >90d.SubscriptionService— stub returningFREEfor everyone, withaurafitness.subscription.overrideenv hook for QA. To be replaced when RevenueCat ships.BehaviorInsightController—GET /insights,GET /insights/cold-start,POST /pin,DELETE /pin,POST /dismiss,GET /detail(90-day calendar + yes/no score split),POST /recompute,POST /backfill.ErrorCodeentries (2030–2032) wired throughGlobalExceptionHandlervia a newBehaviorInsightException.WelchTTestTest— fixture vs hand-computed reference, overlapping samples, undersized samples, identical constants, two-tailed symmetry across signDefaultBehaviorPredicatesTest— every predicate against fixtures + empty-day invariantDailyScoreCalculatorTest— empty / perfect / partial / cap / floorInsightStatsServiceTierFilterTest— free vs pro filtering, pin floats to topFrontend (React Native + Expo, TypeScript)
InsightsScreen— cold-start onboarding under 30 days; otherwise scrollableFlatListwith pull-to-refresh + manual recompute button. React QueryuseQuery+ optimistic pin toggle viaonMutate/revert-on-error.InsightCard— left accent bar (orange positive / muted negative), big ±Δ with up/down arrow, server-formatted sentence,ConfidenceChip, pin button, mandatory AI disclaimer footer per CLAUDE.md.ConfidenceChip— pill with three-tier dot indicator (solid / half-opacity / outlined).InsightsOnboardingCard— 30-day progress bar.AppNavigator; reachable vianavigation.navigate('Insights').InsightCardcovering sentence rendering, sign formatting, confidence chip, AI disclaimer,onPresswiring.Acceptance Criteria coverage
meal_log; 90-day backfillNotable design decisions
BehaviorPredicateinterface, registered once, used by Insights and the upcoming Challenges feature. Avoids the predicate logic drifting in two places.daily_scorestable — Score is computed at deriver time and stored onuser_behavior_days.daily_score. Adding a separate scores table would couple this PR to a schema change that's not yet justified.FREE. The single place to swap isSubscriptionService.getTier().What's intentionally out of scope (follow-ups)
BehaviorDetailSheet(calendar heat-grid + dual-line score split chart) — the API endpoint is implemented and tested; the bottom-sheet UI is a small follow-up so this PR stays reviewable.SubscriptionServiceis a stub.Test plan
WelchTTestTestreference fixture matches scipy to within 0.05 ont, 1e-6 on two-tailed symmetry.DefaultBehaviorPredicatesTestcovers every predicate + empty-day invariant.DailyScoreCalculatorTestcovers empty / perfect / partial / cap / floor.InsightStatsServiceTierFilterTestcovers free vs pro and pin priority.InsightCard.test.tsxcovers sentence + sign + confidence + disclaimer + onPress.tsc --noEmitpasses clean for all new/modified frontend files../gradlew test, apply V53 against fresh Postgres, hitPOST /api/v1/insights/backfill, confirm a few insights render with the AI disclaimer on the screen.