feat: phase-2 + AI calorie pipeline (LiDAR depth, USDA cache, geometric portion refinement) - #227
Merged
Merged
Conversation
…ength tracking Phase 2 mobile-first features inspired by Uber Eats (discovery) and Strava (achievement + social). Plus mobile YouTube playback fix and image perf work already discussed. Mobile perf & playback (carried from earlier in branch): - Rewire YouTubePlayerModal to react-native-youtube-iframe with ID validation, watchdog timeout, error UI, and Linking fallback to the YouTube app. - Swap RN Image for expo-image (memory+disk cache, transition) in WorkoutCard and RecipeImage (native branch). Web paths untouched. - Downgrade YouTube thumbs to mqdefault on native; prefetch first 6 thumbs on Workouts/Recipes screens. keepPreviousData on recommended hooks. F1 — Recipe Discovery Rows: HorizontalRecipeRow component + useRecipesByTag hook; five themed carousels (Recommended, Quick, High protein, Trending, Plant-forward) replace the flat grid on Recipes. F2 — Smart Filter Chips: FilterChipBar with six multi-select chips (time, kcal, protein, diet, budget, saved) filters every row client-side. F3 — Log Again: LogAgainCard on Home dashboard surfaces last 7 days' meals, deduped to 3 most-recent unique names; useReLogMeal mutation re-logs at today's timestamp. F4 — Bottom-sheet detail pattern: reusable DetailBottomSheet primitive (slide-up modal w/ backdrop + drag handle); RecipeCard exposes onOpenDetail prop. F5 — Adaptive Home Hero: AdaptiveHomeHero morphs by time-of-day (morning / afternoon / evening / night) with personalized copy and CTAs. F6 — Achievements V2: AchievementsCard badge wall + useAchievements hook deriving 8 badges from existing data (streak, protein, veggies, cardio, hydration, macro mastery). Tap-to-detail sheet. F7 — Personal Records: PersonalRecordsCard + usePersonalRecords hook on Profile (longest streak, protein PR, most meals, on-target days, cleanest sodium). F8 — Weekly Summary: WeeklySummaryCard with totals, on-target days, PRs; native Share with formatted week recap. F9 — Friends & Activity Feed: FriendsFeedCard + useSocialStore (persistent, Zustand) with mock seed, kudos toggle, privacy flags per activity type. Ready to swap to a real backend feed endpoint when it lands. F10 — Challenges: ChallengesCard + useChallengeStore (persistent) with 4 templates (protein 5/7, breakfast 7, 4-in-14, calories in-range 10/30), progress bars, XP rewards. F11 — In-Workout Timer + Live Status Strip: useWorkoutSessionStore + floating WorkoutSessionStrip mounted at navigator root (persists across tabs). WorkoutCard gets a 'Start' button. F12 — Strength Sessions: StrengthLogModal w/ sets/reps/weight steppers + useStrengthLogStore with auto-PR detection via Epley 1RM. New 'Log strength' FAB on Workouts. F13 — Consistency Heatmap: ConsistencyHeatmap (GitHub-style 12-week grid) + useConsistencyCells hook on Profile. F14 — Day History: DayHistorySheet opens from any heatmap cell, shows that day's meals, strength sessions, and day score. Notes: - All UI follows the project's glass + orange/violet design system. - No backend schema changes; everything piggybacks on existing endpoints (meal-history, weekly-insights, recipes search, recommendations). - Local-only stores (social, challenges, strength, workout session) use Zustand + AsyncStorage. They expose stable APIs so a real backend can replace them later without touching components.
… Redis L2) as lookup fallback When the local food_nutrition table misses, fall back to the USDA FoodData Central API, cached two-level: in-process Caffeine (hot, recurring food vocab) + optional shared Redis (cross-instance, survives Cloud Run churn). Redis is optional/null-safe. Honest scope: density/ nutrition lookups, not the vision call. Tests: cache L1/L2/miss/redis-absent, client JSON parse, local-hit-skips-USDA.
…DAR) devices Adds ARScaleModule (native iOS) + useARScale hook: on ARKit devices without a depth sensor, raycast frame-center to a horizontal plane for camera-to-table distance -> img_w_cm, the same scale signal the LiDAR frame-processor emits (tier between LiDAR +-1cm and plate fallback +-5cm). Degrades to plate fallback when the native module is absent (never fabricates scale). Reference template -- ARKit raycasting needs a real device; capture-screen AR view wiring documented in README.
…) + trigram indexes Bulk-loads ~7,756 authoritative USDA FoodData Central SR Legacy foods (per 100g) into the local food_nutrition table, restoring the USDA grounding dropped in V44 but as LOCAL data (no external API on the hot path). ON CONFLICT (food_key) DO NOTHING preserves the existing hand-curated common foods (incl. Chinese names). Adds pg_trgm GIN indexes for fast fuzzy lookup. Flyway disabled in tests so this is prod-only; validated via sqlite (7.8k rows, ON CONFLICT preserves curated rows).
Roughly halves calorie error vs 2.0-flash at the same ~4s latency (Nutrition5k benchmark, docs/calorie-accuracy-investigation.md). Adds thinkingConfig (budget 0) guarded to 2.5-family models; configurable via GEMINI_MODEL / GEMINI_THINKING_BUDGET. Tests updated + thinkingConfig coverage.
Local Expo module (MetrifulLidar): brief AVCaptureDepthDataOutput session on builtInLiDARDepthCamera, computes center distance + on-device volume/area/height geometry. useLidarScale feeds img_w_cm + geometry into capture metadata. Replaces the non-functional VisionCamera frame-processor depth path; graceful fallback on non-LiDAR/web.
Cross-dataset depth corrector (N5k/MetaFood3D/SimpleFood45 ~25% zero-shot), LiDAR/ARKit integration, DB design, latency, limitations.
…onfig Batches pre-existing working-tree changes: deploy-backend.yml/Dockerfile (Cloud Run 2Gi/2cpu scaling fix), scale.ts ARKit scale module + tests, jest config, RedisConfig, UI components and screens.
…e accuracy Add a physics decomposition that corrects portion (the dominant calorie error) from an absolute LiDAR-measured food volume, keeping the vision model's food identity and energy density. kcal = flashKcal^w x physicsKcal^(1-w) (geometric-mean blend), where physicsKcal = (volume/volumeBias) x massWeightedDensity x sceneEnergyDensity. Mass comes from geometry + a food-intrinsic density table; energy density (kcal/g) stays the model's. This generalizes with no in-domain calibration: on SimpleFood45 (real phone photos, measured volume) pure physics measures ~6% median calorie APE vs ~27% raw, per docs/calorie-accuracy-roadmap-25-to-15.md. Backend: - FoodDensityCategory: add per-category bulk density (g/cm3) LUT. - FoodCategoryClassifier: deterministic food-name -> density category (short cues match whole tokens so "tea" doesn't hide in "steak"). - CaloriePhysicsRefinementService: scene-level geomean portion correction, clamped and a strict no-op without volume / when disabled. - FoodRecognitionRequestMetadata: parse volume_cm3 / area_cm2 / mean_h_cm. - Wire refinement into FoodRecognitionService (provider-agnostic). - application.yml: app.nutrition.physics-refine.* (enabled, blend-weight, volume-bias, min/max-scale), all env-overridable. Frontend: - Thread the on-device LiDAR volumeCm3 through ReviewMealScreen into the analyze request as volume_cm3; type it in nutritionApi. Tests: CaloriePhysicsRefinementServiceTest (geomean math, no-op safety, clamp, density weighting, multi-item ratios, partial items), FoodCategoryClassifierTest, FoodDensityCategory density assertions. Backend suite green (-PskipContainerTests, offline); frontend tsc + 100 jest green.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
aurafit | b075147 | Jul 01 2026, 07:05 AM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
aurafitness | b075147 | Jul 01 2026, 07:06 AM |
Deploying aurafitness-2 with
|
| Latest commit: |
b075147
|
| Status: | 🚫 Build failed. |
Deploying aurafitness with
|
| Latest commit: |
b075147
|
| Status: | 🚫 Build failed. |
Deploying aurafitness-1 with
|
| Latest commit: |
b075147
|
| Status: | 🚫 Build failed. |
- Frontend (HIGH): clear stale LiDAR volumeCm3 on gallery/retake/basic-camera paths so a non-depth image is never refined with a prior capture's volume. - Classifier (MEDIUM): whole-word (token) matching for all single-word cues so "butter" no longer fires on "butterfly" (etc.); regular-plural handling keeps "chip"->"chips". Prefix cues replaced with singular forms; add grapefruit. - Service (LOW): skip refinement below a plausible minimum scene volume (2 cm3) so a near-zero noise reading no-ops instead of clamping to min-scale. - Service (LOW): harden config validation — reject non-finite values and reset an inconsistent min/max-scale pair to safe defaults. Tests added: butterfly/plurals/steak substring-FP guards; tiny-volume no-op; inconsistent-clamp-config reset. Backend suite green (offline); frontend tsc + 100 jest green. Note (pre-existing, out of scope): countable-unit items save grams from the count field, so scaled estimated_grams isn't persisted for pieces/servings — calorie correction still propagates via nutrition. Tracked as follow-up.
Owner
Author
|
C:/Program Files/Git/gemini review |
Reconcile with #225 (phase-2 discovery/social/tracking, already on main). Conflicts (all phase-2, taken from main as canonical): - DashboardScreen: username -> profile.displayName - DetailBottomSheet: onClose optional -> required - application.yml: keep the new gemini thinking-budget + nutrition.physics-refine block
- DashboardScreen: currentUser.data.username (UserProfileResponse has no displayName field, so #225's profile.displayName does not type-check). - DetailBottomSheet: onClose optional so the (title || onClose) guard compiles. Frontend tsc + 100 jest green; backend suite green.
Owner
Author
|
C:/Program Files/Git/gemini review |
Eliaaazzz
added a commit
that referenced
this pull request
Jul 12, 2026
- Add react-native-web-webview: react-native-youtube-iframe's web entry requires it, and the missing package has broken every "Deploy Frontend to Firebase Hosting" run since #227 (Metro: "Unable to resolve module react-native-web-webview"), leaving aurafitness.org on a stale build. - postbuild now runs only the node prerender script: the previous "cp -r public/* dist/" chain was Unix-only (cmd.exe: "system cannot find the path"), silently skipping the prerender on Windows. Expo already copies public/ into the export, and CI keeps its own copy step.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the accumulated
feat/phase-2-discovery-social-trackingbranch tomain. Headline new work is the AI calorie pipeline, capped by a geometric LiDAR-volume portion refinement that attacks the dominant calorie error (portion/grams) while keeping the vision model's strengths (food identity + energy density).What's in this branch (main..HEAD, 10 commits)
kcal = flashKcal^w × physicsKcal^(1-w)(geometric-mean blend), wherephysicsKcal = (volume/volumeBias) × massWeightedDensity × sceneEnergyDensity. Mass from geometry + a food-intrinsic density table; energy density stays the model's. Generalizes with no in-domain calibration — seedocs/calorie-accuracy-roadmap-25-to-15.md§10 (SimpleFood45 real phone photos: ~6% median APE pure physics vs ~27% raw).gemini-2.5-flash(thinking disabled); USDA SR Legacy seed (~7.8k foods) + trigram indexes; USDA FoodData Central two-level cache (Caffeine L1 + Redis L2) as lookup fallback.The calorie-physics feature (detail)
FoodDensityCategory: per-category bulk density (g/cm³) LUT added.FoodCategoryClassifier: deterministic food-name → density category (short cues match whole tokens so"tea"can't hide inside"steak").CaloriePhysicsRefinementService: scene-level geomean portion correction; clamped to[min,max]-scale; strict no-op when disabled or no LiDAR volume is present (non-depth devices, web, gallery photos) — behaviour is never worse than the model alone.FoodRecognitionRequestMetadata: parsevolume_cm3/area_cm2/mean_h_cm; wired provider-agnostically inFoodRecognitionService.app.nutrition.physics-refine.*(enabled,blend-weight,volume-bias,min/max-scale), all env-overridable; all values generalize (no per-domain fit).volumeCm3throughReviewMealScreeninto the analyze request asvolume_cm3.Test plan
./gradlew test -PskipContainerTests --offline(BUILD SUCCESSFUL).CaloriePhysicsRefinementServiceTest(geomean math, no-op safety, clamp, per-food density weighting, multi-item ratio preservation, partial/null items),FoodCategoryClassifierTest,FoodDensityCategorydensity assertions.tsc --noEmitclean;jest --ci100/100 (10 suites).volume_cm3reaches/analyzeand calories shift toward the geometric estimate. (owner: manual)Risk / rollback
volume_cm3; without it the pipeline is unchanged. FlipNUTRITION_PHYSICS_REFINE_ENABLED=falseto disable instantly, orNUTRITION_PHYSICS_BLEND_WEIGHT=1.0to fall back to pure model output.