Integration: working stack (backend+signals+ml+TRIBE+video) + safe sync#3
Open
rishith-c wants to merge 10 commits into
Open
Integration: working stack (backend+signals+ml+TRIBE+video) + safe sync#3rishith-c wants to merge 10 commits into
rishith-c wants to merge 10 commits into
Conversation
Runnable stubs for all four services against shared/contract.md:
- ml (:8003) TRIBE v2 predictor (stub), deviation scoring, Claude intervention
- backend (:8001) orchestrator + SQLite, fans out to signals/ml, degrades gracefully
- signals (:8002) Claude sentiment + typing dynamics, Twilio alerts, collector.js
- frontend (:3000) Vite React check-in flow + Check Engine Light
Plus:
- shared/contract.{md,py} as the source of truth
- per-folder AGENTS.md + .claude/agents/* so each teammate's agent stays in its lane
- standalone imessage/ bridge (AppleScript send + chat.db poll, macOS, owner TBD)
- dev.sh to set up + run everything
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ml (:8003):
- combined_scorer.py merges imessage/typing/facial/voice/tribe streams,
renormalizing weights over whichever streams are present (text-only scores too)
- tribe_inference.py (TribeModel) adds per-region activations + baseline caching
- new endpoints: POST /predict, GET /baseline/{stimulus_id}, POST /score (full bundle)
- claude_interpreter.py (renamed); stimuli/manifest.json; thresholds green<30/yellow<65/red>=65
video (:8004) — NEW service:
- facial_analyzer.py (MediaPipe Face Mesh), voice_analyzer.py (Whisper + librosa)
- main.py with /analyze/video + /analyze/frame; heavy models lazy-load
- opt-in in dev.sh via WITH_VIDEO=1
Contract deltas flagged for Jason in ml/AGENTS.md (score shape, thresholds, /intervention removed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
frontend (Expo, replaces Vite web app):
- bootstrapped Expo + TypeScript (App.tsx placeholder is Wesley's to replace)
- DATA LAYER (Rishith-owned): types/index.ts, services/{api,config}.ts,
hooks/{useResponseTracker,useBurnoutScore,useCamera} — the exact surface Wesley imports
ml (:8003):
- calls TRIBE v2 on Modal (pegasus-tribe) — no stub, lazy + offline-tolerant
- scoring.BurnoutScorer: HF emotion model + deviation math + Claude intervention;
returns full BurnoutResult (deviation, confidence, brain_regions_flagged, breakdown)
- dropped combined_scorer/tribe_inference/claude_interpreter stubs
video (:8004):
- custom facial_analyzer (my own EAR/geometry on MediaPipe landmarks)
- voice_analyzer via NVIDIA STT + librosa; tts.py via NVIDIA TTS (bonus)
docs/secrets:
- frontend/AGENTS.md documents the Rishith/Wesley split
- root AGENTS.md: FINAL team table + the 45-min sync cadence
- .env.examples for ml + video; .env stays gitignored
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ml: - remove Claude entirely; BurnoutScorer now uses huggingface_hub.InferenceClient for BOTH sentiment (emotion model) and interventions (chat model, HF_MODEL). - requirements: drop anthropic/requests, add huggingface-hub. video: - voice_analyzer.py: real NVIDIA parakeet ASR over hosted Riva gRPC (grpc.nvcf.nvidia.com:443) via nvidia-riva-client + function-id. - tts.py: real NVIDIA Chatterbox TTS (function-id ddacc747-...) → WAV. - requirements: drop requests, add nvidia-riva-client. secrets: keys live only in gitignored ml/.env + video/.env (verified ignored). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Assembles the team's branches onto one branch and makes the core flow run end-to-end (verified locally): - my ml/ + video/ + frontend data layer - Jason's backend/ + shared/ (contract.json, stimuli) - Dhruva's signals/ (HF sentiment + Bloo.io SMS) Fixes: - ml: Modal app name -> synapse-tribe-v2-api (real deployed app), env-overridable (MODAL_TRIBE_APP/CLASS/METHOD); breakdown reads backend's facial_score/voice_score. Verified: backend(8001)+signals(8002)+ml(8003)+video(8004) all boot /health; POST /response/submit orchestrates signals->ml->TRIBE(Modal, connected) and returns a full burnout_result; discriminates healthy(26/green) vs burnt(43/yellow). HF chat interventions fall back to canned in this sandbox (HF host unreachable here); work on a non-sandboxed machine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sync.sh runs the safe local half (backup branch -> plain push -> ff-only dev -> merge dev in) and is impossible to use destructively: zero reset --hard/clean/ checkout --theirs/force. Guards two silent-loss traps the adversarial review found: git rerere replaying one-sided resolutions, and merges clobbering gitignored .env. AGENTS.md cadence updated to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hi @rishith-c! 👋
Your private repo does not have access to Sourcery.
Please upgrade to continue using Sourcery ✨
…ntom get_baseline) The deployed Modal app is synapse-tribe-v2-api / class TribeV2 / methods predict(text)+predict_video(url) — there is NO get_baseline, so the old call silently fell back to a default baseline. New ml/tribe_client.py calls the deployed FastAPI endpoint POST /api/analyze-text (avoids the modal.Cls local class-proxy bug), maps the 6 TRIBE macro-groups (Attention/Auditory/Emotion/ Language/Motor/Visual) -> region activations, and caches per stimulus_id. Verified live: /health -> facebook/tribev2; baseline for audio_01 returns real activations (Motor/Auditory/Language dominant — correct for a listening prompt). Endpoint + timeout are env-overridable (TRIBE_JSON_API, TRIBE_TIMEOUT). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ns + PRD fusion scoring Frontend (Expo SDK51, TS): foundation (theme, nav, App), 14 screens/components — HomeScreen + CheckEngine (pulsing orb), Stimulus (typing biometrics capture), CheckIn (camera+NVIDIA), Metrics (charts), Brain, History. Brain3D now loads synapse's real fsaverage5.glb (FreeSurfer surface) via expo-three, recolors by TRIBE region activation, and is lifecycle-safe (single render loop, imperative recolor, dispose, procedural fallback) — fixes the GL-leak the review found. ml: scoring now uses PRD §4.5 weighted fusion (imessage/typing/facial/voice/tribe, renormalized over present streams) so behavioral signals drive the score; TRIBE deviation is directional. RAG intervention engine (HF over a mental-health coping corpus) wired into _intervene with layered fallback. shared/stimuli.json: added calm/activating image stimuli. Verified live: burnt response -> 55 (yellow, typing 88), healthy -> 29 (green); intervention is RAG-sourced; real TRIBE baseline flows in (tribe_client HTTP). Frontend type-check/run needs `cd frontend && npm install` + a device (not run here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…G, face-emotion framework - Talk: tap-to-talk voice call (NVIDIA Magpie TTS + parakeet STT), face outline, Apple-camera shutter, crash-safe focus handling - Wellness score (higher = better), derived live from talks + texts; ML /wellness + /history - SMS/iMessage: Bloo.io + free Mac-iMessage fallback with real image attach, reminders, inbound chat.db poller (Full Disk Access) - RAG companion (HF Qwen + all-MiniLM-L6-v2 over a coping corpus), crisis + masking aware - Meta TRIBE v2 (Modal A10G) baseline + weighted fusion scoring - HF facial emotion + voice fusion; standalone face-emotion-api framework - Apple-HIG light UI, blur-on-scroll, History tab; web admin console (:4000) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ties the team's branches into one branch that runs end-to-end (verified locally).
Assembled
Verified working (local)
/healthPOST /response/submitorchestrates signals → ml → TRIBE (Modal, connected) and returns the fullburnout_resultservices: {signals:true, ml:true}pegasus-tribe→synapse-tribe-v2-api(the real deployed app), classTribePredictorconfirmed reachable; env-overridable. breakdown reads backend'sfacial_score/voice_score.Needs your env to fully light up (not code bugs)
BLOOIO_API_KEY+ a phone (Dhruva's).Also
sync.sh+ hardened 45-min cadence (adversarially reviewed: no revert/deletion; guards git rerere + .env clobber).Frontend screens (Wesley) + 3D brain + RAG interventions are being built next.
🤖 Generated with Claude Code