Pegasus foundation — Expo data layer, Modal TRIBE ML, NVIDIA video#1
Open
rishith-c wants to merge 5 commits into
Open
Pegasus foundation — Expo data layer, Modal TRIBE ML, NVIDIA video#1rishith-c wants to merge 5 commits into
rishith-c wants to merge 5 commits into
Conversation
Runnable stubs for all four services against shared/contract.md:
- ml (:8003) TRIBE v2 predictor (stub), deviation scoring, Claude intervention
- backend (:8001) orchestrator + SQLite, fans out to signals/ml, degrades gracefully
- signals (:8002) Claude sentiment + typing dynamics, Twilio alerts, collector.js
- frontend (:3000) Vite React check-in flow + Check Engine Light
Plus:
- shared/contract.{md,py} as the source of truth
- per-folder AGENTS.md + .claude/agents/* so each teammate's agent stays in its lane
- standalone imessage/ bridge (AppleScript send + chat.db poll, macOS, owner TBD)
- dev.sh to set up + run everything
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hi @rishith-c! 👋
Your private repo does not have access to Sourcery.
Please upgrade to continue using Sourcery ✨
ml (:8003):
- combined_scorer.py merges imessage/typing/facial/voice/tribe streams,
renormalizing weights over whichever streams are present (text-only scores too)
- tribe_inference.py (TribeModel) adds per-region activations + baseline caching
- new endpoints: POST /predict, GET /baseline/{stimulus_id}, POST /score (full bundle)
- claude_interpreter.py (renamed); stimuli/manifest.json; thresholds green<30/yellow<65/red>=65
video (:8004) — NEW service:
- facial_analyzer.py (MediaPipe Face Mesh), voice_analyzer.py (Whisper + librosa)
- main.py with /analyze/video + /analyze/frame; heavy models lazy-load
- opt-in in dev.sh via WITH_VIDEO=1
Contract deltas flagged for Jason in ml/AGENTS.md (score shape, thresholds, /intervention removed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
frontend (Expo, replaces Vite web app):
- bootstrapped Expo + TypeScript (App.tsx placeholder is Wesley's to replace)
- DATA LAYER (Rishith-owned): types/index.ts, services/{api,config}.ts,
hooks/{useResponseTracker,useBurnoutScore,useCamera} — the exact surface Wesley imports
ml (:8003):
- calls TRIBE v2 on Modal (pegasus-tribe) — no stub, lazy + offline-tolerant
- scoring.BurnoutScorer: HF emotion model + deviation math + Claude intervention;
returns full BurnoutResult (deviation, confidence, brain_regions_flagged, breakdown)
- dropped combined_scorer/tribe_inference/claude_interpreter stubs
video (:8004):
- custom facial_analyzer (my own EAR/geometry on MediaPipe landmarks)
- voice_analyzer via NVIDIA STT + librosa; tts.py via NVIDIA TTS (bonus)
docs/secrets:
- frontend/AGENTS.md documents the Rishith/Wesley split
- root AGENTS.md: FINAL team table + the 45-min sync cadence
- .env.examples for ml + video; .env stays gitignored
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rishith-c
changed the title
ml: - remove Claude entirely; BurnoutScorer now uses huggingface_hub.InferenceClient for BOTH sentiment (emotion model) and interventions (chat model, HF_MODEL). - requirements: drop anthropic/requests, add huggingface-hub. video: - voice_analyzer.py: real NVIDIA parakeet ASR over hosted Riva gRPC (grpc.nvcf.nvidia.com:443) via nvidia-riva-client + function-id. - tts.py: real NVIDIA Chatterbox TTS (function-id ddacc747-...) → WAV. - requirements: drop requests, add nvidia-riva-client. secrets: keys live only in gitignored ml/.env + video/.env (verified ignored). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foundation PR (FINAL spec, Hugging Face + NVIDIA stack). Rishith's lanes built & verified; other services scaffolded for their owners.
Rishith's work
useResponseTracker,useBurnoutScore,useCamera). The exact surface Wesley imports.BurnoutScorernow 100% Hugging Face (no Anthropic): emotion model for sentiment + chat model (HF_MODEL) for interventions, viahuggingface_hub.InferenceClient.grpc.nvcf.nvidia.com:443,nvidia-riva-client, function-ids).Secrets
Keys live ONLY in gitignored
ml/.env+video/.env(HF_TOKEN; NVIDIA_STT_KEY/NVIDIA_TTS_KEY)..env.exampletemplates committed. Rotate keys after the event.Scaffolded (other owners)
backend/ (Jason), signals/ (Dhruva —⚠️ still uses Claude; PRD says HF), shared/, frontend screens (Wesley). imessage/ legacy, superseded by Twilio SMS.
Verified
All Python compiles; scorer returns full BurnoutResult;
.envconfirmed gitignored. Not run live (need keys/installs): Modal, HF, NVIDIA Riva, mediapipe/librosa+ffmpeg, Expo.🤖 Generated with Claude Code