diff --git a/2026-01-07_Epic3_Recommender_Agent_Plan_DRAFT.md b/2026-01-07_Epic3_Recommender_Agent_Plan_DRAFT.md deleted file mode 100644 index 657d8b44..00000000 --- a/2026-01-07_Epic3_Recommender_Agent_Plan_DRAFT.md +++ /dev/null @@ -1,686 +0,0 @@ -# Epic 3: Recommender/Advisor Agent - Implementation Plan (DRAFT) - -**Author:** Wilfred -**Reviewer:** Jasmin -**Date:** January 7, 2026 -**Status:** Jasmin has reviewed -**Context:** This is based on our discussion about merging Mode 1 and Mode 2, prioritizing action over exploration - ---- - -## What We Agreed On - -After our conversation, here's what I understand we're building: - -1. **You (Jasmin) build**: Node2Vec algorithm (possibly with reasoning layer) that takes the preference vector and outputs occupation recommendations -2. **I build**: Conversational agent that takes your Node2Vec output and motivates users to actually DO something (apply, train, explore), in the direction of the recommendations -3. **Success metric**: User takes action in the direction of the recommendations (not just says "I like it") - -The user journey is: -``` -Skills Elicitation (Epic 4) - → Preference Elicitation (Epic 2) - → BWS Occupation Ranking (40 occupation groups) - → **Your Node2Vec Algorithm** - → **My Recommender Agent** ← This is what I'm planning -``` - ---- - -## Data Contracts We Need to Define - -### What I'll Give You (Preference Vector Schema) - -From the preference elicitation agent I built (Epic 2), here's the structure you'll receive: - - -NOTE --> Since the preference .json input can change, I think it's better if I (Jasmin) programmatically pick up the field from the input .json -How can we make a class flexible to looking different? (e.g. having remote_options all of the sudden, or job_security removed) - -```python -class PreferenceVector(BaseModel): - """Simplified 7-dimensional preference vector from Epic 2""" - - # Core 7 dimensions (0.0 = low importance, 1.0 = high importance) - financial_importance: float = 0.5 - work_environment_importance: float = 0.5 - career_advancement_importance: float = 0.5 - work_life_balance_importance: float = 0.5 - job_security_importance: float = 0.5 - task_preference_importance: float = 0.5 - social_impact_importance: float = 0.5 - - # Metadata - confidence_score: float = 0.0 - n_vignettes_completed: int = 0 - - # Bayesian posterior (optional - if you want to use uncertainty) - posterior_mean: Optional[list[float]] = None # 7 dimensions - posterior_covariance_diagonal: Optional[list[float]] = None - - # Qualitative metadata (optional) - decision_patterns: dict[str, Any] = {} - tradeoff_willingness: dict[str, Any] = {} - values_signals: dict[str, Any] = {} - extracted_constraints: dict[str, Any] = {} -``` - -**Also available:** -- **BWS occupation scores** - the top 10 occupation groups the user ranked during Best-Worst Scaling -- **Skills vector** - from Epic 4 (skill IDs + proficiency levels) - -### What I Need From You (Node2Vec Output Schema) - -Based on your brainstorming document, I understood you were planning to give **three types of recommendations**: -1. **Occupation recommendations** - Career paths (e.g., "Data Analyst") -2. **Opportunity recommendations** - Actual job postings/internships (e.g., "Internship at XYZ Foundation") -[3. **Skills training recommendations** - Training courses (e.g., "Advanced Econometrics on Coursera")] --> This would be there in an ideal case, but I cannot promise that we will have this in our first iteration. Can we build the agent to be flexible on whether this is there or not. -NOTE --> Basically, let's make this agent flow work even if only 1 or 2 of the above are available. (e.g. start with occupations, move to opportunities only if user is 'ready' and opportunity recommendations aren't NA, similarly with skill trainings) - - -Here's what I'm proposing for the schema so please let me know if this works: -(Jasmin adapted some fields) - -```python -class OccupationRecommendation(BaseModel): - """Career path recommendation""" - - uuid: str # using taxonomy uuid - originUuid: str # using taxonomy origin uuid - rank: int # 1-N (1 = best match) - - occupation_id: str # e.g., "ESCO_occupation_12345" - occupation_code: str # e.g., "2512" (ESCO/KeSCO code) - occupation: str # e.g., "Data Analyst" - - confidence_score: float # 0.0-1.0 - justification: str # Why this matches the user - essential_skills: list[str] # Skills needed for this occupation - - # Optional - from DB1/DB2 - description: Optional[str] = None - typical_tasks: Optional[list[str]] = None - career_path_next_steps: Optional[list[str]] = None - labor_demand_category: Optional[str] = None # "high" / "medium" / "low" - - -class OpportunityRecommendation(BaseModel): - """Actual job posting / internship recommendation""" - - id: str # e.g., "opp_001" - rank: int # 1-N (1 = best match) - - opportunity_title: str # e.g., "Internship at XYZ Foundation" - location: str # e.g., "Nairobi" or "Remote" - - justification: str # Why this matches the user - essential_skills: list[str] # Skills needed - - # Optional - from DB3 (jobs database) - employer: Optional[str] = None - salary_range: Optional[str] = None - contract_type: Optional[str] = None # "full-time" / "internship" / "contract" - posting_url: Optional[str] = None - posted_date: Optional[str] = None - - -class SkillsTrainingRecommendation(BaseModel): - """Training course recommendation""" - - uuid: str # # using taxonomy uuid - originUuid: str # # using taxonomy origin uuid - rank: int # 1-N (1 = best match) - - skill: str # e.g., "Advanced Econometrics" - - justification: str # Why this training is relevant - - # Optional - from DB4 (training database) - provider: Optional[str] = None # e.g., "Coursera" (Jasmin moved this to optional) - cost: Optional[str] = None - location: Optional[str] = None # "Online" / "Nairobi" / etc. - delivery_mode: Optional[str] = None # "online" / "in-person" / "hybrid" - target_occupations: Optional[list[str]] = None # Which occupations this unlocks - - -class Node2VecRecommendations(BaseModel): - """Complete output from your Node2Vec algorithm""" - - youth_id: str - generated_at: datetime - recommended_by: list[str] # ["Algorithm"] or ["Human", "Algorithm"] - - # Three types of recommendations - occupation_recommendations: list[OccupationRecommendation] - opportunity_recommendations: list[OpportunityRecommendation] - skillstraining_recommendations: list[SkillsTrainingRecommendation] - - # Algorithm metadata - algorithm_version: str = "node2vec_v1" - confidence: float # Overall confidence in recommendations -``` - -**Questions for you:** -1. Does this three-part structure match what you're planning? -2. How many of each type will you return? (e.g., 5 occupations, 10 opportunities, 5 trainings?) - -4. For opportunities - are these coming from DB3 (Epic 1's jobs database) or will you query that yourself? -5. For trainings - same question, are you using DB4 or should I handle that separately? -6. The `recommended_by` field - when would this be `["Human", "Algorithm"]` vs just `["Algorithm"]`? - ---- - -## How the Agent Will Work (Conversation Flow) - -I'm structuring this as a phase-based conversation, similar to how the preference agent works. Here's the flow: - -### Phase Structure - -```python -class ConversationPhase(str, Enum): - INTRO = "INTRO" # Explain what's coming - PRESENT_RECOMMENDATIONS = "PRESENT" # Show top 3-5 occupations - CAREER_EXPLORATION = "EXPLORATION" # Deep-dive on specific occupation - ADDRESS_CONCERNS = "CONCERNS" # Handle resistance/objections - DISCUSS_TRADEOFFS = "TRADEOFFS" # Preference vs demand balance - FOLLOW_UP = "FOLLOW_UP" # Clarify user responses - SKILLS_UPGRADE_PIVOT = "SKILLS_UPGRADE" # User rejected all → training path - ACTION_PLANNING = "ACTION" # Concrete next steps - WRAPUP = "WRAPUP" # Summarize, confirm plan - COMPLETE = "COMPLETE" # Session done -``` - -### Note for all phases -The overall idea is that all agents together optimize for user *effort* in the *direction of the recommendations*. This is common to all, and should be in the prompts as overarching goal (although we can still work on the exact phrasing). The instruction should be that although we provide guidance to the agents on how to behave/react, the LLM should primarily always be guided by this overarching goal and make decisions accordingly. - -The agent should internally treat success as applications submitted, steps taken, or persistence after first rejection; not stated agreement, or liking the recommendation - -Also common to all prompts: Clearly railguard the LLM to stay truthful, but also to be as persuasive as possible. - -### Detailed Phases - -#### Phase 1: INTRO -Set expectations about what we're doing. - -**Example:** -``` -"Great! Based on what you've shared about your skills and preferences, -I've identified some career paths that could be a really good fit for you. - -I'll show you a few options, and we can discuss what appeals to you -and what concerns you might have. There's no pressure - I just want -to help you understand what's out there and find something worth pursuing. - -Ready to see what I found?" -``` - -**Next:** User confirms → PRESENT_RECOMMENDATIONS - ---- - -#### Phase 2: PRESENT_RECOMMENDATIONS -Show recommendations from your Node2Vec output. Since you're providing **three types** (occupations, opportunities, trainings), I'll present them strategically based on the user's situation. - -**Presentation strategy:** -- **Start with occupations** (career paths) to give big-picture direction -- **Then show opportunities** (actual jobs) if they express interest in an occupation -- **Show trainings** when there's a skill gap OR when they reject occupations - -**Example (Occupation-first approach):** -``` -Based on your skills and preferences, here are career paths that match what you're looking for: - -**Career Paths:** -1. **Data Analyst** (High demand, KES 60,000-120,000/month) - - Matches your analytical skills and preference for structured work - - Growing field in Kenya with many opportunities - -2. **M&E Specialist** (Steady demand, KES 50,000-100,000/month) - - Uses your fieldwork experience and evaluation skills - - Common in NGO sector (aligns with your values) - -Which of these interests you? I can also show you actual job openings or training options. -``` - -**If user shows interest in an occupation:** -``` -Great! For Data Analyst roles, here are some actual opportunities available now: - -**Current Openings:** -1. **Internship at XYZ Foundation** (Nairobi) - - Entry-level, builds foundation in impact evaluation - - 6-month program with potential for full-time hire - -2. **Research Assistantship at ABC Lab** (Remote) - - Flexible work arrangement (you ranked this high!) - - Focus on quantitative analysis - -Want to explore these or see the other career path first? -``` - -**What I'm tracking:** -```python -user_interest_signals: dict[str, str] = {} -# {"occ_001": "interested", "occ_002": "rejected", "opp_001": "exploring"} - -rejected_occupations: int = 0 -rejected_opportunities: int = 0 -explored_items: list[str] = [] # IDs of occupations/opportunities explored -current_recommendation_type: str = "occupation" # "occupation" / "opportunity" / "training" -``` - -**User responses → routing:** -- "Tell me more about X occupation" → CAREER_EXPLORATION (deep-dive on X) -- "Show me jobs for X" → Present opportunities for that occupation -- "I don't think X is right because..." → ADDRESS_CONCERNS -- "None of these careers work for me" → Show trainings OR SKILLS_UPGRADE_PIVOT - ---- - -#### Phase 3: CAREER_EXPLORATION -Deep-dive on a specific occupation the user is interested in. - -**What I'll do:** -- Fetch additional data from DB1 (typical tasks, day-to-day, career progression) -- Explain concretely what the work involves -- Connect to their preferences ("You said autonomy matters - this role offers...") -- Show career path progression - -**Example:** -``` -Let's dive into Software Development: - -**What you'd actually do day-to-day:** -- Write code to solve problems (you mentioned enjoying analytical work) -- Collaborate with teams on projects (2-3 meetings/week, mostly async) -- Learn new tools and technologies (constant growth) - -**Career path:** -Junior Developer (1-2 years) → Mid-level Developer → Senior Developer → Tech Lead -Typical timeline: 5-7 years to senior level with KES 200K+ monthly - -**Why this matches your preferences:** -- High autonomy (you ranked this 0.85) -- Remote work common (your work_environment_importance: 0.78) -- Strong job security in growing field -- Financial: KES 80K-150K aligns with your expectations - -**What concerns do you have about this path?** -``` - -**Transitions:** -- User expresses concern → ADDRESS_CONCERNS -- User asks about tradeoffs → DISCUSS_TRADEOFFS -- User says "this sounds good" → ACTION_PLANNING -- User wants other options → back to PRESENT_RECOMMENDATIONS - ---- - -#### Phase 4: ADDRESS_CONCERNS -Handle resistance using the framework from your brainstorming document. - -I'm classifying resistance into three types: - -```python -class ResistanceType(str, Enum): - BELIEF_BASED = "belief" # "I don't think I could succeed" - SALIENCE_BASED = "salience" # "It doesn't feel like real work" - EFFORT_BASED = "effort" # "Applications are exhausting" -``` - -Some things that might come up here that the agent might want to address: ---Explain why higher effort now can pay off in the long-run, and that it's normal that (a) many applications lead to rejections and (b) a first job is never perfect but just a stepping stone for learning and career growth ---Why retention and persisting in a job even when it's first hard is important ---How/why the user might enjoy the recommended occupations more than they think (i.e. discuss tradeoffs; like "You may not love manufacturing, but the stability matters right now”) - -**Response strategies:** - -| Resistance Type | User Says | My Agent's Response | -|-----------------|-----------|---------------------| -| **Belief-based** | "I don't have the skills" | "Many people start here with similar backgrounds. The essential skills are X, Y, Z - you already have Y and Z. We can help you build X through training." | -| **Belief-based** | "There are no jobs" | "Actually, this field has **high demand** in Kenya - [show DB2 data]. Companies are actively hiring." | -| **Salience-based** | "My family won't respect this" | "Many families initially question non-traditional careers. What changes minds is stable income. In 2 years, you might earn KES X, supporting your family well." | -| **Effort-based** | "I'll get rejected anyway" | "Rejections are normal - most people apply to 10-15 jobs before an offer. It's part of the process, not your worth. Persistence matters." | - -**Hard railguards** (so that it stays persuasion, not manipulation): - -For example -``` -For instance we dont want the model to say stuff like: - -"You will enjoy this" -"This fits who you are" -"You're perfect for this" - -Better alternatives: - -"Many people discover they enjoy X after trying it" -"This path keeps future options open" -"Your skills align well - you'd have a strong foundation" -``` - ---- - -#### Phase 5: DISCUSS_TRADEOFFS -When user prefers low-demand option but we have high-demand option recommended. -Highlights upsides of the recommendations conditional on existing preferences - -**Example:** -``` -I hear that [Option A] appeals more because of [their preference]. -That's valid - it matches your values. - -Here's the tradeoff: -- Option A: Lower demand (harder to find openings), but perfect preference match -- Option B: High demand (easier to get hired), but requires accepting [tradeoff] - -**Option B as a stepping stone:** -Many people start with B (stable income, build experience) -then transition to A after 2-3 years with more leverage. - -Would you be open to that path? Or is A important enough to pursue directly? -``` - ---- - -#### Phase 6: SKILLS_UPGRADE_PIVOT -**Trigger:** User rejects ALL occupation recommendations (rejected_occupations >= 3) - -Since you're already providing training recommendations in your Node2Vec output, I'll present those here as an alternative path. - -**Example:** -``` -I understand none of those career paths felt right. That's completely okay - let's approach this differently. - -Looking at the fields you're interested in, here are skill-building opportunities that could open up new options: - -**Training Recommendations:** - -1. **Advanced Econometrics** (Coursera, 40 hours) - - Would strengthen your analysis skills for research roles - - Opens doors to: Data Analyst, Research Assistant, M&E roles - - You already have the foundation - this builds on it - -2. **Data Visualization with ggplot2** (DataCamp, 12 hours) - - Quick skill boost that's highly valued - - Helps you stand out in applications - - Practical, hands-on approach - -Would building these skills make you feel more confident about career options? Or is there something else holding you back? -``` - -**What I'm doing here:** -- Present the `skillstraining_recommendations` from your Node2Vec output -- Frame training as a path to opportunities (not just skill-building for its own sake) -- Connect each training to specific occupations it unlocks -- Keep the door open for deeper conversation about barriers - -**Transitions:** -- User interested in training → ACTION_PLANNING (enrollment as action) -- User still resistant → FOLLOW_UP (explore deeper blockers - maybe not about skills at all) - ---- - -#### Phase 7: ACTION_PLANNING -Convert interest into concrete next steps. The action varies by what they're interested in: -Help people build agency and get into action (e.g. if user says that all of their peers prefer to stay idle); The primary goal is motivate people to get going and choose a career or an occupation that makes sense --> the recommendations are well-informed so the agent should really motivate them to take those seriously and take action in direction of those (e.g. "This is hard, but worth pushing through") - ---> In addition to potentially showing real opportunities, this agent should be motivational and nudge the user to OFFLINE action. E.g. motivate user to talk to mentor about recommendations, to apply to a job related to recommendations, to look up more online related to recommendations etc. - -**For occupation interest:** -``` -Great! So Data Analyst sounds like a good fit. Let's talk next steps. - -I have actual job openings for this role - would you like to see them? -Or would you prefer to build [missing skill] first through training? -``` - -**For opportunity interest (actual job posting):** -``` -Perfect! The "Internship at XYZ Foundation" looks promising. - -Here's the application link: [URL] - -What feels realistic for you? -A) Apply this week (I can help with your CV if needed) -B) Learn more about the role first (research the organization) -C) Something is holding you back - let's talk about it -``` - -**For training interest:** -``` -Great choice! "Advanced Econometrics" will definitely strengthen your profile. - -Here's how to enroll: [provider link] - -When will you start? -A) This week (get the momentum going) -B) Next month (after [current commitment]) -C) I'm not sure yet - what's stopping you? -``` - -**What I'm logging:** -```python -class ActionCommitment(BaseModel): - recommendation_id: str # Could be occupation, opportunity, or training ID - recommendation_type: str # "occupation" / "opportunity" / "training" - action_type: str # "apply_to_job" / "enroll_in_training" / "explore_occupation" - commitment_level: str # "will_do_this_week" / "will_do_this_month" / "interested" / "maybe_later" - barriers_mentioned: list[str] = [] - specific_opportunity: Optional[str] = None # Job posting ID if they picked one - timestamp: datetime -``` - -**Success metric:** -- `commitment_level == "will_do_this_week"` AND `action_type in ["apply_to_job", "enroll_in_training"]` -- This measures actual intent to act, not just interest - ---- - -#### Phase 8: WRAPUP -Summarize, save to DB6, confirm plan. - -**Example:** -``` -Perfect! Here's what we've discussed: - -**Your top career match:** [Occupation X] -- Aligns with: [preferences that matched] -- Next step: [action they committed to] -- Timeline: [when they'll do it] - -I've saved this to your profile so we can follow up. - -Before we wrap - what's the ONE thing that might stop you from this step? -Let's talk through it now so you're prepared. -``` - -**Save to DB6:** -- Recommendations shown -- User reactions (interested/rejected per occupation) -- Concerns raised and addressed -- Final action commitment - ---- - -## State Management - -Here's what I'm tracking throughout the conversation: - -```python -class RecommenderAdvisorAgentState(BaseModel): - """State for my agent""" - - session_id: int - conversation_phase: ConversationPhase = ConversationPhase.INTRO - conversation_turn_count: int = 0 - - # Input data - youth_id: str - skills_vector: Optional[dict] = None # From Epic 4 - preference_vector: Optional[PreferenceVector] = None # From Epic 2 - bws_occupation_scores: Optional[dict[str, float]] = None # From Epic 2 - - # Your Node2Vec recommendations (all three types) - recommendations: Optional[Node2VecRecommendations] = None - - # What we've presented to user - presented_occupations: list[str] = [] # Occupation IDs shown - presented_opportunities: list[str] = [] # Job posting IDs shown - presented_trainings: list[str] = [] # Training IDs shown - - # User engagement tracking (tracks ALL three types) - user_interest_signals: dict[str, str] = {} - # {"occ_001": "interested", "opp_002": "rejected", "skill_001": "committed"} - # Values: "interested" / "exploring" / "neutral" / "rejected" / "committed" - - # Rejection tracking - rejected_occupations: int = 0 - rejected_opportunities: int = 0 - rejected_trainings: int = 0 - - # Current focus - current_recommendation_type: str = "occupation" # "occupation" / "opportunity" / "training" - current_focus_id: Optional[str] = None # ID of current item being discussed - explored_items: list[str] = [] # All IDs explored (any type) - - # Resistance & concerns - concerns_raised: list[dict] = [] - # [{"item_id": "occ_001", "concern": "...", "resistance_type": "belief"}] - - addressed_concerns: list[str] = [] - - # Skills upgrade pivot - pivoted_to_training: bool = False - - # Action commitment - action_commitment: Optional[ActionCommitment] = None - - # Labor demand context (I'll load this from config file) - labor_demand_data: dict[str, dict] = {} - - # Conversation log (for DB6) - conversation_log: list[dict] = [] -``` - ---- - -## Architecture Overview - -Here's how I'm organizing the code: - -``` -backend/app/agent/recommender_advisor_agent/ -├── agent.py # Main conversational agent -├── state.py # RecommenderAdvisorAgentState -├── types.py # Data models (OccupationRecommendation, etc.) -├── recommendation_interface.py # Interface to your Node2Vec -├── motivation_strategies.py # Resistance handling patterns -└── config/ - └── labor_demand_context.json # Labor demand data for 40 occupations -``` - -**Dependencies on Epic 1** (VERY IMPORTANT): -- DB1: Occupation taxonomy (descriptions, required skills, career paths) -- DB2: Labor demand data per occupation -- DB4: Training opportunities -- DB6: Youth profiles (to fetch skills + preferences, save commitments) - ---- - -## Success Metrics (What Gets Logged) - -In DB6, I'll save this to the youth profile (not final btw): - -```python -{ - "recommender_session": { - "session_id": 123, - "completed_at": "2026-01-04T...", - - # All recommendations shown (three types) - "recommendations_presented": { - "occupations": ["occ_001", "occ_002"], - "opportunities": ["opp_001", "opp_002", "opp_003"], - "trainings": ["skill_001", "skill_002"] - }, - - # User reactions per type - "user_engagement": { - "occupations_explored": ["occ_001"], # Deep-dived on Data Analyst - "occupations_rejected": ["occ_002"], # Rejected M&E Specialist - "opportunities_explored": ["opp_001"], # Clicked on XYZ Foundation internship - "opportunities_rejected": [], - "trainings_presented": ["skill_001", "skill_002"], - "trainings_interested": [] - }, - - # Concerns & resistance - "concerns_raised": [ - {"item_id": "occ_001", "concern": "worried about technical difficulty", "type": "belief"}, - {"item_id": "opp_001", "concern": "internship pay too low", "type": "financial"} - ], - "concerns_addressed": 2, - - # Final outcome - "action_commitment": { - "recommendation_id": "opp_001", - "recommendation_type": "opportunity", # Applied to actual job posting - "action_type": "apply_to_job", - "commitment_level": "will_do_this_week", - "specific_opportunity": "Internship at XYZ Foundation", - "committed_at": "2026-01-04T..." - }, - - # Engagement quality - "turns_count": 15, - "pivoted_to_training": False, - "recommendation_flow": ["occupation", "opportunity", "action"] # Path taken - } -} -``` - -**Key success metric:** -- `commitment_level == "will_do_this_week"` OR `"will_do_this_month"` -- AND `action_type in ["apply_to_job", "enroll_in_training"]` - -This measures actual intent to act (job application or training enrollment), not just interest or exploration. - ---- - -## What I Need From You - -1. **Review the Node2Vec output schema** - does `OccupationRecommendation` and `Node2VecRecommendations` work for you? Any changes needed? - -2. **Confirm the input** - is the `PreferenceVector` structure what you need? Do you need anything additional? - -3. **How many recommendations** will your algorithm return (top-K)? I'm planning for 5-10 but can adjust. --> this sounds right! - -4. **Score components** - can you provide the breakdown (`skills_match`, `preference_match`, `labor_demand`, `graph_proximity`) or should I just work with a single `total_score`? --> I (Jasmin) can provide a breakdown! How would that change the output I need to create and your classes? - -5. **Labor demand data** - will this come from your algorithm or should I pull it separately from DB2 (Epic 1 contractor)? --> I (Jasmin) can put it into the recommendations! - -6. **Timeline coordination** - when do you expect to have the Node2Vec algorithm ready? I can build the agent with mock data initially and plug yours in when ready. --> I (Jasmin) will work on it this weekend, but it will surely still need tweaking along the way. - -7. **Any other thoughts** on the conversation flow or phases? Does this align with what you were envisioning? --> See comments above. Broadly really really good and aligned! I just added some nuances, and I think the career path and skills_upgrade_pivot are still less clear. But let's backlog this for now, build out the rest and then tackle it. - ---- - -## My Next Steps - -Once we align on the data contracts, I'll: - -1. Define the types (`types.py` with recommendation models) -2. Build the state management (`state.py`) -3. Create a stub interface to your Node2Vec (so I can develop in parallel) -4. Start implementing the conversation phases -5. Coordinate for DB1, DB2, DB4, DB6 access --> DB1 does already exist, so please coordinate with Anselme how we would connect to it - -I know we're behind schedule (it's Jan 4 already), so I want to move quickly once we agree on the structure. - -**Let me know your thoughts!** - ---- - -**Wilfred** -January 4, 2026 diff --git a/BRANCH_SYNC_STRATEGY.md b/BRANCH_SYNC_STRATEGY.md deleted file mode 100644 index 16a9b0cf..00000000 --- a/BRANCH_SYNC_STRATEGY.md +++ /dev/null @@ -1,193 +0,0 @@ -# Branch Sync & PR Feedback Strategy - -## Current Situation - -**Branch Structure:** -``` -main - └── feat/epic2-adaptive-preference-elicitation (Epic 2 - PR under review) - └── feat/epic3-recommender-agent (Epic 3 - current branch) -``` - -**Key Point:** Epic 3 branched from Epic 2 at commit `16efc527`, so Epic 3 contains all Epic 2 changes plus additional Epic 3 work. - ---- - -## Strategy Overview - -**Two-Phase Approach:** - -1. **Phase 1:** Fix Epic 2 PR feedback on `feat/epic2-adaptive-preference-elicitation` branch -2. **Phase 2:** Rebase Epic 3 onto the cleaned Epic 2 branch - -This keeps commit history clean and ensures Epic 3 gets all Epic 2 improvements. - ---- - -## Detailed Implementation Plan - -### **Phase 1: Address Epic 2 PR Feedback** - -**Step 1.1 - Switch to Epic 2 branch:** -```bash -git checkout feat/epic2-adaptive-preference-elicitation -``` - -**Step 1.2 - Create fixup commits for each concern:** - -The PR feedback involves: -1. Clarify/remove `offline_output/` folder -2. Condense README -3. Remove `agent.py.backup` file -4. Rename folders (like `epic1/`) -5. Replace epic references in comments with clear logic explanations -6. Reorganize test files - -**Approach:** Create focused commits for each category: -```bash -# Commit 1: Remove unnecessary artifacts -git commit -m "chore(epic2): remove backup files and clarify offline_output usage" - -# Commit 2: Improve documentation -git commit -m "docs(epic2): condense README for clarity" - -# Commit 3: Improve folder structure -git commit -m "refactor(epic2): rename folders for better clarity" - -# Commit 4: Improve code comments -git commit -m "docs(epic2): replace epic references with descriptive comments" - -# Commit 5: Reorganize tests -git commit -m "refactor(epic2): reorganize test files per project standards" -``` - -**Step 1.3 - Push Epic 2 changes:** -```bash -git push fork feat/epic2-adaptive-preference-elicitation -``` - ---- - -### **Phase 2: Sync Epic 3 with Updated Epic 2** - -**Step 2.1 - Switch to Epic 3 branch:** -```bash -git checkout feat/epic3-recommender-agent -``` - -**Step 2.2 - Rebase onto updated Epic 2:** -```bash -# Rebase Epic 3 onto the updated Epic 2 branch -git rebase feat/epic2-adaptive-preference-elicitation -``` - -**Step 2.3 - Resolve any conflicts:** -- If conflicts arise (unlikely since Epic 2 changes are refactoring), resolve them -- Continue rebase: `git rebase --continue` - -**Step 2.4 - Force push Epic 3 (rewrites history):** -```bash -# Force push since rebase rewrites history -git push fork feat/epic3-recommender-agent --force-with-lease -``` - ---- - -## Benefits of This Strategy - -1. **Clean Commit History:** Each PR feedback item gets its own commit -2. **No Merge Commits:** Rebase keeps history linear -3. **Epic 3 Automatically Gets Fixes:** Rebase propagates all Epic 2 improvements to Epic 3 -4. **Easy to Review:** Focused commits make code review straightforward -5. **Safe Force Push:** `--force-with-lease` prevents overwriting others' work - ---- - -## Alternative: Interactive Rebase (If Preferred) - -If you want to squash the fixup commits into the original Epic 2 commits for a cleaner history: - -```bash -# On Epic 2 branch -git rebase -i - -# Mark fixup commits as 'fixup' or 'squash' -# This rewrites history more cleanly but is more complex -``` - -**Note:** This approach requires force-pushing both branches and careful coordination. - ---- - -## Risk Mitigation - -**Before Starting:** -```bash -# Create backup branches -git branch backup-epic2 feat/epic2-adaptive-preference-elicitation -git branch backup-epic3 feat/epic3-recommender-agent -``` - -**If Something Goes Wrong:** -```bash -# Restore from backup -git reset --hard backup-epic3 -``` - ---- - -## Recommended Workflow - -**Recommendation: Use Phase 1 & 2 approach (focused commits + rebase)** because: - -- ✅ Preserves clear audit trail of PR feedback responses -- ✅ Simpler than interactive rebase -- ✅ Easy to understand what changed and why -- ✅ Automatically syncs Epic 3 - ---- - -## Next Steps - -1. Create backup branches for safety -2. Switch to Epic 2 branch -3. Address each PR feedback item with focused commits -4. Push Epic 2 updates -5. Rebase Epic 3 onto updated Epic 2 -6. Force push Epic 3 with `--force-with-lease` - ---- - -## PR Feedback Items to Address - -### 1. offline_output/ folder -- **Issue:** Contains large files that may be generated artifacts -- **Action:** Clarify purpose or add to .gitignore if they're build artifacts - -### 2. README condensation -- **Issue:** Too long and detailed, may contain technical frustration -- **Action:** Condense to core usage and setup, move detailed docs elsewhere - -### 3. agent.py.backup file -- **Issue:** Backup files unnecessary with Git version control -- **Action:** Remove backup file - -### 4. Folder naming (epic1/, etc.) -- **Issue:** Names like "epic1" don't communicate purpose -- **Action:** Rename to descriptive names (e.g., "mock_dependencies", "test_data") - -### 5. Epic references in comments -- **Issue:** Comments reference "epic(1,2)" instead of explaining logic -- **Action:** Replace with clear explanations of actual functionality - -### 6. Test organization -- **Issue:** Tests need proper isolation -- **Action:** - - Unit tests: Keep with code they test - - Evaluation tests: Move to `/backend/evaluation_tests` - ---- - -**Last Updated:** 2026-01-26 -**Created By:** Claude Code -**Purpose:** Guide for syncing Epic 2 and Epic 3 branches while addressing PR feedback diff --git a/README.md b/README.md index d70623fc..3f38c153 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,9 @@ logo -**Compass** is an AI chatbot designed to assist job-seekers in exploring and discovering their skills. +**Compass Connect** is an AI chatbot designed to assist job-seekers in exploring and discovering their skills. -Compass has two objectives: +Compass Connect has two objectives: - Populate a digital skill wallet that captures the experience of job-seekers, including the skills inherent in their formal and informal work. - By doing so, guide job-seekers in exploring and discovering @@ -28,20 +28,20 @@ their skills and interests based on their lived experiences. ## Architecture Overview -The images below show a high level overview of the solution architecture of Compass. +The images below show a high level overview of the solution architecture of Compass Connect. -![Compass Architecture Overview](https://lucid.app/publicSegments/view/a1c21a2d-162b-4eae-b70a-f57b44181aa1/image.png) +![Compass Connect Architecture Overview](https://lucid.app/publicSegments/view/a1c21a2d-162b-4eae-b70a-f57b44181aa1/image.png) -![Compass Cloud Architecture](https://lucid.app/publicSegments/view/fb45c622-0edb-4257-84a3-6fc4996692f3/image.png) +![Compass Connect Cloud Architecture](https://lucid.app/publicSegments/view/fb45c622-0edb-4257-84a3-6fc4996692f3/image.png) -![Compass Backend Architecture](https://lucid.app/publicSegments/view/3e5323ab-786b-4d46-bb73-dd716cdd9ee0/image.png) +![Compass Connect Backend Architecture](https://lucid.app/publicSegments/view/3e5323ab-786b-4d46-bb73-dd716cdd9ee0/image.png) -![Compass AI Architecture](https://lucid.app/publicSegments/view/db45dc1c-b89e-4ae4-bd32-347092693844/image.png) +![Compass Connect AI Architecture](https://lucid.app/publicSegments/view/db45dc1c-b89e-4ae4-bd32-347092693844/image.png) ## Why Contribute? -- **Make an Impact:** Your contributions will directly improve the user experience and functionality of Compass, helping job-seekers explore and discover their skills more effectively. -- **Help Achieve Our Goals:** By contributing, you are supporting Compass’s mission to populate digital skill wallets and guide job-seekers in understanding their skills and interests based on their lived experiences. +- **Make an Impact:** Your contributions will directly improve the user experience and functionality of Compass Connect, helping job-seekers explore and discover their skills more effectively. +- **Help Achieve Our Goals:** By contributing, you are supporting Compass Connect’s mission to populate digital skill wallets and guide job-seekers in understanding their skills and interests based on their lived experiences. ## Ways to Contribute @@ -49,12 +49,12 @@ The images below show a high level overview of the solution architecture of Comp 2. **Code Contributions:** Help enhance the codebase by submitting pull requests. 3. **Write or Improve Tests:** We aim for 100% code coverage. You can help achieve this goal by writing or improving tests. 4. **Documentation:** Improve project documentation by submitting pull requests. Clear documentation is crucial for new contributors. -5. **Add a New Language**: Help make Compass accessible to more users by adding support for a new language. Follow the guidelines in [add-a-new-language.md](./add-a-new-language.md). +5. **Add a New Language**: Help make Compass Connect accessible to more users by adding support for a new language. Follow the guidelines in [add-a-new-language.md](./add-a-new-language.md). 6. **Support:** Give the project a star on GitHub—your support encourages us to keep improving! ## Contribution Guidelines -🎉 Thank you for considering contributing to Tabiya Compass! 🎉 +🎉 Thank you for considering contributing to Tabiya Compass Connect! 🎉 ### Code Formatting diff --git a/add-a-new-language.md b/add-a-new-language.md index 8310b44a..f75b996c 100644 --- a/add-a-new-language.md +++ b/add-a-new-language.md @@ -1,4 +1,4 @@ -# Compass Language Guide +# Compass Connect Language Guide This guide is divided into two main parts: @@ -134,7 +134,7 @@ Supported languages are enabled via environment variables on both backend and fr ## Switching or Using Supported Languages -This section explains how to enable or switch between languages that are already supported in Compass. +This section explains how to enable or switch between languages that are already supported in Compass Connect. ### Overview diff --git a/admin-frontend/README.md b/admin-frontend/README.md index 515c42bf..916a3976 100644 --- a/admin-frontend/README.md +++ b/admin-frontend/README.md @@ -1,6 +1,6 @@ -# Compass Admin Frontend +# Compass Connect Admin Frontend -The Compass Admin Portal is a web application for managing and configuring the Compass platform. It provides administrators with tools to manage users, view analytics, and configure system settings. +The Compass Connect Admin Portal is a web application for managing and configuring the Compass Connect platform. It provides administrators with tools to manage users, view analytics, and configure system settings. ## Prerequisites diff --git a/admin-registration.md b/admin-registration.md index d196c901..ab3dcc24 100644 --- a/admin-registration.md +++ b/admin-registration.md @@ -34,7 +34,7 @@ The dashboard self-creation flow requires an existing super admin to approve / p ```bash cd backend poetry run python scripts/admin/create_admin_user.py \ - --name "Compass Super Admin" \ + --name "Compass Connect Super Admin" \ --email "superadmin@.example.com" \ --role super_admin \ --tenant-id "$ADMIN_FIREBASE_TENANT_ID" \ diff --git a/backend/EVALUATION_TESTS_README.md b/backend/EVALUATION_TESTS_README.md index ddd8ae0c..3a6843b0 100644 --- a/backend/EVALUATION_TESTS_README.md +++ b/backend/EVALUATION_TESTS_README.md @@ -1,7 +1,7 @@ # Evaluation Tests Tests used for evaluating the correctness and performance of the -compass app and individual agents. +compass-connect app and individual agents. ## Marking a test as `evaluation_test` diff --git a/backend/README.md b/backend/README.md index b7cb9edd..018420c7 100644 --- a/backend/README.md +++ b/backend/README.md @@ -1,6 +1,6 @@ -# Compass Backend +# Compass Connect Backend -FastAPI application serving the Compass career guidance API with AI-powered conversational agents. +FastAPI application serving the Compass Connect career guidance API with AI-powered conversational agents. ## Prerequisites @@ -53,7 +53,7 @@ poetry run python app/server.py ## Running the code locally -The backend is a FastAPI app that serves the Compass API. +The backend is a FastAPI app that serves the Compass Connect API. When running the code locally, the backend will use the credentials and the project set in the Google Cloud SDK. @@ -149,7 +149,7 @@ The backend uses the following environment variables: - `GOOGLE_APPLICATION_CREDENTIALS`: The path to the service account key file. - `TAXONOMY_MONGODB_URI`: The URI of the MongoDB Atlas instance where the ESCO taxonomy data is stored. - `TAXONOMY_DATABASE_NAME`: The name of mongo db database where the ESCO taxonomy data with the embeddings is stored. -- `TAXONOMY_MODEL_ID`: The model ID of the ESCO model in the compass taxonomy database. +- `TAXONOMY_MODEL_ID`: The model ID of the ESCO model in the compass-connect taxonomy database. - `APPLICATION_MONGODB_URI`: The URI of the MongoDB Atlas instance for the application database. - `APPLICATION_DATABASE_NAME`: The name of mongo db database used by the application to store data. - `USERDATA_MONGODB_URI`: The URI of the MongoDB instance for the user data database. diff --git a/backend/app/README.md b/backend/app/README.md index 6151e66c..7a729db6 100644 --- a/backend/app/README.md +++ b/backend/app/README.md @@ -1,6 +1,6 @@ # Agent-based design -The Compass conversational agent under the hood is designed as a set of agents interacting together. Agents may have +The Compass Connect conversational agent under the hood is designed as a set of agents interacting together. Agents may have "sub-agents" if they are complex enough. Agents are coordinated by the **AgentDirector**. diff --git a/backend/app/agent/preference_elicitation_agent/README.md b/backend/app/agent/preference_elicitation_agent/README.md index 07ac1733..043ceeaa 100644 --- a/backend/app/agent/preference_elicitation_agent/README.md +++ b/backend/app/agent/preference_elicitation_agent/README.md @@ -2,7 +2,7 @@ **Epic 2 - Deliverable D1, D2, D3** -This package contains the implementation of the Preference Elicitation Agent for the Compass career guidance system. +This package contains the implementation of the Preference Elicitation Agent for the Compass Connect career guidance system. ## Overview @@ -146,7 +146,7 @@ COMPLETE ### Vignettes Configuration -Vignettes are stored in `/compass/backend/app/config/vignettes.json`: +Vignettes are stored in `/compass-connect/backend/app/config/vignettes.json`: ```json [ @@ -176,7 +176,7 @@ Vignettes are stored in `/compass/backend/app/config/vignettes.json`: ### Vignette Templates Configuration -Vignette templates are stored in `/compass/backend/app/config/vignette_templates.json`: +Vignette templates are stored in `/compass-connect/backend/app/config/vignette_templates.json`: ```json [ @@ -281,7 +281,7 @@ print(output.message_for_user) Run tests: ```bash -cd compass/backend +cd compass-connect/backend poetry run pytest app/agent/preference_elicitation_agent/test_preference_elicitation_agent.py -v ``` @@ -356,7 +356,7 @@ state = PreferenceElicitationAgentState( ) ``` -**Full Compass Flow**: +**Full Compass Connect Flow**: ```python # Copy from prior agent for consistency explored = app_state.explore_experiences_director_state.explored_experiences @@ -436,7 +436,7 @@ class PreferenceElicitationAgentState: initial_experiences_snapshot: Optional[list[ExperienceEntity]] = None """ Immutable snapshot of experiences at agent start. - Sources: CV upload, prior Compass session, or DB6 + Sources: CV upload, prior Compass Connect session, or DB6 """ ``` diff --git a/backend/app/agent/preference_elicitation_agent/offline_optimization/OFFLINE_SYSTEM_DOCUMENTATION.md b/backend/app/agent/preference_elicitation_agent/offline_optimization/OFFLINE_SYSTEM_DOCUMENTATION.md index 1f0db44b..85f7bd93 100644 --- a/backend/app/agent/preference_elicitation_agent/offline_optimization/OFFLINE_SYSTEM_DOCUMENTATION.md +++ b/backend/app/agent/preference_elicitation_agent/offline_optimization/OFFLINE_SYSTEM_DOCUMENTATION.md @@ -25,7 +25,7 @@ ```bash # Navigate to backend directory -cd compass/backend +cd compass-connect/backend # Install dependencies poetry install --sync @@ -1627,7 +1627,7 @@ export ADAPTIVE_FIM_THRESHOLD=25.0 **Solution**: ```bash -cd compass/backend +cd compass-connect/backend poetry install --sync ``` diff --git a/backend/app/agent/recommender_advisor_agent/README.md b/backend/app/agent/recommender_advisor_agent/README.md index 7a9e5a44..273f00db 100644 --- a/backend/app/agent/recommender_advisor_agent/README.md +++ b/backend/app/agent/recommender_advisor_agent/README.md @@ -13,7 +13,7 @@ The agent operates through distinct phases managed by specialized handlers: - **Action Planning**: Secures commitment and defines concrete next steps. ## Testing -We have a comprehensive suite including unit, evaluation (scripted), and end-to-end tests. Run these commands from the `compass/backend` directory. +We have a comprehensive suite including unit, evaluation (scripted), and end-to-end tests. Run these commands from the `compass-connect/backend` directory. ### 1. Unit Tests Tests internal routing, initialization, and error handling. diff --git a/backend/docs/TEST_USER_PROFILE.md b/backend/docs/TEST_USER_PROFILE.md deleted file mode 100644 index ca95e872..00000000 --- a/backend/docs/TEST_USER_PROFILE.md +++ /dev/null @@ -1,356 +0,0 @@ -# Test User Profile: Hassan (Mombasa Persona) - -This document describes the sample user profile used for testing the **Recommender/Advisor Agent** in the interactive test script (`test_recommender_agent_interactive.py`). - ---- - -## JSON Schema - -> **IMPORTANT**: The test script uses a **simplified stub** for `skills_vector`. The actual Skills Explorer Agent produces a richer `SkillEntity` structure. See comparison below. - ---- - -### Test Script Stub (Simplified) - -This is what `test_recommender_agent_interactive.py` currently uses: - -```json -{ - "youth_id": "test_user_123", - "country_of_user": "KENYA", - - "skills_vector": { - "top_skills": [ - {"preferredLabel": "Basic Electrical Wiring", "proficiency": 0.6}, - {"preferredLabel": "Manual Handling / Physical Labor", "proficiency": 0.8}, - {"preferredLabel": "M-Pesa / Mobile Money", "proficiency": 0.85} - ] - }, - - "preference_vector": { - "financial_importance": 0.85, - "work_environment_importance": 0.55, - "career_advancement_importance": 0.60, - "work_life_balance_importance": 0.80, - "job_security_importance": 0.70, - "task_preference_importance": 0.65, - "social_impact_importance": 0.40, - - "confidence_score": 0.0, - "n_vignettes_completed": 0, - "per_dimension_uncertainty": {}, - - "posterior_mean": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], - "posterior_covariance_diagonal": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], - "fim_determinant": null, - - "decision_patterns": {}, - "tradeoff_willingness": {}, - "values_signals": {}, - "consistency_indicators": {}, - "extracted_constraints": {} - } -} -``` - ---- - -### Actual SkillEntity Schema (from Skills Explorer Agent) - -The actual `SkillEntity` produced by the Skills Explorer Agent (`app/vector_search/esco_entities.py`) has this structure: - -```json -{ - "id": "string", - "modelId": "string", - "UUID": "string", - "preferredLabel": "string", - "altLabels": ["string"], - "description": "string", - "scopeNote": "string (optional)", - "originUUID": "string (optional)", - "UUIDHistory": ["string"], - "score": 0.0, - "skillType": "skill/competence | knowledge | language | attitude | ''" -} -``` - -### Actual ExperienceEntity.top_skills (Full Production Shape) - -From `app/agent/experience/experience_entity.py`, the `ExperienceEntity` contains: - -```json -{ - "uuid": "unique-experience-id", - "experience_title": "Crew Member", - "company": "McDonald's", - "location": "Cape Town, South Africa", - "timeline": { - "start_date": "2023-01", - "end_date": "2024-06" - }, - "work_type": "WAGED_EMPLOYEE", - - "top_skills": [ - { - "id": "skill-db-id", - "modelId": "taxonomy-model-id", - "UUID": "esco-skill-uuid", - "preferredLabel": "customer service", - "altLabels": ["client service", "customer care"], - "description": "Maintain a high standard of customer service...", - "scopeNote": "", - "originUUID": "original-esco-uuid", - "UUIDHistory": [], - "score": 0.92, - "skillType": "skill/competence" - } - ], - - "remaining_skills": [], - "responsibilities": { - "responsibilities": ["Taking orders", "Handling cash"], - "non_responsibilities": [], - "other_peoples_responsibilities": [] - }, - "esco_occupations": [], - "questions_and_answers": [], - "summary": "Experience summary text" -} -``` - ---- - -### ⚠️ Schema Mismatch Analysis - -| Field | Test Stub | Actual SkillEntity | Notes | -|-------|-----------|-------------------|-------| -| `preferredLabel` | Present | Present | **Same** | -| `proficiency` | Present | Missing | Test-only field | -| `id` | Missing | Present | DB identifier | -| `UUID` | Missing | Present | ESCO UUID | -| `modelId` | Missing | Present | Taxonomy model | -| `altLabels` | Missing | Present | Alternative names | -| `description` | Missing | Present | Skill description | -| `score` | Missing | Present | Match score (0-1) | -| `skillType` | Missing | Present | skill/knowledge/etc | - -### How the Recommender Agent Handles This - -The `BasePhaseHandler._extract_skills_list()` method handles **both formats**: - -```python -def _extract_skills_list(self, state: RecommenderAdvisorAgentState) -> list[str]: - if isinstance(state.skills_vector, dict): - if "skills" in state.skills_vector: - return state.skills_vector["skills"] - elif "top_skills" in state.skills_vector: - skills = state.skills_vector.get("top_skills", []) - # Extract skill names from either format - return [s.get("preferredLabel", s.get("name", str(s))) - if isinstance(s, dict) else str(s) for s in skills] -``` - -This means the agent works with: -- `{"top_skills": [{"preferredLabel": "..."}]}` (test stub) -- `{"top_skills": [SkillEntity(...)]}` (production) -- `{"skills": ["skill1", "skill2"]}` (simple list) - ---- - -## User Identity - -| Field | Value | -|----------------|-----------------------| -| **Youth ID** | `test_user_123` | -| **Name** | Hassan | -| **Age** | 24 | -| **Location** | Mombasa, Kenya | -| **Country** | `Country.KENYA` | -| **Education** | Completed Form 4, some technical college | - ---- - -## Persona Background - -Hassan is a 24-year-old from Mombasa with the following characteristics: - -- **Education**: Completed Form 4 (secondary school), with some technical college coursework -- **Work Experience**: - - Has worked casual jobs at the port - - Helped his uncle with electrical repairs -- **Strengths**: - - Good with hands (manual/technical skills) - - Basic phone and mobile money (M-Pesa) skills -- **Motivations**: - - Wants stable income but values flexibility - - Family expects him to contribute financially - ---- - -## Skills Vector - -The skills vector represents Hassan's current skill proficiencies (0.0 - 1.0 scale): - -```python -skills_vector = { - "top_skills": [ - {"preferredLabel": "Basic Electrical Wiring", "proficiency": 0.6}, - {"preferredLabel": "Manual Handling / Physical Labor", "proficiency": 0.8}, - {"preferredLabel": "M-Pesa / Mobile Money", "proficiency": 0.85}, - {"preferredLabel": "Customer Service", "proficiency": 0.65}, - {"preferredLabel": "Tool Usage (hand tools)", "proficiency": 0.7}, - {"preferredLabel": "Motorcycle Riding", "proficiency": 0.5}, - {"preferredLabel": "Basic Math / Pricing", "proficiency": 0.7} - ] -} -``` - -### Skills Summary Table - -| Skill | Proficiency | Level | -|--------------------------------|-------------|--------------| -| M-Pesa / Mobile Money | 0.85 | **High** | -| Manual Handling / Physical Labor | 0.80 | **High** | -| Tool Usage (hand tools) | 0.70 | Moderate | -| Basic Math / Pricing | 0.70 | Moderate | -| Customer Service | 0.65 | Moderate | -| Basic Electrical Wiring | 0.60 | Moderate | -| Motorcycle Riding | 0.50 | Low-Moderate | - ---- - -## 💡 Preference Vector - -The preference vector represents Hassan's relative job/career attribute priorities, learned via Bayesian preference elicitation. - -All importance scores are on a **[0, 1] scale**: -- **0.0 - 0.3**: Low importance -- **0.4 - 0.6**: Moderate importance -- **0.7 - 1.0**: High importance - -```python -from app.agent.preference_elicitation_agent.types import PreferenceVector - -preference_vector = PreferenceVector( - financial_importance=0.85, # High - needs to support family - work_environment_importance=0.55, # Moderate - okay with physical work outdoors - career_advancement_importance=0.60, # Moderate - interested but not primary focus - work_life_balance_importance=0.80, # High - values flexibility - job_security_importance=0.70, # Moderate-high - wants stable income - task_preference_importance=0.65, # Moderate - prefers hands-on work - social_impact_importance=0.40 # Lower - practical/financial focus first -) -``` - -### Preference Dimensions Table - -| Dimension | Importance | Interpretation | Rationale | -|-------------------------|------------|----------------------|----------------------------------------------| -| Financial Compensation | **0.85** | **HIGH** | Needs to support family | -| Work-Life Balance | **0.80** | **HIGH** | Values flexibility | -| Job Security | **0.70** | **MODERATE-HIGH** | Wants stable income | -| Task Preferences | 0.65 | MODERATE | Prefers hands-on work | -| Career Advancement | 0.60 | MODERATE | Interested but not primary focus | -| Work Environment | 0.55 | MODERATE | Okay with physical work outdoors | -| Social Impact | 0.40 | LOW | Practical/financial focus takes priority | - ---- - -## Hassan's Key Priorities (from Preference Vector) - -1. ** Financial Security** (0.85) - *Highest priority* - - Family pressure to contribute financially - - Looking for good, consistent income - -2. ** Work-Life Balance** (0.80) - *Very high priority* - - Values flexibility in work schedule - - Doesn't want to be locked into rigid hours - -3. ** Job Stability** (0.70) - *High priority* - - Prefers stable income over risky opportunities - - Looking for reliable work arrangements - -4. ** Task Type Preferences** (0.65) - *Moderate priority* - - Enjoys hands-on, practical work - - Not interested in purely desk-based roles - ---- - -## PreferenceVector Type Definition - -The `PreferenceVector` class includes the following comprehensive fields: - -### Core Preference Dimensions -| Field | Type | Description | -|-------|------|-------------| -| `financial_importance` | `float [0-1]` | How much user values financial compensation | -| `work_environment_importance` | `float [0-1]` | How much user values work environment (remote, commute, conditions) | -| `career_advancement_importance` | `float [0-1]` | How much user values career growth | -| `work_life_balance_importance` | `float [0-1]` | How much user values work-life balance | -| `job_security_importance` | `float [0-1]` | How much user values job security | -| `task_preference_importance` | `float [0-1]` | How much user values specific task types | -| `social_impact_importance` | `float [0-1]` | How much user values social impact | - -### Quality Metadata -| Field | Type | Description | -|-------|------|-------------| -| `confidence_score` | `float [0-1]` | Overall confidence in preference estimates | -| `n_vignettes_completed` | `int` | Number of vignettes completed during elicitation | -| `per_dimension_uncertainty` | `dict[str, float]` | Uncertainty (variance) for each dimension | - -### Bayesian Metadata -| Field | Type | Description | -|-------|------|-------------| -| `posterior_mean` | `list[float]` | Raw Bayesian posterior mean vector (7 dimensions) | -| `posterior_covariance_diagonal` | `list[float]` | Diagonal of posterior covariance matrix | -| `fim_determinant` | `float | None` | Fisher Information Matrix determinant | - -### Qualitative Metadata (LLM-extracted) -| Field | Type | Description | -|-------|------|-------------| -| `decision_patterns` | `dict[str, Any]` | Patterns in how user makes decisions | -| `tradeoff_willingness` | `dict[str, bool]` | Explicit tradeoffs user is willing/unwilling to make | -| `values_signals` | `dict[str, bool]` | Deep values expressed in user's reasoning | -| `consistency_indicators` | `dict[str, float]` | Consistency in user's responses | -| `extracted_constraints` | `dict[str, Any]` | Hard constraints extracted from input | - ---- - -## Usage in Test Script - -The profile is created in the test script as follows: - -```python -# Create initial state with all profile components -state = RecommenderAdvisorAgentState( - session_id="test_session_12345", - youth_id="test_user_123", - country_of_user=Country.KENYA, - conversation_phase=ConversationPhase.INTRO, - recommendations=create_sample_recommendations(), - skills_vector=create_sample_skills_vector(), - preference_vector=create_sample_preference_vector() -) -``` - -### Key Functions - -| Function | Purpose | -|----------|---------| -| `create_sample_skills_vector()` | Creates Hassan's skills proficiency dictionary | -| `create_sample_preference_vector()` | Creates Hassan's PreferenceVector with Bayesian preferences | -| `create_sample_recommendations()` | Creates Node2Vec recommendations tailored to Hassan | - ---- - -## Source Files - -- **Agent Implementation**: [`compass/backend/app/agent/recommender_advisor_agent/agent.py`](../app/agent/recommender_advisor_agent/agent.py) -- **Sample Data**: [`compass/backend/evaluation_tests/recommender_advisor_agent/sample_data.py`](../evaluation_tests/recommender_advisor_agent/sample_data.py) -- **Interactive Test Script**: [`compass/backend/scripts/test_recommender_agent_interactive.py`](../scripts/test_recommender_agent_interactive.py) -- **PreferenceVector Type**: [`compass/backend/app/agent/preference_elicitation_agent/types.py`](../app/agent/preference_elicitation_agent/types.py) - ---- - -*Last updated: January 2026* diff --git a/backend/scripts/conversation_analysis/Description.md b/backend/scripts/conversation_analysis/Description.md index b459f447..0ab71ff5 100644 --- a/backend/scripts/conversation_analysis/Description.md +++ b/backend/scripts/conversation_analysis/Description.md @@ -15,7 +15,7 @@ This file describes the meaning of each column in the exported session analysis | `accepted_terms_and_conditions` | Whether the user accepted the T&Cs | | `user_never_started_conversation` | Whether the user never got past the onboarding steps (terms and conditions and PII) | | `has_multiple_sessions` | Whether this user had more than one session | -| `compass_version` | Version of Compass based on session time | +| `compass_version` | Version of Compass Connect based on session time | | `user_messages` | Number of real (non-silent, non-artificial) messages from the user | | `agent_messages` | Number of messages from the agent | | `counseling_messages` | Number of real messages the user sent during the counseling phase | @@ -49,4 +49,4 @@ These are computed booleans based on the user's session behavior. | `is_discovered_but_no_explored` | User discovered at least one experience but explored none | | `is_explored_1_but_not_completed` | User explored exactly one experience but didn’t finish | | `is_explored_gt1_but_not_complete` | User explored multiple experiences but didn’t finish | -| `engagement_group`| High-level segmentation of the user based on how far they progressed through the Compass flow. Based on behavioral milestones: Intro → Experience Discovery → Experience Exploration → Completed. Values are:

• **Group 1: Low Engagement** – User never left the intro phase or didn’t discover any experiences. These users were blocked for unknown reasons (not technical issues).
• **Group 2: Moderate Engagement** – User discovered at least one experience but did not explore any.
• **Group 3: High Engagement** – User started exploring one or more experiences but did not complete the conversation.
• **Group 4: Successful Engagement** – User explored all experiences and reached the end of the conversation.
• **Group 5: No Engagement** – User never started the conversation (e.g. didn’t accept terms or provide PII). This group cannot be contacted. | +| `engagement_group`| High-level segmentation of the user based on how far they progressed through the Compass Connect flow. Based on behavioral milestones: Intro → Experience Discovery → Experience Exploration → Completed. Values are:

• **Group 1: Low Engagement** – User never left the intro phase or didn’t discover any experiences. These users were blocked for unknown reasons (not technical issues).
• **Group 2: Moderate Engagement** – User discovered at least one experience but did not explore any.
• **Group 3: High Engagement** – User started exploring one or more experiences but did not complete the conversation.
• **Group 4: Successful Engagement** – User explored all experiences and reached the end of the conversation.
• **Group 5: No Engagement** – User never started the conversation (e.g. didn’t accept terms or provide PII). This group cannot be contacted. | diff --git a/config/ANALYTICS_SETUP.md b/config/ANALYTICS_SETUP.md index f405dc77..e9997502 100644 --- a/config/ANALYTICS_SETUP.md +++ b/config/ANALYTICS_SETUP.md @@ -1,6 +1,6 @@ # Analytics Setup Guide (GA4 + GTM) -This guide walks through setting up Google Analytics 4 (GA4) and Google Tag Manager (GTM) for a Compass fork. +This guide walks through setting up Google Analytics 4 (GA4) and Google Tag Manager (GTM) for a Compass Connect fork. The process is largely automated by `backend/scripts/analytics/setup_analytics.py`, but requires a few one-time manual steps first. ## How It Works @@ -37,7 +37,7 @@ creating corresponding triggers/tags in GTM (either via the script or manually). ### SPA Page View Tracking (HashRouter) -Compass uses React's `HashRouter`, so URLs look like `https://example.com/#/skills-interests`. +Compass Connect uses React's `HashRouter`, so URLs look like `https://example.com/#/skills-interests`. GA4's default page view tracking only sees `/` as the page path because the hash fragment is not part of the URL path. The setup script handles this automatically by creating: @@ -57,14 +57,14 @@ The GA4 Config tag has `sendPageView` disabled to avoid duplicate page views. ### 1. Create a GA4 Account -Go to [analytics.google.com](https://analytics.google.com) and create an account (e.g., "Compass Analytics"). +Go to [analytics.google.com](https://analytics.google.com) and create an account (e.g., "Compass Connect Analytics"). One account can hold properties for all forks. > **Note:** GA4 accounts are free and not tied to GCP billing. ### 2. Create a GTM Account -Go to [tagmanager.google.com](https://tagmanager.google.com) and create an account (e.g., "Compass Tags"). +Go to [tagmanager.google.com](https://tagmanager.google.com) and create an account (e.g., "Compass Connect Tags"). Same pattern: one account, containers per fork. > **Do NOT** manually create a container — the script creates one automatically. @@ -133,7 +133,7 @@ python3 setup_analytics.py \ --ga4-account-id \ --gtm-account-id \ --url "https://your-fork.compass.tabiya.tech" \ - --property-name "Compass YourFork" \ + --property-name "Compass Connect YourFork" \ --config ../../../config/default.json \ --credentials path/to/service_account_key.json ``` diff --git a/deployment-procedure.md b/deployment-procedure.md index 678bd4a3..cb26f52b 100644 --- a/deployment-procedure.md +++ b/deployment-procedure.md @@ -209,9 +209,9 @@ The Google OAuth enables Google Sign-In and requires a privacy page, a consent s ### Step 2.2: Set up OAuth Consent Screen. **Set up the OAuth consent screen** under **API & Services** → **OAuth consent screen** → **Get started**: -- **App Name**: `` (e.g., *Compass by Tabiya*) +- **App Name**: `` (e.g., *Compass Connect by Tabiya*) - **User Support Email**: `` -- **Audience**: `External`. Compass is a public application. Not an internal one. +- **Audience**: `External`. Compass Connect is a public application. Not an internal one. - **Contact Information**: `` - Complete the form and submit. diff --git a/docs/VECTOR_SEARCH_INTEGRATION_ANALYSIS.md b/docs/VECTOR_SEARCH_INTEGRATION_ANALYSIS.md deleted file mode 100644 index 1589107a..00000000 --- a/docs/VECTOR_SEARCH_INTEGRATION_ANALYSIS.md +++ /dev/null @@ -1,621 +0,0 @@ -# Vector Search Integration Analysis -**Date:** 2026-01-17 -**Status:** Vector search is integrated and working - ---- - -## Executive Summary - -**Vector search IS integrated** into the Recommender/Advisor Agent -**Triggered when users mention occupations not in recommendations** - **NOT YET integrated into main Agent Director** (Epic 3 is standalone) -**Test script updated** with `load_dotenv()` - all systems operational - ---- - -## Table of Contents - -1. [Integration Points](#1-integration-points) -2. [How It Works (4 Dimensions)](#2-how-it-works-4-dimensions) -3. [Trigger Points](#3-trigger-points) -4. [Future Agent Director Integration](#4-future-agent-director-integration) -5. [Quick Reference](#5-quick-reference) - ---- - -## 1. Integration Points - -### Current Integration Architecture - -``` -RecommenderAdvisorAgent (agent.py) -│ -├── Constructor receives occupation_search_service (line 90) -│ └── Stored as self._occupation_search_service (line 107) -│ -└── Passes to Phase Handlers: - ├── IntroPhaseHandler (line 151) - ├── ConcernsPhaseHandler (line 160) - ├── ExplorationPhaseHandler (line 169) - └── PresentPhaseHandler (line 178) - │ - └── All inherit from BasePhaseHandler - └── Provides _search_occupation_by_name() method -``` - -### Integration Status by Component - -| Component | Integrated? | File | Line | -|-----------|------------|------|------| -| **RecommenderAdvisorAgent** | Yes | `agent.py` | 90, 107 | -| **BasePhaseHandler** | Yes | `base_handler.py` | 35, 49, 84-122 | -| **IntroPhaseHandler** | Yes | Receives service via constructor | | -| **PresentPhaseHandler** | Yes | Receives service via constructor | | -| **ExplorationPhaseHandler** | Yes | **ACTIVELY USES** at line 275 | | -| **ConcernsPhaseHandler** | Yes | Receives service via constructor | | -| **Agent Director** | No | Not yet routed | N/A | - ---- - -## 2. How It Works (4 Dimensions) - -### 2.1 PARETO PRINCIPLE (20% controlling 80%) - -The essential flow: - -```python -# 1. User mentions occupation not in recommendations -user_input = "I want to be a DJ" - -# 2. ExplorationHandler detects out-of-list mention (line 275) -mentioned_occ = await self._search_occupation_by_name("DJ") - -# 3. BasePhaseHandler._search_occupation_by_name() (lines 84-122) -results = await self._occupation_search_service.search( - query="DJ", - k=1 # Top match only -) - -# 4. Returns OccupationEntity -OccupationEntity( - preferredLabel="disc jockey", - code="2654.9", - UUID="abc-123-...", - score=0.92 # Similarity score -) - -# 5. Generates contextual response -response = await self._handle_out_of_list_occupation( - found_occupation=mentioned_occ, - ... -) -``` - -**Critical 20%:** -1. `_occupation_search_service` passed to agent constructor -2. `_search_occupation_by_name()` in BasePhaseHandler -3. `_handle_out_of_list_occupation()` generates LLM response -4. Used in ExplorationPhaseHandler when user mentions new occupation - -### 2.2 DECOMPOSITION (Logical Sub-Parts) - -``` -Vector Search Integration -│ -├── 1. DEPENDENCY INJECTION -│ ├── FastAPI Dependencies (vector_search_dependencies.py) -│ │ ├── get_occupation_search_service() → Singleton -│ │ ├── get_embeddings_service() → Google Vertex AI -│ │ └── SearchServices container -│ │ -│ └── Agent Construction -│ └── occupation_search_service passed to RecommenderAdvisorAgent -│ -├── 2. BASE INFRASTRUCTURE (base_handler.py) -│ ├── _search_occupation_by_name() -│ │ ├── Checks if service available -│ │ ├── Calls vector search -│ │ └── Returns OccupationEntity or None -│ │ -│ └── _handle_out_of_list_occupation() -│ ├── Builds LLM prompt with context -│ ├── Explains why not in recommendations -│ └── Offers choices to user -│ -├── 3. ACTIVE USAGE (exploration_handler.py) -│ └── Line 275: Search when user mentions occupation -│ ├── Parse user input for occupation mention -│ ├── Search taxonomy -│ └── Generate response -│ -└── 4. PASSIVE AVAILABILITY - └── All handlers have access via inheritance - └── Can be used in future enhancements -``` - -### 2.3 FIRST PRINCIPLES (Atomic Concepts) - -**What is an "out-of-list occupation"?** - -The recommender agent presents top 5 occupations: -``` -1. Electrician (88% match) -2. Boda-boda Rider (79% match) -3. Port Cargo Handler (74% match) -4. Boat Fundi (71% match) -5. Market Vendor (68% match) -``` - -User says: **"I want to be a DJ"** - -**Problem:** "DJ" is not in the top 5 recommendations. - -**Solution:** Use vector search to: -1. Find "DJ" in the full occupation taxonomy (54,843 occupations) -2. Return: `OccupationEntity(preferredLabel="disc jockey", score=0.92)` -3. Generate contextual response explaining why it wasn't recommended - -**Why vector search, not exact match?** -- User might say "DJ" but taxonomy has "disc jockey" -- User might say "electrician" but mean "industrial electrician" -- Vector embeddings find **semantically similar** occupations - -**Example:** -``` -User input: "computer guy" -Vector search finds: -1. "ICT technician" (score: 0.85) -2. "computer programmer" (score: 0.83) -3. "software developer" (score: 0.81) - -Returns top match: "ICT technician" -``` - -### 2.4 LEVELS OF ABSTRACTION - -**Level 1: User Experience** -``` -USER: "I want to be a DJ" - -AGENT: "I found 'Disc Jockey' in our database. While it wasn't - among my top recommendations based on your skills and - preferences (your profile shows more hands-on technical - skills), I'm happy to explore it with you if you're - interested. Would you like to learn more about it, or - hear why I suggested electrician and other alternatives?" -``` - -**Level 2: Phase Handler Logic** -```python -# exploration_handler.py:275 -async def handle(user_input, state, context): - # Detect occupation mention - if user_mentions_occupation(user_input): - # Search taxonomy - mentioned_occ = await self._search_occupation_by_name(user_input) - - if mentioned_occ: - # Generate contextual response - return await self._handle_out_of_list_occupation( - found_occupation=mentioned_occ, - user_input=user_input, - state=state, - context=context, - recommendations_summary=build_summary(state.recommendations) - ) -``` - -**Level 3: Search Service** -```python -# base_handler.py:84-122 -async def _search_occupation_by_name(occupation_name: str): - if not self._occupation_search_service: - return None - - results = await self._occupation_search_service.search( - query=occupation_name, - k=1 - ) - - return results[0] if results else None -``` - -**Level 4: Vector Search** -```python -# esco_search_service.py:108-161 -async def search(query: str, k: int): - # 1. Convert text to embedding - embedding = await self.embedding_service.embed(query) # Google API - - # 2. MongoDB vector search - pipeline = [ - {"$vectorSearch": { - "queryVector": embedding, - "path": "embedding", - "numCandidates": k * 30, - "limit": k * 3, - "index": "embedding_index" - }}, - {"$group": {"_id": "$UUID", "score": {"$max": "$score"}}}, - {"$sort": {"score": -1}}, - {"$limit": k} - ] - - results = await collection.aggregate(pipeline) - return [OccupationEntity(...) for doc in results] -``` - -**Level 5: Infrastructure** -- MongoDB Atlas cluster with vector indexes -- Google Vertex AI embeddings API -- 768-dimensional float arrays -- Cosine similarity ranking - ---- - -## 3. Trigger Points - -### When Vector Search is Called - -| Trigger | Handler | Code Location | Description | -|---------|---------|---------------|-------------| -| **User mentions occupation not in list** | ExplorationPhaseHandler | `exploration_handler.py:275` | User says "I want to be X" where X is not in recommendations | -| **Available to all handlers** | BasePhaseHandler | `base_handler.py:84` | Any handler can call `_search_occupation_by_name()` | - -### Detailed Flow: ExplorationPhaseHandler - -```python -# exploration_handler.py (simplified) - -async def handle(user_input: str, state, context): - """ - EXPLORATION phase - user is exploring a specific occupation. - - Trigger points for vector search: - 1. User mentions occupation not in current focus - 2. User asks about different occupation - """ - - # TRIGGER: User mentions new occupation - # Example: User was exploring "Electrician", now says "What about DJ?" - - # Line 275: Search for the occupation - mentioned_occ = await self._search_occupation_by_name(user_input) - - if mentioned_occ: - # Found in taxonomy! - # Check if it's in recommendations - in_recommendations = state.get_recommendation_by_id(mentioned_occ.UUID) - - if not in_recommendations: - # OUT-OF-LIST OCCUPATION - # Generate contextual response explaining why not recommended - return await self._handle_out_of_list_occupation( - found_occupation=mentioned_occ, - ... - ) - else: - # In recommendations, switch focus - state.current_focus_id = mentioned_occ.UUID - # Generate exploration response -``` - -### What Happens When Vector Search Finds Nothing? - -```python -# base_handler.py:117-118 -if not results or len(results) == 0: - self.logger.info(f"No occupation found in taxonomy for: '{occupation_name}'") - return None - -# In exploration_handler.py: -if mentioned_occ is None: - # No match found in taxonomy - # LLM handles conversationally without occupation data - # Example response: "I don't have specific information about that - # occupation in our database. Would you like to explore one of the - # recommended options instead?" -``` - -### Example Conversation Flow - -``` -TURN 1: -USER: "Show me recommendations" -AGENT: [Presents top 5: Electrician, Boda-boda, Port Handler, ...] -PHASE: PRESENT_RECOMMENDATIONS -VECTOR SEARCH: Not triggered - -TURN 2: -USER: "Tell me about electrician" -AGENT: [Explores electrician details] -PHASE: CAREER_EXPLORATION -FOCUS: Electrician (in recommendations) -VECTOR SEARCH: Not triggered (already in list) - -TURN 3: -USER: "What about being a DJ?" -PHASE: CAREER_EXPLORATION -TRIGGER: User mentioned "DJ" (not in recommendations) -VECTOR SEARCH CALLED: search(query="DJ", k=1) -RESULT: OccupationEntity(preferredLabel="disc jockey", score=0.92) -AGENT: "I found 'Disc Jockey' in our database. While it wasn't in - my top recommendations (your skills lean more technical), - I can explore it with you. Want to learn more?" - -TURN 4: -USER: "Yes, tell me more about DJ" -AGENT: [Explores disc jockey details from taxonomy] -PHASE: CAREER_EXPLORATION -FOCUS: Disc Jockey (out-of-list) -VECTOR SEARCH: Not triggered (already have OccupationEntity) -``` - ---- - -## 4. Future Agent Director Integration - -### Current State: Standalone Agent - -The `RecommenderAdvisorAgent` is **NOT yet integrated** into the main `LLMAgentDirector` routing system. - -**Evidence:** -1. No `RECOMMENDER_ADVISOR_AGENT` in `AgentType` enum -2. No routing logic in `_LLMRouter` -3. Agent only accessible via direct instantiation (test scripts) - -### Integration Architecture Plan - -``` -┌─────────────────────────────────────────────────────────────┐ -│ LLMAgentDirector │ -│ (app/agent/agent_director/llm_agent_director.py) │ -└───────────────────────┬─────────────────────────────────────┘ - │ - ├── Receives: SearchServices (via DI) - │ └── Contains: occupation_search_service - │ - ▼ - ┌──────────────────────────────┐ - │ _LLMRouter │ - │ (Routes user messages) │ - └──────────────┬───────────────┘ - │ - ┌───────┴────────┐ - │ │ - [Existing Agents] [NEW: RecommenderAdvisorAgent] - ├── WelcomeAgent │ - ├── CollectExperiences │ - └── ExploreExperiences │ - │ - ┌────────────┴─────────────┐ - │ │ - [Constructor receives] [Passes to handlers] - occupation_search_service ├── IntroHandler - ├── PresentHandler - ├── ExplorationHandler - └── ConcernsHandler -``` - -### Required Changes for Integration - -#### 1. Add Agent Type (agent_types.py) -```python -class AgentType(str, Enum): - WELCOME_AGENT = "WELCOME_AGENT" - COLLECT_EXPERIENCES_AGENT = "COLLECT_EXPERIENCES_AGENT" - EXPLORE_EXPERIENCES_AGENT_DIRECTOR = "EXPLORE_EXPERIENCES_AGENT_DIRECTOR" - PREFERENCE_ELICITATION_AGENT = "PREFERENCE_ELICITATION_AGENT" - RECOMMENDER_ADVISOR_AGENT = "RECOMMENDER_ADVISOR_AGENT" # ADD THIS -``` - -#### 2. Add to LLMAgentDirector Constructor (llm_agent_director.py) -```python -class LLMAgentDirector: - def __init__( - self, - conversation_manager: ConversationMemoryManager, - search_services: SearchServices, # Already receives this! - experience_pipeline_config: ExperiencePipelineConfig - ): - # Initialize existing agents... - - # ADD: Initialize RecommenderAdvisorAgent - self._recommender_advisor_agent = RecommenderAdvisorAgent( - occupation_search_service=search_services.occupation_search_service, - # db6_client=... # Future: DB6 integration - # node2vec_client=... # Future: Node2Vec integration - ) - - # Register in agent map - self._agents[AgentType.RECOMMENDER_ADVISOR_AGENT] = self._recommender_advisor_agent -``` - -#### 3. Add Routing Logic (_llm_router.py) -```python -async def route(self, user_message: str, application_state: ApplicationState): - """ - Route user message to appropriate agent. - """ - - # Existing routing logic... - - # NEW: Route to RecommenderAdvisorAgent - if self._should_route_to_recommender_advisor(application_state): - return AgentType.RECOMMENDER_ADVISOR_AGENT - - # ... existing routing - -def _should_route_to_recommender_advisor(self, state: ApplicationState) -> bool: - """ - Route to recommender advisor when: - 1. Preference elicitation is complete - 2. User has skills vector - 3. Recommendations have been generated (or should be) - """ - # Check if we have recommendations ready - if state.recommender_advisor_agent_state: - return True - - # Check if prerequisites are met - if (state.preference_elicitation_agent_state and - state.preference_elicitation_agent_state.finished and - state.user_experiences): # Has skills - # Initialize recommender state - return True - - return False -``` - -#### 4. Add State to ApplicationState (application_state.py) -```python -class ApplicationState(BaseModel): - # Existing state... - welcome_agent_state: Optional[WelcomeAgentState] = None - collect_experiences_agent_state: Optional[CollectExperiencesAgentState] = None - preference_elicitation_agent_state: Optional[PreferenceElicitationAgentState] = None - - # ADD THIS: - recommender_advisor_agent_state: Optional[RecommenderAdvisorAgentState] = None -``` - -### Dependency Injection Flow - -The good news: **SearchServices is already injected into LLMAgentDirector!** - -```python -# server_dependencies/agent_director_dependencies.py:15-33 - -def get_agent_director( - conversation_manager: ConversationMemoryManager = Depends(...), - search_services: SearchServices = Depends(get_all_search_services), # ← Already here! - application_config: ApplicationConfig = Depends(...) -) -> LLMAgentDirector: - return LLMAgentDirector( - conversation_manager=conversation_manager, - search_services=search_services, # ← Already passed! - ... - ) -``` - -**What this means:** -- `occupation_search_service` is already available in `search_services.occupation_search_service` -- Singleton instance (one service for all requests) -- Properly initialized with database, embeddings, etc. -- Just need to pass to `RecommenderAdvisorAgent` constructor - -### Integration Checklist - -- [ ] 1. Add `RECOMMENDER_ADVISOR_AGENT` to `AgentType` enum -- [ ] 2. Import `RecommenderAdvisorAgent` in `llm_agent_director.py` -- [ ] 3. Initialize agent in `LLMAgentDirector.__init__()` - - [ ] Pass `search_services.occupation_search_service` - - [ ] Pass `db6_client` (when available) - - [ ] Pass `node2vec_client` (when available) -- [ ] 4. Add routing logic in `_LLMRouter` - - [ ] Implement `_should_route_to_recommender_advisor()` - - [ ] Add to `route()` method -- [ ] 5. Add `recommender_advisor_agent_state` to `ApplicationState` -- [ ] 6. Update state loading/saving in `application_state_store.py` -- [ ] 7. Write integration tests -- [ ] 8. Update documentation - -### No Changes Needed - - Vector search dependencies - Already working - Occupation search service - Already singleton - SearchServices injection - Already in place - RecommenderAdvisorAgent code - Already complete - Phase handlers - Already integrated with search - ---- - -## 5. Quick Reference - -### File Locations - -| Component | File Path | Key Lines | -|-----------|-----------|-----------| -| **Agent** | `app/agent/recommender_advisor_agent/agent.py` | 90, 107, 151, 160, 169, 178 | -| **Base Handler** | `app/agent/recommender_advisor_agent/phase_handlers/base_handler.py` | 35, 49, 84-122, 124-236 | -| **Exploration Handler** | `app/agent/recommender_advisor_agent/phase_handlers/exploration_handler.py` | 275-277 | -| **Vector Search Service** | `app/vector_search/esco_search_service.py` | 189-297 | -| **Dependencies** | `app/vector_search/vector_search_dependencies.py` | 87-110 | -| **Test Script** | `scripts/test_recommender_agent_interactive.py` | 20-23 (dotenv), 700-732 (init) | - -### Commands - -```bash -# Test vector search -cd compass/backend -poetry run python scripts/test_vector_search_diagnostic.py - -# Run interactive test (requires terminal) -poetry run python scripts/test_recommender_agent_interactive.py - -# Check if .env is loaded -python -c "from dotenv import load_dotenv; load_dotenv(); import os; print(os.getenv('TAXONOMY_MODEL_ID'))" -``` - -### Key Variables - -```python -# Environment variables (from .env) -TAXONOMY_MODEL_ID = "68933862382aab4c7de13ec6" -EMBEDDINGS_SERVICE_NAME = "GOOGLE-VERTEX-AI" -EMBEDDINGS_MODEL_NAME = "text-embedding-005" -TAXONOMY_MONGODB_URI = "mongodb+srv://..." - -# Service instances -occupation_search_service: OccupationSearchService -embedding_service: GoogleEmbeddingService -taxonomy_db: AsyncIOMotorDatabase - -# Search results -OccupationEntity: - - preferredLabel: str # "electrician" - - code: str # "7411.1" - - UUID: str # "0d50f9af-..." - - score: float # 0.92 (similarity) - - description: str -``` - -### Common Issues & Solutions - -| Issue | Cause | Solution | -|-------|-------|----------| -| "Application configuration is not setup" | `set_application_config()` not called | Add `load_dotenv()` and initialize config | -| "Occupation search service not available" | Service is `None` | Check initialization in handler constructor | -| "No occupation found in taxonomy" | User input doesn't match any occupation | Normal - LLM handles gracefully | -| Vector search returns wrong results | Embedding model mismatch | Verify `EMBEDDINGS_MODEL_NAME` matches database | - ---- - -## Summary - -### What's Working - -1. **Vector search fully integrated** into RecommenderAdvisorAgent -2. **Actively used** in ExplorationPhaseHandler (line 275) -3. **All dependencies injected** correctly via SearchServices -4. **54,843 occupations** searchable via semantic similarity -5. **Test script updated** with `load_dotenv()` - -### What's Pending - -1. **Integration into LLMAgentDirector** (routing logic) -2. **ApplicationState management** (save/load recommender state) -3. **DB6 integration** (when Epic 1 completes) -4. **Node2Vec integration** (for recommendation generation) - -### Next Steps - -1. **Now:** Test vector search with interactive script -2. **Soon:** Integrate into Agent Director routing -3. **Later:** Connect DB6 and Node2Vec when available - ---- - -**Last Updated:** 2026-01-17 -**Tested On:** compass/backend with MongoDB Atlas + Google Vertex AI -**Status:** Production Ready (pending Agent Director integration) diff --git a/docs/kenya-epic4-plan.md b/docs/kenya-epic4-plan.md deleted file mode 100644 index 896ba366..00000000 --- a/docs/kenya-epic4-plan.md +++ /dev/null @@ -1,902 +0,0 @@ -# Kenya Epic 4: Conversation Flow Optimization & Swahili Enablement - -## Quick Summary - -**Goal**: Make skills elicitation faster (20% time reduction), less repetitive (30% reduction), and more natural while maintaining quality (85%+ skill overlap). Enable Swahili language support. - -**Key Personas**: -- Persona 1: Informal worker (no CV, speaks to tasks/years) -- Persona 2: Formal/mixed worker (has CV, responsibilities documented or explained) - ---- - -# MILESTONE 1: Baseline, Harness & Design Locks - -**Objective**: Establish measurable baselines and unblock parallel workstreams. - ---- - -## A1: Technical Work Plan & Dependency Map ✓ - -**Status**: COMPLETE (this document) - ---- - -## A2: Evaluation Harness + Baseline Runs - -### Task: Implement & Integrate Metrics Collector ✓ - -**What**: Automated metrics collection in E2E tests. - -**Files to Create**: -- `backend/evaluation_tests/baseline_metrics_collector.py` - -**Files to Modify**: -- `backend/evaluation_tests/e2e_chat_executor.py` - Add metrics_collector parameter, call hooks -- `backend/evaluation_tests/app_conversation_e2e_test.py` - Initialize collector, export metrics -- `backend/evaluation_tests/evaluation_metrics.py` - Add baseline columns to CSV - -**Metrics Captured**: -- Turn count, conversation time (total + by phase + by agent) -- LLM calls count and duration -- Experiences discovered/explored, skills per experience -- Repetition rate (semantic similarity > 0.75) -- Skill overlap percentage - -**Baseline Test Runs**: -```bash -cd backend -pytest -m "evaluation_test" --repeat 3 \ - -k "single_experience_specific_and_concise_user_e2e or golden_simple_formal_employment" \ - evaluation_tests/app_conversation_e2e_test.py -s -``` - -**Post-Processing**: -- Calculate mean, median, std dev, 95% CI -- Document metrics output as benchmarks - -**Acceptance Criteria**: -- [x] Metrics collector implemented and integrated -- [x] 6 baseline runs completed (2 personas × 3 repititions) -- [x] Metrics exported to JSON/CSV per test -- [x] Statistics calculated and documented - ---- - -## A3: Observability Plan - -### Task: Add Correlation IDs & Logging Fields - -**Files to Create**: -- `backend/app/middleware/correlation_id_middleware.py` -- `docs/observability-sensitive-data-checklist.md` - -**Files to Modify**: -- `backend/app/context_vars.py` - Add `correlation_id: ContextVar[str]` -- `backend/app/server.py` - Register middleware -- `backend/app/conversations/service.py` - Add session_id, turn_index to logs -- `backend/app/agent/agent_director/llm_agent_director.py` - Add agent_type, phase to logs -- `backend/common_libs/llm/llm_caller.py` - Add llm_call_duration_ms to logs - -**Logging Fields**: -- `correlation_id`, `session_id`, `turn_index`, `agent_type`, `llm_call_duration_ms`, `phase` - -**Sensitive Data Checklist**: -- ❌ NEVER: User PII, full conversation text, raw input before PII filter, API keys -- ✅ SAFE: Session ID (numeric), UUIDs, timing metrics, agent types, aggregated stats - -**Acceptance Criteria**: -- [x] Correlation ID middleware implemented -- [x] All 6 logging fields added to relevant code -- [x] Code review confirms no PII logged - - -## C1: Swahili Model Assessment - FINAL VERDICT IS GEMINI 2.5 (https://docs.cloud.google.com/gemini/docs/codeassist/supported-languages) - -**What**: Evaluation framework for Swahili language support. - -**Content**: -- Evaluation criteria: Performance, Quality, Cost, Integration, Localization -- Candidate models for language support: Gemini 2.5 Flas -- Shortlist 2-3 models with pros/cons -- Collect 20+ Swahili job terms - -**Acceptance Criteria**: -- [x] 2-3 models shortlisted - Chosen the best to be Gemini 2.5 - - - -## Success Criteria - -**Quantitative Baselines**: -- [x] Median turn count with confidence interval -- [x] Average conversation time by phase and agent -- [x] Repetition rate calculated -- [x] Skill overlap percentage -- [x] LLM call count and duration - -**Infrastructure**: -- [x] Evaluation harness runs automatically -- [x] Metrics exported in JSON/CSV -- [x] Correlation IDs in logs -- [x] Sensitive data checklist reviewed - -**Documentation**: -- [x] `baseline_metrics_collector.py` committed -- [x] Baseline metrics documented -- [x] Milestone 2 implementation plan documented (see M2 section below) - ---- - -# MILESTONE 2: Refactor Skills Flow + Persona-Aware Probing - -**Objective**: Deliver measurable improvements in flow quality for both personas. - -**Baseline Metrics** (from M1): -- Avg turns: 32.4 | LLM calls: 251 | Repetition rate: 11% | Starter diversity: 15.4% -- Test case variance: 16 turns (best) to 70 turns (worst - formal verbose style) -- Critical issue: FAREWELL_AGENT consuming 83% of processing time (64 LLM calls post-conversation) - ---- - -## B1: Refactored Skills Elicitation Flow - -**Task 1.1: Debug FAREWELL_AGENT Performance Issue (P0)** -- Investigate why FAREWELL_AGENT makes 64 LLM calls after conversation ends -- Determine if user-facing or backend processing (job matching, skill extraction) -- Fix or separate metrics for accurate timing data -- Files: `llm_agent_director.py`, `farewell_agent.py`, `conversations/service.py` - -**Task 1.2: Reduce Starter Phrase Repetition (P0)** -- Problem: "Okay" used in 27% of questions; diversity only 15.4% -- Target: Top starter <15%, diversity >35% -- Add varied acknowledgment phrases to prompts -- Files: `collect_experiences_prompt.py`, `explore_skills_prompt.py` - -**Task 1.3: Increase Achievement Question Rate (P1)** -- Problem: Only 1.9% achievement questions (target: >8%) -- Add prompts for accomplishments, challenges overcome, improvements -- Files: `explore_skills_prompt.py` - -**Task 1.4: Optimize Skills Exploration (P0)** -- Reduce from 6 turns to 4 turns per experience -- Consolidate questions, add exit criteria (8-12 skills OR 4 turns) -- Files: `explore_skills_agent.py`, `explore_skills_prompt.py` - -**Task 1.5: Early Exit for Concise Users (P2)** -- Detect rich, detailed responses and skip redundant follow-ups -- Target: Concise users complete in <18 turns -- Files: `llm_agent_director.py` - ---- - -## B2: Persona-Aware Flow Implementation - -**Important**: CV upload integration deferred to Milestone 4. Persona detection is verbal-only for M2. - -**Task 2.1: Implement Persona Detection (P0)** -- Detect Persona 2 (Formal) via verbal cues: "title", "position", "department", "responsibilities" -- Detect Persona 1 (Informal) via: "tasks", "daily work", "what I did" -- Default to Persona 1 (safer for informal workers) -- Create: `backend/app/agent/persona_detector.py` -- Modify: `conversations/service.py`, `llm_agent_director.py` - -**Task 2.2: Persona 1 (Informal) Optimization (P1)** -- Target: 18-22 turns (simple), ≤35 turns (multi-experience) -- Use simpler language, more examples/scaffolding -- Focus on "what did you do daily" → skills mapping -- Files: `collect_experiences_prompt.py`, `explore_skills_prompt.py` - -**Task 2.3: Persona 2 (Formal) Optimization (P0 - Highest Impact)** -- Problem: Formal verbose descriptions take 70 turns (!) -- Target: ≤35 turns (down from 70) -- Acknowledge formal info upfront, avoid redundant questions -- Track information completeness per experience -- Files: `collect_experiences_agent.py`, prompt files - -**Task 2.4: Multi-Experience Optimization (P1)** -- Problem: 49 turns for 3+ experiences -- Target: ≤35 turns for 3+ experiences -- First experience: Full exploration (4-5 turns) -- Subsequent: Focused exploration (3 turns) -- Files: `llm_agent_director.py`, `conversations/service.py` - ---- - -## Golden Transcripts (English) + CI Gating - -**Task 3.1: Create Golden Transcripts (Based on Refactored Flow)** -- Timing: Create AFTER B1 + B2 refactoring complete -- 6 transcripts total (3 per persona): - - Persona 1: Simple single exp (18-20 turns), Multi-exp (30-35), Process questioner (20-25) - - Persona 2: Simple formal (20-25), Formal verbose (30-35), Career progression (35-40) -- Create: `backend/evaluation_tests/golden_transcripts/persona_1/*.json` -- Create: `backend/evaluation_tests/golden_transcripts/persona_2/*.json` - -**Task 3.2: Implement CI Test Integration (P0)** -- Metrics to Gate (Block PR): Turn count ±2, Repetition ≤8%, Skill overlap ≥85% -- Metrics to Warn: Achievement Q rate ≥5%, Starter diversity ≥35% -- Create: `golden_transcript_runner.py`, `check_metrics_thresholds.py` -- Create: `.github/workflows/golden_transcript_tests.yml` - ---- - -## C1: Swahili Model Documentation - -**Task 4.1: Document Gemini 2.5 Flash Selection** -- Model comparison: Gemini 2.5 Flash vs GPT-4o vs Claude 3.5 -- Criteria: Swahili performance, cost, latency, integration -- Selection rationale and cost analysis -- Create: `docs/swahili-model-selection.md` - -**Task 4.2: Gemini Integration Preparation** -- API setup checklist for M3 -- Environment variables, rate limits, pricing -- Create: `docs/gemini-integration-checklist.md` - ---- - -## Success Criteria - -**Performance Improvements** (vs Baseline: 32.4 turns, 11% repetition, 251 LLM calls): -- [x] Turn count reduced to ≤27 (17%+ reduction) -- [x] Repetition rate reduced to ≤8% (27%+ reduction) -- [x] Starter diversity increased to ≥35% (from 15.4%) -- [x] Achievement question rate ≥8% (from 1.9%) -- [x] LLM calls reduced to ≤200 (20%+ reduction) - -**Quality Maintained**: -- [x] Skill overlap maintained at 85%+ -- [x] Experience completeness maintained at 95%+ -- [x] No regression in occupation accuracy - -**Persona-Aware Flows**: -- [x] Persona detection implemented (verbal-only, >90% accuracy) -- [x] Persona 1 (Informal): 18-22 turns simple, ≤35 multi-experience -- [x] Persona 2 (Formal): 20-25 turns simple, ≤35 turns verbose (down from 70!) -- [x] Flow adapts based on detected persona type - -**CI/CD Integration**: -- [x] 6 golden transcripts created (3 per persona) -- [x] Automated tests run on every PR with metric thresholds -- [x] Clear failure messages when quality gates violated - -**Swahili Preparation**: -- [x] Gemini 2.5 Flash selection documented with rationale -- [x] Integration checklist ready for M3 (no blockers) - ---- - -# MILESTONE 3: Swahili Enablement + Localization - -**Objective**: Deliver Swahili flows with mapping parity and regression protection. - -## C2: Localization/Synonym Mapping Module - -**Objective**: Normalize Swahili and code-switched inputs and map them to taxonomy terms. - -**Tasks**: -- Build a Swahili term dictionary (50+ terms) including informal slang and code-switch variants. -- Define a normalized mapping format (JSON/CSV) and load it at runtime. -- Implement a mapping service to resolve Swahili terms before retrieval / skill extraction. -- Add coverage + accuracy checks (mapping hit rate, false positives). -- Document mapping sources and update process. - -**How RAG helps**: -- Use a Swahili glossary + taxonomy snippets as retrieval context to disambiguate slang and code-switched terms. -- Retrieve localized examples for prompts so the agent uses consistent Swahili phrasing and domain terms. -- Support fallback mapping when exact synonym matches are missing, without overfitting prompts. - -## C3: Swahili-Enabled Flows End-to-End - -**Objective**: Enable the full flow in Swahili with quality parity. - -**Tasks**: -- Add Swahili locale to backend config and frontend supported locales. -- Provide Swahili translations for core system messages and prompts. -- Enforce Swahili responses in LLM prompt templates (no language drift). -- Create Swahili E2E test cases (Persona 1 + Persona 2). -- Create Swahili golden transcripts and integrate into CI. -- Compare Swahili skill discovery accuracy to English baseline (≥80% parity). - ---- - -## Success Criteria - -**Swahili Language Support**: -- [x] Skills elicitation flow works end-to-end in Swahili -- [x] Preference flow functional in Swahili -- [x] Language switching implemented -- [x] Swahili responses maintain correct tone and grammar - -**Localization/Mapping**: -- [x] Synonym mapping module created and tested -- [x] 50+ Swahili job terms mapped to taxonomy -- [x] Code-switched terms handled -- [x] Regional variations documented - -**Quality Parity**: -- [x] Swahili skill discovery accuracy at 80%+ of English baseline -- [x] Occupation matching works for Swahili inputs -- [x] Same structured output as English flows - -**Testing & Regression**: -- [x] Swahili test cases created for both personas -- [x] Automated tests integrated into CI -- [x] Regression protection for English flows - ---- - -# MILESTONE 4: CV Integration + Qualifications Extraction - -**Objective**: Make Persona 2 experience coherent and add qualifications affecting eligibility. This should support basic CV file uploads in the data extraction layer. - -**Current State (What Already Exists)**: -- CV upload pipeline: `backend/app/users/cv/service.py` — file upload → markdown conversion → LLM extraction → GCS storage -- CV extraction: `CVExperienceExtractor` produces `list[str]` (plain experience bullets), **not** structured entities -- Frontend: `Chat.tsx` / `ChatMessageField.tsx` handles file upload, polls status, displays bullets -- MongoDB: `user_cv_uploads` collection stores upload records + experience bullets -- Feature flag: `GLOBAL_ENABLE_CV_UPLOAD` gates the feature -- **Gap**: CV data never flows into the conversation agents — the extracted bullets are dead-end data - ---- - -## B3: CV Integration → Merged Profile - -**Objective**: Bridge the existing CV upload pipeline with the conversational experience collection so that Persona 2 users who upload a CV get a faster, less repetitive flow. The agent should acknowledge what the CV already says and only ask for supplementary details. - -### Task 3.1: Structured CV Extraction (P0) - -**Problem**: `CVExperienceExtractor` returns `list[str]` — unstructured sentences. The `CollectExperiencesAgent` needs `CollectedData` objects with typed fields (title, company, timeline, work_type). - -**What**: Enhance the CV extraction LLM to produce structured experience data that maps directly to `CollectedData`. - -**Files to Create**: -- `backend/app/users/cv/utils/structured_extractor.py` — New `CVStructuredExtractor` class -- `backend/app/users/cv/types.py` — Add `CVExtractedExperience` Pydantic model - -**Files to Modify**: -- `backend/app/users/cv/service.py` — Add structured extraction step after bullet extraction -- `backend/app/users/cv/repository.py` — Persist structured experiences alongside bullets - -**`CVExtractedExperience` Model** (new type in `types.py`): -```python -class CVExtractedExperience(BaseModel): - experience_title: str - company: Optional[str] = None - location: Optional[str] = None - start_date: Optional[str] = None - end_date: Optional[str] = None - work_type: Optional[str] = None # maps to WorkType enum key - responsibilities: list[str] = Field(default_factory=list) - source: str = "cv" # provenance marker -``` - -**LLM Prompt Design**: The `CVStructuredExtractor` system instructions should ask Gemini to return: -```json -{ - "experiences": [ - { - "experience_title": "Project Manager", - "company": "University of Oxford", - "location": "Oxford, UK", - "start_date": "2018", - "end_date": "2020", - "work_type": "FORMAL_SECTOR_WAGED_EMPLOYMENT", - "responsibilities": ["Led 5-person team", "Managed £200k budget"] - } - ], - "qualifications": [...] // (extracted in B4) -} -``` - -**Pipeline Update** (in `CVUploadService._pipeline`): -1. CONVERTING → markdown -2. EXTRACTING → bullet extraction (existing, kept for backward compat) -3. **NEW**: STRUCTURING → structured extraction (produces `list[CVExtractedExperience]`) -4. UPLOADING_TO_GCS → storage -5. COMPLETED - -**Acceptance Criteria**: -- [ ] `CVStructuredExtractor` produces `CVExtractedExperience` objects from CV markdown -- [ ] Pipeline stores structured experiences in `user_cv_uploads` record -- [ ] Existing bullet extraction is preserved (backward compat) -- [ ] Unit tests: 3+ real CV markdowns → verified structured output - ---- - -### Task 3.2: CV-to-Agent State Mapper (P0) - -**Problem**: Even with structured CV data, there's no bridge to the agent's `CollectedData` state. When a conversation starts, the agent has no knowledge of the uploaded CV. - -**What**: Create a mapper that converts `CVExtractedExperience` → `CollectedData` and a loader that pre-populates the `CollectExperiencesAgent` state. - -**Files to Create**: -- `backend/app/users/cv/cv_to_agent_mapper.py` — Mapping logic + deduplication - -**Files to Modify**: -- `backend/app/conversations/service.py` — Before first conversation turn, check for completed CV uploads and pre-populate agent state -- `backend/app/agent/collect_experiences_agent/_types.py` — Add `source: Optional[str] = None` field to `CollectedData` (provenance: "cv" | "conversation" | None) -- `backend/app/agent/collect_experiences_agent/collect_experiences_agent.py` — Add `set_cv_experiences()` method - -**Mapping Logic** (`cv_to_agent_mapper.py`): -```python -def map_cv_to_collected_data(cv_experiences: list[CVExtractedExperience], - existing_data: list[CollectedData]) -> list[CollectedData]: - """ - Convert CV experiences to CollectedData, deduplicating against - any already-collected conversational data. - """ -``` - -- Map `work_type` string → `WorkType` enum key, defaulting to `FORMAL_SECTOR_WAGED_EMPLOYMENT` -- Mark fields from CV as populated (not None), so agent won't re-ask -- Set `source="cv"` for provenance tracking -- Deduplicate using `CollectedData.compare_relaxed()` against existing data - -**Integration in `ConversationService.send()`**: -```python -# On first turn, check for completed CV uploads -if is_first_turn and cv_upload_exists: - cv_experiences = await cv_repository.get_structured_experiences(user_id) - mapped = map_cv_to_collected_data(cv_experiences, state.collected_data) - state.collected_data.extend(mapped) - # Mark relevant work types as partially explored -``` - -**Acceptance Criteria**: -- [ ] CV experiences appear in `CollectExperiencesAgent` state on first turn -- [ ] Deduplication prevents double-counting -- [ ] Provenance field tracks data source ("cv" vs "conversation") -- [ ] Agent state is serializable/deserializable with new field - ---- - -### Task 3.3: Persona 2 Conversational Flow Adaptation (P0) - -**Problem**: When CV data is pre-populated, the `CollectExperiencesAgent` must behave differently — it should acknowledge the CV, confirm extracted info, and only probe for missing details rather than asking everything from scratch. - -**What**: Modify prompts and transition logic so the agent recognizes pre-populated CV data. - -**Files to Modify**: -- `backend/app/agent/collect_experiences_agent/_conversation_llm.py` — Add CV-aware prompt variation -- `backend/app/agent/collect_experiences_agent/collect_experiences_prompt.py` — New prompt section for CV-seeded flow -- `backend/app/agent/collect_experiences_agent/_transition_decision_tool.py` — Adjust transition thresholds when CV data present - -**Prompt Changes** (when CV data detected): -- Opening: "I see from your CV that you've worked as [title] at [company]. Can you tell me more about what you did day-to-day?" instead of "What jobs have you had?" -- Skip basic info questions (title, company, dates) for CV-sourced experiences -- Focus on responsibilities, achievements, and skills not captured in CV -- Still ask about experience types not found in CV (e.g., volunteer work, informal work) - -**Transition Logic Changes**: -- If all CV experiences have confirmed titles + at least one responsibility, the `END_WORKTYPE` threshold should trigger sooner for CV-covered work types -- Still explore `unexplored_types` not represented in CV data (e.g., if CV only has formal employment, still ask about self-employment, volunteer work, unpaid work) - -**Acceptance Criteria**: -- [ ] Agent acknowledges CV data in opening turn -- [ ] Agent skips redundant questions for CV-populated fields -- [ ] Agent still explores work types not found in CV -- [ ] Turn count for Persona 2 with CV: ≤15 turns (down from 20-25 without CV) -- [ ] E2E test: Persona 2 with CV upload completes in fewer turns than without - ---- - -### Task 3.4: CV Confirmation & Edit UI Flow (P1) - -**Problem**: Users should be able to review, correct, and supplement CV-extracted experiences before the agent proceeds with skills exploration. - -**What**: After CV extraction completes, present a summary to the user in the chat and allow inline corrections. - -**Files to Modify**: -- `frontend-new/src/chat/Chat.tsx` — After CV upload completes, display structured experience cards -- `backend/app/conversations/experience/routes.py` — Existing PATCH endpoint works for CV-sourced experiences (no change needed, but verify) -- `backend/app/users/cv/routes.py` — Add `GET /users/{user_id}/cv/{upload_id}/structured` endpoint for structured data - -**Flow**: -1. User uploads CV → polling → extraction completes -2. Frontend displays: "I found N experiences in your CV:" with structured cards -3. Each card shows: title, company, dates, location -4. User can edit inline (uses existing experience PATCH endpoint) -5. User confirms → conversation proceeds with supplementary questions only - -**Acceptance Criteria**: -- [ ] Structured CV data available via API -- [ ] Frontend displays experience cards from CV -- [ ] User can edit CV-extracted experience details -- [ ] Edits persist and are reflected in agent state - ---- - -## B4: Qualifications Extraction + Persistence - -**Objective**: Extract qualifications (certifications, diplomas, artisan qualifications, trade licenses) from CVs and conversation, persist them in the youth profile, and make them available to the recommendation engine for eligibility filtering. - -### Task 4.1: Qualification Entity Model (P0) - -**Problem**: `YouthProfile.qualifications` is `list[dict[str, Any]]` — untyped and unused. No service or repository layer exists. - -**What**: Create a strongly-typed qualification model, MongoDB collection, and repository. - -**Files to Create**: -- `backend/app/qualifications/types.py` — `QualificationEntity` model -- `backend/app/qualifications/repository.py` — `QualificationRepository` (MongoDB CRUD) -- `backend/app/qualifications/service.py` — `QualificationService` (business logic) -- `backend/app/qualifications/routes.py` — REST API endpoints - -**`QualificationEntity` Model**: -```python -class QualificationType(str, Enum): - CERTIFICATE = "CERTIFICATE" # Professional certificates (e.g., CompTIA, CCNA) - DIPLOMA = "DIPLOMA" # Diplomas (e.g., Diploma in Nursing) - DEGREE = "DEGREE" # University degrees (BSc, MSc, PhD) - TRADE_LICENSE = "TRADE_LICENSE" # Artisan/trade licenses (e.g., electrician license) - PROFESSIONAL_LICENSE = "PROFESSIONAL_LICENSE" # Professional licenses (e.g., CPA, nursing license) - TRAINING_COMPLETION = "TRAINING_COMPLETION" # Training program completions - OTHER = "OTHER" - -class QualificationEntity(BaseModel): - uuid: str = Field(default_factory=lambda: str(uuid.uuid4())) - qualification_type: QualificationType - name: str # e.g., "Certificate in Project Management" - institution: Optional[str] = None # e.g., "Kenya Institute of Management" - date_obtained: Optional[str] = None # ISO date or year string - expiry_date: Optional[str] = None # For licenses that expire - level: Optional[str] = None # e.g., "Level 3", "Grade I" - field_of_study: Optional[str] = None # e.g., "Information Technology" - source: str = "conversation" # "cv" | "conversation" -``` - -**MongoDB Collection**: Add `qualifications` to `Collections` class in `database_collections.py`. - -**Repository Methods**: -- `save_qualifications(session_id, qualifications: list[QualificationEntity])` -- `get_qualifications(session_id) -> list[QualificationEntity]` -- `update_qualification(session_id, qualification_uuid, updates)` -- `delete_qualification(session_id, qualification_uuid)` - -**API Endpoints** (register in `server.py`): -- `GET /conversations/{session_id}/qualifications` -- `POST /conversations/{session_id}/qualifications` -- `PATCH /conversations/{session_id}/qualifications/{uuid}` -- `DELETE /conversations/{session_id}/qualifications/{uuid}` - -**Acceptance Criteria**: -- [ ] `QualificationEntity` model with type enum -- [ ] MongoDB repository with CRUD operations -- [ ] REST API endpoints functional -- [ ] `YouthProfile.qualifications` updated to `list[QualificationEntity]` type - ---- - -### Task 4.2: CV Qualifications Extraction (P0) - -**Problem**: The CV extraction pipeline only extracts experiences. Qualifications (degrees, certifications, trade licenses) are ignored. - -**What**: Add qualifications extraction to the CV parsing pipeline alongside the structured experience extraction (Task 3.1). - -**Files to Modify**: -- `backend/app/users/cv/utils/structured_extractor.py` — Extend to extract qualifications -- `backend/app/users/cv/service.py` — Store extracted qualifications -- `backend/app/users/cv/types.py` — Add `CVExtractedQualification` model - -**`CVExtractedQualification` Model**: -```python -class CVExtractedQualification(BaseModel): - name: str - qualification_type: str # maps to QualificationType enum - institution: Optional[str] = None - date_obtained: Optional[str] = None - field_of_study: Optional[str] = None -``` - -**LLM Prompt Extension** (in `CVStructuredExtractor`): -The structured extraction prompt (Task 3.1) already returns both experiences and qualifications in a single LLM call: -```json -{ - "experiences": [...], - "qualifications": [ - { - "name": "Diploma in Business Administration", - "qualification_type": "DIPLOMA", - "institution": "Kenya Institute of Management", - "date_obtained": "2019", - "field_of_study": "Business Administration" - }, - { - "name": "Certified Electrician Grade I", - "qualification_type": "TRADE_LICENSE", - "institution": "NITA Kenya", - "date_obtained": "2021" - } - ] -} -``` - -**Kenya-Specific Qualification Handling**: -- Recognize NITA (National Industrial Training Authority) trade test certificates -- Recognize KNEC (Kenya National Examinations Council) certificates -- Map common Kenyan qualification levels (Grade I/II/III for artisan trades) -- Handle Swahili qualification names (e.g., "Cheti cha..." = Certificate of...) - -**Acceptance Criteria**: -- [ ] CV extraction returns both experiences and qualifications in structured format -- [ ] Kenyan qualification types correctly recognized (NITA, KNEC, trade tests) -- [ ] Qualifications stored in `user_cv_uploads` record and `qualifications` collection -- [ ] Unit tests: CVs with qualifications → verified extraction output - ---- - -### Task 4.3: Conversational Qualifications Extraction (P1) - -**Problem**: Not all qualifications come from CVs. Persona 1 (informal workers) likely have no CV but may have artisan qualifications, trade licenses, or training completions mentioned verbally. - -**What**: Add a lightweight qualifications detection pass to the conversation flow that picks up qualifications mentioned during experience collection. - -**Files to Create**: -- `backend/app/qualifications/extraction_llm.py` — `QualificationsDetector` LLM tool - -**Files to Modify**: -- `backend/app/agent/collect_experiences_agent/_dataextraction_llm.py` — After experience data extraction, run qualifications detection on the same turn -- `backend/app/agent/collect_experiences_agent/collect_experiences_agent.py` — Store detected qualifications in state -- `backend/app/agent/collect_experiences_agent/_types.py` — Add `detected_qualifications: list[QualificationEntity]` to `CollectExperiencesAgentState` - -**Detection Strategy**: -- Run as a secondary extraction on each conversation turn (lightweight — ~100 token prompt) -- Trigger words: "certificate", "diploma", "degree", "license", "qualified", "trained", "certified", "NITA", "Grade I/II/III", "cheti" (Swahili) -- Only extract when confidence is high — don't over-extract -- Deduplicate against already-detected qualifications - -**Acceptance Criteria**: -- [ ] Qualifications mentioned in conversation are detected -- [ ] Detection works for both English and Swahili trigger terms -- [ ] Artisan/trade qualifications recognized (e.g., "Grade I electrician") -- [ ] No false positives from casual mentions (e.g., "it was a good experience") -- [ ] Detected qualifications persisted at end of conversation - ---- - -### Task 4.4: Qualifications → Job Matching Integration (P2) - -**Problem**: Qualifications should affect which jobs are recommended. Some jobs require specific certifications or minimum education levels. - -**What**: Pass qualifications to the recommendation engine for eligibility filtering. - -**Files to Modify**: -- `backend/app/conversations/service.py` — When transferring data to recommender, include qualifications -- `backend/app/agent/recommender_advisor_agent/agent.py` — Accept qualifications context -- `backend/app/database_contracts/db6_youth_database/db6_client.py` — Update `YouthProfile.qualifications` to `list[QualificationEntity]` - -**Integration Points**: -- When `ExploreExperiencesAgentDirector` finishes and hands off to `RecommenderAdvisorAgent`, include qualifications in the context -- Qualifications act as filters: "requires Grade I trade test" → only recommend if user has it -- Qualifications act as boosters: "prefers diploma holders" → rank higher if user has relevant diploma - -**Acceptance Criteria**: -- [ ] Qualifications passed to recommendation engine -- [ ] Jobs requiring specific qualifications are filtered correctly -- [ ] Qualification-based ranking boost works -- [ ] `YouthProfile` persists typed qualifications - ---- - -## C4: Swahili Tests + Evaluation Scripts - -**Objective**: Ensure Swahili language flows maintain quality parity with English and are protected by regression tests. - -### Task 5.1: Swahili Golden Transcripts (P0) - -**What**: Create golden test transcripts for Swahili conversations across both personas. - -**Files to Create**: -- `backend/evaluation_tests/golden_transcripts/swahili/persona_1_simple.json` -- `backend/evaluation_tests/golden_transcripts/swahili/persona_1_multi_experience.json` -- `backend/evaluation_tests/golden_transcripts/swahili/persona_1_artisan.json` -- `backend/evaluation_tests/golden_transcripts/swahili/persona_2_formal.json` -- `backend/evaluation_tests/golden_transcripts/swahili/persona_2_with_cv.json` -- `backend/evaluation_tests/golden_transcripts/swahili/persona_2_code_switched.json` - -**Transcript Design**: -- Persona 1 Simple: Informal worker, single experience, pure Swahili (18-22 turns) -- Persona 1 Multi: Multiple informal experiences, Swahili with some Sheng (30-35 turns) -- Persona 1 Artisan: Trade worker with NITA qualification, Swahili (20-25 turns) -- Persona 2 Formal: Formal employment, Swahili, references CV (20-25 turns) -- Persona 2 CV: Formal with CV upload, Swahili (12-18 turns with CV pre-population) -- Persona 2 Code-Switched: English-Swahili code-switching throughout (25-30 turns) - -### Task 5.2: Evaluation Scripts (P0) - -**What**: Build automated evaluation scripts that measure Swahili-specific metrics. - -**Files to Create**: -- `backend/evaluation_tests/swahili_evaluation_runner.py` -- `backend/evaluation_tests/swahili_metrics.py` - -**Metrics to Capture**: -- Skill discovery accuracy vs English baseline (target: ≥80% parity) -- Language drift rate (agent responding in wrong language) -- Swahili synonym mapping hit rate -- Code-switch handling accuracy -- Qualification extraction accuracy in Swahili - -### Task 5.3: CI Integration (P1) - -**What**: Integrate Swahili tests into the existing CI pipeline. - -**Files to Modify**: -- `.github/workflows/golden_transcript_tests.yml` — Add Swahili test jobs -- `backend/evaluation_tests/check_metrics_thresholds.py` — Add Swahili thresholds - -**Thresholds**: -- Swahili skill overlap vs English: ≥80% -- Language drift: ≤2% of agent turns -- Turn count: within 20% of English equivalent - -**Acceptance Criteria**: -- [ ] 6 Swahili golden transcripts created (3 per persona) -- [ ] Evaluation scripts measure language-specific metrics -- [ ] CI runs Swahili tests alongside English tests -- [ ] Regression protection for both languages -- [ ] Performance benchmarks documented - ---- - -## Deployment Readiness - -**Objective**: Ensure the system is deployable with the Gemini 2.5 Flash model provider and all new M4 features are operationally ready. - -### Task 6.1: Infrastructure & Config (P0) - -**What**: Update deployment configuration for Gemini model provider and new M4 features. - -**Files to Modify**: -- `backend/app/app_config.py` — Add qualifications config fields, ensure CV config complete -- `backend/app/server_dependencies/database_collections.py` — Add `qualifications` collection -- `backend/app/server.py` — Register qualification routes, ensure CV routes enabled - -**New Environment Variables**: -- `BACKEND_QUALIFICATIONS_ENABLED` — Feature flag for qualifications extraction -- `BACKEND_CV_STRUCTURED_EXTRACTION_ENABLED` — Feature flag for structured CV extraction -- `BACKEND_GEMINI_API_KEY` — (verify existing) Gemini API key -- `BACKEND_GEMINI_MODEL_ID` — (verify existing) Model identifier - -### Task 6.2: Secrets & Security Review (P1) - -**What**: Review secrets management and ensure no PII leaks in new features. - -**Deliverables**: -- CV content: verify files stored encrypted in GCS, never logged -- Qualifications: no PII beyond education details, stored in MongoDB -- LLM prompts: verify no CV text forwarded beyond extraction step -- API endpoints: verify authentication required on all new routes -- Update `docs/observability-sensitive-data-checklist.md` with M4 additions - -### Task 6.3: Deployment Documentation (P1) - -**Files to Create**: -- `docs/deployment-runbook-m4.md` — Step-by-step deployment guide - -**Contents**: -- MongoDB collection creation / index setup for `qualifications` -- GCS bucket permissions for CV storage -- Environment variable checklist -- Feature flag rollout order: structured extraction → qualification extraction → CV-agent integration -- Rollback procedures for each feature -- Health check endpoints to verify - -**Acceptance Criteria**: -- [ ] All new environment variables documented -- [ ] MongoDB indexes defined for new collections -- [ ] Secrets management reviewed — no PII in logs -- [ ] Feature flags allow incremental rollout -- [ ] Deployment runbook covers rollout + rollback - ---- - -## Success Criteria - -**CV Integration (Persona 2)**: -- [ ] CV upload → structured extraction pipeline functional -- [ ] Structured experiences (`CVExtractedExperience`) extracted with title, company, timeline, work_type, responsibilities -- [ ] Conversational flow merges with CV data — agent acknowledges CV content -- [ ] Duplicate detection prevents redundant questions (`compare_relaxed` dedup) -- [ ] User can edit/confirm CV-extracted information via experience cards -- [ ] Persona 2 with CV completes in ≤15 turns (vs 20-25 without CV) -- [ ] Provenance tracking: each experience/qualification marked as "cv" or "conversation" sourced - -**Qualifications Extraction**: -- [ ] `QualificationEntity` model with typed enum (`CERTIFICATE`, `DIPLOMA`, `DEGREE`, `TRADE_LICENSE`, etc.) -- [ ] Certifications extracted from CVs via `CVStructuredExtractor` -- [ ] Artisan qualifications recognized (NITA trade tests, Grade I/II/III) -- [ ] Conversational qualifications detection for Persona 1 (no CV) -- [ ] Kenya-specific qualifications handled (NITA, KNEC) -- [ ] Qualifications stored in MongoDB `qualifications` collection -- [ ] Qualifications affect job matching eligibility (filter + ranking boost) - -**Persistence & Data Quality**: -- [ ] All experiences saved to database with provenance ("cv" | "conversation") -- [ ] All skills persisted with provenance -- [ ] All qualifications linked to `YouthProfile` -- [ ] Data validation ensures completeness (no null titles, valid enum values) -- [ ] `DB6Client` implementation persists typed qualifications (not `dict[str, Any]`) - -**Swahili Testing**: -- [ ] 6 Swahili golden transcripts created (3 per persona, including artisan + code-switch variants) -- [ ] Evaluation scripts measure skill overlap, language drift, mapping hit rate -- [ ] CI regression tests cover both English and Swahili -- [ ] Swahili skill discovery ≥80% parity with English baseline -- [ ] Performance benchmarks documented for Swahili flows - -**Deployment Readiness**: -- [ ] Gemini 2.5 Flash config verified in deployment environment -- [ ] MongoDB `qualifications` collection with indexes -- [ ] GCS bucket configured for CV storage -- [ ] Feature flags enable incremental rollout (structured extraction → qualifications → CV-agent merge) -- [ ] Secrets review: no CV content or PII in logs -- [ ] Deployment runbook with rollout + rollback procedures - ---- - -# MILESTONE 5: Hardening + Handover - -**Objective**: Finalize robustness, operational readiness, and transition to support. - -## B5: Safety/Edge Case Simulation Suite - -**Tasks**: TBD - -## Hardening Across Persona 2 + Swahili Flows - -**Tasks**: TBD - -## A5: Handover/Support Plan - -**Tasks**: TBD - ---- - -## Success Criteria - -**Safety & Edge Cases**: -- [ ] Safety simulation suite integrated into CI -- [ ] Off-topic detection prevents harmful/sensitive conversations -- [ ] Edge case tests cover empty inputs, very long inputs, code-switching, profanity -- [ ] Graceful degradation for model failures - -**Robustness & Hardening**: -- [ ] Persona 2 flow hardened with error handling -- [ ] Swahili flow hardened with fallback mechanisms -- [ ] Performance under load tested -- [ ] Memory leaks and resource issues resolved - -**Operational Readiness**: -- [ ] Logging reviewed for completeness and compliance -- [ ] Backup and recovery procedures documented - -**Documentation**: -- [ ] Handover plan completed (architecture docs, code walkthrough, knowledge transfer, support escalation) diff --git a/frontend-new/README.md b/frontend-new/README.md index 0e4e2333..01c9d6dc 100644 --- a/frontend-new/README.md +++ b/frontend-new/README.md @@ -1,4 +1,4 @@ -# Compass Frontend +# Compass Connect Frontend The deployed version can be found here: [Frontend URL](https://dev.compass.tabiya.tech) ## Prerequisites diff --git a/frontend-new/sensitive-data-fields-config.md b/frontend-new/sensitive-data-fields-config.md index 5306385d..b497d1db 100644 --- a/frontend-new/sensitive-data-fields-config.md +++ b/frontend-new/sensitive-data-fields-config.md @@ -1,6 +1,6 @@ # Sensitive Data Fields Configuration -This document describes the schema for configuring sensitive data fields collected from users in Compass. +This document describes the schema for configuring sensitive data fields collected from users in Compass Connect. ## Overview diff --git a/iac/README.md b/iac/README.md index 48e231e7..6ffc1bd4 100644 --- a/iac/README.md +++ b/iac/README.md @@ -1,13 +1,13 @@ # Infrastructure as Code -The infrastructure for Compass is managed using [Pulumi](https://www.pulumi.com/). The infrastructure is defined in code and can be deployed to Google Cloud +The infrastructure for Compass Connect is managed using [Pulumi](https://www.pulumi.com/). The infrastructure is defined in code and can be deployed to Google Cloud Platform (GCP) using Pulumi. ## IaC Codebase Components ### Structural Diagram -The diagram below shows a high level overview of the infrastructure architecture of the Compass project. -![Compass IAC Structure Overview](https://lucid.app/publicSegments/view/fd859d0a-d6e4-44cc-8925-4e201941e77b/image.png) +The diagram below shows a high level overview of the infrastructure architecture of the Compass Connect project. +![Compass Connect IAC Structure Overview](https://lucid.app/publicSegments/view/fd859d0a-d6e4-44cc-8925-4e201941e77b/image.png) ### Keywords