Learning Path Designer – Agentic Workflow Orchestrator

Structured RAG + Planner–Executor agents in Go + Python

Live demo: https://learning-path-designer.vercel.app

Why this exists

This repo is a reference architecture for building production-ready AI agents. It demonstrates how to move beyond toy demos to a robust, distributed system featuring:

• Go gateway coordinating multiple AI microservices.
• RAG over domain content with Qdrant + Postgres.
• Deterministic planner & quiz generator using structured outputs.
• Ready to adapt to any domain: onboarding, internal training, certification prep.

Agent Patterns Implemented

• Planner–Executor: Planner proposes a path; executor decomposes into concrete steps with resources.
• RAG + Re-ranking pipeline: Semantic search (e5-base) → Re-ranking (bge-reranker) → Context window construction.
• Structured JSON outputs: All plans & quizzes are strictly validated against schemas.
• Multi-service Orchestration: Frontend (Next.js) → Go Gateway → Python AI microservices → Qdrant + Postgres.
• Multi-tenancy: Achieved via shared Qdrant collections and Postgres tables with tenant_id filtering.
• Bring Your Own Content (BYOC): Users can ingest their own URLs, which are processed with metadata extraction.
• Distributed Tracing: End-to-end observability with OpenTelemetry.

The Planner–Executor Pattern

The core of this agentic workflow is the Planner–Executor pattern. This involves two distinct roles:

1. The Planner (LLM-based): Takes a high-level goal and breaks it down into actionable steps or "milestones." It considers constraints (time budget, existing skills) and available resources to form a coherent learning path. Its output is a structured, human-readable plan.
2. The Executor (Go Gateway / Orchestrator): Takes the structured plan generated by the Planner and orchestrates the execution. This involves:
   • Retrieving specific resources (via the RAG service).
   • Initiating further AI tasks, such as generating quizzes for milestones (via the Quiz service).
   • Ensuring each step's outcome adheres to defined schemas.

This separation of concerns allows for robust, observable, and debuggable AI workflows. The Planner focuses on strategic breakdown, while the Executor handles the tactical fulfillment and ensures data integrity.

Reference Architecture

This project is designed to be forked and adapted. It serves as a blueprint for:

• Internal L&D Teams: Create personalized onboarding paths from your internal wikis and PDFs.
• EdTech Startups: Build adaptive learning platforms with grounded assessments.
• AI Engineers: See a working example of Go-based agent orchestration handling Python ML services.

Architecture

Frontend (Next.js) → Gateway (Go) → AI Services (FastAPI)
                                    ├─ RAG Service
                                    ├─ Planner Service
                                    └─ Quiz Service
                                    
Data Layer: Qdrant (vectors) + Postgres (metadata) + S3 (snippets)
Observability: OpenTelemetry (Metrics, Traces, Logs) with Jaeger

Tech Stack

• Frontend: Next.js 14, TypeScript, Tailwind CSS, shadcn/ui
• Gateway: Go 1.21+, Gin framework
• AI Services: Python 3.11, FastAPI
• Embeddings: e5-base-v2 via Deep Infra API (768 dims)
• Reranker: Qwen3-Reranker via Deep Infra API
• LLM: OpenRouter (Claude/GPT-4)
• Vector DB: Qdrant Cloud
• Database: PostgreSQL (Neon)
• Auth: Supabase Auth
• Deployment: Google Cloud Run + Vercel
• Inference: Deep Infra (serverless embeddings & reranking)

Quick Start

Prerequisites

• Docker & Docker Compose
• Python 3.11+
• Go 1.21+
• Node.js 20+

Local Development

1. Clone and setup:

   git clone <repo-url>
   cd learnPathDesigner
   
   # Run automated setup (Windows)
   .\setup.ps1
   
   # Or manually:
   cp .env.example .env.local
   python -m venv .venv

2. Activate virtual environment:

   # Windows
   .\.venv\Scripts\Activate.ps1
   
   # Linux/Mac
   source .venv/bin/activate

3. Install Python dependencies:

   pip install -r ingestion/requirements.txt
   pip install -r services/rag/requirements.txt
   pip install -r services/planner/requirements.txt
   pip install -r services/quiz/requirements.txt

4. Start infrastructure:

   docker-compose up -d

5. Run migrations:

   # Windows (with psql installed)
   .\ops\migrate.ps1
   
   # Or via Docker
   Get-Content shared/migrations/001_initial_schema.sql | docker exec -i learnpath-postgres psql -U postgres -d learnpath

6. Seed data:

   # Seed skills
   python -m ingestion.seed_skills
   # Setup Qdrant collection (if not already done)
   python -m ingestion.setup_qdrant
   # Ingest seed resources with embeddings and tenant_id="global"
   python ingestion/ingest_via_api.py --api-url http://localhost:8001 --resources --embeddings --tenant-id global
   # Fix durations for old entries
   docker cp ingestion/fix_durations.py learnpath-rag:/app/fix_durations.py
   docker-compose exec rag-service python /app/fix_durations.py

7. Start services (if not using docker-compose up for dev mode):

   # Terminal 1: RAG service
   cd services/rag
   uvicorn main:app --reload --port 8001
   
   # Terminal 2: Planner service
   cd services/planner
   uvicorn main:app --reload --port 8002
   
   # Terminal 3: Quiz service
   cd services/quiz
   uvicorn main:app --reload --port 8003
   
   # Terminal 4: Gateway
   cd gateway
   go run main.go
   
   # Terminal 5: Frontend
   cd frontend
   npm install
   npm run dev

   Note: Make sure your virtual environment is activated in each terminal before running Python services.

8. Access:

   • Frontend: http://localhost:3000
   • Gateway: http://localhost:8080
   • Jaeger UI: http://localhost:16686 (for tracing)
   • RAG Service: http://localhost:8001
   • Planner Service: http://localhost:8002
   • Quiz Service: http://localhost:8003

Project Structure

/frontend              # Next.js application
/gateway               # Go API gateway
/services/
  /rag                 # RAG service (embeddings, search, rerank)
  /planner             # Planner service (plan generation, replanning)
  /quiz                # Quiz service (generation, grading)
/ingestion             # Data ingestion scripts
/shared/
  /migrations          # Database migrations
  /schemas             # Shared data schemas
/ops                   # Docker files, scripts
/.github/workflows     # CI/CD pipelines

Environment Variables

See .env.example for all required variables.

Key variables:

• DATABASE_URL - Postgres connection string
• QDRANT_URL - Qdrant Cloud URL
• QDRANT_API_KEY - Qdrant API key
• OPENROUTER_API_KEY - OpenRouter API key
• SUPABASE_URL - Supabase project URL
• SUPABASE_ANON_KEY - Supabase anonymous key
• DEEPINFRA_API_KEY - Deep Infra API key for embeddings/reranking
• USE_DEEPINFRA - Set to true for API-based inference

Deployment

Google Cloud Run (Backend Services)

The backend uses a lightweight Deep Infra-based deployment (~500MB images vs ~5GB with PyTorch).

# Deploy all services
gcloud builds submit --config=cloudbuild-deploy-mvp.yaml

Services deployed:

• gateway - API gateway (256Mi, 1 CPU)
• rag-service - RAG with Deep Infra embeddings (512Mi, 1 CPU)
• planner-service - Learning path generation (512Mi, 1 CPU)
• quiz-service - Quiz generation (512Mi, 1 CPU)

All services scale to zero when idle for cost efficiency.

Vercel (Frontend)

1. Connect GitHub repository
2. Set root directory to /frontend
3. Set environment variables:
   • NEXT_PUBLIC_API_URL - Gateway URL
   • NEXT_PUBLIC_SUPABASE_URL
   • NEXT_PUBLIC_SUPABASE_ANON_KEY
4. Deploy

Documentation

• ADR-001: RAG Architecture
• ADR-002: Multi-tenancy Strategy

License

MIT