Feature/migrate workshop3 notebooks

[golldavid]

Migrate Workshop 3 (RAG Evaluation) notebooks

Summary

• Merge ragV3 workshop notebooks (RAG evaluation with RAGAS) into the ragV2 repository
• Rename existing ragV2 notebooks with w2_ prefix, add ragV3 notebooks with w3_ prefix
• Shared setup: one repo, one venv, one .env, one 00_setup.ipynb

Changes

• Notebooks: w2_01–w2_04 (renamed), w3_01–w3_04 (new), config.py, data/ with cache
• Dependencies: added ragas, langchain, langchain-openai, pandas, rapidfuzz
• Setup: 00_setup.ipynb extended with API test + data file checks
• README: restructured with navigation links for Workshop 2 / Workshop 3
• Assets: 00_aisc/img/ logo images for notebook headers

Test plan

• ./scripts/setup_notebooks_kernel.sh completes without errors
• 00_setup.ipynb runs (Qdrant starts, API validates, data files found)
• w2_01_chunking_and_retrieval.ipynb runs (ragV2 still works)
• w3_01_intro_end_to_end.ipynb runs (standalone, no config.py)
• w3_02_ingestion.ipynb runs (from config import * works)
• w3_03_retrieval_evaluation.ipynb runs (RAGAS evaluates)
• w3_04_generation_evaluation.ipynb runs (generation + evaluation works)

Summary by CodeRabbit

Release Notes

• New Features

  • Added two new Workshop 3 notebooks: an end-to-end RAG evaluation workflow and a document ingestion pipeline for chunking and vector storage.
  • Enhanced setup guide with automated environment validation, Docker container management, and API connectivity verification.

• Documentation

  • Updated notebook README with workshop-specific navigation and expanded troubleshooting guidance.

• Chores

  • Added new project dependencies: RAGAS, LangChain, pandas, and RapidFuzz.
  • Updated configuration management and gitignore rules.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR expands workshop infrastructure with centralized configuration, enhanced setup procedures, and two new Workshop 3 notebooks implementing RAG evaluation and document ingestion pipelines. It adds dependencies, configuration module, and cached evaluation metrics for reproducible results.

Changes

Cohort / File(s)	Summary
Project Configuration .gitignore, notebooks/config.py, notebooks/pyproject.toml	Added centralized config.py module exporting repository paths, environment variables, model configuration, and Qdrant settings. Updated .gitignore to ignore workshop directories and added exception for notebooks/data/. Extended dependencies with ragas, langchain, pandas, and rapidfuzz.
Setup & Documentation notebooks/00_setup.ipynb, notebooks/README.md	Transformed 00_setup.ipynb from static guide to executable workflow: added programmatic .env creation, Docker/Qdrant container management, HTTP reachability verification, HPI API endpoint validation (embeddings + generation), and Workshop 3 data file checks. Restructured README.md from quickstart to workshop navigation guide with tables for Workshop 2/3 notebook sequences and expanded troubleshooting for RAGAS/embedding issues.
Workshop 3 Notebooks notebooks/w3_01_intro_end_to_end.ipynb, notebooks/w3_02_ingestion.ipynb	Added w3_01_intro_end_to_end.ipynb implementing complete RAG evaluation workflow: question embedding, vector similarity retrieval, LLM generation, and async RAGAS evaluation (context precision, answer correctness) with manual and synthetic datasets. Added w3_02_ingestion.ipynb for document ingestion: Docling PDF conversion, two chunking strategies (markdown headers + JSON sections), LiteLLM batch embeddings, and Qdrant vector storage with metadata/citations.
Evaluation Cache Data notebooks/data/cache/.../answer_correctness_top5.json, notebooks/data/cache/.../faithfulness_top5.json, notebooks/data/cache/.../generation_responses_top5.json, notebooks/data/cache/.../retrieval_eval_top5.json, notebooks/data/cache/.../retrieval_experiment_topk.json (multiple configurations)	Added precomputed evaluation metrics and cached generation responses across multiple embedding/LLM model combinations (octen-embedding-8b, minilm-embedding, gpt-oss-120b, llama-3-3-70b). Files contain arrays of scored results for reproducible benchmarking.

Sequence Diagram(s)

sequenceDiagram
    participant NB as Notebook<br/>(w3_01)
    participant EMB as Embedding<br/>Service
    participant VS as Vector<br/>Store
    participant LLM as LLM<br/>Service
    participant EVAL as RAGAS<br/>Evaluator

    NB->>NB: Load questions & references
    NB->>EMB: embed_texts(chunks)
    EMB-->>VS: Store vectors in memory
    
    loop For each question
        NB->>EMB: Embed question
        EMB-->>NB: Question embedding
        NB->>VS: Cosine similarity ranking
        VS-->>NB: Top-K chunk contexts
        NB->>LLM: generate(question, contexts)
        LLM-->>NB: Generated answer
        
        NB->>EVAL: Score(contexts, answer,<br/>reference)
        EVAL->>EMB: Verify embeddings for<br/>correctness metric
        EVAL-->>NB: Precision & correctness<br/>scores
    end
    
    NB->>NB: Aggregate metrics<br/>& plot results

Loading

sequenceDiagram
    participant NB as Notebook<br/>(w3_02)
    participant PDF as Docling<br/>PDF Processor
    participant EMB as Embedding<br/>Service
    participant QD as Qdrant<br/>Vector DB

    NB->>PDF: Load & convert PDF
    PDF-->>NB: Markdown content
    
    NB->>NB: Chunk by headers<br/>or JSON sections
    NB->>NB: Normalize OCR text
    
    loop For each chunk batch
        NB->>EMB: Batch embed<br/>(truncated text)
        EMB-->>NB: Embedding vectors
    end
    
    loop For each collection
        NB->>QD: Create collection
        NB->>QD: Upload vectors as<br/>PointStruct(metadata)
    end
    
    NB->>EMB: Embed test question
    NB->>QD: Similarity search
    QD-->>NB: Top results with<br/>scores & citations
    NB->>NB: Print verification

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• PR #3: Modifies overlapping notebook files (00_setup.ipynb, README.md) and .gitignore entries for workshop setup—direct code-level relation to this PR's configuration and documentation changes.

Poem

🐰 Whiskers twitch with RAG delight,
Workshops bloom—from docs to light!
Qdrant hops, embeddings dance,
Config spreads its shared expanse. ✨
Evaluation metrics rise,
Workshop Three now claims the prize! 🏆

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Feature/migrate workshop3 notebooks' directly reflects the main change: migrating Workshop 3 notebooks into the ragV2 repository as part of a feature branch.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/migrate-workshop3-notebooks

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}