Switchable multi-provider LLM support (Gemini / Anthropic / DeepSeek / Groq) - #1
Open
Divergent-Code wants to merge 3 commits into
Open
Switchable multi-provider LLM support (Gemini / Anthropic / DeepSeek / Groq)#1Divergent-Code wants to merge 3 commits into
Divergent-Code wants to merge 3 commits into
Conversation
The v2 notebook's narrative described behavior the code didn't implement. Three substantive fixes plus two smaller ones: - RAG coverage eval always reported 100%: Chroma's query() returns the nearest top_k chunks regardless of match quality, so every requirement looked "covered" and Step 4 could never surface a real gap. Add a cosine distance threshold (and set the collection to cosine space) so weak matches are dropped and gaps show up as empty evidence lists. - Guardrails were decorative: the injection screen ran in the last cell, after the JD had already been fed through the whole pipeline, and rate_limited_call/logged_call were defined but never invoked. Define the guardrails right after setup, screen the JD at intake before any prompt, route every model call through logged_call, and repurpose Step 8 as a monitoring/audit recap. - Agent didn't index its own chunks: run_resume_tailoring_agent logged "Indexed N chunks" but never added them to Chroma and retrieved against the stale global collection. Add a reusable index_chunks() tool and give the agent its own collection. - Chroma stale state: index_chunks() delete-then-recreates the collection so re-runs don't leave chunks from a previous resume behind. - v1 GitHub call crashed on a nonexistent user (API returns a dict, not a list); guard the response and degrade gracefully. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up robustness fixes in the same two notebooks: - extract_jd_requirements: the model can still emit a preamble line or a section header despite "plain list" instructions; strip bullets/numbering and drop blank, ':'-terminated, or over-long lines so they don't become fake requirements. - Guard the GitHub fetch in the linear path too (the agent already did), so a bad/rate-limited username degrades to "no repo evidence" instead of crashing the cell. - fetch_github_repos: request most-recently-updated repos with an explicit per_page, and document the unauthenticated 60/hr + one-page cap. - Add a PII caveat to the Inputs step — the resume and scraped JD are sent to the Gemini API; fitting for a notebook that teaches AI safety. - V1 notebook ended with a dangling "V1.1" header and no content, while V2 repeatedly references "Phase 1 / V1.1"; replace it with a short summary of what V1.1 added so the progression reads coherently. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…Groq) Replace the hardcoded Gemini calls with a single provider-agnostic generate(prompt, system=None) function selected by a PROVIDER flag: - PROVIDER_CONFIG maps each provider to its Colab secret name and model (gemini-flash-latest, claude-haiku-4-5, deepseek-chat, llama-3.3-70b-versatile). - Anthropic uses the anthropic SDK; DeepSeek and Groq share the OpenAI SDK via base_url; Gemini keeps google-generativeai. Default is Anthropic. - Grounding rules (system_rules) now go in the system slot; generate() folds them into the prompt for providers without a separate system role (Gemini). - Embeddings stay local (all-MiniLM-L6-v2), so RAG retrieval is unchanged regardless of provider. In v2, generate() is defined in Setup and every call already routes through logged_call, so the swap is contained to the wrapper plus three call sites (extract/generate/critique drop the Gemini-specific .text). Markdown and the PII note updated to be provider-agnostic; v1's model-list cell now prints the active provider/model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lets users run the workshop against whichever LLM provider they have a key for, defaulting to Anthropic
claude-haiku-4-5.What changed
A single provider-agnostic
generate(prompt, system=None)replaces the hardcoded Gemini calls. Pick a provider with one flag:PROVIDER_CONFIGmaps each to its Colab secret name + model:GOOGLE_API_KEYgemini-flash-latestgoogle-generativeaiANTHROPIC_API_KEYclaude-haiku-4-5anthropicDEEPSEEK_API_KEYdeepseek-chatopenai(base_url)GROQ_API_KEYllama-3.3-70b-versatileopenai(base_url)system_rules) now go in thesystemslot;generate()folds them into the prompt for providers without a separate system role (Gemini).all-MiniLM-L6-v2), so RAG retrieval is identical whichever LLM you choose.generate()is defined in Setup and every call already routes throughlogged_call, so the swap was contained to the wrapper + three call sites (extract / generate / critique drop the Gemini-only.text).Notes
PROVIDER_CONFIGvalue with a comment to update it if a call 404s (esp. the Groq/DeepSeek model names).fix/rag-eval-and-guardrailswork (PR Fix RAG eval, guardrail ordering, and agent indexing in v2 notebook camunity/ai_resume_builder#6 upstream), so against your fork'smain— which doesn't have those yet — this PR also includes those fixes. Merging it brings both the correctness fixes and multi-provider support onto your fork.🤖 Generated with Claude Code