deepeval-metrics

🧪 AI-powered QA Automation Tool that generates, executes, and evaluates API test cases using LLMs, LangGraph, and DeepEval with intelligent error handling and automated test evaluation.

Drop-in TruLens evaluation harness for tool-calling LangGraph agents. Swap LLM providers (OpenAI, Anthropic via LiteLLM, Bedrock, Cortex, Gemini, Ollama) with a single env var. Ships with the RAG Triad plus Plan Quality, Plan Adherence, Execution Efficiency, and Logical Consistency metrics.

Pluggable DeepEval scaffold for RAG, agents, and LLM apps across Anthropic, Bedrock, Azure OpenAI, and Vertex. Ships traceability, test synthesis, safety/PII gating, multi-turn conversation eval, agentic tool-use scoring, JSON validation, judge benchmarks, hyperparameter sweeps, and pytest CI — one Makefile target per feature.

📚 Stateful, self-correcting RAG research assistant built with LangGraph & Streamlit. Features hybrid search (dense + BM25), Cohere reranking, Tavily web/arXiv claim verification, session-isolated Qdrant storage, Guardrails AI, DeepEval testing, LangSmith tracing, /btw off-topic channel and AWS Fargate deployment configured with Terraform & CI/CD.

READ-MAS generates software requirements specifications (SRS) and system designs from natural language queries using multi-agent AI orchestration built on Google ADK.

Improves a production agent system by repairing the evals that judge it. Failures are mined into new eval cases, and gates on both sides control what enters: false negatives the suite would never catch, false positives that refuse a correct change. The system improves by having to pass.

A production-ready LLM evaluation framework built with DeepEval and a custom Groq-powered judge (LLaMA 3.3 70B). Evaluates LLM outputs across 5 key metrics — Answer Relevancy, Faithfulness, Hallucination, Toxicity & G-Eval — without requiring an OpenAI API key. Fast, free-tier compatible.

This is a portable lifecycle benchmark harness for comparing AI coding agents on real QA engineering tasks. It rigorously evaluates agent competencies across test authoring, test repair, root-cause debugging, token efficiency, framework adaptation, and complex browser or mobile automation workflows.

LLM-based RAG evaluation pipeline using DeepEval + Ollama to measure retrieval quality with Contextual Precision, Contextual Recall, and automated evaluation.

RAG pipeline with LLM evaluation using DeepEval, Ollama/Groq judge models, and Langfuse observability.

Successfully developed a multi-domain RAG pipeline that retrieves and answers questions grounded in research papers on lung cancer detection, NSCLC microbiome classification, and Bhutan land cover, with a full safety evaluation suite covering toxicity, information leakage, and scope adherence using DeepEval.

RAG Evaluation Suite on Finance Documents

DeepEval LLM evaluation framework using Claude API — tests GEval, Hallucination, and Prompt Alignment metrics on an AI system that rewrites customer issue reports.

DeepEval LLM evaluation framework using Claude API — tests GEval, Hallucination, Answer Relevancy, Faithfulness, Contextual Precision/Recall, Bias, Toxicity, PII Leakage, and Tool Correctness on a mock AI customer support assistant.