Reference frameworks for browser, API, performance, accessibility, visual, and LLM-evaluation quality engineering — built by Prayag Vyas, Senior Quality Engineer (7+ years). This is not a collection of sample tests: each repository is a governed system with explicit architecture, controlled targets, failure evidence, and CI you can audit.
This portfolio exists to support Staff SDET, Lead SDET, QA Architect, and Quality Engineering Lead conversations. Reach out on LinkedIn or through the personal profile.
Start here: five reference frameworks demonstrate deterministic feedback, diagnosable failure, and governed delivery across UI, API contracts, performance, enterprise JVM execution, and LLM evaluation. Open the dashboard, choose a capability below, and verify the current evidence record before reading the deeper architecture.
The frameworks · Choose by capability · Quality model · Operating model · Review paths · Field results
| Framework | What it demonstrates | Evidence |
|---|---|---|
| Verity Policy Coverage Eval Python · pytest · RAG + tool use |
LLM evaluation engineered like software: a three-tier eval pyramid (deterministic gate → semantic evals → adversarial), cassette replay, seeded defects, and judge calibration with bias controls. The hermetic tier runs without provider keys. |  Eval report · v0.1.0 · Review guide |
| Playwright TypeScript TypeScript · Playwright · Zod · Allure |
Modern browser coverage against a repo-owned target: strict typing, reusable fixtures, typed API clients, visual baselines, Axe accessibility checks, sharded Linux/Windows CI, and a governed retry budget with expiring quarantine. |  Allure report · v1.0.0 · Review guide |
| k6 Performance k6 · TypeScript · Grafana · OpenTelemetry |
Performance testing as a governed system: six workload models, SLO-based gates, reviewed regression baselines, Grafana/InfluxDB observability, and load-safety rules that keep public targets read-only. |  Performance reports · v0.4.0 · Evidence guide |
| ARIA API Java 21 · Pact · WireMock · OpenAPI |
API testing above the endpoint-script level: layered clients and services, a deterministic provider, Pact consumer contracts with provider verification, JSON-schema assertions, OpenAPI endpoint coverage, and redacted diagnostics. |  Allure report · v1.0.0 · Review guide |
| Selenium TestNG Java Java 21 · Selenium 4 · TestNG · Docker Grid |
The JVM execution discipline enterprises still run on: thread-local drivers, explicit-wait-only synchronization, Docker Grid with capacity guidance, multi-browser CI, and redaction-aware Allure diagnostics. |  Allure report · v1.0.0 · Review guide |
| If you are evaluating | Start here | What to inspect first |
|---|---|---|
| AI/LLM quality | Verity | Calibration, replay, adversarial evidence, and explicit live limitations. |
| Modern UI, API, accessibility, and visual testing | Playwright | Controlled target, typed clients, cleanup, and flake governance. |
| Performance governance | k6 | Safety guard, SLOs, baseline provenance, and regression comparison. |
| API and contract quality | ARIA | Owned provider, Pact, OpenAPI, and redacted diagnostics. |
| Enterprise JVM execution | Selenium | Grid lifecycle, waits, parallelism, and failure diagnostics. |
Each repository owns one surface of a layered quality model, and they are meant to be read together: Playwright proves modern browser, visual, and accessibility coverage; ARIA proves API behavior at the contract level; k6 proves performance evidence under governed load; Selenium proves the JVM execution patterns large teams still depend on; and Verity applies the same determinism and evidence standards to the newest surface — LLM evaluation. One quality strategy, demonstrated five ways.
| Time | Path | The question it answers |
|---|---|---|
| 3 minutes | Open the evidence dashboard — refreshed on a schedule from live CI metadata, validated before publish, readable without JavaScript. | Is the CI green, are the releases real, and is the evidence current? |
| 15 minutes | Follow the Verity reviewer guide and the Playwright review guide. | How are LLM non-determinism and browser flakiness actually controlled? |
| Deep review | Work through all five in order — Verity → Playwright → k6 → ARIA → Selenium — each with a documented, time-boxed review path. | Does the same engineering standard hold across every quality surface? |
The documentation workflow in this repository rejects a small, explicit list of unsupported self-assessment terms. A separate evidence contract validates each repository's default-branch run, SHA, verification record, and freshness. These are repository-local and portfolio-contract checks; they are not semantic validation of every organization claim. See the portfolio evidence standard.
These numbers come from one client platform migration, documented in the anonymized platform scaling case study — including the decision record behind them: stop optimizing for raw test count, keep a small set of business-critical E2E journeys, and push repeatable validations down to API level. The fixture, data-builder, and reporting conventions behind those numbers are implemented publicly in the Playwright, ARIA, and k6 repositories.
Every repository was built over a focused, multi-week stretch, working with
AI-assisted engineering tools rather than around them — the same practice
this portfolio treats as a core quality-engineering skill rather than a
shortcut. The frameworks, evidence contract, and governance patterns were
directed and reviewed by hand: architecture decisions, contract boundaries,
threshold placement, and the honesty rules (labeled limitations, quarantined
gates, NOT_REPRODUCED as a distinct status) are engineering judgment calls,
not generated defaults. Treat the commit history and CI runs, not the
calendar, as the evidence of what was actually built and verified.
Reviewing this portfolio? Open the evidence dashboard, pick a framework, inspect its latest CI run and failure evidence, then run its documented local quality gate. Everything above is designed to be challenged.
