Agnox

The AI Quality Orchestrator for modern engineering teams.

Agnox is a unified platform that integrates with your CI/CD pipelines, executes tests across any framework, and delivers actionable AI-powered insights — directly where your team works.

---

The Problem

Test automation teams waste hours every day on:

• Debugging failures — manually sifting through thousands of log lines with no clear root cause
• Flaky tests — intermittent failures that erode confidence but are hard to identify and quarantine
• Wasted CI time — running the full test suite on every PR, even when only a handful of files changed
• Fragmented tooling — different frameworks, environments, and teams with no unified view

---

The Solution

Agnox sits between your CI/CD pipeline and your test framework, bringing everything together in one platform:

Capability	What it does
AI Triage	Dual-agent LLM pipeline analyzes failures, generates a root cause, and self-critiques to eliminate hallucinations
Flakiness Detective	Detects unstable tests across historical runs, scores flakiness, and surfaces actionable recommendations
Stability & Quarantine	Stable per-test identity across runs, stability grades, opt-in auto-quarantine and a one-click manual quarantine — quarantined-only failures no longer block your pipeline
Failure Correlation & Locator Healing	Clusters recurring failures across runs and classifies their cause; suggests verified replacement locators for broken selectors
Smart PR Routing	An AI reads your changed files and dispatches only the relevant tests — not the full suite
Test Cases	Structured test repository with nested folder hierarchy, human-readable IDs (AG-TC-{n}), severity levels, case types, tags, and shareable deep links to any test case view
Test Optimizer	A dual-agent analyzer rewrites test cases into clean BDD format and removes duplicate steps
Quality Chatbot	Ask natural-language questions about your test data — powered by a sanitized, AI-generated aggregation pipeline
Auto Bug Generation	One click generates a structured bug report pre-filled with the AI's root cause analysis, ready to submit to Jira, Linear, or Monday.com
Spec-to-Test Generation	Upload a PDF, DOCX, or Markdown spec — a 4-stage agentic pipeline (Extractor → Generator → Critic → Formatter) streams progress via SSE and generates draft test cases directly into your suite
Automation Planner	Turns your manual test library into a prioritised automation roadmap — AI scores each test's risk level, you select up to 30, and it generates a complete Markdown automation strategy document ready to paste into Cursor or GitHub Copilot
Framework-Agnostic	Bring your own Docker image — Playwright, Cypress, Pytest, JUnit, Selenium, or any custom framework
Multi-Tenant SaaS	Complete data isolation, RBAC, and per-organization quotas and billing
Notifications	Configurable Slack and MS Teams webhooks deliver colour-coded execution summaries with AI analysis snippets and deep links to the Investigation Hub — each channel is independent and per-status configurable
Linear Integration	Create Linear issues from failed executions in one click; bidirectional linkage keeps your issue tracker in sync with your test results
Monday.com Integration	Create Monday.com board items from the Investigation Hub or Auto-Bug Generator; description attached as an update on the item; bidirectional mondayItems[] linkage on each execution
Scheduling	CRON schedules with next-run and last-run status, pause/resume, edit and Run Now
Generic Outbound Webhook	Fire-and-forget execution.finished event POSTed to any HTTPS endpoint on execution completion with optional HMAC-SHA256 signing

---

Core AI Features

AI-Powered Root Cause Analysis

When tests fail, Agnox captures logs automatically and runs a dual-agent LLM pipeline:

1. An Analyzer agent produces a structured root cause and suggested fix as JSON
2. A Critic agent evaluates the analysis against the raw logs and overrides hallucinations
3. The final, vetted analysis is rendered as developer-facing Markdown in the Investigation Hub

Your LLM key stays yours. Agnox is BYOK-only (Bring Your Own Key): AI features run on your organization's own Gemini, OpenAI or Anthropic key, stored encrypted. Add the key in Settings → AI Models to enable AI — it unlocks every model tier, including deep-reasoning models, on every plan.

Flakiness Detective

Agnox scores each test group's flakiness over time and delivers a structured verdict with findings and recommendations — so you can quarantine unstable tests before they derail your pipeline.

Smart PR Routing

Connect Agnox to your GitHub repository via a webhook. When a PR fires:

1. Agnox extracts the changed files from the payload
2. An LLM maps the changes to the most relevant test subfolder
3. A targeted test run is dispatched — not the full suite
4. The execution appears in the Dashboard with a Smart PR badge and full CI context

# GitHub Actions example
- name: Trigger Smart PR Tests
  run: |
    curl -X POST ${{ secrets.AGNOX_API_URL }}/api/webhooks/ci/pr?token=${{ secrets.AGNOX_ORG_ID }}&env=staging \
      -H "Content-Type: application/json" \
      -H "X-Hub-Signature-256: sha256=$(echo -n '${{ toJson(github.event) }}' | openssl dgst -sha256 -hmac '${{ secrets.WEBHOOK_SECRET }}' | cut -d' ' -f2)" \
      -d '${{ toJson(github.event) }}'

Quality Chatbot

Ask questions about your test data in plain English — "What is the flakiness rate for the checkout group this week?" — and get a clear answer backed by real aggregation data from your executions.

---

Supported Frameworks

Agnox is framework-agnostic. Any Docker-based test runner works out of the box:

Framework	Docker Image Example	Command
Playwright	mcr.microsoft.com/playwright:v1.40.0	npx playwright test
Cypress	cypress/included:13.6.0	cypress run
Pytest	python:3.11-slim	pytest --html=report.html
JUnit	openjdk:17-slim	mvn test
Selenium	selenium/standalone-chrome:latest	pytest tests/
Custom	Any Docker image	Any command

---

Connecting Your Project

There are two ways to connect your automation project to Agnox.

Option A: The Agnox CLI (Docker-Hosted Runs)

The Agnox CLI scaffolds a production-ready Docker setup for your test repo in seconds.

npx @agnox/agnox-cli@latest init

What it does:

• Generates a Dockerfile, entrypoint.sh, and .dockerignore tailored to your framework
• Auto-detects your Playwright version from package.json and pins the correct base image
• Builds a multi-platform Docker image (linux/amd64 + linux/arm64) and pushes it to Docker Hub

After running the CLI:

1. Open the Agnox Dashboard
2. Go to Settings → Run Settings
3. Create a project and enter the Docker image name
4. Configure your environment URLs (Dev / Staging / Prod)

The platform uses this image for all runs triggered from the Execution Modal or your CI pipeline.

Updating Your Image & CI/CD:

To update your Docker image seamlessly (e.g., in your CI/CD pipeline), use the push command:

npx @agnox/agnox-cli@latest push -i <image_tag>

This allows you to update the connected image tag programmatically, ensuring Agnox always executes against your latest test suite version.

npm: @agnox/agnox-cli

---

Option B: Native Playwright Reporter (No Docker Required)

Stream live Playwright results directly to your Agnox dashboard — no Dockerfile, no CI changes required.

npm install --save-dev @agnox/playwright-reporter

// playwright.config.ts
import AgnoxReporter from '@agnox/playwright-reporter';

export default defineConfig({
  reporter: [
    ['list'],
    [AgnoxReporter, {
      apiKey:    process.env.AGNOX_API_KEY,
      projectId: process.env.AGNOX_PROJECT_ID,
    }],
  ],
});

Key features:

• Live ingest — results stream to the dashboard as each test completes
• Auto CI detection — automatically attaches branch, PR number, and commit SHA from GitHub Actions, GitLab CI, Azure DevOps, and Jenkins
• Do No Harm — all reporter errors are silently suppressed; your test suite is never affected
• Source filter — runs appear as External CI in the Dashboard, separate from Docker-hosted runs
• Opt-in Test Cycles (v2.2.0) — add createCycle: true to track a run as a Test Cycle, or cycleId to add it to an existing cycle; by default runs stay out of your Test Cycles list
• Structured failure context (v2.0) — optionally re-export test from @agnox/playwright-reporter in your shared fixtures file; an auto-fixture captures the failure URL, page title, console errors, and failed network requests at the moment of failure and injects them directly into the AI root-cause analysis — no test file modifications required

npm: @agnox/playwright-reporter

---

CI/CD Integration

Trigger Agnox from any CI pipeline using a simple curl call.

GitHub Actions

name: Run E2E Tests via Agnox
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Trigger Tests
        run: |
          curl -X POST https://api.agnox.dev/api/ci/trigger \
            -H "Content-Type: application/json" \
            -H "x-api-key: ${{ secrets.AGNOX_API_KEY }}" \
            -d '{
              "projectId": "${{ secrets.AGNOX_PROJECT_ID }}",
              "image": "myorg/my-tests:latest",
              "command": "npx playwright test",
              "folder": "tests/e2e",
              "config": { "environment": "staging" },
              "ciContext": {
                "source": "github",
                "repository": "${{ github.repository }}",
                "prNumber": ${{ github.event.number || 0 }},
                "commitSha": "${{ github.sha }}"
              }
            }'

GitLab CI

e2e-tests:
  stage: test
  script:
    - |
      curl -X POST https://api.agnox.dev/api/ci/trigger \
        -H "Content-Type: application/json" \
        -H "x-api-key: $AGNOX_API_KEY" \
        -d '{
          "projectId": "'"$AGNOX_PROJECT_ID"'",
          "image": "myorg/my-tests:latest",
          "command": "npx playwright test",
          "folder": "tests/e2e",
          "config": { "environment": "staging" },
          "ciContext": {
            "source": "gitlab",
            "repository": "'"$CI_PROJECT_PATH"'",
            "commitSha": "'"$CI_COMMIT_SHA"'"
          }
        }'

For the full API reference, see docs.agnox.dev.

---

Getting Started

1. Sign up at agnox.dev — your organization is created automatically
2. Generate an API Key in Settings → Profile → API Access
3. Connect your project using the CLI or the Native Playwright Reporter
4. Trigger your first run and watch results stream live in the dashboard

---

Security at a Glance

Agnox is built with security-first principles:

• JWT + RBAC — HS256 with Redis-backed revocation; Admin, Developer, and Viewer roles
• AES-256-GCM — all stored secrets (API keys, integration tokens) are encrypted at rest
• Rate Limiting — per-organization and per-IP limits (Redis-backed)
• HMAC Webhook Verification — constant-time X-Hub-Signature-256 validation for PR Routing webhooks
• 5-Layer AI Sanitizer — every LLM-generated MongoDB pipeline is validated against a strict allowlist before execution, preventing NoSQL injection via AI-generated queries
• Log Sanitization — secret values are automatically redacted from all streamed container logs

---

Documentation & Community

Resource	Link
Full Documentation	docs.agnox.dev
Website	agnox.dev
Agnox CLI on npm	@agnox/agnox-cli
Playwright Reporter on npm	@agnox/playwright-reporter
Contact & Support	info@agnox.dev

---

Contributing

This repository contains the public-facing CLI and documentation for Agnox. We welcome bug reports, documentation improvements, and feature suggestions.

• Found a bug in the CLI? Open an issue in this repository.
• Have a feature idea? Start a discussion or reach out at info@agnox.dev.
• Want to read the full docs? Head to docs.agnox.dev.

---

Built for engineering teams who care about quality.
agnox.dev  ·  docs.agnox.dev