Skip to content

api-testing.mdx

Latest commit

 

History

History
80 lines (61 loc) · 4.04 KB

api-testing.mdx

File metadata and controls

80 lines (61 loc) · 4.04 KB
Raw
title API testing
description Create reusable API tests from OpenAPI specs, Postman collections, cataloged endpoints, or natural-language briefs.
icon plug

{/* source: qodeclaw/product.md (API support, Operating Modes); qodeclaw/PRODUCT-CAPABILITIES.md (sections 2, 6, 10); qodeclaw/TESTING-PLATFORM-ARCHITECTURE.md (API Surface, Scenario Generation, API Playground). */}

API testing

Qodex helps you create reusable API tests from specs, collections, cataloged endpoints, or plain-English requests.

You can import an OpenAPI spec or Postman collection, let Qodex catalog every endpoint, then ask the agent to create scenarios for happy paths, validation errors, auth failures, IDOR checks, and other API behaviors. Saved scenarios replay as normal HTTP requests, without an LLM call on every run.

How API testing works

The API testing flow has four parts:

  • Catalog endpoints from OpenAPI, Swagger, Postman, UI discovery, or manual requests.
  • Create scenarios from chat, the API Playground, or manual authoring.
  • Verify scenarios immediately against your staging environment when they are saved.
  • Run scenarios on demand, on a schedule, through webhooks, or in CI.

What you can start with

Upload an OpenAPI 3.x or Swagger 2.0 document and let Qodex catalog every endpoint. Bring a Postman v2.1 collection in with auth and folder structure preserved. Reusable API tests with steps, assertions, captures, tags, and lifecycle state. A Postman-style runner on every endpoint in your project, with environment interpolation and deep links.

Why replays are cheaper

Most AI testing tools charge per replay because every run is an LLM call. Qodex uses the LLM when a scenario is created or repaired. After that, replay is a plain HTTP request against your environment.

That means nightly regression, webhook runs, and CI runs scale with HTTP execution cost, not token cost.

What you can do today

  • Import OpenAPI 3.x, Swagger 2.0, or Postman v2.1 collections.
  • Catalog every endpoint with coverage status (tested, untested, failing).
  • Author scenarios from chat or from the API Playground.
  • Run scenarios as multiple roles using auth profiles (admin, regular user, viewer).
  • Chain steps via captures and reference earlier-step values in later requests.
  • Generate request data automatically or supply it manually.
  • Write test rules in plain English and let Qodex convert to JavaScript.
  • Run scenarios on demand, on a schedule, or via webhook.

Where to go next

Run the same scenario as admin, user, viewer, or any custom role. See pass or fail the moment you save. Coverage status across every endpoint Qodex has seen. Pass tokens and IDs between steps without glue code.

On the roadmap

Self-critique on save (intelligence track): a second LLM reviews every generated scenario against its goal before persistence, with verdict approve / revise / reject and a strength score. Weak scenarios are flagged for humans instead of silently shipped. Findings-aware generation pulls past findings and existing scenarios for the specific endpoint into the authoring prompt. Source-aware generation reads your repo's route table once GitHub is linked, so the agent tests real handlers, not OpenAPI guesses.