Blueprint Bench

A workbench for packaging AI agent skills before they ship to production. A skill here is a prompt plus guardrails, the same unit a "skills management layer" would version and deploy to enterprise customers. Blueprint Bench takes a skill through four steps: define it, audit it for clarity and safety, test it against real inputs, then package it into a versioned JSON manifest.

Live Demo → blueprint-bench.vercel.app

Built with Next.js 14, TypeScript, Tailwind and the Gemini API.

Why this exists

This is a working prototype of the product problem behind an "Agent Blueprints" role: enterprises don't just need AI agents, they need a repeatable way to define, review and ship AI skills the way they'd ship any other software component. See docs/design-decisions.md for the reasoning behind specific choices.

Running locally

npm install
cp .env.local.example .env.local
# add your Gemini API key to .env.local
npm run dev

Open http://localhost:3000.

Get a Gemini API key from Google AI Studio.

Deploying to Vercel

1. Push this repo to GitHub.
2. Import it in Vercel.
3. Add GEMINI_API_KEY under Project Settings → Environment Variables.
4. Deploy.

How it works

• Define: name, category, description, prompt, guardrails. Stored per-blueprint in the browser (lib/storage.ts), no database in this version.
• Audit: app/api/evaluate/route.ts sends the prompt and guardrails to Gemini and asks for a 0-100 score across clarity, safety, scope and completeness, plus a short list of concrete risks.
• Test: each test case runs in two calls. First Gemini plays the agent under the defined prompt and guardrails and produces a real response. Second, a separate judge call grades that response against the expected behaviour. This checks what the skill actually does, not just whether the prompt reads well.
• Package: bumps the version, sets status to tested or production-ready based on the audit score and test pass rate, and downloads a JSON manifest with the full blueprint definition.

What's not built

• No database or auth. State lives in the browser via localStorage, so it resets per device. A real skills registry would need a backend and multi-user access control, closer to what's sketched in ai-governance-console.
• No batch testing or CI integration. Test cases run one at a time, on demand.
• The maturity score is a single model's judgement, not a calibrated rubric. Worth cross-checking against a second model or a human reviewer before treating it as a gate.

Related

• ai-governance-console: feature registry, policy engine, approval queue and audit trail for AI features at a portfolio level.