AgentGate

Guardrail checks for AI-agent-generated pull requests.

AgentGate runs in CI on every PR, inspects the diff for the risk signals that AI agents commonly introduce — leaked secrets, out-of-scope changes, missing tests, surprise dependencies — posts a structured review comment, and sets a pass/fail check. Your team gets eyes on agent work without rubber-stamping it.

Built by Brett Buskirk LLC as part of the Agentic Development Workflow Setup service — a productized safety net for teams shipping with AI coding agents.

💡 Not an engineer? Read the plain-language overview — what AgentGate does and why it matters, no jargon.

📝 The story behind it: A Seatbelt for AI-Generated Pull Requests — why agent PRs need a guardrail, how it was built, and where it's headed.

---

Status

v1.2.1 — stable. Six deterministic rules — including default protection for its own .agentgate.yml config — plus an opt-in LLM intent check, tested (99%+ coverage, enforced in CI). Action bundled (dist/index.js) and listed on the GitHub Marketplace. Dogfood CI runs AgentGate on its own PRs. CLI auto-detects your default branch and ships an init scaffolder. Published to npm as @brett.buskirk/agent-gate.

---

See it in action

When an agent opens a PR that leaks a key, edits a workflow it shouldn't touch, slips in a dependency, and skips tests, AgentGate blocks the merge and explains exactly why:

In CI it posts the same verdict as a single PR comment — here's a sample comment.

---

Quickstart

GitHub Action

# .github/workflows/agentgate.yml
name: AgentGate
on: [pull_request]

jobs:
  agentgate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: brett-buskirk/agent-gate@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

Pin to an exact release (@v1.0.0) for reproducible builds, or track the moving @v1 tag for the latest v1.x.

Add a .agentgate.yml to your repo to configure it (or skip it — the defaults are sane).

CLI

# Auto-detects your default branch (main, master, or origin/HEAD)
npx @brett.buskirk/agent-gate check

Or install globally:

npm install -g @brett.buskirk/agent-gate
agent-gate check                  # diff against the auto-detected default branch
agent-gate check --base develop   # or pick a base explicitly
agent-gate check --json           # machine-readable output
agent-gate init                   # scaffold a .agentgate.yml

---

Configuration

Place a .agentgate.yml in your repo root. Everything has a default — the file is optional.

version: 1
fail_on: error          # error | warning | never
comment: true           # post/update a PR comment

rules:
  secrets:
    enabled: true
    severity: error

  scope:
    enabled: true
    severity: error
    allow:               # if set, only these paths are permitted
      - "src/**"
      - "test/**"
      - "docs/**"
    deny:                # always blocked regardless of allow
      - ".github/workflows/**"
      - "infra/**"
      - "**/*.lock"
      - "package-lock.json"

  diff_size:
    enabled: true
    severity: warning
    max_files: 30
    max_lines: 800

  tests_required:
    enabled: true
    severity: warning
    src_globs: ["src/**"]
    test_globs: ["**/*.test.*", "**/*.spec.*", "tests/**"]

  dependencies:
    enabled: true
    severity: warning
    manifests: ["package.json", "requirements.txt", "go.mod", "Gemfile", "Cargo.toml"]

  dangerous_patterns:
    enabled: true
    severity: error
    patterns:
      - "eval\\("
      - "--no-verify"
      - "child_process\\.exec\\("

  intent:                 # opt-in — needs ANTHROPIC_API_KEY
    enabled: false
    severity: warning
    model: claude-haiku-4-5-20251001
    max_diff_bytes: 60000

fail_on

Value	Behavior
error	Only error-severity findings fail the check (default)
warning	Warnings also fail the check
never	Check always passes; findings are still reported

---

Rules

Rule	Default severity	What it catches
secrets	error	AWS keys, GitHub tokens, private key blocks, high-entropy assignments
scope	error	Files outside the allow list or inside the deny list
diff_size	warning	PRs exceeding max_files (30) or max_lines (800)
tests_required	warning	Source changes with no corresponding test file changes
dependencies	warning	Modified dependency manifests (supply-chain risk)
dangerous_patterns	error	User-defined regex denylist applied to added lines
intent (opt-in)	warning	Uses an LLM to flag changes that go beyond the PR's stated description

---

LLM intent check (optional)

The intent rule is the one check that isn't deterministic: it asks Claude whether your diff actually does what the PR says it does — catching an agent that quietly did more, or other, than the description claims. AI checking AI.

Here it is catching a PR titled "fix a typo in the README" that actually added a whole new module:

It's off by default. Turn it on in .agentgate.yml:

rules:
  intent:
    enabled: true

and give it an Anthropic API key — the ANTHROPIC_API_KEY env var, or the anthropic-api-key Action input:

- uses: brett-buskirk/agent-gate@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}

Locally, supply the intent yourself:

agent-gate check --intent "Add a refund flow to the payment processor"
agent-gate check --intent-file PR_DESCRIPTION.md

Notes:

• Never blocks on infrastructure — a missing key or a failed API call is reported as info, not a failure; your deterministic gate still runs.
• Bounded cost — the diff is summarized and truncated to max_diff_bytes; the default model (claude-haiku-4-5-20251001) is fast and cheap.
• Default severity is warning — LLM judgment is advisory; bump it to error if you want it to block.

---

PR Comment

AgentGate posts a single comment on the PR and updates it in place on re-runs — never spams. It shows the overall verdict, a rule-by-rule summary table, and expandable findings with a file, line, and fix for each.

When a PR trips multiple rules, every finding is grouped with its suggestion:

A clean PR passes quietly:

---

How it works

1. On a pull_request event, the Action fetches the PR diff from the GitHub API
2. The diff is parsed into a structured model (files, chunks, added/removed lines)
3. Each enabled rule runs over the model and returns findings
4. The engine aggregates findings and computes a verdict based on fail_on
5. Reporters post the PR comment, set the check status, and write the Step Summary
6. The check fails if the verdict is fail — blocking merge until the agent's work is reviewed

The CLI (agent-gate check) uses git diff instead of the GitHub API, making it usable locally and in pre-commit hooks.

---

Project structure

agent-gate/
  action.yml              # GitHub Action metadata
  src/
    cli.ts                # CLI entry (commander)
    action.ts             # Action entry
    engine.ts             # Aggregates rules → verdict
    diff/                 # Diff providers + parser
    rules/                # Rule implementations
    report/               # Reporters (comment, check, summary, CLI)
    config/               # Schema (zod) + loader
    utils/                # Glob matching
  test/
    fixtures/             # Sample diffs (clean + dirty per rule)
    rules/                # Rule unit tests
    engine.test.ts
  docs/
    ABOUT.md              # plain-language overview (non-technical)
    DESIGN.md             # architecture & internals
    RELEASING.md          # release + Marketplace runbook
    SPRINTS.md            # sprint plan

---

Stack

Layer	Technology
Language	TypeScript 5, strict mode
Runtime	Node 20+
Action bundler	@vercel/ncc
Config validation	zod
Config format	js-yaml
CLI	commander
GitHub API	@actions/github (Octokit)
Tests	Vitest

---

Contributing

See CONTRIBUTING.md. New rules are the most welcome contribution — there's a dedicated issue template and a clear pattern to follow.

---

License

MIT — see LICENSE.