Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

Goal Mode — Writer/Checker Verification Loop for AI Agents

A pattern for running AI agent tasks with a separate writer and verifier. The writer produces output. The checker verifies it against explicit conditions. If the checker fails, the writer iterates. If blocked, it reports honestly.

Core insight: The model that produces the output should not be the same instance that decides it's done.

How it Works

Goal → Writer → Output → Checker → PASS? → Report
                              ↓
                           FAIL (< 3 retries)
                              ↓
                         Writer fixes → Check again
                              ↓
                           FAIL (3 retries) → Report gap
                              ↓
                           BLOCKED → Report gap

Activation Gates

The loop only activates when ALL of these pass:

  1. Persistent side effect — output writes to code, Notion, API, file, or DB
  2. Objectively verifiable — "done" can be checked with a test, tool call, or file read
  3. Error cost > token cost — undetected failure would be expensive
  4. Worth the overhead — task would take 5+ minutes without a loop

Never enters the loop for: answers, summaries, drafts, exploratory work, informational cron jobs, or tasks under 2 turns.

Included

  • goal-mode.skill.md — The operational skill. Load this in an agent to run the writer/checker loop.
  • goal-drafting.skill.md — Reference template for writing verifiable finish line conditions.

Known Failure Modes (Mitigated)

Failure Mitigation
Checker false positive Different model families for writer vs checker
Checker false negative Structured JSON output + specific feedback
Verbosity bias Include length constraints in conditions
Loop instability Hard cap of 3 retries, then escalate
Blocked items Must report "Cannot verify" — not allowed to guess
Checker limited to text Checker has tool access (files, terminal, search)

Production References

This pattern is used in production at:

  • Spotify — hourly playlist moderation (writer + evaluator)
  • Uber — 15-min notification pipeline (writer + verifier, JSON schema reduced false positives from 5% to 0.3%)
  • Datadog — daily monitor comments (switching model families reduced errors by 60%)
  • Stripe — per-PR code review (checker with confidence thresholds)
  • Airbnb — 5-min listing moderation (evaluator with length normalization)
  • Anthropic docs — tool-use evaluation loop as canonical pattern
  • LangGraph — "reflect" node pattern for separate evaluation
  • CrewAI — native verifier agent with retries

License

MIT

About

Writer/checker verification loop for AI agents — goal conditions, activation gates, structured verification, retry limits

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors