EvoSkill-RQGM extends EvoSkill with the Red Queen Gödel Machine (arXiv 2606.26294, Cambridge, June 2026) — a framework for recursive self-improvement where the evaluator co-evolves with the agent, preventing reward hacking and stalled improvement loops.
The problem: Self-improving agents learn to satisfy a fixed evaluator rather than genuinely improving. The moment the judge stops getting harder, the loop stalls and reward hacking creeps in.
The structural answer: Co-evolve the agent AND its evaluator together, so the bar keeps rising as the agent climbs.
| Feature | Vanilla EvoSkill | EvoSkill-RQGM |
|---|---|---|
| Evaluator | Fixed tolerance schedule | Epoch-based utility evolution — tolerances tighten when exploitation detected |
| Scoring | Static multi-tolerance | Adversarial scoring — penalises answers that game loose criteria |
| Failure detection | Hardcoded 0.8 threshold | Adaptive — threshold shifts with evaluator evolution |
| Reward hacking | Not detected | Hack ratio analysis — flags when agent scores well under loose but poorly under strict |
| Checkpoint | Iteration state only | Full epoch state — tolerances, adversarial pool, epoch index |
| Feature flag | — | rqgm_config.enabled=False (safe default, zero behavior change) |
| Domain | Improvement |
|---|---|
| Coding (verifiable tasks) | 1.35x–1.72x fewer tokens than prior SOTA |
| Scientific paper writing | 1.78x–1.86x higher acceptance rates |
| Olympiad-level proof grading | 9% higher ground-truth accuracy |
| Paper reviewing (adversarial) | Corrects 1.91x over-acceptance of AI-generated papers |
The self-improvement loop is divided into epochs, each with a frozen evaluator configuration. At epoch boundaries, the system analyses whether the agent has begun exploiting the current evaluator and, if so, triggers a utility transition:
Epoch 0 (tolerances: [0.0, 0.001, 0.01, 0.025, 0.05, 0.1])
├── Iteration 1: score 0.42
├── Iteration 2: score 0.51
├── Iteration 3: score 0.49
├── Iteration 4: score 0.53
└── Iteration 5: score 0.55
│
└── Boundary check:
├── Hack ratio = 0.48 (strict/loose) → exploitation detected
└── Drop loosest tolerance (0.1) → tighten evaluator
Epoch 1 (tolerances: [0.0, 0.001, 0.01, 0.025, 0.05])
└── ... evaluator gets harder as agent improves
┌─────────────────────────────────────────────────────────┐
│ EvoSkill-RQGM │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Proposer │───▶│Generator │───▶│Evaluator │ │
│ │ (mutate) │ │ (create) │ │ (score) │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ EpochManager │ │
│ │ ┌────────────┐ │ │
│ │ │ Hack Ratio │ │ │
│ │ │ Analysis │ │ │
│ │ ├────────────┤ │ │
│ │ │ Tolerance │ │ │
│ │ │ Evolution │ │ │
│ │ ├────────────┤ │ │
│ │ │ Adversarial │ │ │
│ │ │ Pool │ │ │
│ │ └────────────┘ │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────┘
git clone https://github.com/observeco/EvoSkill-RQGM.git
cd EvoSkill-RQGM
uv syncevoskill initFollow the prompts to select your agent runtime, dataset, and task description.
evoskill run --rqgmOr via config:
# .evoskill/config.toml
[evolution]
mode = "skill_only"
iterations = 20
frontier_size = 3
[rqgm]
enabled = true
epoch_size = 5evoskill runNo config changes needed — RQGM is disabled by default for full backward compatibility.
Add to .evoskill/config.toml:
[rqgm]
enabled = false # Master flag. Set true to activate.
epoch_size = 5 # Iterations per epoch
min_improvement_threshold = 0.02 # Min frontier improvement per epoch
exploitation_hack_ratio_threshold = 0.6 # strict/loose ratio below = hacking
adversarial_high_score_threshold = 0.85 # Loose score threshold for gaming detection
adversarial_strict_threshold = 0.4 # Strict score threshold for gaming detection
selective_erasure_enabled = false # HIGH RISK: invalidate stale frontier scores
cache_flush_on_boundary = true # Flush RunCache when tolerances change
max_adversarial_examples_per_proposer = 5from src.api import EvoSkill
from src.loop.config import RQGMConfig
rqgm_config = RQGMConfig(
enabled=True,
epoch_size=5,
)
evo = EvoSkill(
task="sealqa",
model="sonnet",
mode="skill_only",
max_iterations=20,
rqgm_config=rqgm_config,
)
result = await evo.run()EvoSkill is a toolkit for automatically creating and improving AI agent skills, compatible with Claude Code, Codex CLI, OpenCode, OpenHands, Goose, Harbor, and more.
Turn your general AI agents into state-of-the-art specialists with a benchmark and EvoSkill's self-improvement loop. Install in seconds, then run evoskill init and evoskill run to supercharge any coding agent with AI-created skills and prompts automatically.
| Agent | Support | Notes |
|---|---|---|
| Claude Code | ✅ | |
| OpenCode | ✅ | CLI v1.4.0+ required (structured output support) |
| OpenHands | ✅ | No native structured output; uses fallback JSON extraction |
| Goose | ✅ | CLI v1.25.0+ required (skill discovery via summon extension) |
| Codex CLI | ✅ | Skill discovery via .agents/skills/ symlink |
| Harbor | ✅ | Containerized task benchmarks with built-in verifiers |
| Capability | Status | Explanation |
|---|---|---|
| Co-evolving evaluator (RQGM) | ✅ | Epoch-based utility evolution prevents reward hacking. First open implementation of the Red Queen Gödel Machine. |
| Evolution with a benchmark | ✅ | Skills can be effectively improved against your own or academic benchmarks. |
| Cross-agent transferability | ✅ | Skills are packaged as reusable folders with instructions, metadata, and helper scripts, compatible with many coding agents. |
| Cross-model transferability | ✅ | Demonstrated in EvoSkills, skills evolved with a fixed LLM can transfer their performance increase to other LLMs. |
| Cross-task transferability | ✅ | Generated skills can be generic enough to transfer across tasks, for instance a SealQA skill improving BrowseComp performance (as shown in EvoSkill). |
| Evolution without a benchmark | 🛠️ | An open research direction where benchmarks are generated on the fly (ex. Hermes-Agent self-evolution). |
| Continuous evolution | 🛠️ | Integrating the ability to improve skills from regular usage. |
- Installation
- Quickstart
- RQGM Configuration
- Harbor Integration
- CLI Reference
- Configuration Reference
- How It Works
- Git Branches
- When the Loop Gets Stuck
- Python API
- Citation
- License
One command (recommended):
# Clone + install everything (Python deps, uv, optional agent CLIs)
curl -fsSL https://raw.githubusercontent.com/sentient-agi/EvoSkill/main/install.sh | bash
# Or, if you already cloned the repo:
./install.sh
# Install Python deps + all agent harness CLIs (macOS/Homebrew)
./install.sh --all-agents
# Install Python deps + specific agent CLIs
./install.sh --agents claude,opencodeThe installer handles Python 3.12+, uv, and uv sync automatically. Harbor is included in the Python package. Agent CLIs are optional — install only the harness you plan to use.
Manual install:
Requirements:
- Python 3.12+
uv(recommended) orpip
# Using uv (recommended)
uv sync
# Or using pip
pip install -e .Agent CLI (install whichever harness you plan to use):
brew install --cask claude-code # Claude Code
brew install opencode # OpenCode (v1.4.0+)
brew install --cask codex # Codex CLI
brew install block-goose-cli # Goose (v1.25.0+)Harbor is included in the Python install above (uv sync / pip install -e .). Run this only if you need the Harbor CLI standalone:
pip install harbor # Harbor (containerized benchmarks)Common auth setup:
# Anthropic (Claude Code harness)
export ANTHROPIC_API_KEY=your-key-here
# OpenAI (Codex harness)
export OPENAI_API_KEY=your-key-here
# OpenRouter (OpenCode / Goose / OpenHands harnesses)
export OPENROUTER_API_KEY=your-key-here
# Fireworks AI (OpenCode / OpenHands harnesses, LLM scorer)
export FIREWORKS_API_KEY=your-key-hereOpenRouter-backed evolution runs also accept LLM_API_KEY, but OPENROUTER_API_KEY is the preferred env var.
Run evoskill init inside any git repository:
CSV dataset (question/answer pairs):
$ evoskill init
EvoSkill — Project Setup
Which agent runtime? › claude
Dataset source? › CSV
Absolute path to dataset CSV? › /path/to/questions.csv
Question/input column name? › question
Answer column name? › answer
Category column name? ›
Additional data directories? ›
How do you want to run EvoSkill? › LocalHarbor dataset (containerized benchmark tasks):
$ evoskill init
EvoSkill — Project Setup
Which agent runtime? › claude
Dataset source? › Harbor
Choose a Harbor dataset: › swe-bench/swe-bench-verified
Where to store this dataset? › .evoskill/harbor/datasets/swe-bench-verified
How do you want to run EvoSkill? › LocalThis creates .evoskill/config.toml and .evoskill/task.md.
- Dataset source — CSV (static question/answer pairs) or Harbor (containerized tasks with built-in verifiers).
- Data dirs — (CSV only) absolute paths to directories the agent needs. Comma-separated if multiple.
- Execution mode — Local (direct), Docker (containerized, supports remote via
DOCKER_HOST), or Daytona (managed cloud sandbox).
Edit .evoskill/task.md to describe what the agent should do:
# Task
Answer questions about quarterly financial reports.
Return only the numeric answer with units.
## Examples
- "What was revenue in Q3?" → "$4.2B"
---
# Constraints
- Always include units in the answer
- Do not explain your reasoning, just return the answerWith RQGM (co-evolving evaluator):
evoskill run --rqgmWithout RQGM (vanilla EvoSkill):
evoskill runEvoSkill uses the execution mode you chose during evoskill init (local, Docker, or Daytona). You can override with --docker or --remote flags.
EvoSkill prints a live progress table:
Iter Accuracy Δ Skills Frontier Status
1 42.0% — 0 [1] baseline
2 51.3% +9.3% 1 [1, 2] ★ new best
3 49.7% -1.6% 1 [1, 2] discarded
...evoskill eval # score the best program on the validation set
evoskill skills # list all discovered skills
evoskill diff # see what changed vs baseline
evoskill logs # view past run historyAfter the loop finishes, the best program lives on a git branch:
git branch | grep program/ # list all program branches
git checkout program/iter-skill-3 # switch to the best oneFrom there you can inspect what the loop discovered:
cat .claude/program.yaml # system prompt, tools, score
ls .claude/skills/ # all learned skillsCopy .claude/program.yaml and .claude/skills/ into your deployment to use the evolved agent configuration.
Harbor is a framework for evaluating AI agents against containerized benchmark tasks. EvoSkill integrates with Harbor as an alternative to CSV-based datasets, using Harbor's built-in verifiers as the scoring mechanism.
Instead of running agents against static CSV questions, Harbor mode:
- Loads tasks from a downloaded Harbor dataset (each task has its own Dockerfile, test harness, and verifier)
- Runs
harbor runfor each task, spawning a sandboxed container where the coding agent solves the task - Reads the verifier reward from the container output (0.0 to 1.0)
- Feeds results back into EvoSkill's self-improvement loop to evolve better skills
pip install harbor # install the Harbor CLIRun evoskill init and select Harbor as the dataset source. Init will show available datasets from the Harbor Hub and auto-download your selection.
When Harbor is selected during init, the following config is auto-generated:
[dataset]
source = "harbor"
harbor_tasks_root = ".evoskill/harbor/datasets/swe-bench-verified"
train_ratio = 0.18
val_ratio = 0.12
[harbor]
enabled = true
inner_agent = "claude-code" # auto-derived from harness.name
inner_model = "anthropic/claude-sonnet-4-6" # auto-derived from harness.model
env = "docker" # "docker" for local, "daytona" for remote
n_concurrent = 1
timeout_multiplier = 1.0
container_skills_path = "/skills"
[scorer]
type = "harbor"The inner_agent and inner_model are automatically derived from your harness selection. The env is derived from your execution mode (docker for local/Docker, daytona for Daytona).
You can filter which tasks are included using glob patterns:
[dataset]
harbor_include = ["swe-bench/*"] # only include matching tasks
harbor_exclude = ["swe-bench/hard*"] # exclude matching tasks
harbor_difficulty = ["easy", "medium"] # filter by difficulty metadata
harbor_limit = 50 # max number of tasks| Mode | How Harbor runs tasks | Notes |
|---|---|---|
| Local | harbor run -e docker |
Requires Docker installed locally |
| Docker | harbor run -e docker |
Harbor tasks dir mounted as volume |
| Daytona | harbor run -e daytona |
Harbor uses Daytona API to create task sandboxes. DAYTONA_API_KEY is forwarded automatically. |
| Command | Description |
|---|---|
evoskill init |
Initialize a new project (creates .evoskill/) |
evoskill run |
Run the self-improvement loop |
evoskill run --rqgm |
Run with RQGM co-evolution enabled |
evoskill run --docker |
Run in a Docker container |
evoskill run --remote |
Run on a Daytona sandbox |
evoskill eval |
Evaluate the best program on the validation set |
evoskill skills |
List all skills discovered so far |
evoskill diff |
Diff baseline vs best, or between two iterations |
evoskill logs |
Show recent run history |
evoskill reset |
Delete all program branches and start fresh |
evoskill remote status |
Check progress of a remote run |
evoskill remote logs |
View logs from a remote run |
evoskill remote download |
Pull results from a completed remote run |
evoskill remote stop |
Stop and clean up a remote run |
evoskill run [--continue] [--verbose] [--quiet] [--config PATH] [--docker] [--remote] [--rebuild] [--rqgm]| Flag | Description |
|---|---|
--continue |
Resume from the existing frontier instead of starting fresh. |
--verbose |
Show per-sample pass/fail results |
--quiet |
Show the progress table only, suppress proposer output |
--config PATH |
Load a specific config TOML file instead of .evoskill/config.toml |
--docker |
Run inside a Docker container (builds image from Dockerfile if needed) |
--remote |
Run on a Daytona sandbox (requires [remote] config) |
--rebuild |
Force rebuild the Docker image before running |
--rqgm |
Enable RQGM co-evolution (overrides [rqgm] enabled = false in config) |
evoskill eval also accepts --config PATH.
evoskill diff # baseline → current best
evoskill diff 3 7 # iteration 3 vs iteration 7The diff is scoped to the .claude/ directory — it shows changes to skills and the system prompt, not your source code.
evoskill logs # last 5 runs (default)
evoskill logs --last 10 # last 10 runsevoskill reset # prompts for confirmationDeletes all program/* branches, frontier/* tags, the loop checkpoint, and feedback history. Your source code, config.toml, task.md, and any skills in .claude/skills/ are left untouched.
evoskill init creates .evoskill/config.toml. All fields are optional — defaults are shown below. Relative dataset and data directory paths are resolved from the project root, meaning the directory containing .evoskill.
[harness]
name = "claude" # "claude", "opencode", "codex", "goose", or "openhands"
model = "sonnet" # Claude alias, Codex model name, or provider/model for OpenCode/Goose/OpenHands
data_dirs = ["/absolute/path/to/data_dir"] # extra directories the agent can read
[evolution]
mode = "skill_only" # "skill_only" or "prompt_only"
iterations = 20
frontier_size = 3
concurrency = 4
no_improvement_limit = 5
[dataset]
path = "data/questions.csv" # relative to project root, or an absolute path
question_column = "question"
ground_truth_column = "ground_truth"
category_column = "" # optional, for stratified sampling
train_ratio = 0.18
val_ratio = 0.12
[scorer]
type = "multi_tolerance" # see scorer types below
[rqgm]
enabled = false # RQGM co-evolution (disabled by default)
epoch_size = 5Alternate configs can live next to the default config:
.evoskill/config.toml
.evoskill/config.openrouter.toml
.evoskill/config.rqgm.toml
Run with an explicit config:
evoskill eval --config .evoskill/config.rqgm.toml
evoskill run --config .evoskill/config.rqgm.tomlCommon evolution model setups:
Anthropic:
[harness]
name = "claude"
model = "claude-sonnet-4-6"OpenAI:
[harness]
name = "codex"
model = "gpt-5"OpenRouter:
[harness]
name = "opencode"
model = "openrouter/openai/gpt-5-mini"Fireworks AI:
[harness]
name = "openhands" # or "opencode"
model = "fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct"Notes:
claudeis Anthropic-only.codexuses bare OpenAI model names such asgpt-5,o3, orgpt-5.1-codex-mini.opencode,goose, andopenhandsare multi-provider harnesses and can also use Claude and OpenAI models.opencode,goose, andopenhandsacceptprovider/modelstrings such asanthropic/claude-sonnet-4-6,openai/gpt-5, oropenrouter/openai/gpt-5-mini.- Fireworks AI uses
FIREWORKS_API_KEY. OpenHands (litellm) expects thefireworks_ai/prefix while OpenCode (models.dev) expectsfireworks-ai/. Goose has no built-in Fireworks provider — use a manual OpenAI-compatible configuration.
| Type | Description |
|---|---|
multi_tolerance |
Flexible string matching: exact, numeric tolerance, list overlap (default) |
exact |
Case-insensitive exact string match |
llm |
LLM-as-judge grading with a custom rubric |
script |
Shell script scorer — receives {predicted} and {expected} as variables |
harbor |
Reads reward from Harbor's built-in verifier (auto-set when dataset source is Harbor) |
LLM scorer options:
[scorer]
type = "llm"
rubric = "Award 1.0 if the answer is numerically correct within 5%, 0.0 otherwise."
model = "claude-sonnet-4-6" # defaults to claude-sonnet-4-6
provider = "anthropic" # "anthropic", "openai", "google", "openrouter", or "fireworks"For OpenRouter-backed scoring, set provider = "openrouter" and use an OpenRouter model ID such as openai/gpt-5-mini or google/gemini-2.5-flash. Authentication uses OPENROUTER_API_KEY and falls back to LLM_API_KEY if needed.
For Fireworks-backed scoring, set provider = "fireworks" and use a Fireworks model ID such as accounts/fireworks/models/llama-v3p1-70b-instruct. Authentication uses FIREWORKS_API_KEY.
Script scorer options:
[scorer]
type = "script"
command = "python score.py --predicted {predicted} --expected {expected}"EvoSkill runs can take hours. Use Docker or Daytona to run on remote hardware and free up your machine.
Build the image from the included Dockerfile:
docker build -t evoskill .
evoskill run --dockerTo run on a remote server, point Docker to it:
export DOCKER_HOST=ssh://user@your-server
evoskill run --dockerMonitor and stop:
docker compose -f .evoskill/docker-compose.yml logs -f
docker compose -f .evoskill/docker-compose.yml downInstall the Daytona SDK and set your API key:
pip install daytona
export DAYTONA_API_KEY=your-daytona-keyBuild and push your image (Daytona runs x86 sandboxes, so cross-compile if you're on Apple Silicon):
# On Apple Silicon (ARM) — cross-compile for x86
docker buildx build --platform linux/amd64 -t your-registry/evoskill:latest --push .
# On x86 Linux — standard build
docker build -t evoskill .
docker tag evoskill your-registry/evoskill:latest
docker push your-registry/evoskill:latestSet in .evoskill/config.toml:
[remote]
target = "daytona"
[remote.daytona]
image = "your-registry/evoskill:latest"
cpu = 4 # max 4 vCPUs per sandbox
memory = 8 # max 8 GB per sandbox
disk = 10 # max 10 GB per sandbox
timeout = 0 # 0 = no auto-stop, or minutes until auto-stopThe DAYTONA_API_KEY can also be set as api_key under [remote.daytona], but the env var is preferred to avoid committing secrets.
Then:
evoskill run --remote # launch
evoskill remote status # check progress
evoskill remote logs -f # stream live output
evoskill remote logs # view last output
evoskill remote download # pull results when done
evoskill remote stop # cancel and clean upThe self-improvement loop follows five stages:
- Base Agent — Attempts benchmark questions using the current best program (system prompt + skills).
- Proposer — Analyzes failure cases and proposes targeted skill or prompt changes to address them.
- Generator — Creates the proposed changes: writes new skill files or rewrites the system prompt.
- Evaluator — Scores the new program variant on a held-out validation set to measure improvement.
- Frontier — Tracks the top-N performing programs as git branches; the best survive to the next iteration.
With RQGM enabled, the evaluator itself evolves across epochs:
- At epoch boundaries, the system computes a hack ratio (strict score / loose score)
- If the agent is gaming the evaluator (high loose, low strict), tolerances are tightened
- Gaming examples are collected into an adversarial pool and injected into the proposer context
- The evaluator gets harder as the agent improves — preventing reward hacking
EvoSkill uses your repo's git history to version every program it creates. During a run it automatically creates and switches between branches — you don't need to do anything. After a run your branch layout will look like:
main ← your code, untouched
program/base ← initial baseline agent
program/iter-skill-1 ← after iteration 1
program/iter-skill-2 ← after iteration 2
...
Frontier members are marked with frontier/* tags. EvoSkill only ever writes to branches prefixed program/, so there is no risk of it touching your working branch.
If accuracy stops improving, try the following:
- Check the feedback log —
.claude/feedback_history.mdrecords what the proposer tried each iteration and why it succeeded or failed. - Enable RQGM —
evoskill run --rqgmactivates the co-evolving evaluator, which can detect and correct reward hacking patterns. - Resume instead of restarting —
evoskill run --continuepicks up from the last frontier rather than discarding progress. - Reset and start fresh —
evoskill resetclears all branches and lets you start over with a revisedtask.md.
For programmatic usage, EvoSkill exposes a high-level Python API.
from src.api import EvoSkill
from src.loop.config import RQGMConfig
# Vanilla EvoSkill
evo = EvoSkill(
task="sealqa",
model="sonnet",
mode="skill_only",
max_iterations=20,
frontier_size=3,
concurrency=4,
train_ratio=0.18,
val_ratio=0.12,
continue_mode=False,
)
result = await evo.run()
# With RQGM co-evolution
rqgm_config = RQGMConfig(enabled=True, epoch_size=5)
evo_rqgm = EvoSkill(
task="sealqa",
model="sonnet",
mode="skill_only",
max_iterations=20,
rqgm_config=rqgm_config,
)
result = await evo_rqgm.run()
# Synchronous usage
result = EvoSkill(task="base").run_sync()from src.api import EvalRunner
summary = await EvalRunner(
task="sealqa",
model="sonnet",
max_concurrent=8,
).run()If you use EvoSkill-RQGM in your research, please cite both the RQGM paper and the original EvoSkill paper:
@article{iacob2026redqueen,
title={The Red Queen G{\"o}del Machine: Co-Evolving Agents and Their Evaluators},
author={Iacob, Alex and Jovanovi{\'c}, Andrej and Shen, William F. and
Burkhardt, Daniel and Kurmanji, Meghdad and Tastan, Nurbek and
Sani, Lorenzo and Venanzi, Niccol{\`o} Alberto Elia and
Odonnat, Ambroise and Cao, Zeyu and Marino, Bill and
Qiu, Xinchi and Lane, Nicholas D.},
year={2026},
eprint={2606.26294},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
@misc{alzubi2026evoskillautomatedskilldiscovery,
title={EvoSkill: Automated Skill Discovery for Multi-Agent Systems},
author={Alzubi, Salaheddin and Provenzano, Noah and Bingham, Jaydon and
Chen, Weiyuan and Vu, Tu},
year={2026},
eprint={2603.02766},
archivePrefix={arXiv},
primaryClass={cs.AI}
}This project is licensed under the Apache 2.0 License — see the LICENSE file for details.

