Skip to content

Add exploration config generator, drop broken clean_trajectories.py#5

Open
xhluca wants to merge 1 commit into
fix/missing-prepare-tasks-intents-promptsfrom
fix/exploration-config-and-clean-trajectories
Open

Add exploration config generator, drop broken clean_trajectories.py#5
xhluca wants to merge 1 commit into
fix/missing-prepare-tasks-intents-promptsfrom
fix/exploration-config-and-clean-trajectories

Conversation

@xhluca

@xhluca xhluca commented May 11, 2026

Copy link
Copy Markdown
Contributor

What

Two gaps reported in follow-up to #4:

  1. Missing exploration-config generator. The released code shipped pre-generated exploration.tasks.json only for gemini-3-pro-preview and had no script to (re)build it for a different exploration model or persona set. This adds scripts/generate_exploration_tasks.py, ported from the internal repo, which reads personas from outputs/personas/<exploration_model>/personas.json (produced by scripts/create_personas.py) and writes one task per (persona, site) pair to agent_as_annotators/configs/<exploration_model>/exploration.tasks.json — the location a3-explore discovers automatically via importlib.resources.

  2. Broken clean_trajectories.py. This file was an unfinished work-in-progress that got accidentally checked in: it imports agent_reward_bench (not in the project's dependencies), references undefined names (judge, provider) at module body, contains a stray breakpoint() at line 103, and is not referenced anywhere in the documented pipeline. The "cleaning" step described in the README is actually performed by scripts/convert_trajectories_to_json.py, which writes its output to trajectories/cleaned/. Removed.

Changes

  • add scripts/generate_exploration_tasks.py — defaults output path to the package's agent_as_annotators/configs/ directory so a3-explore picks up the file with no further configuration. Imports use agent_as_annotators.{prompts,utils} (not the internal llm_annotators package). Behaviour is otherwise identical to the internal version.
  • remove scripts/clean_trajectories.py
  • README.md — Step 2 sub-steps renumbered to 2a (config generation) → 2b (a3-explore) → 2c (extract chat messages) → 2d (prepare prompts) → 2e (generate task intents).

Base

Stacked on top of #4 (which adds extract_chat_messages.py and prepare_tasks_intents_prompts.py). Once #4 merges, this PR will rebase cleanly onto main.

Two gaps reported in follow-up to #4:

1. The released code had no script to (re)build the
   `exploration.tasks.json` config that `a3-explore` consumes — only
   pre-generated configs for `gemini-3-pro-preview`. This adds
   `scripts/generate_exploration_tasks.py`, ported from the internal
   repo, which reads personas from `outputs/personas/<exploration_model>/`
   and writes one task per (persona, site) pair to
   `agent_as_annotators/configs/<exploration_model>/exploration.tasks.json`
   (the location `a3-explore` discovers via importlib.resources).

2. `scripts/clean_trajectories.py` was an unfinished work-in-progress
   that got accidentally checked in: depends on `agent_reward_bench`
   (not in the project deps), references undefined `judge`/`provider`,
   contains a stray `breakpoint()`, and is not part of the documented
   pipeline. The "cleaning" step in the README is actually performed by
   `scripts/convert_trajectories_to_json.py`, which writes JSONs to
   `trajectories/cleaned/`. Removed.

README updated to add `generate_exploration_tasks.py` as sub-step 2a
and renumber the existing sub-steps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant