Add missing extract / prepare scripts for task-intent generation#4
Open
xhluca wants to merge 1 commit into
Open
Add missing extract / prepare scripts for task-intent generation#4xhluca wants to merge 1 commit into
xhluca wants to merge 1 commit into
Conversation
The pipeline section of the README jumps straight from `a3-explore` to `scripts/generate_task_intents.py`, but the latter consumes prepared prompts under `outputs/task_intents/prompts/<exploration_model>/` that the released code did not provide a way to build. This adds the two intermediate scripts and documents the full Step 2 in the README, including which part of the exploration trajectory is fed to the Task Designer and how `step_num` is selected (random sample of N steps per trajectory, dropping step 0). - scripts/extract_chat_messages.py: agentlab `step_*.pkl.gz` -> per-step JSON with chat messages, goal, screenshot path, etc. - scripts/prepare_tasks_intents_prompts.py: per-trajectory random step sample + appended TASK_INTENT_PROMPT_TEMPLATE user turn, written as `task_<i>.step_<j>.json` matching `generate_task_intents.py`'s expected input layout. - README.md: expand Step 2 to spell out the four sub-steps and the sampling semantics. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The documented pipeline jumps from
a3-explorestraight toscripts/generate_task_intents.py, butgenerate_task_intents.pyconsumes prepared prompt files underoutputs/task_intents/prompts/<exploration_model>/that the released code had no script to produce. This PR ports the two intermediate scripts from the internal repo and documents Step 2 properly.Changes
scripts/extract_chat_messages.py— converts an agentlab study directory ($AGENTLAB_EXP_ROOT/<study_dir>/<task_dir>/step_*.pkl.gz) into a parallel JSON tree atoutputs/chat_messages/<study_dir>/<task_dir>/step_*.json. Each output JSON contains the chat messages, goal, screenshot path, and step number for that step. Supports--find-latest <agent>to auto-locate the most recent exploration study.scripts/prepare_tasks_intents_prompts.py— for each trajectory in the chat-messages tree, samples N steps (default 3, skippingstep_0), appendsTASK_INTENT_PROMPT_TEMPLATE(parameterized byWEBARENA_ANNOTATOR_INSTRUCTIONSand--num_intents) as a final user turn, and writes each asoutputs/task_intents/prompts/<exploration_model>/task_<i>.step_<j>.json— exactly the layoutgenerate_task_intents.pyexpects.README.md— Step 2 of the pipeline now lists the four sub-steps (a3-explore→extract_chat_messages.py→prepare_tasks_intents_prompts.py→generate_task_intents.py) and includes a short paragraph explaining what part of the trajectory is fed to the Task Designer and how the step index is sampled.Resolves
Question about the missing intermediate step between
a3-exploreandgenerate_task_intents.py. Documents the trajectory representation passed to the Task Designer (full chat-message history of one randomly sampled step + one appended user turn).Notes
The two scripts are direct ports from the internal repo with the only change being
llm_annotators→agent_as_annotatorsin two import lines.