Skip to content

Tighten starter task prompt guidance - #122

Merged
Jordak merged 5 commits into
mainfrom
codex/remove-visible-validation
Jun 24, 2026
Merged

Tighten starter task prompt guidance#122
Jordak merged 5 commits into
mainfrom
codex/remove-visible-validation

Conversation

@Jordak

@Jordak Jordak commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Summary

This PR tightens the starter task prompts now that test: is grader-facing by default and visible_validation: is explicit.

Changes in the first reviewable layer:

  • Remove all existing task-level visible_validation: blocks from starter/edit-scope task bundles.
  • Regenerate affected task cards.
  • Remove prompt text that explicitly told agents to run relevant or targeted checks.
  • Remove generic scope-coaching phrases such as "keep the patch focused" while preserving concrete task requirements.
  • Reword the React Tabs style-surface instruction as a deliverable requirement instead of process coaching.
  • Keep the Remotion runtime limitation because broad monorepo/browser/e2e validation may be too expensive for routine trials.

Why

The previous behavior made some starter tasks easier for harnesses under test by exposing validation or process hints. This change intentionally leaves graders hidden and asks the implementing agent to infer validation and scope discipline from the task itself.

Validation

  • python3 -m agentlab task validate --check-task-cards tasks
  • python3 -m unittest tests.test_task_bundle_integrity tests.test_task_cards tests.test_tasks tests.test_agent_prompts
  • git diff --check
  • pre-commit task bundle integrity checks

Notes

A low-effort Codex pilot on the tightened prompts was run locally after these changes. Three apparent failures were reviewed separately and looked like grader overfit rather than clear behavioral failures. The final commit on this branch loosens those three starter graders as a separate review layer.

@Jordak
Jordak force-pushed the codex/remove-visible-validation branch from 4abe3bd to cc38298 Compare June 23, 2026 18:58
@Jordak
Jordak marked this pull request as ready for review June 24, 2026 21:36
@Jordak
Jordak merged commit cfe263e into main Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant