Investigate forecast-conditioned task synthesis

[Jordak]

Parent

Related to #60.

Current behavior / design

As of this issue being written, Eval Lab task design is centered on pinned real repositories, issue-derived tasks, reference artifacts, deterministic graders, and human review. It does not yet have a task-candidate method that reduces dependence on directly replaying public historical PRs or issues.

Why change

Public issue/PR replay is realistic, but it is also exposed to model pretraining, synthetic-data generation, and benchmark overfitting. SWE-Future suggests a practical middle ground: use pre-snapshot repository evidence to forecast plausible future task families, then synthesize tasks conditioned on those forecasts rather than copying later PRs directly.

Source:

• SWE-Future: Forecast-Conditioned Data Synthesis for Future-Oriented Software Engineering Agents
• Existing harder-task roadmap parent: Draft PRD for harder post-starter evaluation tasks #60

What to build

Investigate whether Eval Lab should use a lightweight forecast-conditioned task-candidate workflow for future post-starter tasks. This should stay small: one candidate repository, one forecast snapshot definition, one synthetic task sketch, and one contamination/fairness check.

Acceptance criteria

• Select one candidate repository where future-oriented task synthesis might be useful.
• Define a forecast snapshot boundary using only repository evidence available before that point.
• Draft one synthetic task candidate conditioned on forecasted future work, without directly copying a later PR.
• Compare the candidate against the simpler alternative of replaying a real issue/PR.
• Recommend whether this should feed Draft PRD for harder post-starter evaluation tasks #60, become a task-curation tool, or be parked.

Blocked by

None - can start immediately