[luca-belli]

Summary

Design draft for discussion, docs only: docs/design/output-layout.md. It's connected to #268 (naming, config.json, conversation files), #266 (the pipeline as one job list), and #271 (events.jsonl).

Today, a pipeline run with two user models leaves output in four places: one folder per user model, each with its own evaluation, plus a separate pooled folder for the headline score. Finding one run's result takes four timestamps.

Proposal: one folder per run, under the target and the model under test, with one subfolder per stage, mirroring the code and the vera commands:

output/SI/gpt-5.4/20261015-194125_3f9a1c/
├── config.json, state.json, events.jsonl
├── generate/<user model>/      conversations, one file per persona × repeat
├── judge/<judge>_<hash>/       evaluations, results.csv (with a user_model column)
└── score/<judge>_<hash>/       scores.json: all user models together and each one separately

• The run folder is the result. The separate pooled/ folder goes away; vera pool stays only for combining different runs.
• Repeatable judging. Each judging gets its own judge/ and score/ folder, so re-judging never overwrites.
• No c_/u_/j_/p_ prefixes. A folder's position in the tree says what it is.
• No random tags. Conversation names drop the random tag, fixing the resume bug with persona names that contain spaces or dots.

Open questions at the end: stage vs. artifact folder names, naming a judging with two judge models, transcripts not made by vera, combining runs, and old folders.

🤖 Generated with Claude Code