Investigate computer-use proof packet evidence

[Jordak]

Parent

Related to #60 and #108.

Current behavior / design

As of this issue being written, Eval Lab records trial artifacts such as transcripts/traces, diffs, result metadata, run-surface metadata, and human review overlays. It does not have a project-local evidence shape for browser, desktop, or computer-use action loops. That is fine for the current starter suite, but recent agent surfaces make this worth specifying before any future browser/desktop adapter or task appears.

Why change

Computer-use workflows are tempting eval targets because they are realistic, but a final screenshot or final file state is weak evidence by itself. A serious task needs to preserve user intent, observed screen or environment state, proposed action, model-stated intent, executor action, safety policy, approval state, post-action evidence, and replay or artifact receipts.

Sources:

• Gemini Computer Use docs
• Gemini API changelog
• Model Context Protocol specification
• Existing runtime-metadata parent: Audit runtime metadata against agent control-surface sources #108
• Existing harder-task parent: Draft PRD for harder post-starter evaluation tasks #60

What to build

Investigate a small "computer-use proof packet" evidence shape for future tasks. This should be a design recommendation, not a browser/desktop adapter implementation. The output should say whether current run-surface and transcript artifacts are enough, or whether future computer-use tasks need a narrow new evidence object.

Candidate proof-packet fields:

• task goal and allowed action boundary;
• environment type and initial state source;
• screenshot or screen-state cadence;
• action primitive and normalized target;
• model-stated action intent;
• safety policy and prompt-injection scan state when available;
• executor / client that actually performed the action;
• human approval or denial event;
• post-action state and final artifact;
• replay artifact or reason replay is unavailable.

Acceptance criteria

• Map candidate proof-packet fields to existing Eval Lab artifacts where possible.
• Classify missing facts as unsupported/unknown, inferable from transcript, or requires future artifact field.
• Define at least one small future task shape where screenshots alone would be insufficient evidence.
• State non-goals against adding browser automation or desktop adapters before a concrete task requires them.
• If a future schema gap remains, propose one narrow follow-up rather than broad run-surface metadata expansion.

Blocked by

None - can start as a design investigation.