Skip to content

fix(results): disambiguate target artifact directories#1538

Closed
christso wants to merge 1 commit into
mainfrom
fix-av-9919-target-artifact-allocation
Closed

fix(results): disambiguate target artifact directories#1538
christso wants to merge 1 commit into
mainfrom
fix-av-9919-target-artifact-allocation

Conversation

@christso

Copy link
Copy Markdown
Collaborator

Summary

Multi-target runs that reuse the same suite and test ID now write separate physical artifact directories instead of overwriting the same run-1 folder. Single-target paths keep the existing suite/test/run-N or test/run-N layout, and index.jsonl remains the authoritative source for every emitted path.

Validation

  • Red check before the fix: bun test apps/cli/test/commands/eval/artifact-writer.test.ts --test-name-pattern "allocates distinct artifact directories" failed with both rows pointing at suite-a/shared-id.
  • Green focused checks: bun test apps/cli/test/commands/eval/artifact-writer.test.ts, bun test apps/cli/test/commands/eval/aggregate.test.ts, bun test packages/core/test/evaluation/orchestrator.test.ts.
  • Static checks: bun --filter @agentv/core typecheck, bun --filter agentv typecheck, bunx biome check packages/core/src/evaluation/run-artifacts.ts apps/cli/test/commands/eval/artifact-writer.test.ts, git diff --check.
  • Live dogfood: bun apps/cli/src/cli.ts eval run /tmp/agentv-av9919-live-Uxi4Gm/target-artifact-allocation.eval.yaml --targets /tmp/agentv-av9919-live-Uxi4Gm/targets.yaml --target baseline --target candidate --workers 1 --output /tmp/agentv-av9919-live-Uxi4Gm/results passed 2/2 with real OpenAI-backed targets and LLM grading. The manifest recorded shared-id/baseline and shared-id/candidate artifact dirs.

Compound Engineering
Codex

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: bba2f02
Status: ✅  Deploy successful!
Preview URL: https://31ba75e1.agentv.pages.dev
Branch Preview URL: https://fix-av-9919-target-artifact.agentv.pages.dev

View logs

@christso

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by PR #1540 and the av-504 result_dir contract direction. PR #1538 used the older target artifact allocation approach and has failing tests; any useful target-disambiguation ideas should be carried forward through #1540/#1541 instead.

@christso christso closed this Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant