Problem
After a successful harness run, the scratch directory (generated Cargo project, compiled binary, trace output) is not preserved. When debugging failures during development, you need to:
- Run the harness
- Hope the scratch dir still exists (it does on failure, but not success)
- Or add breakpoints / print statements to the harness code itself
Example
After run_spec("specs/edm.primitive_types.spec.yaml") completes successfully:
- Generated
main.rs — gone
traces.json — gone
- Compiled
sg-runner binary — gone (unless cached from a previous failed run)
To inspect actual traces, I had to find the cached binary from a prior run and manually invoke it:
/path/to/sg-runner /tmp/traces.json
Suggestion
Add a mechanism to preserve artifacts:
SPECGATE_KEEP_ARTIFACTS=1 env var
- Or a
run_spec_opts variant with a keep_artifacts: bool parameter
- At minimum, keep
traces.json on success (it's small and valuable for debugging)
Problem
After a successful harness run, the scratch directory (generated Cargo project, compiled binary, trace output) is not preserved. When debugging failures during development, you need to:
Example
After
run_spec("specs/edm.primitive_types.spec.yaml")completes successfully:main.rs— gonetraces.json— gonesg-runnerbinary — gone (unless cached from a previous failed run)To inspect actual traces, I had to find the cached binary from a prior run and manually invoke it:
Suggestion
Add a mechanism to preserve artifacts:
SPECGATE_KEEP_ARTIFACTS=1env varrun_spec_optsvariant with akeep_artifacts: boolparametertraces.jsonon success (it's small and valuable for debugging)