Problem
Error file handling is scattered across multiple task implementations (LocalTask, LocalAsyncTask, SlurmTask) and the output aggregation layer (PipelineOutput). Writing is done via a standalone write_error_file utility in tasks/utils.py; reading is done inline with raw json.loads calls and manual field access. This split means the on-disk schema is implicitly defined in two places with no shared contract.
As a result, the write and read paths can drift independently — field names are duplicated as string literals, error handling diverges (callers catch different exception types), and adding or changing a field requires coordinated edits across unrelated modules. The file attribution field added recently exposed this: the reader had to fall back to a stem-derived name because there was no guarantee the field was present, and no single place enforced that it should be.
Proposal
Introduce an ErrorRecord dataclass that owns the on-disk schema and exposes from_exception(), write(), and read() class methods. Replace all write_error_file call sites with ErrorRecord.from_exception(...).write(path) and all inline json.loads reads with ErrorRecord.read(path). Remove write_error_file from tasks/utils.py.
Problem
Error file handling is scattered across multiple task implementations (
LocalTask,LocalAsyncTask,SlurmTask) and the output aggregation layer (PipelineOutput). Writing is done via a standalonewrite_error_fileutility intasks/utils.py; reading is done inline with rawjson.loadscalls and manual field access. This split means the on-disk schema is implicitly defined in two places with no shared contract.As a result, the write and read paths can drift independently — field names are duplicated as string literals, error handling diverges (callers catch different exception types), and adding or changing a field requires coordinated edits across unrelated modules. The
fileattribution field added recently exposed this: the reader had to fall back to a stem-derived name because there was no guarantee the field was present, and no single place enforced that it should be.Proposal
Introduce an
ErrorRecorddataclass that owns the on-disk schema and exposesfrom_exception(),write(), andread()class methods. Replace allwrite_error_filecall sites withErrorRecord.from_exception(...).write(path)and all inlinejson.loadsreads withErrorRecord.read(path). Removewrite_error_filefromtasks/utils.py.