The Crum transcription is currently materialized in two formats:
Enrichment then runs in JavaScript when the browser loads a page, but never gets materialized. The ask here is to build some format that encodes the enrichment, and track it in Git.
Claude already suggests such a format in #745, to be generated on the fly rather than materialized and committed to Git. Committing it to Git will allow us to simplify the pipeline for #745.
The materialization will also unblock #773.
Quite importantly, it will allow us to inspect the outcome of any changes to the enrichment algorithm, enabling us to verify correctness, and argue about changes.
This may appear as counter to previous directions, which sought to minimize committed data and pipeline steps (#359, #483). But notice that this doesn't introduce any extra pipeline layers.
P.S. Could materializing the TSV itself make it easier for AI to access the source-of-truth without having to curl it or infer it from the output?
The Crum transcription is currently materialized in two formats:
Enrichment then runs in JavaScript when the browser loads a page, but never gets materialized. The ask here is to build some format that encodes the enrichment, and track it in Git.
Claude already suggests such a format in #745, to be generated on the fly rather than materialized and committed to Git. Committing it to Git will allow us to simplify the pipeline for #745.
The materialization will also unblock #773.
Quite importantly, it will allow us to inspect the outcome of any changes to the enrichment algorithm, enabling us to verify correctness, and argue about changes.
This may appear as counter to previous directions, which sought to minimize committed data and pipeline steps (#359, #483). But notice that this doesn't introduce any extra pipeline layers.
P.S. Could materializing the TSV itself make it easier for AI to access the source-of-truth without having to
curlit or infer it from the output?