Skip to content

Commit bb76cf0

Browse files
authored
Merge pull request #3 from stp/drop-known-damage-note
Drop the known-data-damage section from the README
2 parents b81d73f + 8db89f4 commit bb76cf0

3 files changed

Lines changed: 3 additions & 24 deletions

File tree

‎README.md‎

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -119,25 +119,6 @@ that has since moved on.
119119
The website updates when this repo's Pages build finishes. Nothing in `stp/stp`
120120
has to change.
121121

122-
## Known data damage
123-
124-
**`outputs/full-001.jsonl.gz` holds 154,519 of that campaign's 160,610 runs.**
125-
A reboot interrupted the campaign at run 63,533; the output log was being
126-
appended to as a gzip stream, and the kill left an unterminated member with the
127-
resumed run's records appended after it. Walking the members recovers all but
128-
about 6,000 records — the ones still in the compressor's buffer when the
129-
machine went down. The published file has been repaired into a single clean
130-
member, so `zcat` reads it through.
131-
132-
The results themselves are unaffected: they were committed to the database as
133-
each run finished, and the campaign resumed and completed. Only the retained
134-
stdout of those ~6,000 runs is gone, which costs the ability to re-triage
135-
them without re-running.
136-
137-
Both faults behind that are fixed in `scripts/`: the log is repaired before
138-
anything appends to it, it is flushed on the same boundary as the database
139-
commit, and the per-run key no longer restarts at zero when a campaign resumes.
140-
141122
## Licences
142123

143124
What is written here — the harness, the exported results, this documentation —

‎index.html‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,8 @@ <h2>Files</h2>
7171
<h2>Repository</h2>
7272
<p><a href="https://github.com/stp/benchmarks-data">github.com/stp/benchmarks-data</a>
7373
holds all of this plus <code>scripts/</code>, the harness that runs a campaign
74-
and publishes it. Its README covers how a campaign is reproduced, what is
75-
excluded from publication and why, and the one place the data is known to be
76-
incomplete.</p>
74+
and publishes it. Its README covers how a campaign is reproduced, and what is
75+
excluded from publication and why.</p>
7776

7877
</body>
7978
</html>

‎scripts/README.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -218,8 +218,7 @@ before appending to it, so the damage cannot compound; the repair is also what
218218

219219
The published log is keyed by benchmark path, not by the DB's `output_offset`.
220220
That counter used to restart at zero in each process, so a resumed campaign
221-
numbered two different records `0` — see the README's note on `full-001`, which
222-
was recorded that way.
221+
numbered two different records `0`; `full-001` was recorded that way.
223222

224223
## Resuming
225224

0 commit comments

Comments
 (0)