Inference performance: vectorized energy detection, fused per-channel workers, per-cadence streaming, downsample-at-extraction#120
Conversation
…extraction Replace _threshold_hits_worker's per-window Python loop over scipy.stats.normaltest (~8190 windows/channel x 3 ON files x all coarse channels — the #1 inference cost at ~45 min/cadence) with a closed-form vectorized derivation (_sliding_normality_k2): per-block power sums S1..S4 in float64 -> moving window sums -> central moments -> scipy's exact skewtest/kurtosistest Z transforms transcribed elementwise -> k2 = Z1^2 + Z2^2, with chi2.sf evaluated only on the hit subset. Fast path uses step-sized blocks when step divides window; general window/step ratios fall back to gcd-sized blocks via np.add.reduceat. Zero-variance windows yield NaN (never hits), matching scipy. Replace the block-based read/bandpass/threshold stages (3 pools + 2 block-sized SHM copies per 28-channel block) with one fused task per coarse channel (_energy_detect_channel_worker: read -> DC-spike removal -> bandpass flatten -> vectorized threshold -> small hit list) running on a single persistent pool per run (start/stop_energy_detection_pool), which the stamp-extraction stage now reuses. The DC spike sits at the channel center with interpolation offsets of at most +/-3 bins, so per-channel processing touches exactly the bins the block path did. Bandpass flattening is now a pluggable picklable callable (_get_bandpass_flattener; spline implementation today, PFB planned). Every complete coarse channel is processed — the block path silently dropped up to parallel_coarse_chans - 1 trailing channels when n_chans was not an exact block multiple. Downsample-at-extraction: _extract_stamps_worker now downsamples stamps along frequency (downscale_local_mean by data.downsample_factor) before writing, cutting per-cadence .npy storage ~8x and removing the separate downsample pass at load time; metadata JSON gains stored_width / downsample_factor_applied. load_inference_data branches on the stored width: already-downsampled files get log-norm only (vectorized in the workers via the new _lognorm_worker, resolving the old combined downsample+log-norm TODO); legacy full-width files keep the historical downsample-then-log-norm path. New config/flag store_downsampled_stamps (default on) with a stamp_width %% downsample_factor validation. parallel_coarse_chans becomes a progress-logging chunk knob only (default None -> use manager.n_processes); README CLI reference regenerated. find_hits() is now a thin wrapper over plan_cadences() + process_pending_cadence(), which the streaming inference loop drives directly. Part of #115 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…p bug Restructure inference_command around a per-cadence streaming loop (_run_streaming_csv_inference): models load once into a single InferencePipeline; a prefetch-depth-1 background thread preprocesses cadence i+1 (the persistent pool's child processes do the real work) while the main thread loads cadence i's stamps, encodes on the GPUs, runs the RF, and writes per-cadence results. Peak memory is now one cadence's stamps plus the in-flight preprocessing state — independent of catalog size — where the old flow preprocessed and loaded the whole catalog into a single array. The legacy --test-files path keeps its single-shot run_inference_pipeline flow, and the coarse retry loop is unchanged (per-cadence .npy resume already works). Fix the tail-drop bug in prepare_distributed_inf_dataset: inf_steps was n // global_batch with drop_remainder=True and no padding, so the final partial batch was silently never processed (and a cadence smaller than one global batch would process nothing at all). The data is now padded with duplicate rows to the next global-batch multiple (mirroring train.py's viz-dataset pattern) and the encoded latents are truncated back to the real sample count. Populate observational provenance end to end: derive_cadence_provenance maps each cadence's group key + metadata JSON onto target / session / band / cadence_id / timestamp_observed (header tstart) / h5_path, and per-stamp frequency_mhz now comes from the metadata's stamp_frequencies_mhz (resolving the TODO cluster at inference.py and main.py). _write_inference_results also now stores the full (num_observations * latent_dim) latent vector for a candidate snippet — it previously stored a single observation row picked from the wrong index. The per-call tf.function trace and TF session clearing are hoisted so repeated run_inference calls on one pipeline don't accumulate state. Part of #115 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- _sliding_normality_k2 equivalence gates: k2 matches scipy.stats.normaltest per window to rtol=1e-9 across six distributions/scales and four window/step geometries (fast + gcd paths), plus zero-variance NaN, minimum-sample, and window-count parity checks. - Real-data gate: committed fixture tests/fixtures/ed_real_slice.npz (16 x 16384 float32 slice of coarse channel 469 of a GBT DDO210 observation, centered on the DC spike, bitshuffle-decoded on blpc3) must produce identical hit sets and rtol=1e-9 statistics between the vectorized path and the historical scipy loop. - Fused _energy_detect_channel_worker vs an explicitly composed read -> DC spike -> spline flatten -> scipy threshold pipeline; absolute index offsets. - derive_cadence_provenance mapping (case-insensitive columns, unparseable cadence ids, sparse metadata). - _extract_stamps_worker downsample-at-extraction vs downscale_local_mean, and the factor-1 raw path. - _process_cadence end to end on synthetic .h5 cadences (sequential): stamp shapes, metadata fields, resume path, and the store_downsampled_stamps=False full-width layout. - load_inference_data branch tests: downsampled log-norm-only path, legacy downsample+log-norm path, mixed files, unrecognized widths. - prepare_distributed_inf_dataset tail-padding regression tests: the padded tail is processed (old code dropped it; a cadence smaller than one global batch processed nothing) and order/padding are deterministic. Part of #115 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…copies - Streaming inference exits with an error when no cadence produced a stamp .npy (preserving the historical find_hits contract) instead of reporting an empty success. - Skip the redundant float32 copy of each cadence's loaded stamps (astype copy=False) and the double float64 allocation in _sliding_normality_k2 (astype + in-place mean subtraction). Part of #115 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @zachtheyek's task in 2m 23s —— View job Code Review: Inference Performance PR
SummaryThis is a well-executed performance overhaul — 6.2x end-to-end speedup with strong correctness evidence. The architecture is sound: fusing per-channel work into single tasks eliminates pool/SHM churn, the vectorized energy detection is mathematically rigorous with excellent unit-test gates, and per-cadence streaming bounds memory independent of catalog size. The tail-drop bug fix is an important correctness win. Strengths
Issues & SuggestionsMedium Priority
Low Priority / Nits
Security
VerdictApprove with minor suggestions. The correctness evidence is strong (identical hit sets on real data, rtol=1e-9 numeric gates, regression tests for the tail-drop fix), the performance win is substantial and well-measured, and the code quality is high. The medium-priority items above are non-blocking — they're about defensive robustness and code convention rather than correctness. Nice work. |
|
Contribution compliance note: The |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| p1 = np.zeros(width, dtype=np.float64) | ||
| p2 = np.zeros(width, dtype=np.float64) | ||
| p3 = np.zeros(width, dtype=np.float64) | ||
| p4 = np.zeros(width, dtype=np.float64) | ||
| for t in range(time_bins): |
There was a problem hiding this comment.
Nit (performance): The explicit Python for t in range(time_bins) loop over rows, while keeping temporaries at (width,) shape, may become the bottleneck for large time_bins. Consider vectorizing this with np.einsum or direct array ops:
p1 = data.sum(axis=0)
p2 = (data**2).sum(axis=0)
p3 = (data**3).sum(axis=0)
p4 = (data**4).sum(axis=0)The peak memory would be (time_bins, width) per power (one at a time if computed sequentially), which for the typical 16×16384 slice is ~2 MB — negligible. The current row-by-row loop is correct and well-commented, so this is purely an optimization suggestion for a future pass if time_bins grows significantly (e.g., different instruments).
There was a problem hiding this comment.
Added a NOTE in 61f8528: the row loop is deliberate — it caps peak memory at a few (width,) float64 vectors per worker (32 workers x 1M-bin channels). At the default time_bins=16 the loop overhead is noise next to the h5 read + spline fit; the NOTE points to the (channel**k).sum(axis=0) vectorization if time_bins ever grows.
| future = prefetch.submit(preprocessor.process_pending_cadence, units[0]) | ||
|
|
||
| for i, unit in enumerate(units): | ||
| cadence_result = future.result() |
There was a problem hiding this comment.
Potential issue — future submission after exception: If future.result() on line 341 raises (e.g. the prefetch thread hit an unrecoverable error), the exception propagates up, but the previous future submitted on line 345 (from iteration i-1) is still running in the background. The ThreadPoolExecutor context manager will wait for it to finish on exit, but any exception from that abandoned future is silently swallowed.
More importantly, the prefetch.submit(...) on line 338 fires the first unit's preprocessing. If units[0]'s preprocessing itself raises, future.result() on line 341 will raise inside the for-loop, and the code correctly skips to the finally block. That's fine.
But consider: if units[i+1]'s prefetch raises inside the thread (line 345), the exception won't surface until the next iteration's future.result(). At that point cadence_result would be the exception, not None — it would raise there. This is acceptable behavior (the retry loop catches it), but worth a brief comment noting that prefetch exceptions propagate one iteration late.
There was a problem hiding this comment.
Added a comment in 61f8528 clarifying the timing: a prefetched task's exception surfaces one iteration later, when its future.result() is resolved, and then propagates to the retry loop. In practice process_pending_cadence swallows per-cadence failures (returns None), so only infrastructure-level errors (e.g. a broken worker pool) raise here. The with ThreadPoolExecutor exit also waits for any in-flight task before the pool is torn down in finally, so there's no submission-after-shutdown window.
| ) = args | ||
| out = np.lib.format.open_memmap(npy_path, mode="r+") |
There was a problem hiding this comment.
Robustness: np.lib.format.open_memmap(npy_path, mode="r+") will raise if npy_path doesn't exist or is corrupted (e.g. a partial write from a killed previous run). Since this worker is invoked from a pool, the exception propagates as a task failure, which is correct — but it may produce a confusing traceback when the .tmp.npy was partially written due to a prior crash. The atomicity guarantee (.tmp.npy → os.replace → canonical name) protects the canonical path, but the .tmp.npy itself can be left behind by a killed process. Consider adding a note that the caller should remove stale .tmp.npy files before re-dispatching extraction tasks (or do so in _process_cadence before writing the memmap header).
There was a problem hiding this comment.
Addressed in 61f8528: _process_cadence now detects a stale .tmp.npy left by an interrupted run (e.g. SIGKILL between memmap creation and os.replace), logs a warning, and removes it before extraction. Note the worker's open_memmap(..., mode="r+") can only ever see the freshly created tmp file (the parent creates it with w+ immediately before dispatching tasks), so the stale-file risk lived at creation time, not in the workers. The canonical npy_path is only ever produced by os.replace after all workers finish, so a partial canonical file can't exist.
| if n_padded > n_samples: | ||
| pad_count = n_padded - n_samples | ||
| pad_indices = np.arange(pad_count) % n_samples | ||
| inf_data = np.concatenate([data, data[pad_indices]], axis=0) |
There was a problem hiding this comment.
Minor: np.concatenate([data, data[pad_indices]], axis=0) creates a full copy of the original data plus padding. For a large cadence (e.g. 73k stamps × 6 × 16 × 512 × 4 bytes ≈ 14 GB), this doubles peak memory momentarily. Since the streaming path only ever holds one cadence's worth of stamps, this is bounded, but worth noting for future reference that a np.pad-style approach or a custom generator that cycles would avoid the copy.
There was a problem hiding this comment.
Added a NOTE in 61f8528: the concatenate does briefly double the cadence's footprint (~4.6 GB peak for the 73k-stamp DDO210 cadence — fine at per-cadence scale, and far below the old whole-catalog behavior). If this ever matters, the alternative is yielding pad rows from the generator instead of materializing them; deferred until profiling shows a need.
| units = preprocessor.plan_cadences() | ||
| if not units: |
There was a problem hiding this comment.
Design: sys.exit(1) inside a helper function (_run_streaming_csv_inference) called from within the retry loop in inference_command means the retry logic never gets a chance to handle the "no cadence work units" case. This is intentional (an empty catalog is a configuration error, not a transient failure), but it breaks the convention that helpers raise and callers decide exit behavior. Consider raising a dedicated exception (or ValueError) and letting inference_command decide to sys.exit — consistent with how preprocessor.load_inference_data() raises ValueError("No data loaded successfully") in the legacy path.
There was a problem hiding this comment.
Fixed in 61f8528: both permanent failure modes (empty catalog, zero cadences producing stamps) now raise NonRetryableInferenceError and the helper never exits the process itself. inference_command's retry loop catches that exception type before the generic except Exception and fails fast (no retry attempts burned), while transient failures keep the existing retry semantics. Regression tests added in tests/unit/test_main.py.
…otes - _run_streaming_csv_inference now raises NonRetryableInferenceError (instead of calling sys.exit inside the helper) for the two permanent failure modes — empty catalog and zero cadences producing stamps; the retry loop in inference_command re-raises it immediately so permanent config errors don't burn retry attempts, while transient failures keep the existing retry semantics. Regression tests added (tests/unit/test_main.py). - _process_cadence detects a stale .tmp.npy left by an interrupted previous run (e.g. SIGKILL between memmap creation and os.replace), logs a warning, and removes it before extraction; covered by the process-and-resume test. - Clarifying comments: prefetch exceptions surface one iteration later via future.result() (and what actually raises given process_pending_cadence swallows per-cadence failures); the padding concatenate briefly doubles a cadence's memory footprint; the power-sum row loop is deliberate for peak-memory reasons; np.lib.format.open_memmap is a semi-public numpy API. Part of #115 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
All review feedback addressed in 61f8528:
Verification: full default suite re-run in the NGC container on blpc3 at 61f8528 — 239 passed, 2 deselected (includes the 2 new regression tests); |
|
@claude please perform a second-pass code review of this PR. Re-review the current diff for any remaining correctness bugs, security issues, or significant quality problems, and post your findings. If the diff is clean, please say so explicitly. (Final verification round before merge.) |
| edges = np.arange(0, width, block) | ||
| blocks_per_window = window_size // block | ||
| blocks_per_step = step_size // block | ||
|
|
||
| def _window_sums(col_sums: np.ndarray) -> np.ndarray: | ||
| block_sums = np.add.reduceat(col_sums, edges) |
There was a problem hiding this comment.
Low (latent edge case): If width % block != 0, np.add.reduceat produces a partial last block with fewer than block columns. Any sliding window that overlaps that partial block gets a short sum → incorrect k2 statistic (silently wrong, not a crash).
With default config (all powers of 2, coarse channel widths are powers of 2) this never triggers — the concern is a user-configured window_size/step_size whose gcd doesn't divide the coarse channel width.
Consider either asserting width % block == 0 (fail loud on unsupported geometry) or truncating col_sums to (width // block) * block before reduceat (silently exclude trailing columns that can't form a full window anyway — n_windows already excludes incomplete windows from the right edge).
There was a problem hiding this comment.
Fixed in 146845b, using the truncation option you suggested. _window_sums now slices the reduceat output down to the full blocks:
n_full_blocks = width // block
...
block_sums = np.add.reduceat(col_sums, edges)[:n_full_blocks]width // block equals len(edges) when block divides width (so it's a no-op on power-of-2 geometries) and drops exactly the ragged short block otherwise. It's behavior-preserving even on the geometries that currently hit the short block — no in-range window ever reaches it, because every window ends at a multiple of block that is < width (hence <= edges[-1]), and n_windows already excludes the incomplete right-edge windows. The slice just makes that invariant explicit so a partial sum can never silently leak into a k2.
Added test_non_power_of_2_width_short_block_matches_scipy, parametrized over two non-power-of-2 widths — (1234, 100, 30) and (1000, 96, 36) — each of which assert-guards width % block != 0 (so the geometry can't be refactored into a no-short-block case and pass vacuously) and pins k2 to scipy.stats.normaltest at rtol=1e-9. Verified the pure-NumPy/SciPy math against scipy directly (the existing (100, 30)@4096 combo already exercised this path and still matches).
| np.testing.assert_allclose(stat_val, expected[idx], rtol=1e-9) | ||
| np.testing.assert_allclose(pval, stats.chi2.sf(stat_val, 2), rtol=1e-9) | ||
|
|
||
| def test_absolute_indices_offset_by_channel_start(self, make_h5_observation): |
There was a problem hiding this comment.
Nit (test robustness): With stat_threshold=0.0, the all(s >= ch_start ...) assertion at the bottom passes vacuously if hits is empty (Python's all([]) → True). In practice chi-square noise at threshold 0 always produces hits, but the other tests in this class guard against this with an explicit assert len(hits) > 0 — adding one here would close the theoretical gap.
Not blocking, just for consistency with the pattern used a few lines above in test_fused_worker_matches_composed_pipeline.
There was a problem hiding this comment.
Fixed in 146845b: added assert len(starts) > 0 right before the all(...) check, so the offset assertion can no longer pass vacuously on an empty hit list. This matches the assert len(hits) > 0 pattern already used a few lines above in test_fused_worker_matches_composed_pipeline.
Second-pass code reviewReviewed the full diff across all 10 changed files (preprocessing.py, inference.py, main.py, config.py, cli.py, and the three test files). Focused on correctness bugs, security issues, and significant quality problems. Verdict: Clean — no blocking issues found.The vectorized energy detection, fused per-channel workers, per-cadence streaming, downsample-at-extraction, and the tail-drop/latent-vector bug fixes are all correct. The numerical implementation in Low-severity findings (non-blocking, noted inline or below)
Items verified clean
🤖 Generated with Claude Code |
…acuous test Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Applied the three non-blocking hardening items from the review in 146845b:
Verification: the ED math ( |
|
@claude the finding from your previous review has been addressed in a new commit. Please re-review the current diff once more and confirm whether it is now clean, or report any remaining issues. |
Re-review: all three findings addressed — diff is cleanVerified all three fixes from 146845b against the current diff: 1. Partial trailing block in
|
1 participant
Closes #115
Stacked on #116 (
feature/test-suite) — this PR's base isfeature/test-suite; review/merge #116 first. Note: #117 (the parallel train-chain head) shares the config/cli/README-CLI-regen surface with this PR; cross-branch conflicts there are expected to be resolved at rebase time by the coordinator.Summary
CSV inference was CPU-bound at 2 h 13 m for the 2-cadence subset (GPUs busy ~32 s total). This PR overhauls the four dominant costs. Measured result on the same subset, same config, same hardware: 21 m 30 s end to end (~6.2x), with energy detection down from ~45 min/cadence to ~2 min/cadence and stamp storage cut exactly 8x.
1. Vectorized energy detection (
_sliding_normality_k2)Replaces the per-window Python loop over
scipy.stats.normaltest(~8190 windows/channel x 3 ON files x all coarse channels — ~45 min/cadence) with a closed-form vectorized derivation: per-block power sums S1..S4 in float64 -> moving window sums -> central moments -> scipy's exact skewtest/kurtosistest Z transforms transcribed elementwise ->k2 = Z1^2 + Z2^2;chi2.sfruns only on the hit subset. Step-sized blocks when step divides window (the default config), gcd-sized blocks vianp.add.reduceatotherwise. Zero-variance windows yield NaN (never hits), matching scipy.Correctness gates (committed unit tests):
k2matchesscipy.stats.normaltest(window.flatten())to rtol=1e-9 per window across 6 distributions/scales x 4 window/step geometries (measured max rel err ~7e-11; most cases < 1e-13).tests/fixtures/ed_real_slice.npz(16 x 16384 float32 slice of coarse channel 469 of a GBT DDO210 observation, centered on the channel's DC spike, RFI-dense; bitshuffle-decoded on blpc3). After identical DC-spike removal + spline flattening, vectorized vs the historical scipy loop: identical hit sets, statistic max rel err 9.7e-16.2. Fused per-channel workers + one persistent pool
One task per coarse channel (read own h5 slice -> DC-spike removal -> pluggable bandpass flatten (spline now; PFB comes in PR-07) -> vectorized threshold -> small hit list) on a single persistent pool per run, which stamp extraction reuses. Replaces 3 pools + 2 block-sized SHM copies per 28-channel block (x 18 blocks x 3 ON files per cadence). The DC spike sits at the channel center and its interpolation offsets reach at most +/-3 bins, so per-channel processing touches exactly the bins the block path did (verified; noted in code).
parallel_coarse_chansis now a progress-logging chunk knob only (defaultNone->manager.n_processes).Behavior note: every complete coarse channel is now processed — the old block path floored to a block multiple and silently dropped coarse channels 504-511 of the 512-channel subset files (see the +54/+27 extra stamps below).
3. Downsample-at-extraction
_extract_stamps_workerdownsamples stamps along frequency before writing (downscale_local_mean— the same op the loader applied afterward, so end-to-end values are identical): storage / 8, and the separate ~24 min downsample pass disappears (loading now only log-norms, vectorized in the pool workers via the new_lognorm_worker, resolving the old "downsample & log-norm simultaneously" TODOs). Metadata JSON gainsstored_width/downsample_factor_applied;load_inference_databranches on the stored.npywidth so legacy full-width files (e.g. the Julysubset_test_*.npyon blpc3) still load through the historical path. Newstore_downsampled_stampsconfig/flag (default on).4. Per-cadence streaming inference
inference_commandnow drives a per-cadence loop: models load once (InferencePipeline), a prefetch-depth-1 background thread preprocesses cadence i+1 (the persistent pool's child processes do the work) while the main thread loads/encodes/classifies/writes cadence i. Peak memory = one cadence's stamps, independent of catalog size. Per-cadence provenance (target / session / band / cadence_id / timestamp_observed / h5_path, plus per-stampfrequency_mhzfrom the metadata'sstamp_frequencies_mhz) is now populated end to end via the newderive_cadence_provenance— resolving the TODO cluster at inference.py:226-247 / main.py:397-399. The legacy--test-filespath and the coarse retry loop are unchanged (per-cadence.npyresume still works; PR-08 owns the manifest overhaul).Bug fixes along the way:
prepare_distributed_inf_datasetcomputedinf_steps = n // global_batchwithdrop_remainder=True— the final partial batch was silently never processed (the July subset run dropped 2,429 of 94,589 stamps; with per-cadence batches, a cadence smaller than one global batch would process nothing). Now pads with duplicate rows to the next batch multiple (train.py's viz-dataset pattern) and truncates the encoded latents back to the real count. Regression unit tests added. This run processed all 94,670 snippets (padding 73,001->81,920 and 21,669->30,720)._write_inference_resultsstored a single 8-dim latent row picked by snippet index (wrong row, wrong shape) as a candidate's latent vector; it now stores the snippet's full(num_observations * latent_dim)vector, matching the DB contract.Config / CLI
inference.parallel_coarse_chans: default28->None(= usemanager.n_processes); progress-logging chunk only.inference.store_downsampled_stamps(defaultTrue) +--store-downsampled-stamps/--no-store-downsampled-stamps.stamp_width % downsample_factor == 0when downsampled storage is active with--inference-files.Config.to_dict()updated; README CLI Reference regenerated withPYTHONPATH=src python utils/print_cli_help.py all.Verification
Unit (blpc3, NGC 25.02 container,
CUDA_VISIBLE_DEVICES="", branch @ 03e5093)(2 deselected = the gpu/cluster integration smokes.)
pre-commit run --all-filespasses. The suite was run in the container because the local dev box's x86_64 TF wheel aborts at import under Rosetta (cpu_feature_guard CheckFeatureOrDie) — the numeric equivalence checks were additionally exercised locally with a TF module stub.Cluster smoke (blpc3,
subset_test.csv, dummytest_v17model, 5 GPUs,--max-retries 1, fresh--preprocess-output-dirso energy detection ran from scratch; tag20260711_172857)Pipelining evidence from the log timeline: NGC7454's energy detection started at 17:43:10.71 — the same second DDO210's load began on the main thread — and its third ON-source file was being processed (17:44:34) while DDO210's GPU encode ran (17:44:27 -> 17:44:52; GPU util peaked at 63%). Resource-monitor DB for the run: system CPU averaged ~34% during ED (bursting to 100%) and ~49% during extraction — with the per-window scipy cost gone, ED is now bounded by
/datagreads + bitshuffle decode rather than CPU, which is the "minutes, I/O + spline bound" regime the plan predicted (the "CPU near-saturated" expectation is superseded by the bottleneck moving to I/O).Hit-set / artifact comparison vs the July 2026 baseline artifacts (identical config: verified
config_test_v17.jsoncarries the same window/step/threshold/spline/stamp/overlap parameters)stored_width=512,downsample_factor_applied=8in metadata.test_v17RF at threshold 0.99 produced no positives in July either — theinference_resultstable has no rows for any prior tag), so the candidate sets are trivially equal./dev/shmempty post-run, benchmark preprocess dir + temp logs removed, cluster repo restored tomaster.Deferred validation
/datag(blpc3-only).test_v17model yields zero candidates at threshold 0.99, so the "equal candidate sets" check reduces to 0 == 0 here; the stamp-level exactness above (identical inputs to the encoder, bit-identical downsample math) is the substantive equivalence evidence.AI assistance disclosure
Implemented with Claude Code (Anthropic). Extent: implementation, unit tests (including the recorded real-data fixture), documentation/README regeneration, multi-angle code review, and local + cluster verification runs.
🤖 Generated with Claude Code