fix(#696): §4.7.1 gap origin = read invocation start, not summary start

[FileSystemGuy]

Summary

• Fixes the three-way contradiction reported in §4.7.1: mandated write/read split can't meet its own ≤30s gap (read-invocation startup ~46s); submission_checker accepts combined mode — please reconcile #696 between Rules.md §4.7.1 prose, the submission checker's behavior, and physical achievability. The prose demanded a split-mode workflow with a ≤30s gap measured from write.summary.end_time to read.summary.start_time — but the read invocation's own framework startup (Python imports, MPI spawn, CAP env-validation, ~50s in the reporter's evidence) is recorded inside start_time, so the mandated split hard-fails 4.7.1 on a legitimate configuration. Meanwhile the checker also accepted combined mode with no gap check, so a submitter following the prose failed while a submitter following the checker passed.
• Reframes §4.7.1 around what it was actually meant to police — the submitter's failover callout between the writing and reading clients — not framework startup. Split mode's real purpose is to enable storage architectures that need an external callout (e.g., a submitter script) to complete the client-to-client handoff; the 30-second bound exists to prevent long manual procedures, not to time cache flushing.
• Emits a new invocation_start_time field in run metadata, captured at mlpstorage main.py entry before any heavy imports. Storage-only change; zero DLIO edits.

Details

Rules.md

• §4.7.1 rewritten: universal write→read ordering (both modes) up front; combined vs. split described by workload semantics rather than CLI flag names; split-mode's purpose reframed as failover callout (with cache flush as one common in-callout activity, not the point of the split); gap origin defined as read.metadata.invocation_start_time − write.summary.end_time; negative gap treated as NTP clock skew, not a causality violation.
• Table 3 footnote: "two-invocation cache-flush workflow" → "two-invocation failover workflow".

Code (storage-only)

• mlpstorage_py/_invocation.py (new): captures INVOCATION_START = time.time() at first import.
• mlpstorage_py/main.py: imports _invocation before any heavier mlpstorage_py module so the timestamp is recorded before framework startup.
• mlpstorage_py/benchmarks/base.py: Benchmark.metadata emits invocation_start_time (ISO local naive, matching DLIO's summary.json format) for every run — write, read, and combined.
• mlpstorage_py/submission_checker/checks/checkpointing_checks.py::cache_flush_validation: reads the gap-start from read_metadata["invocation_start_time"] instead of read_summary["start_time"]. New "missing invocation_start_time" error tells older submissions to regenerate. Negative gap → warn_violation (clock-skew note), not a hard error. The strict no-overlap ordering check remains in checkpoint_invocation_structure unchanged.

Tests

• mlpstorage_py/tests/conftest.py: fixture always emits invocation_start_time in checkpointing metadata. Existing chkpt_cache_flush_gap_seconds kwarg now controls the read metadata's invocation_start_time relative to write summary end_time; may be negative to model clock skew. New opt-out kwarg chkpt_omit_invocation_start_time exercises the missing-field branch.
• test_checkpointing_check_phase2.py: "missing timestamps" assertion updated for the new write-side message; new tests for missing-invocation_start_time (regenerate-required) and negative-gap (clock-skew warning).

Empirical impact on the reporter's evidence

Using the exact timestamps from the #696 evidence:

Quantity	Old check	New check
Origin (read side)	read.summary.start_time = 03:57:12.759	read.metadata.invocation_start_time ≈ 03:56:21.296 (mlpstorage entry, ~60ms after drop_caches ended)
Origin (write side)	write.summary.end_time = 03:56:18.199	unchanged
Gap	54.6 s → FAIL	~3.1 s → PASS

The unmeetable ceiling becomes trivially meetable without loosening the semantic that the rule was actually trying to enforce.

Test plan

• pytest mlpstorage_py/tests tests/unit -q → 3423 passed, 1 skipped, 0 failures.
• cache_flush_validation covered for: combined mode (silent pass), split with 25s gap (pass), split with 45s gap (fail), missing write end_time (fail w/ regenerate hint on write side), missing read invocation_start_time (fail w/ regenerate hint), negative gap (warning, not violation).
• Reviewer: verify the Rules.md §4.7.1 prose accurately describes the failover-callout intent for your storage system.

Closes #696.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅