[perfloop]

Startup recovery now keeps the !compact row when a parsed raw file validates, so the existing compactor can resume migration after restart. A shared publication gate also lets readers open a raw source before the compactor replaces its metadata and reclaims that source.

Recovery still removes stale, orphaned, size-mismatched, and missing entries, and it skips empty RocksDB writes. Compaction checks that the encoded record and footer fit the configured segment capacity. Entries that do not fit remain raw and indexed until storage restarts with enough capacity.

TestRecoveryRestartPreservesPendingCompaction checks a no-expiry medium Put with no disk cap, reopening the same disk with unchanged thresholds and SegmentSize, exact Get bytes, normal migration, and raw-source reclamation without another Put. TestValueReadersCoordinateWithCompactionPublication forces the handoff for Get, GetWithVersion, and value listing. Point readers hold the shared database-path gate from before metadata lookup through raw-file open; value listing holds it across its iterator reads. The compactor syncs the segment, publishes metadata under the exclusive gate, and queues the raw source only after that write. The existing per-file lock protects an opened stream. No consumer API, configuration, protocol, or stored format changes.

The original-defect check reports violated at 66b97e5abf6a0ec31394974d0bb5eb4c6a1e072c and satisfied on the final source.

Recorded proof ran with retained support that is not included in this pull request.

The storage, server, embedded, compaction, selected workflow, focused race, lint, and license checks passed. Full repository CI remains pending. Production frequency and resource impact, including the gate's runtime cost, remain unmeasured.

---

Generated by Perfloop. Measurements and checks. Reproduce this comparison (tenant access required).

---

Note

Medium Risk
Changes core storage compaction, startup recovery, and read paths with a global publication lock on raw-file handoffs; behavior is heavily tested but affects data migration timing and concurrent read/compaction semantics.

Overview
Fixes pending raw-file compaction after restart and safe RAW_FILE→SEGMENT handoff during reads.

Startup recovery no longer strips every compaction-index row: valid entries stay indexed so the background compactor can resume; corrupted, orphaned, stale, and missing rows are still removed (with metadata cleanup where needed), and empty RocksDB batches are skipped.

Compaction now treats encoded record + segment footer as the capacity check—oversized work is deferred (index row kept, object stays raw) instead of forcing rotation; segment acquisition reserves enough space. Metadata publication after a synced segment is serialized with readers via a shared DB-path publication gate (read lock from metadata lookup through raw-file open; exclusive lock on commit). Comments/docs clarify that this gate is not a CAS ordering mechanism.

Readers (Get, GetWithVersion, paginated list-with-values) participate in that gate for RAW_FILE paths; large-object dangling-file self-heal commentary is updated accordingly.

New integration tests cover restart preservation, reader/compactor coordination, and capacity-shrink deferral.

^{Reviewed by Cursor Bugbot for commit 9c8a440. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Confidence Score: 5/5

[Critical risk] Changes how raw files are compacted and recovered on restart.

The PR appears safe to merge based on the reviewed changes.

Summary

The PR preserves valid pending-compaction rows through startup recovery, coordinates raw-file readers with compaction publication, and prevents compaction records from exceeding segment capacity.

• Recovery skips RocksDB writes for batches with no mutations.
• Restart, reader-handoff, and capacity tests cover the changed behavior.

_{Reviews (2) · Last reviewed commit: "fix: coordinate readers with compaction ..."}