fix(sizes): remove Logical mode — apparent size becomes a classified annotation (sparse/compressed)#31
Merged
Merged
Conversation
…annotation A global "Logical" toggle re-weighted the whole map by st_size, producing un-actionable readings on sparse files (three 512 GiB com.apple.container snapshots occupying ~1.3 GiB; a 2.42 TiB "total" on a 512 GiB volume). The data was correct; the scope was wrong — nobody needs the apparent size of the whole disk, but everyone needs it explained where it diverges. - Everything renders on-disk (allocated blocks): treemap weights, outline, totals, file types. SizeMetric and its cache plumbing are gone. - The scanner requests ATTR_CMN_FLAGS in the same getattrlistbulk call (4 bytes/entry, no extra syscall): UF_COMPRESSED tells compressed apart from sparse (physical < logical without the flag). - FSNode aggregates the gap by cause (aggSparseExcess/aggCompressedExcess, two atomics per directory) through scan, SPEC-11 restat, invalidate, deletions, and the streamed VM scanner (blocks < bytes = sparse). - SizeDivergence annotates where notable (>= 1.5x and >= 8 MiB over): dashed badge + tooltip in the outline, both figures in the breadcrumb, an "apparent" stat in the strip, hover pill and File-types tooltips. "Apparent" is the du --apparent-size vocabulary. - Headless scan prints the split; verified against the real snapshots dir (1.25 GiB on disk / 1.50 TiB apparent, classified sparse) and a system app (compressed). 107 tests pass, 4 new (thresholds, ftruncate sparse, ditto --hfsCompression, exact-mode excess dedup).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The global On disk / Logical toggle re-weighted the entire app by
st_size. On sparse files the result read as a bug while being technically correct: threecom.apple.containersnapshots showed 512 GiB each (~1.3 GiB actually allocated), and the volume "total" reached 2.42 TiB on a 512 GiB disk. The data was right; the scope was wrong — no user question is answered by the apparent size of the whole disk, but "why does this file look huge / what happens if I copy it" needs answering in place.Approach
One truth, annotated where it lies:
SizeMetricand all its dual-dimension cache plumbing (sortCacheMetric,FileListing.metric, layout invalidation by metric, world resets on metric flips) are removed.ATTR_CMN_FLAGSrides the existinggetattrlistbulkrequest (4 bytes/entry, no extra syscall).UF_COMPRESSED⇒ compressed;physical < logicalwithout it ⇒ sparse (block padding goes the other way, so it can never false-positive).FSNodeaggregates the gap by cause —aggSparseExcess/aggCompressedExcess, two atomics per directory node (~11 MB on a 680k-folder scan), maintained through the parallel scan, SPEC-11 restat,invalidate, file/directory deletion (incl. exact-mode hardlink dedup), and the streamed VM scanner (blocks < bytes⇒ sparse — the Docker.raw case; ext4 has no transparent compression to confuse it with).SizeDivergenceannotates where notable (≥ 1.5× and ≥ 8 MiB over):.ext4case) get the same tooltipdu --apparent-sizevocabulary — the term the audience can verify in a shell.Verification
ftruncate), real compressed file (ditto --hfsCompression,UF_COMPRESSEDasserted), exact-mode excess dedup across hardlinks.com.apple.container/snapshots:on-disk 1.25 GiB · apparent 1.50 TiB · sparse 1.50 TiB not allocated✓/System/Applications/Calculator.app:on-disk 3.45 MiB · apparent 5.74 MiB · compressed 2.32 MiB saved✓2.30 GiB total · 2.00 TiB apparent, selection reads694 MiB · 513 GiB apparent.Out of scope (deliberate)