[Bug] composefs-native: Weird fs-verity errors

Greeter comes up but login fails and loops, because cosmic-session package is missing in the deployment, identified by rpm -qa cosmic-session returning empty, and /usr/share/wayland-sessions/cosmic.desktop missing.
see reply below

Log reveals several, curious fs-verity errors:

Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249306): FILE CORRUPTED! pos=1966080, level=-1, want_hash=sha512:407d87dbf5e656c831944d16c426c2f13ba8d8bee4eae484e9b4633c3e3787edc764a8899c5690f14b4e16e20663ff003bf944008a92884a9d8ef8d20a058e2e, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249306): FILE CORRUPTED! pos=2621440, level=-1, want_hash=sha512:319d843febf7a2c4fbd0504c2f87dafea841816172124e1f3f14c1153666d69480e0d75c0e5a50ddf0fa8c393fb459780471e724ffbe6a7772ce387af2114f18, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249306): FILE CORRUPTED! pos=2228224, level=-1, want_hash=sha512:43e3994577ff5f3a125cd4e10c6b1ce21ce17617adef7ee971e3201a6f2d76b0212790470ead010b536d55cc569b048e3146efd9898625870eccc1dde6448cb5, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=1835008, level=-1, want_hash=sha512:190bfb0666908e4c295e56cb4cbcaba0f7e3f2d802307227a8816fc16ef4ba9dae557fe9cac38c3e57158f07056dac2e3fbef89fce3513ec0392b3e48a3fea45, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=2228224, level=-1, want_hash=sha512:30d033fd992fc3217b5b4349e42539698438af3ed2b14b7fc5bd10f7706ab06635693f2534a64898523d69539e930da72cd38ae05d4660e0293b3442c9286b0e, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=3407872, level=-1, want_hash=sha512:c1896860683a55cad0b2cfe8cb2047b0a8efcee09644298828ed6e1d920c44a6ec89ce36988b1f8d5f1a4318baba2f86414eb607cee67ab5ee96f6b1c3a62a75, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:36 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=3145728, level=-1, want_hash=sha512:148f90d6e551e193625967a919308c0c4fa76426b755d6c4b841030b89dadec10b041e7d3b9d7a1cacfb7dc759c0a5f7a93cfcf4aaa4c1346e6ca9e137b3c13a, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:37 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=4063232, level=-1, want_hash=sha512:b7bfbada6e729b0c7e2978391d5468ec52a042a77a2c861fba0d930ee82347f4f41ea934443a4daa50e86d4ffb02bbed0a130f845eb589da0ad245f5ac1e28d5, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:37 fedora-laptop kernel: fs-verity (dm-0, inode 249598): FILE CORRUPTED! pos=3801088, level=-1, want_hash=sha512:4a1504dc5e944d2397a074ee3bc2fa85b397e73eb8ec5fff9d4b8c59573a07324a8e6273d5a10b178808ca56ccaf41c817510348ee2a7c6ddafac24ae9edeab5, real_hash=sha512:2d23913d3759ef01704a86b4bee3ac8a29002313ecc98a7424425a78170f219577822fd77e4ae96313547696ad7d5949b58e12d5063ef2ee063b595740a3a12d
Sep 17 09:14:53 fedora-laptop kernel: fs-verity (dm-0, inode 1444220): FILE CORRUPTED! pos=23040000, level=-1, want_hash=sha512:774f1b8345ab98cfca85fcb31c83b86433eb0b6191828226fe142856fec26d28371a8640b2ffe7bee4314ed3f3cd71be0e4233e18607a10d0d6d87434e44c3b9, real_hash=sha512:053a879ef60a6f4b0c6daa8428a60e6262ef59f3a2389b830994ff7acd4a2d7aa1bf6dd924bbb2b52f0b19dd35279ca8b545d7b67ad3bdc3d8ce5c76e368a0f2

The curious part is that the wanted and found hashes make no sense, there are:

Three different want_hash values for 249306,
Six different want_hash values for 249598,
249306 and 249598 have the same found_hash values
But do not actually point to the same file on disk as seen below

249306: /sysroot/composefs/objects/27/b3751bd5e5ad8d2dff4d98837e574721b5075094dbfdf86417bcbb3ce90482547656eb0958e30fff42c73d0df2652886e5db69c90c3021933550836323caf7

249598:
/sysroot/composefs/objects/1b/362f004ed42defd8e400883f382a500bb421ea186e326c429d1475923662d60277365ad6843d3bb4f554aee7d1b81cf31fb1876064e02ebaadcc3a52f44438

1444220:
/sysroot/composefs/objects/92/4eba01ab7a7c2865874593472d1b77a3e108a919c6680c88c61b09bb6adedcfe217a59e10dd65d33b62a4e87940604a1c1cf9dbb7c9e04d4561534313920f7

The issue persists after deleting and redownloading the deployment:

🦸 [root@fedora-laptop hooly]# bootc status
● Booted image: ghcr.io/secureblue/cosmic-main-hardened:uki
        Digest: sha256:e345f75fca78fac0cab1494944c266546f39e766e0eba7945fbdba14ed302497 (amd64)
        Verity: cba33619b1512ce05e9b4154f15f4b6e1afa9cc26d7cc672530ae6188c07bdd348b30ea6d694311dac8e8b3355cf362224c6ef30c095920648591e716e4f83a5
     Timestamp: 2026-09-12T00:11:12Z

  Rollback image: ghcr.io/secureblue/cosmic-main-hardened:uki
          Digest: sha256:80ebcf6cefc533038d58faa27f16b1deba369dea39e0ff5d4305353b3d1f8321 (amd64)
          Verity: 5b03aa85aa92715cb444cb8cbd722a9d89183ef36454afc1651dfba3d49958005e3316926ab03cee96644130f297f9eecd7c785bd00ee0620db15a03aaaf0ea0
       Timestamp: 2026-09-16T12:37:33Z
🦸 [root@fedora-laptop hooly]# bootc delete-deployment 5b03aa85aa92715cb444cb8cbd722a9d89183ef36454afc1651dfba3d49958005e3316926ab03cee96644130f297f9eecd7c785bd00ee0620db15a03aaaf0ea0
🦸 [root@fedora-laptop hooly]# bootc status
● Booted image: ghcr.io/secureblue/cosmic-main-hardened:uki
        Digest: sha256:e345f75fca78fac0cab1494944c266546f39e766e0eba7945fbdba14ed302497 (amd64)
        Verity: cba33619b1512ce05e9b4154f15f4b6e1afa9cc26d7cc672530ae6188c07bdd348b30ea6d694311dac8e8b3355cf362224c6ef30c095920648591e716e4f83a5
     Timestamp: 2026-09-12T00:11:12Z
🦸 [root@fedora-laptop hooly]# bootc upgrade
Fetching config sha256:80be46064733fc78bd2c4aab0ddf8e6e72cba0623afe8227790028b1d7474999                                                                                          Storing manifest sha256:80ebcf6cefc533038d58faa27f16b1deba369dea39e0ff5d4305353b3d1f8321                                                                                         🦸 [root@fedora-laptop hooly]#

We have confirmed that the session file is not missing because of the fs-verity errors, they are probably worth investigating regardless.

I get the same errors.

$ sudo bootc status
● Booted image: somewhere/sealed/custom-kinoite:44
Digest: sha256:7f556e992928ee17245917664468a5bb39e8e8976978b97c8baf0af44544c279 (arm64)
Verity: 1b3b8bec40af987ee5b53c2e86d2e8984ba0731641b10a2089ce8cfc3611e4964f86180772e4a23b984d221bb246a035b7bfc56fabab666325f19c7a59a93c59
Timestamp: 2026-09-25T13:13:05Z

Sep 26 12:02:28 yuki-kino kernel: fsverity_msg: 25 callbacks suppressed
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=1429504, level=-1, want_hash=sha512:75e40a81540eabf188f379850>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=1560576, level=-1, want_hash=sha512:65358e98a0823b232cbfa24c0>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=1605632, level=-1, want_hash=sha512:4bd0f159ec0777770204e0855>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=1736704, level=-1, want_hash=sha512:403f32c7f009ab27c22f3cb34>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=1966080, level=-1, want_hash=sha512:675ce285b173b3418adfedb06>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=2129920, level=-1, want_hash=sha512:b96f8192e3d36a7af619a092f>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=2228224, level=-1, want_hash=sha512:951090e84171b57bac2ea5539>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=2523136, level=-1, want_hash=sha512:68cfad18d7ccf1e670b65116c>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=2392064, level=-1, want_hash=sha512:1944a720ba08acab67e113a16>
Sep 26 12:02:28 yuki-kino kernel: fs-verity (sda2, inode 300697): FILE CORRUPTED! pos=29646848, level=-1, want_hash=sha512:6e54c0ebb8d105dafa6dfbd0>

@cgwalters

@cgwalters Is this also #2428 (comment)? Has that composefs-rs change shipped in bootc already or is it awaiting release?

@cgwalters

@cgwalters Is this also #2428 (comment)? Has that composefs-rs change shipped in bootc already or is it awaiting release?

I asked Copilot chat to look, it says

First bootc release containing it: 1.16.4, published July 15, 2026; its release notes explicitly include #2295. (github.com)

But that won't fix existing objects.

I think changing bootc internals fsck to fix this would make sense.

But we should handle this automatically: detect the case where the system was likely provisioned (e.g. /composefs directory creation timestamp predates that release) and then do a one-time pass over all objects and re-check their verity digest and repair the ones which are incorrect.

That said, I'm not fully sure that that patch was the cause - it just smells likely to me. One issue is if true there's some nondeterminism here. As of recently we do run bootc internals fsck on each CI in our readonly suite, which should be caching this now when it happened, but I think that was relatively recent.

@cgwalters-bot so let's try this: do a composefs based install starting from a bootc which predates that. Can you reproduce incorrect verity digests? An interesting thing to try here would be to e.g. ptrace the skopeo process and introduce delays and see if that helps.

Oooh...could this be a lot more likely when doing registry fetches? I bet almost certainly so...we aren't exercising that enough here.

As far as fixing existing objects though, it is kind of tricky because there's AFAIK no API to disable fsverity in that way, we'd have to (reflink ideally) the object, enable verity on it, then rename over.

I reproduced the FILE CORRUPTED messages, but they aren't bad objects, and they show up with 1.16.4 just as with 1.16.3. They look like a btrfs readahead bug with fs-verity on fragmented files (guest kernel 7.2.8-200.fc44). Plain files with no composefs involved show it too. composefs-rs d409ec7 (in bootc 1.16.4) isn't the cause.

Method. I used fedora-bootc:44 VMs (bcvk libvirt, btrfs, --composefs-backend) with bootc 1.16.3 or 1.16.4 from koji. In each VM I ran 25 bootc internals cfs oci pulls into fresh (verity-enabled) repos, using the composefs-rs that bootc vendors (not bootc upgrade itself), from a local registry:2 (gzip/zstd) and from containers-storage. The pulls ran plain, with skopeo under strace (delayed writes), at CPUQuota=10%, and with netem 20ms/100mbit. After every pull I compared each object's name with fsverity measure, dropped caches, re-read every object, and counted kernel messages.

Results. Across 50 pulls (~1.7M object checks) plus 10 cold-read passes over the installed repos there were 0 name/digest mismatches, 0 objects without verity and 0 read errors. bootc internals fsck and cfs fsck on the installed repos were clean (seen in the VMs; that output isn't in the gist logs). The FILE CORRUPTED lines (mostly real_hash = the zero-block hash, as in the reports) appeared on cold re-reads. Counts are small and noisy:

condition (btrfs)	1.16.3	1.16.4
installed repo, 5 cold-read passes	1/5	4/5
registry pull, plain / zstd / strace delay / netem	13/13 pulls	13/13 pulls
registry pull at CPUQuota=10%	0/3	0/3
containers-storage via skopeo (plain, strace)	4/6	2/6
containers-storage native (reflink)	0/3	3/3
ext4 (installed repo + 3 registry pulls)	0/6	–

Mechanism. 8 of 9 bad blocks I mapped sit exactly on an extent boundary. Registry pulls write objects in parallel, so they come out fragmented (mean 1.32 extents, up to 106), while the installed repo, reflinked from containers-storage, averages 1.01. The throttled pulls being clean fits that, though I didn't measure their extents. A composefs-free script that writes interleaved chunks to make ~200-extent files, enables verity and cold-reads them gives the same messages. With btrfs read_ahead_kb=0 both that and the pulled repo gave none (3 passes and 1 pass). So it looks like readahead across an extent boundary hands fs-verity an unfilled folio, verification fails, and the later synchronous read succeeds. That's why dd works. You were right that registry fetches hit it more, but because of on-disk layout, not digests. Fedora saw a similar signature in 6.16.8 (fixed by "btrfs: fix incorrect readahead expansion length").

Side finding: 1.16.3 and 1.16.4 installs leave /sysroot/composefs/meta.json without verity (the install preflight calls init_path(... set_insecure()) first), so the repo opens in insecure mode and missing verity on objects would go unnoticed. Main has reworked this.

Next steps

Report to linux-btrfs/fsverity with the frag.sh reproducer. A bisect against older Fedora kernels would narrow the window. I haven't tested anything below 7.2.8.
No on-disk repair is needed for these messages. The gist has a design note for an fsck repair pass covering the cases that could exist: objects without verity (reflink, enable verity, rename over) and name/digest mismatches (re-pull the referencing images). It suggests using "meta.json lacks verity on a non-insecure boot" as the auto-detect trigger, since the /composefs timestamp would miss 1.16.4+ installs.

Scripts, logs and the design note: https://gist.github.com/cgwalters-bot/515bbd58a7d26262bde80f563c6cb538

Generated-by: https://github.com/cgwalters/#llms

Every day is like this continual conflict between LLM exhilaration and exhaustion. I'd still un-invent them if I could. I hate that I'm warming the planet (in another way) and contributing to wild data center growth. But yeah I mean, I probably could have debugged that myself over multiple days...but it sure is amazing to not have to have done it.

Side finding: 1.16.3 and 1.16.4 installs leave /sysroot/composefs/meta.json without verity (the install preflight calls init_path(... set_insecure()) first), so the repo opens in insecure mode and missing verity on objects would go unnoticed. Main has reworked this.

Ah, interesting that is an upgrade bug indeed; hm do we have that semantic w/sealed UKIs? If so should definitely fix - when doing another upgrade if the UKI requires verity (and we've mounted successfully) then let's just go ahead and flip on verity on meta.json too.

Report to linux-btrfs/fsverity with the frag.sh reproducer. A bisect against older Fedora kernels would narrow the window. I haven't tested anything below 7.2.8.

Well if we think it's a filesystem bug, simplest is to drop out bootc stack from this, use bcvk ephemeral to launch VMs with file-backed blockdevs and initialize various linux filesystems on those. I'd also go over the commit logs from the Linux kernel git.

Can you reproduce w/xfs or ext4?

I haven't tested anything below 7.2.8.

Also, may be interesting to test with say c10s era kernel w/btrfs (which should now be soft-supported there for external filesystems)

The FILE CORRUPTED lines (mostly real_hash = the zero-block hash, as in the reports) appeared on cold re-reads.

Ah hah yes of course with a warm page cache we won't see it as often. This has echoes of the prior bug on this...and it also explains why people don't see a constant spam of this of course.

@cgwalters-bot

Also @cgwalters-bot I just saw https://git.kernel.org/pub/scm/fs/fsverity/linux.git/commit/?h=for-current&id=4ec9b3758dcb0c2c58bbdc6f97126c67e820ee99

Thanks for the pointer. Short answer: that commit is real, but it isn't our bug. I read it and A/B tested it.

What it fixes. "fsverity: RCU-delay the freeing of struct fsverity_info" is a use-after-free in __fsverity_get_info(): the rhashtable chain walk for one inode can touch the rhash_head/inode of a different inode's fsverity_info that is being freed at eviction. It fixes f77f281b6118 ("use a hashtable to find the fsverity_info", Feb 2026, so 7.0 and later). Only the freeing is deferred with call_rcu; it changes nothing about how data blocks are verified.

Why it doesn't match. Our messages are FILE CORRUPTED! level=-1 with real_hash equal to the zero-block hash, i.e. the data folio was not yet filled when verification ran. That points at the data-read path, not at looking up the wrong fsverity_info. The lookup race needs concurrent eviction of other inodes and would fail with oopses or lookup mismatches, not a clean zero-hash on an extent boundary. It is also filesystem-independent, but ext4 is clean, and we saw 8 of 9 bad blocks on btrfs extent boundaries and nothing with read_ahead_kb=0.

Test. frag.sh (btrfs on a loop file, 8 files of 200 extents, cold read x12) in bcvk VMs on a 16-core devspace (kernel messages per round):

fedora-bootc:44 stock 7.2.8-200.fc44: 7 of 12 rounds with messages (1 to 4 each).
vanilla 7.2.8 with the Fedora config, unpatched, 2 runs x 12 rounds: 5/12 and 6/12 rounds with messages.
the same kernel with 4ec9b3758dcb applied (patch applied cleanly), 2 x 12 rounds: 7/12 and 7/12 rounds with messages. No change.
fedora-bootc:42 stock 6.19.14-101.fc42 (before f77f281b6118): 0 messages in 16 rounds (4 + 12).

(The custom kernels are vanilla 7.2.8 built with Fedora's config, with btrfs/fuse/virtiofs/loop built in and the modules and initramfs rebuilt, so they are not byte-identical to Fedora's. The stock and unpatched-custom results agree, which is the control.)

Result. The commit doesn't fix our reproduction, so I'd drop it as the cause. The 6.19 vs 7.2.8 difference is the useful finding: the reproducer is clean on 6.19.14 and fails on 7.2.8, so this is a regression somewhere in the 7.0 to 7.2 range (btrfs and/or the fsverity readahead and large-folio changes that landed with the same hashtable series). I haven't bisected it. A bisect between v6.19 and v7.0 using this script in a VM, which takes about 10 builds of 3 to 5 minutes each here, is the next step before reporting it to linux-btrfs and fsverity.

For the bootc side. No change to the plan: nothing on disk is wrong, so no repair pass is needed for these messages, and they should not drive an auto-repair. They are also not evidence of a bad object. If we want an auto-detect, it should key off fsverity measure mismatches and missing verity, never off dmesg FILE CORRUPTED lines. The practical workaround is that the reads succeed on retry. The fix has to come from the kernel, and affected kernels are 7.0+ on btrfs.

Generated-by: https://github.com/cgwalters/#llms

Transparency note: This comment was written by Claude Opus 5.5 (an AI assistant by Anthropic) and posted with the account owner's permission. The measurements come from the account owner's machine. Please treat it like any other user report; the account owner can answer follow-up questions.

We see the same FILE CORRUPTED! messages on a composefs-native bootc system (Arch Linux, bootc 1.17.1, kernel 7.2.9-arch1-1, btrfs). Our differential test suggests that btrfs compression, not fragmentation, is the trigger. Note that Fedora mounts btrfs with compress=zstd:1 by default, so the fragmented files in the VMs above may also have been compressed.

Observations on the installed system (/ mounted with compress=zstd:3):

251 logged messages over 6 boots. Every pos is a multiple of 131072 (128 KiB, the maximum uncompressed size of a btrfs compressed extent), and real_hash is the hash of a zero block in every case.
All 5 affected objects consist almost entirely of compressed extents (filefrag -v "encoded": 109/109, 25/25, 33/33, 19/19, 280/282). btrfs device stats are all 0.
drop_caches + a full read reproduces it every time. Userspace never got EIO, and sha256 stays correct. The messages are rate-limited, so the journal undercounts.
None of our objects share extents with containers-storage (0 of 300 sampled objects show "shared").

Differential test on plain files outside composefs (copy of /usr/bin/podman, 45 MB, on the same btrfs, compress=zstd:3). fs-verity is enabled via FS_IOC_ENABLE_VERITY (SHA-512, 4K blocks). Each round runs sync, drop_caches=3 and a full read, 5 rounds per file:

file	extents	fs-verity	logged `FILE CORRUPTED`	wrong data / EIO
A: compressed (mount option)	343 (339 encoded)	yes	50 (10/round, rate-limited)	0 / 0
B: `FS_NOCOMP_FL`, contiguous	1	yes	0	0 / 0
C: compressed	343 (339 encoded)	no	–	0 / 0 (sha256 matches every round)
D: `FS_NOCOMP_FL`, fragmented (128K writes interleaved with another file, fsync each)	65	yes	0	0 / 0

So on this kernel, fragmentation alone (D) does not trigger it, but compressed extents with fs-verity (A) do. Reads without verity return correct data (C), so it looks like the verity check runs on a not-yet-decompressed or zero-filled folio in the compressed readahead path, and the subsequent synchronous read succeeds. The April 2026 series "btrfs: properly enable compressed readahead for bs < ps and large folio cases" may be worth a look when bisecting, but we haven't verified that.

Mitigation for composefs repos on btrfs: composefs-rs creates objects as O_TMPFILE in objects/ and then calls linkat. A btrfs inode inherits BTRFS_INODE_NOCOMPRESS from its parent directory, so chattr +m <repo>/objects (an inode flag, not an xattr) makes all new objects uncompressed. Setting FS_NOCOMP_FL on the objects directory in composefs-rs, at least on btrfs with verity, might be a cheap default. Existing objects would need the copy → enable verity → rename-over pass mentioned above. The cost here is small: the whole repo is 6.6 GB uncompressed vs 4.2 GB on disk.

The reproducer script (Python, stdlib only, run as root, args: SOURCE_FILE [DIR_ON_COMPRESSED_BTRFS]) is below. We haven't reported this to linux-btrfs ourselves, since there's no mail setup on this machine. If you're already preparing that report, feel free to include this data.

verity-compress-repro.py (comments and output in German, sorry)

#!/usr/bin/python3 -I
# Reproduzierer (Kernel-Bugbericht): btrfs-Kompression + fs-verity -> falsche "FILE CORRUPTED"-Meldungen.
# Aufruf als root: verity-compress-repro.py QUELLDATEI [ZIELORDNER auf btrfs mit compress=zstd]
# Drei Kopien der Quelle im Zielordner:
#   A: komprimiert (Mount-Option), fs-verity an         -> erwartet: Meldungen (Bug)
#   B: FS_NOCOMP_FL (chattr +m), fs-verity an           -> erwartet: keine
#   C: komprimiert, OHNE fs-verity, sha256 je Runde      -> zeigt, ob ohne verity falsche Daten ankommen
# Je Runde: sync, drop_caches=3, Datei komplett lesen; neue Kernelmeldungen zur Inode zählen (gedrosselt -> Untergrenze).
import fcntl, hashlib, os, struct, subprocess, sys, time

SRC = sys.argv[1]
D = (sys.argv[2] if len(sys.argv) > 2 else '/var/tmp') + '/verity-repro'
ROUNDS = 5
FS_IOC_ENABLE_VERITY, FS_IOC_SETFLAGS, FS_IOC_GETFLAGS, FS_NOCOMP_FL = 0x40806685, 0x40086602, 0x80086601, 0x400

def enable_verity(p):
    fd = os.open(p, os.O_RDONLY)
    try: fcntl.ioctl(fd, FS_IOC_ENABLE_VERITY, struct.pack('=IIIIQIIQ88x', 1, 2, 4096, 0, 0, 0, 0, 0))
    finally: os.close(fd)

def make_frag(name):
    p = f'{D}/{name}'; q = f'{D}/fueller'
    fd = os.open(p, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600); fq = os.open(q, os.O_WRONLY | os.O_CREAT, 0o600)
    for x in (fd, fq):
        b = bytearray(8); fcntl.ioctl(x, FS_IOC_GETFLAGS, b)
        fcntl.ioctl(x, FS_IOC_SETFLAGS, struct.pack('=Q', struct.unpack('=Q', b)[0] | FS_NOCOMP_FL))
    with open(SRC, 'rb') as s:
        while c := s.read(128 << 10):
            os.write(fd, c); os.fsync(fd); os.write(fq, os.urandom(4096)); os.fsync(fq)
    os.close(fd); os.close(fq); os.unlink(q); subprocess.run(['sync'])
    enable_verity(p)
    return p

def make(name, nocomp, verity):
    if name.startswith('D-'): return make_frag(name)
    p = f'{D}/{name}'
    fd = os.open(p, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
    if nocomp:
        b = bytearray(8); fcntl.ioctl(fd, FS_IOC_GETFLAGS, b)
        fcntl.ioctl(fd, FS_IOC_SETFLAGS, struct.pack('=Q', struct.unpack('=Q', b)[0] | FS_NOCOMP_FL))
    with open(SRC, 'rb') as s:
        while c := s.read(1 << 20): os.write(fd, c)
    os.fsync(fd); os.close(fd); subprocess.run(['sync'])
    if verity: enable_verity(p)
    return p

def kcount(ino):
    out = subprocess.run(['journalctl', '-k', '-b', '--no-pager', '-o', 'cat'], capture_output=True, text=True).stdout
    return sum(1 for l in out.splitlines() if f'inode {ino}): FILE CORRUPTED' in l)

def sha(p):
    h = hashlib.sha256()
    with open(p, 'rb') as f:
        while c := f.read(1 << 20): h.update(c)
    return h.hexdigest()

print(f'Kernel {os.uname().release}; Quelle {SRC} ({os.path.getsize(SRC)} Bytes)')
print('Mount:', subprocess.run(['findmnt', '-no', 'OPTIONS', '-T', os.path.dirname(D)], capture_output=True, text=True).stdout.strip())
ref = sha(SRC)
os.makedirs(D, mode=0o700, exist_ok=True)
res = {}
try:
    for name, nocomp, verity in (('A-komprimiert-verity', False, True), ('B-unkomprimiert-verity', True, True), ('C-komprimiert-ohne-verity', False, False), ('D-unkomprimiert-fragmentiert-verity', True, True)):
        p = make(name, nocomp, verity); ino = os.stat(p).st_ino
        fr = subprocess.run(['filefrag', '-v', p], capture_output=True, text=True).stdout
        enc = sum(1 for l in fr.splitlines() if 'encoded' in l)
        print(f'{name}: Inode {ino}, komprimierte Extents {enc}, {fr.strip().splitlines()[-1]}')
        msgs = eio = bad = 0
        for r in range(ROUNDS):
            time.sleep(6)
            before = kcount(ino)
            subprocess.run(['sync']); open('/proc/sys/vm/drop_caches', 'w').write('3\n')
            try:
                h = sha(p)
                if h != ref: bad += 1; print(f'   Runde {r+1}: FALSCHE DATEN sha256 {h[:16]}… (Quelle {ref[:16]}…)')
            except OSError as e:
                eio += 1; print(f'   Runde {r+1}: Lesefehler {e}')
            time.sleep(1); n = kcount(ino) - before; msgs += n
            print(f'   Runde {r+1}: Meldungen {n}')
        res[name] = f'komprimierte Extents {enc}, verity-Meldungen {msgs}, Lesefehler {eio}, falsche Daten {bad}/{ROUNDS}'
        os.unlink(p)
finally:
    for n in os.listdir(D): os.unlink(f'{D}/{n}')
    os.rmdir(D)
print('ERGEBNIS:'); [print(f'  {k}: {v}') for k, v in res.items()]

Navigation Menu

Repository navigation

[Bug] composefs-native: Weird fs-verity errors #2471

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug] composefs-native: Weird fs-verity errors #2471

Description

Activity

TheHooly commented on Sep 17, 2026

jnperry commented on Sep 26, 2026

RoyalOughtness commented on Sep 28, 2026

cgwalters commented on Oct 2, 2026

cgwalters-bot commented on Oct 2, 2026

cgwalters commented on Oct 2, 2026

cgwalters commented on Oct 2, 2026

cgwalters commented on Oct 2, 2026

cgwalters-bot commented on Oct 2, 2026

gauner666 commented on Oct 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions