Skip to content

fix(#186): evsieve build box retries with fuse-overlayfs on overlay-mount failure - #189

Merged
aradanmn merged 1 commit into
mainfrom
fix/186-evsieve-overlay-mount
Aug 1, 2026
Merged

aradanmn merged 1 commit into
mainfrom
fix/186-evsieve-overlay-mount

Conversation

@aradanmn

@aradanmn aradanmn commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Closes #186.

Problem

Found on the Deck during #122 hardware validation, on two separate installs
right after --purge wiped the podman/distrobox store:

```
Preparing mcss-evsieve-build box (first run pulls a debian image) (4s)
evsieve build box could not be created; seamless controller reconnect
unavailable (v1.2 proxy feature stays OFF); your v1.1 install is unaffected.
```

Fails in ~4s — a real error, not a timeout. Fail-open worked exactly as
designed (v1.2 proxy off, v1.1 unaffected, install completes), but seamless
reconnect (#38) silently stops being available on this Deck.

Root cause

Reproduced directly, outside any of our scripts:

```
$ distrobox create --yes --name diag --image debian:12
Error: creating container storage: creating an ID-mapped copy of layer
"...": creating overlay mount to .../merged, mount_data="...userxattr":
invalid argument
```

Podman's native (kernel) overlay driver rejects an ID-mapped mount on this
Deck's kernel (6.16.12-valve24.5-1-neptune). fuse-overlayfs sidesteps
the kernel driver entirely and is already shipped on the Deck — verified
fix, end to end on real hardware:

```
$ distrobox create --yes --name diag --image debian:12
--additional-flags "--storage-opt overlay.mount_program=/usr/bin/fuse-overlayfs"
(create: exit 0; enter: exit 0)
```

Fix

`_evsieve_create_box()` is the one new encoding, called from both places
`_evsieve_ensure_box()` creates a box (the initial create AND the
heal-after-broken-box retry): try the plain form first, and only on
failure
— with fuse-overlayfs actually present — retry once with the
storage-opt via distrobox's --additional-flags passthrough.

A host where the native driver already works sees zero behavior change:
no extra flag, no risk of a fuse-overlayfs-specific quirk on a host that
never needed one.

Tests

`tests/test_evsieve_management.sh`: 11→14, with a distrobox stub that fails
unless its create call carries the exact expected storage-opt value:

  • T12 — a healthy host (plain create succeeds) never even checks for
    fuse-overlayfs, let alone attempts the fallback
  • T13 — plain fails, fuse-overlayfs absent → degrades exactly as before
    (one attempt, clean failure, no retry loop)
  • T14 — plain fails, fuse-overlayfs present → the fallback retry
    succeeds

Mutation-tested, five mutations. Four caught immediately; one (a typo'd
fuse-overlayfs path) initially slipped past because the first test stub
matched any substring containing "fuse-overlayfs" — a typo'd value still
contains that substring. Tightened the stub to an exact last-argument match;
all five caught after.

No CI baseline change — test_evsieve_management.sh has no dedicated CI
gate entry (verified against ci.yml), consistent with the other
network/toolchain-dependent module suites.

Not done

Not yet re-validated on the Deck with this exact fix (the original bug was
found and diagnosed live during #122; this is the permanent fix, verified
manually against real distrobox/podman but not yet through the full
install_evsieve() pipeline on hardware). Will batch into the same Deck
sitting as #184/#185.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dru6wVc2ZcjiTpa6p7yJDh

…ount failure

Found on the Deck during #122 hardware validation, on two separate installs
right after --purge wiped the podman/distrobox store:

    Preparing mcss-evsieve-build box (first run pulls a debian image)  (4s)
    evsieve build box could not be created; seamless controller reconnect
    unavailable (v1.2 proxy feature stays OFF); your v1.1 install is unaffected.

Fails in ~4s — a real error, not a timeout. Fail-open worked exactly as
designed (v1.2 proxy off, v1.1 unaffected, install completes), but seamless
reconnect (#38) silently stops being available.

Root cause, reproduced directly with distrobox/podman outside any of our
scripts:

    $ distrobox create --yes --name diag --image debian:12
    Error: creating container storage: creating an ID-mapped copy of layer
    "...": creating overlay mount to .../merged, mount_data="...userxattr":
    invalid argument

Podman's native (kernel) overlay driver rejects an ID-mapped mount on this
Deck's kernel (6.16.12-valve24.5-1-neptune). fuse-overlayfs sidesteps the
kernel driver entirely and is already shipped on the Deck — verified fix,
end to end on real hardware:

    $ distrobox create --yes --name diag --image debian:12 \
        --additional-flags "--storage-opt overlay.mount_program=/usr/bin/fuse-overlayfs"
    (create: exit 0; enter: exit 0)

_evsieve_create_box() is the one new encoding, called from both places
_evsieve_ensure_box() creates a box (the initial create AND the heal-after-
broken-box retry): try the plain form first, and ONLY on failure — with
fuse-overlayfs actually present — retry once with the storage-opt via
distrobox's `--additional-flags` passthrough. A host where the native driver
already works sees zero behavior change: no extra flag, no risk of a
fuse-overlayfs-specific quirk on a host that never needed one.

tests/test_evsieve_management.sh: +3 (11→14), with a distrobox stub that
fails unless its create call carries the exact expected storage-opt value:
  T12 — a healthy host (plain create succeeds) never even checks for
        fuse-overlayfs, let alone attempts the fallback
  T13 — plain fails, fuse-overlayfs absent -> degrades exactly as before
        (one attempt, clean failure, no retry loop)
  T14 — plain fails, fuse-overlayfs present -> the fallback retry succeeds

Mutation-tested (PRINCIPLES #4), five mutations, four caught immediately;
one (a typo'd fuse-overlayfs path) initially slipped past because the test
stub matched any substring containing "fuse-overlayfs" — a typo'd value
still contains that substring. Tightened the stub to an exact last-argument
match before re-running; all five caught after the fix.

No CI baseline change: test_evsieve_management.sh has no dedicated CI gate
entry (checked ci.yml — it isn't in BASELINE), consistent with the other
network/toolchain-dependent module suites.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dru6wVc2ZcjiTpa6p7yJDh
@aradanmn
aradanmn merged commit fc81d7a into main Aug 1, 2026
4 checks passed
@aradanmn
aradanmn deleted the fix/186-evsieve-overlay-mount branch August 1, 2026 13:19
aradanmn added a commit that referenced this pull request Aug 1, 2026
…#190)

Archives the morning's PLAN.md (docs/PLAN-20260801.md, per its own
versioning rule) and updates the canonical version:

- #122 (workdir consolidation + uninstaller --purge) SHIPPED, both
  hardware-validated on a real destructive purge + reinstall cycle.
- That validation found #184/#185/#186 — a readonly-var env-passing bug, a
  fake --yes flag plus two prompts that died silently on EOF, and an evsieve
  build failure caused by this Deck's kernel rejecting native-overlay
  ID-mapped mounts. All three fixed, CI-green, PRs #187/#188/#189 open,
  awaiting one batched Deck sitting to validate before merge.
- v1.2.4 milestone/estimate updated to reflect the remaining scope (#126,
  #36, plus merging+validating the three new fixes).
- Deck state refreshed: two full purge/reinstall cycles today, MC back on
  26.1.2 (matches the home-server pin, resolving the earlier 26.2 drift),
  current backup locations noted.
- New risk R11: a test-stub mutation-testing gap (a loose substring match
  let a typo'd value pass) — worth a second look wherever a stub does a
  substring case-match on a path/flag value.


Claude-Session: https://claude.ai/code/session_01Dru6wVc2ZcjiTpa6p7yJDh

Co-authored-by: Scott <scott@example.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
aradanmn added a commit that referenced this pull request Aug 1, 2026
…#191)

- v1.2.4 status: #187/#188/#189 merged AND hardware-validated tonight — a
  real `./install-minecraft-splitscreen.sh --yes < /dev/null` run completed
  end to end (no readonly-var error, no silent EOF death, evsieve built via
  the fuse-overlayfs fallback, confirmed live-running not a lucky native
  pass).
- #126a decided: CI cross-build, not build-on-Deck-and-upload. Full reasoning
  posted to the issue — short version: _evsieve_host_verify has passed on
  every build since PR1, which is direct proof stock SteamOS already ships
  libevdev.so.2, so the only real ABI risk is glibc, and SteamOS's (2.41) is
  newer than even ubuntu-24.04's (2.39). Static-linking libevdev, floated in
  the issue, isn't needed. release.yml already builds on tags; this is one
  more job in it, costing zero recurring Deck time versus a manual build
  that would need repeating on every evsieve pin/patch change.
- Deck state refreshed: fully validated working install, evsieve ON, backup
  restored, old MC backups cleaned (benchmark data deliberately kept).


Claude-Session: https://claude.ai/code/session_01Dru6wVc2ZcjiTpa6p7yJDh

Co-authored-by: Scott <scott@example.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

evsieve build box fails: overlay ID-mapped mount rejects userxattr on this kernel (fuse-overlayfs fixes it)

1 participant