No separation of concerns: `ROOTFS_INIT_SCRIPT` hardcodes application-level env vars + various tmpfs mounts (=> rootfs is partly kept in memory)

[codethief]

Problem statement

By default, Gondolin configures various tmpfs mounts and environment variables, see host/src/alpine/init-scripts.ts:

mount -t tmpfs tmpfs /tmp || log "[init] mount tmpfs /tmp failed"
mount -t tmpfs tmpfs /root || log "[init] mount tmpfs /root failed"
# […]
mount -t tmpfs tmpfs /var/tmp || log "[init] mount tmpfs /var/tmp failed"
mount -t tmpfs tmpfs /var/cache || log "[init] mount tmpfs /var/cache failed"
mount -t tmpfs tmpfs /var/log || log "[init] mount tmpfs /var/log failed"

mkdir -p /tmp/.cache /tmp/.config /tmp/.local/share

export HOME=/root
export TMPDIR=/tmp
export XDG_CACHE_HOME=/tmp/.cache
export XDG_CONFIG_HOME=/tmp/.config
export XDG_DATA_HOME=/tmp/.local/share
export UV_CACHE_DIR=/tmp/.cache/uv

These are getting in my way for various reasons (see below), so I would rather configure them myself. Unfortunately, the above code is part of the default ROOTFS_INIT_SCRIPT. My only remedy, therefore, would be to build a custom image and provide the entire init script myself via the init.rootfsInit build option. That'd be throwing out the baby with the bathwater, though, and I'd likely just be duplicating the remaining > 300 lines of the init script. Alternatively, I could provide init.rootfsInitExtra and undo the tmpfs mounts and env var exports one by one. Either option sounds rather dirty & brittle.

Suggested solution

• Remove the tmpfs mounts from the init script. Let the user configure whether they want a (partly or fully) memory- vs. disk-backed rootfs via VMOptions.vfs.mounts and VMOptions.rootfs.mode.
• Remove the env vars from the init script. Let the user configure those via VMOptions.env.

Background

• I am running my VMs with 1G of RAM, which should be plenty for what my agents do. Unfortunately, though, that's not enough if a big part of my rootfs (caches) live in memory: The kernel's default tmpfs mount size is ½ of the available RAM and, even if I resized it (e.g. sudo mount -o remount,size=… /tmp), all tmpfs mounts together still couldn't hold more than the available RAM. (Unless I configure swapping but that, too, would just be a workaround.) And, indeed, I ran into this issue the second my agent tried to download & extract various packages & tarballs. The thing is: The VM already uses an ephemeral disk-backed COW overlay for the rootfs, so, personally, I see no point in keeping data-heavy directories (e.g. UV_CACHE_DIR) in memory. RAM is precious!

• I want to run my workloads as a non-root user and configure $HOME myself.

• Similarly for $XDG_… – I want to mount various dotfiles and directories into the guest user's home dir and I need applications to find them.