Skip to content

fix(installer): support containerd config version 4 - #1

Merged
mismosmi merged 1 commit into
mainfrom
containerd-config-v4
Aug 3, 2026
Merged

mismosmi merged 1 commit into
mainfrom
containerd-config-v4

Conversation

@mismosmi

@mismosmi mismosmi commented Aug 3, 2026

Copy link
Copy Markdown
Member

Why

Our Scaleway Kapsule preview nodes were re-imaged onto containerd 2.3.1, which writes a version 4 /etc/containerd/config.toml. zeropod's installer is built against containerd/v2 v2.1.x, whose config loader caps at version 3, so ds/zeropod-node went into Init:CrashLoopBackOff:

configuring containerd v2.3.1
error installing runtime: unable to configure containerd: loading containerd config:
  failed to load TOML from /etc/containerd/config.toml:
  expected containerd config version equal to or less than `3`, got `4`

The shim and CRIU binaries are already installed by that point — only the runtime never gets registered — so every pod using the zeropod RuntimeClass is stuck in FailedCreatePodSandBox: no runtime for "zeropod" is configured. That took down all of our preview environments.

What

  1. Stop routing the host config through containerd's own loader. The installer only needs three things out of it: the config version, the imports, and the opt plugin path. Those are now parsed directly with pelletier/go-toml/v2, which the installer already depends on — so no dependency bump, no change to the shim or manager, and the installer stays forward compatible with config versions that don't exist yet instead of breaking again on the next one.
  2. Write the version 3 runtime drop-in for any config version >= 3, not only for exactly 3. The CRI plugin was renamed to io.containerd.cri.v1.runtime in config version 3 and has kept that name since, and a drop-in may declare a lower version than the root config (containerd migrates it forward). Without this, a version 4 host got the pre-v3 io.containerd.grpc.v1.cri drop-in, which registers the runtime into a section the CRI plugin no longer reads — a silent failure rather than a loud one.

Testing

  • go test ./cmd/installer/ — all TestConfigureContainerd subtests pass, including a new v4 config case.
  • Ran the patched installer against the actual config.toml from the affected node and had a real containerd 2.3.1 binary load the result.

Worth sending upstream to ctrox/zeropod as well — there is no issue or PR filed for this yet, and upstream main still pins containerd/v2 v2.1.6.

The installer loaded the host's containerd configuration through
containerd's own config loader, which refuses any config whose version is
higher than the containerd release the installer was built against. With
containerd/v2 v2.1 that cap is version 3, so on a host running containerd
v2.3 — which writes version 4 — the install fails outright:

  error installing runtime: unable to configure containerd: loading
  containerd config: failed to load TOML from /etc/containerd/config.toml:
  expected containerd config version equal to or less than `3`, got `4`

The shim binaries are already installed at that point, only the runtime
never gets registered, so every pod using the zeropod RuntimeClass is
stuck in FailedCreatePodSandBox with `no runtime for "zeropod" is
configured`.

The installer only needs three things out of that config: the version,
the imports and the opt plugin path. Parse those directly with the TOML
library the installer already depends on instead of going through the
containerd loader, so the installer keeps working against config versions
that did not exist when it was built.

Also write the version 3 runtime drop-in for any config version >= 3
rather than only for exactly 3. The CRI plugin was renamed to
io.containerd.cri.v1.runtime in config version 3 and kept that name
since, and a drop-in may declare a lower version than the root config, so
the version 3 form stays correct. Without this, a version 4 host got the
pre-v3 io.containerd.grpc.v1.cri drop-in, which registers the runtime
into a section the CRI plugin no longer reads.
@mismosmi
mismosmi force-pushed the containerd-config-v4 branch from aa6463c to 7df2979 Compare August 3, 2026 11:47
@mismosmi
mismosmi merged commit d036666 into main Aug 3, 2026
7 of 8 checks passed
@mismosmi
mismosmi deleted the containerd-config-v4 branch August 3, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant