Support suspending and resuming Linux VMs - #1297
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d2a3fcca1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
| } | ||
|
|
||
| private func regenerateMachineIdentifier() throws { |
There was a problem hiding this comment.
When a Linux .tvm archive is imported more than once, or imported on a host that still has the exported VM, Import.swift only fixes MAC collisions; the new persisted VZGenericMachineIdentifier is copied unchanged from config.json. Since this helper is private and only the clone path can call it, those imported local Linux VMs remain suspendable with the same supposedly unique platform identifier, defeating the uniqueness invariant this change adds for save/restore. Please make this available to import and regenerate it when a local Linux identifier collision exists.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch. I updated Import to regenerate the VZGenericMachineIdentifier when it collides with an existing local Linux VM. Regeneration also removes state.vzvmsave if a new identifier is minted, since that saved state is tied to the old identifier and cannot be restored using the new one.
I added an import test using distinct MAC addresses to isolate the machine-identifier collision. It verifies that the identifier changes, the MAC address remains unchanged, and the incompatible saved state is
removed.
By the way, I think the same collision risk may exist for macOS VMs. Darwin's ecid is a VZMacMachineIdentifier, and the clone and import paths appear to preserve it. I kept that preexisting behavior out of this PR, but it may be worth unifying the identity handling in a follow-up.
There was a problem hiding this comment.
I took a step back to think about why this would be any different for the Linux vs Darwin platforms, and ended up simplifying the whole PR to make the new behavior more consistent with how Darwin currently handles its ecid. Existing identifiers are now preserved just like Darwin, and the only Linux-specific clone behavior is initializing the missing identifier for a stopped legacy config.
I agree that Import should probably do the machineIdentity/ecid collision check, but it should be done consistently for Darwin and Linux and that felt out of scope for this PR.
0d2a3fc to
769f6a2
Compare
|
I'm very excited about getting this merged in. My org has a suite of playwright tests that run against a containerized stack of services running on a Linux host. Bringing that stack up and down to reset state for each test is expensive at around 30s per respin. Some initial experiments with code behind this new PR show that we can take a snapshot of a VM running our system, and then resume it in ~2s before each test. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 769f6a21be
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if populateCache { | ||
| let expectedPushedVMDir = try ociStorage.create(pushedRemoteName) | ||
| try localVMDir.clone(to: expectedPushedVMDir, generateMAC: false) | ||
| try localVMDir.clone(to: expectedPushedVMDir, generateMAC: false, generateMachineIdentifier: false) |
There was a problem hiding this comment.
When --populate-cache is used after pushing a suspended Linux VM, this cache clone still copies state.vzvmsave, even though pushToRegistry only uploads config, disk, and NVRAM. A later tart clone <that ref> can then open the populated OCI cache, see the cached VM as .Suspended, preserve the same Linux machine identifier, and produce a local clone that restores a snapshot that was never part of the pushed image; strip the saved state from the pushed-image cache (or clone without it) before linking the ref.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, but that's really an issue that pre-dates this PR and also applies to Darwin. I removed all of my changes to Push and Clone (other than the new machineIdentifier generation for legacy configs) so this PR doesn't change that behavior at all.
Similar to the other comment, this is probably a good idea to fix consistently for both platforms but I think it's out of scope for the current PR.
Persist a VZGenericMachineIdentifier in Linux VM configurations and reuse it when building VZGenericPlatformConfiguration. Saved machine state is tied to this identifier, so generating a new one prevents restoration. Legacy configurations remain loadable but cannot be suspended. When cloning a stopped legacy Linux VM, initialize the missing identifier in the clone. Existing identifiers and suspended state remain unchanged. Linux now conforms to PlatformSuspendable and retains USB keyboard support in suspendable mode, but omits VZUSBScreenCoordinatePointingDeviceConfiguration because it passes save/restore validation but causes restoration to fail.
769f6a2 to
01d9359
Compare
1 participant
Fixes #1177
Summary
Previous investigation in #796 and #1177 implied that restoring suspended Linux VMs wasn't supported by Virtualization.framework. Some testing on current macOS shows that Linux restoration does actually work properly, and that Tart's existing suspend/restore mechanisms were very close to working.
The
Darwinplatform implementation persists the VM's ECID and assigns it toVZMacPlatformConfiguration.machineIdentifier. However, theLinuxplatform didn't specify amachineIdentifierwhen constructing itsVZGenericPlatformConfiguration, which caused a new identifier to be generated each time the platform was instantiated.Unfortunately, Virtualization.framework returns a very opaque error if saved machine state is restored with a mismatched identifier:
This PR generates and persists a
VZGenericMachineIdentifierin new Linux VM config files, then reuses it whenever the platform is constructed. The goal is to get the Linux behavior as close as possible to the existing Darwin behavior (with the exception of handling legacy config files with no machineIdentifier). Future work could probably align these more closely, but I'd like to keep this PR as minimal as I can.Now, existing Linux configs without a
machineIdentifierare loadable for normal use but can't be suspended. When cloning a stopped legacy Linux VM, the missingmachineIdentifieris initialized in the new clone. Existing identifiers are preserved, matching Tart's current behavior for Darwin ECIDs, and suspended clones keep their original identity and saved state.The
Linuxplatform now conforms to PlatformSuspendable. Suspendable Linux VMs retain USB keyboard support but omit VZUSBScreenCoordinatePointingDeviceConfiguration, which passes validateSaveRestoreSupport() but causes restoration to fail.Testing
The matching boot ID and PID with a continuously incrementing counter confirm that saved machine state was restored rather than cold-booted.