Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,8 @@ test-isolated target=default-target features="" :
{{ cargo-cmd }} test {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F " + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host --lib -- sandbox::uninitialized::tests::test_log_trace --exact --ignored
{{ cargo-cmd }} test {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F " + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host --lib -- sandbox::outb::tests::test_log_outb_log --exact --ignored
{{ cargo-cmd }} test {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F " + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host --test integration_test -- log_message --exact --ignored
@# CPU vendor check, gated to known CI runner hardware
{{ cargo-cmd }} test {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F " + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host --lib -- sandbox::snapshot::file::config::tests::cpu_vendor_current_is_recognized --exact --ignored
@# metrics tests
{{ cargo-cmd }} test {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F function_call_metrics," + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host --lib -- metrics::tests::test_metrics_are_emitted --exact

Expand Down
29 changes: 18 additions & 11 deletions docs/snapshot-oci-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,10 @@ Three blob kinds per tag:
pointer record selected via `index.json`. References one config and
one layer by digest.
* **config** (`application/vnd.hyperlight.snapshot.config.v1+json`). The
snapshot descriptor: arch, hypervisor, ABI version, resume address
and captured registers, memory layout, registered host functions,
snapshot generation counter. Loaded eagerly and fully parsed.
snapshot descriptor: arch, hypervisor, CPU vendor, ABI version,
resume address and captured registers, memory layout, registered
host functions, snapshot generation counter. Loaded eagerly and
fully parsed.
* **layer / memory** (`application/vnd.hyperlight.snapshot.memory.v1`).
The raw guest memory image, exactly `memory_size` bytes. mmap'd on
restore.
Expand All @@ -48,8 +49,9 @@ A single saved `Snapshot` consists of exactly:

* one entry in `index.json`, carrying the `tag` as
`org.opencontainers.image.ref.name`, plus advisory
`dev.hyperlight.snapshot.arch` and
`dev.hyperlight.snapshot.hypervisor` annotations that mirror the
`dev.hyperlight.snapshot.arch`,
`dev.hyperlight.snapshot.hypervisor`, and
`dev.hyperlight.snapshot.cpu.vendor` annotations that mirror the
config blob for tooling visibility,
* one **manifest** blob (referenced by that index entry),
* one **config** blob (referenced by the manifest's `config` field),
Expand Down Expand Up @@ -103,8 +105,8 @@ the manifest, config, or snapshot blobs against their sha256 digests.
digest returned by `save`. `Snapshot::checked_load` adds the digest
check on those three blobs, catching accidental corruption on disk.
Both run every other check (OCI structure, descriptor sizes, schema
versions, arch / hypervisor / ABI tags, layout bounds, entrypoint
bounds). The caller is responsible for trusting the source.
versions, arch / hypervisor / CPU vendor / ABI tags, layout bounds,
entrypoint bounds). The caller is responsible for trusting the source.

A reference that matches no manifest, or a tag that matches more than
one manifest in `index.json`, is rejected.
Expand All @@ -113,7 +115,12 @@ one manifest in `index.json`, is rejected.

## Portability

Snapshot images are bound to a specific CPU architecture and
hypervisor. Both are recorded in the config blob and checked at load
time, with mismatches rejected with a clear error. The hypervisor
tag (`kvm`, `mshv`, `whp`) constrains the host OS.
Snapshot images are bound to a specific CPU architecture, hypervisor,
and CPU vendor. All three are recorded in the config blob and checked
at load time, with mismatches rejected with a clear error. The
hypervisor tag (`kvm`, `mshv`, `whp`) constrains the host OS. The CPU
vendor is the x86_64 CPUID leaf-0 vendor string (e.g. `GenuineIntel`)
or the aarch64 `MIDR_EL1` implementer byte. A load on a different
vendor is rejected because the resumed CPU state can be incompatible.
A future version may relax this binding once a wider compatibility set
is proven safe.
67 changes: 67 additions & 0 deletions src/hyperlight_host/src/sandbox/snapshot/file/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,42 @@ impl Hypervisor {
}
}

#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(super) struct CpuVendor(String);

impl CpuVendor {
/// The vendor identifier of the running host.
pub(super) fn current() -> Self {
#[cfg(target_arch = "x86_64")]
{
// SAFETY: CPUID leaf 0 is always available on x86_64.
// TODO: Remove the `unsafe`/allow when MSRV is raised above
// 1.89. On Rust 1.89 `__cpuid` requires `unsafe`; on newer
// compilers it is safe and clippy flags it as unnecessary.
#[allow(unused_unsafe)]
let r = unsafe { core::arch::x86_64::__cpuid(0) };
let mut bytes = [0u8; 12];
bytes[0..4].copy_from_slice(&r.ebx.to_le_bytes());
bytes[4..8].copy_from_slice(&r.edx.to_le_bytes());
bytes[8..12].copy_from_slice(&r.ecx.to_le_bytes());
Self(String::from_utf8_lossy(&bytes).into_owned())
}
#[cfg(all(target_arch = "aarch64", target_os = "linux"))]
{
let midr: u64;
// SAFETY: Linux emulates MIDR_EL1 reads from EL0.
unsafe { core::arch::asm!("mrs {}, MIDR_EL1", out(reg) midr) };
let implementer = (midr >> 24) & 0xff;
// `0x` prefix padded to width 4, e.g. Apple `0x61`, Arm `0x41`.
Self(format!("{implementer:#04x}"))
}
}

pub(super) fn as_str(&self) -> &str {
&self.0
}
}

// --- Config JSON shape ----------------------------------------------

/// Top-level Hyperlight snapshot config JSON. Lives at
Expand All @@ -123,6 +159,8 @@ pub(super) struct OciSnapshotConfig {
/// Memory blob ABI version. See `SNAPSHOT_ABI_VERSION`.
pub(super) abi_version: u32,
pub(super) hypervisor: Hypervisor,
/// CPU vendor captured at snapshot time. Checked on load.
pub(super) cpu_vendor: CpuVendor,
Comment thread
ludfjig marked this conversation as resolved.
/// Top of the guest stack, in guest virtual address space.
pub(super) stack_top_gva: u64,
/// Guest virtual address the loader resumes the paused call at.
Expand Down Expand Up @@ -331,6 +369,16 @@ impl OciSnapshotConfig {
self.hyperlight_version
));
}
let current_vendor = CpuVendor::current();
if self.cpu_vendor != current_vendor {
return Err(crate::new_error!(
"snapshot CPU vendor mismatch: file was created on {} but the current CPU is {} \
(snapshot produced by hyperlight {})",
self.cpu_vendor.as_str(),
current_vendor.as_str(),
self.hyperlight_version
));
}
// Bound memory size early so the subsequent file-size check
// does not have to deal with absurd values.
if self.memory_size == 0 || self.memory_size > SandboxMemoryLayout::MAX_MEMORY_SIZE as u64 {
Expand Down Expand Up @@ -574,4 +622,23 @@ mod tests {
assert_eq!(back, r, "return type {:?} did not round-trip", r);
}
}

/// `CpuVendor::current` returns the expected host vendor. Ignored
/// by default and run explicitly in CI, where the runner hardware
/// is known. Extend the allowlist when new runner hardware is
/// added.
#[test]
#[ignore = "hardware-specific; run explicitly in CI"]
fn cpu_vendor_current_is_recognized() {
let vendor = CpuVendor::current();
let v = vendor.as_str();
#[cfg(target_arch = "x86_64")]
assert!(
matches!(v, "GenuineIntel" | "AuthenticAMD"),
"unrecognized x86_64 CPU vendor: {v:?}"
);
#[cfg(all(target_arch = "aarch64", target_os = "linux"))]
// MIDR_EL1 implementer byte for Apple silicon.
assert_eq!(v, "0x61", "unexpected aarch64 CPU implementer");
}
}
12 changes: 7 additions & 5 deletions src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,12 @@ pub(super) const SNAPSHOT_ABI_VERSION: u32 = 1;
/// the Image Layout spec.
pub(super) const ANNOTATION_REF_NAME: &str = "org.opencontainers.image.ref.name";

/// Advisory annotation keys recording the guest arch and hypervisor
/// backend on the manifest descriptor in `index.json`. These mirror
/// the authoritative `arch` and `hypervisor` fields in the config
/// blob so registry UIs and tools like `oras` and `skopeo` can show
/// them. The loader validates against the config blob.
/// Advisory annotation keys recording the guest arch, hypervisor
/// backend, and CPU vendor on the manifest descriptor in
/// `index.json`. These mirror the authoritative `arch`, `hypervisor`,
/// and `cpu_vendor` fields in the config blob so registry UIs and
/// tools like `oras` and `skopeo` can show them. The loader validates
/// against the config blob.
pub(super) const ANNOTATION_ARCH: &str = "dev.hyperlight.snapshot.arch";
pub(super) const ANNOTATION_HYPERVISOR: &str = "dev.hyperlight.snapshot.hypervisor";
pub(super) const ANNOTATION_CPU: &str = "dev.hyperlight.snapshot.cpu.vendor";
31 changes: 20 additions & 11 deletions src/hyperlight_host/src/sandbox/snapshot/file/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ use oci_spec::image::{
ImageManifestBuilder, MediaType, SCHEMA_VERSION,
};

use self::config::{Arch, HostFunction, Hypervisor, MemoryLayout, OciSnapshotConfig};
use self::config::{Arch, CpuVendor, HostFunction, Hypervisor, MemoryLayout, OciSnapshotConfig};
use self::digest::{Digest256, oci_digest, parse_oci_digest, verify_blob_bytes, verify_blob_file};
use self::fsutil::{put_blob, put_blob_if_absent, read_bounded, replace_file_atomic};
use self::media_types::{
ANNOTATION_ARCH, ANNOTATION_HYPERVISOR, ANNOTATION_REF_NAME, MT_CONFIG_CURRENT, MT_CONFIG_V1,
MT_SNAPSHOT_CURRENT, MT_SNAPSHOT_V1, SNAPSHOT_ABI_VERSION,
ANNOTATION_ARCH, ANNOTATION_CPU, ANNOTATION_HYPERVISOR, ANNOTATION_REF_NAME, MT_CONFIG_CURRENT,
MT_CONFIG_V1, MT_SNAPSHOT_CURRENT, MT_SNAPSHOT_V1, SNAPSHOT_ABI_VERSION,
};
use self::reference::{OciDigest, OciReference, OciTag};
use super::{NextAction, Snapshot};
Expand Down Expand Up @@ -282,10 +282,12 @@ impl Snapshot {
///
/// # Portability
///
/// Snapshot images are bound to the specific CPU architecture and
/// hypervisor that the snapshot was created on. For example, a
/// snapshot taken on x86_64 with KVM can only be loaded on an
/// x86_64 host running KVM. Loading on any other host is rejected.
/// Snapshot images are bound to the specific CPU architecture,
/// hypervisor, and CPU vendor that the snapshot was created on.
/// For example, a snapshot taken on an Intel x86_64 host with KVM
/// can only be loaded on an Intel x86_64 host running KVM. Loading
/// on any other host is rejected. A future version may relax this
/// binding once a wider compatibility set is proven safe.
///
/// # Compatibility
///
Expand Down Expand Up @@ -523,6 +525,10 @@ impl Snapshot {
ANNOTATION_HYPERVISOR.to_string(),
cfg.hypervisor.as_str().to_string(),
);
anns.insert(
ANNOTATION_CPU.to_string(),
cfg.cpu_vendor.as_str().to_string(),
);
DescriptorBuilder::default()
.media_type(MediaType::ImageManifest)
.digest(oci_digest(&manifest_digest)?)
Expand Down Expand Up @@ -565,6 +571,7 @@ impl Snapshot {
abi_version: SNAPSHOT_ABI_VERSION,
hypervisor: Hypervisor::current()
.ok_or_else(|| crate::new_error!("no hypervisor available to tag snapshot"))?,
cpu_vendor: CpuVendor::current(),
stack_top_gva: self.stack_top_gva,
entrypoint_addr,
sregs: *sregs,
Expand Down Expand Up @@ -602,10 +609,12 @@ impl Snapshot {
///
/// # Portability
///
/// Snapshot images are bound to the specific CPU architecture and
/// hypervisor that the snapshot was created on. For example, a
/// snapshot taken on x86_64 with KVM can only be loaded on an
/// x86_64 host running KVM. Loading on any other host is rejected.
/// Snapshot images are bound to the specific CPU architecture,
/// hypervisor, and CPU vendor that the snapshot was created on.
/// For example, a snapshot taken on an Intel x86_64 host with KVM
/// can only be loaded on an Intel x86_64 host running KVM. Loading
/// on any other host is rejected. A future version may relax this
/// binding once a wider compatibility set is proven safe.
///
/// # Compatibility
///
Expand Down
25 changes: 25 additions & 0 deletions src/hyperlight_host/src/sandbox/snapshot/file_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -502,6 +502,31 @@ fn cfg_current_hypervisor() -> &'static str {
}
}

#[test]
fn cpu_vendor_mismatch_rejected() {
let snapshot = create_snapshot();
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("snap");
snapshot
.save(&path, &OciTag::new("latest").unwrap())
.unwrap();

rewrite_config(&path, |cfg| {
cfg["cpu_vendor"] = Value::from("not-this-cpu-vendor");
});

let err = unwrap_err_snapshot(Snapshot::checked_load(
&path,
OciTag::new("latest").unwrap(),
));
let msg = format!("{}", err);
assert!(
msg.contains("vendor"),
"expected CPU vendor mismatch, got: {}",
msg
);
}

// A call snapshot must carry sregs. serde rejects a config that
// omits the field.

Expand Down
Loading