The system implements attested inference through a client-server architecture where the CMaaS Client communicates directly with Confidential Inference Nodes running in TEE environments. The Model Studio Gateway authenticates the client and routes requests to the appropriate nodes, while the TEE-Vendor Backed Attestation Service provides attestation verification support. Additionally, software supply chain verification mechanisms enable customers to verify the provenance of the code running in the inference nodes. This is achieved through transparency log services such as Rekor and open-source code.
The system builds trust through three complementary security mechanisms:
CMaaS establishes secure communication between the Client and Confidential Inference Node through remote attestation and end-to-end encryption. The CMaaS Client communicates with the CMaaS Proxy running inside the Confidential Inference Node, which attests the node to the client and forwards encrypted requests to the LLM Backend within the same TEE node.
Remote Attestation based Key Negotiation
The CMaaS Client initiates a key negotiation session with the CMaaS Proxy. The CMaaS Proxy responds with an Attestation Bundle containing the TDX Quote, EventLog, and transparency log data (release manifest and proof entries) as proof of its TEE identity and software provenance. The client performs remote attestation by verifying the Quote's signature (with support from the TEE-Vendor Backed Attestation Service), validating TEE measurements (MRTD, RTMR values, and TD Features), replaying the EventLog to ensure boot chain integrity, and verifying the transparency log entries to confirm the node runs a known, audited software version. Upon successful verification, both parties establish a shared ephemeral Session Key that exists only in volatile memory and is never persisted to disk.
End-to-End Encrypted Inference
The CMaaS Client encrypts inference requests using the Session Key and sends them through the Model Studio Gateway to the confidential inference node. Within the TEE, the CMaaS Proxy first decrypts the request, forwards it to the LLM Backend for processing, encrypts the inference result, and returns it to the client. All plaintext data (user prompts and model responses) exists only within the secure boundaries of the client's local environment and TEE memory. Plaintext never leaves these boundaries or appears on the network path, host system, or cloud infrastructure outside the TEE. When the session ends, the Session Key is destroyed from memory, providing forward secrecy.
CMaaS ensures the integrity and isolation of the execution environment through hardware-based TEE technology and VM image hardening.
The Confidential Inference Node runs as a confidential virtual machine within Intel TDX Trust Domain with NVIDIA Confidential Computing support:
- TDX Isolation: Provides hardware-level memory encryption, CPU state isolation, and DMA protection to ensure the TD is completely isolated from the host OS/VMM
- CPU-GPU Secure Communication: PCIe traffic is protected through SPDM-based key negotiation and encryption, preventing interception or tampering by the host
- GPU Isolation: The GPU's Compute Protected Region (CPR) isolates GPU memory from the host, with direct ingress and egress blocked by the PCIe firewall. Sensitive data and model weights remain protected within the GPU's secure memory boundaries
The confidential VM image is built with security hardening to minimize attack surface and ensure immutability. All security-critical components are cryptographically measured during boot and recorded in measurement registers:
- MRTD (Measurement Register of TD): Initial TD measurement, contains VM configuration and initial code
- RTMR[0]: Firmware measurement (UEFI, bootloader)
- RTMR[1]: Kernel measurement (Linux kernel, initrd and filesystem)
- RTMR[2]: Application measurement (cmaas-proxy, LLM backend)
These measurements are included in the TDX Quote, which is cryptographically signed by Intel's CPU and verified by the client during attestation. The EventLog provides a complete record of all measurement events, allowing clients to replay and verify the integrity of the boot chain.
In addition, the VM image hardening includes:
- RootFS Integrity: Enables dm-verity to verify the integrity of the read-only root filesystem during system startup and prevent runtime tampering
- RootFS Encryption: Enables dm-crypt for the root filesystem and data volumes, using one-time encryption keys
- Minimal Attack Surface: Strips the VM image down to essential components, removes cloud-init, SSH, getty, Alibaba Cloud Assistant, and other operational channels, and prevents dynamic image changes after deployment
CMaaS enables customers to verify that inference nodes run known, audited code versions through software supply chain verification mechanisms.
Alibaba Cloud publishes the code baseline values running in the TEE to transparency log services, ensuring that these values are publicly accessible, auditable, and traceable.
A trusted baseline can serve as a foundation for verification, but is not sufficient by itself to achieve complete security auditing. Alibaba Cloud therefore conducts rigorous code audits of all components that can access plaintext prompts and responses. These audits confirm that the system contains no unintended operational channels or backdoors and eliminate potential paths for plaintext leakage. Alibaba Cloud also makes the deployed binary artifacts available to customers so they can independently recompute artifact reference values and conduct security audits.
Trusted baseline values and audited source code still need to be linked to the binaries that actually run. Reproducible builds establish this link: identical source code and build environments produce identical binaries. Reproducible builds are already used in some scenarios, and Model Studio CMaaS will continue integrating this capability.