Certs improv#52
Closed
esposem wants to merge 267 commits into
Closed
Conversation
Example kata-agent policies
The conversion is part of the code
The config field serves as default when --runtime-class flag is not provided, so a single value makes more sense than an array. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
The default attestation initContainer (from --init-container flag) appears first in the manifest. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Users can now specify custom annotations in the config file that will be automatically added to pods. Only annotations with non-empty values are applied. Default annotations include: - io.katacontainers.config.runtime.create_container_timeout - io.katacontainers.config.hypervisor.machine_type - io.katacontainers.config.hypervisor.image Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Users can now configure both the default init container image and command in the config file. The precedence order is: 1. CLI flags (--init-container-img, --init-container-cmd) 2. Config file values (init_container_image, init_container_cmd) 3. Hardcoded defaults Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Move the default config values to consts and populate it during non-interactive config file creation Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
- Create integration_test/ directory - Create testdata/ with subdirectories (configs, manifests, policies, certs, expected, tmp) - Add .gitkeep for tmp directory - Update Makefile with TEST environment variable support for regex filtering - Update clean target to remove test artifacts Usage: make test TEST=<regex> Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Config fixtures: - valid-config.toml: Complete valid configuration - invalid-no-trustee.toml: Missing required trustee_server - config-with-annotations.toml: Custom annotations - config-minimal.toml: Minimal configuration - config-with-ca.toml: With CA certificate - config-with-policy.toml: With custom policy Manifest fixtures: - simple-pod.yaml: Basic pod - pod-with-annotations.yaml: Pod with existing annotations - pod-with-initcontainers.yaml: Pod with existing initContainers - pod-with-multiple-containers.yaml: Multi-container pod - pod-with-volumes.yaml: Pod with volumes - pod-with-secrets.yaml: Pod with secret references - invalid.yaml: Invalid manifest (missing spec) - deployment.yaml: Deployment manifest Policy fixtures: - custom-policy.rego: Custom policy allowing exec - default-policy.rego: Default restrictive policy Certificate fixtures: - test-ca.crt: Test CA certificate for testing Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests: - TestConfig_Create_NonInteractiveWithDefaults: Verify default values populated - TestConfig_Create_CustomOutputPath: Create config at custom location - TestConfig_LoadAndValidate_ValidConfig: Load and validate valid config - TestConfig_LoadAndValidate_MissingTrusteeServer: Validation fails without trustee - TestConfig_NormalizeURL_AddHTTPSPrefix: URL normalization with/without protocol - TestConfig_WithAnnotations: Load config with custom annotations - TestConfig_GetConfigPath: Verify default config path Run: make test TEST=TestConfig Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests: - TestSealed_Generate_ValidKBSURI: Generate sealed secret from KBS URI - TestSealed_Generate_MultipleURIs: Generate secrets for multiple URIs - TestSealed_Format_Base64URLEncoding: Verify base64url encoding (unpadded) - TestSealed_Payload_JSONStructure: Verify JSON payload structure - TestSealed_ParseResourceURI_ValidURI: Parse KBS URIs and error handling Run: make test TEST=TestSealed Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests: - TestInitData_Generate_MinimalConfig: Generate initdata with minimal config - TestInitData_Generate_WithCACert: Include CA certificate in initdata - TestInitData_Generate_WithCustomPolicy: Use custom policy file - TestInitData_Generate_DefaultPolicy: Verify default restrictive policy - TestInitData_Encoding_GzipAndBase64: Verify gzip compression and base64 encoding - TestInitData_AAToml_Structure: Verify aa.toml structure (token_configs.kbs) - TestInitData_CDHToml_Structure: Verify cdh.toml structure (kbc configuration) Run: make test TEST=TestInitData Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests: - TestManifest_Load_ValidPod: Load and parse valid pod manifest - TestManifest_Load_InvalidYAML: Error handling for invalid YAML - TestManifest_SetRuntimeClass_SimplePod: Set runtimeClassName in spec - TestManifest_SetAnnotation_NewAnnotation: Add new annotation - TestManifest_SetAnnotation_ExistingAnnotations: Preserve existing annotations - TestManifest_AddInitContainer_NoPreviousInit: Add initContainer to pod without existing - TestManifest_AddInitContainer_PrependToExisting: Prepend initContainer to existing list - TestManifest_AddVolume_EmptyDir: Add emptyDir volume to spec - TestManifest_AddVolumeMount_AllContainers: Add volumeMount to all containers - TestManifest_SaveAndBackup: Save manifest and create backup with -coco suffix - TestManifest_GetSecretRefs: Find secret references in env and volumes - TestManifest_ReplaceSecretName: Replace secret names in manifest - TestManifest_CompleteTransformation: Full workflow with all transformations Run: make test TEST=TestManifest Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests: - TestWorkflow_BasicTransformation: Load config, transform manifest, generate initdata - TestWorkflow_WithInitContainer: Add initContainer with config defaults - TestWorkflow_WithSecretDownload: Secret download via initContainer and volume - TestWorkflow_WithCustomAnnotations: Apply custom annotations from config - TestWorkflow_CompleteWithAllFeatures: Full transformation with all features - TestWorkflow_MultipleManifests: Apply same config to multiple manifests - TestWorkflow_PreserveExisting: Verify existing content is preserved These tests demonstrate complete end-to-end workflows combining: - Config loading and validation - Manifest transformation (runtime class, annotations, volumes) - InitData generation (aa.toml, cdh.toml, policy.rego) - Sealed secret generation - InitContainer injection - Volume and volumeMount management Run: make test TEST=TestWorkflow Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Implement three new configuration parameters for Confidential Data Hub: - container_policy_uri: Maps to image_security_policy_uri in CDH config - registry_cred_uri: Maps to authenticated_registry_credentials_uri - registry_config_uri: Maps to registry_configuration_uri These parameters are included in the [image] section of cdh.toml when generating initdata, enabling proper image security policy enforcement, authenticated registry access, and registry configuration management. Updated README to document the new parameters with examples. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Currently only for darwin and linux Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Update README with install instructions from GH releases Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Added two new build targets in the release-all target: - make release GOOS=linux GOARCH=ppc64le - make release GOOS=linux GOARCH=s390x Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Rename the create-config command to init to better reflect its purpose of initializing the CoCo environment. The current focus is developer mode only. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Integrate Trustee KBS deployment into the init command for simplified infrastructure setup for a typical app developer workflow. Key features: - Interactive mode: prompts for Trustee URL or auto-deploys if empty - Non-interactive mode: auto-deploys unless --skip-trustee-deploy - Auto-detects if Trustee already running in namespace - Uses all-in-one KBS image with built-in AS and RVPS - Namespace handling: defaults to current kubectl context namespace - Updates config with deployed Trustee service URL New flags: - --skip-trustee-deploy: Skip automatic deployment - --trustee-namespace: Specify deployment namespace - --trustee-url: Provide existing Trustee URL (skip deployment) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Change default behavior to non-interactive mode. Users must now explicitly use -i or --interactive flag to enable interactive prompts for configuration values. Updated: - cmd/init.go: Replace --non-interactive flag with --interactive/-i - README.md: Update documentation to reflect new default behavior - claude.md: Update development documentation Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add support for specifying RuntimeClass via --runtime-class flag in non-interactive mode. This allows users to set a custom runtime class without interactive prompts. Usage: kubectl coco init --runtime-class kata-cc kubectl coco init --runtime-class kata-remote --trustee-url https://trustee.example.com Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add CPU resource constraints to prevent "Too many open files" error:
- Request: 1 CPU
- Limit: 2 CPUs
This fixes the error: "called `Result::unwrap()` on an `Err` value: Os {
code: 24, kind: Uncategorized, message: \"Too many open files\" }"
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add sockets = ["0.0.0.0:8080"] to [http_server] section of kbs-config.toml to allow external connectivity Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Remove the --secret flag and handleSecret() function to prepare for automatic K8s secret to sealed secret conversion. The new implementation will automatically detect and convert secrets without requiring manual flag specification. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Create pkg/secrets package to detect K8s secret references in manifests. Supports detection of: - Environment variable secrets (secretKeyRef) - Volume-mounted secrets - EnvFrom secrets (entire secret as env vars) The detector identifies: - Secret names and namespaces - Specific keys (when specified in manifest) - Usage patterns (env, volume, envFrom) - Container names and mount paths - Flags when kubectl lookup is needed This forms the foundation for automatic secret conversion to sealed secrets. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add functionality to query K8s secrets via kubectl to discover all keys in a secret when they are not explicitly specified in the manifest. This enables automatic detection of secret keys for: - Volume-mounted secrets without explicit items - EnvFrom references (entire secret as env vars) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add converter to transform K8s secret references into sealed secret format using KBS resource URIs. Conversion flow: 1. Build resource URI from namespace/secret/key 2. Generate sealed secret using sealed.GenerateSealedSecret() 3. Return SealedSecretData with URI and sealed value Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Add functionality to generate Trustee KBS configuration files and display setup instructions for sealed secrets. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Updates apply-creates-object to check for and use kata-qemu-coco-dev instead of kata-cc, matching the CoCo dev Helm chart deployment. Install with: helm install coco oci://ghcr.io/confidential-containers/charts/confidential-containers --namespace coco-system --create-namespace Uninstall: helm uninstall coco --namespace coco-system && kubectl delete namespace coco-system Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Replace the skip-if-no-Trustee guard with proper workflow scenarios: - cluster/kbs-start: runs `kubectl coco kbs start` and waits for Trustee - cluster/kbs-populate: uploads a test resource via the KBS HTTP API - cluster/apply-transforms-with-secrets: applies a pod with a K8s secret using --skip-apply, verifies the -coco.yaml has sealed secret refs - cluster/apply-creates-pod: creates a pod with kata-qemu-coco-dev (skipped unless the RuntimeClass is present) Also fix two assertion bugs: check for -sealed suffix (not 'sealed.' which never appears in the pod manifest), and embed namespace in the manifest so kubectl apply targets coco-eval not the context default. Also back up and restore ~/.kube/coco-config.toml so kbs start does not permanently alter the developer's config. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests that `kubectl coco init` creates ~/.kube/coco-config.toml and deploys Trustee to the eval namespace. The subsequent kbs-start test now exercises detection of an already-deployed Trustee rather than initial deployment. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Verifies that --init-container injects an initContainers section into the transformed manifest. Uses --skip-apply so no kata RuntimeClass is required; the test validates the transformation output only. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Verifies that apply works on a Deployment (not just Pod) by checking the Deployment object lands in the cluster with the expected name. Skipped when kata-qemu-coco-dev RuntimeClass is absent. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Exercises the --from-k8s-secret input mode of kbs populate, which is a separate code path from --path/--resource-file. Creates a real K8s secret then uploads all its keys to KBS via the HTTP API. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
After kbs-populate uploads a resource, kubectl exec into the Trustee pod to confirm the file actually exists in the KBS on-disk repository. This catches cases where populate exits 0 but the write silently failed. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Verifies --sidecar injects the coco-secure-access container into the transformed manifest. Uses --sidecar-skip-auto-sans to avoid needing a Service object; --skip-apply keeps this cluster-state independent. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests the end-to-end pipe: initdata create → dump (base64+gzip) → validate (reads encoded blob from stdin). Uses the config written by init/kbs-start so the initdata embeds the real in-cluster KBS URL. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
init now runs with --enable-sidecar so the client CA is present when apply-sidecar needs to sign the server certificate. apply-sidecar passes --sidecar-image (required when config has no image set) and an explicit --sidecar-san-dns so the cert generation succeeds without a live Service for auto-detection. backupCertDir helper added to restore ~/.kube/coco-sidecar/ state after the eval so init --enable-sidecar does not permanently alter the system. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
…ions Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests the volumes[].secret.secretName code path, which is distinct from env secretKeyRef conversion. Creates a K8s secret, applies a pod with a volume mount referencing it, then verifies: - -coco.yaml has the -sealed suffix on the volume secretName - trustee-secrets file is generated for the user to populate KBS - kbs populate -f uploads the key to KBS - kubectl exec confirms the key exists in the KBS repository Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests the 'I already have a KBS' flow: init with --skip-trustee-deploy and an explicit --trustee-url writes the config without touching the cluster. Verifies: - Command exits 0 - Config contains the provided KBS URL - Pod count in the eval namespace is unchanged (no new deployment) Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests cococtl apply on a two-document YAML (Pod + Service). Key behavioral assertion: cococtl extracts the primary workload (Pod), transforms it with cc_init_data and runtimeClassName, and saves only the Pod to -coco.yaml. The Service document is NOT applied — users apply it separately. Verifies: - -coco.yaml contains cc_init_data on the Pod - -coco.yaml does NOT contain the Service document - With kata RuntimeClass: Pod is created in the cluster, Service absent Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Runs cococtl apply twice on the same manifest and verifies both invocations exit 0. Two assertions: 1. --skip-apply (always): second run produces byte-identical -coco.yaml confirming sealed-secret token and initdata generation are stable. 2. Full cluster apply (skipped without kata): two consecutive apply calls both exit 0 and leave the pod object in the cluster (kubectl apply is idempotent; sealed-secret creation via kubectl apply is too). Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Tests `kbs start --mode external --url <url>` which registers a pre-existing KBS without any cluster interaction. Verifies: - Command exits 0 - Config contains the registered URL - Pod count in the namespace is unchanged (no deployment occurred) Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Verifies cococtl apply works on a StatefulSet with correct annotation placement. The critical invariant: cc_init_data must appear inside spec.template.metadata.annotations, NOT in the top-level StatefulSet metadata.annotations. The test splits the -coco.yaml on 'template:' and checks the annotation is absent before and present after. With kata RuntimeClass: StatefulSet object lands in the cluster and spec.template.spec.runtimeClassName is set correctly. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
…entry each generateCDHToml was putting the entire certPEM string (which may contain multiple PEM-encoded certificates from --cacert <multi-cert> or --capath <dir>) as a single extra_root_certificates array element. The validate command already enforced that each element must contain exactly one certificate. The create path now matches: it decodes the PEM block by block and appends each certificate as its own array entry. Before: extra_root_certificates = ["cert1\ncert2"] (combined, rejected by validate) After: extra_root_certificates = ["cert1", "cert2"] (one per element, accepted) Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
fix(initdata): split multi-cert PEM into one extra_root_certificates entry each
The binary name is cococtl for standalone. Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Provide two usage mode - standalone and kubectl
Supposing a pod called hello-http-server is created:
we would have
kind: Secret
metadata:
name: sidecar-tls-hello-http-server
namespace: default
type: kubernetes.io/tls
data:
tls.crt: ...
tls.key: ...
But in the pod-coco.yaml we have
- env:
- name: TLS_CERT_URI
value: kbs:///default/sidecar-tls-hello-http-server/server-cert
- name: TLS_KEY_URI
value: kbs:///default/sidecar-tls-hello-http-server/server-key
- name: CLIENT_CA_URI
value: kbs:///default/sidecar-tls/client-ca
this doesn't make sense as applying the secret and adding it into kbs will create /default/sidecar-tls-hello-http/server/tls.crt not /default/.../server-cert.
Fix it with this commit.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
…pply mode The sidecar TLS secret is uploaded to KBS, not consumed directly by the pod. Place it in the trustee namespace so 'kbs populate --from-k8s-secret -n' can find both the KBS pod and the secret in the same namespace. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Contributor
Author
|
Putting this in draft because I need to test this |
bpradipt
reviewed
Jul 5, 2026
| "name": "sidecar-tls-" + appName, | ||
| "namespace": namespace, | ||
| }, | ||
| "type": "kubernetes.io/tls", |
Contributor
There was a problem hiding this comment.
The Secret type should be changed to Opaque given the key rename.
bpradipt
reviewed
Jul 5, 2026
| } else { | ||
| // Skip-apply mode: save certs to file instead of uploading | ||
| certFilePath, err := saveSidecarCertsToYAML(manifestPath, serverCert, appName, namespace) | ||
| certFilePath, err := saveSidecarCertsToYAML(manifestPath, serverCert, appName, trusteeNamespace) |
Contributor
There was a problem hiding this comment.
This should be namespace only, which is the workload namespace.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As per issue #51
Plus fix namespace error in generated trustee secret