Feat/efs storage - #41
Closed
supernovae wants to merge 22 commits into
Closed
Conversation
Adds enable_layer_openshift_ai with full operator install chain: NFD -> NVIDIA GPU Operator -> Red Hat OpenShift AI (DataScienceCluster). Includes S3 bucket + IAM for RHOAI data connections, comprehensive GPU instance type reference, and fix for coalesce() regression on HCP/Classic cluster outputs. Made-with: Cursor
…Serve prereqs - Switch OpenShift AI operator channel from stable to fast-3.x for v3+ - Add Service Mesh and Serverless operator installs as KServe prerequisites - Add KnativeServing CR for serverless inference - Annotate RHOAI service accounts with IRSA role ARN for S3 access - Create S3 data connection secret for RHOAI workbenches - Expand README with Day 2 setup, S3 access, and GPU verification docs - Update example tfvars with component overrides and GPU machine pool examples Made-with: Cursor
RHOAI v3+ no longer requires S3 for model serving — OCI images and PVC are the recommended storage backends. S3 is now opt-in, only needed when datasciencepipelines component is Managed. - Default openshift_ai_create_s3 to false across all environments - Update README with v3 storage table (OCI/PVC/S3 by use case) - Add KServe examples for OCI, PVC, and S3 model serving - Update air-gapped guidance to recommend OCI modelcar images - Note pipelines component requires S3 in DSC component table Made-with: Cursor
…prereqs RHOAI v3+ uses KServe RawDeployment (Headed) mode which eliminates the need for Service Mesh and Serverless operators as prerequisites. This simplifies the install from 6 operators to 3. - Upgrade DSCInitialization and DataScienceCluster templates to v2 API - Remove Service Mesh and Serverless operator install (Stage 3 removed) - Update component list: remove modelmeshserving, codeflare, datasciencepipelines; add aipipelines, nim, trainer, feastoperator, llamastackoperator, mlflowoperator, sparkoperator - Update README architecture diagram and component table - Update variable descriptions across all environments Made-with: Cursor
These components exist in ODH upstream main but are not yet in the RHOAI 3.x CRD schema. Removes them from the template, defaults, variable descriptions, and documentation. Made-with: Cursor
RHOAI 3.x on fast-3.x channel uses v1 API, not v2. The v2 API with aipipelines/nim/trainer is ODH upstream only. - Revert DSC and DSCI templates to v1 API - Remove kserve.serving subfield (original bug, removed in v3 CRD) - Restore v1 components: datasciencepipelines, modelmeshserving, codeflare - Keep feastoperator and llamastackoperator (confirmed in v3 CRD) - Revert variable descriptions and defaults to match v1 schema - Service Mesh auto-managed by RHOAI operator, not installed separately Made-with: Cursor
RHOAI v3 admission webhook rejects Managed state for deprecated components. ModelMesh and CodeFlare are deprecated in v3. Made-with: Cursor
Made-with: Cursor
Drop modelmeshserving, codeflare, kueue from template — these are deprecated in RHOAI v3 and the operator manages their defaults. Only declare components we actively configure. Made-with: Cursor
The RHOAI IAM role is now created whenever the OpenShift AI layer is enabled, independent of the S3 toggle. S3 bucket and its IAM policy are conditional on create_s3. A new ECR push/pull policy is attached when ecr_repository_arn is provided, allowing RHOAI service accounts to publish OCI model images to ECR via IRSA. Made-with: Cursor
The ECR repository ARN isn't known until apply, so it can't drive count. Use a separate create_ecr_policy boolean derived from the static var.create_ecr flag instead. Made-with: Cursor
The role ARN isn't known until apply so it can't drive for_each. Use a new openshift_ai_create_irsa boolean derived from the static enable_layer_openshift_ai flag instead. Made-with: Cursor
- Add availability_zone field to HCP machine pools with az_subnet_map resolution, enabling pools to target specific AZs for instance types with limited availability (e.g., g7e only in us-east-1b/1d) - Update GPU reference: g7e uses NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7, native FP4/FP8) not L40S - Document AZ constraints for GPU instance types in OpenShift AI README with lookup commands and targeting examples - Byron dev env: multi-AZ, m6i.4xlarge workers, Graviton monitoring pool with tolerations, full observability stack Made-with: Cursor
Update default and example versions across commercial environments and examples to the latest 4.20 patch. GovCloud remains on 4.16.x EUS. Made-with: Cursor
Add single_nat_gateway variable (default null = auto) so multi-AZ deployments can opt into a shared NAT gateway instead of 1-per-AZ, saving ~$64/month for dev/lab clusters at the cost of AZ resilience. Made-with: Cursor
- README: mention shared NAT option in cost tables and features - BYO-VPC: add NAT Gateway Topology section with cost/resilience table - Machine Pools HCP: add AZ targeting example for GPU pools, update GPU instance reference with g7e/g6e/p5, document availability_zone and az_subnet_map inputs Made-with: Cursor
openshift-logging and openshift-operators-redhat exist on fresh clusters. Switch from kubernetes_namespace_v1 (which errors on existing resources) to kubectl_manifest with server_side_apply for idempotent namespace creation. Made-with: Cursor
CLO 6.x observability.openshift.io/v1 API no longer accepts spec.collector.type — Vector is the only collector and the field was removed from the schema. Made-with: Cursor
The cert-manager-openshift-routes controller uses --enable-leader-election which requires get/create/update/patch on leases in the cert-manager namespace. Add a Role and RoleBinding granting this to the cert-manager service account. Made-with: Cursor
…_pools type The environment variables.tf machine_pools type was missing these fields, causing Terraform to silently drop them. The module-level type had them but values never reached it. Made-with: Cursor
kubernetes_secret_v1 data field already handles base64 encoding. The explicit base64encode() calls caused double encoding, resulting in Loki resolving sts.dXMtZWFzdC0x.amazonaws.com instead of sts.us-east-1.amazonaws.com. Made-with: Cursor
Add AWS EFS support as a new gitops layer, gated by enable_layer_efs_storage. Creates EFS filesystem, mount targets, NFS security group, and IRSA role for the EFS CSI driver. Installs the operator via OLM and deploys an efs-sc StorageClass with dynamic access point provisioning. Partition-aware ARNs for GovCloud parity. Works on Classic/HCP clusters. No node-level policies required (IRSA-only authentication model). Made-with: Cursor
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+37
to
+47
| resource "aws_efs_file_system" "this" { | ||
| encrypted = var.efs_encrypted | ||
| kms_key_id = var.kms_key_arn != "" ? var.kms_key_arn : null | ||
| performance_mode = var.efs_performance_mode | ||
| throughput_mode = var.efs_throughput_mode | ||
|
|
||
| tags = merge(var.tags, { | ||
| Name = "${var.cluster_name}-efs" | ||
| "rosa-gitops-layer" = "efs-storage" | ||
| }) | ||
| } |
Check failure
Code scanning / checkov
Ensure resource is encrypted by KMS using a customer managed Key (CMK) Error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the ability to create an "efs-sc" storage class with everything necessary to support EFS volumes on ROSA HCP / CLASSIC in GovCloud and Commercial.