Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,10 @@ talos_backup_*/
*.backup
*.tfstate
*.tfstate.*

# Terraform local state (bootstrap)
backend/
*.tfstate
*.tfstate.backup
.terraform/
.terraform.lock.hcl
5 changes: 5 additions & 0 deletions core/vpc/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
> [!NOTE]
> Experimental VPC module.
> Not used by any environment in infra v1.
> Reserved for future refactors (infra v2).

# vpc

<!-- BEGIN_TF_DOCS -->
Expand Down
78 changes: 63 additions & 15 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,48 @@ A complete overview of the **ShopStack** project architecture and repository rel
## 📦 **Repositories Overview**

| Repository | Purpose | Main Technologies |
|-------------|----------|-------------------|
| **infra-terraform** | Defines and provisions the foundational infrastructure (S3 backend, DynamoDB for locks, Terraform remote state). | Terraform, AWS |
| **cluster-gitops** | Manages cluster provisioning, configuration, and repeatability testing using Kind and GitOps workflows. | Kind, Kubectl, Makefile |
| **apps-sre** | Hosts applications and observability stacks that run inside the Kubernetes cluster. | Docker, Helm, ArgoCD *(future)* |
|-----------|---------|-------------------|
| **infra-terraform** | Defines and provisions the foundational AWS infrastructure (Terraform remote state backend, IAM/OIDC integration, ECR, Secrets Manager, and build hooks). | Terraform, AWS (S3, DynamoDB, IAM, ECR, Secrets Manager) |
| **cluster-gitops** | Manages cluster provisioning, configuration, and repeatability testing using Kind and GitOps workflows. | Kind, kubectl, Makefile |
| **apps-sre** | Hosts applications and observability stacks that run inside the Kubernetes cluster. | Docker, Helm, Argo CD *(future)* |

---

## ☁️ **AWS Integration Scope (infra v1)**

This section defines the AWS integration scope for **infra v1** of ShopStack.

- **Terraform remote state backend**
- `core/backend` module creates:
- An S3 bucket to store Terraform remote state.
- A DynamoDB table to provide state locking.
- Used by `envs/dev` to safely manage infrastructure changes.

- **Secrets Manager**
- AWS Secrets Manager will store runtime secrets for ShopStack (API keys, database credentials, etc.).
- `core/secrets` module (infra-terraform) manages the secret metadata and naming convention (e.g. `shopstack/dev/app-config`).
- Secret values can be injected later into workloads (Kubernetes manifests or app configs) in future sprints.

- **ECR (Elastic Container Registry)**
- `core/ecr` module provisions an ECR repository per environment (e.g. `shopstack-dev`).
- Used to store container images built for ShopStack applications.
- Images are pushed from CI (GitHub Actions / CodeBuild) and later pulled by the runtime environment (Kind/Talos/EKS).

- **CodeBuild (build integration)**
- A CodeBuild project will be used as a managed build step to:
- Build Docker images from application repositories.
- Push images to the ECR repository created by `core/ecr`.
- CodeBuild will be triggered via GitHub Actions or future CI workflows using IAM roles instead of long-lived credentials.

- **GitHub OIDC (federated IAM access)**
- `core/github-oidc` module defines:
- An IAM OIDC provider for `token.actions.githubusercontent.com`.
- An IAM role that can be assumed only by specific GitHub repositories/branches (e.g. `repo:<org>/<repo>:ref:refs/heads/main`).
- This role will grant least-privilege access to:
- Read/write the Terraform remote state in S3/DynamoDB.
- Push images to ECR.
- Interact with CodeBuild when needed.
- This removes the need for static AWS keys in GitHub and aligns with AWS recommended security practices.

---

Expand All @@ -21,7 +59,10 @@ Developer
├── infra-terraform
│ ├── S3 bucket (Terraform backend)
│ └── DynamoDB table (state lock)
│ ├── DynamoDB table (Terraform state lock)
│ ├── AWS Secrets Manager (app secrets skeleton)
│ ├── ECR repository (container images per env)
│ └── IAM + OIDC role (GitHub Actions / CI access)
├── cluster-gitops
│ ├── Kind cluster (shopstack)
Expand Down Expand Up @@ -76,29 +117,36 @@ Results are saved to repeat.log.
```plaintext
chmod 600 ~/.kube/config
```
Docker daemon restricted to local use.
Terraform state secured in private AWS S3 bucket with DynamoDB locking.
- Terraform state stored in a private AWS S3 bucket with DynamoDB locking.
- GitHub CI uses federated IAM access via OIDC instead of static AWS credentials (infra v1 scope).

---

## 🧩 **Future Improvements**

- Add CI/CD pipeline for infrastructure changes.
- Integrate EKS managed cluster for cloud testing.
- Enable security scanning tools (Trivy, kube-bench).
- Automate bootstrap across all repositories with a unified Makefile.
- Add CI/CD pipeline for infrastructurechanges using GitHub Actions + OIDC.
- Integrate EKS managed cluster for cloud testing.
- Enable additional security scanning tools (Trivy, kube-bench, tfsec/Trivy for Terraform).
- Automate bootstrap across all repositories with a unified Makefile and shared scripts.
---

## 🗂️ **Folder Structure**
```bash
infra-terraform/
├── core/
│ ├── backend/ # S3 + DynamoDB for Terraform remote state
│ ├── vpc/ # Experimental VPC module (not used in infra v1)
│ ├── github-oidc/ # IAM OIDC provider + role for GitHub Actions (planned)
│ ├── ecr/ # ECR repository definitions (planned)
│ └── secrets/ # Secrets Manager layout (planned)
├── envs/
│ └── dev/
│ ├── backend.tf
│ ├── main.tf
│ ├── backend.tf # Backend configuration using S3/DynamoDB
│ ├── main.tf # Environment stack (modules wiring)
│ ├── outputs.tf
│ └── versions.tf
├── .github/
│ └── workflows/terraform-ci.yml
├── talos/ # Talos cluster configuration files (local cluster)
── backend/ # Local bootstrap state (not used in normal flows)
├── Makefile
└── docs/
└── ARCHITECTURE.md
Expand Down
9 changes: 0 additions & 9 deletions envs/dev/!!

This file was deleted.

18 changes: 0 additions & 18 deletions envs/dev/1!

This file was deleted.

24 changes: 14 additions & 10 deletions vpc/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
> [!NOTE]
> Active VPC module for ShopStack infra v1.
> Used by `envs/dev/main.tf` via `source = "../../vpc"`.

# vpc

<!-- BEGIN_TF_DOCS -->
Expand Down Expand Up @@ -26,19 +30,19 @@ No resources.

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_azs"></a> [azs](#input\_azs) | n/a | `list(string)` | n/a | yes |
| <a name="input_env"></a> [env](#input\_env) | n/a | `string` | n/a | yes |
| <a name="input_private_subnet_cidrs"></a> [private\_subnet\_cidrs](#input\_private\_subnet\_cidrs) | n/a | `list(string)` | n/a | yes |
| <a name="input_project"></a> [project](#input\_project) | n/a | `string` | n/a | yes |
| <a name="input_public_subnet_cidrs"></a> [public\_subnet\_cidrs](#input\_public\_subnet\_cidrs) | n/a | `list(string)` | n/a | yes |
| <a name="input_region"></a> [region](#input\_region) | n/a | `string` | n/a | yes |
| <a name="input_vpc_cidr"></a> [vpc\_cidr](#input\_vpc\_cidr) | n/a | `string` | n/a | yes |
| <a name="input_azs"></a> [azs](#input\_azs) | List of Availability Zones where subnets will be created. | `list(string)` | n/a | yes |
| <a name="input_env"></a> [env](#input\_env) | Deployment environment (e.g., dev, prod). | `string` | n/a | yes |
| <a name="input_private_subnet_cidrs"></a> [private\_subnet\_cidrs](#input\_private\_subnet\_cidrs) | List of CIDR blocks for the private subnets. | `list(string)` | n/a | yes |
| <a name="input_project"></a> [project](#input\_project) | Project name used for resource naming and tagging. | `string` | n/a | yes |
| <a name="input_public_subnet_cidrs"></a> [public\_subnet\_cidrs](#input\_public\_subnet\_cidrs) | List of CIDR blocks for the public subnets. | `list(string)` | n/a | yes |
| <a name="input_region"></a> [region](#input\_region) | AWS region where the VPC resources are created. | `string` | n/a | yes |
| <a name="input_vpc_cidr"></a> [vpc\_cidr](#input\_vpc\_cidr) | Primary CIDR block for the VPC. | `string` | n/a | yes |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_private_subnets"></a> [private\_subnets](#output\_private\_subnets) | n/a |
| <a name="output_public_subnets"></a> [public\_subnets](#output\_public\_subnets) | n/a |
| <a name="output_vpc_id"></a> [vpc\_id](#output\_vpc\_id) | n/a |
| <a name="output_private_subnets"></a> [private\_subnets](#output\_private\_subnets) | List of private subnet IDs created in the VPC. |
| <a name="output_public_subnets"></a> [public\_subnets](#output\_public\_subnets) | List of public subnet IDs created in the VPC. |
| <a name="output_vpc_id"></a> [vpc\_id](#output\_vpc\_id) | ID of the VPC created by this module. |
<!-- END_TF_DOCS -->
3 changes: 3 additions & 0 deletions vpc/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ provider "aws" {
region = var.region
}


# Base VPC module for the ShopStack environment.
# Uses terraform-aws-modules/vpc. NAT Gateway is disabled to reduce cost.
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "5.8.1"
Expand Down
10 changes: 7 additions & 3 deletions vpc/outputs.tf
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
output "vpc_id" {
value = module.vpc.vpc_id
description = "ID of the VPC created by this module."
value = module.vpc.vpc_id
}

output "public_subnets" {
value = module.vpc.public_subnets
description = "List of public subnet IDs created in the VPC."
value = module.vpc.public_subnets
}

output "private_subnets" {
value = module.vpc.private_subnets
description = "List of private subnet IDs created in the VPC."
value = module.vpc.private_subnets
}

21 changes: 14 additions & 7 deletions vpc/variables.tf
Original file line number Diff line number Diff line change
@@ -1,27 +1,34 @@
variable "project" {
type = string
type = string
description = "Project name used for resource naming and tagging."
}

variable "env" {
type = string
type = string
description = "Deployment environment (e.g., dev, prod)."
}

variable "region" {
type = string
type = string
description = "AWS region where the VPC resources are created."
}

variable "vpc_cidr" {
type = string
type = string
description = "Primary CIDR block for the VPC."
}

variable "azs" {
type = list(string)
type = list(string)
description = "List of Availability Zones where subnets will be created."
}

variable "public_subnet_cidrs" {
type = list(string)
type = list(string)
description = "List of CIDR blocks for the public subnets."
}

variable "private_subnet_cidrs" {
type = list(string)
type = list(string)
description = "List of CIDR blocks for the private subnets."
}