Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
360fb9a
refactor(core): restructure vpc and backend modules
juandiegocv27 Nov 2, 2025
1fa6030
Merge pull request #7 from juandiegocv27/feature/eks-config
juandiegocv27 Nov 2, 2025
55bc591
feat(talos): bootstrap of Talos Linux control plane and secure .gitig…
juandiegocv27 Nov 8, 2025
4807710
chore(talos): add exported machine configs and etcd snapshot
juandiegocv27 Nov 15, 2025
809b7da
refactor: remove EKS and update Terraform configs for Talos
juandiegocv27 Nov 17, 2025
7934f68
cleanup: remove unused EC2/EKS references and fix outputs
juandiegocv27 Nov 17, 2025
a59b5a5
Merge remote-tracking branch 'origin/dev' into talos-linux-implementa…
juandiegocv27 Nov 17, 2025
a0dd2f2
chore: restore vpc module and fix dev config
juandiegocv27 Nov 17, 2025
7191703
Merge pull request #8 from juandiegocv27/talos-linux-implementation
juandiegocv27 Nov 17, 2025
b13128b
feat: add providers.tf and finalize S3+DynamoDB backend setup
juandiegocv27 Nov 19, 2025
1b05f79
chore: add terraform pre-commit hooks (fmt, validate, tflint, trivy, …
juandiegocv27 Dec 2, 2025
cc8fc72
Merge pull request #9 from juandiegocv27/feat/validate-backend-and-ce…
juandiegocv27 Dec 2, 2025
a6d35ed
chore: finalize infra v1 with VPC module docs and pre-commit
juandiegocv27 Dec 2, 2025
d7b224d
docs: define AWS integration scope for infra v1
juandiegocv27 Dec 2, 2025
0bc56bf
Merge pull request #10 from juandiegocv27/feat/review-infra-v1
juandiegocv27 Dec 2, 2025
41531a6
Add high-level architecture overview for ShopStack infra
juandiegocv27 Dec 3, 2025
947b9e9
Merge pull request #11 from juandiegocv27/architecture-design
juandiegocv27 Dec 3, 2025
803ded4
Add high-level architecture overview for ShopStack infra pt2
juandiegocv27 Dec 3, 2025
9a887f2
Merge pull request #12 from juandiegocv27/architecture-design
juandiegocv27 Dec 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,17 +1,52 @@
# Talos / Kubernetes credentials
talos/talosconfig
talos/kubeconfig
talos/controlplane.yaml
talos/worker.yaml
talos/*.pem
talos/*.key
talos/*.crt
talos/*.csr
talos/support-*.zip
talos_backup_*/talos_backup_*/talos_backup_*/

# Local Kubectl
.kube/
*.log


# Terraform
.terraform/
*.tfstate*
.crash
*.lock.hcl
*.tfstate.backup
.terraform.lock.hcl

crash.log
*.tfvars
*.tfvars.json

# AWS credentials
*.pem
*.key

# Editor files
# file editors
.DS_Store
.idea/
.vscode/
*.swp


kubeconfig*
cilium-quick-install.yaml
talos_backup_*/
*.backup
*.tfstate
*.tfstate.*

# Terraform local state (bootstrap)
backend/
*.tfstate
*.tfstate.backup
.terraform/
.terraform.lock.hcl
24 changes: 24 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
default_stages: [pre-commit]

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.96.3
hooks:
- id: terraform_fmt
args:
- --args=-recursive

- id: terraform_validate

- id: terraform_tflint

- id: terraform_trivy
args:
- --hook-config=--parallelism-limit=1
- "--args=--severity=CRITICAL"
- "--args=--skip-dirs=**/.terraform"

- id: terraform_docs
args:
- --hook-config=--create-file-if-not-exist=true
- "--args=--lockfile=false"
Binary file added ARCHITECTURE_OVERVIEW.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions core/backend/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# backend

<!-- BEGIN_TF_DOCS -->
## Requirements

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.6.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.60 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | ~> 5.60 |

## Modules

No modules.

## Resources

| Name | Type |
|------|------|
| [aws_dynamodb_table.lock](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dynamodb_table) | resource |
| [aws_s3_bucket.state](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
| [aws_s3_bucket_lifecycle_configuration.state](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration) | resource |
| [aws_s3_bucket_public_access_block.state](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
| [aws_s3_bucket_server_side_encryption_configuration.state](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource |
| [aws_s3_bucket_versioning.state](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) | resource |

## Inputs

No inputs.

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_tflock_table"></a> [tflock\_table](#output\_tflock\_table) | n/a |
| <a name="output_tfstate_bucket"></a> [tfstate\_bucket](#output\_tfstate\_bucket) | n/a |
<!-- END_TF_DOCS -->
File renamed without changes.
File renamed without changes.
1 change: 0 additions & 1 deletion backend/versions.tf → core/backend/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ terraform {
version = "~> 5.60"
}
}
backend "s3" {}
}
provider "aws" {
region = "us-east-1"
Expand Down
49 changes: 49 additions & 0 deletions core/vpc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
> [!NOTE]
> Experimental VPC module.
> Not used by any environment in infra v1.
> Reserved for future refactors (infra v2).

# vpc

<!-- BEGIN_TF_DOCS -->
## Requirements

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.6.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.60 |

## Providers

No providers.

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | 5.8.1 |

## Resources

No resources.

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_azs"></a> [azs](#input\_azs) | n/a | `list(string)` | <pre>[<br/> "us-east-1a",<br/> "us-east-1b"<br/>]</pre> | no |
| <a name="input_env"></a> [env](#input\_env) | n/a | `string` | `"dev"` | no |
| <a name="input_private_subnet_cidrs"></a> [private\_subnet\_cidrs](#input\_private\_subnet\_cidrs) | n/a | `list(string)` | <pre>[<br/> "10.0.10.0/24",<br/> "10.0.11.0/24"<br/>]</pre> | no |
| <a name="input_project"></a> [project](#input\_project) | n/a | `string` | `"shopstack"` | no |
| <a name="input_public_subnet_cidrs"></a> [public\_subnet\_cidrs](#input\_public\_subnet\_cidrs) | n/a | `list(string)` | <pre>[<br/> "10.0.0.0/24",<br/> "10.0.1.0/24"<br/>]</pre> | no |
| <a name="input_region"></a> [region](#input\_region) | AWS region (currently unused in this module) | `string` | n/a | yes |
| <a name="input_vpc_cidr"></a> [vpc\_cidr](#input\_vpc\_cidr) | n/a | `string` | `"10.0.0.0/16"` | no |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_private_subnet_ids"></a> [private\_subnet\_ids](#output\_private\_subnet\_ids) | n/a |
| <a name="output_public_subnet_ids"></a> [public\_subnet\_ids](#output\_public\_subnet\_ids) | n/a |
| <a name="output_vpc_id"></a> [vpc\_id](#output\_vpc\_id) | Output VPC and subnet identifiers |
<!-- END_TF_DOCS -->
41 changes: 41 additions & 0 deletions core/vpc/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
module "vpc" {
# Use the official AWS VPC module from the Terraform Registry
source = "terraform-aws-modules/vpc/aws"
version = "5.8.1"

# Define VPC name based on project and environment
name = "${var.project}-${var.env}"

# Set the CIDR block for the VPC
cidr = var.vpc_cidr

# Define the Availability Zones to use
azs = var.azs

# Define CIDR blocks for public and private subnets
public_subnets = var.public_subnet_cidrs
private_subnets = var.private_subnet_cidrs

# Enable NAT Gateway for outbound traffic from private subnets
enable_nat_gateway = false

# Create one NAT Gateway per AZ for high availability
single_nat_gateway = false
one_nat_gateway_per_az = false

# Enable DNS hostnames and DNS resolution support within the VPC
enable_dns_hostnames = true
enable_dns_support = true

# Common tags for resource identification and management
tags = {
Project = var.project
Env = var.env
Managed = "terraform"
}
}

# Output VPC and subnet identifiers
output "vpc_id" { value = module.vpc.vpc_id }
output "public_subnet_ids" { value = module.vpc.public_subnets }
output "private_subnet_ids" { value = module.vpc.private_subnets }
35 changes: 35 additions & 0 deletions core/vpc/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
variable "project" {
type = string
default = "shopstack"
}

variable "env" {
type = string
default = "dev"
}

# tflint-ignore: terraform_unused_declarations
variable "region" {
type = string
description = "AWS region (currently unused in this module)"
}

variable "vpc_cidr" {
type = string
default = "10.0.0.0/16"
}

variable "azs" {
type = list(string)
default = ["us-east-1a", "us-east-1b"]
}

variable "public_subnet_cidrs" {
type = list(string)
default = ["10.0.0.0/24", "10.0.1.0/24"]
}

variable "private_subnet_cidrs" {
type = list(string)
default = ["10.0.10.0/24", "10.0.11.0/24"]
}
10 changes: 10 additions & 0 deletions core/vpc/versions.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
terraform {
required_version = ">= 1.6.0"

required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.60"
}
}
}
82 changes: 67 additions & 15 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,48 @@ A complete overview of the **ShopStack** project architecture and repository rel
## 📦 **Repositories Overview**

| Repository | Purpose | Main Technologies |
|-------------|----------|-------------------|
| **infra-terraform** | Defines and provisions the foundational infrastructure (S3 backend, DynamoDB for locks, Terraform remote state). | Terraform, AWS |
| **cluster-gitops** | Manages cluster provisioning, configuration, and repeatability testing using Kind and GitOps workflows. | Kind, Kubectl, Makefile |
| **apps-sre** | Hosts applications and observability stacks that run inside the Kubernetes cluster. | Docker, Helm, ArgoCD *(future)* |
|-----------|---------|-------------------|
| **infra-terraform** | Defines and provisions the foundational AWS infrastructure (Terraform remote state backend, IAM/OIDC integration, ECR, Secrets Manager, and build hooks). | Terraform, AWS (S3, DynamoDB, IAM, ECR, Secrets Manager) |
| **cluster-gitops** | Manages cluster provisioning, configuration, and repeatability testing using Kind and GitOps workflows. | Kind, kubectl, Makefile |
| **apps-sre** | Hosts applications and observability stacks that run inside the Kubernetes cluster. | Docker, Helm, Argo CD *(future)* |

---

## ☁️ **AWS Integration Scope (infra v1)**

This section defines the AWS integration scope for **infra v1** of ShopStack.

- **Terraform remote state backend**
- `core/backend` module creates:
- An S3 bucket to store Terraform remote state.
- A DynamoDB table to provide state locking.
- Used by `envs/dev` to safely manage infrastructure changes.

- **Secrets Manager**
- AWS Secrets Manager will store runtime secrets for ShopStack (API keys, database credentials, etc.).
- `core/secrets` module (infra-terraform) manages the secret metadata and naming convention (e.g. `shopstack/dev/app-config`).
- Secret values can be injected later into workloads (Kubernetes manifests or app configs) in future sprints.

- **ECR (Elastic Container Registry)**
- `core/ecr` module provisions an ECR repository per environment (e.g. `shopstack-dev`).
- Used to store container images built for ShopStack applications.
- Images are pushed from CI (GitHub Actions / CodeBuild) and later pulled by the runtime environment (Kind/Talos/EKS).

- **CodeBuild (build integration)**
- A CodeBuild project will be used as a managed build step to:
- Build Docker images from application repositories.
- Push images to the ECR repository created by `core/ecr`.
- CodeBuild will be triggered via GitHub Actions or future CI workflows using IAM roles instead of long-lived credentials.

- **GitHub OIDC (federated IAM access)**
- `core/github-oidc` module defines:
- An IAM OIDC provider for `token.actions.githubusercontent.com`.
- An IAM role that can be assumed only by specific GitHub repositories/branches (e.g. `repo:<org>/<repo>:ref:refs/heads/main`).
- This role will grant least-privilege access to:
- Read/write the Terraform remote state in S3/DynamoDB.
- Push images to ECR.
- Interact with CodeBuild when needed.
- This removes the need for static AWS keys in GitHub and aligns with AWS recommended security practices.

---

Expand All @@ -21,7 +59,10 @@ Developer
├── infra-terraform
│ ├── S3 bucket (Terraform backend)
│ └── DynamoDB table (state lock)
│ ├── DynamoDB table (Terraform state lock)
│ ├── AWS Secrets Manager (app secrets skeleton)
│ ├── ECR repository (container images per env)
│ └── IAM + OIDC role (GitHub Actions / CI access)
├── cluster-gitops
│ ├── Kind cluster (shopstack)
Expand Down Expand Up @@ -76,33 +117,44 @@ Results are saved to repeat.log.
```plaintext
chmod 600 ~/.kube/config
```
Docker daemon restricted to local use.
Terraform state secured in private AWS S3 bucket with DynamoDB locking.
- Terraform state stored in a private AWS S3 bucket with DynamoDB locking.
- GitHub CI uses federated IAM access via OIDC instead of static AWS credentials (infra v1 scope).

---

## 🧩 **Future Improvements**

- Add CI/CD pipeline for infrastructure changes.
- Integrate EKS managed cluster for cloud testing.
- Enable security scanning tools (Trivy, kube-bench).
- Automate bootstrap across all repositories with a unified Makefile.
- Add CI/CD pipeline for infrastructurechanges using GitHub Actions + OIDC.
- Integrate EKS managed cluster for cloud testing.
- Enable additional security scanning tools (Trivy, kube-bench, tfsec/Trivy for Terraform).
- Automate bootstrap across all repositories with a unified Makefile and shared scripts.
---

## 🗂️ **Folder Structure**
```bash
infra-terraform/
├── core/
│ ├── backend/ # S3 + DynamoDB for Terraform remote state
│ ├── vpc/ # Experimental VPC module (not used in infra v1)
│ ├── github-oidc/ # IAM OIDC provider + role for GitHub Actions (planned)
│ ├── ecr/ # ECR repository definitions (planned)
│ └── secrets/ # Secrets Manager layout (planned)
├── envs/
│ └── dev/
│ ├── backend.tf
│ ├── main.tf
│ ├── backend.tf # Backend configuration using S3/DynamoDB
│ ├── main.tf # Environment stack (modules wiring)
│ ├── outputs.tf
│ └── versions.tf
├── .github/
│ └── workflows/terraform-ci.yml
├── talos/ # Talos cluster configuration files (local cluster)
── backend/ # Local bootstrap state (not used in normal flows)
├── Makefile
└── docs/
└── ARCHITECTURE.md
```
# Architecture Overview

![Architecture Overview](ARCHITECTURE_OVERVIEW.png)

---

## 🧭 **Additional Documentation**
Expand Down
Binary file added docs/ARCHITECTURE_OVERVIEW.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading