[EPAC-2060]: migrate Terraform state to S3 remote backend with bootstrap-from-zero

.gitignore

@@ -112,4 +112,10 @@ backend/*/loader
 *.tfstate
 *.tfstate.backup
 *.tfstate.lock.info
+*.tfvars
 *.tfplan
+infra/terraform/**/.terraform/
+infra/terraform/**/*.tfstate
+infra/terraform/**/*.tfstate.backup
+infra/terraform/**/*.tfvars
+infra/terraform/**/*.tfplan

infra/terraform/README.md

@@ -2,13 +2,67 @@
 
 Terraform modules for the epac project:
 
-- **[core/](./core/)**: Core infrastructure for Terraform remote state plus artifact hosting (S3, CloudFront, OAC, ACM, Route53).
+- **[core/](./core/)**: Core infrastructure for shared artifact hosting (S3, CloudFront, OAC, ACM, Route53).
 - **[staging/](./staging/)**: Staging environment resources (Lambda, API Gateway, Route53).
 - **[production/](./production/)**: Production environment resources (Lambda, API Gateway, Route53).
 
 ## Remote State
 
-All modules use the remote S3 backend with DynamoDB locking.
+All modules use an S3 backend with DynamoDB locking in `us-east-1`.
 
-1. **Bootstrap**: The `core` module has been applied and its state is stored in S3.
-2. **Migration**: Once the core resources exist, run `terraform init` in `staging` and `production` to migrate existing local state to the remote backend.
+| Workspace | State bucket | State key | Lock table |
+|---|---|---|---|
+| core | `epac-tfstate-core-227530433709` | `core.tfstate` | `epac-tfstate-lock-core` |
+| staging | `epac-tfstate-staging-227530433709` | `staging.tfstate` | `epac-tfstate-lock-staging` |
+| production | `epac-tfstate-production-227530433709` | `production.tfstate` | `epac-tfstate-lock-production` |
+
+The account ID is hard-coded in each backend block because Terraform loads backend configuration before variables are available.
+
+## Bootstrap
+
+Run the bootstrap script before `terraform init` in a fresh account or before migrating existing state:
+
+```bash
+cd infra/terraform
+export AWS_PROFILE=riddim-agent
+./bootstrap.sh staging
+```
+
+`bootstrap.sh staging` creates the `core` and `staging` backend resources. `bootstrap.sh production` creates the `core` and `production` backend resources. The script is idempotent; existing buckets and lock tables produce `bucket exists, no-op` and `lock table exists, no-op` log lines.
+
+No human AWS console or pre-created bucket/table steps are required for state storage.
+
+## Local Terraform Use
+
+```bash
+cd infra/terraform/staging
+export AWS_PROFILE=riddim-agent
+terraform init
+terraform plan
+```
+
+For the shared artifact infrastructure, use `infra/terraform/core`. For production, use `infra/terraform/production` and keep the existing production human gate before any apply.
+
+To migrate an already-initialized local checkout from the previous backend configuration to the current backend:
+
+```bash
+terraform init -migrate-state -force-copy
+terraform state list
+terraform plan
+```
+
+Do not commit generated state, plan, variable, lock, or `.terraform/` files.
+
+## Bootstrap Smoke Test
+
+From a scratch AWS account or disposable sub-account with the IAM permissions documented in `bootstrap.sh`, run:
+
+```bash
+cd infra/terraform
+./bootstrap.sh staging
+./bootstrap.sh staging
+aws s3api get-bucket-versioning --bucket "epac-tfstate-staging-$(aws sts get-caller-identity --query Account --output text --region us-east-1)" --region us-east-1
+aws dynamodb describe-table --table-name epac-tfstate-lock-staging --region us-east-1 --query 'Table.BillingModeSummary.BillingMode'
+```
+
+The first run creates the `core` and `staging` buckets/tables. The second run exits 0 with no-op lines for both workspaces.

infra/terraform/bootstrap.sh

@@ -0,0 +1,100 @@
+#!/usr/bin/env bash
+#
+# Bootstrap Terraform remote state storage for epac.
+#
+# Required environment:
+# - AWS credentials for the target account, supplied by the standard AWS
+#   credential chain. AWS_PROFILE is optional for local use.
+# - The script always targets us-east-1; no AWS_REGION override is required.
+#
+# Required IAM permissions:
+# - s3:CreateBucket
+# - s3:PutBucketVersioning
+# - s3:PutBucketEncryption
+# - s3:PutBucketPublicAccessBlock
+# - s3:HeadBucket
+# - dynamodb:CreateTable
+# - dynamodb:DescribeTable
+# - sts:GetCallerIdentity
+#
+# Usage:
+#   ./bootstrap.sh staging
+#   ./bootstrap.sh production
+set -euo pipefail
+
+readonly REGION="us-east-1"
+
+usage() {
+  printf 'Usage: %s ENVIRONMENT\n\nENVIRONMENT must be staging or production.\n' "$0" >&2
+}
+
+if [ "$#" -ne 1 ]; then
+  usage
+  exit 2
+fi
+
+readonly ENVIRONMENT="$1"
+
+case "$ENVIRONMENT" in
+  staging | production)
+    ;;
+  *)
+    usage
+    exit 2
+    ;;
+esac
+
+ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text --region "$REGION")"
+readonly ACCOUNT_ID
+
+ensure_bucket() {
+  local workspace="$1"
+  local bucket="epac-tfstate-${workspace}-${ACCOUNT_ID}"
+
+  if aws s3api head-bucket --bucket "$bucket" --region "$REGION" >/dev/null 2>&1; then
+    printf '[%s] bucket exists, no-op: %s\n' "$workspace" "$bucket"
+    return 0
+  fi
+
+  printf '[%s] creating bucket: %s\n' "$workspace" "$bucket"
+  aws s3api create-bucket --bucket "$bucket" --region "$REGION" >/dev/null
+  aws s3api put-bucket-versioning \
+    --bucket "$bucket" \
+    --versioning-configuration Status=Enabled \
+    --region "$REGION"
+  aws s3api put-bucket-encryption \
+    --bucket "$bucket" \
+    --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}' \
+    --region "$REGION"
+  aws s3api put-public-access-block \
+    --bucket "$bucket" \
+    --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true \
+    --region "$REGION"
+}
+
+ensure_lock_table() {
+  local workspace="$1"
+  local table="epac-tfstate-lock-${workspace}"
+
+  if aws dynamodb describe-table --table-name "$table" --region "$REGION" >/dev/null 2>&1; then
+    printf '[%s] lock table exists, no-op: %s\n' "$workspace" "$table"
+    return 0
+  fi
+
+  printf '[%s] creating lock table: %s\n' "$workspace" "$table"
+  aws dynamodb create-table \
+    --table-name "$table" \
+    --attribute-definitions AttributeName=LockID,AttributeType=S \
+    --key-schema AttributeName=LockID,KeyType=HASH \
+    --billing-mode PAY_PER_REQUEST \
+    --region "$REGION" \
+    >/dev/null
+  aws dynamodb wait table-exists --table-name "$table" --region "$REGION"
+}
+
+# The core workspace hosts shared resources and has its own backend. Bootstrapping
+# staging or production also ensures core can initialize from a fresh account.
+for workspace in core "$ENVIRONMENT"; do
+  ensure_bucket "$workspace"
+  ensure_lock_table "$workspace"
+done

infra/terraform/core/README.md

@@ -1,22 +1,23 @@
-# epac terraform bootstrap
+# epac core infrastructure
+
+This module manages shared epac infrastructure:
 
-This module manages the core infrastructure required for Terraform remote state:
-- S3 bucket for state storage with versioning and encryption
-- DynamoDB table for state locking
 - S3 + CloudFront + OAC artifact hosting in [`artifacts/`](./artifacts/)
 
+Terraform remote state storage is created by [`../bootstrap.sh`](../bootstrap.sh), not by this module. That avoids the bootstrap cycle where Terraform needs a remote backend before it can create its own state bucket.
+
 ## Usage
 
-This module uses **remote state** (S3).
+This module uses S3 remote state with DynamoDB locking.
 
 ```bash
-cd infra/terraform/core
+cd infra/terraform
+./bootstrap.sh staging
+cd core
 export AWS_PROFILE=riddim-agent
 terraform init
 terraform plan
 terraform apply
 ```
 
-Once applied, other modules (staging, production) can use the created resources in their `backend "s3"` blocks.
-
 The artifact module defaults to `epac-assets.riddimsoftware.com` because `epac.app` is not currently present as a public Route 53 hosted zone. See [`artifacts/README.md`](./artifacts/README.md) for switching to `assets.epac.app` after that zone exists and for the post-apply smoke test.

infra/terraform/core/main.tf

@@ -1,60 +1,3 @@
-resource "aws_s3_bucket" "terraform_state" {
-  bucket = "epac-terraform-state"
-
-  lifecycle {
-    prevent_destroy = true
-  }
-
-  tags = {
-    Project   = "epac"
-    ManagedBy = "terraform"
-    Ticket    = "EPAC-1852"
-  }
-}
-
-resource "aws_s3_bucket_versioning" "terraform_state" {
-  bucket = aws_s3_bucket.terraform_state.id
-  versioning_configuration {
-    status = "Enabled"
-  }
-}
-
-resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
-  bucket = aws_s3_bucket.terraform_state.id
-
-  rule {
-    apply_server_side_encryption_by_default {
-      sse_algorithm = "AES256"
-    }
-  }
-}
-
-resource "aws_s3_bucket_public_access_block" "terraform_state" {
-  bucket = aws_s3_bucket.terraform_state.id
-
-  block_public_acls       = true
-  block_public_policy     = true
-  ignore_public_acls      = true
-  restrict_public_buckets = true
-}
-
-resource "aws_dynamodb_table" "terraform_locks" {
-  name         = "epac-terraform-locks"
-  billing_mode = "PAY_PER_REQUEST"
-  hash_key     = "LockID"
-
-  attribute {
-    name = "LockID"
-    type = "S"
-  }
-
-  tags = {
-    Project   = "epac"
-    ManagedBy = "terraform"
-    Ticket    = "EPAC-1852"
-  }
-}
-
 module "artifacts" {
   source = "./artifacts"
 

infra/terraform/core/outputs.tf

@@ -1,11 +1,3 @@
-output "terraform_state_bucket_name" {
-  value = aws_s3_bucket.terraform_state.id
-}
-
-output "terraform_locks_table_name" {
-  value = aws_dynamodb_table.terraform_locks.name
-}
-
 output "artifacts_bucket_arn" {
   description = "ARN of the epac artifacts bucket."
   value       = module.artifacts.bucket_arn

infra/terraform/core/versions.tf

@@ -2,10 +2,10 @@ terraform {
   required_version = ">= 1.5"
 
   backend "s3" {
-    bucket         = "epac-terraform-state"
-    key            = "epac/core/terraform.tfstate"
+    bucket         = "epac-tfstate-core-227530433709"
+    key            = "core.tfstate"
     region         = "us-east-1"
-    dynamodb_table = "epac-terraform-locks"
+    dynamodb_table = "epac-tfstate-lock-core"
     encrypt        = true
   }
 

infra/terraform/production/README.md

@@ -14,6 +14,7 @@ Terraform module for the epac production backend: the production HTTP API, produ
 ```bash
 cd infra/terraform/production
 export AWS_PROFILE=riddim-agent
+../bootstrap.sh production
 terraform init
 terraform plan
 ```
@@ -106,4 +107,4 @@ The production API also contains older staging-target integrations left over fro
 
 - Lambda code is not managed by Terraform. `placeholder.zip` is used only to create missing production functions; `lifecycle.ignore_changes` prevents Terraform from overwriting deployed code.
 - Production stage `auto_deploy` remains `false` to preserve the existing production release gate.
-- State is remote (S3). Remote state (S3 + DynamoDB lock) was added in EPAC-1852. State is stored in the `epac-terraform-state` bucket with locks in `epac-terraform-locks`.
+- State is remote (S3). Production state is stored in `epac-tfstate-production-227530433709` with locks in `epac-tfstate-lock-production`.

infra/terraform/production/versions.tf

@@ -2,10 +2,10 @@ terraform {
   required_version = ">= 1.5"
 
   backend "s3" {
-    bucket         = "epac-terraform-state"
-    key            = "epac/production/terraform.tfstate"
+    bucket         = "epac-tfstate-production-227530433709"
+    key            = "production.tfstate"
     region         = "us-east-1"
-    dynamodb_table = "epac-terraform-locks"
+    dynamodb_table = "epac-tfstate-lock-production"
     encrypt        = true
   }
 

infra/terraform/staging/README.md

@@ -13,6 +13,7 @@ Terraform module for the epac staging backend: 10 Lambda functions, API Gateway
 ```bash
 cd infra/terraform/staging
 export AWS_PROFILE=riddim-agent
+../bootstrap.sh staging
 terraform init
 terraform plan   # should show 0 changes after import
 terraform apply  # idempotent once imported
@@ -65,4 +66,4 @@ The EPAC-1914 artifact-backed functions (`members`, `sittings`, and `bills`) may
 
 - Lambda **code** is managed by `.github/workflows/backend-staging.yml`, not Terraform. The `placeholder.zip` is used only when creating a function from scratch; `lifecycle.ignore_changes` prevents Terraform from overwriting CI-deployed code.
 - State is remote (S3). `terraform.tfstate` files should not be committed (covered by the repo root `.gitignore`).
-- Remote state (S3 + DynamoDB lock) is enabled. State is stored in the `epac-terraform-state` bucket with locks in `epac-terraform-locks`.
+- Remote state (S3 + DynamoDB lock) is enabled. Staging state is stored in `epac-tfstate-staging-227530433709` with locks in `epac-tfstate-lock-staging`.

infra/terraform/staging/versions.tf

@@ -2,10 +2,10 @@ terraform {
   required_version = ">= 1.5"
 
   backend "s3" {
-    bucket         = "epac-terraform-state"
-    key            = "epac/staging/terraform.tfstate"
+    bucket         = "epac-tfstate-staging-227530433709"
+    key            = "staging.tfstate"
     region         = "us-east-1"
-    dynamodb_table = "epac-terraform-locks"
+    dynamodb_table = "epac-tfstate-lock-staging"
     encrypt        = true
   }