From db61544c47c06011213593a32c086e5039cf04bc Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Wed, 5 Aug 2026 08:19:05 -0500 Subject: [PATCH 01/18] docs(prd): correct stale status headers found during Terragrunt migration research MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Checkpoint before syncing this branch with origin/main β€” these fixes were made against an older snapshot and will likely need rework once current upstream content is merged in. Co-Authored-By: Claude Sonnet 5 --- docs/prd/code-generation.md | 2 ++ docs/prd/custom-hooks.md | 2 +- docs/prd/dag-concurrent-execution.md | 10 +++++----- docs/prd/import-adapter-registry.md | 2 ++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/prd/code-generation.md b/docs/prd/code-generation.md index c9080c78934..b918293a4e3 100644 --- a/docs/prd/code-generation.md +++ b/docs/prd/code-generation.md @@ -1,5 +1,7 @@ # PRD: Generate Section in Atmos Stack Config +**Status:** 🟒 Shipped β€” `cmd/terraform/generate/files.go`, `atmos terraform generate files`, and `auto_generate_files` are implemented and documented at `website/docs/stacks/generate.mdx`. + ## Overview Add a `generate` section to Atmos stack configuration that allows users to declaratively define files to be generated alongside Terraform components. Files are written to the component directory and support templating with full component context. diff --git a/docs/prd/custom-hooks.md b/docs/prd/custom-hooks.md index 6d17963227e..4ffed10acb9 100644 --- a/docs/prd/custom-hooks.md +++ b/docs/prd/custom-hooks.md @@ -1,6 +1,6 @@ # Custom Hooks -**Status**: 🟑 In Progress (kind system, scanner kinds, `--skip-hooks`, dependency auto-install, and workdir compatibility have shipped on the in-flight PR; Pro upload backend still pending β€” see Implementation Plan below) +**Status**: 🟒 Shipped (core) β€” kind system, scanner kinds (`infracost`/`trivy`/`checkov`/`kics`), the generic `command` kind, `--skip-hooks`, dependency auto-install, and workdir compatibility are all shipped. Only the Atmos Pro upload backend remains outstanding β€” see Implementation Plan below. Core hook functionality is production-ready today. **Last Updated**: 2026-05-22 diff --git a/docs/prd/dag-concurrent-execution.md b/docs/prd/dag-concurrent-execution.md index a69be25e0d5..bbc9e6444b9 100644 --- a/docs/prd/dag-concurrent-execution.md +++ b/docs/prd/dag-concurrent-execution.md @@ -1,8 +1,8 @@ # PRD: DAG-Based Concurrent Execution -**Status:** Draft +**Status:** Mostly Shipped β€” Phases 1–3 (foundation packages, `pkg/scheduler/` + Terraform adapter with `--max-concurrency` on `plan`/`apply`/`deploy`/`destroy`, and `--affected`/`--query` routing consolidation onto the scheduler) are implemented (`pkg/scheduler/scheduler.go`, `pkg/scheduler/adapters/terraform.go`, `internal/exec/terraform_affected.go`, `internal/exec/terraform_query.go`). Phase 4 (per-type concurrency limits, critical-path scheduling, TUI progress display, resumability) remains open. **Version:** 2.0 -**Last Updated:** 2026-03-16 +**Last Updated:** 2026-07-11 **Author:** Erik Osterman --- @@ -219,10 +219,10 @@ Fully implemented and shipped (#1516): - `builder.go` β€” `GraphBuilder` for constructing graphs - `filter.go` β€” Filter by type, stack, component; connected components -### Dependency-Ordered Execution (`internal/exec/terraform_all.go`) -- `ExecuteTerraformAll()` β€” Builds DAG from `settings.depends_on`, executes in topological order +### Dependency-Ordered Execution (`internal/exec/terraform_all.go`, `pkg/scheduler/`) +- `ExecuteTerraformAll()` β€” Builds DAG from `settings.depends_on`/`dependencies.components`, executes via `pkg/scheduler/adapters/terraform.go` - `buildTerraformDependencyGraph()` β€” Constructs graph from stack configs -- `executeInDependencyOrder()` β€” **Currently sequential** (iterates sorted nodes one by one) +- Execution now runs through the ready-queue `Scheduler` (`pkg/scheduler/scheduler.go`) with `--max-concurrency` controlling worker count (default `1`, sequential) β€” no longer a plain sequential loop - Reverse order for `destroy` - Cross-stack dependency support diff --git a/docs/prd/import-adapter-registry.md b/docs/prd/import-adapter-registry.md index f07390245e4..4e95085195d 100644 --- a/docs/prd/import-adapter-registry.md +++ b/docs/prd/import-adapter-registry.md @@ -1,5 +1,7 @@ # Import Adapter Registry Pattern +**Status:** Phase 1 (registry pattern, `pkg/config/import_adapter.go`, `pkg/config/import_adapter_registry.go`) is implemented. Phase 2's `terragrunt://` HCLβ†’YAML transform adapter is **not implemented** β€” only referenced in a doc comment as a planned adapter type. There is no automatic `terragrunt.hcl` β†’ Atmos YAML conversion today; migration is manual/agent-assisted. + ## Overview This document describes the import adapter registry pattern for Atmos, which provides a modular, extensible architecture for custom import schemes that require **transformation** (like `terragrunt://` for HCLβ†’YAML conversion, or `mock://` for testing). Standard go-getter schemes continue to work through the existing infrastructure. From 88fc22baaa959c51c1ca548ab153381ba7d3e4dc Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Wed, 5 Aug 2026 15:56:41 -0500 Subject: [PATCH 02/18] docs(migration): add Terragrunt migration skill reference and correct stale PRD statuses Adds the atmos-migration skill's Terragrunt reference (classic and Stacks patterns, concept mapping, migration workflow), hands-on-validated against a real Terragrunt Stacks example run end to end on the floci/aws emulator. Corrects four PRD status headers that had gone stale relative to shipped code, fixes pre-existing EditorConfig indentation violations the commit hook surfaced in two of those files, and documents the mocks/--use-mocks feature in the website Terragrunt migration guide as the direct equivalent of mock_outputs. Co-Authored-By: Claude Sonnet 5 --- agent-skills/skills/atmos-migration/SKILL.md | 7 +- .../references/from-terragrunt.md | 369 ++++++++++++++++++ docs/prd/code-generation.md | 22 +- docs/prd/custom-hooks.md | 2 +- docs/prd/dag-concurrent-execution.md | 31 +- docs/prd/import-adapter-registry.md | 2 +- website/docs/migration/terragrunt.mdx | 43 ++ 7 files changed, 445 insertions(+), 31 deletions(-) create mode 100644 agent-skills/skills/atmos-migration/references/from-terragrunt.md diff --git a/agent-skills/skills/atmos-migration/SKILL.md b/agent-skills/skills/atmos-migration/SKILL.md index 89bf8dbd8d5..6ee135873ad 100644 --- a/agent-skills/skills/atmos-migration/SKILL.md +++ b/agent-skills/skills/atmos-migration/SKILL.md @@ -9,6 +9,7 @@ references: - references/from-terraform-workspaces.md - references/remote-state-bridge.md - references/from-component-updater.md + - references/from-terragrunt.md --- # Migrating to Atmos @@ -24,7 +25,8 @@ For full prose tutorials aimed at end users, link to: - [Migrating from Native Terraform](https://atmos.tools/migration/native-terraform) - [Migrating from Terraform Workspaces](https://atmos.tools/migration/terraform-workspaces) -- [Migrating from Terragrunt](https://atmos.tools/migration/terragrunt) (not covered by this skill) +- [Migrating from Terragrunt](https://atmos.tools/migration/terragrunt) -- see + [from-terragrunt.md](references/from-terragrunt.md) for the agent-actionable recipes ## Terraform or OpenTofu @@ -75,6 +77,7 @@ different reference: | `terraform.workspace`-driven environments with shared state backend | [from-terraform-workspaces.md](references/from-terraform-workspaces.md) | | Need to read outputs from un-migrated TF (legacy or another repo) | [remote-state-bridge.md](references/remote-state-bridge.md) | | `cloudposse/github-action-atmos-component-updater` | [from-component-updater.md](references/from-component-updater.md) | +| Terragrunt (`terragrunt.hcl` or `terragrunt.stack.hcl`) | [from-terragrunt.md](references/from-terragrunt.md) | The remote-state-bridge pattern is what makes **progressive, component-by-component migration** possible. Without it, a team is forced into a big-bang cutover. Cover it any time the user has @@ -188,3 +191,5 @@ Things to push back on if a user (or another agent) proposes them during migrati workspaces to stacks without losing state - [References/remote-state-bridge.md](references/remote-state-bridge.md) -- the dummy-component and abstract-component patterns for reading state from un-migrated or external Terraform +- [References/from-terragrunt.md](references/from-terragrunt.md) -- concept mapping and migration + workflow for classic Terragrunt and Terragrunt Stacks diff --git a/agent-skills/skills/atmos-migration/references/from-terragrunt.md b/agent-skills/skills/atmos-migration/references/from-terragrunt.md new file mode 100644 index 00000000000..70edf69dbe1 --- /dev/null +++ b/agent-skills/skills/atmos-migration/references/from-terragrunt.md @@ -0,0 +1,369 @@ +# Migrating from Terragrunt + +This reference covers both Terragrunt shapes: the classic pattern (`terragrunt.hcl`, +`include`, `find_in_parent_folders()`) and the newer Stacks pattern +(`terragrunt.stack.hcl`, `unit` blocks). For the full user-facing prose tutorial with +complete concept and function mapping tables, see +[atmos.tools/migration/terragrunt](https://atmos.tools/migration/terragrunt). This +reference distills that tutorial into agent-actionable recipes and adds material the +tutorial does not cover: how to translate Terragrunt Stacks. + +## Identifying the User's Shape + +Check the repository for these signals before proposing anything: + +| Signal | Shape | Recipe | +|---|---|---| +| `terragrunt.hcl` files, no `terragrunt.stack.hcl` | Classic | This reference, sections below | +| `terragrunt.stack.hcl` present | Stacks | This reference, plus [Terragrunt Stacks](#terragrunt-stacks-map-directly-to-atmos-stacks) below | +| A Gruntwork "Runbooks" scaffold on top of Stacks | Stacks + scaffolding | Same as Stacks β€” the scaffolding layer does not change the mapping | + +Mixed repositories exist. Treat each `terragrunt.hcl` or `terragrunt.stack.hcl` unit +independently, the same way multiple root modules get treated independently in a +native-Terraform migration. + +## Core Concept Mapping + +Each pair below shows the Terragrunt construct and its direct Atmos equivalent. Full +function-by-function tables live in the canonical tutorial; this section covers only +the constructs an agent needs to translate a real project. + +### `include` and `find_in_parent_folders()` β†’ stack imports + +Terragrunt's DRY mechanism climbs the directory tree at run time. Atmos resolves +inheritance declaratively through `import:` and deep-merge, with no directory-walk +step: + +```hcl +# terragrunt.hcl +include "root" { + path = find_in_parent_folders("root.hcl") +} +``` + +```yaml +# stacks/orgs/acme/prod/us-east-1.yaml +import: + - orgs/acme/_defaults + - mixins/region/us-east-1 +``` + +Atmos's import system supports deeper inheritance chains (org β†’ tenant β†’ stage β†’ +region) than Terragrunt's parent-folder walk, and it works the same way whether the +imported file lives next to the stack or in a separate repository. Route deeper +organization questions to the [atmos-design-patterns](../../atmos-design-patterns/SKILL.md) +skill. + +### `generate "backend"` / `generate "provider"` β†’ automatic backend and provider generation + +Terragrunt's `root.hcl` typically has a `generate "provider"` block and a +`remote_state { generate = {...} }` block that each write one file. Atmos generates +both automatically from stack configuration β€” no `generate` block needed for either: + +```hcl +generate "provider" { + path = "provider.tf" + if_exists = "overwrite_terragrunt" + contents = <` (or `--affected` for a subset). +Do not reach for `compositions` here β€” that primitive groups components for local +multi-kind operation (container, compose, terraform together) and explicitly does not +define execution order. Ordering and value-passing across units is +`dependencies.components` and `!terraform.state`, exactly as shown above, whether the +source project uses classic Terragrunt or Terragrunt Stacks. + +One difference worth naming to the user: Terragrunt Stacks pins each unit's catalog +source and version inline in the `unit` block. Atmos separates that concern into +`vendor.yaml` (or direct component authoring), so a stack manifest's `components:` +section only carries values, not source references. This is a cleaner separation of +concerns, not a missing capability β€” see +[from-terragrunt.md](#terraform-source-pinning--vendoring) above for the vendoring +side. + +## Migration Workflow + +1. **Inventory the source repository.** Find every `terragrunt.hcl` and + `terragrunt.stack.hcl`, and every `include`/`dependency` reference between them, to + build a migration order β€” the same reconnaissance step a native-Terraform migration + starts with. +2. **Stand up `atmos.yaml` and one stack file** for a single unit, converting `include` + and `inputs` per the concept mapping above. +3. **Wire `dependency` blocks** to `dependencies.components` and `!terraform.state`, + translating `mock_outputs` to the `// "default"` pattern. +4. **Convert `generate` blocks.** Backend and provider generation need no + configuration at all; anything else becomes a `generate:` stack section. +5. **Validate** with `atmos validate stacks`, then compare `atmos describe affected` + against the equivalent Terragrunt change-detection output. +6. **Make the result testable.** Wire the component's stack manifest to an + `aws/emulator` (or the matching cloud target) identity so `atmos terraform plan`/ + `apply` runs with zero real cloud credentials before the user connects a real + account. See the [atmos-emulator](../../atmos-emulator/SKILL.md) skill. +7. **Repeat per unit**, using [remote-state-bridge.md](remote-state-bridge.md) to keep + not-yet-migrated units reachable via `!terraform.state` during the transition. + +## When to Escalate to Other Skills + +- **Stack organization (orgs, tenants, accounts, regions)** β†’ [atmos-design-patterns](../../atmos-design-patterns/SKILL.md) +- **Vendoring converted module sources** β†’ [atmos-vendoring](../../atmos-vendoring/SKILL.md) +- **Abstract components, inheritance, catalog patterns** β†’ [atmos-components](../../atmos-components/SKILL.md) +- **Deep merging, imports, overrides** β†’ [atmos-stacks](../../atmos-stacks/SKILL.md) +- **Provider credentials, identity chaining** β†’ [atmos-auth](../../atmos-auth/SKILL.md) +- **Local emulator setup for testing the migrated stack** β†’ [atmos-emulator](../../atmos-emulator/SKILL.md) +- **YAML function selection** (`!terraform.output`, `!terraform.state`, `!store`) β†’ [atmos-yaml-functions](../../atmos-yaml-functions/SKILL.md) +- **Hooks beyond packaging** (scanners, cost estimation, custom commands) β†’ [atmos-hooks](../../atmos-hooks/SKILL.md) +- **Progressive, component-by-component migration** β†’ [remote-state-bridge.md](remote-state-bridge.md) + +## Anti-Patterns + +- **"Terragrunt Stacks needs the `compositions` feature."** No β€” an ordinary stack + manifest with `dependencies.components` already covers it. `compositions` solves a + different problem (grouping components of different kinds for local operation). +- **"Atmos cannot run components in parallel like `run-all --parallelism`."** No β€” + `atmos terraform apply --all --max-concurrency N` runs a dependency-ordered + concurrent scheduler. Confirm the installed Atmos version if `--max-concurrency` + does not appear in `--help`; it shipped after a long sequential-only period. +- **"Port every `generate` block one-to-one with a `kind: command` hook."** No β€” check + whether the block only writes a backend or provider file first (automatic, no + configuration needed) before reaching for a hook. +- **"There is a `terragrunt://` import scheme that converts `terragrunt.hcl` + automatically."** No β€” this is a documented future proposal, not a shipped feature. + Translation is manual or agent-assisted. + +## Additional Resources + +- [remote-state-bridge.md](remote-state-bridge.md) β€” progressive migration technique, + identical to the native-Terraform case +- [atmos.tools/migration/terragrunt](https://atmos.tools/migration/terragrunt) β€” the + full prose tutorial, including complete concept and function mapping tables diff --git a/docs/prd/code-generation.md b/docs/prd/code-generation.md index b918293a4e3..8c777be793c 100644 --- a/docs/prd/code-generation.md +++ b/docs/prd/code-generation.md @@ -1,6 +1,6 @@ # PRD: Generate Section in Atmos Stack Config -**Status:** 🟒 Shipped β€” `cmd/terraform/generate/files.go`, `atmos terraform generate files`, and `auto_generate_files` are implemented and documented at `website/docs/stacks/generate.mdx`. +**Status:** Shipped. Atmos implements this feature in `cmd/terraform/generate/files.go` and the `atmos terraform generate files` command, and documents `auto_generate_files` at `website/docs/stacks/generate.mdx`. ## Overview @@ -433,10 +433,10 @@ func init() { 1. **No CommandProvider needed**: Unlike top-level commands, subcommands register directly with their parent via `GenerateCmd.AddCommand(filesCmd)` in `init()` 2. **Update GenerateCmd help text**: Add "files" to the list in `cmd/terraform/generate/generate.go`: - ```go - Long: `... - - 'files' to generate files from the generate section for an Atmos component.`, - ``` + ```go + Long: `... + - 'files' to generate files from the generate section for an Atmos component.`, + ``` 3. **Pattern follows existing commands**: See `cmd/terraform/generate/varfiles.go` as the reference implementation @@ -543,12 +543,12 @@ File generation is implemented as a built-in feature rather than user-configurab 2. **Command tests** in `cmd/terraform/generate/files/files_test.go` 3. **Integration tests** in `tests/` with fixtures in `tests/test-cases/generate-files/` 4. **Test cases:** - - Single file generation (JSON, YAML, HCL, string) - - Multi-level merge behavior - - Template variable substitution - - Auto-generation trigger - - Dry-run mode - - Error cases + - Single file generation (JSON, YAML, HCL, string) + - Multi-level merge behavior + - Template variable substitution + - Auto-generation trigger + - Dry-run mode + - Error cases ## Documentation diff --git a/docs/prd/custom-hooks.md b/docs/prd/custom-hooks.md index 6edb1b65c12..32f448a55ad 100644 --- a/docs/prd/custom-hooks.md +++ b/docs/prd/custom-hooks.md @@ -1,6 +1,6 @@ # Custom Hooks -**Status**: 🟒 Shipped (core) β€” kind system, scanner kinds (`infracost`/`trivy`/`checkov`/`kics`), the generic `command` kind, `--skip-hooks`, dependency auto-install, and workdir compatibility are all shipped. Only the Atmos Pro upload backend remains outstanding β€” see Implementation Plan below. Core hook functionality is production-ready today. +**Status**: Shipped (core). Atmos ships the kind system, the scanner kinds (`infracost`, `trivy`, `checkov`, `kics`), the generic `command` kind, `--skip-hooks`, dependency auto-install, and workdir compatibility. Only the Atmos Pro upload backend remains outstanding β€” see Implementation Plan below. Core hook functionality is production-ready today. **Last Updated**: 2026-07-24 diff --git a/docs/prd/dag-concurrent-execution.md b/docs/prd/dag-concurrent-execution.md index bbc9e6444b9..8b3e0f6206d 100644 --- a/docs/prd/dag-concurrent-execution.md +++ b/docs/prd/dag-concurrent-execution.md @@ -1,6 +1,6 @@ # PRD: DAG-Based Concurrent Execution -**Status:** Mostly Shipped β€” Phases 1–3 (foundation packages, `pkg/scheduler/` + Terraform adapter with `--max-concurrency` on `plan`/`apply`/`deploy`/`destroy`, and `--affected`/`--query` routing consolidation onto the scheduler) are implemented (`pkg/scheduler/scheduler.go`, `pkg/scheduler/adapters/terraform.go`, `internal/exec/terraform_affected.go`, `internal/exec/terraform_query.go`). Phase 4 (per-type concurrency limits, critical-path scheduling, TUI progress display, resumability) remains open. +**Status:** Mostly shipped. Atmos implements Phases 1–3 β€” foundation packages, the `pkg/scheduler/` Terraform adapter with `--max-concurrency` on `plan`/`apply`/`deploy`/`destroy`, and the `--affected`/`--query` routing consolidation onto the scheduler β€” in `pkg/scheduler/scheduler.go`, `pkg/scheduler/adapters/terraform.go`, `internal/exec/terraform_affected.go`, and `internal/exec/terraform_query.go`. Phase 4 (per-type concurrency limits, critical-path scheduling, a TUI progress display, and resumability) remains open. **Version:** 2.0 **Last Updated:** 2026-07-11 **Author:** Erik Osterman @@ -70,10 +70,10 @@ Terragrunt originally organized units into "run groups" β€” sets of units at the Two problems drove the change: 1. **The "slowest unit" problem.** From the RFC: - > *"There is wasted time in a run, as groups execute when they have no dependent groups they are waiting on. A group dependent on another group will only start running when the slowest Unit in the dependency completes."* + > *"There is wasted time in a run, as groups execute when they have no dependent groups they are waiting on. A group dependent on another group will only start running when the slowest Unit in the dependency completes."* 2. **Failure blast radius.** From the RFC: - > *"Individual Units failing during runs can cause entire groups, and dependent groups to fail, ultimately meaning that individual failing Units can cause widespread failure for a Stack."* + > *"Individual Units failing during runs can cause entire groups, and dependent groups to fail, ultimately meaning that individual failing Units can cause widespread failure for a Stack."* The RFC includes timing diagrams proving that the worst case for runner pool equals the best case for level-based β€” it can never be slower, only faster. @@ -162,8 +162,8 @@ The industry standard is **modified Kahn's algorithm with a ready queue and work 2. Seed ready queue with all zero-in-degree nodes (roots) 3. Workers pull from ready queue (bounded by --max-concurrency) 4. On node completion: - a. Atomically decrement in-degree of all dependents - b. Any dependent reaching in-degree 0 enters the ready queue + a. Atomically decrement in-degree of all dependents + b. Any dependent reaching in-degree 0 enters the ready queue 5. Repeat until queue empty + all workers idle, OR error ``` @@ -194,11 +194,8 @@ The real difference shows with **asymmetric diamonds** where branches have diffe ### Diamond Dependencies (Fan-Out/Fan-In) ``` - A - / \ - B C - \ / - D +A -> B -> D +A -> C -> D ``` Handled naturally by in-degree counting: @@ -649,13 +646,13 @@ Stream injection is a prerequisite for the scheduler, not a future optimization. 1. **`prefixedWriter`** β€” follows the exact same `maskedWriter` pattern in `pkg/io/streams.go` (lines 97-124). Wraps an `io.Writer` and prepends a configurable prefix (e.g., `[vpc/tenant1-ue2-dev]`) to each line of output. Line-prefixed output is a general I/O concern reusable beyond scheduling. 2. **`NewOutput()`** β€” factory that composes an execution-scoped output pipeline: - ```go - func NewOutput(opts OutputOptions) Output - ``` - Each output pipeline branches after masking and prefixing so the terminal, log file, and capture sinks receive consistently labeled output: - - `maskedWriter` β†’ applies secret masking (shared global `Masker` β€” thread-safe, secrets are process-wide) - - `prefixedWriter` β†’ labels each line with the configured prefix - - `io.MultiWriter` β†’ fans out the composed writer to terminal, file, and capture sinks + ```go + func NewOutput(opts OutputOptions) Output + ``` + Each output pipeline branches after masking and prefixing so the terminal, log file, and capture sinks receive consistently labeled output: + - `maskedWriter` β†’ applies secret masking (shared global `Masker` β€” thread-safe, secrets are process-wide) + - `prefixedWriter` β†’ labels each line with the configured prefix + - `io.MultiWriter` β†’ fans out the composed writer to terminal, file, and capture sinks **Note on `maskedWriter` vs `dynamicMaskedWriter`:** The `dynamicMaskedWriter` pattern (line 129 of `streams.go`) resolves writers at write time via `getWriter func() io.Writer`. This is needed for the global singletons but NOT for per-node streams β€” each node has fixed writers. Use the simpler `maskedWriter` directly. diff --git a/docs/prd/import-adapter-registry.md b/docs/prd/import-adapter-registry.md index 4e95085195d..7acd8774c5a 100644 --- a/docs/prd/import-adapter-registry.md +++ b/docs/prd/import-adapter-registry.md @@ -1,6 +1,6 @@ # Import Adapter Registry Pattern -**Status:** Phase 1 (registry pattern, `pkg/config/import_adapter.go`, `pkg/config/import_adapter_registry.go`) is implemented. Phase 2's `terragrunt://` HCLβ†’YAML transform adapter is **not implemented** β€” only referenced in a doc comment as a planned adapter type. There is no automatic `terragrunt.hcl` β†’ Atmos YAML conversion today; migration is manual/agent-assisted. +**Status:** Atmos implements Phase 1 β€” the registry pattern β€” in `pkg/config/import_adapter.go` and `pkg/config/import_adapter_registry.go`. Atmos has not implemented Phase 2's `terragrunt://` HCL-to-YAML transform adapter; a doc comment references it only as a planned adapter type. Atmos has no automatic `terragrunt.hcl`-to-Atmos-YAML conversion today, so migration remains manual or agent-assisted. ## Overview diff --git a/website/docs/migration/terragrunt.mdx b/website/docs/migration/terragrunt.mdx index 88dbdd3f874..1afa59c17dc 100644 --- a/website/docs/migration/terragrunt.mdx +++ b/website/docs/migration/terragrunt.mdx @@ -210,6 +210,48 @@ If you're familiar with Terragrunt, the concepts below will help you translate w +### mock_outputs β†’ Component Mocks + + + + Terragrunt lets a `dependency` block declare `mock_outputs` so `plan` and `validate` still work before the dependency has been applied. `mock_outputs_allowed_terraform_commands` restricts which commands are allowed to use the mock values, so a real `apply` never runs against fake data by accident. + + + ```hcl + dependency "vpc" { + config_path = "../vpc" + mock_outputs = { + vpc_id = "vpc-mock1234" + private_subnet_ids = ["subnet-a", "subnet-b"] + } + mock_outputs_allowed_terraform_commands = ["validate", "plan"] + } + ``` + + + + Atmos has a direct equivalent: a component declares literal output values under `mocks`, and any command resolves them only when you explicitly pass `--use-mocks`. `!terraform.state` and `!terraform.output` read from the `mocks` map instead of real state when the flag is set, with no Terraform init, authentication, or backend access at all. + + + ```yaml + components: + terraform: + vpc: + mocks: + vpc_id: vpc-mock1234 + private_subnet_ids: [subnet-a, subnet-b] + ``` + + + ```shell + atmos terraform plan eks -s prod --use-mocks + atmos describe component eks -s prod --use-mocks + ``` + + `--use-mocks` is rejected on `apply`, `deploy`, and `destroy` before stack resolution even happens, so a mock value can never reach a mutating Terraform operation β€” a stronger guarantee than opting individual commands in through `mock_outputs_allowed_terraform_commands`. An undeclared `mocks` entry is also a hard error rather than a silent fallback. See the provider-free `examples/terraform-component-mocks` example in the Atmos repository for a complete walkthrough. + + + ### Inputs β†’ Vars @@ -979,6 +1021,7 @@ In Atmos terminology, root modules are called **components**. Each component is - [ ] Convert `terragrunt.hcl` files to stack YAML - [ ] Extract common config to `_defaults/` - [ ] Convert `dependency` blocks to remote state +- [ ] Convert `mock_outputs` to component `mocks` and `--use-mocks` - [ ] Update backend configuration - [ ] Test with `atmos terraform plan` - [ ] Update CI/CD pipelines From bfde6a61fd23d12f1c554246dbda18df0067c310 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Wed, 5 Aug 2026 16:32:33 -0500 Subject: [PATCH 03/18] fix(docs): use clean !terraform.state syntax and fix EditorConfig indentation CI caught two real issues in the new Terragrunt migration reference: - Two examples used the legacy doubled-double-quote YQ escaping (!terraform.state x ".field // ""default""") instead of the clean current syntax (!terraform.state x .field // "default"), which scripts/check- terraform-example-syntax.sh flags outside its designated compatibility fixtures. - The "Migration Workflow" numbered list used 3-space continuation indentation, not a multiple of the repo's 2-space EditorConfig setting. Co-Authored-By: Claude Sonnet 5 --- .../references/from-terragrunt.md | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/agent-skills/skills/atmos-migration/references/from-terragrunt.md b/agent-skills/skills/atmos-migration/references/from-terragrunt.md index 70edf69dbe1..be499e9d541 100644 --- a/agent-skills/skills/atmos-migration/references/from-terragrunt.md +++ b/agent-skills/skills/atmos-migration/references/from-terragrunt.md @@ -174,7 +174,7 @@ undeclared `mocks` entry is a hard error rather than a silent fallback. A workin provider-free reference lives at `examples/terraform-component-mocks` in the Atmos repository. -Reserve the YQ-default pattern (`!terraform.state vpc ".vpc_id // ""vpc-mock1234"""`) +Reserve the YQ-default pattern (`!terraform.state vpc .vpc_id // "vpc-mock1234"`) for the narrower case of a placeholder that should also apply during a normal, non-mock run against a dependency that genuinely has not deployed yet β€” for example, while bringing up a dependency graph for the first time. `mocks` and `--use-mocks` are @@ -291,7 +291,7 @@ components: vars: name: my-service runtime: nodejs22.x - iam_role_arn: !terraform.state iam-role ".arn // ""pending""" + iam_role_arn: !terraform.state iam-role .arn // "pending" iam-role: vars: name: my-service-role @@ -315,23 +315,23 @@ side. ## Migration Workflow 1. **Inventory the source repository.** Find every `terragrunt.hcl` and - `terragrunt.stack.hcl`, and every `include`/`dependency` reference between them, to - build a migration order β€” the same reconnaissance step a native-Terraform migration - starts with. + `terragrunt.stack.hcl`, and every `include`/`dependency` reference between them, to + build a migration order β€” the same reconnaissance step a native-Terraform migration + starts with. 2. **Stand up `atmos.yaml` and one stack file** for a single unit, converting `include` - and `inputs` per the concept mapping above. + and `inputs` per the concept mapping above. 3. **Wire `dependency` blocks** to `dependencies.components` and `!terraform.state`, - translating `mock_outputs` to the `// "default"` pattern. + translating `mock_outputs` to the `// "default"` pattern. 4. **Convert `generate` blocks.** Backend and provider generation need no - configuration at all; anything else becomes a `generate:` stack section. + configuration at all; anything else becomes a `generate:` stack section. 5. **Validate** with `atmos validate stacks`, then compare `atmos describe affected` - against the equivalent Terragrunt change-detection output. + against the equivalent Terragrunt change-detection output. 6. **Make the result testable.** Wire the component's stack manifest to an - `aws/emulator` (or the matching cloud target) identity so `atmos terraform plan`/ - `apply` runs with zero real cloud credentials before the user connects a real - account. See the [atmos-emulator](../../atmos-emulator/SKILL.md) skill. + `aws/emulator` (or the matching cloud target) identity so `atmos terraform plan`/ + `apply` runs with zero real cloud credentials before the user connects a real + account. See the [atmos-emulator](../../atmos-emulator/SKILL.md) skill. 7. **Repeat per unit**, using [remote-state-bridge.md](remote-state-bridge.md) to keep - not-yet-migrated units reachable via `!terraform.state` during the transition. + not-yet-migrated units reachable via `!terraform.state` during the transition. ## When to Escalate to Other Skills From 99be34a1b9e474d2993373531ff115b1a4673eab Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 10:04:19 -0500 Subject: [PATCH 04/18] fix(docs): address CodeRabbit findings and field-test gaps on Terragrunt migration guide Reconciles PRD status claims that contradicted themselves (dag-concurrent-execution.md Phase 3 is only partially shipped, not fully; custom-hooks.md's relative "today" date), completes the from-terragrunt.md 5-level merge listing, and fixes a hallucinated `settings.terraform.provider_overrides` key found via hands-on field testing. Also recommends `atmos list affected` over `atmos describe affected` for human-run migration comparisons (table output vs. a wall of YAML), notes both diff committed trees only, and updates the Change Tracking table to the current `dependencies.files`/`folders` syntax instead of the legacy inline `kind: file`/`kind: folder` form. Co-Authored-By: Claude Sonnet 5 --- .../references/from-terragrunt.md | 18 +++++++----- docs/prd/custom-hooks.md | 2 +- docs/prd/dag-concurrent-execution.md | 28 +++++++++---------- website/docs/migration/terragrunt.mdx | 19 +++++++------ 4 files changed, 37 insertions(+), 30 deletions(-) diff --git a/agent-skills/skills/atmos-migration/references/from-terragrunt.md b/agent-skills/skills/atmos-migration/references/from-terragrunt.md index be499e9d541..09581906587 100644 --- a/agent-skills/skills/atmos-migration/references/from-terragrunt.md +++ b/agent-skills/skills/atmos-migration/references/from-terragrunt.md @@ -91,16 +91,18 @@ terraform: Atmos writes `backend.tf.json` from this section automatically at plan/apply time. No provider-file generation is needed either β€” set provider region and account restrictions through stack `vars:` that the component's own `providers.tf` reads, or -through `settings.terraform.provider_overrides` when the provider block itself needs -per-stack values Atmos does not already inject through an identity. +through the `providers:` stack section (`website/docs/stacks/providers.mdx`, e.g. +`terraform.providers:` at the component-type level or `providers:` on the component) +when the provider block itself needs per-stack values Atmos does not already inject +through an identity. If a Terragrunt unit's `generate` block writes something other than a backend or provider file, Atmos has a direct, more general equivalent: the declarative `generate:` stack section (`website/docs/stacks/generate.mdx`), which writes arbitrary files from stack configuration with full templating and a 5-level merge (global, component-type, -component, and override). This covers the general case Terragrunt's `generate` block -handles; backend and provider files are simply the two cases Atmos automates without -any `generate:` configuration at all. +base component, component, and override). This covers the general case Terragrunt's +`generate` block handles; backend and provider files are simply the two cases Atmos +automates without any `generate:` configuration at all. ### `dependency` blocks β†’ `dependencies.components` and `!terraform.state` @@ -324,8 +326,10 @@ side. translating `mock_outputs` to the `// "default"` pattern. 4. **Convert `generate` blocks.** Backend and provider generation need no configuration at all; anything else becomes a `generate:` stack section. -5. **Validate** with `atmos validate stacks`, then compare `atmos describe affected` - against the equivalent Terragrunt change-detection output. +5. **Validate** with `atmos validate stacks`, then compare `atmos list affected` + (human-readable table; commit your change first β€” it diffs committed trees, not + the working tree) against the equivalent Terragrunt change-detection output. Use + `atmos describe affected` instead when scripting/CI needs the JSON/YAML form. 6. **Make the result testable.** Wire the component's stack manifest to an `aws/emulator` (or the matching cloud target) identity so `atmos terraform plan`/ `apply` runs with zero real cloud credentials before the user connects a real diff --git a/docs/prd/custom-hooks.md b/docs/prd/custom-hooks.md index 32f448a55ad..1bbdb57082e 100644 --- a/docs/prd/custom-hooks.md +++ b/docs/prd/custom-hooks.md @@ -1,6 +1,6 @@ # Custom Hooks -**Status**: Shipped (core). Atmos ships the kind system, the scanner kinds (`infracost`, `trivy`, `checkov`, `kics`), the generic `command` kind, `--skip-hooks`, dependency auto-install, and workdir compatibility. Only the Atmos Pro upload backend remains outstanding β€” see Implementation Plan below. Core hook functionality is production-ready today. +**Status**: Shipped (core). Atmos ships the kind system, the scanner kinds (`infracost`, `trivy`, `checkov`, `kics`), the generic `command` kind, `--skip-hooks`, dependency auto-install, and workdir compatibility. Only the Atmos Pro upload backend remains outstanding β€” see Implementation Plan below. Core hook functionality is production-ready as of 2026-07-24. **Last Updated**: 2026-07-24 diff --git a/docs/prd/dag-concurrent-execution.md b/docs/prd/dag-concurrent-execution.md index 8b3e0f6206d..a8ca7e64fdd 100644 --- a/docs/prd/dag-concurrent-execution.md +++ b/docs/prd/dag-concurrent-execution.md @@ -1,6 +1,6 @@ # PRD: DAG-Based Concurrent Execution -**Status:** Mostly shipped. Atmos implements Phases 1–3 β€” foundation packages, the `pkg/scheduler/` Terraform adapter with `--max-concurrency` on `plan`/`apply`/`deploy`/`destroy`, and the `--affected`/`--query` routing consolidation onto the scheduler β€” in `pkg/scheduler/scheduler.go`, `pkg/scheduler/adapters/terraform.go`, `internal/exec/terraform_affected.go`, and `internal/exec/terraform_query.go`. Phase 4 (per-type concurrency limits, critical-path scheduling, a TUI progress display, and resumability) remains open. +**Status:** Mostly shipped. Atmos implements Phases 1–2 in full, and Phase 3's Terraform-routing items β€” foundation packages, the `pkg/scheduler/` Terraform adapter with `--max-concurrency` on `plan`/`apply`/`deploy`/`destroy`, and the `--affected`/`--query` routing consolidation onto the scheduler β€” in `pkg/scheduler/scheduler.go`, `pkg/scheduler/adapters/terraform.go`, `internal/exec/terraform_affected.go`, and `internal/exec/terraform_query.go`. Phase 3's multi-type-DAG items (Packer/Ansible scheduler adapters, cross-type `depends_on`) and all of Phase 4 (per-type concurrency limits, critical-path scheduling, a TUI progress display, and resumability) remain open. **Version:** 2.0 **Last Updated:** 2026-07-11 **Author:** Erik Osterman @@ -256,10 +256,10 @@ The I/O package provides the stream isolation primitives that per-node output wi - Propagates TTY: injects `ATMOS_FORCE_TTY=true` when parent has TTY - `terraform_plan_diff.go` swaps global `os.Stdout` to capture output β€” race condition under concurrency -### Routing Gap -- `--all` for Terraform goes through `ExecuteTerraformAll()` (dependency-aware, sequential) -- `--components`, `--query` still route through `ExecuteTerraformQuery()` (no DAG awareness) -- No `--all` equivalent exists for other component types +### Routing Gap (resolved in Phase 3) +- `--all` for Terraform goes through `ExecuteTerraformAll()` (dependency-aware, `--max-concurrency` controls parallelism) +- `--components`, `--query` now route through `ExecuteTerraformQuery()` β†’ `scheduleradapters.ExecuteTerraform()` (DAG-aware via the scheduler, consolidated in Phase 3) +- No `--all` equivalent exists yet for other component types --- @@ -838,13 +838,13 @@ When users enable `--max-concurrency > 1`, understanding the DAG is critical for ## Phased Rollout -### Phase 1: Foundation Packages +### Phase 1: Foundation Packages β€” βœ… Shipped 1. Create `pkg/process/` with `Runner` interface, `TaskSpec`, `Streams`, `Result`, default exec-based implementation 2. Extend `pkg/io/` with `prefixedWriter` and `NewOutput()` factory (reuse existing `maskedWriter` pattern) 3. Refactor `ExecuteShellCommand()` to accept optional `process.Streams` parameter (backward-compatible: `nil` means current behavior) 4. Eliminate the `os.Stdout` swap pattern in `terraform_plan_diff.go` β€” replaced by stream injection -### Phase 2: Scheduler + Terraform Adapter +### Phase 2: Scheduler + Terraform Adapter β€” βœ… Shipped 1. Create `pkg/scheduler/` with pure scheduling logic (`Scheduler`, `Node`, `Dispatcher` interface, `AggregateResult`) 2. Create `pkg/scheduler/orchestrator.go` combining scheduler + process runner + adapter registry 3. Create `pkg/scheduler/adapters/` with `TerraformAdapter` (Prepare/Finalize calling existing exec functions) @@ -854,13 +854,13 @@ When users enable `--max-concurrency > 1`, understanding the DAG is critical for 7. JSON summary output (`--output json`) 8. Require `-auto-approve` when `--max-concurrency > 1` for `apply`/`destroy` -### Phase 3: Routing Consolidation + Multi-Type DAGs -1. Converge `--components` and `--query` onto the DAG-backed executor (currently `ExecuteTerraformQuery`) -2. Unify `--affected` path to use the scheduler -3. Add `--fail-fast` / `--keep-going` flags -4. Create `PackerAdapter`, `AnsibleAdapter` (wraps `ComponentProvider`), `ComponentProviderAdapter` for registered types -5. Extend graph building to include Packer and Ansible nodes -6. Cross-type `depends_on` syntax (e.g., `component: ami-builder, type: packer`) +### Phase 3: Routing Consolidation + Multi-Type DAGs β€” partially shipped +1. βœ… Shipped β€” Converge `--components` and `--query` onto the DAG-backed executor (currently `ExecuteTerraformQuery`) +2. βœ… Shipped β€” Unify `--affected` path to use the scheduler +3. βœ… Shipped β€” Failure-mode handling equivalent to `--fail-fast` / `--keep-going`, implemented as a single `--failure-mode {fail-fast,keep-going}` flag on `plan`/`apply`/`deploy`/`destroy` +4. Open β€” Create `PackerAdapter`, `AnsibleAdapter` (wraps `ComponentProvider`), `ComponentProviderAdapter` for registered types +5. Open β€” Extend graph building to include Packer and Ansible nodes +6. Open β€” Cross-type `depends_on` syntax (e.g., `component: ami-builder, type: packer`) ### Phase 4: Advanced Scheduling 1. Per-type concurrency limits (resource pools, like Ninja) diff --git a/website/docs/migration/terragrunt.mdx b/website/docs/migration/terragrunt.mdx index 1afa59c17dc..9e3d0165d61 100644 --- a/website/docs/migration/terragrunt.mdx +++ b/website/docs/migration/terragrunt.mdx @@ -788,8 +788,8 @@ Terragrunt exposes HCL functions for path discovery, environment lookup, command | Terragrunt Function | Atmos Equivalent | Notes | |---------------------|------------------|-------| -| `mark_as_read()` | `dependencies.components` with `kind: file` | Declare explicit file dependencies so `describe affected` can track changes. | -| `mark_glob_as_read()` | `dependencies.components` with `kind: folder` | Declare folder dependencies instead of imperatively marking glob reads. | +| `mark_as_read()` | `dependencies.files` | Declare explicit file dependencies so `describe affected`/`list affected` can track changes. | +| `mark_glob_as_read()` | `dependencies.folders` | Declare folder dependencies instead of imperatively marking glob reads. | | `constraint_check()` | Sprig `semverCompare` or tool dependency constraints | Use `semverCompare` in templates for conditional config, or Atmos toolchain dependency constraints for tool versions. | :::tip YAML Functions vs Templates @@ -854,8 +854,8 @@ See [YAML Functions](/functions/yaml) for the complete reference. **GitOps-native:** Atmos goes beyond simple "run all" with intelligent change detection: ```bash - # See what changed between commits (compares current branch to main) - atmos describe affected + # See what changed between commits (compares current branch to main; table output for humans) + atmos list affected # Plan ONLY components affected by changes in current branch atmos terraform plan --affected @@ -864,14 +864,17 @@ See [YAML Functions](/functions/yaml) for the complete reference. atmos terraform apply --affected # Compare against a specific branch or commit - atmos describe affected --ref refs/heads/feature-branch - atmos describe affected --sha abc123def + atmos list affected --ref refs/heads/feature-branch + atmos list affected --sha abc123def # Include dependent components (if vpc changed, also plan eks that depends on it) - atmos describe affected --include-dependents=true + atmos list affected --include-dependents=true + + # Same comparison as JSON/YAML, for scripting and CI + atmos describe affected --base main ``` - This is purpose-built for CI/CD: instead of planning everything on every PR, Atmos analyzes Git changes to determine exactly which components and stacks are affectedβ€”including changes to stack configs, component code, and even local Terraform modules. + This is purpose-built for CI/CD: instead of planning everything on every PR, Atmos analyzes Git changes to determine exactly which components and stacks are affectedβ€”including changes to stack configs, component code, and even local Terraform modules. `list affected` and `describe affected` both diff **committed** trees, not the working treeβ€”commit a change before comparing. From 05fc85602aa752160b9e18bf8b4c62f66c7b3bc7 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 12:32:25 -0500 Subject: [PATCH 05/18] fix(docs): resolve second CodeRabbit review round on Terragrunt migration guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 3 of the migration workflow still mapped mock_outputs to the YQ // "default" pattern, contradicting the mocks:/--use-mocks mapping documented a few paragraphs earlier. Quotes the YQ default expressions for consistency with atmos-yaml-functions/SKILL.md and atmos-components/SKILL.md. dag-concurrent-execution.md had two more self-contradictions: the Subprocess Execution section still described the os.Stdout race that Phase 1 already fixed (terraform_plan_diff.go now captures via bytes.Buffer), and the Resolved Questions section claimed cross-type dependency syntax was "solved by PR #2193" β€” traced the code and found pkg/scheduler/adapters/terraform.go explicitly skips any dependency whose kind isn't "terraform", so the kind field is schema-parseable but not yet consumed by the scheduler; corrected to match the already-accurate Phase 3 status. terragrunt.mdx's list-affected example claimed to compare against main by default without passing --ref; list affected has no --base flag (unlike describe affected), so made the comparison explicit with --ref main instead. Co-Authored-By: Claude Sonnet 5 --- .../skills/atmos-migration/references/from-terragrunt.md | 8 +++++--- docs/prd/dag-concurrent-execution.md | 4 ++-- website/docs/migration/terragrunt.mdx | 4 ++-- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/agent-skills/skills/atmos-migration/references/from-terragrunt.md b/agent-skills/skills/atmos-migration/references/from-terragrunt.md index 09581906587..7ca3a80dc31 100644 --- a/agent-skills/skills/atmos-migration/references/from-terragrunt.md +++ b/agent-skills/skills/atmos-migration/references/from-terragrunt.md @@ -176,7 +176,7 @@ undeclared `mocks` entry is a hard error rather than a silent fallback. A workin provider-free reference lives at `examples/terraform-component-mocks` in the Atmos repository. -Reserve the YQ-default pattern (`!terraform.state vpc .vpc_id // "vpc-mock1234"`) +Reserve the YQ-default pattern (`!terraform.state vpc '.vpc_id // "vpc-mock1234"'`) for the narrower case of a placeholder that should also apply during a normal, non-mock run against a dependency that genuinely has not deployed yet β€” for example, while bringing up a dependency graph for the first time. `mocks` and `--use-mocks` are @@ -293,7 +293,7 @@ components: vars: name: my-service runtime: nodejs22.x - iam_role_arn: !terraform.state iam-role .arn // "pending" + iam_role_arn: !terraform.state iam-role '.arn // "pending"' iam-role: vars: name: my-service-role @@ -323,7 +323,9 @@ side. 2. **Stand up `atmos.yaml` and one stack file** for a single unit, converting `include` and `inputs` per the concept mapping above. 3. **Wire `dependency` blocks** to `dependencies.components` and `!terraform.state`, - translating `mock_outputs` to the `// "default"` pattern. + mapping `mock_outputs` to the producer component's `mocks:` field and using + `--use-mocks` for read-only planning/description (reserve the YQ `// "default"` + pattern for real dependencies that simply have not deployed yet). 4. **Convert `generate` blocks.** Backend and provider generation need no configuration at all; anything else becomes a `generate:` stack section. 5. **Validate** with `atmos validate stacks`, then compare `atmos list affected` diff --git a/docs/prd/dag-concurrent-execution.md b/docs/prd/dag-concurrent-execution.md index a8ca7e64fdd..969792cbe12 100644 --- a/docs/prd/dag-concurrent-execution.md +++ b/docs/prd/dag-concurrent-execution.md @@ -254,7 +254,7 @@ The I/O package provides the stream isolation primitives that per-node output wi - Merges environment: system + `atmos.yaml` global env + command-specific env - Preserves exit codes: `exec.ExitError` β†’ `errUtils.ExitCodeError` - Propagates TTY: injects `ATMOS_FORCE_TTY=true` when parent has TTY -- `terraform_plan_diff.go` swaps global `os.Stdout` to capture output β€” race condition under concurrency +- `terraform_plan_diff.go` no longer swaps global `os.Stdout` to capture output β€” it now captures via a local `bytes.Buffer` (see Phase 1, shipped), eliminating the concurrency race this section originally flagged ### Routing Gap (resolved in Phase 3) - `--all` for Terraform goes through `ExecuteTerraformAll()` (dependency-aware, `--max-concurrency` controls parallelism) @@ -902,7 +902,7 @@ Notable details from PR #2159 that should be preserved: - **No new exec files**: Adapters and orchestrator live in `pkg/scheduler/` and `pkg/scheduler/adapters/`, NOT in `internal/exec/`. The long-term goal is to eliminate `internal/exec/`. - **`pkg/process/` vs `pkg/runner/`**: `pkg/process/` is subprocess-level (spawn, streams, signals, exit codes). `pkg/runner/` is task-level (shell tasks, atmos sub-commands). Different abstraction levels, complementary. - **Default concurrency**: `1` (sequential, backward-compatible), configurable via `atmos.yaml`, ENV, or CLI flag -- **Cross-type dependency syntax**: Solved by PR #2193 β€” new `dependencies.components` format with `kind` field for cross-type dependencies (terraform/helmfile/packer/plugin). The scheduler consumes this format via the graph builder. +- **Cross-type dependency syntax**: Partially solved. PR #2193 added the `dependencies.components` `kind` field (terraform/helmfile/packer/plugin) at the schema level, and it's normalized/validated today. The Terraform scheduler adapter does not yet consume it, though: `addTerraformDependencies()` (`pkg/scheduler/adapters/terraform.go`) explicitly skips any dependency whose `kind` isn't `terraform` (`if dep.Kind != "" && dep.Kind != cfg.TerraformComponentType { continue }`), so a declared `kind: packer` edge is parsed but never added to the DAG the scheduler executes. Real cross-type scheduling requires the Phase 3 items still open below (`PackerAdapter`, `AnsibleAdapter`, graph building for non-Terraform nodes). --- diff --git a/website/docs/migration/terragrunt.mdx b/website/docs/migration/terragrunt.mdx index 9e3d0165d61..05bcee38fd8 100644 --- a/website/docs/migration/terragrunt.mdx +++ b/website/docs/migration/terragrunt.mdx @@ -854,8 +854,8 @@ See [YAML Functions](/functions/yaml) for the complete reference. **GitOps-native:** Atmos goes beyond simple "run all" with intelligent change detection: ```bash - # See what changed between commits (compares current branch to main; table output for humans) - atmos list affected + # See what changed between commits (table output for humans) + atmos list affected --ref main # Plan ONLY components affected by changes in current branch atmos terraform plan --affected From 52df4de13788c158ffb9dcde0504383720b30df2 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 14:13:52 -0500 Subject: [PATCH 06/18] chore: trigger CI re-run From 6c2250349d25ad86528adb802a765b9f38fc7c81 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 17:29:04 -0500 Subject: [PATCH 07/18] fix(mocks): correct provenance rendering, error wording, and // default parity A field test of --use-mocks found `describe component` silently rendering empty output whenever a component's provenance path wasn't matched due to an unnormalized lookup, a mock-output error that mislabeled the output name as a component name, and a YQ `//` default that only rescued a missing key inside a declared `mocks` map, not a component with no `mocks` section at all -- inconsistent with how `//` already rescues real state. Also cross-references the mocks:/--use-mocks feature from the docs pages and skill most likely to be read first. Co-Authored-By: Claude Sonnet 5 --- .../skills/atmos-yaml-functions/SKILL.md | 29 +++++++++ errors/errors.go | 2 +- internal/exec/terraform_mocks.go | 11 +++- internal/exec/terraform_mocks_test.go | 23 ++++++++ pkg/provenance/data_transform.go | 44 +++++++------- pkg/provenance/data_transform_test.go | 59 +++++++++++++++++++ pkg/provenance/tree_renderer.go | 7 +-- .../2026-07-15-terraform-component-mocks.mdx | 2 +- .../components/terraform/stack-config.mdx | 18 ++++++ .../docs/functions/yaml/terraform.output.mdx | 9 +++ .../docs/functions/yaml/terraform.state.mdx | 9 +++ website/docs/migration/terragrunt.mdx | 2 +- 12 files changed, 185 insertions(+), 30 deletions(-) diff --git a/agent-skills/skills/atmos-yaml-functions/SKILL.md b/agent-skills/skills/atmos-yaml-functions/SKILL.md index 7a98a063481..b2ff41a2e79 100644 --- a/agent-skills/skills/atmos-yaml-functions/SKILL.md +++ b/agent-skills/skills/atmos-yaml-functions/SKILL.md @@ -102,6 +102,35 @@ vars: The deployed upstream output supersedes the fallback automatically. Dependency metadata controls deployment order; it does not create state before an aggregate plan. +### Reusable Mocks vs. a One-Off `//` Default + +The `//` default above is a per-expression fallback. For a producer component's mock outputs to be +declared once and resolved consistently by every consumer, use the component's `mocks:` stack-config +section together with `--use-mocks` (supported by `atmos terraform plan` and +`atmos describe component`) instead of repeating a `//` default in every consuming expression: + +```yaml +components: + terraform: + vpc: + mocks: + vpc_id: vpc-mock1234 + private_subnet_ids: [subnet-a, subnet-b] + app: + vars: + vpc_id: !terraform.state vpc vpc_id +``` + +```shell +atmos terraform plan app -s dev --use-mocks +``` + +`mocks:` is Terraform-only, never templated or YAML-function-processed, and (unlike a `//` +default) requires the referenced component to declare `mocks:` -- with one exception: a `//` +default in the caller's expression is still honored even when the referenced component declares +no `mocks:` section at all, mirroring how a `//` default rescues a component with no real state. +Do not conflate this feature with the `//`-default idiom above; they're separate mechanisms. + ## `!terraform.output` -- Remote State Access Reads Terraform outputs by running `terraform output`. Requires Terraform initialization diff --git a/errors/errors.go b/errors/errors.go index ff5ebf6b56e..ae71711b1a0 100644 --- a/errors/errors.go +++ b/errors/errors.go @@ -215,7 +215,7 @@ var ( // --use-mocks errors. ErrTerraformComponentMocksNotDeclared = errors.New("terraform component does not declare `mocks` required by --use-mocks") - ErrTerraformMockOutputNotDeclared = errors.New("mocked terraform output is not declared for component") + ErrTerraformMockOutputNotDeclared = errors.New("mocked terraform output is not declared") // API/infrastructure errors - should cause non-zero exit. // These errors indicate backend API failures that should not use YQ defaults. diff --git a/internal/exec/terraform_mocks.go b/internal/exec/terraform_mocks.go index f9fed356aa8..b2050ce6045 100644 --- a/internal/exec/terraform_mocks.go +++ b/internal/exec/terraform_mocks.go @@ -50,7 +50,14 @@ func resolveTerraformMockOutput( mocks, ok := componentSection[cfg.MocksSectionName].(map[string]any) if !ok || mocks == nil { - return nil, true, fmt.Errorf("%w: component %q in stack %q", errUtils.ErrTerraformComponentMocksNotDeclared, component, stack) + // A `//` default in the caller's expression is honored the same way + // whether or not the component declares mocks, mirroring how a `//` + // default rescues a component with no real state at all. Without a + // default, an undeclared `mocks` map is a hard error. + if !hasYqDefault(output) { + return nil, true, fmt.Errorf("%w: component %q in stack %q", errUtils.ErrTerraformComponentMocksNotDeclared, component, stack) + } + mocks = map[string]any{} } value, err := tb.GetTerraformBackendVariable(atmosConfig, mocks, output) @@ -58,7 +65,7 @@ func resolveTerraformMockOutput( return nil, true, fmt.Errorf("failed to resolve mocked Terraform output %q for component %q in stack %q: %w", output, component, stack, err) } if value == nil && !hasYqDefault(output) && !mockOutputExists(mocks, output) { - return nil, true, fmt.Errorf("%w %q for component %q in stack %q", errUtils.ErrTerraformMockOutputNotDeclared, output, component, stack) + return nil, true, fmt.Errorf("%w: %q for component %q in stack %q", errUtils.ErrTerraformMockOutputNotDeclared, output, component, stack) } log.Debug( diff --git a/internal/exec/terraform_mocks_test.go b/internal/exec/terraform_mocks_test.go index 21f1686b66f..feb995e8318 100644 --- a/internal/exec/terraform_mocks_test.go +++ b/internal/exec/terraform_mocks_test.go @@ -75,3 +75,26 @@ func TestTerraformComponentMocksFailClosed(t *testing.T) { require.Error(t, err) assert.Contains(t, err.Error(), "is not declared") } + +// TestTerraformComponentMocksYqDefaultDoesNotRequireMocks verifies that a YQ `//` +// default in the caller's expression is honored even when the referenced +// component declares no `mocks` section at all, mirroring how a `//` default +// already rescues a component with no real state. Without a default, an +// undeclared `mocks` map must still hard-error (see TestTerraformComponentMocksFailClosed). +func TestTerraformComponentMocksYqDefaultDoesNotRequireMocks(t *testing.T) { + sandbox, err := testhelpers.SetupSandbox(t, "../../tests/fixtures/scenarios/terraform-component-mocks") + require.NoError(t, err) + t.Cleanup(sandbox.Cleanup) + t.Chdir(sandbox.OriginalWorkdir) + for key, value := range sandbox.GetEnvironmentVariables() { + t.Setenv(key, value) + } + + atmosConfig, err := cfg.InitCliConfig(schema.ConfigAndStacksInfo{Stack: "dev"}, true) + require.NoError(t, err) + + // "app" declares no `mocks` section at all. + value, err := processTagTerraformState(&atmosConfig, `!terraform.state app .missing // "yq-fallback"`, "dev", &schema.ConfigAndStacksInfo{UseMocks: true}) + require.NoError(t, err) + assert.Equal(t, "yq-fallback", value) +} diff --git a/pkg/provenance/data_transform.go b/pkg/provenance/data_transform.go index 9082d9cac2f..6dc45b2faff 100644 --- a/pkg/provenance/data_transform.go +++ b/pkg/provenance/data_transform.go @@ -2,6 +2,7 @@ package provenance import ( "fmt" + "strings" m "github.com/cloudposse/atmos/pkg/merge" "github.com/cloudposse/atmos/pkg/perf" @@ -79,30 +80,31 @@ func filterEmptySections(data any, ctx *m.MergeContext) any { filtered := make(map[string]any) for key, value := range dataMap { - // Check if this key or any of its array elements have provenance. - // When ctx is nil (provenance tracking disabled), keep all keys. - hasProvenance := ctx == nil - if ctx != nil { - hasProvenance = ctx.HasProvenance(key) - - // If no direct provenance, check for array element provenance. - if !hasProvenance { - // Check up to maxArrayCheckLimit array elements (reasonable limit). - for i := 0; i < maxArrayCheckLimit; i++ { - arrayPath := fmt.Sprintf("%s[%d]", key, i) - if ctx.HasProvenance(arrayPath) { - hasProvenance = true - break - } - } - } - } - - // Keep if has provenance. - if hasProvenance { + if hasSectionProvenance(ctx, key) { filtered[key] = value } } return filtered } + +// hasSectionProvenance reports whether any recorded provenance path belongs to +// the given top-level section key, once normalized the same way findProvenance +// does (stripping the "components..." prefix). Recorded paths +// are always prefixed that way (e.g. "components.terraform.app.vars.vpc_id"), +// so a raw, unnormalized key like "vars" would never match without this. +// When ctx is nil (provenance tracking disabled), every key is kept. +func hasSectionProvenance(ctx *m.MergeContext, key string) bool { + if ctx == nil { + return true + } + + for _, storedPath := range ctx.GetProvenancePaths() { + normalized := normalizeProvenancePath(storedPath) + if normalized == key || strings.HasPrefix(normalized, key+pathSeparator) || strings.HasPrefix(normalized, key+"[") { + return true + } + } + + return false +} diff --git a/pkg/provenance/data_transform_test.go b/pkg/provenance/data_transform_test.go index e1274367f52..11872e9df14 100644 --- a/pkg/provenance/data_transform_test.go +++ b/pkg/provenance/data_transform_test.go @@ -64,3 +64,62 @@ func TestUpdateImportsProvenancePreservesChain(t *testing.T) { afterEntries[1].File, afterEntries[1].Depth) } } + +// TestFilterEmptySectionsKeepsComponentSections is a regression test for a bug +// where every top-level component section (vars, metadata, settings, and +// others) was silently dropped from `describe component` output whenever the +// stack manifest had no identically-named stack-root section. Provenance for +// a component's own fields is always recorded under a +// "components..." prefix (e.g. +// "components.terraform.app.vars.foo"), never under the bare section name +// ("vars") -- filterEmptySections must normalize before comparing, the same +// way findProvenance does, or it drops every section it should keep. +func TestFilterEmptySectionsKeepsComponentSections(t *testing.T) { + ctx := m.NewMergeContext() + ctx.EnableProvenance() + ctx.RecordProvenance("components.terraform.app.vars.foo", m.ProvenanceEntry{ + File: "dev.yaml", Line: 5, Type: m.ProvenanceTypeInline, Depth: 1, + }) + + data := map[string]any{ + "vars": map[string]any{"foo": "bar"}, + "backend": map[string]any{}, + "metadata": map[string]any{}, + } + + filtered := filterEmptySections(data, ctx) + filteredMap, ok := filtered.(map[string]any) + if !ok { + t.Fatalf("expected filterEmptySections to return a map, got %T", filtered) + } + + if _, ok := filteredMap["vars"]; !ok { + t.Errorf("expected 'vars' to survive filtering (has provenance under a prefixed path), but it was dropped: %v", filteredMap) + } + if _, ok := filteredMap["backend"]; ok { + t.Errorf("expected empty 'backend' (no provenance recorded) to be filtered out, but it survived: %v", filteredMap) + } + if _, ok := filteredMap["metadata"]; ok { + t.Errorf("expected empty 'metadata' (no provenance recorded) to be filtered out, but it survived: %v", filteredMap) + } +} + +// TestFilterEmptySectionsNilContextKeepsEverything verifies the documented +// behavior that a nil MergeContext (provenance tracking disabled) keeps every +// top-level key rather than filtering anything. +func TestFilterEmptySectionsNilContextKeepsEverything(t *testing.T) { + data := map[string]any{ + "vars": map[string]any{"foo": "bar"}, + "backend": map[string]any{}, + } + + filtered := filterEmptySections(data, nil) + filteredMap, ok := filtered.(map[string]any) + if !ok { + t.Fatalf("expected filterEmptySections to return a map, got %T", filtered) + } + + if len(filteredMap) != len(data) { + t.Errorf("expected all %d keys to survive with a nil context, got %d: %v", len(data), len(filteredMap), filteredMap) + } +} diff --git a/pkg/provenance/tree_renderer.go b/pkg/provenance/tree_renderer.go index a4170d60ac8..ccf4b120290 100644 --- a/pkg/provenance/tree_renderer.go +++ b/pkg/provenance/tree_renderer.go @@ -25,10 +25,9 @@ const ( SymbolComputed = "∴" // Rendering constants. - defaultSeparatorWidth = 60 // Width of separator lines - commentSpaceNeeded = 60 // Space needed for provenance comments - maxLineLength = 10 // Buffer subtracted from comment column - maxArrayCheckLimit = 1000 // Maximum array elements to check for provenance + defaultSeparatorWidth = 60 // Width of separator lines + commentSpaceNeeded = 60 // Space needed for provenance comments + maxLineLength = 10 // Buffer subtracted from comment column // String constants used repeatedly. pathSeparator = "." diff --git a/website/blog/2026-07-15-terraform-component-mocks.mdx b/website/blog/2026-07-15-terraform-component-mocks.mdx index 2803eca8e32..4d3e56c080b 100644 --- a/website/blog/2026-07-15-terraform-component-mocks.mdx +++ b/website/blog/2026-07-15-terraform-component-mocks.mdx @@ -64,7 +64,7 @@ The [advanced quick start](/quick-start/advanced/) uses this pattern for its fir When a Terraform lookup is mocked, Atmos loads the referenced component's merged `mocks` map and evaluates the requested expression against it. It does **not** initialize Terraform, resolve the referenced component's credentials, read a backend, or populate the real-state lookup cache. -That makes mock mode useful for local plans and `describe component` output that should be independent of the remote dependency. It also means `--use-mocks` never silently falls back to real state: a missing mock map or output is an actionable error, not a surprise backend read. +That makes mock mode useful for local plans and `describe component` output that should be independent of the remote dependency. It also means `--use-mocks` never silently falls back to real state: a missing mock map or output is an actionable error, not a surprise backend read β€” unless the expression itself supplies a YQ `//` default, which is honored the same way it is for real state, whether or not the referenced component declares `mocks` at all. Mock values themselves are literal. Atmos does not evaluate templates or YAML functions inside `mocks`, so a mock cannot accidentally call the real dependency it was meant to replace. diff --git a/website/docs/components/terraform/stack-config.mdx b/website/docs/components/terraform/stack-config.mdx index a4dc87202ea..0b970bb5318 100644 --- a/website/docs/components/terraform/stack-config.mdx +++ b/website/docs/components/terraform/stack-config.mdx @@ -109,6 +109,24 @@ components: AWS_PROFILE: production ``` + +
`mocks` (optional)
+
+ A free-form map of literal Terraform output values this component provides for other components to + consume, without requiring `terraform init`, a real backend, or cloud credentials. `mocks` values are + used only when a consuming component reads them via [`!terraform.state`](/functions/yaml/terraform.state) + or [`!terraform.output`](/functions/yaml/terraform.output) and the reader is run with `--use-mocks` + (supported by `atmos terraform plan` and `atmos describe component`). Values are always literal β€” Atmos + never evaluates templates or YAML functions inside `mocks`. See + [Component Mocks for Terraform YAML Lookups](/changelog/terraform-component-mocks). + + **Example:** + ```yaml + mocks: + vpc_id: vpc-mock1234 + private_subnet_ids: [subnet-a, subnet-b] + ``` +
## Context Variables diff --git a/website/docs/functions/yaml/terraform.output.mdx b/website/docs/functions/yaml/terraform.output.mdx index be82b5b6501..454a00f3892 100644 --- a/website/docs/functions/yaml/terraform.output.mdx +++ b/website/docs/functions/yaml/terraform.output.mdx @@ -176,6 +176,15 @@ in the YQ expression using the `//` operator. Atmos will evaluate the default wh This allows you to mock outputs when executing `atmos terraform plan` where there are dependencies between components, and the dependent components are not provisioned yet. +:::tip Looking for reusable, component-owned mock fixtures? +A `//` default is a one-off fallback baked into a single expression. If you want a producer +component to declare its mock outputs once and have every consumer resolve them consistently, +use the dedicated [`mocks`](/components/terraform/stack-config#mocks) stack-config section with the +`--use-mocks` flag instead β€” see [Component Mocks for Terraform YAML Lookups](/changelog/terraform-component-mocks). +A `//` default in the caller's expression still applies even when the referenced component +declares no `mocks` section at all; without one, an undeclared `mocks` section is a hard error. +::: + :::tip Default Value Behavior Atmos distinguishes between **recoverable errors** (component not provisioned, output missing) and **non-recoverable errors** (API failures, network timeouts): diff --git a/website/docs/functions/yaml/terraform.state.mdx b/website/docs/functions/yaml/terraform.state.mdx index f89c754ba69..f9c06a80c36 100644 --- a/website/docs/functions/yaml/terraform.state.mdx +++ b/website/docs/functions/yaml/terraform.state.mdx @@ -171,6 +171,15 @@ in the YQ expression using the `//` operator. Atmos will evaluate the default wh This allows you to mock outputs when executing `atmos terraform plan` where there are dependencies between components, and the dependent components are not provisioned yet. +:::tip Looking for reusable, component-owned mock fixtures? +A `//` default is a one-off fallback baked into a single expression. If you want a producer +component to declare its mock outputs once and have every consumer resolve them consistently, +use the dedicated [`mocks`](/components/terraform/stack-config#mocks) stack-config section with the +`--use-mocks` flag instead β€” see [Component Mocks for Terraform YAML Lookups](/changelog/terraform-component-mocks). +A `//` default in the caller's expression still applies even when the referenced component +declares no `mocks` section at all; without one, an undeclared `mocks` section is a hard error. +::: + :::tip Default Value Behavior Atmos distinguishes between **recoverable errors** (component not provisioned, output missing) and **non-recoverable errors** (backend API failures): diff --git a/website/docs/migration/terragrunt.mdx b/website/docs/migration/terragrunt.mdx index 05bcee38fd8..3a41b4b5e3b 100644 --- a/website/docs/migration/terragrunt.mdx +++ b/website/docs/migration/terragrunt.mdx @@ -248,7 +248,7 @@ If you're familiar with Terragrunt, the concepts below will help you translate w atmos describe component eks -s prod --use-mocks ``` - `--use-mocks` is rejected on `apply`, `deploy`, and `destroy` before stack resolution even happens, so a mock value can never reach a mutating Terraform operation β€” a stronger guarantee than opting individual commands in through `mock_outputs_allowed_terraform_commands`. An undeclared `mocks` entry is also a hard error rather than a silent fallback. See the provider-free `examples/terraform-component-mocks` example in the Atmos repository for a complete walkthrough. + `--use-mocks` is rejected on `apply`, `deploy`, and `destroy` before stack resolution even happens, so a mock value can never reach a mutating Terraform operation β€” a stronger guarantee than opting individual commands in through `mock_outputs_allowed_terraform_commands`. An undeclared `mocks` entry is also a hard error rather than a silent fallback, unless the expression supplies a YQ `//` default β€” which is honored the same way it is for real state, whether or not the referenced component declares `mocks` at all. See the provider-free `examples/terraform-component-mocks` example in the Atmos repository for a complete walkthrough. From d06ce50358349cd051a69987b41d52aadcda141c Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 18:55:57 -0500 Subject: [PATCH 08/18] test(snapshots): regenerate describe_component golden snapshots after provenance fix The filterEmptySections fix (6c22503) corrected describe_component to stop silently dropping real sections (backend, metadata, env, overrides) that lack a stack-root section of the same name. CI caught the resulting golden snapshot drift on both linux and macos; regenerated via `-regenerate-snapshots` per CLAUDE.md, verified the diffs only add the previously-hidden, now-correct content. Co-Authored-By: Claude Sonnet 5 --- ...component_help_shows_stack_flag.stdout.golden | 1 - ...e_component_provenance_advanced.stdout.golden | 3 ++- ...ribe_component_provenance_basic.stdout.golden | 3 ++- ...t_name_(backward_compatibility).stdout.golden | 16 ++++++++++++++++ ...nent_with_current_directory_(.).stdout.golden | 16 ++++++++++++++++ ...onent_with_provenance_and_stack.stdout.golden | 3 ++- ...be_component_with_relative_path.stdout.golden | 16 ++++++++++++++++ 7 files changed, 54 insertions(+), 4 deletions(-) diff --git a/tests/snapshots/TestCLICommands_describe_component_help_shows_stack_flag.stdout.golden b/tests/snapshots/TestCLICommands_describe_component_help_shows_stack_flag.stdout.golden index d119873b74a..e45a484e818 100644 --- a/tests/snapshots/TestCLICommands_describe_component_help_shows_stack_flag.stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_help_shows_stack_flag.stdout.golden @@ -153,4 +153,3 @@ GLOBAL FLAGS -v, --verbose Enable verbose error output with full context, stack traces, and detailed information - diff --git a/tests/snapshots/TestCLICommands_describe_component_provenance_advanced.stdout.golden b/tests/snapshots/TestCLICommands_describe_component_provenance_advanced.stdout.golden index af0e5818a7c..57bff43f8c6 100644 --- a/tests/snapshots/TestCLICommands_describe_component_provenance_advanced.stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_provenance_advanced.stdout.golden @@ -9,6 +9,8 @@ import: # β—‹ [3] mixins/stage/dev.yam - catalog/mock/defaults # β—‹ [3] mixins/stage/dev.yaml:3 - mixins/stage/dev # β—‹ [2] orgs/acme/_defaults.yaml:3 - orgs/acme/_defaults # ● [1] orgs/acme/dev/us-east-1.yaml:3 +metadata: # β—‹ [4] catalog/mock/defaults.yaml:6 + component: mock # β—‹ [4] catalog/mock/defaults.yaml:6 vars: # β—‹ [4] catalog/mock/defaults.yaml:8 enabled: true # β—‹ [4] catalog/mock/defaults.yaml:8 environment: development # β—‹ [3] mixins/stage/dev.yaml:13 @@ -22,4 +24,3 @@ vars: # β—‹ [4] catalog/mock/default stage: dev # β—‹ [3] mixins/stage/dev.yaml:15 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:14 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:6 - diff --git a/tests/snapshots/TestCLICommands_describe_component_provenance_basic.stdout.golden b/tests/snapshots/TestCLICommands_describe_component_provenance_basic.stdout.golden index af0e5818a7c..57bff43f8c6 100644 --- a/tests/snapshots/TestCLICommands_describe_component_provenance_basic.stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_provenance_basic.stdout.golden @@ -9,6 +9,8 @@ import: # β—‹ [3] mixins/stage/dev.yam - catalog/mock/defaults # β—‹ [3] mixins/stage/dev.yaml:3 - mixins/stage/dev # β—‹ [2] orgs/acme/_defaults.yaml:3 - orgs/acme/_defaults # ● [1] orgs/acme/dev/us-east-1.yaml:3 +metadata: # β—‹ [4] catalog/mock/defaults.yaml:6 + component: mock # β—‹ [4] catalog/mock/defaults.yaml:6 vars: # β—‹ [4] catalog/mock/defaults.yaml:8 enabled: true # β—‹ [4] catalog/mock/defaults.yaml:8 environment: development # β—‹ [3] mixins/stage/dev.yaml:13 @@ -22,4 +24,3 @@ vars: # β—‹ [4] catalog/mock/default stage: dev # β—‹ [3] mixins/stage/dev.yaml:15 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:14 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:6 - diff --git a/tests/snapshots/TestCLICommands_describe_component_with_component_name_(backward_compatibility).stdout.golden b/tests/snapshots/TestCLICommands_describe_component_with_component_name_(backward_compatibility).stdout.golden index 4ca830f651b..7f1119b8870 100644 --- a/tests/snapshots/TestCLICommands_describe_component_with_component_name_(backward_compatibility).stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_with_component_name_(backward_compatibility).stdout.golden @@ -5,6 +5,18 @@ # Stack: orgs/cp/tenant1/dev/us-east-2 +backend: # β—‹ [2] catalog/terraform/vpc.yaml:9 + acl: bucket-owner-full-control + bucket: cp-ue2-root-tfstate + dynamodb_table: cp-ue2-root-tfstate-lock + encrypt: true + key: terraform.tfstate + profile: cp-gbl-root-tfstate + region: us-east-2 + role_arn: null + workspace_key_prefix: top-level-component1 +component: top-level-component1 # β—‹ [2] catalog/terraform/test-component-override-2.yaml:29 +env: {} # β—‹ [2] catalog/terraform/test-component-override-2.yaml:34 import: # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/echo-server # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/infra-server # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 @@ -43,6 +55,10 @@ import: # β—‹ [3] orgs/cp/tenant1/_def - orgs/cp/_defaults # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - orgs/cp/tenant1/_defaults # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 - orgs/cp/tenant1/dev/_defaults # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml:5 +metadata: {} # β—‹ [2] catalog/terraform/vpc.yaml:7 +overrides: # β—‹ [2] catalog/terraform/vpc.yaml + providers: # β—‹ [2] catalog/terraform/vpc.yaml + context: {} # β—‹ [2] catalog/terraform/vpc.yaml providers: # β—‹ [2] catalog/terraform/vpc.yaml:52 context: # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml delimiter: '-' # β—‹ [4] orgs/cp/_defaults.yaml:64 diff --git a/tests/snapshots/TestCLICommands_describe_component_with_current_directory_(.).stdout.golden b/tests/snapshots/TestCLICommands_describe_component_with_current_directory_(.).stdout.golden index 4ca830f651b..7f1119b8870 100644 --- a/tests/snapshots/TestCLICommands_describe_component_with_current_directory_(.).stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_with_current_directory_(.).stdout.golden @@ -5,6 +5,18 @@ # Stack: orgs/cp/tenant1/dev/us-east-2 +backend: # β—‹ [2] catalog/terraform/vpc.yaml:9 + acl: bucket-owner-full-control + bucket: cp-ue2-root-tfstate + dynamodb_table: cp-ue2-root-tfstate-lock + encrypt: true + key: terraform.tfstate + profile: cp-gbl-root-tfstate + region: us-east-2 + role_arn: null + workspace_key_prefix: top-level-component1 +component: top-level-component1 # β—‹ [2] catalog/terraform/test-component-override-2.yaml:29 +env: {} # β—‹ [2] catalog/terraform/test-component-override-2.yaml:34 import: # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/echo-server # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/infra-server # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 @@ -43,6 +55,10 @@ import: # β—‹ [3] orgs/cp/tenant1/_def - orgs/cp/_defaults # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - orgs/cp/tenant1/_defaults # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 - orgs/cp/tenant1/dev/_defaults # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml:5 +metadata: {} # β—‹ [2] catalog/terraform/vpc.yaml:7 +overrides: # β—‹ [2] catalog/terraform/vpc.yaml + providers: # β—‹ [2] catalog/terraform/vpc.yaml + context: {} # β—‹ [2] catalog/terraform/vpc.yaml providers: # β—‹ [2] catalog/terraform/vpc.yaml:52 context: # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml delimiter: '-' # β—‹ [4] orgs/cp/_defaults.yaml:64 diff --git a/tests/snapshots/TestCLICommands_describe_component_with_provenance_and_stack.stdout.golden b/tests/snapshots/TestCLICommands_describe_component_with_provenance_and_stack.stdout.golden index af0e5818a7c..57bff43f8c6 100644 --- a/tests/snapshots/TestCLICommands_describe_component_with_provenance_and_stack.stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_with_provenance_and_stack.stdout.golden @@ -9,6 +9,8 @@ import: # β—‹ [3] mixins/stage/dev.yam - catalog/mock/defaults # β—‹ [3] mixins/stage/dev.yaml:3 - mixins/stage/dev # β—‹ [2] orgs/acme/_defaults.yaml:3 - orgs/acme/_defaults # ● [1] orgs/acme/dev/us-east-1.yaml:3 +metadata: # β—‹ [4] catalog/mock/defaults.yaml:6 + component: mock # β—‹ [4] catalog/mock/defaults.yaml:6 vars: # β—‹ [4] catalog/mock/defaults.yaml:8 enabled: true # β—‹ [4] catalog/mock/defaults.yaml:8 environment: development # β—‹ [3] mixins/stage/dev.yaml:13 @@ -22,4 +24,3 @@ vars: # β—‹ [4] catalog/mock/default stage: dev # β—‹ [3] mixins/stage/dev.yaml:15 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:14 tenant: acme # β—‹ [2] orgs/acme/_defaults.yaml:6 - diff --git a/tests/snapshots/TestCLICommands_describe_component_with_relative_path.stdout.golden b/tests/snapshots/TestCLICommands_describe_component_with_relative_path.stdout.golden index 4ca830f651b..7f1119b8870 100644 --- a/tests/snapshots/TestCLICommands_describe_component_with_relative_path.stdout.golden +++ b/tests/snapshots/TestCLICommands_describe_component_with_relative_path.stdout.golden @@ -5,6 +5,18 @@ # Stack: orgs/cp/tenant1/dev/us-east-2 +backend: # β—‹ [2] catalog/terraform/vpc.yaml:9 + acl: bucket-owner-full-control + bucket: cp-ue2-root-tfstate + dynamodb_table: cp-ue2-root-tfstate-lock + encrypt: true + key: terraform.tfstate + profile: cp-gbl-root-tfstate + region: us-east-2 + role_arn: null + workspace_key_prefix: top-level-component1 +component: top-level-component1 # β—‹ [2] catalog/terraform/test-component-override-2.yaml:29 +env: {} # β—‹ [2] catalog/terraform/test-component-override-2.yaml:34 import: # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/echo-server # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - catalog/helmfile/infra-server # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 @@ -43,6 +55,10 @@ import: # β—‹ [3] orgs/cp/tenant1/_def - orgs/cp/_defaults # β—‹ [3] orgs/cp/tenant1/_defaults.yaml:4 - orgs/cp/tenant1/_defaults # β—‹ [2] orgs/cp/tenant1/dev/_defaults.yaml:5 - orgs/cp/tenant1/dev/_defaults # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml:5 +metadata: {} # β—‹ [2] catalog/terraform/vpc.yaml:7 +overrides: # β—‹ [2] catalog/terraform/vpc.yaml + providers: # β—‹ [2] catalog/terraform/vpc.yaml + context: {} # β—‹ [2] catalog/terraform/vpc.yaml providers: # β—‹ [2] catalog/terraform/vpc.yaml:52 context: # ● [1] orgs/cp/tenant1/dev/us-east-2.yaml delimiter: '-' # β—‹ [4] orgs/cp/_defaults.yaml:64 From 95289ba739934cae71027b89f6359418cf01a9d6 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 6 Aug 2026 19:01:39 -0500 Subject: [PATCH 09/18] fix(provenance): address CodeRabbit review on PR #2878 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add periods to the rendering-constant comments (godot's inline-comment scope missed these, but CLAUDE.md's comment convention still applies), and cover the array-element provenance path (vars[0].foo) alongside the already-tested dot-nested form. The trailing-period finding on ErrTerraformMockOutputNotDeclared was already resolved by an earlier commit in this PR β€” no change needed there. Co-Authored-By: Claude Sonnet 5 --- pkg/provenance/data_transform_test.go | 30 +++++++++++++++++++++++++++ pkg/provenance/tree_renderer.go | 6 +++--- 2 files changed, 33 insertions(+), 3 deletions(-) diff --git a/pkg/provenance/data_transform_test.go b/pkg/provenance/data_transform_test.go index 11872e9df14..0d6abad96e8 100644 --- a/pkg/provenance/data_transform_test.go +++ b/pkg/provenance/data_transform_test.go @@ -104,6 +104,36 @@ func TestFilterEmptySectionsKeepsComponentSections(t *testing.T) { } } +// TestFilterEmptySectionsKeepsArrayElementSections is a regression test for +// the array-element form of the same prefixed-path lookup: a provenance path +// like "components.terraform.app.vars[0].foo" must also be recognized as +// belonging to the "vars" section, not just the dot-nested form. +func TestFilterEmptySectionsKeepsArrayElementSections(t *testing.T) { + ctx := m.NewMergeContext() + ctx.EnableProvenance() + ctx.RecordProvenance("components.terraform.app.vars[0].foo", m.ProvenanceEntry{ + File: "dev.yaml", Line: 5, Type: m.ProvenanceTypeInline, Depth: 1, + }) + + data := map[string]any{ + "vars": []any{map[string]any{"foo": "bar"}}, + "backend": map[string]any{}, + } + + filtered := filterEmptySections(data, ctx) + filteredMap, ok := filtered.(map[string]any) + if !ok { + t.Fatalf("expected filterEmptySections to return a map, got %T", filtered) + } + + if _, ok := filteredMap["vars"]; !ok { + t.Errorf("expected 'vars' to survive filtering (has provenance under an array-element path), but it was dropped: %v", filteredMap) + } + if _, ok := filteredMap["backend"]; ok { + t.Errorf("expected empty 'backend' (no provenance recorded) to be filtered out, but it survived: %v", filteredMap) + } +} + // TestFilterEmptySectionsNilContextKeepsEverything verifies the documented // behavior that a nil MergeContext (provenance tracking disabled) keeps every // top-level key rather than filtering anything. diff --git a/pkg/provenance/tree_renderer.go b/pkg/provenance/tree_renderer.go index ccf4b120290..ffe110da991 100644 --- a/pkg/provenance/tree_renderer.go +++ b/pkg/provenance/tree_renderer.go @@ -25,9 +25,9 @@ const ( SymbolComputed = "∴" // Rendering constants. - defaultSeparatorWidth = 60 // Width of separator lines - commentSpaceNeeded = 60 // Space needed for provenance comments - maxLineLength = 10 // Buffer subtracted from comment column + defaultSeparatorWidth = 60 // Width of separator lines. + commentSpaceNeeded = 60 // Space needed for provenance comments. + maxLineLength = 10 // Buffer subtracted from comment column. // String constants used repeatedly. pathSeparator = "." From 7b23760fc481fb4a76dfc9cda307f1957d2c7890 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Thu, 20 Aug 2026 08:08:25 -0500 Subject: [PATCH 10/18] fix(test): widen RunSession timeouts to fix Windows CI flake MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Acceptance Tests (windows, shard 4/10) failed with ErrWaitTimeout in TestRunSessionExecutesScriptedShellActions and TestRunSessionAppliesDirectoryAndEnvironment: the write->echo->match round trip against a spawned child (no PTY on Windows, unlike session_unix.go) never completed within the 2s wait/3s context budget. Widened both to 8s/15s across all four RunSession-based tests; no production code changed since static review found no concrete pipe-wiring bug. Not reproduced locally (no Windows environment available) β€” documented in docs/fixes/ per this repo's convention for unconfirmed Windows-only CI fixes. Co-Authored-By: Claude Sonnet 5 --- ...asciicast-runsession-windows-ci-timeout.md | 88 +++++++++++++++++++ pkg/asciicast/session_test.go | 23 +++-- 2 files changed, 104 insertions(+), 7 deletions(-) create mode 100644 docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md diff --git a/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md b/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md new file mode 100644 index 00000000000..9378182c0da --- /dev/null +++ b/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md @@ -0,0 +1,88 @@ +# Fix: `TestRunSessionExecutesScriptedShellActions`/`TestRunSessionAppliesDirectoryAndEnvironment` timeout on Windows CI + +**Date:** 2026-08-19 + +## Summary + +`Acceptance Tests (windows, shard 4/10)` failed with: + +``` +--- FAIL: TestRunSessionExecutesScriptedShellActions (2.07s) + session_test.go:821: RunSession error: timed out waiting for cast output +--- FAIL: TestRunSessionAppliesDirectoryAndEnvironment (3.18s) + session_test.go:863: RunSession error: timed out waiting for cast output +``` + +Both tests spawn the test binary itself as a fake interactive shell +(`_ATMOS_ASCIICAST_SESSION_HELPER=1`, handled by `runAsciicastSessionHelper` in +`testmain_test.go`), write scripted input, and wait for the helper's echoed +response. The error is `ErrWaitTimeout` ("timed out waiting for cast output"), +raised by `waitForOutput`'s own per-action `Timeout` field (2s in both +failing tests) β€” not the outer test context β€” meaning the write -> echo -> +match round trip observed *zero* matching output within a full 2-second +window on the Windows runner. + +## Context + +`pkg/asciicast/session_unix.go` spawns the child over a real PTY +(`github.com/creack/pty`); `pkg/asciicast/session_windows.go` has no PTY +equivalent and instead wires the child via plain `cmd.StdinPipe()` + +`io.Pipe()`-backed combined stdout/stderr. Every other `RunSession` test in +this file passed: the two that failed are also the only two that depend on +completing a real write -> echo -> match round trip with the spawned child +(`TestRunSessionDefaultsNilOptions` runs no actions at all; +`TestRunSessionReturnsActionErrors` fails immediately on an unknown action +type before any I/O). That isolates the timing pressure to the round trip +itself, not process spawn/exit alone. + +I could not reproduce this on a Windows machine (no Windows environment +available in this session β€” darwin only, matching the pattern noted in +`docs/fixes/2026-08-07-windows-parallel-shell-child-quoting.md` and +`docs/fixes/2026-08-08-toolchain-live-renderer-windows-ci-deadlock.md`). +Static review of `session_windows.go`'s pipe wiring didn't surface a clear +correctness bug (the reader goroutine starts immediately after `cmd.Start()` +and drains continuously, so there's no obvious deadlock analogous to the +un-drained-pipe bug in the toolchain live-renderer fix above). Windows +process creation (`CreateProcess`) and pipe scheduling are well-documented as +slower than POSIX `fork`/`exec` under load, and this CI run is a newly +10-way-sharded, parallel matrix leg (`ci(test): shard acceptance tests 10-way +per OS to cut CI runtime (#2940)`) β€” plausible enough to explain a 2-second +window occasionally not being enough, but not confirmed as the sole cause. + +## Changes + +`pkg/asciicast/session_test.go`: + +- `TestRunSessionExecutesScriptedShellActions` and + `TestRunSessionAppliesDirectoryAndEnvironment`: outer `context.WithTimeout` + raised from 3s to 15s, and every `wait` action's own `Timeout` raised from + `"2s"` to `"8s"`, giving the round trip real headroom under Windows CI load + without slowing down the happy path (these are ceilings, not fixed + sleeps β€” both tests still complete in under a second locally). +- `TestRunSessionDefaultsNilOptions` and `TestRunSessionReturnsActionErrors`: + outer context also raised 3s -> 15s for consistency, even though they + didn't fail this run, since they exercise the same spawn path under the + same CI conditions. + +No production code (`session.go`, `session_windows.go`, `session_unix.go`) +was changed β€” nothing in this investigation identified a concrete logic bug +to fix there, only a plausibly-too-tight test timeout. + +## Validation + +- `go build ./...` and `GOOS=windows GOARCH=amd64 go build ./pkg/asciicast/...` β€” clean. +- `GOOS=windows GOARCH=amd64 go vet ./pkg/asciicast/...` β€” clean. +- `go test ./pkg/asciicast/... -run 'TestRunSession' -v` and + `go test ./pkg/asciicast/...` (full package) β€” all pass on darwin/arm64. +- `atmos lint --changed` β€” pending. +- **Not reproduced on Windows** (no Windows environment available). If the + next `Acceptance Tests (windows)` run still shows this exact timeout with + the widened budget, the round trip itself needs deeper investigation on a + real Windows runner (e.g. adding diagnostic logging around `readOutput`) + rather than further timeout increases. + +## Follow-ups + +If this recurs even at 8s/15s, treat it as a genuine correctness bug in +`session_windows.go`'s pipe wiring, not a timing issue, and investigate with +Windows-side diagnostics (this session had no way to attach one). diff --git a/pkg/asciicast/session_test.go b/pkg/asciicast/session_test.go index a2add58b13c..411f81fa1a1 100644 --- a/pkg/asciicast/session_test.go +++ b/pkg/asciicast/session_test.go @@ -805,7 +805,12 @@ func TestRunSessionExecutesScriptedShellActions(t *testing.T) { } t.Setenv(asciicastSessionHelperEnv, "1") - ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + // Windows CI spawns a real child process over anonymous pipes (no PTY, + // see session_windows.go) and has been observed to take measurably + // longer than Unix to complete the write->echo->match round trip under + // sharded/parallel CI load, so both the overall context and the "wait" + // action's own timeout need more headroom than a local run would need. + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) defer cancel() err = RunSession(ctx, &SessionOptions{ Shell: shell, @@ -814,7 +819,7 @@ func TestRunSessionExecutesScriptedShellActions(t *testing.T) { Actions: []SessionAction{ {Type: "write", Text: "printf ready", Rate: "0"}, {Type: "key", Key: "enter"}, - {Type: "wait", Text: "ready", Timeout: "2s"}, + {Type: "wait", Text: "ready", Timeout: "8s"}, }, }) if err != nil { @@ -838,7 +843,11 @@ func TestRunSessionAppliesDirectoryAndEnvironment(t *testing.T) { cwdPattern := "cwd=(" + regexp.QuoteMeta(dir) + "|" + regexp.QuoteMeta(expectedDir) + ")" t.Setenv(asciicastSessionHelperEnv, "1") - ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + // See the matching comment in TestRunSessionExecutesScriptedShellActions: + // Windows CI needs more headroom than Unix for the real-process write -> + // echo -> match round trip, both for the overall context and each "wait" + // action's own timeout. + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) defer cancel() err = RunSession(ctx, &SessionOptions{ Shell: shell, @@ -855,8 +864,8 @@ func TestRunSessionAppliesDirectoryAndEnvironment(t *testing.T) { {Type: "pause", Duration: "300ms"}, {Type: "write", Text: "print context", Rate: "0"}, {Type: "key", Key: "enter"}, - {Type: "wait", Regex: cwdPattern, Timeout: "2s"}, - {Type: "wait", Text: "marker=from-session", Timeout: "2s"}, + {Type: "wait", Regex: cwdPattern, Timeout: "8s"}, + {Type: "wait", Text: "marker=from-session", Timeout: "8s"}, }, }) if err != nil { @@ -878,7 +887,7 @@ func TestRunSessionDefaultsNilOptions(t *testing.T) { t.Setenv("SHELL", shell) t.Setenv(asciicastSessionHelperEnv, "1") - ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) defer cancel() if err := RunSession(ctx, nil); err != nil { t.Fatalf("RunSession with nil opts: %v", err) @@ -906,7 +915,7 @@ func TestRunSessionReturnsActionErrors(t *testing.T) { } t.Setenv(asciicastSessionHelperEnv, "1") - ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) defer cancel() err = RunSession(ctx, &SessionOptions{ Shell: shell, From 89385eef617585e1f4a0f467cca4c914a60438b7 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 06:39:00 -0500 Subject: [PATCH 11/18] docs(fixes): address CodeRabbit findings on PR #2878 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add a text language tag to the failure-output fenced block (markdownlint MD040). - Correct the documented timeout values to match what actually shipped after merge-conflict resolution: 10s per wait (not 8s), and 25s outer context for TestRunSessionAppliesDirectoryAndEnvironment's two sequential waits (not 15s) β€” the outer context must exceed the sum of sequential wait timeouts, not just one of them, per waitForOutput's ctx.Done()-vs-deadline-timer race. Co-Authored-By: Claude Sonnet 5 --- ...asciicast-runsession-windows-ci-timeout.md | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md b/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md index 9378182c0da..664fa9e4cdd 100644 --- a/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md +++ b/docs/fixes/2026-08-19-asciicast-runsession-windows-ci-timeout.md @@ -6,7 +6,7 @@ `Acceptance Tests (windows, shard 4/10)` failed with: -``` +```text --- FAIL: TestRunSessionExecutesScriptedShellActions (2.07s) session_test.go:821: RunSession error: timed out waiting for cast output --- FAIL: TestRunSessionAppliesDirectoryAndEnvironment (3.18s) @@ -53,12 +53,18 @@ window occasionally not being enough, but not confirmed as the sole cause. `pkg/asciicast/session_test.go`: -- `TestRunSessionExecutesScriptedShellActions` and - `TestRunSessionAppliesDirectoryAndEnvironment`: outer `context.WithTimeout` - raised from 3s to 15s, and every `wait` action's own `Timeout` raised from - `"2s"` to `"8s"`, giving the round trip real headroom under Windows CI load - without slowing down the happy path (these are ceilings, not fixed - sleeps β€” both tests still complete in under a second locally). +- `TestRunSessionExecutesScriptedShellActions`: outer `context.WithTimeout` + raised from 3s to 15s, and its single `wait` action's own `Timeout` raised + from `"2s"` to `"10s"`, giving the round trip real headroom under Windows CI + load without slowing down the happy path (these are ceilings, not fixed + sleeps β€” it still completes in under a second locally). +- `TestRunSessionAppliesDirectoryAndEnvironment`: outer context raised from 3s + to 25s, and both of its sequential `wait` actions raised from `"2s"` to + `"10s"` each. The outer context has to exceed the *sum* of sequential wait + timeouts (not just one of them) with margin, or it silently caps the second + wait short regardless of what its own `Timeout` says β€” `waitForOutput` races + `ctx.Done()` against the action's own deadline timer, and whichever fires + first wins. - `TestRunSessionDefaultsNilOptions` and `TestRunSessionReturnsActionErrors`: outer context also raised 3s -> 15s for consistency, even though they didn't fail this run, since they exercise the same spawn path under the @@ -83,6 +89,6 @@ to fix there, only a plausibly-too-tight test timeout. ## Follow-ups -If this recurs even at 8s/15s, treat it as a genuine correctness bug in +If this recurs even at these widened budgets, treat it as a genuine correctness bug in `session_windows.go`'s pipe wiring, not a timing issue, and investigate with Windows-side diagnostics (this session had no way to attach one). From 0a8e4e49741b6abc214a315ba35f78e9a1e6ea58 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 07:55:01 -0500 Subject: [PATCH 12/18] docs(fixes): document CI exit-code test failure as a registry network flake Acceptance Tests (linux, shard 9/10) failed TestCLICommands/atmos_exit_code_should_be_same_as_command_exit_code_(2) with "Expected exit code 2, got 1". The real cause was tofu init timing out reaching registry.opentofu.org (context deadline exceeded) before any plan could run -- confirmed the fixture has no registry-mirror config to regress, and the sibling (0)/(1) exit-code cases in the same file passed. No code change: there's nothing in this repo that fixes a transient outage on a public third-party registry, and loosening the exit-code assertion would mask a real CLI exit-code-propagation regression if one ever occurs. Co-Authored-By: Claude Sonnet 5 --- ...ommand-exit-code-registry-network-flake.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 docs/fixes/2026-08-31-exec-command-exit-code-registry-network-flake.md diff --git a/docs/fixes/2026-08-31-exec-command-exit-code-registry-network-flake.md b/docs/fixes/2026-08-31-exec-command-exit-code-registry-network-flake.md new file mode 100644 index 00000000000..02db4808c06 --- /dev/null +++ b/docs/fixes/2026-08-31-exec-command-exit-code-registry-network-flake.md @@ -0,0 +1,80 @@ +# Fix: `atmos exit code should be same as command exit code (2)` failed on a transient registry outage, not a code bug + +**Date:** 2026-08-31 + +## Summary + +`Acceptance Tests (linux, shard 9/10)` failed with: + +```text +--- FAIL: TestCLICommands/atmos_exit_code_should_be_same_as_command_exit_code_(2) (13.04s) + cli_test.go:1569: Reason: Expected exit code 2, got 1 + cli_test.go:1363: Description: Ensure the exit code equals the command exit code for a passing terraform plan (expected to be 2) +``` + +Captured stderr showed the real cause was never in Atmos: + +```text +Error: Failed to resolve provider packages + +Could not resolve provider hashicorp/null: could not connect to +registry.opentofu.org: failed to request discovery document: Get +"https://registry.opentofu.org/.well-known/terraform.json": context +deadline exceeded +``` + +## Context + +`tests/test-cases/exec-command.yaml`'s `(2)` case runs a real, uncached +`atmos terraform plan component1 -s test -- -detailed-exitcode` against +`tests/fixtures/scenarios/exitCode` and expects OpenTofu's own +`-detailed-exitcode` exit code of `2` (diff present). That fixture's +`atmos.yaml` has no provider-mirror/registry-cache configuration, so `tofu +init` resolves `hashicorp/null` directly from `registry.opentofu.org` on +every run -- by design for this minimal fixture, not a gap introduced by +this branch. + +On this run, the discovery-document request to `registry.opentofu.org` +itself timed out (`context deadline exceeded`) before `tofu init` could even +start downloading the provider. `tofu` correctly failed with a generic exit +code `1`, and Atmos correctly propagated that exit code -- the test's +expectation of `2` assumes `init` succeeds and only the plan diff drives the +exit code, which is a reasonable assumption that a registry outage breaks +regardless of anything in this codebase. + +Checked and ruled out as codebase causes: +- No provider-mirror/network-mirror config exists anywhere in + `.github/workflows/` or the `exitCode` fixture that this run could have + regressed. +- Atmos's own `terraform cache`/registry-mirror feature (see + `docs/fixes/2026-08-25-provider-mirror-concurrency-test-flake.md` for a + prior, code-fixable flake in that subsystem) is opt-in and not wired into + this fixture at all -- this test has always talked to the real registry. +- No test-framework-level retry mechanism exists for network-dependent CLI + acceptance tests in `tests/cli_test.go` (the only `retry` support in + `tests/test-cases/*.yaml` is Atmos's own workflow-retry *feature* under + test, unrelated to the test harness retrying itself). + +## Changes + +None. There is no code, test, or configuration change in this repository +that fixes a transient DNS/network failure reaching a public, +third-party registry from a CI runner. Making a speculative change here +(e.g., loosening the exit-code assertion) would mask a real regression in +the CLI's exit-code propagation if one ever occurs, which is exactly the +property this test exists to catch. + +## Validation + +- Confirmed via the fixture's `atmos.yaml` and a repo-wide grep that no + registry-mirror/cache config applies to this test, ruling out a + configuration regression. +- Confirmed no other subtest in the same shard's run failed from the same + cause (the sibling `(0)` and `(1)` exit-code cases in the same file + passed), consistent with an isolated, transient outage rather than a + systemic issue. + +## Follow-ups + +None. Re-running the failed CI job is expected to pass; no issue is being +opened since there is no actionable code change to track. From fc068755e4527786da9b04b7ef0529891601315a Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 08:56:10 -0500 Subject: [PATCH 13/18] fix(ci): don't fail test-required/k3s-required on a cancelled run All five attached failure logs (Acceptance Tests linux/macos/windows, [k3s] demo-helmfile, Build windows) traced to one event: workflow run 33394180592 on this PR was cancelled (confirmed via gh api), not failed. The test/k3s matrix jobs were skipped as a result, but the -required gate jobs (if: always()) still ran and misreported the cancellation as a hard failure ("expected 10 shard jobs, found 0" / "k3s matrix result was 'skipped'"). needs.test.result and needs.k3s.result both report "skipped" for a genuine upstream failure and for a whole-run cancellation alike, so they can't distinguish the two - cancelled() can, and is the fix. It's only valid in an if:, not inside a run: script (caught by actionlint), so both gates get a "Skip verification" step under if: cancelled() plus if: !cancelled() on their existing check steps, leaving the fail-loudly-on-genuine-anomalies logic untouched for real failures. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/test.yml | 24 +++++- ...uired-check-gates-fail-on-cancelled-run.md | 83 +++++++++++++++++++ 2 files changed, 106 insertions(+), 1 deletion(-) create mode 100644 docs/fixes/2026-08-31-required-check-gates-fail-on-cancelled-run.md diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index c0da6830ce9..9d8768e0b9a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -590,7 +590,21 @@ jobs: matrix: check: ["Acceptance Tests (linux)", "Acceptance Tests (macos)", "Acceptance Tests (windows)"] steps: + # A whole-run cancellation (e.g. superseded by a newer push to the same + # PR) skips the `test` matrix before it ever dispatches shard jobs, so + # the API query below would legitimately find 0 shards - not a real + # anomaly to fail loudly on. cancelled() reflects the run itself, unlike + # needs.test.result, which reports "skipped" for both this case and a + # genuine upstream (e.g. build) failure - so it can't replace this + # check. cancelled() is only valid in an `if:`, not inside a script, so + # the verification steps below are skipped via `if: ${{ !cancelled() }}` + # rather than early-exiting from within their run: blocks. + - name: Skip verification (workflow run was cancelled) + if: ${{ cancelled() }} + run: echo "workflow run was cancelled (e.g. superseded by a newer push); nothing to verify" + - name: Check per-OS test matrix result + if: ${{ !cancelled() }} env: GH_TOKEN: ${{ github.token }} CHECK: ${{ matrix.check }} @@ -638,7 +652,7 @@ jobs: # terraform-registry-cache only runs a linux/windows matrix (no macos # leg - see its `matrix.flavor` above), so its aggregated result must # not gate the macos alias. - if: matrix.check != 'Acceptance Tests (macos)' + if: ${{ !cancelled() && matrix.check != 'Acceptance Tests (macos)' }} run: | if [ "${{ needs.terraform-registry-cache.result }}" != "success" ]; then echo "terraform-registry-cache result was '${{ needs.terraform-registry-cache.result }}'" @@ -1019,7 +1033,15 @@ jobs: runs-on: ubuntu-latest if: ${{ always() }} steps: + # See the matching comment on test-required's per-OS check step: a + # whole-run cancellation is not a genuine k3s failure, and cancelled() + # is only valid in an `if:`, not inside a script. + - name: Skip verification (workflow run was cancelled) + if: ${{ cancelled() }} + run: echo "workflow run was cancelled (e.g. superseded by a newer push); nothing to verify" + - name: Check k3s matrix result + if: ${{ !cancelled() }} run: | if [ "${{ needs.k3s.result }}" != "success" ]; then echo "k3s matrix result was '${{ needs.k3s.result }}'" diff --git a/docs/fixes/2026-08-31-required-check-gates-fail-on-cancelled-run.md b/docs/fixes/2026-08-31-required-check-gates-fail-on-cancelled-run.md new file mode 100644 index 00000000000..c34ebb73778 --- /dev/null +++ b/docs/fixes/2026-08-31-required-check-gates-fail-on-cancelled-run.md @@ -0,0 +1,83 @@ +# Fix: `test-required`/`k3s-required` reported failure instead of passing through on a cancelled run + +**Date:** 2026-08-31 + +## Summary + +Five attached CI failure logs (`Acceptance Tests (linux/macos/windows)`, `[k3s] demo-helmfile`, +`Build (windows)`) all turned out to be one event: workflow run `33394180592` on PR #2878 +(`head_sha: 0a8e4e4974`, this branch) was **cancelled**, not failed -- confirmed via `gh api +repos/cloudposse/atmos/actions/runs/33394180592` (`conclusion: cancelled`) and per-job status +(`Build (windows)`: `cancelled`; the `test` matrix template job: `skipped`). The two `-required` +gate jobs still ran (`if: always()`) and mis-reported the cancellation as a genuine failure: + +```text +expected 10 'linux' shard jobs, found 0 +##[error]Process completed with exit code 1. +``` + +```text +k3s matrix result was 'skipped' +##[error]Process completed with exit code 1. +``` + +## Context + +`test-required` and `k3s-required` gate the sharded `test`/`k3s` matrix jobs and intentionally use +`if: always()` so the required check always resolves rather than staying pending forever. When the +whole workflow run is cancelled (this run's `head_sha` matches a commit pushed mid-session; the very +next push, made shortly after, is the likely trigger), GitHub skips the `test`/`k3s` matrix before +it ever dispatches jobs -- but the gate jobs still execute and, finding 0 shard jobs (or +`needs.k3s.result == 'skipped'`), correctly-by-their-own-logic-but-misleadingly report a hard +failure. `test-required`'s own comment already documents a deliberate design choice to "fail loudly +... rather than silently treating a missing shard as a pass" for genuine anomalies (a renamed job, +an API hiccup, or a real upstream `build` failure that legitimately produces the same "skipped" +result) -- that property needed to stay intact, so the fix could not simply treat any "skipped"/ +non-success result as acceptable. + +`needs.test.result` and `needs.k3s.result` both report `"skipped"` for two different situations +that must be told apart: a genuine upstream failure (should still fail loudly) and a whole-run +cancellation (should not). The `cancelled()` expression function distinguishes them -- it reflects +the run's own cancellation state, not any specific job's result -- so it's the correct signal here. +`cancelled()` is only valid in a job or step `if:`, not inside an interpolated `run:` script +(confirmed by `actionlint`), so the fix uses step-level `if: ${{ !cancelled() }}` guards plus a +small explicit "Skip verification" step for a clear log line, rather than an early `exit 0` inside +the existing script bodies. + +No other workflow in `.github/workflows/` has a `concurrency:` block referencing `test.yml`'s jobs, +and `test.yml` itself declares none -- the cancellation was not GitHub's automatic +`concurrency.cancel-in-progress` behavior (that requires an explicit `concurrency:` key, which this +workflow doesn't have). The exact cancelling actor (manual, Mergify, or another integration) wasn't +identified and doesn't change the fix; a run being superseded by a newer push to the same PR is +normal and expected regardless of the mechanism. + +## Changes + +`.github/workflows/test.yml`: + +- `test-required`: added a "Skip verification (workflow run was cancelled)" step + (`if: ${{ cancelled() }}`) and gated both existing check steps ("Check per-OS test matrix result", + "Check terraform-registry-cache result") with `if: ${{ !cancelled() }}` (the latter combined with + its existing `matrix.check != 'Acceptance Tests (macos)'` condition), so a cancelled run resolves + as a passing (all-steps-skipped) job instead of a hard failure. The per-shard/API-based + verification logic itself is unchanged. +- `k3s-required`: same pattern -- an explicit skip step plus `if: ${{ !cancelled() }}` on the + existing "Check k3s matrix result" step. + +No changes to `test`, `k3s`, `build`, or any other job -- this only affects how the two gate jobs +report a cancellation that already happened upstream. + +## Validation + +- `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/test.yml'))"` -- valid YAML. +- `actionlint .github/workflows/test.yml` -- 0 issues (an earlier draft using `${{ cancelled() }}` + directly inside a `run:` script was caught and corrected by this check: "calling function + 'cancelled' is not allowed here"). +- Confirmed via `gh api` that all five attached failures share `run_id: 33394180592`, and that this + run's own conclusion is `cancelled` (not `failure`), ruling out a genuine test/build regression. +- Not exercised end-to-end (no way to trigger and cancel a real Actions run from this session); the + fix is a narrow, `actionlint`-verified expression-logic correction with an unambiguous mechanism. + +## Follow-ups + +None. From eef85a8666f44ed00d2c692bcfb128806b454b6b Mon Sep 17 00:00:00 2001 From: "atmos-pro[bot]" <173522224+atmos-pro[bot]@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:16:52 +0000 Subject: [PATCH 14/18] [autocommit] formatting fixes --- NOTICE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/NOTICE b/NOTICE index e2dee243e1f..7ef9d6bd8c9 100644 --- a/NOTICE +++ b/NOTICE @@ -63,7 +63,7 @@ APACHE 2.0 LICENSED DEPENDENCIES - cuelang.org/go License: Apache-2.0 - URL: https://github.com/cue-lang/cue/blob/v0.16.1/LICENSE + URL: Unknown - github.com/Azure/go-autorest/autorest/to License: Apache-2.0 From 5ac5d9739f3cfe7265a1ced99c84954b01de12d2 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 09:25:59 -0500 Subject: [PATCH 15/18] fix(ci): pin cuelang.org/go's NOTICE URL to a deterministic override Review Dependency Licenses failed: NOTICE had "URL: Unknown" for cuelang.org/go, but a fresh generate-notice.sh run resolved a real URL, tripping the out-of-date check. Root cause was a race, not one bad run: this branch's merge commit already had the correct URL, but a subsequent [autocommit] formatting fixes commit (atmos-pro[bot]) regenerated NOTICE under a network condition where go-licenses' live resolution for cuelang.org/go failed, silently reverting it to "Unknown" and committing that regression - exactly the oscillation scripts/generate-notice.sh's REPO_OVERRIDES mechanism exists to prevent for modules go-licenses can't resolve reliably, cuelang.org/go just wasn't in the list yet. Added it (repo github.com/cue-lang/cue, no tag prefix, LICENSE path), which reconstructs the exact URL CI itself resolved (https://github.com/cue-lang/cue/blob/v0.16.1/LICENSE) from go.mod's pinned v0.16.1 with no network dependency, and applied that one-line NOTICE fix by hand: a local generate-notice.sh run silently produced a truncated 102-dependency report (vs. CI's 643) with 0 Apache-2.0/BSD licenses found, consistent with this machine lacking a Linux-targeting C cross-compiler for CGO_ENABLED=1 GOOS=linux GOARCH=amd64 - so that broken local output was discarded rather than committed, and the NOTICE line was hand-verified against the override's own URL-construction formula and go.mod's version instead. Co-Authored-By: Claude Sonnet 5 --- NOTICE | 2 +- scripts/generate-notice.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/NOTICE b/NOTICE index 7ef9d6bd8c9..e2dee243e1f 100644 --- a/NOTICE +++ b/NOTICE @@ -63,7 +63,7 @@ APACHE 2.0 LICENSED DEPENDENCIES - cuelang.org/go License: Apache-2.0 - URL: Unknown + URL: https://github.com/cue-lang/cue/blob/v0.16.1/LICENSE - github.com/Azure/go-autorest/autorest/to License: Apache-2.0 diff --git a/scripts/generate-notice.sh b/scripts/generate-notice.sh index bbe4b3f06f4..7c18b1afd3e 100755 --- a/scripts/generate-notice.sh +++ b/scripts/generate-notice.sh @@ -43,6 +43,7 @@ inet.af/netaddr|github.com/inetaf/netaddr||LICENSE go4.org/intern|github.com/go4org/intern||LICENSE go4.org/netipx|github.com/go4org/netipx||LICENSE go4.org/unsafe/assume-no-moving-gc|github.com/go4org/unsafe-assume-no-moving-gc||LICENSE +cuelang.org/go|github.com/cue-lang/cue||LICENSE cloud.google.com/go|github.com/googleapis/google-cloud-go||LICENSE cloud.google.com/go/auth|github.com/googleapis/google-cloud-go|auth|auth/LICENSE cloud.google.com/go/auth/oauth2adapt|github.com/googleapis/google-cloud-go|auth/oauth2adapt|auth/oauth2adapt/LICENSE From 7487aa69a0804bd62382aa5a5c10ba7d4c249510 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 11:44:16 -0500 Subject: [PATCH 16/18] fix(ci): retry go install go-licenses on transient sum.golang.org failures Review Dependency Licenses failed installing go-licenses@v1.6.0: a mid-stream HTTP/2 reset (stream ID 1155; INTERNAL_ERROR) reading sum.golang.org during go install's go.sum verification, unrelated to any actual dependency problem and unrelated to the immediately preceding commit on this branch (confirmed via gh api against head_sha 5ac5d9739f, which only touched an unrelated NOTICE URL override). Same failure class already fixed once for go mod download (docs/fixes/2026-08-25-build-atmos-go-mod-download-retry.md, later ported to magefiles/build.go's runGoModDownload) - just hit a different network call (go install's dependency-graph resolution) in a different script. Wrapped generate-notice.sh's bare go install in the same 3-attempt/15s-backoff until loop, matching .github/actions/download-artifact-retry's convention. Co-Authored-By: Claude Sonnet 5 --- ...-08-31-notice-go-licenses-install-retry.md | 54 +++++++++++++++++++ scripts/generate-notice.sh | 19 ++++++- 2 files changed, 72 insertions(+), 1 deletion(-) create mode 100644 docs/fixes/2026-08-31-notice-go-licenses-install-retry.md diff --git a/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md b/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md new file mode 100644 index 00000000000..cab63723df0 --- /dev/null +++ b/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md @@ -0,0 +1,54 @@ +# Fix: retry `go install go-licenses` in `scripts/generate-notice.sh` on transient sum.golang.org failures + +**Date:** 2026-08-31 + +## Summary + +`Review Dependency Licenses` failed with: + +```text +go: github.com/google/go-licenses@v1.6.0: version constraints conflict: + github.com/google/go-licenses@v1.6.0 indirectly requires github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa: verifying go.mod: github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa/go.mod: reading https://sum.golang.org/tile/8/0/x042/147: stream error: stream ID 1155; INTERNAL_ERROR; received from peer +##[error]Process completed with exit code 1. +``` + +A transient mid-stream HTTP/2 reset talking to `sum.golang.org` (the Go checksum database) during +`go install`'s go.sum verification, unrelated to any actual dependency problem -- the exact same +failure class already fixed once for `go mod download` in +`docs/fixes/2026-08-25-build-atmos-go-mod-download-retry.md` (later ported to +`magefiles/build.go`'s `runGoModDownload` per +`docs/fixes/2026-08-26-merge-main-go-mod-download-retry-port.md`), just hitting a different +network call (`go install`'s dependency-graph resolution, not `go mod download`) in a different +script. + +## Context + +`scripts/generate-notice.sh`'s `go install "github.com/google/go-licenses@${GO_LICENSES_VERSION}"` +call had no retry logic, so a single mid-stream `sum.golang.org` hiccup failed the whole NOTICE +regeneration outright, before it ever got to actually scanning dependencies. Confirmed via `gh api +repos/cloudposse/atmos/actions/jobs/99522359137` that this ran against `head_sha: 5ac5d9739f` -- +the immediately preceding commit on this branch, which itself only touched the `REPO_OVERRIDES` +list and `NOTICE` for an unrelated `cuelang.org/go` URL fix +(`docs/fixes/2026-08-31-required-check-gates-fail-on-cancelled-run.md`'s sibling commit) -- ruling +out a regression from that change. + +## Changes + +- `scripts/generate-notice.sh`: wrapped the bare `go install` call in a 3-attempt/15s-backoff + `until` retry loop, matching the established convention (`.github/actions/download-artifact-retry`, + `magefiles/build.go`'s `runGoModDownload`). `set -euo pipefail` doesn't short-circuit this, since + a command used as an `until`/`while` condition is exempt from `set -e`'s early-exit behavior + (same reasoning documented in the original `go mod download` fix). + +## Validation + +- `bash -n scripts/generate-notice.sh` -- clean. +- `shellcheck scripts/generate-notice.sh` -- only two pre-existing, unrelated SC2129 style notes + (lines 221/238, `cat >>` redirects) untouched by this change; zero new findings. +- Not exercised end-to-end against a real `sum.golang.org` outage (not reproducible on demand); + the fix is a direct, narrow port of an already-validated pattern (see the two referenced prior + fix docs) to a second call site hitting the same failure class. + +## Follow-ups + +None. diff --git a/scripts/generate-notice.sh b/scripts/generate-notice.sh index 7c18b1afd3e..cb33b6a36d4 100755 --- a/scripts/generate-notice.sh +++ b/scripts/generate-notice.sh @@ -124,7 +124,24 @@ EOF GO_LICENSES_BIN="$(command -v go-licenses || true)" if [ -z "${GO_LICENSES_BIN}" ]; then echo "Installing go-licenses ${GO_LICENSES_VERSION}..." - go install "github.com/google/go-licenses@${GO_LICENSES_VERSION}" + # `go install` resolves this module's full dependency graph, including a + # go.sum verification round-trip against sum.golang.org - a transient + # mid-stream HTTP/2 reset there fails the whole install outright. Retry a + # few times with a short cooldown, matching the 3-attempt/15s-backoff + # convention already used for artifact downloads (see + # .github/actions/download-artifact-retry) and go mod download + # (magefiles/build.go's runGoModDownload). + attempt=1 + max_attempts=3 + until go install "github.com/google/go-licenses@${GO_LICENSES_VERSION}"; do + if [ "$attempt" -ge "$max_attempts" ]; then + echo "go install github.com/google/go-licenses failed after $max_attempts attempts" >&2 + exit 1 + fi + echo "go install github.com/google/go-licenses failed (attempt $attempt/$max_attempts), retrying in 15s..." >&2 + sleep 15 + attempt=$((attempt + 1)) + done GOBIN="$(go env GOBIN)" if [ -z "${GOBIN}" ]; then GOBIN="$(go env GOPATH)/bin" From a9cf70f0615a2641d8cdc8d7392610a7e9cf24d0 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 12:05:27 -0500 Subject: [PATCH 17/18] fix(docs): replace a literal tab with spaces in a fix-log fenced block Run pre-commit hooks failed atmos-validate-editorconfig: a fenced code block in docs/fixes/2026-08-31-notice-go-licenses-install-retry.md quoted a Go toolchain error message verbatim, including its original tab-indented continuation line - violating this repo's *.md indent_style=space rule. Replaced the literal tab with two spaces (matching indent_size=2), content otherwise unchanged. Scanned every other 2026-08-31 fix-log doc added this session for the same issue; none found. Co-Authored-By: Claude Sonnet 5 --- docs/fixes/2026-08-31-notice-go-licenses-install-retry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md b/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md index cab63723df0..8391a938791 100644 --- a/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md +++ b/docs/fixes/2026-08-31-notice-go-licenses-install-retry.md @@ -8,7 +8,7 @@ ```text go: github.com/google/go-licenses@v1.6.0: version constraints conflict: - github.com/google/go-licenses@v1.6.0 indirectly requires github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa: verifying go.mod: github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa/go.mod: reading https://sum.golang.org/tile/8/0/x042/147: stream error: stream ID 1155; INTERNAL_ERROR; received from peer + github.com/google/go-licenses@v1.6.0 indirectly requires github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa: verifying go.mod: github.com/googleapis/enterprise-certificate-proxy@v0.0.0-20220520183353-fd19c99a87aa/go.mod: reading https://sum.golang.org/tile/8/0/x042/147: stream error: stream ID 1155; INTERNAL_ERROR; received from peer ##[error]Process completed with exit code 1. ``` From 3e0515ebfdda9cd5b4ae9d2b5f57add49f3aef25 Mon Sep 17 00:00:00 2001 From: Erik Osterman Date: Mon, 31 Aug 2026 21:40:37 -0500 Subject: [PATCH 18/18] docs(blog): bump terraform-component-mocks date to when its content changed fix(mocks) commit 6c2250349d edited this post's body (the // default behavior clarification) on 2026-08-06, but the post kept displaying/sorting under its original 2026-07-15 publish date since Docusaurus has no separate date. Added an explicit date: frontmatter override for the edit date, matching this repo's existing convention for date overrides (e.g. 2026-01-02-unified-task-runner.mdx). Co-Authored-By: Claude Sonnet 5 --- website/blog/2026-07-15-terraform-component-mocks.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/website/blog/2026-07-15-terraform-component-mocks.mdx b/website/blog/2026-07-15-terraform-component-mocks.mdx index 4d3e56c080b..712a676ca57 100644 --- a/website/blog/2026-07-15-terraform-component-mocks.mdx +++ b/website/blog/2026-07-15-terraform-component-mocks.mdx @@ -1,6 +1,7 @@ --- slug: terraform-component-mocks title: "Component Mocks for Terraform YAML Lookups" +date: 2026-08-06T12:00:00.000Z authors: [osterman] tags: [feature, dx] ---