From 50cccc9515ef906b70d17099ed38f921d7144db4 Mon Sep 17 00:00:00 2001 From: Tyler Lynch Date: Thu, 9 Jul 2026 14:11:11 -0400 Subject: [PATCH] run-acceptance-tests: add environment variable setup guidance --- .../skills/run-acceptance-tests/SKILL.md | 40 ++++++++++++++----- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/terraform/provider-development/skills/run-acceptance-tests/SKILL.md b/terraform/provider-development/skills/run-acceptance-tests/SKILL.md index d22de2a..4abb26b 100644 --- a/terraform/provider-development/skills/run-acceptance-tests/SKILL.md +++ b/terraform/provider-development/skills/run-acceptance-tests/SKILL.md @@ -1,6 +1,6 @@ --- name: run-acceptance-tests -description: Guide for running acceptance tests for a Terraform provider. Use this when asked to run an acceptance test or to run a test with the prefix `TestAcc`. +description: Guide for running acceptance tests for a Terraform provider. Use this when asked to run an acceptance test or to run a test with the prefix `TestAcc`, when a test fails with missing environment variables, or when diagnosing a failing or suspiciously passing acceptance test. license: MPL-2.0 metadata: copyright: Copyright IBM Corp. 2026 @@ -9,16 +9,33 @@ metadata: An acceptance test is a Go test function with the prefix `TestAcc`. -To run a focussed acceptance test named `TestAccFeatureHappyPath`: +Before running: acceptance tests create **real infrastructure** against the +provider's live API, which may incur cost. Confirm the configured +credentials point at a test account before proceeding. -1. Run `go test -run=TestAccFeatureHappyPath` with the following environment - variables: +To run a focused acceptance test named `TestAccFeatureHappyPath`: + +1. Run `go test -run=TestAccFeatureHappyPath -timeout 60m` with the + following environment variables: - `TF_ACC=1` - - Default to non-verbose test output. + + Default to non-verbose test output. Always pass an explicit `-timeout`: + `go test` kills any test run after 10 minutes by default, and acceptance + tests routinely exceed that. 1. The acceptance tests may require additional environment variables for - specific providers. If the test output indicates missing environment - variables, then suggest how to set up these environment variables securely. + specific providers. To discover which ones: + - Read the test's `PreCheck` / `testAccPreCheck` function and search the + test files: `grep -rn "os.Getenv" --include="*_test.go"`. + - Check the repository's README, CONTRIBUTING, or `.env.example` for + documented test setup. + - The provider's `Configure` method shows how credentials are resolved; + use the `provider-configuration` skill (if available) to understand a + credential provider chain. + + Set the variables for the single test invocation + (`EXAMPLE_API_KEY=... TF_ACC=1 go test ...`) rather than exporting them + into the shell profile, and never write secret values into files inside + the repository. To diagnose a failing acceptance test, use these options, in order. These options are cumulative: each option includes all the options above it. @@ -29,7 +46,7 @@ options are cumulative: each option includes all the options above it. 1. Offer debug-level logging. Enable debug-level logging with the environment variable `TF_LOG=debug`. 1. Offer to persist the acceptance test's Terraform workspace. Enable - persistance with the environment variable `TF_ACC_WORKING_DIR_PERSIST=1`. + persistence with the environment variable `TF_ACC_WORKING_DIR_PERSIST=1`. A passing acceptance test may be a false negative. To "flip" a passing acceptance test named `TestAccFeatureHappyPath`: @@ -39,3 +56,8 @@ acceptance test named `TestAccFeatureHappyPath`: 1. Run the acceptance test. Expect the test to fail. 1. If the test fails, then undo the edit and report a successful flip. Else, keep the edit and report an unsuccessful flip. + +If a test run is interrupted, real resources may be left behind; run the +provider's sweepers if it registers them (see the `provider-test-patterns` +skill's sweeper reference, if available). For writing or restructuring +tests, use the `provider-test-patterns` skill.