You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
5. Runs platform-specific test suite (`.github/scripts/gh-actions.ps1` or `gh-winactions.ps1`)
251
251
252
-
### AppVeyor CI (appveyor.yml)
252
+
### ci-azure (.github/workflows/ci-azure.yml)
253
253
254
-
Runs on every push, 5 scenarios in parallel:
255
-
-**2008R2:** SQL Server 2008 R2 Express
256
-
-**2016:** SQL Server 2016 Developer
257
-
-**2016_2017:** SQL Server 2016 + 2017 (for Copy-* commands)
258
-
-**service_restarts:** Service restart testing
259
-
-**default:** 2008 R2 + 2016 combo
254
+
The test suite of record. Runs on pushes to `development` and on PRs targeting it, on self-hosted Azure VMSS runners, with these scenarios in parallel:
255
+
-**SINGLE:** SQL Server 2022, split into five parts
256
+
-**MULTI:** SQL Server 2022 + 2017, for tests needing two instances
257
+
-**COPY:** SQL Server 2017 + 2022, for `Copy-*` commands
258
+
-**HADR:** SQL Server 2019 with availability groups configured
259
+
-**RESTART:** SQL Server 2019, for service restart testing
260
+
-**default:** no instance, for tests that don't need a running server
261
+
262
+
It drives the `tests/appveyor.*.ps1` build scripts, which kept their name when the project moved off AppVeyor; `tests/gha.shim.ps1` supplies the AppVeyor build API they call. There is no `appveyor.yml` in the repository.
260
263
261
264
**Magic commit commands:**
262
265
- Add `(do Get-DbaFoo)` to commit message to run only `Get-DbaFoo` tests
We use Appveyor for running the Pester tests for each function. We have this repository to store content that is used in our tests. This keeps the current repository clean from excess files just for testing. **Maintainers will determine if tests require files be placed in this repository**.
21
+
We have this repository to store content that is used in our tests. The name predates the move off AppVeyor; CI still clones it for test fixtures. This keeps the current repository clean from excess files just for testing. **Maintainers will determine if tests require files be placed in this repository**.
Our project uses [Pester](https://pester.dev) for our testing framework. Appveyor runs a matrix of environments that test against various versions of SQL Server. We do have some commands that cannot be tested easily but the majority we use Pester to ensure the code behaves properly. Appveyor runs these tests for each-and-every commit in our repository.
131
+
Our project uses [Pester](https://pester.dev) for our testing framework. CI runs a matrix of environments that test against various versions of SQL Server. We do have some commands that cannot be tested easily but the majority we use Pester to ensure the code behaves properly. CI runs these tests for each-and-every commit in our repository.
132
132
133
133
We strive to have the Pester tests where any user can pull the project and run the same test in their environment (on a test lab of course or in our Docker image).
134
134
@@ -146,34 +146,33 @@ Tests make sure a "contract" is made between the code and its behavior: once a t
146
146
147
147
You can inspect/copy/cannibalize existing tests. You'll see that every test file is named with a simple convention `Verb-Noun*.Tests.ps1`, and this is required by [Pester](https://GitHub.com/pester/Pester), which is the de-facto standard for running tests in PowerShell.
148
148
149
-
## AppVeyor Environment
149
+
## CI Environment
150
150
151
-
AppVeyor is hooked up to test any commit, including PRs. Each commit triggers several builds, each referred to as a "scenario". We have the scenarios setup where the dbatools log is published as an artifact should you need to view why test are failing.
151
+
CI is hooked up to test any commit, including PRs. The suite of record is the `ci-azure` GitHub Actions workflow (`.github/workflows/ci-azure.yml`), which runs on self-hosted Azure VMSS runners. Each commit triggers several builds, each referred to as a "scenario". We have the scenarios setup where the dbatools log is published as an artifact should you need to view why test are failing.
152
152
153
-
- SINGLE: a server with a single SQL Server 2022 instance available ($TestConfig.InstanceSingle)
153
+
- SINGLE: a server with a single SQL Server 2022 instance available ($TestConfig.InstanceSingle), split into five parts that run in parallel
154
154
- MULTI: a server with two instances (SQL Server 2022 and SQL Server 2017) available for tests that need multiple instances ($TestConfig.InstanceMulti1 and $TestConfig.InstanceMulti2)
155
155
- COPY: a server with two instances (SQL Server 2017 and SQL Server 2022) available for tests that need multiple instances ($TestConfig.InstanceCopy1 and $TestConfig.InstanceCopy2)
156
-
- HADR: a single SQL Server 2022 instance available with Hadr configured ($TestConfig.InstanceHadr)
157
-
- RESTART: used to test service restarts ($TestConfig.InstanceRestart)
158
-
- 2008R2SP2Express: a server with a single SQL Server 2008 R2 Express Edition available to test some commands against an old version ($TestConfig.InstanceSingle)
156
+
- HADR: a single SQL Server 2019 instance available with Hadr configured ($TestConfig.InstanceHadr)
157
+
- RESTART: a single SQL Server 2019 instance used to test service restarts ($TestConfig.InstanceRestart)
159
158
- default: a server with no instance for all tests that don't need a running instance
160
159
161
-
Builds are split among "scenario"(s) because not every test requires everything to be up and running, and resources on AppVeyor are constrained.
162
-
AppVeyor is set up to recognize what "scenario" is required by your test, simply inspecting for the presence of $TestConfig.Instance*.
160
+
Builds are split among "scenario"(s) because not every test requires everything to be up and running, and runner capacity is constrained.
161
+
CI is set up to recognize what "scenario" is required by your test, simply inspecting for the presence of $TestConfig.Instance*.
163
162
164
163
Most PRs will target `public/*.ps1` files to add functionality or resolve bugs.
165
164
Our test runner will try and figure out what tests needs to be run based on the files modified in the PR, plus all the dependencies.
166
165
167
-
If the automatic detection doesn't work, and you don't want to wait for the entire test suite to run (i.e. you need to run only `Get-DbaFoo`on AppVeyor), you can use a **_magic command_** within the commit message, namely `(do Get-DbaFoo)` . This will run only test files within the test folder matching this mask `tests\*Get-DbaFoo*.Tests.ps1`.
166
+
If the automatic detection doesn't work, and you don't want to wait for the entire test suite to run (i.e. you need to run only `Get-DbaFoo`in CI), you can use a **_magic command_** within the commit message, namely `(do Get-DbaFoo)` . This will run only test files within the test folder matching this mask `tests\*Get-DbaFoo*.Tests.ps1`.
168
167
169
-
<!-- TODO: how to run your own AppVeyor before pushing a PR -->
168
+
The build scripts CI drives are the `tests/appveyor.*.ps1` files. They keep that name for historical reasons - the harness was carried over unchanged when the project moved off AppVeyor, and `tests/gha.shim.ps1` supplies the AppVeyor build API it still calls.
170
169
171
170
## Codecov
172
171
173
172
We utilize Codecov as many other PowerShell community projects are doing. You can see those coverage reports directly on Codecov site for our project.
This system allows us to see the percentage of the coverage for our tests. A rough goal is getting as close to 80% coverage, some commands we have that are just not achievable due to various limitations. As part of our test framework that runs on Appveyor we upload a coverage file to Codecov so it stays current.
176
+
This system allows us to see the percentage of the coverage for our tests. A rough goal is getting as close to 80% coverage, some commands we have that are just not achievable due to various limitations. As part of our test framework that runs in CI we upload a coverage file to Codecov so it stays current.
178
177
179
178
If you want to start contributing new tests, choose the ones with no coverage. You can also inspect functions with low coverage and improve existing tests. [See improving test](https://dbatools.io/improving-tests/).
Copy file name to clipboardExpand all lines: tests/CLAUDE.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ This guide provides the ongoing standards and best practices for running and wri
4
4
5
5
**This file is the single source for test policy.** General PowerShell style remains authoritative in the repository's root `CLAUDE.md` and applies to test files too.
6
6
7
-
## Local (not appveyor) Testing Setup
7
+
## Local (not CI) Testing Setup
8
8
9
9
To test commands locally during development:
10
10
@@ -34,7 +34,7 @@ This allows you to manually test commands against actual SQL Server instances be
34
34
35
35
## CHOOSING A TEST INSTANCE
36
36
37
-
`$TestConfig` exposes several SQL Server instances, defined in `private/testing/Get-TestConfig.ps1`. The choice is not cosmetic: `pester.groups.ps1` autodetects which `$TestConfig.Instance*` variable a test file references and assigns the file to that AppVeyor scenario. **Always pick the lightest instance that can demonstrate the behaviour** - reaching for a heavier one moves the test into a scarcer, slower scenario for no benefit.
37
+
`$TestConfig` exposes several SQL Server instances, defined in `private/testing/Get-TestConfig.ps1`. The choice is not cosmetic: `pester.groups.ps1` autodetects which `$TestConfig.Instance*` variable a test file references and assigns the file to that CI scenario. **Always pick the lightest instance that can demonstrate the behaviour** - reaching for a heavier one moves the test into a scarcer, slower scenario for no benefit.
Two consequences worth knowing before you write the test:
55
55
56
-
-**Only `InstanceSingle`, `InstanceMulti1`and `InstanceMulti2` exist on GitHub Actions.** A test written against `InstanceCopy*`, `InstanceHadr` or `InstanceRestart` runs on AppVeyor only. For new or changed command behavior, the required real-boundary coverage must also run on GitHub Actions or an Azure test runner; provision the needed boundary instead of omitting the regression test.
56
+
-**Every scenario runs on the self-hosted Azure runners** driven by `.github/workflows/ci-azure.yml`, so `InstanceCopy*`, `InstanceHadr`and `InstanceRestart` all get exercised there. The container-based `integration-tests.yml` workflow is narrower: only `InstanceSingle`, `InstanceMulti1` and `InstanceMulti2` exist on it. For new or changed command behavior, the required real-boundary coverage must run on one of these; provision the needed boundary instead of omitting the regression test.
57
57
-**`InstanceRestart` tests share machine state.** They run in file order within a `Describe`, and one test's leftovers become the next test's fixture. If a test mutates machine state that `AfterAll` cannot reliably undo - archived certificates, service accounts, registry values - restore it in a `try`/`finally` inside the `It` itself.
0 commit comments