Skip to content

Commit 966913e

Browse files
Docs - Point CI documentation at ci-azure instead of AppVeyor (#10521)
1 parent ca40234 commit 966913e

5 files changed

Lines changed: 38 additions & 33 deletions

File tree

.github/CODEOWNERS

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,6 @@
55
/.github/CODEOWNERS @potatoqualitee @niphlod @andreasjordan
66
/.github/copilot-instructions.md @potatoqualitee @niphlod @andreasjordan
77
/.github/dependabot.yml @potatoqualitee @niphlod @andreasjordan
8-
/appveyor.yml @potatoqualitee @niphlod @andreasjordan
8+
/tests/appveyor.*.ps1 @potatoqualitee @niphlod @andreasjordan
9+
/tests/gha.shim.ps1 @potatoqualitee @niphlod @andreasjordan
10+
/tests/pester.groups.ps1 @potatoqualitee @niphlod @andreasjordan

.github/CONTRIBUTING-TESTING.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -446,18 +446,19 @@ git commit -m "Fix login handling (do Get-DbaLogin, Set-DbaLogin)"
446446
git commit -m "Update comments [skip ci]"
447447
```
448448

449-
### AppVeyor Build Matrix
449+
### CI Build Matrix
450450

451-
The CI runs tests across multiple scenarios in parallel:
451+
The `ci-azure` workflow runs tests across multiple scenarios in parallel on self-hosted Azure runners:
452452

453453
```yaml
454-
# From appveyor.yml
454+
# From .github/workflows/ci-azure.yml
455455
matrix:
456-
- scenario: SINGLE (split into 3 parts for speed)
457-
- scenario: MULTI
458-
- scenario: COPY
459-
- scenario: HADR
460-
- scenario: RESTART
456+
- scenario: SINGLE (split into 5 parts for speed, sql2022)
457+
- scenario: MULTI (sql2022 + sql2017)
458+
- scenario: COPY (sql2017 + sql2022)
459+
- scenario: HADR (sql2019)
460+
- scenario: RESTART (sql2019)
461+
- scenario: default (no instance)
461462
```
462463
463464
---

.github/copilot-instructions.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -223,7 +223,7 @@ dbatools/
223223
### Configuration Files
224224
- **CLAUDE.md** - Single source for repository-wide PowerShell style and workflow conventions
225225
- **tests/CLAUDE.md** - Single source for ongoing test policy and Pester 6 rules
226-
- **appveyor.yml** - AppVeyor CI (Windows testing with SQL Server 2008-2017)
226+
- **.github/workflows/ci-azure.yml** - the test suite of record (Windows testing on self-hosted Azure runners, SQL Server 2017/2019/2022)
227227
- **.github/workflows/integration-tests.yml** - GitHub Actions (cross-platform testing)
228228

229229
## CI/CD Validation Pipeline
@@ -249,14 +249,17 @@ dbatools/
249249
4. Installs SqlPackage
250250
5. Runs platform-specific test suite (`.github/scripts/gh-actions.ps1` or `gh-winactions.ps1`)
251251

252-
### AppVeyor CI (appveyor.yml)
252+
### ci-azure (.github/workflows/ci-azure.yml)
253253

254-
Runs on every push, 5 scenarios in parallel:
255-
- **2008R2:** SQL Server 2008 R2 Express
256-
- **2016:** SQL Server 2016 Developer
257-
- **2016_2017:** SQL Server 2016 + 2017 (for Copy-* commands)
258-
- **service_restarts:** Service restart testing
259-
- **default:** 2008 R2 + 2016 combo
254+
The test suite of record. Runs on pushes to `development` and on PRs targeting it, on self-hosted Azure VMSS runners, with these scenarios in parallel:
255+
- **SINGLE:** SQL Server 2022, split into five parts
256+
- **MULTI:** SQL Server 2022 + 2017, for tests needing two instances
257+
- **COPY:** SQL Server 2017 + 2022, for `Copy-*` commands
258+
- **HADR:** SQL Server 2019 with availability groups configured
259+
- **RESTART:** SQL Server 2019, for service restart testing
260+
- **default:** no instance, for tests that don't need a running server
261+
262+
It drives the `tests/appveyor.*.ps1` build scripts, which kept their name when the project moved off AppVeyor; `tests/gha.shim.ps1` supplies the AppVeyor build API they call. There is no `appveyor.yml` in the repository.
260263

261264
**Magic commit commands:**
262265
- Add `(do Get-DbaFoo)` to commit message to run only `Get-DbaFoo` tests

CONTRIBUTING.md

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ As of 2023 the project has been restructured but we also have a few different re
1818

1919
1. [dataplat/appveyor-lab](https://github.com/dataplat/appveyor-lab)
2020

21-
We use Appveyor for running the Pester tests for each function. We have this repository to store content that is used in our tests. This keeps the current repository clean from excess files just for testing. **Maintainers will determine if tests require files be placed in this repository**.
21+
We have this repository to store content that is used in our tests. The name predates the move off AppVeyor; CI still clones it for test fixtures. This keeps the current repository clean from excess files just for testing. **Maintainers will determine if tests require files be placed in this repository**.
2222

2323
1. [dataplat/docs](https://github.com/dataplat/docs)
2424

@@ -128,7 +128,7 @@ A few notes:
128128

129129
## Pester 🧪
130130

131-
Our project uses [Pester](https://pester.dev) for our testing framework. Appveyor runs a matrix of environments that test against various versions of SQL Server. We do have some commands that cannot be tested easily but the majority we use Pester to ensure the code behaves properly. Appveyor runs these tests for each-and-every commit in our repository.
131+
Our project uses [Pester](https://pester.dev) for our testing framework. CI runs a matrix of environments that test against various versions of SQL Server. We do have some commands that cannot be tested easily but the majority we use Pester to ensure the code behaves properly. CI runs these tests for each-and-every commit in our repository.
132132

133133
We strive to have the Pester tests where any user can pull the project and run the same test in their environment (on a test lab of course or in our Docker image).
134134

@@ -146,34 +146,33 @@ Tests make sure a "contract" is made between the code and its behavior: once a t
146146

147147
You can inspect/copy/cannibalize existing tests. You'll see that every test file is named with a simple convention `Verb-Noun*.Tests.ps1`, and this is required by [Pester](https://GitHub.com/pester/Pester), which is the de-facto standard for running tests in PowerShell.
148148

149-
## AppVeyor Environment
149+
## CI Environment
150150

151-
AppVeyor is hooked up to test any commit, including PRs. Each commit triggers several builds, each referred to as a "scenario". We have the scenarios setup where the dbatools log is published as an artifact should you need to view why test are failing.
151+
CI is hooked up to test any commit, including PRs. The suite of record is the `ci-azure` GitHub Actions workflow (`.github/workflows/ci-azure.yml`), which runs on self-hosted Azure VMSS runners. Each commit triggers several builds, each referred to as a "scenario". We have the scenarios setup where the dbatools log is published as an artifact should you need to view why test are failing.
152152

153-
- SINGLE: a server with a single SQL Server 2022 instance available ($TestConfig.InstanceSingle)
153+
- SINGLE: a server with a single SQL Server 2022 instance available ($TestConfig.InstanceSingle), split into five parts that run in parallel
154154
- MULTI: a server with two instances (SQL Server 2022 and SQL Server 2017) available for tests that need multiple instances ($TestConfig.InstanceMulti1 and $TestConfig.InstanceMulti2)
155155
- COPY: a server with two instances (SQL Server 2017 and SQL Server 2022) available for tests that need multiple instances ($TestConfig.InstanceCopy1 and $TestConfig.InstanceCopy2)
156-
- HADR: a single SQL Server 2022 instance available with Hadr configured ($TestConfig.InstanceHadr)
157-
- RESTART: used to test service restarts ($TestConfig.InstanceRestart)
158-
- 2008R2SP2Express: a server with a single SQL Server 2008 R2 Express Edition available to test some commands against an old version ($TestConfig.InstanceSingle)
156+
- HADR: a single SQL Server 2019 instance available with Hadr configured ($TestConfig.InstanceHadr)
157+
- RESTART: a single SQL Server 2019 instance used to test service restarts ($TestConfig.InstanceRestart)
159158
- default: a server with no instance for all tests that don't need a running instance
160159

161-
Builds are split among "scenario"(s) because not every test requires everything to be up and running, and resources on AppVeyor are constrained.
162-
AppVeyor is set up to recognize what "scenario" is required by your test, simply inspecting for the presence of $TestConfig.Instance*.
160+
Builds are split among "scenario"(s) because not every test requires everything to be up and running, and runner capacity is constrained.
161+
CI is set up to recognize what "scenario" is required by your test, simply inspecting for the presence of $TestConfig.Instance*.
163162

164163
Most PRs will target `public/*.ps1` files to add functionality or resolve bugs.
165164
Our test runner will try and figure out what tests needs to be run based on the files modified in the PR, plus all the dependencies.
166165

167-
If the automatic detection doesn't work, and you don't want to wait for the entire test suite to run (i.e. you need to run only `Get-DbaFoo` on AppVeyor), you can use a **_magic command_** within the commit message, namely `(do Get-DbaFoo)` . This will run only test files within the test folder matching this mask `tests\*Get-DbaFoo*.Tests.ps1`.
166+
If the automatic detection doesn't work, and you don't want to wait for the entire test suite to run (i.e. you need to run only `Get-DbaFoo` in CI), you can use a **_magic command_** within the commit message, namely `(do Get-DbaFoo)` . This will run only test files within the test folder matching this mask `tests\*Get-DbaFoo*.Tests.ps1`.
168167

169-
<!-- TODO: how to run your own AppVeyor before pushing a PR -->
168+
The build scripts CI drives are the `tests/appveyor.*.ps1` files. They keep that name for historical reasons - the harness was carried over unchanged when the project moved off AppVeyor, and `tests/gha.shim.ps1` supplies the AppVeyor build API it still calls.
170169

171170
## Codecov
172171

173172
We utilize Codecov as many other PowerShell community projects are doing. You can see those coverage reports directly on Codecov site for our project.
174173

175174
[Codecov - dataplat/dbatools](https://app.codecov.io/gh/dataplat/dbatools/tree/development)
176175

177-
This system allows us to see the percentage of the coverage for our tests. A rough goal is getting as close to 80% coverage, some commands we have that are just not achievable due to various limitations. As part of our test framework that runs on Appveyor we upload a coverage file to Codecov so it stays current.
176+
This system allows us to see the percentage of the coverage for our tests. A rough goal is getting as close to 80% coverage, some commands we have that are just not achievable due to various limitations. As part of our test framework that runs in CI we upload a coverage file to Codecov so it stays current.
178177

179178
If you want to start contributing new tests, choose the ones with no coverage. You can also inspect functions with low coverage and improve existing tests. [See improving test](https://dbatools.io/improving-tests/).

tests/CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This guide provides the ongoing standards and best practices for running and wri
44

55
**This file is the single source for test policy.** General PowerShell style remains authoritative in the repository's root `CLAUDE.md` and applies to test files too.
66

7-
## Local (not appveyor) Testing Setup
7+
## Local (not CI) Testing Setup
88

99
To test commands locally during development:
1010

@@ -34,7 +34,7 @@ This allows you to manually test commands against actual SQL Server instances be
3434

3535
## CHOOSING A TEST INSTANCE
3636

37-
`$TestConfig` exposes several SQL Server instances, defined in `private/testing/Get-TestConfig.ps1`. The choice is not cosmetic: `pester.groups.ps1` autodetects which `$TestConfig.Instance*` variable a test file references and assigns the file to that AppVeyor scenario. **Always pick the lightest instance that can demonstrate the behaviour** - reaching for a heavier one moves the test into a scarcer, slower scenario for no benefit.
37+
`$TestConfig` exposes several SQL Server instances, defined in `private/testing/Get-TestConfig.ps1`. The choice is not cosmetic: `pester.groups.ps1` autodetects which `$TestConfig.Instance*` variable a test file references and assigns the file to that CI scenario. **Always pick the lightest instance that can demonstrate the behaviour** - reaching for a heavier one moves the test into a scarcer, slower scenario for no benefit.
3838

3939
| Instance | Use it for | Scenario |
4040
|---|---|---|
@@ -53,7 +53,7 @@ Get-TestInstanceUsage | Group-Object -Property InstanceList -NoElement | Sort-Ob
5353

5454
Two consequences worth knowing before you write the test:
5555

56-
- **Only `InstanceSingle`, `InstanceMulti1` and `InstanceMulti2` exist on GitHub Actions.** A test written against `InstanceCopy*`, `InstanceHadr` or `InstanceRestart` runs on AppVeyor only. For new or changed command behavior, the required real-boundary coverage must also run on GitHub Actions or an Azure test runner; provision the needed boundary instead of omitting the regression test.
56+
- **Every scenario runs on the self-hosted Azure runners** driven by `.github/workflows/ci-azure.yml`, so `InstanceCopy*`, `InstanceHadr` and `InstanceRestart` all get exercised there. The container-based `integration-tests.yml` workflow is narrower: only `InstanceSingle`, `InstanceMulti1` and `InstanceMulti2` exist on it. For new or changed command behavior, the required real-boundary coverage must run on one of these; provision the needed boundary instead of omitting the regression test.
5757
- **`InstanceRestart` tests share machine state.** They run in file order within a `Describe`, and one test's leftovers become the next test's fixture. If a test mutates machine state that `AfterAll` cannot reliably undo - archived certificates, service accounts, registry values - restore it in a `try`/`finally` inside the `It` itself.
5858

5959
## TESTING FOR WARNINGS

0 commit comments

Comments
 (0)