Summarize the problem
The sets up a mirror test in the "Test version in Gallery" workflow fails intermittently, because nothing waits for the second SQL Server container to be ready before the test run starts.
Seen on PR #10588, whose changes cannot reach this test at all - it only touches Get-DbaWsfc*, which cannot even run on Linux. Re-running the identical commit passed, so no code changed between the failure and the success.
The failure
WARNING: [Invoke-DbaDbMirroring] Failure on primary | An exception occurred while executing a
Transact-SQL statement or batch. Communications to the remote server instance 'TCP://mssql2:5022'
failed before database mirroring was fully started. The ALTER DATABASE command failed. Retry the
command when the remote database is started.
[-] sets up a mirror 2.06s
Expected 'Success', but got $null.
at Invoke-DbaDbMirroring @params | Select-Object -ExpandProperty Status | Should -Be "Success",
.github/scripts/gh-actions.ps1:83
That message is SQL Server's own "the partner was not listening yet" error.
Mechanism
.github/workflows/gallery.yml starts three containers with docker run -d and then moves straight on:
docker run -p 1433:1433 ... --name mssql1 ... -d dbatools/sqlinstance
docker run -p 14333:1433 ... --name mssql2 ... -d dbatools/sqlinstance2
docker run -p 14334:1433 ... --name mssql3 ... -d dbatools/sqlinstance2
There is no health check and no wait loop. The steps that follow - cloning appveyor-lab, importing dbatools - happen to give the containers a few seconds, and that is usually enough. When it is not, the first test that needs the second instance fails.
Mirroring makes this more likely than the other tests, because it needs more than a reachable engine: mssql2 must also have its mirroring endpoint listening on 5022, and the primary reaches it container to container over localnet as TCP://mssql2:5022, not through the published port. The engine can be accepting logins on 1433 while the endpoint is still starting.
How often
Once in the last 41 runs of the workflow, so this is rare rather than routinely flaky. Opening it anyway because the failure is indistinguishable from a real regression at first glance, and it cost a review cycle on an unrelated PR to establish that it was not one.
Suggested fix
Wait for readiness in the workflow before running the tests, rather than relying on the incidental delay of the intervening steps. Something that polls each instance until it accepts a connection, and additionally waits for the mirroring endpoint on mssql2:
SELECT state_desc FROM sys.database_mirroring_endpoints
until it reports STARTED.
A retry around the single test would also make it green, but it would only hide the same race for whichever test happens to run first against mssql2.
I have not touched this, since it is CI scripting rather than module code.
created by Claude
Summarize the problem
The
sets up a mirrortest in the "Test version in Gallery" workflow fails intermittently, because nothing waits for the second SQL Server container to be ready before the test run starts.Seen on PR #10588, whose changes cannot reach this test at all - it only touches
Get-DbaWsfc*, which cannot even run on Linux. Re-running the identical commit passed, so no code changed between the failure and the success.The failure
That message is SQL Server's own "the partner was not listening yet" error.
Mechanism
.github/workflows/gallery.ymlstarts three containers withdocker run -dand then moves straight on:There is no health check and no wait loop. The steps that follow - cloning appveyor-lab, importing dbatools - happen to give the containers a few seconds, and that is usually enough. When it is not, the first test that needs the second instance fails.
Mirroring makes this more likely than the other tests, because it needs more than a reachable engine:
mssql2must also have its mirroring endpoint listening on 5022, and the primary reaches it container to container overlocalnetasTCP://mssql2:5022, not through the published port. The engine can be accepting logins on 1433 while the endpoint is still starting.How often
Once in the last 41 runs of the workflow, so this is rare rather than routinely flaky. Opening it anyway because the failure is indistinguishable from a real regression at first glance, and it cost a review cycle on an unrelated PR to establish that it was not one.
Suggested fix
Wait for readiness in the workflow before running the tests, rather than relying on the incidental delay of the intervening steps. Something that polls each instance until it accepts a connection, and additionally waits for the mirroring endpoint on
mssql2:until it reports
STARTED.A retry around the single test would also make it green, but it would only hide the same race for whichever test happens to run first against
mssql2.I have not touched this, since it is CI scripting rather than module code.
created by Claude