Fix four sources of CI failure - #746
Merged
Merged
Conversation
GetInfo gave the 'dotnet --info' process a 2000ms cancellation token. That is not a budget the call fails politely when it misses; CliWrap kills the process and the OperationCanceledException comes back out of whichever TestInitialize happened to ask, failing that test and no other. It showed up as CanGetPackage failing alone out of 709 in IKVM.Tests on net8.0 win-x64, with the stack running through AssemblyClassLoaderTests.TestInitialize into DotNetSdkResolver. Two things make 2000ms tight enough to lose. 'dotnet --info' enumerates every installed SDK and runtime, and CI installs five of each. And ResolvePath was uncached, so every test initializer in the five projects that ask for reference assembly paths spawned the process again, several of them at once, on a machine already busy running the tests. Cache the result and give the call two minutes. The installed SDK cannot change while the run is in progress, so once is enough, and the lock around the cache also keeps concurrent initializers from spawning the process side by side, which was part of what made it slow in the first place.
WindowUpdateFocusabilityTest was already excluded on macosx-all. It has now failed on net472 win-x64 in two separate runs, on main and on a documentation branch a week later, both times with 'window1 is not focusable'. It is an applet test that clicks a button and waits for the window manager to hand focus over, which a runner with no interactive desktop session does not reliably do. Widened to macosx-all,windows-all rather than generic-all, since it has not been seen failing on Linux. Both the .html and .java halves move together, as they were aligned to. SetFullScreenTest is new. It does not exist in 8u482 and arrived with the jdk8u504-b01 upgrade, which makes it a fourth test from that upgrade needing an exclusion; it was missed because unlike the other three it fails intermittently rather than every time. It goes full screen and asks Robot for the colour of the centre pixel, expecting the red frame it just showed, and read 12,12,12 - the desktop background - 300ms after waitForIdle. Every other test in java/awt/FullScreen is already excluded, for the reason this one failed, so it is listed generic-all with that reasoning rather than pinned to the one platform a single occurrence happens to name. Not excluded here: LegacyDHEKeyExchange failed once, on net8.0 win-x64 on 15 August, when the client hung up mid-handshake and the server side reported 'Software caused connection abort: recv failed'. It is not graphical and one occurrence is not enough to call it, so it is left as evidence.
wasabii
force-pushed
the
ci/fix-flaky-tests
branch
from
August 29, 2026 16:22
2d97e3d to
4a0d6bc
Compare
Both transfer tests started a server task, slept a second, and then connected, which leaves two ways to fail on a loaded machine. CanReceiveBlocking bound port 42341. A fixed port is not ours to claim: anything else holding it, including the previous run's socket still in TIME_WAIT, fails the bind. It also bound the wildcard address rather than loopback. CanReceiveNonBlocking asked for port 0 correctly but read the assigned port back out of a variable the server task writes, one second after starting it. A task that has not been scheduled by then leaves the port at 0 and the client connects to nothing. Bind on the test thread in both cases. Port 0 lets the OS pick, and the port is known before the task starts. Returning from bind also establishes the listen backlog, so the blocking test can connect straight away and its sleep is gone. The non-blocking one keeps its delay, which now only gives the selector time to register and is no longer load bearing.
wasabii
changed the title
VerifyCACerts walks every root in the shipped cacerts bundle and errors on any that expires within 90 days: ERROR: cert "entrustevca [jdk]" expiry "Fri Nov 27 20:53:42 UTC 2026" will expire within 90 days That root expires 2026-11-27 20:53:42 UTC, so the 90 day window opened at 2026-08-29 20:53:42 UTC. This is not intermittent, and the two CI runs on this branch happen to bracket the instant: partition 13 net472 win-x64 ran 17:34 to 18:46 UTC and passed, and the next run crossed the boundary and failed on net472 win-x64 and net8.0 linux-x64 together. Every run from here fails on every platform until the bundle changes. Excluded generic-all, which is where the evidence points. The real fix is upstream refreshing cacerts, which jdk8u does periodically, so this entry should come out with the next update that removes or replaces that root rather than being left in place indefinitely.
wasabii
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four causes. Three are intermittent; one is not, and is currently keeping
mainred regardless of this PR.VerifyCACerts — not a flake, fails from here on
sun/security/lib/cacerts/VerifyCACerts.javaerrors on any root within 90 days of expiry:The window opened at 2026-08-29 20:53:42 UTC. The two CI runs on this branch bracket that instant: partition 13
net472:win-x64ran 17:34–18:46 UTC and passed; the next run crossed it and failed onnet472:win-x64andnet8.0:linux-x64together. Every run fails from here, on every platform and branch, until the bundle changes. Excludedgeneric-all, with the comment saying to drop the entry when jdk8u next refreshes cacerts rather than leaving it in indefinitely.dotnet --infoDotNetSdkResolver.GetInfogave the process a 2000ms cancellation token, andResolvePathwas uncached, so every test initializer in the five projects that ask for reference assembly paths respawned it, several concurrently, on a machine already running the tests. When it missed the deadline CliWrap killed the process and theOperationCanceledExceptionsurfaced out ofTestInitialize. Seen onmain(net8.0 win-x86),docs/claude-md(net8.0 win-x64) andtest/reenable-passing. Cached, and given two minutes.ServerSocketChannel tests
CanReceiveBlockingbound a fixed port 42341, which anything else on the machine — including the previous run's socket in TIME_WAIT — can be holding.CanReceiveNonBlockingasked for an ephemeral port correctly but read it back out of a variable the server task assigns, one second after starting the task, so a task not yet scheduled left the client connecting to port 0. Both now bind on the test thread before the task starts.Two AWT tests
WindowUpdateFocusabilityTestwas already excluded on macOS and failed twice on Windows withwindow1 is not focusable. Widened tomacosx-all,windows-all; not seen on Linux, so the scope stops there, and the.htmland.javahalves move together.SetFullScreenTestdoes not exist in 8u482 and arrived with jdk8u504-b01 — a fourth test from that upgrade needing an exclusion, missed because it fails intermittently rather than every time. It Robot-samples the centre pixel expecting red and read 12,12,12, the desktop background. Every other test injava/awt/FullScreenis already excluded for that reason.Scopes were checked against
RegressionParameters.getPlatforms, which buildsos.family + "-all", andExcludeList.readBugIds, which comma-splits the second column.Not addressed
URLTests.CanGetFromURLmakes seven live HTTPS requests to badssl.com and has failed three times, including onmain. Left alone deliberately; replacing that dependency is a separate decision.ButtonGroupLayoutTraversalTest,SetLocationRelativeToTest,ConsumeNextMnemonicKeyTypedTest,TestGCMKeyAndIvCheck,TransformationWarningsTest. At 48 OpenJDK jobs per run, retiring these one occurrence at a time will not converge.LegacyDHEKeyExchange, one occurrence on 15 August. Not graphical; left as evidence.IKVM.Tests:net8.0:win-x86on 29 August. No change: retrying the step papers over it rather than fixing anything.developfails to build becauseIKVM.slnx:90is the only.ikvmprojentry without aTypeGUID.mainhas noIKVM.slnx.