CI reliability is poor

[mhsmith]

Over the last month, CI on the main branch has failed about 50% of the time. There appear to be several different issues, but most of them are related to Linux.

[freakboy3742]

To clarify - the issues I'm aware of are:

1. Windows app startup failure

The app build successfully, but returns "status code 1" when it runs. I haven't had any luck getting any more diagnostic information about this; and I've never been able to reproduce it locally.

Anecdotally, it appears most common on ARM, with PySide. However, I've also seen it on x86_64, on PyGame, and very rarely with Toga. It's only ever the app test that fails, not the Visual Studio project, AFAICT.

2. OpenSuSE repo failures

Docker| #9 ERROR: process "/bin/sh -c zypper install -y -t pattern devel_basis" did not complete successfully: exit code: 4
Docker| ------
Docker|  > [6/7] RUN zypper install -y -t pattern devel_basis:
Docker| 0.738 Resolving package dependencies...
Docker| 0.770 
Docker| 0.770 Problem: 1: the to be installed pattern:devel_basis-20170319-13.4.x86_64 requires 'patterns-devel-base-devel_basis', but this requirement cannot be provided
Docker| 0.770 not installable providers: patterns-devel-base-devel_basis-20170319-13.4.x86_64[repo-oss]
Docker| 0.770 
Docker| 0.770  Solution 1: downgrade of libncurses6-6.6.20260613-109.3.x86_64 to libncurses6-6.6.20260613-109.2.x86_64
Docker| 0.770  Solution 2: do not install pattern:devel_basis-20170319-13.4.x86_64
Docker| 0.770  Solution 3: break pattern:devel_basis-20170319-13.4.x86_64 by ignoring some of its dependencies
Docker| 0.770 
Docker| 0.770 Choose from above solutions by number or cancel [1/2/3/c/d/?] (c): c

As best as I can make out, this is caused by OpenSuSE being a "rolling" distribution, and they occasionally push unstable configurations. I don't think there's much we can do here, other than dropping CI support for OpenSuSE.

3. OpenSuSE runtime failures

The Toga tests occasionally fail on OpenSuSE with this error:

[verifyapp] Starting app...
===========================================================================
/app/verifyapp-0.0.1/usr/lib64/verifyapp/app_packages/gi/overrides/__init__.py:159: PyGIDeprecationWarning: GLib.unix_signal_add_full is deprecated; use GLibUnix.signal_add_full instead
  value = getattr(proxy, attr)
WARNING: Glycin running without sandbox.
>>> successfully started...exiting <<<
Forcibly killing verifyapp...
2026/07/29 02:18:20 socat[31810] W exiting on signal 15

Problem running app verifyapp (return code -9).

The OpenSuSE tests are the last platform to run, so the same app code runs happily on Ubuntu, Debian, Fedora and Arch before getting to the OpenSuSE tests; so this seems to suggest pretty strongly that the issue is OpenSuSE.

4. Github Failures

From time to time, Github has issues connecting to the internet to download repo updates for distros. There's very little we can do here.

---

Honestly, I think the most immediate pragmatic solution might be to disable the OpenSuSE runner. OpenSuSE stability has been problem for as long as we've had it in CI; and while it's nice to support it, if we drop it we can shave a couple of minutes of each CI run and get back some stability.

Investigating the source of the Windows failure would also be worthwhile, but I'm not even sure what the next course of action would be on that. It doesn't occur locally; and all the tricks that I know for debugging Github Actions involve getting shell access to the test machine... and tmate doesn't work on Windows, AFAIK. I'm open to suggestions.

[mhsmith]

Since OpenSUSE was disabled, CI on the main branch has failed 9 out of 13 times (July 30 to August 10, not counting interrupted runs). Causes:

• Build app (pyside6, windows-11-arm): 5
• Build app (pygame, windows-latest): 1
• Build app (toga, windows-latest): 1
• GitHub infrastructure failure: 3
• Real failure: 2

Numbers add up to more than 9 because some runs had more than one failure.

Based on that, I think we should disable the pyside6, windows-11-arm job until we work out what's going on.

[freakboy3742]

Based on that, I think we should disable the pyside6, windows-11-arm job until we work out what's going on.

Done in #3000 (although ironically, CI failed on the Windows Console x86-64 pass because GitHub wasn't able to download RCEdit...).