Skip to content

Surface downstream resolver incompatibilities#117

Merged
ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/surface-downstream-resolver-errors
Jul 22, 2026
Merged

Surface downstream resolver incompatibilities#117
ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/surface-downstream-resolver-errors

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 15, 2026

Copy link
Copy Markdown
Member

Ignore this PR until reviewed by @ChrisRackauckas.

Summary

  • Remove the reusable downstream workflow's ResolverError catch that converted incompatible environments into green jobs.
  • Let dependency resolution and test failures propagate normally.
  • Add regression assertions that forbid the prior success-exit path.

PR #115 has merged, and the shared v1 tag now points to its merge commit f8bbdfe. This branch includes that released workflow state; the remaining diff is the resolver-failure propagation change.

Local verification

  • julia +1.12 --startup-file=no test/runtests.jl — passed; all testsets passed, including monorepo selection 11/11 and resolver propagation 3/3.
  • python3 -m unittest discover -s test -p 'test_resolve_monorepo_tagbot.py' -v — 9/9 passed.
  • Executed the exact YAML run block against two local path packages with mutually incompatible Compat requirements. A real Pkg.Resolve.ResolverError propagated on Julia 1.10 and 1.12.
  • actionlint -color — passed.
  • julia +1.12 --startup-file=no -m Runic --check . — passed.
  • git diff --check — passed.

Default-branch runtime evidence

The latest conventional downstream-workflow run census covered 193 jobs across 30 repositories. Of 186 downloadable logs, 58 concluded success after executing the compatibility catch and emitted zero test summaries. All 58 are listed in the audit record at ChrisRackauckas/InternalJunk#70. A post-release NonlinearSolve rerun independently reproduced the same behavior in all seven matrix rows.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

Copy link
Copy Markdown
Member Author

Added merge commit c37a258 to carry #115's latest caller-side monorepo support without losing resolver propagation. The prior stack had a real content conflict. The combined exact YAML body now passes the two-source-package success fixture and propagates a genuine ResolverError fixture on Julia 1.10 and 1.12; both full repository test runs, actionlint, Runic, and diff checks pass.

Copy link
Copy Markdown
Member Author

CI timer check (2026-07-15 09:46 EDT): the latest workflow run completed successfully. actionlint + shellcheck, monorepo TagBot resolver tests, and compute_affected_sublibraries.jl tests all passed. The PR is draft, mergeable/clean, and git log HEAD..upstream/master --oneline is empty, so the branch is current with master.

This stacked PR still includes #115 and its new reusable-workflow inputs will not be available to callers until the shared workflow changes are merged and the v1 tag is advanced.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

Copy link
Copy Markdown
Member Author

⎿ Setting 1-hour timer for CI completion as per CLAUDE.md instructions...
Current time: 2026-07-16 12:50:40 EDT
Will check CI results at: 2026-07-16 13:50:40 EDT

Copy link
Copy Markdown
Member Author

Additional real-workflow confirmation from the post-#115 v1 release:

  • NonlinearSolve PR #1080 job 87689840720 selected GROUP=All and developed exactly lib/NonlinearSolveBase, lib/SciMLJacobianOperators, and lib/NonlinearSolveFirstOrder into BoundaryValueDiffEq.
  • Pkg.test then hit the real test-environment incompatibility: BoundaryValueDiffEq restricts LinearSolve to 3.x while the developed NonlinearSolveFirstOrder requires LinearSolve 5.
  • The currently released workflow caught that ResolverError, printed Not compatible with this release. No problem., and marked the job green before any BoundaryValueDiffEq test group ran.
  • I replayed the same current BoundaryValueDiffEq + three local NonlinearSolve subprojects on Julia 1.12.6 using this PR's uncaught run block. Pkg.test(coverage=false) exited 1 with the LinearSolve incompatibility, as intended.

This is direct evidence that the removed catch was still producing a silent downstream no-op in a real monorepo integration job. The incompatibility itself is legitimate; the fix here is that it can no longer masquerade as a passing test run.

Copy link
Copy Markdown
Member Author

Expanded log scan of NonlinearSolve PR #1080's completed IntegrationTest run found the same silent-success path in all 7/7 downstream rows: each log contains the executed compatibility-catch message and zero test summaries. The affected declared groups span BoundaryValueDiffEq, DiffEqCallbacks, ModelingToolkit (three groups), and OrdinaryDiffEq (two groups).

In the same post-release window, completed corrected rows in RecursiveArrayTools, MaybeInplace, SciMLLogging, SciMLOperators, and Static produced real Test Summary: output. This isolates the problem to resolver incompatibilities being converted to success, rather than the new group/subdirectory routing itself.

Copy link
Copy Markdown
Member Author

Expanded default-branch evidence confirms this is organization-wide, not an isolated NonlinearSolve case.

For the latest default-branch push run in 30 repositories with conventional Downstream.yml workflows:

  • 193 jobs were recorded.
  • 186 logs were downloadable; 7 returned HTTP 404.
  • 123 logs contained a real Test Summary: or package-passed marker.
  • 5 were visible failures before a final summary.
  • 58 jobs concluded success after executing this resolver catch and emitted zero test summaries.

The released script text itself appears once in every relevant log; these 58 contain the second compatibility-message occurrence from executing the catch. Every one has a green conclusion.

Source distribution: SciMLBase 21, OrdinaryDiffEq 12, NonlinearSolve 5, RecursiveArrayTools 4, LinearSolve 4, SciMLLogging 3, FindFirstFunctions 3, DataDrivenDiffEq 2, and one each in SciMLSensitivity, FunctionProperties, DiffEqProblemLibrary, and FastBroadcast.

The full row list and per-repository counts are archived in InternalJunk #70. This directly supports propagating ResolverError as a red integration result.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 22, 2026 08:10
@ChrisRackauckas
ChrisRackauckas merged commit 995e242 into SciML:master Jul 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants