Skip to content

Automate the recipe upgrades - #612

Merged
psaavedra merged 4 commits into
mainfrom
auh/metadata
Sep 22, 2026
Merged

psaavedra merged 4 commits into
mainfrom
auh/metadata

Conversation

@psaavedra

@psaavedra psaavedra commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Adds a nightly job that runs the auto-upgrade-helper over the layer and proposes each successful upgrade as its own pull request.

This PR also hands seven recipes back to meta-openembedded, which already has all of them at newer versions.

conf/layer.conf gains multimedia-layer in LAYERDEPENDS_webkit, which is a important change for consumers that do not have meta-multimedia in BBLAYERS.

What AUH needed from the layer

AUH skips every recipe without a RECIPE_MAINTAINER, so a run against this layer upgraded nothing at all. conf/include/maintainers.inc supplies the maintainer of every recipe and, for the ones that must not move on their own, a RECIPE_NO_UPDATE_REASON stating why:

  • The WPE WebKit stack. wpewebkit and webkitgtk are driven by .github/scripts/do-release.sh, and cog, libwpe and wpebackend-fdo follow the series pinned in conf/templates/template/presets/wpe-*.conf. An isolated bump of any of them would either be ignored by those pins or break the combination. Without this, a run proposes cog 0.18.5 -> 0.19.1 and webkitgtk 2.50.1 -> 2.51.1, which are development series, and wpebackend-rdk 1.20221201 -> 5.3.0, which is not a comparable version at all.
  • The recipes that follow a SRCREV rather than a release, plus python-is-python3, which has no upstream. Moving those is a decision about which revision the layer ships. ruby-json belongs here despite its _1.8.6.bb file name, since it tracks a SRCREV on the v1.8 branch; a stock run proposed 3.0.2.

conf/templates/template/presets/auh.conf is the build configuration the job uses. It pulls the maintainers file in and pins no version on purpose: a PREFERRED_VERSION would hide the real state of the tree from the upgrade check.

What the job does

.github/workflows/auh-nightly.yml runs at 02:00 UTC on the self-hosted runner, and can be dispatched by hand with a recipe list, a pull request cap, a compile switch and a dry run switch.

.github/scripts/auh-run refreshes a repo workspace, fetches the helper at a pinned revision, renders .github/auh/upgrade-helper.conf.in into the build directory and runs the upgrade attempt. It runs with --skip-compilation, so it fetches the new sources and refreshes the checksums without building the result; the build tests of the resulting pull request are what prove the upgrade. Building inside the job is available as a dispatch option for the cases that deserve it.

.github/scripts/auh-pr turns the results into pull requests, one branch auh/<recipe>_<version> per upgraded recipe. It never proposes the same work twice, through three guards rather than one: the branch already exists, a pull request for that branch was opened before (closed and merged ones included, which is what covers branches deleted on merge), or that version reached the base branch some other way. A rejected upgrade therefore stays rejected, and the permanent form of that decision is a RECIPE_NO_UPDATE_REASON.

AUH_MAX_PRS, two by default, bounds how many pull requests one run opens, since each one starts a full round of build tests on the same runner the job itself uses. The rest are named in the log and wait for the next night.

What was dropped, and why

recipe was here provided by
bubblewrap 0.8.0 meta-oe 0.11.2
xdg-dbus-proxy 0.1.4 meta-oe 0.1.7
dav1d 0.9.1 meta-multimedia 1.5.3
libavif 0.11.1, a git snapshot meta-multimedia 1.4.1
libvpx 1.10.0 meta-oe 1.16.0
highway 1.0.4 meta-oe 1.3.0+git
libjxl 0.8.1, pinned to the 0.8.x branch meta-oe 0.11.2

None of them were preferred on merit: BBFILE_PRIORITY_webkit is 7 against oe-core's 5, so this layer's copy won regardless of being older. A layer that exists for the WebKit ports was holding seven of their dependencies back.

Nothing referred to them by path. wpewebkit, webkitgtk and packagegroup-wpewebkit-depends name them as dependencies, which any layer providing them satisfies.

The carried patches are covered. libavif 1.4.1 ships both CVE fixes that were patched in here, libjxl 0.11.2 ships the CVE-2024-11403 fix, and meta-oe applies the same blank prefix patch to libvpx, rebased onto its own version; its recipe is otherwise this one down to the configure options, other than falling back to generic-gnu where this one named an armv5te or armv6 target.

highway and libjxl were removed together on purpose. highway was here only as a dependency of libjxl, and this libjxl is pinned to 0.8.x and built against a shared highway 1.0.4, while meta-oe builds libjxl 0.11.2 against a static highway 1.3. Removing only one of them would have produced a pair nobody builds. The same reasoning applies to dav1d and libavif.

What the removals imply

meta-multimedia becomes a hard dependency. libavif now comes from there, and the avif PACKAGECONFIG of wpewebkit is on by default, so the layer declares multimedia-layer in LAYERDEPENDS_webkit. A build without meta-multimedia now fails while the layers are read, with a clear message, rather than in the middle of resolving a dependency. The bblayers template already includes it, so CI is unaffected, but downstream users who do not have it will need to add it.

Testing

Verified in a wrynose repo workspace built exactly as the job builds it, with source setup-environment auh qemux86-64 poky layers.webkit conf_v4.auh:

  • devtool check-upgrade-status reports every opt-out with its reason, so a RECIPE_NO_UPDATE_REASON set from a conf file does reach the code path AUH uses.
  • All three fixed recipes resolve a version now: libarchive-zip-perl 1.68, hyphen 2.8.8, which is the latest one upstream published, and openxr 1.1.63.
  • Every removed recipe resolves to its meta-openembedded provider, and wpewebkit, webkitgtk and cog still resolve to this layer.
  • The layer parses with the new LAYERDEPENDS_webkit.

auh-pr was exercised in dry run mode against the output of a real AUH run: subject parsing, branch naming, git am, the rewritten commit, the pull request body, the three guards, the deferred list at the cap, and cleanup after a patch that no longer applies. oelint-adv is clean on the changed recipes and shellcheck -S warning on both scripts.

Not built. No image or recipe was compiled for this branch. The removals are the part that needs it, since wpewebkit and webkitgtk now build against libjxl 0.11.2, libavif 1.4.1, dav1d 1.5.3, libvpx 1.16.0 and highway 1.3 instead of the older pins. The build tests of this pull request are the gate.

Notes for reviewers

Before the job can work, the repository needs an AUH_PR_TOKEN secret and an auh label. GITHUB_TOKEN would open the pull requests, but a pull request it creates does not trigger other workflows, so the build tests would never run on an upgrade.

The bot signs off as itself, meta-webkit CI Bot <meta-webkit-bot@igalia.com>, rather than as a person, since nobody has reviewed the change at that point. AUH_SIGNED_OFF_BY changes that in one place.

webkitgtk cannot be opted out by omission. oe-core's conf/distro/include/maintainers.inc assigns Unassigned <unassigned@yoctoproject.org> to webkitgtk, libwpe and wpebackend-fdo, and the distro conf is parsed after local.conf, so it overrides what this layer says. A non-empty maintainer is all AUH requires, which is why the opt-out has to be an explicit RECIPE_NO_UPDATE_REASON rather than a missing entry.


Prepared with the assistance of Claude Opus 5.

Comment thread .github/scripts/auh-pr Outdated
Add conf/include/maintainers.inc, holding the maintainer of every recipe
in the layer and the list of recipes that are not upgraded automatically,
and conf/templates/template/presets/auh.conf, a preset that pins no
version and pulls that file in.

The auto-upgrade-helper skips every recipe without a RECIPE_MAINTAINER,
so the layer offers it nothing to work on as it stands. Recipes that must
not move on their own carry a RECIPE_NO_UPDATE_REASON instead: the WPE
WebKit stack, whose versions come from .github/scripts/do-release.sh and
from the wpe-*.conf presets, and the recipes pinned by SRCREV. What is
left to the automatic upgrades are the third party support recipes.

Neither file takes part in a normal build. The preset is meant to be used
on its own, since a PREFERRED_VERSION would hide the real state of the
tree from the upgrade check:

    source setup-environment auh qemux86-64 poky layers.webkit conf_v4.auh

Change-Type: patch
AI-Generated: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Set UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX in libarchive-zip-perl
and openxr, and declare UPSTREAM_VERSION_UNKNOWN in hyphen. The three of
them reported no upstream version at all.

openxr aimed its check at the GitHub release downloads, which serve no
listing. It reads the release page now and resolves 1.1.63.

libarchive-zip-perl was worse than uninformative. cpan-base.bbclass
builds the regex out of the whole SRC_URI entry, parameters included,
so the subdir=${BP} parameter makes it define the pver group twice,
the regex fails to compile, and the exception takes down the check for
every recipe in the batch. An explicit regex in the recipe overrides
the generated one, and the check resolves 1.68.

hyphen cannot be checked at all. SOURCEFORGE_MIRROR serves no listing,
the tarballs sit in a per series directory, and the fetcher only walks
those directories while UPSTREAM_CHECK_URI is unset, which leads back
to the mirror. Aiming the check at a single series would answer with
that series for good, which is a silent wrong answer, so the recipe
states that the version cannot be determined instead.

This metadata feeds the automatic upgrades. A recipe that cannot report
a version never gets upgraded, so either the check is fixed or the
recipe says outright that there is nothing to check.

Change-Type: patch
AI-Generated: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Add .github/workflows/auh-nightly.yml and the two scripts behind it.
auh-run refreshes a repo workspace, configures the auto-upgrade-helper
from .github/auh/upgrade-helper.conf.in and runs it over the layer, and
auh-pr turns every successful upgrade into a branch named
auh/<recipe>_<version> and a pull request for it.

Nothing is proposed twice. A recipe is skipped when the branch is there
already, when a pull request for it was opened before, closed ones
included, and when that version reached the base branch some other way.
A rejected upgrade therefore stays rejected, and the permanent form of
that decision is a RECIPE_NO_UPDATE_REASON in
conf/include/maintainers.inc.

AUH_MAX_PRS, two by default, bounds how many pull requests a single run
opens, since each one starts a full round of build tests on the same
runner the job itself uses. The rest are named in the log and wait for
the next night. A dry run applies every patch and prints the commits
and the pull request bodies without pushing anything.

The job refreshes the checksums without building the result, so the
build tests of the pull request are what proves the upgrade. Building
it inside the job is a dispatch option for the cases that deserve it.

Maintenance-Type: ci
AI-Generated: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Remove bubblewrap, dav1d, highway, libavif, libjxl, libvpx and
xdg-dbus-proxy. meta-oe and meta-multimedia carry every one of them at a
newer version, while the copies here went as far back as 2021 and won
anyway, because BBFILE_PRIORITY_webkit outranks both layers. A layer
that exists for the WebKit ports was holding seven of their dependencies
back.

What the layer keeps is a dependency on meta-multimedia, since the avif
PACKAGECONFIG of wpewebkit is on by default and libavif comes from there
now. LAYERDEPENDS_webkit says so, so a build without meta-multimedia
fails while the layers are read rather than in the middle of resolving a
dependency.

Change-Type: major
AI-Generated: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
@psaavedra
psaavedra merged commit 64681dd into main Sep 22, 2026
4 of 7 checks passed
@psaavedra
psaavedra deleted the auh/metadata branch September 22, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants