Add R feedstock v0 -> v1 recipe migration: GenericV0ToV1Migrator + RV0ToV1Migrator

[pb01ka]

Description

This is step 2 of the R feedstock v0 -> v1 migration plan, split into two layers, both planned in this PR:

• GenericV0ToV1Migrator (implemented): a MiniMigrator class that only knows the conda-recipe-manager (crm) conversion mechanics. It has no R-specific logic.
• RV0ToV1Migrator (in progress, to be pushed in next commit of this PR): an R-specific subclass layered on top of the base class, adding the R-specific pre/post-processing and registered for R feedstocks. See TODOs below for what's still outstanding.

Why split into a base class + subclass

Keeping the crm-mechanics layer free of R-specifics means:

• It's independently testable against crm's actual behavior (raised exceptions vs. warnings vs. errors), without needing R recipe fixtures.
• It's reusable if we migrate other package clusters through the same v0 -> v1 mechanism later.
• RV0ToV1Migrator can extend IGNORED_WARNINGS and add its own pre/post-processing without touching the safe-to-auto-convert decision itself.

TODOs - remaining work in this PR

• Implement RV0ToV1Migrator: layer R-specific pre/post-processing on top of GenericV0ToV1Migrator: {{ compiler('c') }} / {{ stdlib('c') }} / native/posix/m2w64 token cleanup (comparable to what RUCRTCleanup already does for the ucrt migration), and multi-output handling for CRAN "recommended" vs. non-recommended packages.
• Re-run the feasibility sample against a larger set of R feedstocks once RV0ToV1Migrator exists, to catch any remaining edge cases.
• Multi-output recipes (e.g. r-base itself) currently crash crm's render_to_v1_recipe_format() with a raw AttributeError in its dependency-upgrade logic; RV0ToV1Migrator catches this and skips the recipe rather than converting it. Add a local workaround so multi-output CRAN "recommended" vs. non-recommended recipes actually convert instead of always being skipped.

Cross-refs, links to issues, etc:

https://gist.github.com/pb01ka/4d915b2196470ca7c95643c31212b52b

[codecov]

Codecov Report

❌ Patch coverage is 96.63158% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.71%. Comparing base (b0eba38) to head (978a5ac).
⚠️ Report is 20 commits behind head on main.

Files with missing lines	Patch %	Lines
conda_forge_tick/migrators/v0_to_v1.py	92.96%	14 Missing ⚠️
conda_forge_tick/migrators/r_v0_to_v1.py	91.30%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6381      +/-   ##
==========================================
+ Coverage   81.27%   81.71%   +0.43%     
==========================================
  Files         146      150       +4     
  Lines       17537    18013     +476     
==========================================
+ Hits        14253    14719     +466     
- Misses       3284     3294      +10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jaimergp]

Don't we need to flag this somewhere as an error in the bot or something? This may look like a successful migration otherwise 🤔

[pb01ka]

I don't think so. It's because the v1_content is never written into recipe.yaml for this case. The warning being logged is Skipping v0 -> v1 conversion... and the control returns. So migration would never actualise.

May be I am missing something? Do you want to elevate this to an error instead of a warning? Please let me know.

[jaimergp]

Who's going to see the warning? I'm expecting something we can track in its status page (e.g. like https://conda-forge.org/status/migration/?name=supportwindowsarm64platform), so unless it's an error, it will be a skipped feedstock (or even worse, a "successfully migrated" one in the eyes of the bot metadata?). Need to verify that.

[pb01ka]

I see. It makes sense. Here are a couple (exactly two 😆 TBH) of options I would like to propose,

A. Promote it to a real top-level Migrator. i.e., give it a name, build it as a GraphMigrator (AFAICT, it's currently not) with its own effective_graph/pr_limit, and register it in make_migrators.py like ArchRebuild/OSXArm/MigrationYaml. I think this would be the only option matching the exact status-page precedent you shared.

B. Write a per-node status field via a new MiniMigrator-scoped mechanism. AFAIK, pr_info.pre_pr_migrator_status only covers top-level Migrator exceptions. RV0ToV1Migrator.migrate() would write to attrs["pr_info"], and status_report.py would need extending to read and report it.

I would recommend option A - I guess it is what we intend to do anyways (?). Please let me know TY.

[jaimergp]

Ah, true, why is this a minimigrator? 🤔 I had missed that part at the beginning assuming we wanted a full graph migrator.

[isuruf]

I think a mini-migrator makes sense. We can piggy back off r 4.6 migrator

[isuruf]

If the v0->v1 conversion fails, then the PR for r_base 4.6 will still be made right? Then at the end of the migration, we can retry for the handful left.

[pb01ka]

Thanks @isuruf I think your comment is quite helpful in converging on a better plan than the standalone-migrator approach I'd have gone with earlier. Here's where I've landed:

1. Mini-migrator, piggy-backed on the r-base 4.6 migration

Reverting RV0ToV1Migrator from a standalone GraphMigrator approach back to a MiniMigrator approach, and scoping it to the r-base 4.6 rebuild migration specifically. I guess this follows the exact precedent already in this codebase for RUCRTCleanup:

if migration_name.startswith("r-base44"):
    piggy_back_migrations.append(RUCRTCleanup())

The equivalent hook might look like if migration_name.startswith("r-base46"): piggy_back_migrations.append(RV0ToV1Migrator()). Since bumping the r-base pin forces a rebuild of essentially every R feedstock, this gets us close to full coverage of the R feedstock population in one campaign. Importantly, it also combines the r-base 4.6 rebuild and the recipe format conversion into a single PR per feedstock, instead of two separate ones.

One concern worth flagging explicitly: AFAICT, the r-base 4.6 migration doesn't exist yet. I checked that there's no reference to it anywhere in this code base.
Assuming that the r-base 4.6 migration will be created in the near future, and since the make_migrators.py hook is a one-line addition, the RV0ToV1Migrator mini-migrator can be built and merged independently of that, and then piggy-backed onto the r-base 4.6 migration once it exists.
If it already exists, then please point me to it and I'll add the piggy-back hook immediately.

a v0 -> v1 failure shouldn't block the r-base 4.6 PR. I think, as long as RV0ToV1Migrator.migrate() doesn't raise on a skip (it doesn't, it just logs a warning and returns gracefully), the containing r-base 4.6 migrator's PR gets created normally regardless of whether the conversion succeeded, failed, or was skipped for that feedstock.

2. Track failures so the handful left can actually be retried

The catch with (1) alone: AFAIK, a mini-migrator's migrate() return value is discarded by the caller (and also warnings are non-persistent), so no record of which feedstocks were skipped is kept. To retry for the skipped/failed recipe conversions, at the end of the migration wave, we need to have such a record.

So I am thinking of adding a lightweight tracking mechanism (as described in Option B above): when RV0ToV1Migrator.migrate() skips a feedstock, write a small status note into that node's persisted pr_info (e.g. skipped: True + the blocking reason(s)) instead of only logging. It should be just enough data to query "which R feedstocks are still on schema v0, and why" after the migration wave finishes, and drive a manual or scripted retry pass over just those.

Please correct me wherever I'm wrong. TY.

[jaimergp]

Ok Isuru's approach seems sensible, let's do that. I don't think we need to track which ones were skipped. Instead, the 2nd sweep will traverse the full R graph and skip any feedstock that is already recipe.yaml.

[pb01ka]

I see.

I don't think we need to track which ones were skipped. Instead, the 2nd sweep will traverse the full R graph and skip any feedstock that is already recipe.yaml.

It would be slightly wasteful to revisit the recipes (going through all the feedstocks and checking if recipe.yaml already exist for each of those) which are already successfully converted. However, I don't mind doing so.

Final decision - I will keep the RV0ToV1Migrator as a mini-migrator for now. I wouldn't go for step 2 (Track failures so the handful left can actually be retried) in #6381 (comment). It's anyways an independent of the logical implementation of the migrator. So, if the need arises in the future, we can add tracking mechanism later on.

[pb01ka]

I am making this comment just for record purposes. No action necessary (but thoughts/opinions welcome).

With a0e437c, r-kedd was failing with Could not patch unrecognized license: GPL (>= 2). The conversion was failing.

However, with 08ab378, conversion happened. Here are the relevant files,

Original meta.yaml

Click here

{% set version = "1.0.4" %}

{% set posix = 'm2-' if win else '' %}

package:
  name: r-kedd
  version: {{ version|replace("-", "_") }}

source:
  fn: kedd_{{ version }}.tar.gz
  url:
    - https://cran.r-project.org/src/contrib/kedd_{{ version }}.tar.gz
    - https://cran.r-project.org/src/contrib/Archive/kedd/kedd_{{ version }}.tar.gz
  sha256: bb944389a61cff22bb3bdde938a0ff6c16700a1ca725ef36b391b97812c45b26

build:
  noarch: generic
  number: 2
  rpaths:
    - lib/R/lib/
    - lib/

requirements:
  build:
    - r-base

  run:
    - r-base

test:
  commands:
    - $R -e "library('kedd')"  # [not win]
    - "\"%R%\" -e \"library('kedd')\""  # [win]

about:
  home: https://CRAN.R-project.org/package=kedd
  license: GPL (>= 2)
  summary: Smoothing techniques and computing bandwidth selectors of the nth derivative of a probability density for one-dimensional data.
  license_family: GPL3
  license_file: '{{ environ["PREFIX"] }}/lib/R/share/licenses/GPL-3'  # [unix]
  license_file: '{{ environ["PREFIX"] }}\R\share\licenses\GPL-3'  # [win]

extra:
  recipe-maintainers:
    - conda-forge/r

---

Converted recipe.yaml

Click here

schema_version: 1

context:
  version: "1.0.4"
  posix: "'m2-' if win else ''"

package:
  name: r-kedd
  version: ${{ version|replace("-", "_") }}

source:
  url:
    - https://cran.r-project.org/src/contrib/kedd_${{ version }}.tar.gz
    - https://cran.r-project.org/src/contrib/Archive/kedd/kedd_${{ version }}.tar.gz
  sha256: bb944389a61cff22bb3bdde938a0ff6c16700a1ca725ef36b391b97812c45b26
  file_name: kedd_${{ version }}.tar.gz

build:
  number: 2
  noarch: generic
  dynamic_linking:
    rpaths:
      - lib/R/lib/
      - lib/

requirements:
  build:
    - r-base
  run:
    - r-base

tests:
  - script:
      - if: not win
        then: $R -e "library('kedd')"
      - if: win
        then: "\"%R%\" -e \"library('kedd')\""

about:
  license: GPL-2.0-or-later
  summary: Smoothing techniques and computing bandwidth selectors of the nth derivative of a probability density for one-dimensional data.
  license_file: ${{ (env.get("PREFIX") ~ '/lib/R/share/licenses/GPL-3') if unix else (env.get("PREFIX") ~ '\\R\\share\\licenses\\GPL-3') if win else '' }}
  homepage: https://CRAN.R-project.org/package=kedd

extra:
  recipe-maintainers:
    - conda-forge/r

[pb01ka]

The steps remaining for this PR to be marked as "Ready for review",

• Fix the parsing error encountered r-stringi's v0 to v1 conversion - Out of all the packages in r_v1_sample_feedstocks.csv its the only one left to be addressed by the RV0ToV1Migrator.
• Test the RV0ToV1Migrator on a larger sample - ~300 R feedstocks.

[danielnachun]

I have a couple thoughts on this having worked quite a bit on the last two migrations.

First is handling of patches - although rattler-build has improved a lot in this regard, I still occasionally run into failures because an existing patch is formatted in a way that is incompatible with the patching library rattler-build uses. I think this will probably be something that just has to be handled by maintainers manually - Claude is quite good now at regenerating patches to work with what rattler-build wants. But if there is an automated way to fix patches to work with rattler-build that would save some work.

Second is handling of build scripts - we'll definitely have to regenerate build scripts (or better yet, move the build steps in to the recipe itself). Existing build scripts reference ${R} which is now undefined in rattler-build (we could change this but I'm unsure we should). We do rarely have build scripts that have to set environment variables (usually for detecting shared libraries or setting compilation flags). This could be slightly tricky from an automation perspective because sometimes the package might build correctly even without the flags but will incorrectly use a bundled library, for example, instead of the conda-forge one.

At the same time we still have some packages with old build scripts which build the package like normal for all platforms except osx-arm64, where the script instead tries to copy files from a now non-existent prebuilt package. I assume this is a leftover from some sort of short term work around to support osx-arm64 before we had cross-compilation working.

So the question is how much of this the migration bot should try to clean up itself versus flagging the issue for manual intervention?

[pb01ka]

TYSM @danielnachun for highlighting these important points.

So the question is how much of this the migration bot should try to clean up itself versus flagging the issue for manual intervention?

I will complete testing my migrator on a larger sample of R-feedstocks. Subsequently I will use it to figure out answers to your questions (the two points you have highlighted and how much the bot will be able to do on its own) by tomorrow EOD. I hope that will work.

TY again. <3.

[danielnachun]

That's great. If your larger test doesn't already hit some of these snags, I can definitely at least find some recipes with more complicated build.sh scripts as a smoke test for the bot. It's harder to anticipate patches failing.

One other thing I realized now too that can come up - in every R migration we always run into at least a handful of CRAN package which have been archived and are therefore unmaintained (see https://cran.r-project.org/web/packages/RandomFields/index.html for example). Some fraction of those packages will fail to build, and we end up archiving those feedstocks.

Potentially it would be worth checking the package page to see if it's archived and instead of migrating it, have the bot open an issue indicating the feedstock should be archived.

[pb01ka]

Two questions: recipes with a license string crm can't patch

While testing RV0ToV1Migrator against a sample of R feedstocks, a handful failed to convert because crm couldn't map their license field to an SPDX identifier or due to duplicate key in meta.yaml.

1. Ambiguous/custom identifiers with no deterministic SPDX equivalent

• r-stringi: license: FOSS
• r-rttf2pt1: license: LicenseRef-BSDLike

AFAICT, FOSS is most probably a shorthand for "this package is open source," without saying which license. LicenseRef-BSDLike at least hints at a family ("BSD-like"), but not a specific variant. Neither maps deterministically to a single SPDX identifier the way GPL-2 -> GPL-2.0-only or LicenseRef-Unlimited -> Unlimited does, so there isn't a safe automatic mapping to add here.

A few ways we could handle this, roughly in increasing order of acceptability:

1. Skip the conversion (current behavior). Safe, but means these recipes need a manual v0 -> v1 conversion later.
2. Let it through verbatim. crm already leaves the field unchanged when it can't map it, so this is a one-line change (add the warning to IGNORED_WARNINGS). A small concern: the v1 recipe schema expects a real SPDX expression in about.license, and a vague/custom string like FOSS isn't one. This would likely just move the failure from "bot skips the recipe" to "bot opens a PR that fails conda-forge's own recipe-lint/schema validation."
3. Try to infer the real license from the package's own source. For r-stringi, the actual license lives in the upstream package's LICENSE file (e.g. https://github.com/gagolews/stringi/blob/master/LICENSE), which CRAN's DESCRIPTION/meta.yaml doesn't surface. We could fetch and inspect that file (or CRAN's package page) to guess the SPDX identifier automatically. Worth mentioning as an option but I feel it would be a long shot to implement and maintain, and would still be error-prone (e.g. if the license changes upstream).

Would wait for your thoughts on this.

---

2. Unselector-ed duplicate top-level sections (e.g. r-loose.rock)

Separately, r-loose.rock failed with a raw DuplicateKeyException from crm rather than a clean warning. Its meta.yaml has two full, unconditional build: sections (no # [selector] on either) - one declares noarch: generic, the other doesn't. That's not the common key: val # [selector] idiom crm's duplicate-key merge handles.
It's most likely a copy-paste mistake in the source recipe, and worth flagging on its own since (I think) plain YAML semantics silently pick one of the two build: blocks - meaning noarch: generic may already be getting silently dropped in the currently-published package, independent of this migrator.

RV0ToV1Migrator (via GenericV0ToV1Migrator) now detects a genuinely duplicated, unselector-ed top-level section before crm even runs, and reports it clearly:

top-level `build:` section appears 2 times with no selector to distinguish them - likely an authoring mistake in the source recipe that needs manual review

The recipe is still skipped either way - this only replaces the internal DuplicateKeyException with something a maintainer can act on directly. See the exact logs here.

---

Other than these two questions, from my end I am marking this PR as ready for review.

Now I will work on answering @danielnachun's questions.

[pb01ka]

https://github.com/conda-forge/conda-forge-bot/actions/runs/31217780965/job/92995304151?pr=6381 seems to be unrelated failure. A simple re-run should make it pass I guess.

[pb01ka]

If your larger test doesn't already hit some of these snags, I can definitely at least find some recipes with more complicated build.sh scripts as a smoke test for the bot. It's harder to anticipate patches failing.

@danielnachun I found out the following sample for the two points you shared. I will experiment using these sample and get back to you with a concrete well-tested response. If you have any other feedstocks worth experimenting with, please share with me, I will include it in my sample.

r-msqc
r-pca3d
r-cffr
r-juniperkernel
r-systemfonts
r-rcppdpr
r-geodist
r-leadercluster
r-xtensor

[pb01ka]

@danielnachun As promised, here are my thoughts on your comment above.

1. Patches

I think this can stay out of scope for the migrator (please let me know if you think otherwise).
AFAICT, we only rewrite the recipe YAML's structure and never the files a patches: list points at. Also, a patch incompatible with rattler-build's patching library will convert "successfully" and only fail later, at build time.

I also checked how often this actually comes up and AFAICT, only 1 of our sample of ~350 feedstocks - r-websocket - declares a patches: list. That being said, I would still propose a 3-step pipeline (if the problem is way more frequent than I predict using my sample)

• try the patch as-is under rattler-build; only on failure,
  • apply it with plain CLI tools against the unpacked source;
  • then re-derive a rattler-build-compatible patch from the result

---

2. Build scripts

Same as patches, build.sh content passes through completely untouched, by both crm and our migrator.
As expected, almost every R feedstock has a build.sh. So the useful question here is which recurring pattern inside these scripts are worth flagging (and which of those can be automatically fixed by the migrator).
I found the osx-arm64/osx-64 prebuilt-copy related workaround you described in almost third of my sample. Specifically, in the feedstocks I checked (e.g. r-pca3d, r-msqc), the outer if already routes osx-64 to a normal install, so the install_name_tool dylib-path fix - nested inside target_platform == osx-64 in the else branch osx-arm64 falls into - is unreachable dead code. May be, that block was added in before osx-arm64 support existed, so it's very likely never run. Therefore, I flag it for manual review instead of trying to fix it. In this commit, I implemented it as a non-blocking, post-conversion warning (see below),

build.sh's `install_name_tool` dylib-path fixes for osx-64 are commonly nested inside the branch that already excludes osx-64 (the one osx-arm64 falls into) - making them dead code that's likely never run. Verify whether that's the case here, and if so, whether osx-arm64 needs the missing fix.

I think we can extend this going forward: catalog and add detectors for concrete, recurring patterns as we find them - same as osx-arm64 - rather than guessing at build-script complexity up front.

Please let me know your opinions on both of these. Please also note my comment above. TY.

[danielnachun]

For 1, I think it's fine not to try to fix patches - at the very least Claude Code is very good at regenerating the patches so it's not a huge maintenance burden.

For 2, the pattern of checking the platform in scripts like this should be removed entirely - the install_name_tool fix has been irrelevant for a long time as well. The only issue with outright replacing build.sh and bld.bat with an in recipe call to R CMD INSTALL --build . ${R_ARGS} is just that we do occasionally have to set environment variables or make patches with sed.

I think what I should do is just suffer through cloning the conda-forge/feedstocks repo and then having Claude on my machine locally comb through all the build.sh scripts for the R feedstocks to figure out which feedstocks even have this issue and what kind of patterns are there.

The cleanest ideal outcome for this whole thing is that are able to move the vast majority of recipes to have R CMD INSTALL --build . ${R_ARGS} in the recipe file itself, with environment variables being set using the recipe mechanisms instead of export/set. I'd love to see the R feedstocks get as close to declarative as possible, thought that may take multiple iterations to fully achieve.

Other thing that was missed is that I thought ${R}/%R% is undefined in rattler-build (at least it is when I build locally). I would think that would require editing every build.sh/bld.bat to fix that?