Define build_platform and pass it to conda-build and rattler-build

[jaimergp]

Checklist

• Added a news entry with any new deprecations added to the Deprecated section.
• Regenerated schema JSON if schema altered (python -m conda_smithy.schema)
• Regenerated linter documentation if any rules were added, removed or modified (python -m conda_smithy.linter.messages)

Comes from conda-forge/conda-forge.github.io#1940 (comment).

Changes:

• On Windows, define %BUILD_PLATFORM% directly from GHA/AZP job matrix
• We also implement a fallback where BUILD_PLATFORM is inferred from PROCESSOR_ARCHITECTURE
• For conda-build, we export CONDA_SUBDIR=%BUILD_PLATFORM%
• For rattler-build, we pass --build-platform %BUILD_PLATFORM%

Needs these PRs for full effect:

• Wrap native win-arm64 scripts with a start /machine launcher conda/conda-build#6047
• feat: make CMD subprocesses match build_platform architecture when running emulated prefix-dev/rattler-build#2664
• Do not override BUILD_PLATFORM if already defined conda-forge-ci-setup-feedstock#423
• Fix conda platform when using emulation on windows conda/conda#16417

[jaimergp]

cc @isuruf @baszalmstra

[isuruf]

You can just install the win-64 version of rattler-build on a win-arm64 machine. I don't see why this is necessary.

[baszalmstra]

This is necesary because win-64 rattler-build will default to a build environment that is win-64 instead of arm64.

[isuruf]

This is necesary because win-64 rattler-build will default to a build environment that is win-64 instead of arm64.

Which is exactly what we want.

[baszalmstra]

This is my understanding:

We have a bit of a boostrapping problem. We want to use native win-arm64 runners. We need conda-forge-ci-setup to run on win-arm64 runners. conda-forge-ci-setup is an arch specific package with no support for win-arm64. This would require, amongst others, packages like conda which depends on arch specific python packages like py-rattler that need to be build for win-arm64. Since we do not have native win-arm64 runners yet, we need to cross compile them (which is what we have been doing for other packages). But that requires cross-python, which is something we do not want to support for win-arm64 (because its hacked together as far as I understand).

The solution we discussed in the last conda meeting was to enable using win-arm64 runners but install conda-forge-ci-setup for win-64. Emulating the ci setup. This will bring in rattler-build (or conda-build) for win-64, but we use it to build a native package (not cross-compiling from win-64 -> win-arm64). So the build-platform and host-platform will both be win-arm64 but the build will be orchestrated by win-64 rattler-build.

But, rattler-build compiled for win-64 will think the native platform (build-platform) is win-64. So we need to force it to use win-arm64 as the build-platform because we are actually running it on a win-arm64 runner.

@jaimergp Is this correct?

[jaimergp]

You can just install the win-64 version of rattler-build on a win-arm64 machine. I don't see why this is necessary.

That's what we are doing, but as Bas said:

But, rattler-build compiled for win-64 will think the native platform (build-platform) is win-64. So we need to force it to use win-arm64 as the build-platform because we are actually running it on a win-arm64 runner.

---

@jaimergp Is this correct?

100% spot on.

---

This is necesary because win-64 rattler-build will default to a build environment that is win-64 instead of arm64.
Which is exactly what we want.

No, because in that case we have build_platform != target_platform and then we hit a wall with cross_python not being available. If we define build_platform as win-arm64, then we can build Python packages that use recipe.yaml (e.g. see conda-forge/menuinst-feedstock#61). conda-build seems to be picking up the build platform just fine (even if running from win-64 envs) and doesn't need this (example at conda-forge/pycosat-feedstock#37).

---

I also considered creating a separate win-arm64 environment for rattler-build alone and executing the binary without activation, but that diff is larger than needed for this workaround. And worst case, the --build-platform flag is redundant and doesn't add anything.

[baszalmstra]

Ill try this with py-rattler.

[isuruf]

No, because in that case we have build_platform != target_platform and then we hit a wall with cross_python not being available. If we define build_platform as win-arm64, then we can build Python packages that use recipe.yaml (e.g. see conda-forge/menuinst-feedstock#61). conda-build seems to be picking up the build platform just fine (even if running from win-64 envs) and doesn't need this (example at conda-forge/pycosat-feedstock#37).

This should be fixed instead of being worked around by a hack. Maybe change build_platform != target_platform to build_platform != target_platform and not win

[jaimergp]

But I don't see it as a hack 🤔 The build_platform is win-arm64 because we are using an ARM runner on Github Actions (windows-11-arm). If anything the hack is on having the build tool installed on a win-64 environment to work around the bootstrapping problem for now. Once we have all the necessary dependencies available then we can install base for win-arm64 and then the --build-platform directive will be redundant (but not incorrect).

[jaimergp]

Ok after talking with Isuru I've understood why there's a problem with this approach: cmd.exe thinks is running on win-64 (see this line despite everything else being ARM). We'd need to find a way for a win-64 process to start a fresh win-arm64 process, and I don't know if that's possible.

Putting this in draft until then.

[jaimergp]

Ok, there seems to be a way to pass the desired target architecture for the subprocess on Windows: start /machine arm64. So if we contribute a fix to conda-build and rattler-build, this should tell CMD what it's actually running on. Docs do say that's in preview, so we need to check which version the CI runners are on. cc @baszalmstra.

[baszalmstra]

But does that mean that if a win-64 process starts a sub-process in emulation that that subprocess also runs under win-64?

[isuruf]

But does that mean that if a win-64 process starts a sub-process in emulation that that subprocess also runs under win-64?

Yes, this is true universally. For eg: Rosetta 2 and qemu userspace.

[baszalmstra]

So after some investigation I think this is false.

Launching cmd.exe does not make it inherently win-64. On a Windows-on-ARM machine, resolving C:\Windows\System32\cmd.exe normally launches the native ARM64 (or Arm64X) system cmd.exe;
a child can have a different architecture from its parent.

However, PROCESSOR_ARCHITECTURE is not a reliable way to establish that: it is just an inherited/mutable environment variable. In particular, rattler-build currently forwards its own value:

  vars.insert(var.to_string(), std::env::var(var).ok());

So an emulated win-64 rattler-build could pass along AMD64 even if its cmd.exe child is native ARM64.

[baszalmstra]

Yes, this is true universally. For eg: Rosetta 2 and qemu userspace.

How though? If you have an executable that is only available as arm64, rosetta would have to convert arm64 to x64 and back?

[isuruf]

How though? If you have an executable that is only available as arm64, rosetta would have to convert arm64 to x64 and back?

Most binaries on Apple silicon are fat and have both arm64 and x86_64 slices.

[baszalmstra]

So after some investigation I think this is false.

I retract that statement after trying it, I think you're right

[baszalmstra]

Putting this in draft until then.

But is this really a problem for now? We would use this to build the low level packages and can switch to completely native after that.

The compiler activations etc are still properly targeting win-arm64 (as can be seen a few lines up in the logs) which I think is what really matters.

[jaimergp]

Ok, as per the results in https://github.com/jaimergp/windows-11-arm-tests/actions/runs/29523869103/job/87707238242, there's a way to do it with start:

# /b keeps it headless, /wait propagates the exit code
native_wrapper = f"start /b /wait /machine arm64 cmd.exe BATCH_FILE_HERE"
# shell=True because 'start' is a cmd.exe built-in
result = subprocess.run(
    native_wrapper,
    shell=True,
    capture_output=True,
    text=True
)

[h-vetinari]

Not specifically about the win-arm64 migration (and I'll admit I'm not up to speed with the fat binary vs. emulation vs. xyz stuff), I think we should just generally pass the --build-platform to rattler-build; I recently ran into an issue where a v1 translation couldn't use build_platform in the recipe context:, and according to @wolfv, the only way rattler can know about this is by passing the respective CLI argument.

[jaimergp]

So as we stand, I think this PR still has merit, but both conda-build and rattler-build need to patch how CMD is invoked:

• Wrap native win-arm64 scripts with a start /machine launcher conda/conda-build#6047
• feat: make CMD subprocesses match build_platform architecture when running emulated prefix-dev/rattler-build#2664

[isuruf]

It seems like we get build_platform == win-arm64 on a x64 emulated machine on windows-arm64. This is python/cpython#98962

We should first fix that in conda, so that the build_platform reported is correct. Otherwise you get platform == win-arm64 on a win-64 installation of conda. cc @zooba

[isuruf]

rattler/rattler-build doesn't have this problem because it correctly uses the rust functions and not python

[zooba]

It seems like we get build_platform == win-arm64 on a x64 emulated machine on windows-arm64. This is python/cpython#98962

The answer here is what I was saying in the linked issue - either you're querying the "architecture of subprocesses that can be launched on this machine" (platform) or the "architecture of the running process" (sys).

Windows applies emulation layers at every process boundary, so every time you launch a new process it'll check the executable file and see which subsystem it should run under. This has been the design since 16/32-bit were the options, and has been the same for 64/32-bit and now for ARM64/64/32 (on ARM processors). It's always layered for maximum compatibility.

My recommendation would be to always specify the architecture something should build for (CLI arg or env var of your own design), allow overriding it (probably the same arg but exposed to the user/CI script), and default to sysconfig.get_platform (a.k.a. the build target that will be compatible with the running process).

All the Windows compilers are cross-compiling all the time, which is largely why you don't see special support for it - either it's so transparent that it's ignored/forgotten, or it's trivially supported with a single flag, and occasionally it requires a flag and a directory. Swapping the x86 built compiler (the default) for the x64 one makes no difference to the output unless you tell it to build for x64 (which vcvarsall.bat will do for you; hence, transparent). Rust likewise makes you specify the target triple, which encodes the information needed here.

So assume that you need to always specify the target platform, use sysconfig.get_platform as a default, but override it in this case so that the x64 environment can be used to build ARM64 binaries. conda-forge has got by far the best setup for being able to do this easily, you should be able to get a really long way building on regular x64 machines this way.

[jaimergp]

Upstream PRs in conda and conda-build are now merged and scheduled for the July release in the coming days. Marking this as ready again, and will come in full effect once the tools have new releases. This only impacts win-arm64 PRs so I guess we can merge before tools are ready though, and included it in the next smithy release.

[h-vetinari]

Upstream PRs in conda and conda-build are now merged and scheduled for the July release in the coming days.

Do we need to raise the lower bounds for conda/conda-build here?