Rebuilds into an existing out/ are fragile: clean bundle_dir and guard the copied project tree

[timholy]

Building the OLS tutorial example (#35) a second time into an existing out/ — and, separately, with a Python venv created inside the entry-project directory — surfaced two build failures. Both ultimately come from upstream behavior in JuliaC / PackageCompiler, but JuliaLibWrapping chooses the layout and orchestrates the bundle step, so it can defend against them.

1. Bundle output is not idempotent (cert.pem already exists)

A rebuild without first clearing out/ fails:

ERROR: ArgumentError: '/tmp/lib/out/ols-bundle/share/julia/cert.pem' exists. `force=true` is required to remove ... before copying.
  [5] bundle_cert @ PackageCompiler.jl:1687
  [6] bundle_products(recipe::BundleRecipe) @ JuliaC .../bundling.jl:23
  [7] _build_library_juliac(...) @ JuliaLibWrappingJuliaCExt.jl:44
  [9] build_library(...) @ src/build.jl:117

PackageCompiler.bundle_cert does a plain cp(cert, dst) with no force=true, so it refuses to overwrite a pre-existing bundle. juliac writes into bundle_dir (defaulting to joinpath(libdir, libname * "-bundle"), i.e. out/<lib>-bundle), but build_library does not clean that directory before invoking the backend at src/build.jl:117.

Notably, _copy_bundle_into_python_package already wipes its own destination for exactly this reason (src/build.jl:152-155):

# Wipe any stale copy so a smaller-than-before bundle does not leave
# orphan files behind ...
ispath(dest) && rm(dest; recursive = true)
cp(bundle_dir, dest)

Recommendation: mirror that wipe for bundle_dir itself — ispath(bundle_dir) && rm(bundle_dir; recursive=true) before the ext._build_library_juliac call when bundle is true — so a rebuild into a populated out/ is idempotent. (Or, upstream, have JuliaC/PackageCompiler clear the bundle destination / pass force=true.)

2. Whole-project copy + recursive chmod following symlinks (EPERM)

juliac copies the entire entry-project directory to a temp dir and then walks the tree chmod-ing every entry (JuliaC compiling.jl:99-105), following symlinks. Two consequences with standard_build's default layout, where out = joinpath(dir, "out") is nested inside project = dir:

• Wasteful: every rebuild recursively copies the previous out/, including the full bundle/ runtime closure (libjulia, stdlibs, BLAS, …), into the temp project.
• Fragile: any symlink in the project that points at an unwritable target breaks the build. Concretely, a venv created inside the project (python/bin/python -> /usr/bin/python) yields:

ERROR: IOError: chmod("/tmp/jl_XXXX/python/bin/python", 0o100755): operation not permitted (EPERM)
  [5] compile_products(recipe::ImageRecipe) @ JuliaC .../compiling.jl:102
  ...
  [8] build_library(...) @ src/build.jl:117

(The chmod follows the symlink to the root-owned system Python.)

Recommendations (any subset):

• Add a fail-fast preflight, a sibling of _validate_sources_absolute, that rejects/​warns when the project directory contains symlinks to unwritable targets or a nested output dir — with a message pointing the user to keep venvs and build artifacts out of the entry project.
• Reconsider nesting the default out/ inside the copied project tree (e.g. default it to a sibling of dir, or otherwise keep it out of what juliac copies).
• At minimum, document the gotcha: the entry-project directory is copied wholesale by juliac, so it must not contain venvs or large generated trees.

Repro

# in an ols-style example dir
julia --project=. build.jl          # first build: OK
python -m venv ./python              # venv created *inside* the project dir
julia --project=. build.jl          # -> EPERM on python/bin/python
rm -rf ./python
julia --project=. build.jl          # -> cert.pem 'exists' from the prior bundle

Upstream

The chmod-follows-symlink behavior (JuliaC compiling.jl) and the cp-without-force in PackageCompiler.bundle_cert are arguably upstream bugs; worth cross-linking if filed there. This issue tracks the defensive changes JuliaLibWrapping can make regardless.

🤖 Generated with Claude Code