Burrito as a NIF package manager

[dbernheisel]

Proposal: nif_deps — Cross-compilation C library management for NIFs

Problem

Burrito cross-compiles NIFs using zig, but NIFs that depend on third-party C libraries (e.g., libusb, openssl, zlib) fail at runtime. There are two failure modes:

1. Host library paths leak into cross-builds. Makefiles and rebar.config invoke pkg-config which returns host-platform paths (e.g., macOS Homebrew paths). The linker either uses incompatible host libraries or produces a .so with missing NEEDED entries.

2. musl/glibc mismatch at runtime. Burrito targets x86_64-linux-musl for Linux, producing statically-linked binaries. When a cross-compiled NIF loads system libraries from a glibc-based distro (e.g., Debian), the glibc-specific symbols (like __snprintf_chk) are not available in musl, causing the NIF to fail to load.

Both issues share the same root cause: cross-compiled NIFs need target-platform C libraries available at build time, but only host-platform libraries exist on the build machine.

Solution

Add a nif_deps key to Mix.Project.config() that declares the C library dependencies required by NIFs. Burrito fetches, caches, and provides these libraries to the NIF build via pkg-config configuration when recompiling for the release.

Configuration

def project do
  [
    app: :my_app,
    version: "0.1.0",
    deps: deps(),
    releases: releases(),
    nif_deps: nif_deps()
  ]
end

defp nif_deps do
  libusb_alpine = [alpine: [package: "libusb-dev", branch: "v3.21", repo: "main"]]

  [
    {"libusb-1.0",
      targets: [
        linux_amd64: [alpine: [package: "libusb-dev", branch: "v3.21", repo: "main"]],
        linux_arm64: [alpine: [package: "libusb-dev", branch: "v3.21", repo: "main"]],
        macos_arm64: [homebrew: [formula: "libusb"]],
        macos_x86:   [homebrew: [formula: "libusb"]],
        windows:     [url: "https://github.com/libusb/libusb/releases/download/v1.0.27/libusb-1.0.27.7z"]
      ]
    }
  ]
end

Each entry is a tuple of {pkg_config_name, opts}:

• pkg_config_name — the name that Makefiles/rebar.config pass to pkg-config (e.g., "libusb-1.0", "openssl", "zlib")
• targets: — a keyword list keyed by the target aliases from the release config, each specifying a source

Source types

Each target specifies one source for its libraries:

Alpine (for musl Linux targets):

[alpine: [package: "libusb-dev", branch: "v3.21", repo: "main"]]

Required keys: package, branch, repo. Arch is derived from the Burrito target.

Debian (for glibc Linux targets):

[debian: [package: "libusb-1.0-0-dev", release: "bookworm", component: "main"]]

Required keys: package, release, component. Arch is derived from the Burrito target.

Homebrew (for macOS targets):

[homebrew: [formula: "libusb"]]

Required keys: formula. Arch and macOS version are derived from the Burrito target.

Direct URL (for any target):

[url: "https://example.com/libfoo-1.0-linux-x86_64.tar.gz"]

The archive must contain headers, static libraries (.a), and ideally .pc files.

Resolution and caching

Burrito resolves packages from ecosystem-specific indexes:

Source	Index URL	Size	Key fields
Alpine	https://dl-cdn.alpinelinux.org/alpine/v{branch}/{repo}/{arch}/APKINDEX.tar.gz	~800KB	p:pkg-config({name}) provides mapping
Debian	https://deb.debian.org/debian/dists/{release}/{component}/Contents-{arch}.gz + Packages.gz	~30MB + ~15MB	pkgconfig/{name}.pc file path search
Homebrew	https://formulae.brew.sh/api/formula/{formula}.json	~5KB per formula	bottle.stable.files.{arch} for download URL
URL	Direct download	—	—

Everything is cached in deps/_burrito_nifs/:

deps/_burrito_nifs/
  _indexes/
    alpine/v3.21/main/x86_64/APKINDEX
    alpine/v3.21/main/aarch64/APKINDEX
    debian/bookworm/main/amd64/Contents
    debian/bookworm/main/amd64/Packages
  libusb-1.0/
    x86_64-linux-musl/
      usr/include/libusb-1.0/libusb.h
      usr/lib/libusb-1.0.a
      usr/lib/pkgconfig/libusb-1.0.pc
    aarch64-macos/
      include/libusb-1.0/libusb.h
      lib/libusb-1.0.dylib
      lib/pkgconfig/libusb-1.0.pc
    x86_64-windows/
      include/libusb-1.0/libusb.h
      lib/libusb-1.0.lib

How it integrates with NIF compilation

Before invoking rebar3 (i have a branch that tries to support rebar3, separate effort) or elixir_make for a cross-build target, Burrito:

1. Reads nif_deps from Mix.Project.config()
2. For the active target, checks which nif_deps entries have a matching target key
3. Ensures each package is fetched and cached (downloads if missing)
4. Sets PKG_CONFIG_LIBDIR to point at the cached .pc files for the active target
5. Sets PKG_CONFIG_SYSROOT_DIR to the cached library root

When the dep's Makefile runs pkg-config --libs libusb-1.0, it finds the cached musl-compiled .pc file which points at the musl-compiled static .a — no host paths leak in, no glibc dependency at runtime.

Detection and warnings

Burrito scans each NIF dep's build files (Makefiles, rebar.config) for pkg-config invocations. Any detected pkg-config dependency not covered by nif_deps produces a warning before compilation:

--> Detected pkg-config dependency "libusb-1.0" in usb (from Makefile)
--> WARNING: "libusb-1.0" not declared in nif_deps — NIF may fail at runtime on cross-compiled targets

This helps users discover what they need to declare without trial and error.

Example: full project with multiple NIF deps

defmodule MyApp.MixProject do
  use Mix.Project

  def project do
    [
      app: :my_app,
      version: "0.1.0",
      elixir: "~> 1.18",
      deps: deps(),
      releases: releases(),
      nif_deps: nif_deps()
    ]
  end

  def releases do
    [
      my_app: [
        steps: [:assemble, &Burrito.wrap/1],
        burrito: [
          targets: [
            macos:       [os: :darwin, cpu: :aarch64],
            linux_amd64: [os: :linux, cpu: :x86_64],
            linux_arm64: [os: :linux, cpu: :aarch64],
            windows:     [os: :windows, cpu: :x86_64]
          ]
        ]
      ]
    ]
  end

  defp nif_deps do
    alpine_3_21 = [branch: "v3.21", repo: "main"]

    [
      {"libusb-1.0",
        targets: [
          linux_amd64: [alpine: [package: "libusb-dev"] ++ alpine_3_21],
          linux_arm64: [alpine: [package: "libusb-dev"] ++ alpine_3_21],
          macos:       [homebrew: [formula: "libusb"]],
          windows:     [url: "https://github.com/libusb/libusb/releases/download/v1.0.27/libusb-1.0.27.7z"]
        ]
      },
      {"openssl",
        targets: [
          linux_amd64: [alpine: [package: "openssl-dev"] ++ alpine_3_21],
          linux_arm64: [alpine: [package: "openssl-dev"] ++ alpine_3_21],
          macos:       [homebrew: [formula: "openssl@3"]],
          windows:     [url: "https://slproweb.com/download/Win64OpenSSL-3_4_1.exe"]
        ]
      }
    ]
  end

  defp deps do
    [
      {:usb, "~> 0.2"},
      {:some_ssl_nif, "~> 1.0"},
      {:burrito, path: "../burrito"}
    ]
  end
end