Skip to content

install-deps: pre-seed bzip2 tarball for vcpkg under 1ES network isolation - #910

Merged
hmmorales merged 2 commits into
developfrom
hmmorales/fix-vcpkg-bzip2-network-isolation
Jul 28, 2026
Merged

install-deps: pre-seed bzip2 tarball for vcpkg under 1ES network isolation#910
hmmorales merged 2 commits into
developfrom
hmmorales/fix-vcpkg-bzip2-network-isolation

Conversation

@hmmorales

Copy link
Copy Markdown
Member

Problem

The platform build's amd64 legs (debian11/12/13-amd64, ubuntu2204/2404-amd64) fail in Install additional dependencies because of 1ES network isolation on the amd64 build pool.

do_install_delta() in scripts/install-deps.sh builds a set of vcpkg ports in classic mode (zlib, zstd, bzip2, gtest, openssl, e2fsprogs, jsoncpp, libconfig, fmt, bsdiff). Every one of those ports fetches its sources from github.com — which the pipeline reaches fine — except bzip2, whose vcpkg portfile downloads bzip2-1.0.8.tar.gz from sourceware.org (with a www.mirrorservice.org fallback).

Under 1ES isolation the amd64 pool only reaches an allowlist of package mirrors. Neither sourceware host is on it, so the download times out and vcpkg install bzip2 fails with BUILD_FAILED, which aborts the entire dependency install and fails every amd64 leg.

Downloading https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz -> bzip2-1.0.8.tar.gz...
...
error: curl failed to download ... Connection timed out after ... ms
Building bzip2:x64-linux failed with: BUILD_FAILED

Note: this only became visible after the sibling apt-mirror fix let the build progress past the earlier apt failure. The delta/vcpkg git clones already succeed, confirming github.com is reachable — bzip2's non-github distfile is the lone gap.

Fix

Pre-seed vcpkg's download cache with the byte-for-byte identical tarball fetched from an allowlisted distro archive pool before running the vcpkg installs:

  • Debian containers reach deb.debian.org; Ubuntu containers reach azure.archive.ubuntu.com — both carry bzip2_1.0.8.orig.tar.gz, which is identical to sourceware's bzip2-1.0.8.tar.gz (verified: 810029 bytes, same SHA512).
  • The file is written to $VCPKG_ROOT/downloads/bzip2-1.0.8.tar.gz — the exact filename the port's vcpkg_download_distfile(... FILENAME "bzip2-${VERSION}.tar.gz") expects. vcpkg then validates the SHA512 and skips its upstream download.

Properties:

  • SHA512-verified against the value pinned in the vcpkg baseline (e0edebd, bzip2 1.0.8 port-version 6) — 083f5e67…a30b9f3. A mismatch discards the file.
  • Idempotent — skips if the cache is already present and valid.
  • Soft-fail — if no mirror is reachable, it warns and lets vcpkg attempt its normal upstream download (no worse than today).
  • No-op on non-isolated pools — the cache pre-seed simply saves vcpkg a download.

Only bzip2 is affected because it is the only delta dependency whose vcpkg port pulls from a non-github, non-allowlisted host.

Validation

  • bash -n scripts/install-deps.sh passes.
  • Functional dry-run of the pre-seed block against a temp VCPKG_ROOT: cold cache downloads from deb.debian.org, SHA512 verifies, warm cache skips, no leftover .tmp.
  • Confirmed the vcpkg baseline pinned by this script (e0edebd1dc2d03cf7d02349df91de74ef4d0c00e) resolves bzip2 to 1.0.8 with FILENAME "bzip2-1.0.8.tar.gz" and SHA512 083f5e67…a30b9f3, matching the pre-seed exactly.
  • End-to-end pipeline validation of the amd64 legs is in progress.

/cc @adu-client

…ation

The delta-library dependency install (`do_install_delta`) builds a set of
vcpkg ports in classic mode. Every port fetches its sources from github.com
except bzip2, whose vcpkg portfile downloads bzip2-1.0.8.tar.gz from
sourceware.org (with a www.mirrorservice.org fallback). Under 1ES network
isolation the amd64 build pool only reaches an allowlist of package mirrors;
neither sourceware host is on it, so the download times out and
`vcpkg install bzip2` fails with BUILD_FAILED, breaking the whole dependency
install and every amd64 leg of the platform build.

Pre-seed vcpkg's download cache with the byte-for-byte identical tarball,
fetched from an allowlisted distro archive pool (deb.debian.org for Debian
containers, azure.archive.ubuntu.com for Ubuntu containers) under the exact
filename the port expects (bzip2-1.0.8.tar.gz). vcpkg validates the SHA512
and skips its upstream download. The pre-seed is SHA512-verified, idempotent,
and soft-fails (warns and lets vcpkg try upstream) if no mirror is reachable,
so it is a no-op on non-isolated pools.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f9dea48f-3bbf-423c-b8cd-4c78631b2d76
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f9dea48f-3bbf-423c-b8cd-4c78631b2d76
@hmmorales
hmmorales merged commit 4f3e5c1 into develop Jul 28, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants