Skip to content

Pin protoc so the build is reproducible - #73

Merged
psobot merged 3 commits into
masterfrom
psobot/pin-protoc
Aug 7, 2026
Merged

Pin protoc so the build is reproducible#73
psobot merged 3 commits into
masterfrom
psobot/pin-protoc

Conversation

@psobot

@psobot psobot commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Problem

Building the generated code needed a protoc on $PATH, unpinned. protoc 25+ embeds a runtime-version check into its output that refuses to load under an older google.protobuf than the one it was built against — so a contributor with a current Homebrew protoc (35.x) gets generated code that won't import in the same virtualenv that just built it:

google.protobuf.runtime_version.VersionError: Detected incompatible Protobuf
Gencode/Runtime versions ... gencode 7.35.1 runtime 6.32.1

That reads as "this repo is broken" rather than "your protoc is too new". Combined with the README pointing at a dumper/Makefile that has never existed, this is #67.

Fix

dumper/run.py downloads and uses a pinned protoc 21.9 — the version CI already used — into .protoc/, and deliberately does not fall back to $PATH.

21.9 predates the runtime-version check, which keeps protobuf>=3.13.0 an honest floor for end users. Pinning a modern protoc instead would have forced every installed user up to protobuf>=5.28.1; that's a separate decision, not a side effect of fixing the build.

--protoc /path/to/protoc overrides.

Also in here

  • dumper.protodump is now imported lazily. It depends on Protobuf internals removed in protobuf 4 (SkipField), and importing it at module scope made even the compile-only path — which needs none of that — fail on a modern protobuf. Recompiling the checked-in protos now works on any supported protobuf; only the --app-path extraction step still needs protobuf<4, which is documented and declared in the script header.
  • check=True on the protoc call. A protoc failure previously left a half-populated gencode directory and let the build carry on.
  • Workflow cleanup: removed the now-redundant protoc download, the ::set-env command (disabled by GitHub since 2020, kept alive only by ACTIONS_ALLOW_UNSECURE_COMMANDS), and a pip install protobuf>=4 that contradicted the protobuf<4 pin installed two lines later.
  • README: documents rebuilding gencode and adding a new Keynote version, replacing the cd dumper && make clean && make instructions.

Verification

From a clean tree with no .protoc/ and no gencode, under protobuf==6.32.1 (which failed before this change):

INFO  Downloading protoc 21.9 from ...
INFO  Using protoc: .../.protoc/21.9/bin/protoc
INFO  Done!
20 passed

Second run reuses the cached protoc without re-downloading.

Fixes #67.

psobot and others added 2 commits August 7, 2026 10:29
Building the generated code required a `protoc` on $PATH, unpinned. protoc 25
and newer embed a runtime version check into their output that refuses to load
under an older google.protobuf than the one they were built against, so a
contributor with a current Homebrew protoc (35.x) gets gencode that fails to
import in the very same virtualenv that built it:

    google.protobuf.runtime_version.VersionError: Detected incompatible
    Protobuf Gencode/Runtime versions ... gencode 7.35.1 runtime 6.32.1

which reads as "this repo is broken" rather than "your protoc is too new".

dumper/run.py now downloads and uses a pinned protoc 21.9 - the same version CI
already used - from .protoc/, and deliberately does not fall back to $PATH.
21.9 predates the version check, which keeps `protobuf>=3.13.0` an honest floor
for end users rather than silently requiring a much newer runtime. `--protoc`
overrides it.

Also:
 - Import dumper.protodump lazily. It uses Protobuf internals removed in
   protobuf 4, and importing it at module scope made even the compile-only path
   (which needs none of that) fail on a modern protobuf.
 - Pass check=True to protoc, so a compile failure stops the build instead of
   leaving a half-populated gencode directory behind.
 - Drop the now-redundant protoc download and the disabled ::set-env command
   from both workflows, along with a `pip install protobuf>=4` that contradicted
   the pin installed two lines later.
 - Document how to rebuild gencode and how to add a new Keynote version. The
   README told people to run `cd dumper && make`, and there has never been a
   dumper/Makefile.

Fixes #67.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179M4xvAKPGrgKpy4AeCsM7
CI installs ruff unpinned and the repo declares no rule selection, so the lint
gate was whatever that month's ruff considered default. Ruff's defaults have
since grown well beyond the original E4/E7/E9/F, and `ruff check .` now reports
125 errors on unmodified master - across LOG015, UP031, BLE001, SIM115 and
others - so every pull request fails lint for reasons that have nothing to do
with its contents.

Select the rules explicitly and pin ruff in CI. The selection is exactly what
master already passes, so this restores the gate to what it was actually
enforcing rather than silently adopting a much stricter one; broadening it is
now a deliberate edit instead of a side effect of a ruff release.

target-version is py310 rather than the py39 inferred from requires-python,
because dumper/extract_mapping.py already uses a match statement and the CI
matrix starts at 3.10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179M4xvAKPGrgKpy4AeCsM7
…-protoc

# Conflicts:
#	.github/workflows/python-package.yml
@psobot
psobot merged commit c2c5de7 into master Aug 7, 2026
4 checks passed
@psobot
psobot deleted the psobot/pin-protoc branch August 7, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Protobuf update commands with make not working

1 participant