Pin the packages the images install, and keep them pinned

[hacker-cb]

images/esp-idf/Dockerfile installed its test harness with no version specifier
at all, so what the image carried was decided by the day it was built — one tag has
already shipped pytest-embedded 2.2.1 / pytest 8.4.2 and 2.8.1 / 9.1.1 across two
rebuilds. This pins all nine packages with == and adds the check that keeps them
pinned.

Closes #18, taking option A from
#18 (comment).

The pins, and why esptool is among them

esptool is not a test dependency and nothing here installed it directly. It
arrives through pytest-embedded-serial-esp, whose 2.x line requires
esptool<6,>=5.2, and pip overwrote the copy ESP-IDF ships — while
components/esptool_py/esptool/esptool.py is a nine-line shim around
python -m esptool. So an unpinned test package decided what idf.py flash runs.

The image therefore runs esptool 5.3.1 against ESP-IDF 5.5's own esptool~=4.12.
That divergence is deliberate and documented in the Dockerfile, together with the
three ways of "fixing" it that do not work — editing the constraints file inside
the image (a cache with a one-day TTL), re-pointing PIP_CONSTRAINT at Espressif's
copy (mutable upstream; the pytest pins were already deleted from it), and arming
IDF_PYTHON_CHECK_CONSTRAINTS (every idf.py call would then exit non-zero). It
resolves itself at ESP-IDF 6.0, whose constraints already pair esptool 5.x with
pytest-embedded 2.x.

python -m pip check now closes that layer, and the verification RUN asserts the
exact esptool==<version> rather than the name — pip freeze prints the name
under any version, which is precisely the drift the layer exists to catch.

scripts/check-pins.sh

images/versions.json pins what CI builds; nothing pinned what a build
installs. The new script gates that, per package manager:

manager	rule
pip	== plus an exact version (==9.* is a PEP 440 prefix match, so it is a range); a VCS reference needs a full commit; a URL needs #sha256=; a -r file is followed into the build context and held to the same rule, while a -c file pins nothing by itself
pio	name@version, never @^version — the caret is a range, which is how Unity@^${UNITY_VERSION} ships 2.6.1 under an ARG saying 2.6.0
apt	advisory only. Debian rewrites its pool at every point release, so a pinned version 404s a quarter later; in Ubuntu the only version that never disappears is the release-pocket one, and pinning jq=1.7.1-3build1 rolls back six CVE fixes

Specs are compared after ARG expansion, so a range cannot hide in a Dockerfile
default. Exceptions are # pin-allow: <package> - <reason> comments that name one
package rather than muting a rule, work above or below the install, and fail the
check when they cover nothing.

It runs in lint.yml, not in prepare: a Dockerfile belongs to one image, so an
unpinned package in one must not withhold another image's tags. The
cross-image blast radius is justified for images/versions.json alone, where the
data really is shared.

What it deliberately does not claim. Packages that a script an image runs
installs for itself — esp-matter's ./install.sh --no-host-tool — are outside it,
and the script says so on its own output rather than passing over them in silence.
The closing line reads "every package these Dockerfiles name is pinned".

platformio is exempt for now

Its three unpinned packages carry a pin-allow with the reason. Pinning them
belongs to platformio's own change: protobuf and jinja2 have no consumer in the
image (neither PlatformIO core nor the espressif32 platform requires them), so
"keep or drop" comes before "which version"; and dropping the caret from Unity
changes what the image contains and wants the matching ARG bump beside it.

Also

• esp-matter regenerates /opt/esp/python-packages.txt instead of inheriting it —
  install.sh populates the same venv, so the base's snapshot described an
  environment that image no longer has, while the esp-idf README sends readers to
  that very file.
• .github/workflows/lint.yml's justification for keeping hadolint advisory was
  factually wrong on two of its three points (no apt package ever passes through a
  build arg, and SC3046 is a state-tracking bug that appears after an ENV, not a
  missing SHELL). Replaced with the measured reasons.
• .github/dependabot.yml said the pip installs carry no constraints, which stops
  being true with this change; it now says why pinning them still changes nothing
  for Dependabot (its pip ecosystem reads manifests, never a pip install line).

Verified locally

• ./scripts/build.sh esp-idf on both IDF lines (v5.5.5 and v5.4.1): builds,
  pip check clean, idf.py --version still exits 0, pytest --help still carries
  --embedded-services, esptool 5.3.1 in both.
• The esptool assertion confirmed to fail against a 4.12.0 freeze.
• check-pins.sh against 16 fixtures covering every rule and every false-positive
  case above; check-versions.sh, actionlint, shellcheck (clean at every
  level, not just --severity=error), hadolint (DL3013 is gone from esp-idf; the
  remaining findings are the documented deliberate ones).

esp-matter is not buildable locally in reasonable time — CI covers it, which is the
point of it being build-validated on pull requests.

[Copilot]

Pull request overview

This PR makes the developer Docker images reproducible by pinning package installs (especially the ESP-IDF pytest harness) and adding a repo-level lint gate to prevent future “floating” dependencies from silently changing image contents across rebuilds.

Changes:

• Pin the ESP-IDF pytest harness (and its transitive flasher dependency esptool) to exact versions and add stronger verification (pip check, frozen package snapshot).
• Add scripts/check-pins.sh and wire it into CI (🧹 Lint) and local linting to enforce pinning rules per package manager (pip/pio gated, apt advisory).
• Update documentation and guidance (root README, image READMEs, CLAUDE.md, Dependabot notes) to reflect the new pinning/verification regime.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
scripts/lint.sh	Adds local execution of the new pin-check gate alongside existing lint steps.
scripts/check-pins.sh	New enforcement script that validates pinned install specs in Dockerfiles with pin-allow exceptions.
README.md	Documents check-pins.sh and clarifies which lint checks are gates vs advisory.
images/versions.json	Notes that Dockerfile-level package pins are validated via check-pins.sh rather than versions.json.
images/platformio/Dockerfile	Adds pin-allow exceptions for currently-unpinned pip/pio installs (protobuf/jinja2/Unity).
images/esp-matter/Dockerfile	Regenerates the python package freeze in the derived image and surfaces esptool/pytest info in verification.
images/esp-idf/README.md	Documents the fully-pinned harness and explains the esptool pin rationale and where to inspect the freeze.
images/esp-idf/Dockerfile	Pins the full pytest harness + esptool, adds pip check, and asserts esptool exact version in the freeze.
CLAUDE.md	Codifies the “pin what you install; verify what you pin” guidance and the pinning rules per manager.
.github/workflows/lint.yml	Adds a CI gate step running ./scripts/check-pins.sh and updates lint rationale text.
.github/dependabot.yml	Updates notes to reflect that Dockerfile pip pins are now explicit but still not Dependabot-managed.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.