Skip to content

fix: sniff .bin magic before treating weights as pickle - #17

Merged
layer8labs8 merged 2 commits into
layer8labs8:mainfrom
ShahadatTest:fix/bin-magic-bytes
Nov 6, 2026
Merged

layer8labs8 merged 2 commits into
layer8labs8:mainfrom
ShahadatTest:fix/bin-magic-bytes

Conversation

@ShahadatTest

@ShahadatTest ShahadatTest commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Name-only detection treated every *model*.bin as a legacy PyTorch pickle, so gpt-oss Metal weights (GPT-OSS v1.0…) raised QPF014 incorrectly.

Per maintainer review on #6 and PR #17:

  • Read 8 bytes of candidate files (local open / remote Range: bytes=0-7).
  • Invert check: count as pickle unless the first bytes match known non-pickle prefixes (GPT-OSS or GGUF). This ensures classic pickle protocols 0 and 1 (which do not begin with 0x80 or PK) are correctly flagged.
  • Fetch remote legacy weight headers in parallel using concurrent.futures.ThreadPoolExecutor to keep remote scans fast.
  • If the bytes cannot be read or fetch errors out, keep warning (scanner errs toward caution).
  • Added tests for protocol 0 pickle named pytorch_model.bin (local and remote), alongside fake Metal .bin and zip .bin.

Closes #6

Testing

python -m pip install -e ".[dev]"
pytest
# 7406 passed in 18.15s
quantpreflight validate-profiles
# Validated 10 architecture profiles, 15 GPU profiles, and 4 engine profiles.
python tools/run_benchmark.py
# Benchmark: 132 cases, 96.9% recall, 100% precision.

AI disclosure

The original PR was drafted with AI assistance (Cursor cloud agent). This follow-up addressing maintainer review feedback was implemented and validated using Google Gemini / Antigravity at the account owner's request. No independent human review or unexecuted tests are claimed.

Only count name-matched .bin/.pt/.pth files as legacy torch pickle when
the first bytes are a zip (PK) or pickle protocol (0x80). Metal gpt-oss
weights and other non-pickle formats no longer trip QPF014 (layer8labs8#6).

@layer8labs8 layer8labs8 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix, and for the AI disclosure. The code is clean and the tests read well. Before merging, there's one problem in the rule itself, and that's on me, because the issue asked for exactly this.

  1. Pickle protocols 0 and 1 don't start with 0x80. pickle.dumps(obj, protocol=0) starts with c, and loading it runs code just like protocol 2 and later. With this change, a pytorch_model.bin written that way would no longer be flagged, so QPF014 would miss the attack it exists for. Please invert the check: count a file as pickle unless its first bytes match a format we know isn't pickle. For now that's GPT-OSS (gpt-oss Metal weights) and GGUF, so read 8 bytes instead of 4. Unreadable files should still count, as you have it.
  2. Please add a test with a protocol 0 pickle named pytorch_model.bin that must still be reported.
  3. Older repos can have dozens of .bin shards, and the remote check reads them one request at a time. Please fetch the first bytes in parallel, the way loader.py already fetches shard headers with a ThreadPoolExecutor, so a scan doesn't slow down.

I've updated #6 with the corrected rule. Thanks again for picking this up so fast. With these changes it's ready to merge.

Count candidate legacy weight files as pickle unless their first 8 bytes
match known non-pickle prefixes (GPT-OSS or GGUF). This avoids missing
legacy pickle protocols 0 and 1 while preserving zero-warning behavior
for Metal weights. In addition, fetch remote legacy weight headers in
parallel with ThreadPoolExecutor (layer8labs8#6).
@layer8labs8
layer8labs8 self-requested a review November 6, 2026 20:44
@layer8labs8
layer8labs8 merged commit 8e056e7 into layer8labs8:main Nov 6, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check the first bytes of .bin files before calling them pickle weights

2 participants