Repository navigation
fix: sniff .bin magic before treating weights as pickle - #17
Merged
Merged
Conversation
Only count name-matched .bin/.pt/.pth files as legacy torch pickle when the first bytes are a zip (PK) or pickle protocol (0x80). Metal gpt-oss weights and other non-pickle formats no longer trip QPF014 (layer8labs8#6).
layer8labs8
requested changes
Nov 3, 2026
layer8labs8
left a comment
Owner
There was a problem hiding this comment.
Thanks for the quick fix, and for the AI disclosure. The code is clean and the tests read well. Before merging, there's one problem in the rule itself, and that's on me, because the issue asked for exactly this.
- Pickle protocols 0 and 1 don't start with
0x80.pickle.dumps(obj, protocol=0)starts withc, and loading it runs code just like protocol 2 and later. With this change, apytorch_model.binwritten that way would no longer be flagged, soQPF014would miss the attack it exists for. Please invert the check: count a file as pickle unless its first bytes match a format we know isn't pickle. For now that'sGPT-OSS(gpt-oss Metal weights) andGGUF, so read 8 bytes instead of 4. Unreadable files should still count, as you have it. - Please add a test with a protocol 0 pickle named
pytorch_model.binthat must still be reported. - Older repos can have dozens of
.binshards, and the remote check reads them one request at a time. Please fetch the first bytes in parallel, the wayloader.pyalready fetches shard headers with aThreadPoolExecutor, so a scan doesn't slow down.
I've updated #6 with the corrected rule. Thanks again for picking this up so fast. With these changes it's ready to merge.
Count candidate legacy weight files as pickle unless their first 8 bytes match known non-pickle prefixes (GPT-OSS or GGUF). This avoids missing legacy pickle protocols 0 and 1 while preserving zero-warning behavior for Metal weights. In addition, fetch remote legacy weight headers in parallel with ThreadPoolExecutor (layer8labs8#6).
layer8labs8
self-requested a review
November 6, 2026 20:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Name-only detection treated every
*model*.binas a legacy PyTorch pickle, so gpt-oss Metal weights (GPT-OSS v1.0…) raised QPF014 incorrectly.Per maintainer review on #6 and PR #17:
Range: bytes=0-7).GPT-OSSorGGUF). This ensures classic pickle protocols 0 and 1 (which do not begin with0x80orPK) are correctly flagged.concurrent.futures.ThreadPoolExecutorto keep remote scans fast.pytorch_model.bin(local and remote), alongside fake Metal.binand zip.bin.Closes #6
Testing
AI disclosure
The original PR was drafted with AI assistance (Cursor cloud agent). This follow-up addressing maintainer review feedback was implemented and validated using Google Gemini / Antigravity at the account owner's request. No independent human review or unexecuted tests are claimed.