Repository navigation
Fix ModelPruning._make_pruning_permanent_on_state_dict suffix stripping and O(M^2) sanitization - #22045
Open
deep1blue wants to merge 1 commit into
Open
Fix ModelPruning._make_pruning_permanent_on_state_dict suffix stripping and O(M^2) sanitization#22045deep1blue wants to merge 1 commit into
ModelPruning._make_pruning_permanent_on_state_dict suffix stripping and O(M^2) sanitization#22045deep1blue wants to merge 1 commit into
Conversation
…ng and O(M^2) sanitization
1. In ModelPruning._make_pruning_permanent_on_state_dict(),
k.replace("_mask", "") replaced all occurrences of "_mask" in the
state_dict key rather than only the trailing "_mask" suffix. When any
pruned submodule or parameter path contains "_mask" (e.g.,
self.attention_mask_proj = nn.Linear(...) with keys
attention_mask_proj.weight_orig and attention_mask_proj.weight_mask),
k.replace("_mask", "") produced attention_proj.weight and caused
state_dict.pop(tensor_name + "_orig") to crash with KeyError during
checkpoint saving. Use k.removesuffix("_mask") so only the trailing
suffix is stripped.
2. In ModelPruning.sanitize_parameters_to_prune(), convert
current_modules to a set before validating explicit
(module, name) pairs (O(1) lookup per pair instead of O(M) list
scan), and avoid calling getattr(m, p, None) twice per (p, m)
pair when parameters_to_prune is defaulted.
Signed-off-by: Parth Vadera <parthvadera@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
deep1blue
requested review from
ethanwharris,
justusschock and
tchaton
as code owners
October 22, 2026 15:16
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes a$O(M^2)$ linear list scan in
KeyErrorbug when saving checkpoints withModelPruning(..., make_pruning_permanent=True)on models whose submodule names contain"_mask", and removes anModelPruning.sanitize_parameters_to_prune:"_mask"suffix in_make_pruning_permanent_on_state_dict:map_pruned_params = {k.replace("_mask", "") for k in state_dict if k.endswith("_mask")}replaced every occurrence of"_mask"in the key string rather than only the trailing"_mask"suffix added bytorch.nn.utils.prune."_mask"(for example,self.attention_mask_proj = nn.Linear(...), whose prunedstate_dictkeys are"attention_mask_proj.weight_orig"and"attention_mask_proj.weight_mask"),k.replace("_mask", "")corruptedtensor_nameto"attention_proj.weight", causingstate_dict.pop(tensor_name + "_orig")on the next line to crash withKeyError: 'attention_proj.weight_orig'duringTrainer.save_checkpoint().k.replace("_mask", "")withk.removesuffix("_mask")strips only the trailing"_mask"suffix.getattrcalls insanitize_parameters_to_prune:parameters_to_pruneis provided as a list/tuple of(module, name)pairs,if module not in current_modules:previously performed ancurrent_modulesfor every tuple (current_modulestocurrent_modules_set = set(current_modules)makes membership checksparameters_to_pruneis empty,isinstance(getattr(m, p, None), nn.Parameter)already evaluates toFalsewhengetattrreturnsNone, avoiding callinggetattr(m, p, None)twice per(p, m)pair.Acknowledgments
Credit to
@tchaton,@carmocca,@awaelchli,@Borda,@SkafteNicki(#21191, #21223), and@chanokin(#20783) for prior work onModelPruning.Before submitting
PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:
Reviewer checklist