Skip to content

Fix syncbn training state - #1695

Open
Marchematics wants to merge 5 commits into
open-mmlab:mainfrom
Marchematics:fix-syncbn-training-state
Open

Fix syncbn training state#1695
Marchematics wants to merge 5 commits into
open-mmlab:mainfrom
Marchematics:fix-syncbn-training-state

Conversation

@Marchematics

@Marchematics Marchematics commented Aug 30, 2026

Copy link
Copy Markdown

Motivation

Preserve the source module's training mode when converting BatchNorm to SyncBatchNorm. Without this, evaluation-mode layers are silently switched to training mode, which can cause incorrect fine-tuning behavior and NCCL timeouts. Fixes #1624.

Modification

Copy module.training to the converted module and add a regression test for evaluation-mode conversion.

BC-breaking

No.

Checklist

  1. Targeted lint checks pass.
  2. Added test_convert_sync_batchnorm_keeps_training_state.
  3. No downstream changes are required.
  4. No documentation changes are needed.

Copilot AI lite review requested due to automatic review settings August 30, 2026 16:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Marchematics and others added 3 commits September 1, 2026 04:09
@Marchematics
Marchematics force-pushed the fix-syncbn-training-state branch from 57be875 to 8b1dbfb Compare August 31, 2026 20:09
@Marchematics

Copy link
Copy Markdown
Author

Removed the duplicate isfile mock that caused the lint failure in the checkpoint test. The changed test now passes the repository lint checks locally.

@Marchematics

Copy link
Copy Markdown
Author

Targeted regression test passes: tests/test_model/test_convert_sync_batchnorm_training.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] convert_sync_batchnorm missing 'training' attribute

2 participants