Unofficial reproduction of Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training (NeurIPS 2025, arXiv:2504.13161) — search-found mixtures beat uniform baselines +0.014–0.031 STEM at d28; novel finding: selection-mechanism winner's curse; Ascend NPU backend.
lightgbm reproduction data-curation climb paper-reproduction llm ascend-npu pretraining-data mid-training data-mixing data-mixture nanochat llm-pretraining nemotron-climb
-
Updated
Oct 10, 2026 - Python