Fix bias gradient computations - #76

Closed

baberabb wants to merge 11 commits into

EleutherAI:mainfrom

Commits on Dec 19, 2025

fix: sum bias gradients over sequence dim only, not batch + tests
baberabb
committed
0c74db9
fix: add normalizer bias support. fix trainer callback. add tests
baberabb
committed
ac89086
add `scale_by_lr` method to AdafactorNormalizer
baberabb
committed
d581873
fix tests
baberabb
committed
b88717d
fix device
baberabb
committed
f433265
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
d41264e
fix hf loop
baberabb
committed
db58e89
handle bias but not norm
baberabb
committed
46bdd34
fix test
baberabb
committed
e488e29

Commits on Jan 14, 2026

Merge branch 'main' into bias
baberabb
committed
cb42f90
pacify pre-commit
baberabb
committed
0a41a1d