Smart sampling approach - #9

Open

toslali-ibm wants to merge 74 commits into

mainfrom

Commits on Jun 16, 2025

Define buffers and hooks
toslali-ibm
committed
315de2e
Fix add exploration and add idx to dataset - next generation and checkupdate
toslali-ibm
committed
7d9b53f
Debug hook2
toslali-ibm
committed
d884d42
Debug hook2
toslali-ibm
committed
9d718cc
Debug hook2
toslali-ibm
committed
b2d18a8
Debug hook2
toslali-ibm
committed
4e2a4a5
Debug hook2
toslali-ibm
committed
a77ec55

Commits on Jun 17, 2025

Debug hook2
toslali-ibm
committed
3de3119
Debug hook2
toslali-ibm
committed
1a575e9
Exploration is always at the end by budget
toslali-ibm
committed
20e597d
Exploration is always at the end by budget
toslali-ibm
committed
7d7ad33
Exploration is always at the end by budget
toslali-ibm
committed
3f1421b
Add adjust_batch for buffer and replace logics
toslali-ibm
committed
50b00a1
Run comparison experiments
toslali-ibm
committed
169a53d
Fix model card issue
toslali-ibm
committed
4f05867
Smart sampling as config
toslali-ibm
committed
4b29c87
Fix small bug and run experiments
toslali-ibm
committed
6fe47e0
Debug size mismatch
toslali-ibm
committed
7b9ecef
Fix shape mismatch issue
toslali-ibm
committed
e165058
Fix shape mismatch issue
toslali-ibm
committed
3c38bc7
Export advantages as metrics
toslali-ibm
committed
574ed99

Commits on Jun 18, 2025

Fix completion ids length
toslali-ibm
committed
7a88dbe
Make smart sampling multi proc for server mode
toslali-ibm
committed
5e0b504
Fix broadcast issue of completion ids
toslali-ibm
committed
6bf517b
Clean up before actual tst
toslali-ibm
committed
de9053b
Clean up before actual tst
toslali-ibm
committed
50bee9e

Commits on Jun 19, 2025

Clean up before actual tst
toslali-ibm
committed
3a95d78
Debug smart sampling exploration bug
toslali-ibm
committed
90c22f2
Fix broadcast length issue in exploration
toslali-ibm
committed
e4376db
Fix broadcast length issue in exploration
toslali-ibm
committed
c0d1dee
Fix tensor object confusion in adjust_batch
toslali-ibm
committed
7256288
Debugging adjust batch
toslali-ibm
committed
7bae9fd
Fix mismatch between ranks for adjust batch
toslali-ibm
committed
c960386
Fix mismatch between ranks for adjust batch
toslali-ibm
committed
989340e
Fix mismatch between ranks for adjust batch
toslali-ibm
committed
b238dcc
Gather across all ranks not only main
toslali-ibm
committed
4475bf6
Fix reward gathering
toslali-ibm
committed
f5d2873
Fix rewards - dont split
toslali-ibm
committed
3fa3c5d
Fix device mismatch
toslali-ibm
committed
a820967
Fix train dataset for multi proc
toslali-ibm
committed
f07085f
Fix train dataset for multi proc
toslali-ibm
committed
42bddce
Fix train dataset for multi proc
toslali-ibm
committed
be746c5
Confirmed all ranks have train dataset
toslali-ibm
committed
0a92e1c

Commits on Jun 24, 2025

Fix coloc mode - split completions correctly
toslali-ibm
committed
a093228
Set seed for random exploration
toslali-ibm
committed
d19c41c
Debug uneven bug
toslali-ibm
committed
e42b608
Fix reward uenven problem
toslali-ibm
committed
f355320
Fix reward uenven problem
toslali-ibm
committed
15b3ff2
Fix reward gathering with gather object
toslali-ibm
committed
37532d3

Commits on Jun 25, 2025

Fix uneven len of all varaibles and remove a lot of gathers
toslali-ibm
committed
b07944d
Fix uneven len of all varaibles and remove a lot of gathers
toslali-ibm
committed
87b2811
Remove reward gathering, and fix ternsor and list issues
toslali-ibm
committed
87d474f
Move tensors to rank0 for cat
toslali-ibm
committed
4529145
scalar tensors reshaped to 1d
toslali-ibm
committed
4343fc6
Fix conversion after all gather obj
toslali-ibm
committed
c9dd764
Loaner commit
Mert Toslali
committed
b54ca75
Refactor adjust batch in loaner
Mert Toslali
committed
f735315
Bring back original all gather key by key
Mert Toslali
committed
5c40bcc

Commits on Jun 26, 2025

Add wandb metrics
toslali-ibm
committed
80ce077
Add prints to debug wandb metrics
toslali-ibm
committed
011bfb0

Commits on Jun 27, 2025

Add promising buffer metric to wandb
Mert Toslali
committed
e409f93
Make logic smarter - only push medium samples and discard hard and easy
Mert Toslali
committed
08f03cf

Commits on Jun 30, 2025

Fix prints
Mert Toslali
committed
c7cb093
Fix prints
Mert Toslali
committed
8d9d507
Fix prints
Mert Toslali
committed
0db81c1
Fix prints
Mert Toslali
committed
c4d578a
Replace easy samples too

Mert Toslali
authored and
Mert Toslali
committed
bafc31e

Commits on Jul 3, 2025

Move old_per_token_logps after adjust batch
Mert Toslali
committed
b7b704e
Fix forgetton comma in prompt completion ids
Mert Toslali
committed
7c1ed0f
Fix attention mask
Mert Toslali
committed
35be057
Debug
Mert Toslali
committed
83c9aa5
Debug
Mert Toslali
committed
bbb7398
Debug seession with fabian
Mert Toslali
committed
60483e6
New advantage metrics non zero, sum of pos and neg
Mert Toslali
committed
48cae3b