Perf optimizations & support for CDF plots - #29
Draft
gdadlaney wants to merge 39 commits into
Draft
Conversation
gdadlaney
changed the title
gdadlaney
marked this pull request as draft
January 26, 2026 18:43
…where they are required (when var=good)
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
January 28, 2026 05:55
f953b1b to
c9704a9
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
January 28, 2026 15:30
1a94f45 to
1d3d637
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 2, 2026 18:57
1d3d637 to
36fead0
Compare
…iminating GPU scynchronizations
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 3, 2026 02:23
3d8cf53 to
deecaae
Compare
… significance. 1. Autocasting from FP32 to F16 to use the hardware acceleration on TensorCores in A40s 2. Decreasing max_sequence_length 3. Increase batch size 4. Keep on GPU: Accumulating output_labels, logits & loss on the GPU for every batch & only moving them to the CPU after all batches. Earlier it would happen at the end of every batch & .item() & .cpu() force the GPU flush its pipeline (like cuda.synchronize) 5. Only copy the test dataset in GPU memory once
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 4, 2026 07:37
ae07f98 to
247ec3e
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
2 times, most recently
from
February 4, 2026 09:39
ed8f6f6 to
8b2681e
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
2 times, most recently
from
February 9, 2026 06:36
ef9dd4a to
a48550f
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 9, 2026 18:04
a48550f to
ee69946
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 9, 2026 18:13
ee69946 to
9cacdc1
Compare
gdadlaney
force-pushed
the
perf/iteration_time
branch
2 times, most recently
from
February 11, 2026 04:32
4b91a2e to
f8e3cea
Compare
…en an error or exception is encountered
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 11, 2026 04:42
f8e3cea to
1dd480b
Compare
…e dataset & model
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 17, 2026 20:12
c1f24bb to
d381722
Compare
…essage in fwdllm_aggregator.py
gdadlaney
force-pushed
the
perf/iteration_time
branch
from
February 25, 2026 06:18
99c5b54 to
b916755
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.