By: Sana Ul Mustafa Qadri
Hardware: Tesla T4 (sm_75) + NVIDIA H100 (sm_90)
Language: CUDA C++ (C++17), portable CC 7.0+
Status: ✅ Completed — Tested on 2 GPU architectures
Implemented and optimized a CUDA SGEMM kernel (C = A x B, float32) from a naive global-memory baseline to a shared-memory tiled implementation. Performed full correctness audit (13-category checklist), performance audit with roofline analysis, and verified on two GPU architectures.
Key results:
- Tesla T4: Tiled kernel achieved 393.86 GFLOP/s (Google Colab, verified)
- NVIDIA H100: Tiled kernel achieved 6,813.25 GFLOP/s — ACCEPTED on Tensara.org
- 17.3x scaling confirmed from T4 to H100 on non-square matrices up to 8192x8192
Matrix multiplication (GEMM) is the core operation in deep learning training and inference. On GPU hardware, naive implementations leave most of the device's compute and memory bandwidth unused. The goal was to:
- Implement a correct naive CUDA kernel (baseline)
- Optimize using shared-memory tiling to increase arithmetic intensity
- Verify correctness rigorously (not just "output looks right")
- Analyze performance using roofline model on real hardware
- Explain the gap between theoretical and measured speedup
| N | Naive | Tiled | Speedup |
|---|---|---|---|
| 512 | 275.35 GFLOP/s | 350.56 GFLOP/s | 1.27x |
| 1024 | 332.88 GFLOP/s | 389.39 GFLOP/s | 1.17x |
| 2048 | 335.50 GFLOP/s | 393.86 GFLOP/s | 1.17x |
| Matrix Shape | GFLOP/s | Status |
|---|---|---|
| 4096x4096x4096 | 6,769.54 | ✅ ACCEPTED |
| 8192x8192x4096 | 6,822.33 | ✅ ACCEPTED |
| 4096x4096x8192 | 6,817.78 | ✅ ACCEPTED |
| 8192x8192x8192 | 6,843.56 | ✅ ACCEPTED |
Average H100: 6,813.25 GFLOP/s (6.8 TFLOPS)
T4 → H100 scaling: 17.3x confirmed
| Kernel | Registers/thread | Shared Memory | Spills |
|---|---|---|---|
| Naive | 58 | 0 bytes | 0 |
| Tiled | 62 | 8,192 bytes | 0 |
All sizes tested: 512x512, 1024x1024, 2048x2048 — PASS (0 mismatches) on all.
H100: All 4 test cases (up to 8192x8192) — ACCEPTED on Tensara.org.
Full 13-category correctness audit completed — no Tier 0 or Tier 1 issues found.
T4 Ridge Point = 8,100 GFLOP/s / 320 GB/s = 25.3 FLOPs/byte Kernel Structural AI BW Ceiling Measured Naive 0.25 FLOPs/byte 80 GFLOP/s 332.88 GFLOP/s (4.2x above ceiling) Tiled 8.0 FLOPs/byte 2,560 GFLOP/s 393.86 GFLOP/s (far below ceiling)
Naive measured 332 GFLOP/s — 4.2x above its own structural ceiling of 80 GFLOP/s. The warp broadcast pattern on A and T4's 4MB L2 cache absorb most HBM traffic. Effective AI of naive is greater than 1.0 FLOPs/byte at HBM — not 0.25 as the structural model predicts.
Tiled used only 49.2 GB/s of 320 GB/s peak HBM (15.4%) — latency-bound, not bandwidth-bound. After every syncthreads(), all 32 warps simultaneously stall on HBM loads. Compute phase (32 FMA = ~32 cycles) is 6-12x shorter than HBM latency (~200-400 cycles).
- HBM3: 3.35 TB/s vs 320 GB/s (10.5x bandwidth)
- 132 SMs vs 40 SMs (3.3x parallelism)
- 2,048 max threads/SM — allows 2 blocks/SM, partial latency recovery
Thread coarsening (2 output elements per thread, 16x32 block):
- Doubles compute phase to 64 FMA per tile
- Restores latency hiding via multiple blocks per SM
- Register impact: [NEEDS PROFILING]
- Structural roofline does not equal actual behavior — L2 cache closes the gap before shared memory helps
- Synchronized barriers destroy latency hiding — all warps stall together
- Correctness requires rigor — float32 non-associativity and integer overflow are real production risks
- Explaining results matters more than numbers — understanding why 1.17x points to thread coarsening
- Hardware scaling validates kernel design — 17x T4→H100 confirms no architecture-specific bottlenecks
| File | Description |
|---|---|
matmul_fixed.cu |
CUDA source — naive + tiled kernels, host code, timing harness |
portfolio_case_study.md |
Detailed case study with full analysis |
CUDA C++ · C++17 · nvcc · cudaEvent timing · ptxas -v · Roofline Model · Tesla T4 (sm_75) · NVIDIA H100 (sm_90) · Tensara.org