Skip to content

Latest commit

 

History

24 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Utkarsh Bahuguna

CS undergrad at BITS Pilani × Scaler School of Technology.
I build systems, then write the harnesses that try to break them.


Three things worth your time

I found real nondeterminism in vLLM's deterministic mode. Byte-identical requests to an unchanged server disagreed under concurrency. Filed as vllm#51187, independently reproduced by another engineer on different hardware, then traced into the CUDA C++ RMSNorm kernel: a reduction-width switch at a 256-token boundary. Confirmed by pre-registered measurement, 1,120 comparisons, zero exceptions.

I published a paper arguing against a technique everyone uses. When Self-Consistency Backfires (arXiv:2608.11403), sole author, accepted at the 2nd Workshop on Efficient Reasoning, COLM 2026. Sampling more and voting makes accuracy worse on 56.6% and 65.7% of hard science problems across two models.

I publish what the measurements say, including when they contradict me. In Lockstep, all-pairs comparison showed half my original finding was a first-batch artifact, so the retraction sits beside the result. In Shadowbook, seven benchmark runs gave p50 anywhere from 385 to 424ns, so the published number is the median rather than the good one.


Building

Project What it is Live
Lockstep Batch-invariant LLM inference engine built as the fixture for its real deliverable: a certifier that checks output stays bit-identical however requests are batched, ordered, preempted or evicted. Paged KV, continuous batching, fixed-reduction Triton kernels. The harness is tested like the engine, 10/10 mutation operators killed, 65/65 invariance relations.
Shadowbook Single-instrument limit order book matching engine in Rust. Price-time priority over a 65,536-tick band, preallocated order arena. 100M differentially fuzzed operations against a reference oracle, zero divergences. p50 416ns insert on a stated machine. Zero hot-path allocation is enforced by a counting global allocator, not asserted.
MIRR Incident-response environment where agents diagnose and recover failing microservices from partial telemetry. Five services, one hidden fault, ±15% metric noise, logs that cost a step. Published numbers are generated from a fingerprinted artifact and a pre-commit hook rejects any drift between README and measurement. Space
Starling Real-time collaborative editor on a Fugue CRDT. Treap-backed document, custom binary wire encoding, relay and provider, ProseMirror binding. 377 tests. 60,000 deletions encode to 15 bytes on the wire. Published as starling-crdt. Demo
CAIRN Self-hosted Git platform in Java on a real content-addressable VCS engine. Packfiles with delta compression, commit DAG with generation numbers, Myers diff, three-way merge, trigram code search. Cross-verified against the git binary and serving live over smart-HTTP. Deploy
Flint Bootable x86-64 kernel in Rust under QEMU. Physical and virtual memory, allocator, preemptive scheduler, interrupts, syscall boundary and a shell. W^X and ring 0 / ring 3 isolation proven by a harness that tries to break out.
Tessera Poisoning-resistant distribution for ML models and datasets. Publisher signs once, untrusted mirrors distribute, any consumer verifies the exact signed bytes with full provenance. Tamper-evident, replay-proof, revocation-aware.

Open source

Merged upstream  ·  11 pull requests across 7 repositories

  • gccrs (GCC Rust frontend, C++) #4731, #4728  ·  dead-code lint missing types used as generic arguments; rustc-compatible E0259/E0260 diagnostics, design agreed with two maintainers on Zulip before writing code
  • jaeger-ui (CNCF) #4053, #4271  ·  GenAI span classification over OpenTelemetry semantic conventions, and a trace view that bypassed the shared span detector
  • huggingface/OpenEnv #742  ·  SSRF-safe URL parser
  • NVIDIA/garak #1842  ·  provider-aware parameter suppression, restoring Bedrock scans that were failing for every Claude 4.x user
  • dottxt-ai/outlines #1867  ·  RFC 4291 IPv6 structured-output type
  • vllm-project/llm-compressor, AI Village

In review

  • kubernetes-sigs/resource-state-metrics #84 (Go)  ·  a CEL resolver accepted int64 inside lists but dropped it inside maps, so a metric silently vanished depending on where the value sat. Reproduced at scrape level with before-and-after output, plus a parity test asserting the two paths agree. I disclosed that the fix changes series identity and breaks existing dashboards.
  • urunc-dev/urunc #886 (Go)  ·  decode-before-validate ordering bug in a Kubernetes unikernel runtime, found by fuzzing and pinned by a committed regression seed
  • NVIDIA/TensorRT-LLM, NVIDIA/cuda-python, microsoft/PyRIT, promptfoo, ml-explore/mlx-lm

Research

  • When Self-Consistency Backfires  ·  sole author, accepted at the 2nd Workshop on Efficient Reasoning, COLM 2026. A pre-registered study finding that majority-vote sampling degrades accuracy on most expert-level problems.
  • Filed an exact failing-set predicate on an open PyTorch numerics issue (#147284) while building Lockstep's fp64 reference.

Elsewhere

  • Contract engineer at Handshake, authoring containerised evaluation environments that measure how well AI coding agents perform real engineering tasks. Each task ships as a Docker image plus a deterministic verifier, calibrated so a reference solution passes and a strong agent does not.
  • Global Top 20 finalist, Meta × PyTorch Hackathon (the project became MIRR)
  • Delegate, Harvard HPAIR 2026  ·  Top 1% delegate, Japan Youth Summit 2025 (UNESCO affiliated)

Stack

Languages  Rust C++ Python Go Java TypeScript Systems  CUDA Triton QEMU Linux internals x86-64 Kubernetes Docker Verification  differential fuzzing reference oracles property-based testing mutation testing deterministic replay Focus  inference systems compilers observability low-latency engines

Portfolio  ·  LinkedIn  ·  Email

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors