Senior AI Research Scientist & Engineer at Intel | 20+ years of industrial research (Intel Labs, IBM Research)
I work on making LLM inference faster and more efficient -- speculative decoding, CPU optimization, and serving infrastructure.
- Speculative decoding -- adaptive token scheduling, vocabulary-aware drafting, heterogeneous vocabularies (TLI, SLEM)
- CPU inference optimization -- bringing speculative decoding and high-throughput serving to Intel Xeon platforms
- LLM serving infrastructure -- contributing to vLLM and SGLang for production-grade inference
Enabling speculative decoding when draft and target models use different tokenizers.
- TLI -- Token-Level Intersection enabling any draft model regardless of tokenizer
- SLEM -- cross-vocabulary speculative decoding
- Dynamic speculative token count -- adaptive scheduling that improves acceptance rates
- Adaptive dynamic speculative tokens -- further acceleration via heuristic scheduling
- LM Head pruning for USD -- vocabulary mismatch handling for Universal Speculative Decoding
- Default behavior of assisted decoding -- setting speculative decoding defaults for the ecosystem
- Grammar-aware draft token sampling for structured outputs -- constrained decoding during speculative drafting
- Dynamic speculation length with confidence-threshold early exit -- adaptive draft length based on model confidence
- CPU spec decode stream fix -- removing incorrect overrides in CPUModelRunner
- Spec decode kernel compatibility -- Triton-CPU integration
- CPU rotary embedding fix -- fixing CPU path without flash-attn ops
- Vocabulary compatibility validation -- safety checks for speculative decoding in standalone mode
- SPEED-Bench dataset support -- new benchmark dataset for serving evaluation
- Distributed Speculative Inference (DSI) -- co-developed implementation for distributed draft-target decoding
- Redistributing Drafter Kernels -- target distribution pruning and analysis for speculative decoding efficiency
