My recent work focuses on VLA post-training, reinforcement learning for LLM agents, multimodal inference efficiency, and RL training systems.

Current technical interests include self-evolving post-training, rollout-based policy optimization, credit assignment, verifiable rewards, multimodal inference optimization, and efficient RL infrastructure.

Failure-Informed Self-Evolution for Vision-Language-Action Models

A self-evolving post-training framework for driving VLA models.

FIRE-VLA retains GRPO rollout learning while identifying unresolved failure groups from reward and rollout diversity. A frozen privileged teacher provides on-policy supervision for difficult samples, allowing the student policy and its failure distribution to evolve together across training rounds.

Avg L2 improves from 0.6421 → 0.6023 over standard GRPO, while severe long-error rollouts are reduced from 81 → 50 under G=4 evaluation.

Code · Paper

Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading Agents

CIGPO studies a failure mode of outcome-only GRPO in multi-turn evidence-reading agents: as reward variance disappears, training can enter a zero-advantage lock-in and lose effective policy-gradient signals.

Instead of assigning credit only to the final answer, CIGPO estimates how much each newly observed piece of evidence increases confidence in the correct answer and converts this contextual information gain into process-level rewards.

On HotpotQA with Qwen2.5-3B, standard F1 improves from 0.252 → 0.518, while the GRPO baseline eventually collapses to 0.000.

Code · Paper

When Do Fewer Visual Tokens Actually Accelerate Multimodal Inference?

A systems study of the break-even point of visual-token reduction in multimodal LLM inference.

The project measures not only LLM prefill and decoding, but also routing overhead, image preprocessing, vision encoding, and reusable computation to determine when reducing visual tokens actually improves end-to-end latency.

Experiments on Qwen2.5-VL-3B show up to 5.62% mean latency reduction on RTX 3090, while demonstrating that removing more visual tokens does not necessarily result in larger latency savings.

Code · Paper

A compact framework for understanding and reproducing LLM policy optimization

rlite exposes the complete online-RL training path instead of hiding it behind a large distributed training stack:

vLLM rollout → verifiable rewards → grouped experience → policy optimization → versioned LoRA synchronization

It provides GRPO, DAPO, and GSPO objectives through a common algorithm interface, with Ray-based rollout/trainer actors, bounded experience buffering, Hugging Face and vLLM rollout backends, token-aware LoRA microbatch training, and explicit rollout-policy version consistency.

The framework also implements versioned adapter synchronization and guards against stale experience, policy-version mismatch, response-token misalignment, and cache reuse across policy updates.

Small-scale Qwen2.5-1.5B-Instruct experiments on GSM8K complete 200 GRPO / GSPO updates on two RTX 3090 GPUs, serving as an end-to-end validation of the training runtime.

Code

Some earlier research and engineering work:

• VeriSeek — scientific evidence QA with SFT + RL, reaching 79.3% answer accuracy on SciFact.
• tiny-r1 — a minimal and reproducible SFT → GRPO training pipeline.
• AutoPilot — a tool-using intelligent cockpit Agent built with Go, Python and Next.js.
• TideKV — distributed KV storage implemented in Go with Bitcask, Raft and Bloom Filters.
• go-ai-copilot — a RAG-based AI coding assistant with streaming dialogue and a complete Web backend.