Blog
Blog
Technical deep dives on ML infrastructure, GPU inference, distributed training, and the systems engineering behind quantitative platforms. 2000–4000 words. Written from a practitioner's perspective.
- LServe and SampleAttention: What Sparse Attention Actually Changes in Prefill and Decode
Structured sparsity aligned with GPU attention kernels: LServe’s unified serving stack for prefill and decode, and SampleAttention’s empirical patterns plus CRA as a runtime quality floor.
- How KV-Cache Paging Works in vLLM — and Why It Matters for Production
A technical walkthrough of PagedAttention: the memory management innovation that makes vLLM practical for serving many concurrent LLM sessions without fragmenting GPU memory.