Topic
ML & GPU
Deep dives into machine learning systems, GPU programming, LLM inference, attention mechanisms, speculative decoding, vector search, and model optimization. From CUDA kernels to production serving.
10 posts · ~64 min of reading
Jul 8, 2026
FlashAttention-3: Warp Specialization and the 75% FLOPS Barrier on Hopper GPUs
How FlashAttention-3 exploits H100 warp specialization, asynchronous pipelines, and FP8 quantization to push attention throughput past 75% of peak Tensor Core FLOPS.
Jul 8, 2026
J-Space: Anthropic Discovered a Global Workspace Inside Claude
Anthropic found that Claude has developed a privileged internal workspace, the J-space, where concepts light up silently during reasoning. It mirrors Global Workspace Theory from neuroscience, and it lets researchers read what the model is thinking but not saying.
Jul 8, 2026
Mixture of Depths: Dynamic Token Routing for 50% Faster Transformer Inference
How Google DeepMind's Mixture of Depths achieves equivalent language model quality at a fraction of the FLOPs by learning which tokens can skip entire transformer layers, yielding up to 50% faster inference with a static computation graph.
Jul 8, 2026
RadixAttention: Prefix-Aware KV Cache Reuse for 10x LLM Serving Throughput
How SGLang's RadixAttention uses a radix tree to automatically detect and reuse shared prefixes in KV caches, eliminating redundant computation and achieving up to 10x throughput gains for multi-turn LLM workloads.
Jul 8, 2026
Ring Attention: Distributing Million-Token Contexts Across Devices
How Ring Attention eliminates the memory wall for long-context transformers by overlapping blockwise attention computation with KV-cache communication in a ring topology, enabling near-linear context scaling across devices.
Jul 6, 2026
EAGLE-3: Why the Best Draft Models Stopped Predicting Features
EAGLE-3 gets up to 6.5x decoding speedup by abandoning the feature-prediction objective that defined its predecessors. The interesting part is why feature prediction became the bottleneck, and how a trick called training-time test fixes the train/inference mismatch it leaves behind.
Jul 6, 2026
Native Sparse Attention: Why Trainable Sparsity Beats Post-Hoc Pruning
DeepSeek's NSA makes attention sparsity a first-class citizen of pretraining instead of an inference-time hack, and pairs it with a kernel design that actually turns theoretical FLOP savings into wall-clock speedups. A close read of the architecture and why most sparse attention schemes before it failed to deliver.
Jul 6, 2026
RaBitQ: 32x Vector Compression With an Error Bound You Can Actually Prove
Product Quantization has powered billion-scale vector search for 15 years, but it can fail badly on real datasets and offers no theoretical guarantees. RaBitQ (SIGMOD 2024) compresses vectors to one bit per dimension, estimates distances with a popcount, and comes with a provable O(1/sqrt(D)) error bound. Here is how a random rotation makes that possible.
Jul 5, 2026
How I Built This Portfolio (and Its Blogging Engine) in Angular 22
A technical walkthrough of rebuilding my portfolio from Angular 9 to Angular 22 with spartan-ng, Tailwind v4, SSG prerendering, a WOW animation system, and a zero-backend blogging engine powered by markdown files.
Jul 5, 2026
Two Workloads in a Trench Coat: Prefill/Decode Disaggregation in LLM Serving
Prefill and decode have opposite hardware profiles, and serving them on the same GPUs wastes both. A practical tour of DistServe and Mooncake, the two papers behind the biggest architecture shift in LLM inference.