Topic
Data Engineering
Data engineering patterns: lakehouse architectures (Apache Iceberg), cache eviction (SIEVE, LRU), object storage (S3 conditional writes), database internals, file formats, and query engines.
10 posts · ~71 min of reading
Jul 8, 2026
Adaptive Radix Trees: How Modern Databases Achieve Sub-Microsecond Lookups
Dissecting ART (Adaptive Radix Tree), the cache-conscious indexing structure that outperforms B-trees for in-memory workloads by collapsing node sizes, eliminating key comparisons, and exploiting CPU cache hierarchies.
Jul 8, 2026
Bw-Tree: Lock-Free B+ Trees for Multi-Core Database Scalability
How the Bw-Tree achieves latch-free concurrent access to B+ tree indexes through delta chains, an indirection mapping table, and epoch-based garbage collection, enabling linear scalability on modern many-core hardware.
Jul 8, 2026
PGM-Index: How Piecewise Linear Models Replace B-Trees with O(log log n) Lookups
The Piecewise Geometric Model index uses linear regression segments to predict key positions, achieving O(log log n) point queries with orders-of-magnitude less space than B-trees. Here's the theory, the recursive structure, and why production systems are starting to care.
Jul 8, 2026
Roaring Bitmaps: Compressed Set Operations in Constant Time Per Container
How Roaring Bitmaps achieve intersection, union, and cardinality on billion-element sets in microseconds by partitioning integers into typed containers, each optimized for its density regime, and accelerated with SIMD vectorization.
Jul 8, 2026
Swiss Tables: How SIMD Rewrote the Rules of Hash Map Design
Flat hash maps based on Swiss Table design now dominate C++, Rust, Go, and Zig standard libraries. The key insight is not a better hash function or collision strategy, it is using SIMD to probe 16 slots in a single instruction, turning the control byte array into a hardware-accelerated Bloom filter.
Jul 8, 2026
Vectorized Execution: How DuckDB Processes a Billion Rows Per Second on a Laptop
Push-based pipelines, morsel-driven parallelism, and selection vectors: the three architectural bets that let an in-process database saturate modern hardware without a cluster.
Jul 7, 2026
Iceberg v3 Deletion Vectors: Fixing Merge-on-Read With One Bitmap Per File
Apache Iceberg v2 made row-level deletes possible with position delete files, and large deployments have regretted the details ever since. Format version 3 deprecates them in favor of deletion vectors, Roaring bitmaps stored in Puffin files with a hard invariant of at most one vector per data file. Here is what was broken, how the new binary format works down to the byte level, and why the length field is big-endian on purpose.
Jul 7, 2026
S3 Conditional Writes: The CAS Primitive That Killed the Coordination Sidecar
For years, every database built on object storage needed a DynamoDB table or a ZooKeeper cluster on the side just to answer "who is the writer?" In late 2024, S3 quietly shipped If-Match and If-None-Match support on PutObject, turning the object store itself into a compare-and-swap register. Here is why that one HTTP header changes how you architect storage systems, and how projects like SlateDB use it for formally verified writer fencing.
Jul 6, 2026
RaBitQ: 32x Vector Compression With an Error Bound You Can Actually Prove
Product Quantization has powered billion-scale vector search for 15 years, but it can fail badly on real datasets and offers no theoretical guarantees. RaBitQ (SIGMOD 2024) compresses vectors to one bit per dimension, estimates distances with a popcount, and comes with a provable O(1/sqrt(D)) error bound. Here is how a random rotation makes that possible.
Jul 5, 2026
SIEVE: The Cache Eviction Algorithm That Beats LRU by Doing Less
A 2024 NSDI paper showed that a FIFO queue, one bit per object, and a lazy hand pointer can out-perform LRU, ARC, and friends on web workloads, while removing the lock that makes LRU a scalability bottleneck. Here is how SIEVE works and why its simplicity is the whole point.