PagedAttention
Virtual memory pagination for the KV cache
Performance
In plain English
Breaking KV cache notes into neat, indexed memory pages so no memory is wasted on empty space.
Technically
An algorithm (Kwon et al., 2023) inspired by OS virtual memory that partitions the KV cache into fixed-size physical blocks. Eliminates external memory fragmentation and enables dynamic memory sharing across parallel sampling and speculative draft paths.