FlashInfer
Ultra-fast customizable GPU kernel library for LLM serving
Tools
In plain English
A toolkit of lightning-fast GPU micro-recipes used by top serving engines to calculate attention in record time.
Technically
A kernel library specializing in LLM serving with support for heterogeneous page sizes, fused RoPE, grouped-query attention, and FP8/FP4 tensor formats across varying batch and context sizes.