Google Gemma 2
Google's lightweight open models with logit soft-capping and sliding window attention
Models
In plain English
Google's fast, efficient open models (2B, 9B, 27B) engineered to fit into laptops and consumer GPUs without losing accuracy.
Technically
Google DeepMind's open model series (2B, 9B, 27B) featuring interleaved sliding window attention and global attention, logit soft-capping (preventing logit explosion), and post-norm KV projections. The 27B model fits cleanly in 4-bit on a single 24GB RTX 4090/3090.