Learn · Foundations · The Three Axes
Capacity — What Fits
In plain English
You can't fit an elephant into a Mini Cooper! 🐘🚗 Models take up space, and your computer needs enough RAM (memory) to hold the whole model before it can run.
What is actually going on
A model's footprint in memory is determined by its parameter count and the precision of those parameters. To run a model, your system's unified memory or discrete VRAM must exceed the model's footprint plus context size.
Why it matters
If a model doesn't fit in VRAM, it will offload to system RAM or storage, tanking performance entirely.
Another way to think about it
RAM is like a backpack. If a book (model) is too big, it just won't fit in the bag. 🎒
Check yourself
What hardware component dictates if a model 'fits'?
- CPU clock speed
- SSD size
- VRAM / System RAM
- Network speed
Show answer
VRAM / System RAM — Models must be loaded into memory (RAM or VRAM) to be executed efficiently.
An 8-Billion parameter model takes up exactly 8 Gigabytes of RAM.
- True
- False
Show answer
False — The size depends on precision. At 16-bit (FP16), an 8B model takes ~16GB. Quantized to 4-bit, it takes ~4-5GB.
What happens when a model is larger than your GPU VRAM?
- It runs faster
- It crashes or offloads to slow system RAM
- It automatically shrinks
- It connects to the cloud
Show answer
It crashes or offloads to slow system RAM — Out Of Memory (OOM) errors occur, or if offloaded to CPU RAM, speed drops drastically.