AWQ
Activation-aware Weight Quantization preserving salient weights
Formats
In plain English
Protecting the 1% most important neuron channels in full quality while compressing the remaining 99%.
Technically
Activation-aware Weight Quantization (Lin et al., 2023). Observes activation distributions on calibration data to identify the top ~1% salient channels and scales them before uniform INT4 quantization, outperforming pure round-to-nearest methods.