Why Uniform Quantizers Break ViTs

The article introduces IGQ-ViT, an adaptive quantization method designed to improve Vision Transformer efficiency without the accuracy loss seen in traditional uniform, layer-wise quantizers. The authors show that activations and softmax attentions vary widely across channels and tokens, making fixed quantization intervals suboptimal. IGQ-ViT applies more flexible group-wise quantization, along with a layer-specific group size allocation technique that minimizes prediction drift under computational constraints. The result is a more precise, scalable, and hardware-friendly quantization pipeline for modern ViT architectures.