Why Dynamic Grouping Beats Traditional Quantizers for Vision Transformers

The paper introduces IGQ-ViT, a dynamic, instance-aware group quantization method that tackles the significant scale variations in Vision Transformer activations and softmax attentions. By assigning channels and tokens into statistically aligned groups—rather than fixed or layer-wide buckets—IGQ-ViT applies more precise quantization with minimal computational overhead. Experiments across multiple ViT architectures show that this approach surpasses layer-wise, channel-wise, and existing group quantizers, approaching upper-bound performance even with small group sizes. Adaptive group-size allocation further boosts accuracy, confirming IGQ-ViT as a more stable and effective quantization framework.