The Conductor in Your Pocket: How PowerInfer-2 Orchestrates Smartphone Hardware for LLM Inference

PowerInfer-2 is a smartphone LLM inference framework that uses "neuron clusters" to optimize for heterogeneous hardware and minimize I/O overhead.