TurboSparse-LLM Performance: Outperforming Mixtral and Gemma with Extreme Sparsity

TurboSparse-Mistral-7B and Mixtral-47B deliver elite performance on the OpenLLM Leaderboard while activating as few as 3B parameters. Discover how ReLU-based intrinsic sparsity maintains accuracy with significant FLOPs reduction.