Generalizing Sparse Spectral Training Across Euclidean and Hyperbolic Architectures

Sparse Spectral Training (SST) introduces a low-rank optimization technique that enhances both Euclidean and hyperbolic neural networks. Tested on machine translation benchmarks like IWSLT and Multi30K, SST consistently outperformed LoRA, ReLoRA*, and even full-rank training, delivering higher BLEU scores and preventing overfitting in high-dimensional hyperbolic spaces. The results highlight SST’s ability to generalize efficiently while maintaining stability and robustness across architectures.