How Nvidia Made Its ASR Models 3x Faster Than the Competition

Nvidia's Parakeet models sit 3x clear of the rest of the Hugging Face Open ASR Leaderboard on throughput, with competitive accuracy. The reason is the Token-and-Duration Transducer (TDT), a small modification to RNN-T that adds a second head predicting how many encoder frames each token covers. Instead of advancing one frame at a time, the decoder skips. The result is up to 2.82x faster inference at comparable or better word error rate.