GPUs Trade Complexity for Massive Parallelism: What Every Machine Learning Engineer Should Know

The goal of this article is to show the fundamental differences between CPU threads and GPU threads. It will also show how GPUs deliberately simplify per-thread control to pack in far more parallelism. The goal of a SW thread is to create parallel execution flows in software so it can exploit the available HW threads.