Dino in the Machine: Surviving the Transformer Latency Trap in C++

Porting from YOLOv8 to Grounding DINO in a zero-copy C++ ONNX pipeline exposed severe CPU cache bottlenecks, thread thrashing, and unstable graph optimizations. Transformer self-attention shattered the prior scaling logic, forcing a rethink of worker-to-thread ratios, abandonment of aggressive ONNX graph fusion, and a strategic pivot to INT8 quantization. The result: stable, quantized CPU inference without falling for the “optimize everything” myth.