Key Bottlenecks in PC Parallelization

This article examines the major bottlenecks in parallelizing probabilistic circuits (PCs), focusing on the forward pass. While product layers scale efficiently, sum layers create unexpected overhead due to repeated IO reloads of product node outputs. This imbalance leads to drastically higher runtime costs than predicted by memory read/write counts. The proposed solution involves smarter grouping of sum edges across processors, reducing redundant reloads and framing core operations as matrix multiplications. By doing so, modern GPUs can leverage Tensor Cores to significantly reduce computational overhead, making PC implementations faster and more efficient.