Three A/B Testing Mistakes I Keep Seeing (And How to Avoid Them)

Many A/B testing problems come from using statistical methods without checking if they fit the situation. The three most common mistakes are: (1) using the Mann–Whitney test to compare medians (2) using bootstrapping for every dataset and (3) always using default error rates (5% Type I, 20% Type II) Better assumptions lead to better A/B test decisions.