How Spam Filters Shaped the Field of Adversarial ML

The 2000s spam arms race was an early stress test for adversarial ML. Spammers learned to manipulate inputs without seeing the model, close feedback loops with tracking pixels, and poison training data with as little as 1% corrupted samples. Every one of those attacks has a modern descendant in today's AI systems. The lesson the spam arms race exposed still holds: accuracy alone is not a sufficient measure of performance when an adversary can manipulate both model inputs and training data.