Most AML testing relies on production data that was never designed for testing or performance evaluation. It is legally sensitive, difficult to access, historically biased, and severely label-scarce. As a result, it produces weak and often misleading evidence about how AML systems actually perform.

This piece explains why production data is the wrong foundation for AML testing, why the problem becomes more acute with modern and AI-based models, and how synthetic datasets with known ground truth enable objective, repeatable evaluation without relying on production data.