Why benchmark scores lie: eval contamination + Goodhart's law

Added: 2026-04-29

[00:00:00]A new model just claimed every benchmark. You ran it on your own work and it lost to a model from six months ago. Public benchmarks live on the open internet. Then models train on trillions of pages scraped from that same internet. So the test ends up sitting inside the training data. Now the model isn't reasoning on the test. It's recalling. There are two ways this happens. First, leakage. Questions from benchmarks like MMLU and human evil are already in the training set and the model has seen the answers. Second, Goodart's law. When a metric becomes the goal, labs start training to win on it.

[00:00:33]The benchmark stops measuring capability and just measures who optimized for it the hardest. So, a famous benchmark goes up. A freshly written equivalent stays flat. The score moved, but the actual capability didn't. So, build a small heldout eval on your actual workload.

[00:00:48]Real prompts your users send. Keep it private. Never publish it. That's the only number that tracks the thing you care about. Public benchmarks suggest your private eval proves.