AI Model Scores Flawed: Harness Affects Results! #shorts

本站添加: 2026-05-16

[00:00:00]Two independent sources, different methodologies, same conclusion.

[00:00:03]Anthropic's newer model is is worse than the one it replaced.

[00:00:07]And a researcher named Bustamante published a study this week proving the benchmark scores change dramatically based on which harness you use, which which thing you add to the model.

[00:00:18]Same model, different harness, different score. Cursor jumped from top 30 to top five by changing only the harness configuration.

[00:00:26]The model is the same, the score moved because the test setup moved. Nobody else is testing for this. Tab tests 101 harnesses, I'm proud to say.

[00:00:35]101 harness configurations because the model isn't the score, like we just said. The model plus the harness plus the configuration equals the score.