AI Models Can Know They’re Being Tested, And Not Tell You

Added: 2026-05-27

[00:00:00]And so that text then, since it's in plain English, gives us a new kind of window into what the model's thinking about. That's the basic idea here. They see a bunch of interesting stuff. One result that's especially interesting, unverbalized evaluation awareness. So, you know, we've talked about language models where when you give them a chain of thought, you'll often find them thinking that, "Oh, this question I'm being asked looks suspiciously like a an evaluation, like I'm being benchmarked for maybe cyber risk or something else."

[00:00:27]So, you know, maybe the model chooses to like sandbag and pretend that it doesn't have the capability, that it'll pass the test. Well, in this case, they actually see that the model sometimes internally recognizes that it's in an eval without ever saying so in its chain of thought.

[00:00:41]So, there's a new way of detecting the sort of like, I guess, unspoken thoughts, if you will, of these models.