Test-time verification for AI agents: New from Microsoft Research #ai #agenticai #verification

Hinzugefügt: 2026-05-25

[00:00:00]Now let me go tell you how it works in a real-world setting.

[00:00:04]We are finding that Intervene leads to state-of-the-art results on agentic benchmarks such as Tau Too Bench, where even small models can rival the accuracy of frontier models.

[00:00:14]Let's see how this works.

[00:00:16]First step is to extract verifiable properties. These are structured atomic properties that any solution should satisfy.

[00:00:24]For example, in Tau Too Bench, we have a scenario with a retail agent, and you'll have a policy which is a lot of text, but then it gets converted to verifiable properties such as a refund must go to the original payment method.

[00:00:37]And with this property, we can now actually automatically generate Python code that acts as a verifier for it. And this is a one-time operation that can actually be assessed and checked. And then the magic happens at runtime when the variables of the Python verifier are dynamically filled in based on the user's context and [music] the model's current response.