Your AI Agents Need an Adversary

[00:00:00]After you run a plan mode in Claude Coder or Codex, do you ever go back through with another agent to review the plan before you just blindly let Claude Code implement it? If not, you may not be getting the best output that you can from your LLMs. So, I'm going to show you how to write an adversarial review agent to go through your plans and make sure that it's written in such a way that is scalable, has security in mind, that is maintainable, and also has performance and a bunch of other things that you should be checking for before you even kick off the plan. These plans can cost a lot of money when they're actually being implemented. They can run for a very long time. And so, you want to make sure that the plan that you're implementing from the get-go is of high quality. So, what we're looking at right here is an adversarial review agent. The overall idea of this agent is that it should ask a bunch of questions over the plan that was being written. And you want to make sure that it's checking for a lot of these things: observability, backwards compatibility, for costs, security, scalability, reliability.

[00:00:55]Basically, all of the senior software engineering terms you should know by now. They should be automated from an agent to make sure that the plans are solid. And not only that, but the code that's implemented is also looking pretty good, too. Now, one main goal of this agent is that it should be really challenging everything about your plan.

[00:01:12]Challenge the design rather than summarizing it. For every major decision, ask, "What assumptions does this depend on? What happens when those assumptions are wrong? What breaks under load? What breaks under partial failure?" So, we're basically having this agent go through and pick the plan apart not only from the reviewing of the behavior, the architecture. Do we have retries and timeouts? Do we have queuing if we need to handle performance and retriability? Are we caching things properly? If this is a feature that may require a lot of performance issues, do we have a proper cache system set up?

[00:01:46]Are there single points of failure with this plan that's being implemented? We also have a code change reviewer. So, this needs to go through and make sure that the code, the actual syntax, is aligning with something that we consider good code versus bad code. So, do we have logical correctness, error handling, input validation, um etc. Now, granted, you can break this plan up into smaller different agents. You could have three to five of these all do a separate thing. Like, I could probably even pull this out to its own separate agent. This could be an architecture review sub-agent so that they all keep their context window nice and concise when they're actually reviewing the plan. So, we're going to kick this one off and actually see how it does with Grok 4.5.

[00:02:25]I am using a lesser intelligent model to kind of prove this and also so that we don't sit here waiting forever for this model to finish, right? But, starting off with a more intelligent model like Fable 5 or Soul GPT 5.6, the review agents may find less at the end of it, but I guarantee you they will still find things. I asked it to investigate another application I'm building out called Fleet, and I want to see if it can add in a new feature. It recommended that I should add in the ability to automatically reply to get up issues or get up comments when it's done running its agent. I basically said here is I'm going to kick this off in the plan mode, and I'm going to be using Cursor for this instance. I'm not using Cloud Coder or Codex. I'm actually using Cursor so that I can actually use Grok 4.5. I do find Grok a pretty fast model. It's just not that intelligent. I have caught it doing some pretty dumb things, but it's decent and it's very cheap. So, let's let this run through its plan mode, and at this point it should be going off and investigating my code base to try to understand what exactly what it need to potentially change in order to implement GitHub reactions or comments, okay? So, we're going to let this run, and then I want to run through the adversarial reviewer agent to show how much stuff the initial plan is probably going to miss. All right, so I'm going to say three so I can actually type in a revision, and I'll say use the @adversarialreviewagent to check the plan and report back with an updated plan and the things you've changed or found that were missed.

[00:03:53]So, again, we're going to let the agent go through with all that information I kind of showed at the beginning of the video, and we're going to see how much it actually missed from the original plan, which is why you should do this even a couple times. Don't just do this adversarial review once. Often these models will cut off after a certain point. You can keep on iterating and keep on having it do it over and over again until the plan is extremely solid.

[00:04:15]All right, so their adversarial reviewer is done and it did find three main blockers the plan before implementation. Critical and high missed by the original plan. Do not post error.run critical run error is untrusted runner agent text with no free text scrubber. Obviously, you do not want a potential agent that errored out to take that error and write it directly into a GitHub comment. So that's a huge critical security issue that it found.

[00:04:42]Errors can have sensitive information, and the fact that the original plan was blindly just going to take the output of the agent run regardless if it was successful or not and just put that in a comment is a problem. Number two that it found, duplicate comments on retry. So the outbox is at least once great issue comments is not item potent unlike reactions. Workers die after post second comment after lease expiry. So in this system, I do have some retry set up and it's saying that it could potentially duplicate the post even if the worker were to die. So cool, the adversarial reviewer found yet another thing that it should probably add into the original plan. And then number three that it found, PR comment 403 high create issue comment mints only issues right.

[00:05:25]Reactions already mint issues plus pull request fleet on PRs would fail silently under retries. Okay, so again, this is part of the retry mechanism. It's saying that the way the plan was written as of right now, it would fail on the retries anyway. Now scrolling down, there is a medium section. So run link is off workspace scoped label as member only comments must stand alone. And two other things that it found as well. So, what I'm trying to show here is that even though you have an initial plan, you should run another review over that plan. Honestly, I've even ran two or three additional reviews using different models. Like, you can have Fable 5 write the initial plan, and then GPT-5.6 Soul review it. You can even have Grok maybe review it. I will say that when you use a less intelligent model to review stuff, sometimes it finds things that aren't actually real issues. And so, you do have to keep that in mind that the more intelligent models are probably the ones you should be sticking with with the plans. But, the actual models you use for the implementation could potentially be a little bit dumber models like Haiku or Sonnet or even Grok 4.5. You want the plan to be as solid as possible before you kick off the implementation because again, you are charged on the output tokens. And as it's implementing and generating a lot of code, that's where a lot of the charges kind of occur. So, that's basically all I want to show you. And then, it's obviously after you implement this plan, you should probably have another set or fleet or swarm of agents going through and reviewing all the code to make sure A that it actually followed the plan, and B that the code that was written is actually secure. There's no syntax errors because there are different levels of security. There's security in terms of the architecture and the high-level design of your system. But then later on in the code, the implementation, there are security concerns there with how you're calling functions, what are you passing the functions, what are you catching in errors and printing out a logs. These are all things that are very important that you need to have agents review through before you can come at the very end and just do a human review to make sure nothing was missed. All right, I hope you guys enjoyed this little overview of adversarial reviewer agents.

[00:07:25]Definitely hook them into your workflows to make sure that all the code and the plans that you're writing are the highest quality possible. All right, that's about it. Have a good day, and happy prompting.