We built a bulk debugging tool (fix 20+ bugs at once)

Added: 2026-05-09

[00:00:00]Hi, my name is Devon. I'm one of the founders of Devonabox. Devonabox is a tool for basically taking source code and finding all of the software and security vulnerabilities and all of the bugs that exist within your code. It's a tool for basically being able to rapidly take a piece of software and identify all of the issues in it to reach a higher standard of software quality. And it's gotten really good at doing this uh to the point where I've been dog fooding it a lot with a lot of AI generated projects. And basically on every project I'm pulling out about 50 to 60 issues, which is really cool. However, I've been running into the problem of going through and fixing 40 to 50 bugs is actually taking a lot of time. And that's because we have a feature where you can generate a patch, but it's only on a defectby defect basis. And this is good because basically it uses our big software map that we use to find all the vulnerabilities to find out what the root cause of the issue was and then basically communicates that issue. It gives you a unit test so you can prevent it from happening again and gives you the source code needed to fix that issue. However, this is a timeconuming process and it's also a very uh token expensive process, right? Because even when I go and get that fixed, I still want to use a high-end model to go and fix this up and to review my changes and there's a lot of effort that gets put into solving an individual defect by itself. However, a lot of the defects we're pulling out of the software are all in the same file and they're all slightly related or they all feed off each other in unique ways and we're not really considering how to deal with that as a cohesive hole rather than as a singular issue and I think that's lowhanging fruit to make our system far more effective. So the goal has been basically the inspiration is to build a Google maps like routing algorithm which founds the best route through the software and through the patching process to basically in the least amount of steps possible patch all of the files to eliminate all of the defects. And my kind of thinking behind that is I kind of want a prompt for the implementation, a prompt for the verification and I also want a checklist that I as a human being can come and sit down and go okay I've done that, I've done that. I've done that. I've done that. Everything has been checked. this is good to go. So the process for building that has basically come down to how can we take advantage of this map that we've built for the software. So Devon in a box the one thing that we're doing and we spend a lot of effort building is basically we take raw source code and we transform that into artifacts and then we use those artifacts to bootstrap up to a higher level of understanding about the software. Uh so we're spending a lot of tokens to basically build this high level map of the software and it gives us an incredible amount of potential and incredible amount of detail. So the idea was well I need an algorithm which can basically look at that map calculate a path through it and then take that as a suggestion give it to another large language model which can see broader context and bring in more of the fuzzy logic of okay how do different pieces work and those two combined can create an optimal plan for patching the software and then each individual step I would generate an implementation prompt and I'm also going to generate a uh verification prompt my logic behind that is that every single thing that I do with largeang language models, I always make sure that I have an implementation step where I instruct the model to implement things and then I'm very very uh one of my big beliefs is that you should get the system to prove that it's correct and basically do an analysis. So that's kind of what I'm trying to build into the workflow. Of course, you can just use the implementation check or the checklist, but I think that a lot of people are going to get a lot of value out of having a verification uh step which explains to them that this has actually been implemented correctly and we've verified it. So, where are we at now? I'm really excited to say that right now this is live. So, you can come into Devonbox today and you can click on your scan and then in the top right corner you'll see a generate bulk patch button. You just click on that and you can see all of the defects that are in your project and you can basically filter them. So, I can go for the active defects for all of the vulnerabilities and I can select say critical severity or high severity. And one of the other cool things is that I can also select the AI tool that I'm using to do my patching. So if I'm using cursor, I can get a specific implementation, a specific verification prompt that's tightly wound and tightly designed around the way the cursor workflow works. So I'm not just putting in a prompt and it's the same prompt across all these different providers. I'm taking advantage of the specific tools that these providers have. So if you use Claude code, it's like really focused in on the tools that Claude Code has available and also the things that Claude is good at. Uh and make sure that the constraints and everything we put around the request are focused for what Claude will need to do its job right.

[00:04:26]And of course, I'm really looking forward to hearing the feedback and the iteration that we can get on this to make sure that the prompts that we are generating for you are the highest quality that do let your AI system take care of the step in a very uh bounded and logical way. So then basically I can also filter by my active defects uh if that's a bug or a security vulnerability. We've also got advanced settings so you can filter by the area the assigne the consequences and then you just click generate bul patch and then that typically takes about 2 to three minutes. There's a lot of processing going behind the scenes.

[00:04:57]Again our philosophy is that basically we take this highle model and we look at the entire context of the software rather than just looking at individual files. One of my mottos is that software is not its files. We are not trying to analyze files here. We are trying to analyze software and software is not the files. It is the branching logic and the chains of source to sync artifacts that exist in the software which just do not exist in source code. So that's why it takes a little bit longer. But the quality that you're going to get for that is uh immeasurably better, I think, because you're essentially going to get a step-by-step plan with prompts that are specifically written to give the LMM all of the context it needs to tell it exactly where to look so that it can do a patch without stumbling over or getting confused. And then the valification patch gives it the broader context of where to look to make sure that nothing is around that system has broken. And then you as the human being can come in, click through the checklist and go, "Yes, this is working." Uh, and I think that's going to be a really fantastic tool for people to basically get more value out of what we're trying to do and make their software higher quality faster. Again, like the point of all of this is software quality. And I keep harking on about this idea that when you fix these bugs, you are not just going to get the benefit of that piece of code being bug free. The quality of the code getting better actually seems to have some sort of emergent effect on the model itself. in that when you give an LLM higher quality code to work with, it writes more high quality code. And of course, having Devon in a box as a feedback loop and a mechanism, I think it's going to make it so that you can work with an LLM or leverage these tools in a way where you don't get beat blown off track and you have to spend time, you know, banging your head against a wall because stuff just isn't working properly. You go and you find the bugs, you go and you find the defects, you make your system higher quality, and then you build upon it in a way that's sustainable. And I think that's going to lead to a golden age of software quality. So what's next for us?

[00:06:54]Well, I've talked about this before. I want to add an optimization scanner. We have this high level map of the software. I think it's not possible not just pull out bugs and security, but also identify where you could save money in your project. Uh, and also where you could make the project faster. Uh, and I'm already sort of seeing the benefits of this myself. I've been using this on our own application and I've dropped our response time of our server to sub 50 milliseconds which is pretty cool and also I've dropped my storage cost by 80%. Uh which is a really big benefit to us and I think that as we roll this out to more and more people there is going to be some pretty dramatic optimizations and some massive amounts of money that people can save just by applying this scan to their project and looking at all of the different places. And I'm kind of thinking that the logic is going to be that you scan for your security flaws, you scan for your bugs, and then the next step is you scan for your optimizations, and you go to a higher level of engineering quality because you you've you've clarified by getting rid of bugs and security flaws, you've clarified for the LLM exactly the core kernel of what this software is meant to do, which means that when you bring in the optimization scan, it's going to be an even higher quality. And that's what I'm really excited about. Uh, I'm hoping to get that up and running very soon and I'm looking forward to showing it off.

[00:08:09]So, thank you. Uh, as always, if you want to check out the project and you want to try scanning your project, you can sign up at devonabox.com.

[00:08:17]Uh, we don't have a free trial and this is because we're a bootstrap startup. We don't have any venture capital funding, but if you would like to try it for free, just contact me at devondevabox.com.

[00:08:26]I'll hook you up with as many credits as you need and I'll help you as much as I can. I really want to see lots and lots of highquality software projects being produced because we were able to help them.