I Fixed a 5-Year Magento Bug with AI Agents (Using Instrumentation + MCP)

Added: 2026-05-06

[00:00:01]Hi everyone, welcome to my channel.

[00:00:04]Today I really glad to share with you really cool approach to tackle quite complicated issues in big systems using laidback approach with LLMs and other additional tools that help LLMs to to debug your application. So in this particular case we have Magento 2 just empty project from GitHub we have Luma front end PHP back end we also going to have automated test because for this particular case it's necessary and I would recommend to have test end to end test in general because it's like non-negotiable nowadays.

[00:00:53]So we going send the instrumentation for the bug to spee database which was designed for that and it's called SEQ data loss. So our agent I going to use codex. Our agent going to read those instrumentation records or in other world words props and it going to understand why issue happens send us summary and we decide what to do with this. I choose to tackle pretty old issue which is still opened.

[00:01:34]It's been opened like 2021. It's like previous life for me. So yeah, it says that when you send a ton of GraphQL queries such as like simple some get queries, PHP going to open sessions in radius many times. It's just I'm not going to dig into too much details here because it's laidback session. It's debugging without even digging into it like deeply.

[00:02:14]So yeah, just let's say we have problem with radies and we need to optimize performance here and yeah it has a huge history. people have workarounds.

[00:02:29]There is no currently there is no pull request. So we may even contribute today. Let's see if it if it happens.

[00:02:40]And yeah, let's go back to our diagram.

[00:02:45]I going to introduce you introduce to you tools I built to work with SEQ in easier way. So I developed really straightforward plug-in that going to allow us to publish events to database from the storefront and from the back end and it also saves all the login to this database because it's easier to read for for agent. It's easier to read it from the database than from files. So it doesn't make sense to have files locally.

[00:03:25]This magenta module is completely for local development. I wouldn't recommend to push it to production because in production you have new relic and all the other cool stuff. So it's needed only for local LLM development.

[00:03:41]>> Yeah.

[00:03:43]On receiving side it's like right hand side and on receiving side we have MCP extension for intellig. So intelligy has its own mcp server with different tool sets and we just extend it with our own tools.

[00:04:02]So yeah and it's like agent diagnostic.

[00:04:05]It's compatible with anything like cursor codeex whatever you want to use and yeah it's not necessary to have this MCP extension agent can use I don't know wget get what else CLI commands it's like C z is so necessary but it just going to simplify your life for agent and maybe save some token because the schema of request are defined and the capabilities of ECQ is also defined.

[00:04:45]So agents and wouldn't need to think how to form requests and so on just do searches right away.

[00:04:54]Yeah. So we have those two tools and to save time I also prepared three prompts.

[00:05:02]First one is for for investigation.

[00:05:08]Second is for proposals and third is for fixing.

[00:05:15]So let's start with first one. So using our skill file in new magentto module we will debug the problem. I've prepared I've prepared spec playright spec. It's going to try to overwhelm GraphQL server with multiple simple queries to to reproduce the issue. So, let's try to let's try to run this test first and then we need to make sure it fails.

[00:05:58]Let's try this one. Then we're going to fix the issue and run test again. So, it's going to be green.

[00:06:08]So, yeah, let's try to run it. As we can see, we expect to have request faster than half a second and instead of that, we got a ton of request like way slower which confirms the issue.

[00:06:29]Okay. So, let's go ahead and start a new chat and using first prompt, we going to investigate what's going on with Magento for five years. Five years, guys. Today is like 2026 April.

[00:06:56]Yeah, it's definitely high time to fix this issue. I already asked Chad to get familiar with the environment.

[00:07:06]It's actually Marx shoes container. Very popular, very powerful, my f one of my favorite tools ever and highly recommend using it.

[00:07:21]And yeah, so let's go ahead and paste it here so our codeex can investigate what going on and we can wait for first events in the sec. I currently have some events. I believe we can clear them all to not step over the those bits from pre previous investigation.

[00:07:52]Yeah, we cleared them all.

[00:07:56]Now we can only wait and see what happens next. Okay, some new files started to pop up. So progress it's quite quick I would say. I guess there is no events yet. Yeah, there is no events because it's needed to at least run the spec one time.

[00:08:26]So yeah, let's wait.

[00:08:29]I don't know. It's so fun to wipe code on camera. I need to do it more often I would say because it's just like sitting with smart face while LLM do the job and you just like chilling. We got first events here. So our workflow is working. So LLM using our module posting some events here to debug what's actually happens.

[00:09:02]How cool is that? I believe it's super cool for me. It's like it's magic. 10 minutes of execution. Quite a task.

[00:09:12]Not easy. Not easy.

[00:09:16]Even for codex on high mode with like high context window still takes it still takes time. But like I believe it's going to be less than six years. So we can wait for even I believe even our going to be justified maybe even two. So yeah just we need to be patient. Of course I need to be patient because I going to cut all this.

[00:09:49]So you going to see the result right away.

[00:09:54]Okay. After 26 minutes of investigation, we got findings.

[00:10:03]So the rooies are starting customer session during user containing before. Okay. Blah blah blah.

[00:10:13]Guys, this is a laidback wipe cleaning session. We don't care about all this to be honest. We find root cause.

[00:10:26]Great. So it's high time to do what?

[00:10:32]To propose some solutions.

[00:10:35]The main requirement here is to be fully backward compatible because it's Magento. We don't want to break anyone.

[00:10:48]I noticed the pattern that works really cool. I actually like notice I just stole it this pattern from one guy. I don't even remember because it used to be like I don't know it was like maybe more than one years year ago.

[00:11:11]So when you not just asking LLM to fix the issue but you asking it to provide ratings for possible varants with ratings it's like it works just better. I don't know why but it works better. So I always include please score it like from 1 to 10 1 to 100 or something like this with this properties. So it's it works great. So you can use this life hack from me or you can share your own life hacks in comments below. So yeah, let's go ahead and put this prompt into area so we can proceed with our fun contribution evening I would say.

[00:12:13]Okay. So, we got options how to fix it after quite quite a short time I would say. Yeah.

[00:12:29]Option one, introduce GraphQL specific customer session user context implementation.

[00:12:36]that short circle guest unless the request is actually trying to identificate a customer.

[00:12:47]Okay. So again I want to dig into the details and it's not our goal because we are coding here. So yeah, Vitali, forget about this. Just let's look at the numbers.

[00:13:02]And those are quite high, I would say.

[00:13:06]But this one is I don't know like nine from 10. I'm not agree with nine from 10.

[00:13:19]No public beh okay maybe not a big deal but let's see seven from it's no for sure it's red flag no and again compatibility is great but others are okay so yeah option one wins option one wins in. So, let's go ahead and copy the sword prompt and we go with option one.

[00:14:06]We go with option one.

[00:14:11]So, yeah, we going to fix Magento today, guys. Can you imagine this?

[00:14:17]Whoops.

[00:14:19]And after providing the fix, we still have red tests happens.

[00:14:25]Okay, let's give it some time. So, I believe it's it will figure out what went some sideways. Yeah, Codex having fine time investigating it. Okay, we officially working on it for 1 hour. For one hour.

[00:14:46]Yeah, it's fun. Like for me, not a big deal of course, but yeah, we spend a ton of electricity somewhere.

[00:15:00]And I would say this issue is perfect showcase how LLMs are in inconsistent because I've instruct LLM to use MCPS whenever it's possible to use project wrappers and it's described in general rules.

[00:15:24]It's described in skills. If we go to skills here, one skill which leads to my Magento SEQ and it also deliberately says to use MCPS and over time LLM just forgetting about it whatever I do and I need to stop the process and remind please use tool set correct tool set because it's when it forgets about it, it forgets also about projects the correct project structure because it's not standard Magento installation. It has different folders and it start patching wrong places running commands in wrong places and it's it's terrible and yeah and currently I have no cure against it especially for task with such a huge volume because it's it start to concentrate on issue itself and just forgetting about anything else.

[00:16:33]Okay. So, we on the second loop now.

[00:16:37]Yeah. So, let's go ahead and repeat this one because first fix wasn't good enough.

[00:16:52]So, we need to do the research one more time and see the options we got.

[00:17:02]Okay. So, we got some options.

[00:17:06]Yeah, let's check option one. It seems to be the best dateless.

[00:17:16]But yeah, again, backward compatibility is not ideal.

[00:17:21]CH limited and okay, let's hope it going to work out.

[00:17:28]Yeah. and third prompt.

[00:17:34]Hopefully, it's going to be last iteration because this way I going to spend all my tokens this evening. It seems like it can take more time because LLM is searching for any dependencies that happens on GraphQL request that can load at customer session and yeah it takes some time of course. So the work went wrong completely on Sundays.

[00:18:07]Sundays I got multiple issues with like MCP and also model started to behave strange and I run out of tokens. Yeah, had to interrupt that and go to sleep.

[00:18:22]But I'm determined to fix this issue because it's been in Magentto for too long. So now is 2 days after I finally have time to proceed.

[00:18:34]And yeah, I have a little prompt for reminding to model what we were doing. So the idea is because I try to embrace what it is doing to understand whether the direction is correct because in this particular case when we have such huge and let's say tricky issue it's not always good case just to just to rely on AI and not be in the loop. The idea is to use a separate session context which going to be like stateless and not going to load any kind of session when we have cachable get requests. And yeah, LLM tries to replace it whatever whenever it possible only for those requests and the task is not so easy. To be honest, I don't know how backward compatible it is. Maybe not, but we may polish it later on.

[00:19:51]At first we need to fix the issue and maybe we can introduce separate quick route quick query so we can ask user to switch to new query without bug to not break backward compatibility. Let's see but for now we need to just to to fix it in general and then we can decide what to do with it. So I going to go ahead and proceed with session replacement.

[00:20:24]Interesting that the LLM identified that we have Magento tax model is being loaded during during request for store config which is just store config. Why would we load taxes? It's interesting.

[00:20:47]So yeah, mod modularity of machanto sometimes like very s is very s surprising.

[00:20:55]So yeah, and hopefully we will fix this issue as well like we going to have final ver verification and if everything is green I going to make a pull request.

[00:21:16]So let's see how it goes.

[00:21:21]So it was a lot of fun. I believe it took about four hours or something but a lot of time was spent due to issues not only because of like model thinking and that's it we had issues with different stuff like with permissions with MCP yeah with different things. So in ideal world it would take maybe 2 hours two 2 and 1/2 hours something like this but again quicker than 5 years. As you can see guys test is green requests are quick so we can do pull request.

[00:22:08]I want to thank you if you watched to this moment. I hope you enjoyed this journey with me and if you want to support this channel, please just subscribe. You can buy me a token. Thank you so much for your attention.

[00:22:26]See you next time.