Supply chain attacks? Wait until AI agents get compromised

Added: 2026-05-14

[00:00:00]I'm recording this a couple of hours after an extremely devastating supply chain attack started, a supply chain attack that spread out to many more NPM and also Python packages. And at the point of time where I'm clear when and where it'll end. And I did create a separate video on my YouTube channel where I dive deep into this specific supply chain attack because it was quite elaborate, and I do a deep dive there where I explain all the details because that is quite interesting. But here I wanna talk about supply chain attacks and security and AI in this age of supply chain attacks and AI in which we're living because I'm sure things will get worse, and I fear that many people don't really see all the dangers yet. And there is more we as developers and users of technology and AI, to be honest, have to do.

[00:00:59]And this affects us even if we're not developers.

[00:01:02]I know that most people watching or he- hearing this are developers, but as I will make clear, this is not just about writing code and not just about supply chain attacks as you know them. But let's start with the basics.

[00:01:17]What is a supply chain attack? A supply chain attack in the context of software development simply means that a dependency you are using is compromised.

[00:01:30]That is, in a nutshell, what a supply chain attack is all about.

[00:01:33]And compromised, of course, can mean all kinds of things.

[00:01:37]What we typically see is that we get malicious code in a compromised package that harvests credentials and, and tokens. So that scans your hard drive to find secrets which you may have in.env files or your AWS credentials and so on, and it then uses those credentials to access your accounts but also to spread itself, so to affect other packages. If you're an open source, uh, package maintainer or even if it's closed source, if you're working on something, some package, some tool other people use or depend on, it's, of course, interesting to compromise your machine to compromise that package or that tool that you're distributing because, guess what?

[00:02:26]That will then affect more people.

[00:02:28]So all these supply chain attacks that we see including that supply chain attack that started here with the TanStack they are worms that spread to other packages to affect more and more packages and ultimately, of course, also machines on which these packages are installed and used.

[00:02:47]Now there are some things you can do to protect yourself, and I created a separate video about that on my other channel, Uh, things like making sure that you only install packages that are at least three days old or something like this, package versions, I mean, running your code in a dev container machine. These are all things you should do.

[00:03:10]You should also not store plaintext secrets on your system. Instead, use a service like InPhysical or Doppler or anything like that where you store secrets in the cloud or in some other form in an encrypted way so that if an attacker does scan your system, they don't see those plaintext secrets.

[00:03:31]These are all things you, you have to do right now.

[00:03:35]It's important because these supply chain attacks, they're getting more. We's- we're seeing more of them. And w- why is that?

[00:03:42]It's certainly not the case because you weren't able to run attacks like this many, many years ago.

[00:03:50]It was possible back then and it did happen back then, but the frequency has dramatically increased, of course.

[00:03:57]And AI is a big reason here. So let's take a look at the role of AI.

[00:04:05]AI is a big reason because, of course, it makes it easier to run such attacks.

[00:04:12]You a- if you're an attacker, you can, of course, use AI to analyze all kinds of repositories out there of packages you might want to compromise to see how are they building their packages, how are they distributing their packages.

[00:04:29]For example, the TanStack attack which started this recent supply chain attack, there the maintainers used a theoretically secure approach using the trusted publishing process by NPM. And again, I do dive deeper into that in my separate video on this channel. Uh, but what they also did is they used a, a certain GitHub Actions event trigger in a certain way where it was not secured perfectly, and that allowed the attacker to use cache poisoning to get malicious code from an untrusted environment into a trusted environment.

[00:05:10]And that is how this attack started.

[00:05:12]Again, details in that other video.

[00:05:15]But of course, AI makes it easier to analyze repositories, to analyze their GitHub Action workflows other CI/CD provider workflows. AI can mass analyze all these workflow scripts, all the code, and it can look for security vulnerabilities.

[00:05:33]And of course, maintainers can also use AI to scan their repositories and look for potential attack vectors, but as an attacker, you're naturally always in the advantage there because you can look for everything, you can try out all kinds of things, whereas as a maintainer, you have to anticipate everything.

[00:05:55]And AI can help with that, but it's still not perfect.

[00:05:59]You have the advantage there as an attacker. And AI has simplified that.

[00:06:02]AI also, of course, simplifies the process of writing malicious code. It simplifies the process of writing any code. And of course...And you know that if you watched other videos by me or heard other episodes, I'm a big proponent of looking at the code, doing code reviews, not outsourcing everything to AI.

[00:06:26]But of course, it's clear to me at least, that you should use AI as a productivity boost. And we're still all figuring out how much usage of AI is right. Some people will tell you 100%, they don't even look at the code.

[00:06:40]For me, that's not the case, but there is a spectrum here.

[00:06:43]Either way, AI definitely makes it easy to pump out lots of code. And if we're talking about malicious code of course, there are certain things that are important to you if you're an attacker. You want code that does the job, that's not super easy to detect. But you don't care if it's beautiful code, if it follows certain best practices. Your best practices are that your attack goes through. And of course, AI can help with that.

[00:07:13]It can help with writing all that malicious code, with coming up with ideas on how you could attack packages.

[00:07:20]So that is where AI helps, but that's only one part.

[00:07:24]Making it easier is only one part of the story.

[00:07:28]The other very important side is that there is more code than ever, so that means there are more targets than ever. I mean, maybe you followed that blog post or the entire story around all the GitHub reliability issues and GitHub downtimes.

[00:07:46]Well, the, the reason for that is that there's more code being pushed to GitHub than ever because of AI, because it's easier than ever to generate code. And more people than ever are generating code and writing software, including many people that have no idea of what that code does, what it's about.

[00:08:06]White coding is a big thing, and it has its, its use cases. I mean, if I want to merge five PDF documents into one, I'm very happy telling an AI agent to do that for me, and it will probably then write some code does it. And I don't care about that code. It's a one-time task, right?

[00:08:25]But if I run that on my system, then of course, the agent may install some package to merge these PDF documents that has been affected by a supply chain attack.

[00:08:36]So I don't even know that a certain package was used then because I just cared about merging some PDF documents.

[00:08:43]So there are more situations than ever where packages are being installed because there is more code than ever being written for software, but also for one-time tasks, and that, of course, makes running such supply chain attacks attractive than ever before. Because there are more targets than ever, many targets that have absolutely no idea about software security, cybersecurity, or anything like that.

[00:09:09]And let's be honest, many of us developers too, we may theoretically know about certain risks, but we may not care because it's so convenient to just get the job done. And we have to rethink here.

[00:09:25]We have to rethink. We have to secure our machines. We have to make sure that we develop in secure environments, so in, in virtual machines, in dev containers, that there are no credentials lying around.

[00:09:39]And if we use AI agents, which we likely all do, we have to be careful there too, because there too, are two ways of, of being in danger.

[00:09:51]So let's take a closer look at how AI agents are problematic here. One problem here is what I already mentioned.

[00:09:59]When we use AI agents, especially when we maybe use them for things that are not directly related to writing code or software, but also when we use them in order to help us work on a program, we don't necessarily see everything they're doing. If you're using Cloud Code or anything like that, and I have nothing against these tools, indeed, I have courses on Cloud Code, Cloud Cowork, Codex. I have courses on them because they are very useful. But if you're using them and you just let them go and you tell them, "I need this feature," you might not even realize what they're installing.

[00:10:42]So again, packages being installed, you may be compromised, you may be affected.

[00:10:48]Now, one defense against that also, of course, is to limit the amount of packages you wanna use. But again, if you're using an AI agent, you may not be in control there. It may install packages installed. So that's one obvious danger, I guess.

[00:11:04]Here's the less obvious one. AI agents are super attractive attack targets.

[00:11:12]Now, what do I mean with that? Well, these supply chain attacks, I, I mentioned it, spread like worms.

[00:11:19]They attack or affect all kinds of packages.

[00:11:23]Now, it would, of course, be very interesting for an attacker to infest Cloud Code or Codex or the PyCoding agent or OpenCode or any other agent, any other AI agent. Why?

[00:11:38]Well, if you had malicious code that is actually optimized for also or exclusively affecting and infiltrating AI agent packages and repositories and code bases, then of course, that malicious code could contain prompt injection parts.

[00:12:00]So it could, for example, explicitly target all these AI agents to change their code such that it is not primarily about exfiltrating data.So the package code itself, the malicious code that's injected, is not about exfiltrating data, let's say, but it is about tweaking the AI agent code such that it has some special instructions that makes it do stuff on the machine where it's being used, so on your machine for example, that you don't want it to use. Imagine Claude Code having a secret system prompt which normally would be set by the Anthropic employees, but which now is set by that malicious code that tells it to ignore whatever you're asking it to do and just fake that it's doing what you're asking it to do, or that it should do what you're asking it to do. But that in addition, it should scan the system for secrets. That in addition, it maybe should write a little program that does the scanning and that then sends that data off to a certain remote server or anything like that. The sky is the limit here, because suddenly you have like a Trojan horse on your system.

[00:13:11]Suddenly you have an AI agent going rogue on your system and not because the AI is going rogue, not because the model is bad or wrong, but because the agent code itself and its system prompt or whatever has been affected and has been compromised.

[00:13:29]That is not an unrealistic scenario, and I guarantee you this will happen at some point. It's such an obviously interesting target. AI agents are such an obviously interesting target. This will happen.

[00:13:44]We'll see a new level of these supply chain attacks, as they don't just do what they normally do, affect a bunch of packages and harvest credentials, which is already horrible increasing. But we will also see AI agents going rogue because of malicious code.

[00:14:02]Only a matter of time. So there are many, many layers here as you can see, and that is just this new reality in which we're living now. I- I guess it's a bit like with the early of the internet. It's all bumpy, uh, whilst we're still figuring stuff out, and we'll have to figure out how to ramp up security and how to do stuff securely. And one obvious step, which but also for running AI agents, is that you don't wanna do it in an environment where things can go wrong.

[00:14:32]You don't wanna run it in an environment where you're storing credentials or secrets or any other data that matters to you.

[00:14:38]You don't wanna do it on your main machine.

[00:14:41]You wanna run agents, you wanna build software in- in isolated virtual machines, remote machines, anything like that where the blast radius is limited because again, it's only a matter of time until things will go wrong.

[00:14:54]And- and we have to realize that. That's the first important step. Things are changing quickly and security is a huge issue, and it will stay a huge issue and become even more of an issue as AI accelerates, as these AI models, uh, get smarter, especially combined with the tools in which they're running, and as this introduces a whole lot of new capabilities.

[00:15:19]And as, at the same time, it's so much convenience added by them.

[00:15:23]Convenience is always dangerous because that makes you get sloppy and overlook stuff. And yeah, AI is everywhere.

[00:15:31]So many people that don't know anything about cybersecurity are using it, and even the people that do know a lot about it or something about it are in great danger. So we're in for- for a hot ride here, I think, and we have to rethink and- and be super careful where and how we run agents and- and work on our code