Automating the Hardest Parts of Game Dev (Unity AI)

Added: 2026-05-10

[00:00:00][music] [music] [music] [music] >> Hey guys, welcome. So, as you know, game developers are living in a really weird time right now. You've got people on one side that scoff at the mention of AI, and then you've got people on the other side that are fully embracing it. I sit somewhere right in the middle. For art, I don't really use it for more than placeholders, but for coding, for debugging, for planning, my day-to-day workflow looks totally different now than it did a couple of years ago.

[00:00:53]Especially now that you have agentic workflows like what you get with Cloud Code. And about a week ago, I learned that Unity was launching the open beta for Unity AI, bringing that same kind of agentic context-aware workflow right into the Unity editor. And they had some sponsorship opportunities for it. And since I just paid my taxes and my fridge also just died, the timing for this was kind of perfect because I need to buy a new fridge. Thank you, Unity, by the way, for sponsoring this video and paying for my new fridge. And I don't often agree to doing sponsorships before being able to test what I'm going to be showcasing in the video beforehand because it's risky. If I hate the product, then that shows. If the product sucks, then that shows. I can't really fake my way through that. And with AI, it's even more risky because I'm kind of at the mercy of hoping that their AI model doesn't suck. So, what I pitched for my video idea to Unity was making a fire simulation using Unity AI, and not just like fire visuals, but a full-on simulation that spreads fire and heat across the walls and the floor. I figured that the simulation logic and the compute shaders and all that good stuff would take a couple of days to get through. Long story short, it didn't. It took barely an afternoon. So, just for fun and because I had the extra time now, I decided to make way more stuff for this video just to kind of show you what Unity AI can do. How it works is it has specific skills that it'll activate only when necessary to save context. And it gave me what I expected, UI, input, etc., etc. It didn't mention anything about jobs or compute shaders, and I also needed to know if it could do calculus and the other types of math that are usually required for really complex simulations. So, I pushed it a little and it gave me an answer that I liked a lot better. I was also curious if it knew about new features, and at the time of recording this, Unity 6.4 was released about a month and a half ago. In 6.4 they added 2D custom render passes, which I was really excited about, and it gave me info about that as well. So, I was satisfied that it sounded like it knew what it was talking about. So, it was time to start a plan.

[00:02:51]And if you're not familiar with Agentic workflows, plan mode is awesome. Unity thinks about it and reasons everything out step-by-step. It has full access to your project, and for the fire system, I actually already had a system that I wanted to integrate it with. So, that was a good test as well. I wanted to keep everything on the GPU side as much as possible, so I'm avoiding the use of looking for colliders and stuff like that. Instead, we have a texture of the level. So, it can read the texture and use that to determine where the floors and the walls [music] are. Once it's done planning, it spits it out in a markdown format and you can give it feedback if you need anything revised.

[00:03:26]Once you approve the plan, it'll go ahead and save that plan to your project, and then it will ask you if you want it to go ahead and execute it. So, it breaks it all down into steps and checks them off one by one. And importantly, you have full control of what it is allowed to do. So, by default, it needs your permission to do just about everything, and you can change those permissions anytime in your preferences. So, here's the very first result, which I was actually really surprised by. In my experience, you're rarely going to get exactly what you picture in your head on the first go, but you can iterate really rapidly. And there's a lot of correct elements here.

[00:03:59]You can see it's sort of following my walls and floor, but there is a bit of an offset happening. And at the side, it actually does really look like fire. So, to fix the offset, I had it create a debug overlay that I could look at, and you can see there definitely is an offset happening here for some reason.

[00:04:14]Embarrassingly enough, the issue was actually coming from the texture of my level that I baked and then gave to Unity AI. So, my tool was actually the issue here, and it also fixed that for me. Now, the level lines up with the debug view, which let our fire properly follow the floor and the walls. Now again, it looks good on the ends where it's disintegrating, but not at the top, which is the most important part. So, I opened another planning session and told it what changes I wanted. This time I got the visuals way, way better with lots of control options to stylize it however I like. The new issue is now when I hit play, the whole level just sort of ignites almost right away, which is not what I was going for. So, I kept chipping away at it, and I did get to see how it handles errors in the console. It's actually really easy. You can just right-click and hit add to assistant and say, "Can you fix these errors?" But, I was still having trouble getting the fire to spread in a way that I liked. So, I had it build me a debug visualization tool that let me see how the actual heat was spreading. And with this, you can see there's obviously way too much of a bias for the heat spreading vertically, hardly spreads horizontally at all. So, I told Unity AI what the problem was, and then we had a result that was much more natural and much closer to what I wanted. Now, when I have scripts like this that have too many sliders or too many fields, I almost always end up creating an editor script to kind of pretty them up and just keep them organized and easier to manage. [music] And it had no issues at all with that. We made a couple of last-minute adjustments, and now we had a really nice working fire simulation that followed my level layout. This is what I thought would take me 3 days and it was done in about 3 hours. So, for this video, Unity made the mistake of giving me way too many AI credits and I have all sorts of other things that I'm curious to try out with it. And one thing I've been super interested in lately is destructible terrain. I'm thinking something similar to Worms Armageddon. I love that game.

[00:06:11]So, we made another plan and this is not what I was picturing at all. It's kind of cool, too, but it's not what I wanted. Though, when I look back on my prompt, I wasn't very specific or clear.

[00:06:22]Plus, it said right there in the plan that it was going to remove tiles within a certain radius and I approved that plan. So, I tried again and this is much more like it. Though, we aren't actually falling through the holes and we had some bigger problems like these new invisible colliders everywhere in my level.

[00:06:39]So, it cleverly, for performance reasons, broke the level into a bunch of chunks and then scanned each of those chunks for colliders. And it looked right as it scanned through everything, except the squares were actually overlapping and creating invisible colliders everywhere. It fixed that easily enough and you can see now we're drawing gizmos everywhere to ensure that it's working properly. And it does appear to be working well now. The colliders update, it looks just how I want it, except it's kind of slow. This is bringing my scene down to about 70 frames per second. If I bring the cell size even lower, we'll dip [music] down into the 40s. So, I asked it to analyze the performance and why it was running so terribly. I didn't even have to exit play mode. I just asked and let it run.

[00:07:19]It did do a couple of small optimizations for me, but apparently 75% of my issue was actually on draw gizmos.

[00:07:25]So, after a few small changes and disabling the drawing, I was back into my normal 6 to 700 frames per second range. Now, I also saw that you could create sprites and sounds and all sorts of stuff. And I was curious as to whether it could set up a simple particle system for me. So, I asked it to create some explosion sound effects and a particle system and play both of them whenever we blow a hole into the train. So, you can see it working away there. You don't even have to open a different window if you don't want. You can do it right here with Assistant.

[00:07:52]It's making the sounds. Looks like it created a little puff of smoke to use as a texture in our particle system. Now, for me, some of the sounds it produced were really quiet and some of them were insanely loud. So, they're not meant to be used for final production-ready assets, but they are fantastic for placeholders. But, I am really impressed that it was just working out of the gate. Except, something with the smoke looked more blocky than I was expecting.

[00:08:14]And I don't know why AI has so much trouble creating assets with no background. But, yeah, you can see it made the smoke texture with the actual checkered background as if it was transparent as a part of the background, which is just kind of funny. Although, there is a remove background button right here that is a part of these sprite creation tools with Unity AI. So, I saved that, and now it's looking much nicer. So, now that I was actually impressed, I got really excited because I've had all this really weird stuff that I've wanted to try for a long time, but I've never had a great excuse to try them. So, I made a boids simulation. And if you don't know what that is, it's a sort of bird flock simulation. If you get your settings right, you should get objects that fly around, and one, they don't crash into each other. Two, they fly the same direction as their neighbors. And three, they stay with the group. I've always wanted to see little triangles flying around in my scene acting like actual birds. And Unity AI didn't struggle with this one too much.

[00:09:07]It chose to use Unity's job system to navigate them. But, if we wanted a lot of boids, then we would need a compute shader because just at 2,500 boids, I was already down to like 200 frames per second. And it didn't really struggle with that, either. We're now at 50,000 boids and running at a nice 6 to 700 frames per second. I don't have any idea why I would ever need this many. It looks kind of insane. It did actually remind me of some specific flock patterns that I've seen in real life.

[00:09:33]And I did a quick Google, and I discovered this is called a murmuration, in case you were wondering. All right, next. I have always wanted to dabble in procedural generation, specifically trees, because I find drawing any kind of tree that's not an evergreen tree to be really difficult to do. And this time I was a little bit less specific in my request and gave it a little bit more room just to see what it would come up with. But I really like this because what it did was it asked me what direction I'd like to take while it was creating its plan. So it's always trying to keep you in control and try to create your vision, which I love. Another cool thing is you'll often notice that as it's implementing your plan, it's going to take a screen capture of the scene just to make sure that what you want is actually working. And usually this is how fractal trees look. They're perfectly symmetrical. I wanted a bit more randomness to it to make it look a bit more natural. Plus that way we'd get a bit more variety cuz maybe I want to use this as a tool to be able to spit out as many tree meshes as I want. They would all look unique, but they would all have the same visual style. So I figured that adding some branching probability as well as more angle variance would make a huge difference, and it turns out that it very much did and I'm already way happier with this.

[00:10:42]We added some finer controls for the color to make it look more tree-like, but there was still no real leaves [music] on this thing, which is what I asked for next. I told it to just draw some simple arrowhead shapes, which I think for this style of the tree it fits fairly well, as well as just generate random colors for the leaves between two color fields that I choose in the inspector. So, now that I'd created life, I wanted to destroy it.

[00:11:06]So this one is really cool. It takes a snapshot of the tree and turns that into a render texture and scans through every single pixel on that render texture and for every pixel that is a part of the tree, it turns that to dust and makes it fall and pile up on the ground. And at the same time it uses this scanline feature on the shader of the tree to make it disappear at just the right spot. I also created a second totally different type of random tree generation from what's called an L system. These are actually really strange. It reads and builds the shape of your tree from strings that you punch in. So, this is not just limited to trees. You can make all sorts of random stuff. But, I was more interested in trees right now, so you can see this one's a bit different.

[00:11:45]I'm not as big a fan of this one, but I still think it's really cool. And of course, same with this one, I wanted to turn this one to dust as well at the click of a button. So, with Unity AI, there are a lot of really great features here. If you enable checkpoints, then you can roll back any change to a previous state if you don't like the new changes. And I'm sure you're familiar with the term AI slop, and I think Unity is as well. From the way that this was designed, it's pretty clear to me that Unity is positioning this as a tool to help you iterate and build games faster rather than eventually replace you. Any asset that you create using Unity AI is tagged with Unity AI, which is going to make it really easy for you to find them all and swap them out later on. You can even hook up your own favorite AI if you prefer Claude or Gemini or whatever, and it comes built in with a secure MCP server if you want to connect those to Unity AI if you want. This is a genuinely useful tool. It's going to help you debug. It's going to help you plan. It's going to help you iterate and rapidly prototype, and be able to prototype with assets that aren't just capsules. Let's use animations for example, just in case you happen to need something to get those creative juices flowing a little bit. I didn't really know what it was looking for in the first and last frame, so I put my very best drawings in there.

[00:12:58]>> [laughter] >> And the results are actually weirdly good considering that I asked for an idle, a walk, and a jump animation. I guess that you're meant to ask for one at a time.

[00:13:07]It's actually quite good when you give it actual art, and I just had it generate this character. So, if you are wanting to check out the Unity AI open beta, there is a link down below in the description. It is really fun to just play around with it. I highly recommend it. They give you 1,000 free credits to play around with. I used less than half of that for everything [music] that you saw in this video. And I'm curious to know, if Unity AI, will it change how you end up prototyping your game? Or maybe there's something in your workflow that you would use this to automate. Let me know in the comments. I'm curious to hear what you would use this for. For me personally, I'm going to be using this a lot for [music] UI, I think. And that is all I got. I'll see you soon, guys. Bye.