How to Train a Neural Network like a Boss

本站添加: 2026-05-19

[00:00:00]So, I'm sure most of you have seen the figure robot. There's a live stream going on. It's day five right now where it's just continually sorting these packages. And the objective for the robot is to use its cameras, its sensors, to analyze each package to ensure that the white label on it is down. It's facing down and then it just pushes it off. Now, how does that work?

[00:00:24]Well, that is basically a neural network. It uses a camera to identify a label, a white label on differently colored packages and if it's facing up, flip it over and push it off. That is a neural network and it's exactly the thing that I had to create and figure out how to create myself. So many of you know I've been building this pool projection system and it's for helping you get better at pool. You can create really cool drills and really cool games. And one of the problems that I recently solved with a neural network is being able to accurately identify each ball. Not just in my environment, that's easy. But in all environments, like look at all these images right here. These are not real images. These are actually Blender renders of different layouts of pool balls in different colors like a different lighting environments, different felt and cloth colors and also different camera distortions as well like a barrel roll, focus, exposure, etc. And so you might be wondering, well, what's the purpose of that? Well, in order for this system, this software to work in a variety of different lighting environments and pool tables and pool balls and all that good stuff, it has to be trained. A neural network has to be trained on literally thousands of images. So, I'm going to show you exactly how the process works. So, the first step involved finding a realistic pool table. If you ask Claude Code or any other AI agent to use Blender to create its own 3D, you know, pool table, for instance, it's going to look like crap. So, I went on Turbos Squid, which is a 3D asset website. I found a pool table that I liked. I purchased it for like $30, and it gives you access to all the files that you need to make it custom and unique. I fed that into Claude Code, and I told it, listen, we need to get a top-down view of the actual pull table, and we need to make it look as realistic as possible. Now, I had to do a bunch of back and forth in order to get it really dialed in. But then the next step involved also running it through what's called an ISP sensor.

[00:02:28]And an ISP is just a camera sensor essentially. And we need to simulate different types of cameras. So maybe this camera is overexposed a little bit.

[00:02:38]Maybe this one's slightly out of focus.

[00:02:40]You know, maybe there's barrel distortion, barrel roll at the edges.

[00:02:44]These are all things that we need to as accurately as possible represent in the training data so that when it comes across it in real life, it'll say, "Oh, I've seen that before." So, the goal is to create 10,000 renders of just different layouts where the balls are in different positions. Some are in the pockets, some are, you know, against the rails, all that good stuff. It has to be able to identify each one. Now, there are two different ways to approach training a neural network. You can train it on real data from like my actual camera, which I already did. I did about 50 different pictures of ball layouts in different areas. But the problem with that is you have to make sure that they're all labeled. Each ball needs to have its own crop around it along along with the accurate label like is this the one ball, is this the nine ball, the two ball, etc. That can take a long time even with autoleabeling because the auto labeling doesn't always get it right.

[00:03:39]Now, when you use synthetic data, you don't have to do the labeling because the computer is the one who already positioned each ball. It knows by default. So, it creates the labels for you. That's why you're able to generate way more training data because it doesn't take that human in a loop um in approving each label. So, generating 10,000 Blender renders on a single 4090 GPU like the like the workstation that I use here in the PC would take about 3 days. So, instead of waiting 3 days, you can just use a service. you can rent cloud compute. So, I used one service called modal.com. This is not a sponsored video. And it worked really well. Essentially, instead of having just one computer, I was able to have nine computers, nine different GPUs, all generating over 10,000 images of the Blender render. And once all the images are done, okay, you have an image and then you also have the meta data associated with that image, which says, you know, all the different labels of each ball and which ball is which. At that point, you then have to actually train the model. And this takes a long time, too. It's a it's a timeintensive process. So, I used modal.com as well to take care of that. So, the whole process, which would have taken days on my end, end up taking like five or six hours total, and it was $80. That's about it for me. That's definitely worth it. It could have been free if I just ran it on my own computer, but it was $80. Now, once the model's complete, that's where the real test comes into play. So I tested out on my environment and I also tested it out at my brother's environment. He's the first user of the software and it was not trained on any images in his environment and guess what it works. So so to me this provides such a massive unlock to understanding computer vision and neural networks. Not obviously at the very low level but a high level. It unlocks a lot of possibilities in terms of what you can actually build. There's a lot of people right now, vibe coders, they're focused on CRUD apps. Very boring in my opinion.

[00:05:38]I love infusing real world stuff into these apps because fewer people are doing it. And it's also so freaking fun.

[00:05:45]So, if you're looking to build something that's truly unique and you're not sure what to build, consider building a neural network in an app that interfaces with it somehow in some way, shape, or form because it's so freaking incredibly fun. Just a real quick video here. I wanted to share that because it was a problem that I was facing. I could get it working in my environment with my 50, you know, organic images, but to really make it work in a robust amount of environments, then synthetic data is the way to go. All right, everybody. I will see you soon. Goodbye.