Local AI Just Hit the Tipping Point (Colibri)

[00:00:00]Watch this. A real frontier AI model, bigger than GPT 5.5, is answering live on a mini PC with no graphics card in it at all. The model has 744 billion parameters. The machine has 25 gigs of RAM. On paper, this cannot happen. It happens anyway.

[00:00:18]And that demo is the visible edge of something bigger. Local AI just crossed a tipping point. For years, running AI on your own hardware meant a smaller, weaker model. You settled. That trade is over now. Three walls fell at nearly the same moment. And walls like these do not build themselves back up. The proof is a tiny open project named Colibri, hummingbird in Spanish. The whole engine is one file of C. Six weeks ago, it had 4,000 GitHub stars. Today, it has over 14,000, built by 39 people. I will be straight with you, the way most of these videos are not. At the far edge, this is slow, painfully slow. But slow is not fake. This is genuinely GLM 5.2, the open model that outscores GPT 5.5 on real coding. So, here is the plan. The three walls, why Colibri lit the fuse, and why this tipping point holds. A tipping point is not one dramatic breakthrough. It is a stack of slow lines that finally crossed on the same afternoon. Here, three crossed together.

[00:01:16]The quality of open models, the price of memory, and the tricks for folding a giant model into a small box. Take them one crossing at a time. Start with quality, because that is the wall that stood the longest. Two years back, local meant hobby. You ran a small open model.

[00:01:32]It felt like a toy next to the paid services, and you told yourself that privacy was worth the drop in smarts.

[00:01:37]Then the open labs caught up, not creeping along behind, drawing dead level, and on a growing list of jobs, pulling out in front. The moment that made it undeniable has a name. That name is GLM 5.2, from a Chinese lab called Z.ai. It runs 744 billion parameters.

[00:01:55]The full weights are free under an MIT license and it scores 62.1 on the SWE bench pro coding test ahead of GPT 5.5 at 58.6.

[00:02:06]An open model beating a closed flagship on the benchmark that pays your salary and that benchmark is not a trivia quiz.

[00:02:13]SWE bench pro drops a model into real GitHub repositories and asks it to fix actual bugs start to finish about the closest thing we have to the job you would hire an engineer for.

[00:02:23]Winning there is what made people stop scrolling and GLM is not a one-off because it is not alone.

[00:02:30]DeepSeek V4, Kimmy K2, 6 QN3, 6 MiniMax M3, a whole shelf of open models now trades punches with the closed ones and on coding the gap has basically closed.

[00:02:42]Pick a frontier task and there is an open model with an arms reach of the best. So that wall is simply gone. Local no longer means worse. Running one of these at home is no longer a privacy compromise where you accept dumber answers. It is the same class of model the big labs rent back to you by the token. And that changes your actual day, not just a leaderboard. A model you run yourself has no meter ticking on it.

[00:03:05]Nothing you type leaves the room. The monthly bill disappears. The rate limits disappear and no vendor can ever deprecate the thing you built on. You own the model itself, not a login to it, which brings us to the memory wall, the nastiest of the three. To run at all, a model has to sit in fast memory. GLM 5.2 at full precision is about 1.4 terabytes of raw numbers, which is pure data center territory. Even crushed hard, it is still near 239 gigs and 25 gigs of RAM cannot hold even 5% of that. How does anyone actually get under a wall like that? There turned out to be three different ways and all three got cheap in the same short stretch of years. One way is Apple's unified memory. On a Mac, the system RAM and the graphics memory are one shared pool. So a Mac Studio max to 256 gigs can load models that would otherwise need a small pile of GPUs. It is not cheap, near $15,000, but it is a single box on a desk instead of a server rack. Another way is old gaming silicon.

[00:04:06]A used RTX 3090 hands you 24 gigs of video memory for around $600, the same memory as a card that still sells new for three times that. People bolt two or three together and suddenly own a serious rig for the price of a laptop.

[00:04:20]But, the way this whole video circles is the third path. Stop loading the model at all. Keep only the sliver you need this exact instant in RAM and stream the rest off a fast SSD on demand. That is the move that folds a 744 billion parameter model down into 25 gigs. This works because of how GLM 5.2 is wired inside. It is a mixture of experts, hundreds of small specialist networks, and for any single word, a tiny router wakes only a handful of them. So, out of 744 billion parameters, only about 40 billion actually fire on a given token.

[00:04:52]95% of the model is just sitting there asleep. On top of that, every number gets shrunk. A tool called Unsloth crushes each weight from 16 bits down to about two, a six-to-one cut in raw size.

[00:05:04]The trade is kinder than it sounds because it holds onto roughly 82% of the model's quality. So, you give up a little sharpness and most of the bulk.

[00:05:12]So, the memory math finally works out.

[00:05:14]The dense core every token needs, the attention layers, and one shared expert, about 10 gigs, stays resident in RAM the whole time. The other 370 gigs of experts sit parked on the SSD, and only the eight the router calls for get pulled in for each word. RAM holds the working set, never the whole model. To feel why Colibri matters, look at the official way to run this same model.

[00:05:36]Z.ai points you at four data center GPUs or that maxed out Mac Studio. Even a llama, the friendly one-click tool, refuses to load these weights on your own machine and hands you off to the cloud instead. Colibri runs exactly where the official path gives up. All of that theory needs someone to actually build it and that someone is Colibri.

[00:05:55]The core engine is a single C file that runs without a heavy math library, without Python underneath, and without ever touching a GPU. It treats your video memory, your RAM, and your disk as one long ladder of storage and slides the model up and down it as each word needs. Watch it boot. It reads the model index, parks the dense core in memory, and it is ready in about 30 seconds holding just 9.9 gigs resident. Then you type a question and the answer streams back. The genuine GLM 5.2 one word at a time straight off the drive and the thing running it is not a lab. Colibri comes from one developer testing on a plain 12-core laptop with 25 gigs of RAM and an ordinary NVMe drive, the exact machine a lot of you already have shoved in a drawer. That is the whole point.

[00:06:41]The barrier here was never really the hardware. And this is where the tipping point turns into something you can see.

[00:06:46]Six weeks ago Colibri was that one person's experiment. Today it carries 14,500 stars, more than 1,000 forks, and 39 contributors, one of whom is literally Claude. A week-old repo became a small movement in about a month. Now the honest bill, because I did promise it. To write a single word, the drive has to physically read about 11 gigs of expert data. So on a slow disk, you get one word roughly every 10 seconds and a full answer takes minutes. You hand this thing a job at night, not a quick question over lunch, but the slowness is a dial, not a fixed ceiling. A recent caching trick already pushed Colibri to 1.41 tokens a second on a single mid-range card. Load the experts into real RAM on a box with a few used GPUs and people are seeing close to seven.

[00:07:32]Money buys speed here. Patience buys the same model for almost nothing and it stays honest while it speeds up. That caching trick nudges the the toward experts already sitting in memory and it still matches the model's true pick about 94% of the time. So, you buy the speed without trading away the answer.

[00:07:49]So, why is this the tipping point and not last year or next year? Because the three walls that fell do not rebuild themselves. Once you are standing on the far side, you stay there. The clearest reason is the weights themselves. GLM 5.2 is open, so it's full parameters are already downloaded, copied, and mirrored across thousands of machines around the planet. A company can switch off an API tomorrow morning. It cannot reach into your drawer and delete a file you already own. The second reason cracks the industry's oldest excuse. That entire GLM 5 family was trained on 100,000 Huawei chips inside China with zero Nvidia hardware anywhere in the pipeline for a reported $25 million.

[00:08:28]So, frontier training is no longer a private club that one vendor or one country gets to control. And physics is on your side, too. The bottleneck here is the SSD, and SSDs get quicker every single year. Today's PCIe 5 drives already clear 13 to 15 gigs each second, and a stripe pair sales past 20.

[00:08:47]The model size holds still while the hardware under it keeps accelerating.

[00:08:50]So, the slow run you just watched only gets faster from here. And a 25 gig machine is not even where this lands for most people. The sweet spot is the middle of the range. A 64 or 128 gig box, about the cost of a used gaming PC, already runs a frontier class open model at a comfortable pace today. The extreme demo just shows how far the floor has dropped. So, here is where we actually landed.

[00:09:13]The thing the whole industry swore needed a data center now runs, slowly but for real, on a second-hand computer in a bedroom. The old moat was compute and secrecy, and both just sprang a leak. That is the tipping point, and it happened while the argument stayed stuck on chatbots. Now, running at all is not the same as running well, and which machine hits that sweet spot is its own rabbit hole. We took GLM 5.2 apart trick by trick on a plain 25 gig laptop in the video on screen now. Go see how deep it goes.

[00:09:43]If this rewired how you think about local AI, subscribe because taking these systems apart piece by piece is the whole job here every week.