A 744B AI Model Runs On A Laptop (GLM-5.2 Via Colibri)

[00:00:00]A 744 billion parameter AI model is currently running on a normal 12-core laptop with zero GPUs and 25 to 32 GB of memory. A solo developer who goes by just bug on GitHub figured out how to do it with a free tool called Calibri and it proves that headline parameter counts are essentially a lie. Because within the first minute of looking at the receipt, you realize how GP AI's GLM 5.2 actually works. Out of those 744 billion parameters, the model only activates about 40 billion parameters per token.

[00:00:35][music] And of those 40 billion, only about 11 GB actually change from one token to the next. Right now, if you want to run GLM 5.2 locally, the official math says you can't. [music] Unless you happen to have the GDP of a small island nation lying around. To run it unquantized with its full 1 million token context window, the deployment guides tell you to spin up an eight-node H200 cluster. But Calibri breaks that entire narrative. It is about 1300 lines of pure C code. There is no Python, there is no PyTorch, there is no CUDA. It is just a bare-metal engine that reads bytes from a disk.

[00:01:13][music] Which means if you have 25 to 32 GB of system memory, you don't need an H200 cluster to run the model. You just need a really clever way to stream it.

[00:01:23]We have been conditioned by the big AI labs to believe [music] that frontier weights belong behind a paywall, locked in a data center guarded by a corporate billing department. If you blindly read the AI news, you [music] would think the only way to get intelligence is to rent it. And if you are a hobbyist building complex pipelines, maybe hooking a chat interface up to open web UI or running agents that write code, you usually face a brutal decision. You either pay those violent cloud API costs, financing a Honda Civic over a month of heavy Anthropic usage, or you try to run it locally. But, running a frontier model locally usually means buying an absurd multi-GPU rig or maxing out a workstation's RAM just to get out of memory errors. Getting GLM 5.2 local via disk streaming changes that choice entirely. Instead of buying more RAM or a bigger GPU for a hobby project, you are trading speed for memory using the solid-state drive you already have. It pulls the heaviest models out of the cloud and puts them back on your physical machine. But, to understand how a 744 billion parameter behemoth fits onto a laptop that struggles to run Chrome, you have to understand the trick GPoo AI used to build it. GLM 5.2 is a mixture of experts model, and mixture of experts is basically a giant corporate office building.

[00:02:45]>> [music] >> When you ask a standard dense neural network a question, every single parameters in the model fires up to calculate the answer. Every neuron does the math. It is horribly inefficient.

[00:02:56]But, in a mixture of experts model, the architecture is divided into specialized subnetworks. When you feed a prompt into GLM 5.2, a routing mechanism looks at the token and decides which specific experts are actually qualified to handle it. It only activates a tiny slice of the network, the 40 billion parameters we talked about, and leaves the other 700 billion completely dormant.

[00:03:18]>> [music] >> The rest of the parameters are essentially just taking a nap. So, if the vast majority of the network is inactive for any given token, loading the entire 744 billion parameter file into your RAM is a massive waste of resources. The model's total size is just a storage problem. It is not a memory problem. This is exactly what Just Bug realized when they sat down to write Calibri. The developer converted the GLM 5.2 weights into a highly compressed 4-bit integer format. In this int4 format, the dense core of the model, the the mechanisms, the embeddings, the shared experts that have to run for every single token, shrinks down to about 9.9 GB. [music] Calibri loads that 9.9 GB core directly into your laptop's RAM and just leaves it there. But, what about the rest of the routed experts? Instead of loading them into memory, >> [music] >> Calibri leaves all 370 GB of them sitting on your NVMe solid state drive.

[00:04:13]When the model's router predicts it needs a specific expert for the next token, Calibri streams that exact file from the SSD into memory on demand. It is a stunning piece of raw systems engineering. You are quite literally swapping the physical limitations of GPU memory for the read speeds of your storage drive. But, swapping memory for disk speed introduces a very obvious, very brutal physical reality. Solid state drives are fast, but they are not VRAM. When you fire up Calibri and ask it a question with a completely cold cache, meaning none of the experts it needs are currently sitting in RAM, the engine has to physically read about 11 GB of fresh data off the SSD just to generate a single word. The result is a generation speed of roughly 0.05 to 0.1 tokens per second. You do not watch the text stream in smoothly like a cloud API. You type a prompt, you hit [music] enter, and then you sit there staring at the terminal like a crypto miner from 2021, waiting for a block to validate. A short paragraph long answer can take several minutes to generate. It is agonizingly slow.

[00:05:17]>> [music] >> The Hacker News thread for this project was basically split right down the middle between developers worshipping the hacker spirit of the code and prompt users asking if a tool this slow actually has a real-world use case. But, Calibri is not just passively accepting that speed limit. It is actively trying to cheat it. To fight the SSD bottleneck, Just Bug implemented native multi-token prediction, or MTP. Instead of generating just one token and then going back to the disk to fetch the next set of experts, speculative decoding allows the model to guess several tokens ahead into the future. If the needed experts for those future tokens are already warm in the cache, it can verify them all in a single forward pass. When the cache is warm, Colibri can spit out 2.2 to 2.8 verified tokens per pass. Of course, MTP is a double-edged sword. If the cache is completely cold, guessing future tokens means fetching extra speculative experts from the disk, which can actually slow the generation down even further. But hardware changes the math. Testers running Colibri on an Apple M5 Max with fast internal storage are clocking roughly 1.06 tokens per second. One token a second is not an API level speed, [music] but it crosses the threshold from a neat science experiment to an actual usable tool.

[00:06:28]>> [music] >> You can kick off a complex reasoning task, go get a coffee, and come back to an answer generated by one of the smartest open-weight models in existence. And GLM 5.2 is genuinely one of the smartest models available right now. Zhipu AI released the original GLM 5 back in February 2026, [music] trained on a massive 28.5 trillion tokens. They updated it with GLM 5.1 in April, but on June 13th, they dropped GLM 5.2 under an unrestricted MIT license, and they aggressively tuned it for coding and reasoning tasks. It currently holds a score of 51 on the Artificial Analysis Intelligence Index V4.1, which is the highest score among open-weight models.

[00:07:08]On Terminal Bench 2.1, which tests long-horizon coding capabilities, GLM 5.2 scores an 81.0, reportedly beating OpenAI's GPT 5.5 on those specific developer tasks. It features two selectable reasoning modes, high and max, and a 1 million token context window. Though it is worth noting that the long context sparse attention required to actually use that million token window locally is not fully implemented in Colibri yet. The quality of this model is why developers are so desperate to run it locally. They want that GPT 5.5 level coding capability running directly on their own machine without paying cloud pricing. It is a genuine frontier model and G Pooh handed it away for free. The only remaining question is how much degradation the model suffers from the four-bit quantization, which the community is still actively benchmarking. When Just Vug posted Colibri to Hacker News on July 9th with the title getting GLM 5.2 running on my slow computer, it went instantly viral. It hit over 900 points, generated hundreds of comments, and the GitHub repository exploded past 11,000 stars in a matter of days. And it blew up because it represents a complete shift in how we think about computing power. You do not need a data center to test frontier weights. If you are willing to trade speed for memory, you can run the absolute cutting edge of artificial intelligence on the laptop currently sitting on your desk. The developer themselves said it best in their release post. They were not chasing speed. They said, [music] "The important thing was the journey to reach this goal. I just wanted it to work at all costs, even [music] slowly." That is the difference between treating AI like a consumer product and treating it like computer science. Anyways, [music] that's all, folks.