Run a 27B AI Model on Your Laptop & Phone (Bonsai 27B)

[00:00:00]Watch this. A phone airplane mode on. No cloud. Someone asks it a hard reasoning question and it thinks and answers. The model doing that has 27 billion parameters full size. It is a 54 GB file. It is running in about 4 gigs on the phone. This is Bonsai 27B. The idea is almost silly. It stores every weight in the model as just plus one, zero, or minus one. It ships in two sizes. a seven gig build that runs on a laptop and a 4 gig build small enough for a phone. And here is the part that should not work. It keeps about 95% of the original models intelligent because they did not train a fresh model. They took Quen 3 6, a top open model, and re-encoded its weights to take far less room. The weights are free, Apache licensed, and they landed on July 14th.

[00:00:50]You can download them right now. Why should you care? Your prompts never leave the device. There is no monthly bill and it works with no signal at all.

[00:00:58]And it is the first model this capable to genuinely fit a phone. Not a toy demo. The real thing in your pocket. So, let me show you how they folded 54 gigs into your pocket and then whether the little thing can still actually think.

[00:01:11]Start with what a model even is. Under the hood, it is billions of numbers called weights. The tiny knobs it tuned during training. Ask it something and answering is just a storm of arithmetic over those numbers. Normally each weight is a 16- bit number. Line up 27 billion of them at 16 bits a piece and the file lands around 54 GB. That is squarely data center territory. Nothing you casually drop on a laptop and hopeless on a phone that ships with 8 or 12 gigs of memory total. People already shrink these. The usual move is a so-called 4-bit build. And for this model that still weighs about 17.6 6 gigs. Better, sure, but too heavy for a phone, too heavy for most laptops, and it barely dents the real problem. And a phone is a nastier gate than the raw number suggests. iOS lets a single app touch only about half of physical memory. So, a 12 gig iPhone really hands your app somewhere near 6 gigs to live in.

[00:02:04]Everything, the model, its scratch space, has to fit inside that. Run that math and no full 27 billion model had ever cleared the bar. The good ones were four to nine times too big for the budget. That gap between a model worth running and a device that can hold it is the wall Bonsai set out to break. The team behind it is a startup called Prism ML, a group out of Caltech backed by Kla, Google, and Samsung. And they have been chewing on one specific problem for years. How to crush a network down without wrecking the reasoning inside it. Their idea starts with a blunt question. How precise does one weight actually need to be? A 16- bit number can hold 65,000 different values. And it turns out for most of those weights, you need almost none of that. So, they throw the precise number away. Each weight becomes one of three choices: minus one, zero, or plus one. Think of it as a no, a shrug, and a yes. That is the turnary build. And that middle zero is doing more work than it looks. Three options carries only about 1.6 bits of information. Add one shared scaling number for every group of 128 weights.

[00:03:08]and the true cost settles at 1.71 bits per weight. Compare that to 16 and you are storing the same model in roughly a ninth of the space. Concretely, that single change takes the 54 gig model down to about 6 gigs of ideal size near 7 once it is packed for today's hardware. That is the laptop build full 27B sitting in ordinary memory. It also means 27Bclass quality now fits on a single 24 gig graphics card with headroom to spare. Want it even smaller?

[00:03:37]Drop the middle option entirely. Let each weight be only minus one or plus one. Pure yes or no, a single bit. That is 1.125 bits per weight. The file collapses to 3.9 gigs. And that is the exact build that slips onto the phone.

[00:03:53]And they do this across the whole network. The attention layers, the big feed forward blocks, the embeddings, the output head. There is no secret full precision layer parked in the back to prop up the score. When they say one bit, they mean the whole brain is one bit. The thing keeping it alive is that little shared scale. The plus and minus signs say which direction each weight pushes. The 116 bit number per group of 128 restores how loud that whole group should be. Directions are cheap to store. Loudness you share that split is the whole trick. One more piece people miss. It can see. There is a small vision tower bolted on at 4bit. So, the same pocket model reads screenshots, documents, and camera frames, not just text. It loads only when an image actually shows up. So, plain chat never pays for it. Now, the fear you should have. Squeeze a model this hard and it ought to fall apart. A confident sounding answer machine that gets the hard stuff wrong while sounding exactly as short as when it is right. So, this is the real test of the whole thing. And that collapse is exactly what ordinary two-bit builds do. They save the space, then hollow out the reasoning. The genuinely sneaky part is how well they hide the damage from a casual look. On easy short quizzes, they still look great. A typical two-bit squeeze of this model scores near 89 on a popular knowledge test. You would try it for 2 minutes and think it was completely fine. Then you hand it real work. On competition level math, that same build crashes to 57 out of 100. On live coding problems, 56. The knowledge quiz stayed high and masked it, but the actual reasoning had already fallen out the bottom. Bonsai does not crack there and that is the whole point. Same brutal math exam. It holds around 88. Live coding, low8s, tool calling, following instructions, vision, all within a few points of the full 54 gig original. It keeps the exact skills the cheap squeeze throws away. It is also refreshingly honest about its own size. Plenty of builds labeled 4-bit are really five bits once you measure them. A lot of two-bit ones are nearly three. Bonsai's one-bit build genuinely averages about one bit. The name matches the file for once. There is even a clean way to score this. Intelligence per gigabyte. Take how smart a build is, divide by how much room it eats. On that measure, Bonsai lands more than 10 times higher than the full precision model and well over double the best conventional squeeze.

[00:06:13]Every gig you store buys far more thinking. Put another way that is almost hard to believe. This gives you 27 billion class answers in less memory than a full precision two billion model would take. The small careful model now outpunches the big sloppy one gig for gig. And because the whole model actually sits in fast memory, it is quick. This is not the other local trick where a giant model streams off a disc one painful word every few seconds.

[00:06:38]Every weight it needs is already right there. So, it answers at a normal conversational pace. The numbers back that up. On a recent Apple laptop, it runs around 26 to 66 words a second. On a high-end gaming card, over 150. And on the iPhone itself, a steady 11 a second, quick enough to read comfortably as it types the answer out. They even measured the battery cost. On an iPhone, the 1-bit build produces roughly 672 words of output for every single% of charge per word. Running it on a laptop came out about 10 times more energy efficient than answering from a data center GPU. a serious model you can leave running on the train without watching the battery drain. And it is not a goldfish. It carries a 262,000 token context window on the device, enough to drop an entire codebase or a stack of long reports into it at once that stays affordable because most of its attention layers use a lighter linear design that does not balloon as the conversation grows. It also bundles a speed trick called DSpark. A tiny helper model guesses the next few words in a block. And the big model checks them all in one pass, keeping the ones that match on a server card that is about a third more speed.

[00:07:48]And because the big model still verifies every token, the answer is provably identical. But the biggest payoff is agents. The valuable AI work now is not one reply. It is a model that loops over a task, calling tools, running for a while before it hands anything back. In the cloud, every one of those hundreds of steps is a metered request carrying your private files across the network.

[00:08:09]When a model this capable lives on the device, that math flips a 100st step loop cost you nothing extra. And not one of those steps leaves the machine.

[00:08:18]Ondevice assistants that work offline, that reason over your private files by construction, that is the door a 4 gig 27B actually opens. And you do not have to pick a side. The pattern people are landing on is a split. Let the local model handle the routine. private high volume steps for free and only phone out to an expensive Frontier model for the two or three genuinely hardest ones cheaper and more private at once. And running it is not exotic. The weights load through tools you already know.

[00:08:46]Llama.cpb, Apple's MLX, LM Studio, a Llama on a Mac, an iPhone, an iPad, or an Nvidia box. And there is a free preview API if you just want to poke at it before downloading anything. Now, let me be straight about the catch because there is one. Long multifile coding jobs, the run it, test it, fix it, grind are not its strong suit yet. The team says a version tuned for exactly that is the next thing on the road map. But step back and the line has clearly moved. A full modern model, real reasoning, vision, tool use now fits the laptop on your desk and the phone in your pocket running offline for free at 90 to 95% of the original. A year ago, that was a punchline. Now, the small build is being pulled well over a million times a month. If you want more breakdowns that actually open the box like this, this is the channel for it. I read the white paper so you did not have to. Now go run the thing.