Did Kimi K3 really beat Fable?

[00:00:00]Moonshot AI just dropped Kimmy K3 and I'm out here baking in the 100 degree heat to tell you about it. This might be the next deepseek moment. This Chinese AI lab just dropped the best open-source open weights model on the planet. And it's actually competitive with Fable 5 and GPT 5.6, at least on some measurements. But there's more to this story than just a really good model. And if you like when I review open-source models, hit the like button, subscribe to the channel. It very much does help.

[00:00:33]Thank you in advance. And also, I just kicked off the Rubik's Cube simulator with Kimmy K3. It's being built right now. Stick around to the end of the video. I'll reveal how it actually did.

[00:00:44]So, this is the benchmark that everybody is pointing at and saying, "Wow, China and specifically open- source has finally reached the frontier." And in fact, it actually beat the frontier.

[00:00:56]This is Arena AI's front-end development benchmark. And what do we see? Kimmy K3 at the top above Fable 5, above GPT 5.6.

[00:01:08]That means in front-end development, Kimmy K3 is the best model on the planet right now. And that is wild. And by the way, it's not by a small margin. Here it is. 76% versus number two, Fable 5 at 63%.

[00:01:26]All right, so let me get into the details for a minute before I show you the bigger picture of this story. First, this is Kimmy K3. This is a 2.8 trillion parameter model that is the biggest open-source model to date. And no, you probably cannot run this on your home computer. This is one of those models that's going to need to be served by a data center. It also has a million token context window, which is excellent. It is designed for long horizon coding, knowledge work, and reasoning. Now, I know I've talked a lot about US versus China in the AI race. And I made an entire video about how the economics around US open- source are not favorable to building the best open source models on the planet. And what do we see? Look at this. Thinking Machines, which just released what is their bestin-class open- source model, 975 billion parameters, and really pales in comparison to Kimmy K3, coming in at 2.8 trillion. And yes, size is not everything. But Kimmy K3 is also just significantly better on the intelligence scale. All right, let's watch this one minute demo video from Kimmy K3. And as you watch it, I want you to keep something in mind. Kimmy K3 was the model that actually edited the video.

[00:02:44]Check this out. So, create a new character. Let's see. There we go. By the way, I keep hearing it is incredible at design. It is incredible at 3D asset creation. And that's what we're seeing here with that entire simulated world.

[00:02:56]Add real-time reflections, dynamic daylight cycled. It's basically building a video game. And it looks really good.

[00:03:02]This is like Red Dead Redemption style.

[00:03:04]So, you get the picture. Well, what is open source really known for? It is known for efficiency and cost. If you're using an open source model, you should expect to pay far less than going with a Fable 5 or a GPT 5.6. And it's kind of the case. Let me show you. So, for the million tokens, Kim K3 input price with a cash mix, $3 per million input, $15 per million output. To put that in perspective, that is about half the price of GPT 5.6 six soul, which sounds inexpensive, but that's not all there is to the picture. You also have to factor in the intelligence density. What is the intelligence per token? Because if you're using twice as many tokens for the same amount of intelligence, then the price is effectively the same. I'm going to show you that in a second. All right. So, currently my favorite benchmark out there, Deep Suite, which really has just nailed what kind of the public perception of all of these models are. Here's what it has to say about Kimmy K3. Now, on the X-axis, over here on the bottom, we have average cost per task. The more to the right, the better.

[00:04:13]That's cheaper. Over here on the Yaxis, we have the completion success rate. So, what you're looking for is up here. The farther up it is, and the farther to the right it is, the better. Now, what we're seeing here is Kimmy K3 Max sits right there, right under GPT 5.6 Soul.

[00:04:32]However, it's effectively the same price. They're both about $4.70.

[00:04:38]Now, what does that actually mean? That means Kimmy K3, given that it is half the price, takes twice the amount of tokens for the same exact task. And the number one model on this right now is GPT 5.6 Soul Max right there. But then you're basically doubling the price. And here's Fable if you were wondering. And by the way, if you want to learn more about open source, we actually have a number of incredible original essays on forwardfuture.com.

[00:05:06]Go to forwardfuture.com, subscribe to our newsletter, and really stay uptodate on the latest on all things artificial intelligence. Now, here is David Saxs, the AI ZAR of the United States. This is concerning. For the first time, a Chinese model Kimmy Kade 3 has taken number one on the front-end code arena and is scoring at or near the frontier on other benchmarks. Now, I don't know if that's actually true. It's probably fairing pretty well, but still GPT 5.6 and Fable are more generalized models.

[00:05:40]They do better across the board than Kimmy K3. Now, I don't want to go too deep into the politics in this video.

[00:05:45]I've already covered China versus the United States plenty, but what David Sax is pointing out here is as we continue, we as the United States regulate our AI industry, have kind of this patchwork of AI regulation across all of the states in the United States. It really slows down our frontier labs. And don't forget, Fable was released, pulled down, GPT 5.6 was delayed for weeks. This all allows China to continue to release incredible models much more quickly because they don't have to follow our rules. And I'm not saying China is bad here. I'm not saying these Chinese AI labs are bad. In fact, quite the opposite. Look at what they're doing.

[00:06:28]They are releasing insanely good models for free, open sourcing them, and open weighting them. I can't say anything negative about that. I am grateful that they're doing this, and I'm going to touch more on that later in the video.

[00:06:40]Now, here's GMO Roush, the CEO and co-founder of Verscell. Kim K3 is the best performing model on Nex.js.org.

[00:06:48]Evals, ahead of Fable, reaching a comparable success rate in less time.

[00:06:52]This is the first time that an open model is ahead of all proprietary ones for this comprehensive web engineering benchmark. And if you see me sweating, it's not because I'm getting nervous about Chinese AI. It is literally 100° out here. Yeah. So, here's the success rate coming in at 92% on the agent performance results. Success with agents.md, the number one model on the planet right now. Here's another one.

[00:07:15]Turns out Kimmy K3 is also really good at writing. So, big news from our internal writing benchmark early results. Kimmy K3 is our now number one for writing in our editorial voice 2840 ELO, surpassing Claude Fable 5. That jump is from place 21 to number one over its predecessor Kimk 2.6. five times cheaper than the model it just displaced at the top. So, it's it's much much cheaper at writing. Now, here's where we have to add caveats left and right. Can we actually trust these benchmarks? A lot of these benchmarks have been completely saturated. And remember, Anthropic did accuse Moonshot of distillation attacking their models, meaning they basically stole data from Anthropic to train the Kimmy K3 model.

[00:08:04]That is Anthropic's claim. Now, whether they stole enough data to be meaningful, who knows? But here's the thing, it's open source. You can actually go look at exactly how they built the model. You can replicate it. Technically, it's open source and open weights. They revealed all of their algorithmic unlocks, which thank you. Amazing. But we won't know until we actually test it thoroughly, actually in production environments. But what have I been saying for a while? It turns out you don't need the absolute frontier for the vast majority of work, especially in the enterprise. I know you're probably sick of me saying that by now, but it's awesome that Moonshot released this open source, but it's also definitely a threat to the United States and it is a threat to the Frontier Labs.

[00:08:52]Those are two different things, by the way. Now, here's the thing. You might be thinking, "Wow, China and open source in general is at the frontier along with the closed source US labs." That is not true. What happens with these open source labs, especially the Chinese labs, is the minute they're done baking the model, they release it to the world.

[00:09:10]However, Fable 5.1, Fable 5.2 has probably already been baked. Anthropic is just testing it. So I still think that Anthropic is probably 8 to 10 months ahead of open source because remember we found out Anthropic had Mythos way back in January, 5 months before they actually released it publicly. That's probably what both of the major labs, OpenAI and Anthropic are doing right now. GPT6 is probably being tested thoroughly internally at OpenAI right now. They've probably had it for months. They're just doing all of their safety evaluations. They're making sure that they're doing kind of the post training properly, extracting the most they can out of the model. So, I still think US closed source labs are far in the lead. We're still way out there. And again, I want to say it's awesome that these Chinese labs are open sourcing their models because guess what? It helps everybody, including US Open source and open AI and Enthropic. It helps everybody. and they have all of these incredible algorithmic discoveries with Kimmy K3 and they just give it away for free. It's incredible and it puts a lot of pressure and increased competition on the US closed source labs. And here's the thing, ultimately when there's really good open-source models, every part of the AI stack wins, maybe with the exception of the closed source labs. That's it. Every other part of the stack wins because models get better. they get cheaper which means Javon's paradox more tokens get used better applications get built on top of it the inference providers make more money and of course Nvidia sells more chips so it's kind of good across the board that China is open-sourcing these models the risk which I've detailed thoroughly in other videos is that if US enterprise gets built on top of Chinese open source models and then these Chinese open source models are optimized for Chinese chips that will make us dependent on Chinese chips. That is a big problem. Now, by the way, I kicked off the Rubik's Cube experiment 30 minutes ago and it is still going. And this is one of the criticisms I've heard about Kimmy K3. It is quite slow. So, it is token hungry and slow and hopefully it finishes soon.

[00:11:30]And in fact, right after this, I'm just going to show you a quick demo of it.

[00:11:34]But for now, it hasn't finished after about 30 minutes. And by the way, here we go. The Rubik's Cube finished. Let's check it out. It looks really good. I can see the reflections on the side of the cube right there. Let's try scrambling it.

[00:11:47]Scramble looks correct. Everything looks really good there.

[00:11:52]And let's see what happens when we solve it.

[00:11:56]It looks like it's going to solve it.

[00:11:59]There it is. Perfect. So, works really well. This is another model that can absolutely crush the Rubik's Cube. And by the way, I detailed my thoughts on the AI race between China and the US.