Moonshot AI Makes Waves Despite China's Chip Constraints

[00:00:00]Anthony, talk to us about your assessment of the capabilities of these Chinese AI models.

[00:00:07]Yeah. So the technical details for Kimi, Kimi K3, the Moonshot IO, AI's latest model, is a little bit, kind of dense, but let's go through it step by step. So there is a lot of software innovation in there in what is objectively a very large model. So the first thing is the ambition and the scale of the model is a step further, and that's probably why it has caught the US models on the frontier front. But the second most important aspect is the resource optimization around the model. It is very memory efficient. In short, there are proprietary innovations in there to also help with the usage of of HBM and GPUs where China is obviously constrained. And the fact that this model was trained with those constraints in mind is the important aspect to markets here.

[00:00:56]China has found a software workaround to its hardware constraints, and that is what the market is pricing in here on the AI picks and showers trade, and that is what will continue to play out. It is also interesting that Alibaba owns such a sizable chunk of Moonshot AI. So when those two companies come head to head, it's a bit of a family feud here.

[00:01:19]Anthony, how are China's AI leaders able to make such a competitive product without the access to the cutting-edge technology that its US competitors have?

[00:01:30]It's software innovation in in a word. Right? Like, there is a lot of software that is dealing very creatively with the constraints around compute and memory. And how they're doing it is by using extreme probabilistic mathematics, basically, to try to understand how a machine thinks in a more efficient manner and correlate that with the kind of use cases that the model is most likely to be used for. So it's not trying to do everything for everyone.

[00:01:57]It's trying to do the most probabilistically useful thing with the resources that it has available.

[00:02:03]And that approach has really paid off on the benchmarking. As as you guys have covered, you know, relentlessly, the biggest use case for, AI at the moment is encoding. And in approaching coding, this model is extremely efficient in how it uses its resources.

[00:02:20]How does this affect the demand for memory and perhaps what this also means for the big chip makers, TSMC, Nvidia?

[00:02:30]So at the margin, the fact that it was trained on a compute-restricted ecosystem has worried people. Like, how has China done this without access to TSMC's and Nvidia's top-of- the-line chips? But overall, the debate has moved on to now the price of these chips are so price of these models are so constrained that it might open up new use cases for AI and as a result, increase the size of the market. So this debate is very live. Does the market grow overall as a result of the Chinese models, or does the Chinese model's resource optimization reduce the size of the GPU market? On memory, the efficiency of memory is a clear negative to the kind of on ongoing kind of bull market in memory, but they are very large models which require a lot of HBM up front. So for the moment, the HBM market looks very stable, but there is a concern that the memory market is being pushed by Chinese innovation towards the DDR 5 or the DRAM market where China is more competitive and China can bring on more capacity to bear.

[00:03:34]And that that capacity is going to become the new touchpoint for markets on the memory trade in the days to come.