From Air Blowers to Liquid Baths: How AI's Oven Problem Got Solved

Added: 2026-05-06

[00:00:00]AI is becoming an electricity story. The real cost is power.

[00:00:04]Every question you ask triggers data centers >> [music] >> to move huge amounts of data and run massive matrix math.

[00:00:12]So why does talking to an AI translate into a power bill?

[00:00:16]Let's break it down from the ground up.

[00:00:20]AI models are built through training phases.

[00:00:23]This is like making a kid read every book in human history in a few months.

[00:00:28]To do this, thousands of GPUs need to run at full power.

[00:00:31]GPT-3 used about 1,287 megawatt hours for training.

[00:00:38]That's like hundreds of US homes for a year.

[00:00:41]GPT-4 is much larger by scale, 1.8 trillion parameters.

[00:00:45]>> [music] >> It's training costs over 40 times more.

[00:00:48]Training is expensive once, but the reasoning process that serves users is the real cost.

[00:00:54]Research shows 80% to 90% of AI's total energy use goes to inference.

[00:00:59]One short query to GPT-4 uses about 0.42 watt hours.

[00:01:05]Now multiply that by billions of requests per day for all over the world.

[00:01:10]The meter spins [music] like crazy.

[00:01:12]But even for the same task, some AIs consume massive energy while some are efficient because different models have different appetites.

[00:01:20]Old school idea of bigger is better [music] made energy use went out of control.

[00:01:25]But efficient models like a llama 3.18b use only 0.43 watt hours to process a long prompt. That's because same AI shows different efficiencies in different centers.

[00:01:37]Deep Seek R1 ran on a cloud platform.

[00:01:40]It used 70% less energy than on its own servers.

[00:01:44]So an AI's carbon footprint isn't just about the algorithm.

[00:01:48]It's about the data centers power usage effectiveness and cooling.

[00:01:52]The future isn't just who is smarter.

[00:01:53][music] It's who is greener.

[00:01:57]With the AI boom, data centers now run hot.

[00:02:00]Really hot.

[00:02:01]The power density of racks packed with top AI chips now uses 100 [music] to 200 kilowatts or even higher.

[00:02:08]Just a single rack.

[00:02:10]That's 39 times more than a normal home.

[00:02:13]Pack thousands of those racks into one building, you get a giant oven that never cools down.

[00:02:19]Air has a physical limit.

[00:02:20]>> [music] >> It can't pull heat away fast enough anymore.

[00:02:23]Traditional cooling can't keep up with AI. So cooling technology went through three stages.

[00:02:29]>> [music] >> Stage one, air cooling pushed to the limit.

[00:02:33]Engineers tried everything.

[00:02:36]They sealed hot and cold aisles.

[00:02:38]Cold air goes straight to the front of servers.

[00:02:41]Hot air exits the back.

[00:02:43]Prevent mixing.

[00:02:45]This pushed power usage effectiveness [music] from 1.8 down to about 1.4.

[00:02:51]But when one rack passes 40 kilowatts, running full tilt for 24 hours uses about 960 kilowatt hours.

[00:02:59]That's more than one US home uses in a month.

[00:03:01]At that level, even the strongest fans can't pull the heat out.

[00:03:05]>> [music] >> Stage two, cold plate liquid cooling.

[00:03:08]Air cooling hit limit, >> [music] >> then liquid cooling appears.

[00:03:12]A cold metal plate sits on top of the chip. [music] Cool water runs through it. Heat gets carried away.

[00:03:19]Liquid conducts heat way better than air.

[00:03:21]It easily handles hundreds [music] of watts per chip.

[00:03:25]Power usage effectiveness can drop to about 1.3.

[00:03:29]But it's partly cooling.

[00:03:30]Other parts still need fans and there's a leak risk.

[00:03:36]Stage three, immersion [music] cooling. The most complete solution is dunk the whole server in non-conductive liquid. The liquid touches every component, 360°. [music] Heat transfer efficiency is over 1,000 times better than air.

[00:03:52]If take single phase immersion, liquid just circulates and [music] carries heat away. Alibaba used it. Total power dropped 36%. Power usage effectiveness hit 1.07.

[00:04:03]>> [music] >> If take two phase immersion, liquid boils into vapor when hot. That absorbs huge amounts of heat.

[00:04:08]>> [music] >> Power usage effectiveness can go low as 1.02 to 1.05, the theoretical limit.

[00:04:14]>> [music] >> This cuts hardware failure rates by 53%.

[00:04:18]And the waste heat comes out at 60 to 80° C.

[00:04:21]>> [music] >> That's enough to heat city buildings.

[00:04:23]AI's fever warms your radiators.

[00:04:26]Real efficiency gains don't just come from cooling.

[00:04:29]They come from everything working together.

[00:04:31]>> [music] >> Google proved it. Optimize every layer, hardware, model, system. The effects are amazing.

[00:04:38]It needs three tricks. First trick, custom hardware. Build your own AI accelerators, low power from the ground up. Second trick, model slimming.

[00:04:47]Use quantization to lower accuracy.

[00:04:50]Keep answer quality, cut energy use.

[00:04:53]Third trick, use the right tool.

[00:04:56]A small specialized model is 15 to 50 times more efficient than a giant general model.

[00:05:01]Less waste.

[00:05:03]Through a combo of hardware, cooling, compression, >> [music] >> and smart distribution, Gemini cut its energy use to 1/33 of what it was.

[00:05:13]The revolution in cooling and algorithms reflects the growth in computing power.

[00:05:17]The future isn't just who is smarter, but also about who is greener and cooler.

[00:05:22]For this, different countries give us different answers.

[00:05:26]China sets up the East Data West Computing project. Build compute centers west.

[00:05:32]Target 80% renewable by 2025.

[00:05:34]And at peak power times, by compute energy synergy, compute centers power down.

[00:05:40]EU law says the PUE of data centers must be no more than 1.3.

[00:05:45]>> [music] >> US depends on market driven.

[00:05:48]But some grids are already stressed.

[00:05:50]Microsoft even signed to restart Three Mile Island nuclear plant.

[00:05:54]We talked about how hungry AI is. Next time, let's talk about how dumb it can be.

[00:06:00]It will confidently make up papers that don't exist.

[00:06:03]These aren't random bugs. They're built into how AI works. See you next time.