New video codecs explained: AV2, H.265, H.266 | Lex Fridman Podcast

Added: 2026-05-11

[00:00:02]Let's talk about the future of um multimedia ffmpeg VC but uh some of the codecs we didn't really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265 H.266. So AV1 is this codec that is done by the aliens for open media right where there is uh Google, Netflix, Amazon, Apple, uh Vidolan where we try to make a royaltyfree very good codec right and now it's being deployed but actually the codec was finished in 2018 but a codec takes years to be used in wide scenarios right so uh AV2 is the next generation of this codec it's 30% better, right? So, if you keep the same uh quality, you got 30% bandwidth reduction compared to AV1.

[00:00:58]>> What's the connection with the David and AV2?

[00:01:01]>> We going to do a David 2, right? That I called David because D is two in French.

[00:01:08]>> Well done.

[00:01:08]>> And and you have to know that David is an actual uh what we call recursive acronym, right? Because it means D David is an AV1 decoder, right? So, >> Oh, nice. Nice. I didn't even think about that. And uh people should know that Dave is spelled with a one.

[00:01:24]>> Yes, it's um and so David 2, >> it's gonna be spelled with a two.

[00:01:28]>> It's going to be day a V2D. Sorry, I don't know how you pronounce that. Uh and again, we did a demo at the CES of VC running the first demo of AV2.

[00:01:39]>> So can you clarify to me the specification of AV2 and then the encoding and the decoding?

[00:01:47]>> Sure. So the specification is like the document who explains how the codec is supposed to work, right?

[00:01:55]>> And that's really AV2.

[00:01:56]>> That is AV2 like H.264. Yeah.

[00:01:59]>> Right.

[00:02:00]>> Then you have an encoder. The current encoder is called AVM and there will probably be other encoders probably one called SVTA AV2 and those are the encoder. The same way x264 is an encoder to H.264. The same way that x265 is an encoder for the H.265 codec. And the decoders for AV1 is David. The decoder for AV2 is David 2. The decoder for H.264 is FFH.264 inside FFmpeg.

[00:02:36]The decoder for HVC is FFHVC inside FFmpeg. Um and there is a next generation codec from the meg world after H.264 H.265. There is one that is called H.266 or also known as VVC.

[00:02:54]>> So HEVC is H.265.

[00:02:57]VVC is H.266.

[00:03:01]Why is H.266 super sexy so much better?

[00:03:05]>> So the the question often we have is why are they two names? because most of the time it is a conjunct work from the ISO world and the ITU which is the international telecommunication union.

[00:03:19]>> These are these two regulatory bodies.

[00:03:21]>> One is a private entity and one is the United Nations.

[00:03:23]>> Which one is the private?

[00:03:24]>> ISO is private.

[00:03:26]>> In theory, H.264 is MEG 410 H.264/AVC.

[00:03:36]>> Mhm. And this is the full name.

[00:03:38]>> Nice.

[00:03:39]>> So it's the concatenation of the ISO name and the ITU name.

[00:03:43]>> Yeah.

[00:03:43]>> Even though they work together, this is this is politics historical, you know.

[00:03:48]>> And for HVC, it's MEGH, H.265, HVC.

[00:03:53]>> Got it.

[00:03:54]>> And there is H.266 which is also named VVC.

[00:03:57]>> Is there a high level thing to say about the improvement?

[00:04:00]>> 30% each generation is a best summary.

[00:04:04]This is true both for the AV codeex and the H.26456.

[00:04:10]>> So the professional who are listening to us are going to kill us because they say no it's 35% 25% blah blah but globally you need to know that HVC is 30% better than than H.264 H.26 is 30% better than H.265 265 because there is so many cases, so many scenarios. For example, there are cases especially for screen recording where the gains are >> humongous because you arrive you have the right tool that is done for that.

[00:04:38]And so for a specific video, a new generation is going to give you 70% gain or 80% gain, right?

[00:04:45]>> Um, >> but there used to be a ton more codecs, but now the two main codecs for transmission are the H.264, H265, H.266, 366 and the other is AV1 AV2 >> and I guess the the major difference would be the cost of encoding. Yes.

[00:05:00]>> Yes. And the royalty of the patents and this is the reasons why you see the AV version of Codex is because um they try to be as royalty-free which means no cost for the patents as much as possible because what you need to know and we've not talked about that so far is that multimedia is what we call a patent minefield. there is two places where it's you have the most patents. It's everything related to 3G, 4G, 5G, RF and multimedia. Um because it's very mathematical and you can get great gains and so on. Um so Google and Meta and Netflix wanted something where it was royalty-free. There are people who says that they have patents outside but they are fringe patents, right? So it's mostly true that is patent free. We should extend patent patent checking was done as part of the standardization process in AV1 AV2 whereas patents are not even discussed in the meg world.

[00:06:01]Patents are off topic completely.

[00:06:03]>> Can you educate me at the patents side?

[00:06:05]So usually so meek does a format right and then there is um everyone comes around and say well I have all those patents sort of formats and they do usually a union called um was called megla meg licensing association um and you put all the all your patents in and then you ask everyone who's using this format to pay for it.

[00:06:26]>> Wait, can you elaborate? What does it mean to have a patent of a codec? Why is there many patents? Uh imagine I'm doing something where I'm going to def instead of doing blocks which are square I'm going to do rectangles right.

[00:06:39]>> Oh so every idea yes somebody patents it. Yes.

[00:06:44]>> Oh man.

[00:06:46]>> Yes. Oh man, people and how many lawyers are I mean it pays for a lot of lawyers, right? Like >> the biggest issue is not the following, right? Because at time of H264 the patents were let's call it like sane but there was there was so much money in that >> that for HVC a lot there were a ton of things that were pushed inside the specification which are not useful in 99.9% of the time but just one could add a patent on it and so it became that for HVC licensing there was MELA plus another patent pool called HV VC advance plus um I think was Nokia was outside of the patent pool. a few of them are outside and some other one >> and so it was impossible to license right and I think that several months ago HP decided that they were going to remove support from HVC in their Windows laptops because the the cost was increasing of those patents and it arrived where a point where and there was uncapped pad and so for YouTube or Netflix we could talk about hundreds of millions of dollars of licensing for patents per Yeah. And they said you know what at [laughter] 100 million per year you know I could create my own codec and this is what they did and so that's why we have the open media alliance alliance for open media where we are part of that is that created AV1 and creates AV2 uh we create also audio codecs but yes so the main difference would be that and because you need to work around the patents or go do some things that are not patented a lot of things are different right The basing things that were done in ME 2 30 years ago are caught out of patents. But so for example, there is things like a golden frame, a S frame or or or different type of >> these are all patented ideas.

[00:08:43]>> Yeah. No, it's I can't believe it's not butter. I can't believe it's not a B frame.

[00:08:47]I mean I mean it's kind of what it is in some ways. It's like a >> Oh, so it's a different variant of a B frame.

[00:08:52]>> Yeah. That's to try and sidestep things like that.

[00:08:55]>> And so you need to have double creativity, right? creativity in terms of being more efficient but creativity of being sure that you don't infringe existing patents and so for example VVC is has all the patents of HVC plus new ones right while AV2 tries to be as royalty-free as possible >> to what degree does FFmpeg and VC have to think about this kind of stuff >> we don't and one of the reasons why VC was in France is that France rejects software patents so most of those patents are illegal in France because I once made the comput the calculus that if I had to pay all the licensing fee for VC, I needed to pay more than €200 per user, right? The same in dollars. Um, but most of those patents are invalid in Europe because those are called it's basically mathematical patents or idea patents and they are not valid in Europe.