Autoencoders - Explained

Added: 2026-05-22

[00:00:00]Take a tiny picture, just 49 pixels.

[00:00:03]That's 49 numbers. Now, imagine squeezing all of that down into only two numbers, and then expanding those two numbers back into the same picture. It sounds impossible. And the really strange part is that nobody ever tells the network what a seven actually looks like. It figures that out on its own.

[00:00:23]The trick is to wire the network into the shape of an hourglass. Information flows in on the left. The layers get narrower and narrower in the middle, and then they widen back out on the right.

[00:00:35]That skinny waist is the bottleneck. And the whole game is to make what comes out of the right side match what went into the left. We want X hat to be approximately X. Because the middle is so narrow, the network cannot just copy the input. It has to find structure.

[00:00:54]It helps to split this hourglass into two halves. The left half is the encoder. An encoding a function F that takes the input X and produces a small code Z. The right half is the decoder. A function G that takes that code Z and tries to rebuild X from it. So, the full round trip is X hat equals G of F of X.

[00:01:17]And that code Z, the thing carrying all of the meaning, is really just a few numbers.

[00:01:23]How does the network actually learn this?

[00:01:26]We measure the reconstruction loss. The squared distance between the original X and the rebuilt X hat.

[00:01:34]Training just means slowly shrinking that gap step by step until the curve bottoms out. And notice, there are no labels anywhere here. We never tell the network what the data is. We only tell it that its guess should match the input.

[00:01:51]And here's the beautiful part. Once training is done, that code Z lives in a space we can actually look at. Similar inputs end up close together. And as you walk from one point to another, the decoder's output smoothly morphs. The bottleneck has quietly organized the data by meaning.

[00:02:12]That same idea powers a lot of things.

[00:02:15]Feed in a noisy image, and the decoder gives you a clean one back.

[00:02:20]Score new data by its reconstruction error, and you've got an anomaly detector.

[00:02:26]And if you sample a code at random and pass it through the decoder, the decoder becomes a generator. The best way to understand data really is to learn how to rebuild it. And that's basically it.

[00:02:40]If you found this helpful, hit that like button, subscribe for more, and drop a comment if you have any questions. See you in the next one. Bye-bye.