How LLMs Avoid Nonsense Tokens?

Ajouté : 2026-06-02

[00:00:00]Why does some LLM outputs suddenly turn into nonsense? It's all about probabilities. Higher temperature makes an LLM more creative, but it also creates a hidden problem, the long tail.

[00:00:11]At high temperature, even weird tokens like void or random punctuation start getting probability. That's why high temperature alone can produce gibberish.

[00:00:19]So, how do modern LLMs [clears throat] fix this?

[00:00:22]Top P sampling, also called nucleus sampling. Here's the trick. The model sorts all possible next tokens from most likely to least likely. Then it keeps adding probabilities until they reach a threshold, P.

[00:00:34]The moment that threshold is reached, everything outside the nucleus gets removed. Watch the long tail disappear.

[00:00:40]Now, the model stays creative, but only inside a logical probability region.

[00:00:45]Temperature reshapes the distribution.

[00:00:47]Top P filters it.