AI Tokens Explained: What They REALLY Are! #shorts

追加: 2026-05-23

[00:00:00]Or Aunt Pruitt is in the Discord chat and he's asking, I think a question we probably assume everybody knows what we mean when we say token.

[00:00:07]He's saying, "Well, what is a token?

[00:00:08]Does it equate to a dollar? What is it?"

[00:00:13]Kind of like a token is a dollar. You do it. Um Token is the basic unit of inference, of intelligence. So, your AI doesn't think in words or individual numbers or even bits and bytes. It thinks in tokens, which are often number number of words. When you give it a prompt, it takes that, tokenizes it, and sends it off to the inference engine. So, that's tokens in, and then it responds to you in what looks like English, but really what it's done is it's taken the tokens you've sent it, run it through the LLM, which has all these weights, and produced a series of tokens, not words, but like words, might be more than one word, might be a word and a half, but a series of tokens which then are translated into English on your screen.

[00:01:02]That's tokens out.