The Origins Of The Term "AI Alignment"

Added: 2026-05-06

[00:00:00]If if you've heard the term AI alignment, that term came out of a brainstorming session between folks at MIRI and um Stuart Russell, the Berkeley professor who wrote the AI book. I think the idea was his word.

[00:00:13]>> And when you say wrote the AI book, he sort of wrote the textbook for anyone at university sort of studying artificial intelligence. That's the sort of go-to >> the standard. Yeah. It's called AI, a modern approach and sort of the standard textbooks taught to students. He wrote that book. I think he's the one who came up with it with the idea for the the word AI alignment, but it was in a brainstorming session because we used to use the term friendly artificial intelligence, like friendly AI, and we were like, man, that sounds a little bit sillier. We should come up with a new word. Um, an alignment AI that is aligned to our goals.

[00:00:39]>> Yeah. So, the the idea of the alignment problem is actually, as we sort of originally conceived it, the the part that I was sort of trying to get at in this brainstorming session with this word is there's one question of, you know, what stuff should you have an AI try to do? like if you had this like very very powerful AI, what should you try to have it do? And that's sort of a different question from how could you make an AI that actually does that stuff instead of something else? And so there's one question of like where would you like the arrow to land? And there's another question of like how do you aim the arrow at all? And in in fact it's actually very hard to aim these sorts of arrows which is a whole a whole different discussion but a lot of people think alignment is all about you know what should the AI do. It's really a lot more about the lion share of the problem from my perspective is how would you make a very very smart AI that where the things it's actually doing are tightly related to what the developers wish it were doing. And so I wrote a lot of the the very early papers uh that were sort of talking about this term. You know the the term now means something like um we say an AI is aligned if it won't give you a math recipe. Um which is which is sort of a different concept. But yeah I was I was involved in the the very beginning of that field. Uh, you know, I spoke to a lot of these guys before they made their AI companies, uh, when they were like, well, if you know, what if this other bad guy gets control of the AIS and I'm, you know, I was the guy on the sidelines saying, it's not about who holds the leash. It's about if you make really, really smart AIs, they don't stay on your leash. Uh, that project largely failed. We didn't make enough progress.

[00:02:03]The field went much too quickly. Modern AIs, we have very little understanding of them. They're grown more like an organism. Uh, and so in in something of an act of desperation, I sort of switched to trying to inform the world that were in very bad shape