The Impossible Quest for Safe Superintelligence #shorts

追加: 2026-05-05

[00:00:00]I was convinced [music] we can make safe AI, but the more I looked at it, the more I realized it's not something we can actually do. We don't know how to make them safe. And yet we still have the smartest people in the world competing to win the race to superintelligence. Unfortunately, while we know how to make our systems much more capable, we don't know how to make them safe, how to make sure they don't do something we will regret. And that's the state of the art right now. So, while progress in AI capabilities is exponential, or maybe even hyper-exponential, progress in AI safety is linear or constant. The gap is increasing. That's exactly right. So, even people making those systems have to run experiments on their product to learn what it's capable of. And so, maybe it takes a year to train it and then 6 months to get some fundamentals about what it's capable of, some safety overhead, but we still discover new capabilities in old models. So, it's no longer engineering how it was the first 50 years where someone was a knowledge engineer programming an expert system AI to do specific things. And then we study to see what it's doing. He puts safety second to winning this race to superintelligence, being the guy who created God and controlling polite corner of the universe, needing a perfect safety for superintelligence, perpetual safety as it keeps improving, modifying, interacting with people.

[00:01:20]You're never going to get there. It's impossible. There's a big difference between difficult problems in computer science and P-complete problems and impossible problem. And I think control indefinite control of superintelligence is such a problem. I'm trying to prove that it is specifically that. Once we establish something is impossible, fewer people will waste their time claiming they can do it and find looking for money. So many people go and give me a billion dollars in 2 years and I'll solve it for you. Well, I don't think you will. If we know that it's impossible to make it right, to make it safe, then this direct path of just build it as soon as you can becomes suicide mission. Hopefully fewer people will pursue that.