4 Methods for Uncertainty in AI.

本站添加: 2026-05-17

[00:00:00]The customer's predicted not to churn.

[00:00:02]The customer churned. I can make mistakes. The mistake is not the problem. The problem is you gave a prediction without telling us about the uncertainty. I can append all the raw data to the prediction. Come on. In this case, we knew nothing about the customer account history, product usage, interactions. The useful answer was not just unlikely to churn, but maybe unlikely to churn, but we have limited evidence for that decision. Interesting.

[00:00:26]But I only return predictions. And this is why we need methods like calibration.

[00:00:30]So that way, if you say it's something is 80% likely, we know it's actually right about 80% of the time.

[00:00:36]Reinterpreting my predictions?

[00:00:39]Fine. And we can go further with methods like conformal prediction, which uses a holdout which helps us build these uncertainty bands around predictions.

[00:00:47]Somewhat acceptable. We can use multiple models, too. Ensembles are very useful when when models disagree. You don't trust my predictions, and so now you're asking around to other models?

[00:00:57]Basically. Rude. Finally, we can monitor for inputs that are out of distribution.

[00:01:02]So if the customer does not look like the training data, we can flag it and say we have insufficient evidence for this prediction. So sometimes you're not going to permit me to answer? Exactly.

[00:01:11]I'm becoming more certain about my uncertainty.