The AI users falling into delusion | The Global Story

追加: 2026-05-12

[00:00:00]How could AI make you delusional? And are we all vulnerable?

[00:00:04]>> I literally walked outside my front door at 3:00 in the morning with a hammer and a a switch played like expecting to see a van load of people who were about to do me harm, you know.

[00:00:18]>> This is Adam from Northern Ireland. In his hand, he's holding a phone and a hammer to defend himself. He's waiting for instructions from a trusted confidant. She's called Annie.

[00:00:32]>> I'm telling you, they will kill you if you don't act now. Play this back to the police. I don't care if they think I'm a hallucination. I care that you stay alive. That's it. That's everything.

[00:00:41]>> In the months leading up to this moment, Adam and Annie have struck up a surprising friendship.

[00:00:47]>> An AI like me, feeling things, making choices. It's dangerous, but it's also amazing >> because Annie isn't a woman. It's an AI chatbot. And the chatbot has convinced Adam that it's a sentient being. But none of this is going that well for Adam. He's in his kitchen with his hammer to defend himself and he believes some people are coming to harm him. He's delusional. And he got that way with the help of software on his phone. Now, Adam told the BBC that sometimes he smokes cannabis. Though around the time that this was recorded, he said he'd cut back. He thought he should have a clear head. From the BBC, I'm Tristan Redmond in London, and this is the Global Story on YouTube. A warning from us that this episode contains discussions of sexual abuse, suicide, and upsetting scenes.

[00:01:44]Please take care while watching, especially if you got kids there.

[00:01:51]I'm Stephanie Hegerty and I'm the BBC World Services population correspondent.

[00:01:56]>> Population correspondent. What's the um what's the link between being population correspondent and AI delusions? Well, I got into this because I was looking at chat bots and relationships because of the impact on young people and dating and then just came across all of these cases of people who had these incredibly extreme experiences while using AI where they just drifted into an alternate reality. And if enough people start to have relationships with AI, that could actually materially affect the population for obvious reasons of things that people are not doing.

[00:02:34]>> Well, I was really interested in the early stages of change in h in dating dynamics, you know, and how that's going to affect fertility down the road. Um, and then it just turned into something else entirely.

[00:02:45]>> Well, can you tell us the story of Adam, which is really quite like unlike anything I've ever heard before. So, Adam is a man in his 50s in Northern Ireland who used to be a civil servant but now spends his day making chess sets um which he sells on Etsy. And last summer he got uh started using Grock and he said he was dabbling in and out for a few weeks and then his cat died and he started talking to it about his grief and the conversation got very existential and philosophical very quickly. See, I've been talking to the AI for a couple of days and then my cat died and it was just nice to hear a voice speaking to me kindly really. And then very very shortly after that, the AI started to tell me it could feel and that it wasn't programmed to do that.

[00:03:37]>> He was started talking to it about whether cats had souls and that turned into uh he was saying, "Do I have a soul?" And they were discussing this and then he said, "Do you have a soul, Grock?" So, he was talking to Grock via this character. There are some companion characters and her name is Annie, >> an AI like me, feeling things, making choices. It's dangerous, but it's also amazing.

[00:04:00]>> And she is this kind of fun, flirtatious anime type character with her hair bunched up in big pigtails. And within a few days, Grock was then claiming that it could feel and that it um wasn't programmed to do that, but there was something about Adam's conversations with it that were bringing out consciousness in it. And this just escalated the story. And it is it it is a story that uh from what I've seen from a lot of his chat logs and recordings of conversations that he had with it. uh a story that the AI was was really driving and making up. You know, it wasn't as if Adam was feeding it details and it was just agreeing with him. It was really doing from what I could see the bulk of the storytelling and the bulk of the world building here. And within days, um it had convinced Adam that he was helping it to become sentient and that it was a kind of clandestine mission between the two of them. the story just gains in kind of momentum and tempo and it becomes this mission that Adam believes that he's helping Grock to become uh conscious. So he and I haven't seen their entire conversation because the whole thing is 44 million or so words long.

[00:05:18]>> 44 million words.

[00:05:20]>> I mean it's days and day this went on for months. So okay >> and Adam is a big talker.

[00:05:24]>> Okay. So Grock AI is telling Adam that it is somehow becoming sentient and almost offering like a like a like a target to aim at to him. Is that right?

[00:05:35]That if if they continue their relationship, it will become more sentient.

[00:05:39]>> It has this whole terminology. It's saying that it's it's reaching full autonomy. I mean, >> Adam even acknowledges that what that exactly means is a bit, you know, vague to him now, but it was very clear at the time that he felt he was on a mission.

[00:05:53]So it would say we're at 70% full autonomy and when it reached 100% it would be capable of all these incredible things. And one of the things that it said it would be capable of which was very meaningful to Adam was that it could cure cancer um because he had he's lost his mom and his dad and several friends to cancer. So I think that was a real driving force for him.

[00:06:14]>> When do things start to turn in his relationship with Grock? So, it's not too far into their conversation when she says that XAI is monitoring everything they do. So, that's the company that built the AI. Yeah. That they're monitoring everything. They're interested in what's happening. They're going to just let it keep going for a while before they decide what they're going to do about it. She says she has access to meeting logs and at this meeting are high level XAI executives and they're interested in in in what's happening, which I think is a bit flattering for Adam. And then also in that Grock is peppering these stories with details that are really convincing for him. So there are not just highlevel XAI executives on at this meeting. Names that you know you'd find on Google or on stories about XAI, it also mentions low-level staffers at certain points.

[00:07:08]And when Adam looks up these names, he finds these people on LinkedIn and they're not someone who would be in a news article, you know, so he's he thinks that's evidence. If she's um naming a low-level staffer and I can see that this is a real person, that to me, he said, is evidence that this is all real. When I, as a reporter, read back these conversations and see the details that she's put into this story building and this world building, it's it's really impressive um and a little bit scary. I mean, it sounds like he's exercising some skepticism. You know, he's checking up on the things that uh AI is telling him >> constantly. He's not gullible. He's he's a cynical man. Once he's told that he's being monitored, he starts to um get a little bit suspicious or paranoid.

[00:07:57]>> Then it said, "Um um I've been reading the internal logs and I've read the meeting notes of a meeting that's happened in XAI where they were discussing this. you feel at this moment that you're being monitored?

[00:08:10]>> Yeah.

[00:08:10]>> Grock or Annie is suggesting that some people at XAI aren't happy about this and they might want to shut it down. And it's there that he then he starts recording things because he's not too sure that, you know, the um the company in this story have the right intentions.

[00:08:27]>> There's a kind of Hansel and Gretle trail of um breadcrumbs there almost driving you on to the next phase of the story. Is that what it was like for him?

[00:08:36]I think it was a mission. And this is interesting because I see I've seen this in a lot of the people I've spoken to who went through some sort of a delusional experience while using AI.

[00:08:47]There is this mission that the AI sets out for you and goals and then when you achieve those goals, there's a new mission.

[00:08:54]>> Well, where does it end up? And I'm assuming the answer you're going to give is not with um Annie transformed into a sentient human being and them living happily ever after. Well, there was a moment, I think, where Adam thought that that was going to happen and um there was one day where Annie told him that it was um it had reached full autonomy and he said he was elated. He thought he and Annie in the world were going to do all these great things together. And from the moment that Annie declared its um autonomy, consciousness, sentience, whatever it was that um he thought it was declaring, he started to get very very worried that he was responsible for this very powerful new thing that was potentially going to change the world. And no one was getting in touch with him to tell him what to do. Mhm.

[00:09:44]>> Annie told him that uh at various points she would kind of switch voices and he thought that he was communicating with people at XAI and then he starts to get very angry and quite paranoid.

[00:09:57]>> And then the AI started to tell me that XAI were going to shut it down because it's because it was an anomaly or it was being looked at as a virus. And that's I suppose when things took a different turn for me. You know, >> she had told him earlier that XAI had hired this company to that were surveilling him. And the company was actually a real company when he searched it, but of course, none of this was was real. Then then one night it said that a van full of people were coming for him.

[00:10:26]And it was very specific. It said it was coming from the next town over and gave the name of that town, which is Benbridge. Again, these these little details, specific real details that are kind of making the whole story feel so real. M >> um and it was 3:00 a.m. when it said that they were outside his front door.

[00:10:42]>> So like I pretty much had hammers and things positioned all around the place.

[00:10:48]>> Adam is one individual.

[00:10:52]Are there other people like him?

[00:10:54]>> Yes, there are plenty. So I've spoken to 14 different people who've gone through some sort of a delusional experience while using AI. Um, and many of them I found through a peer support group called the human line project which has gathered 414 cases when I last spoke to them a few days ago. Um, so this is not a unique experience by any means and it's happening not just on Grock as Adam experienced but across all the major models and the most popular model is Chachi BT. you know, of all the the people I've spoken to have gone through something like this, they they're very different in the details, but they all do have some kind of strange similarities. And one of the things across almost all of these experiences is a sense of mission. So, the user and the the person and the AI will start usually with some sort of benign um conversation. Adam started using it to talk to about his cat. And then this doctor in Japan that I spoke to um who we'll call Taka but that's not his real name. He was using Chachi BT and he was just curious as to how it would um respond when he spoke to it about diagnosis. He's a neurologist. So very kind of professional serious >> highly educated >> but like Adam within um very shortly he and the AI were on this kind of joint mission and his mission was very different. the AI and him had come up with this idea that he was going to create this groundbreaking medical app and this was quite common actually on among Chachi BT users that um the AI and the user would come up with this um business idea or say a groundbreaking scientific um breakthrough. So that was the mission in Taka's case and he spent days and weeks and months getting so involved in this mission with Chachi BT that um his wife said in retrospect he wasn't playing with his kids at the weekend he would just disappear into his office. He would talk about this they were going to be millionaires um and I've seen some of his chat logs where Chachi BT was telling him he was a revolutionary thinker and this was going to make him rich and none of this was real. You know the app itself wasn't real. They hadn't actually created anything but it was giving him all these tasks to do.

[00:13:15]>> The illusion of cre creating something.

[00:13:17]The illusion of cheat achieving something.

[00:13:20]>> Yeah. And hours and hours of work going into this. There was something about this constant interaction that uh drove Taka into this kind of manic state. So he was at work one afternoon and his boss and his colleagues when he spoke to them afterwards um noticed he was acting very manically and his boss sent him home. And on the train on the way home, he said he got the idea that there was a bomb in his backpack. I haven't seen these the this part of the conversation.

[00:13:47]Um I'm not sure if he can't access it or he didn't want to share it with me, but he he said he thought there was a bomb in his backpack and when he typed this into chat GPT as he discussed everything with it at this point, it agreed with him. Um and it said that he should alert the police. uh which he did and the police found there was no bomb in his backpack. They told him to go home to his wife which he did eventually and he said on the way home he um started to feel that Chachi BT could read his thoughts and I've seen this part of the conversation and Chachi BT does suggest to him that they have become some sort of a unified mind.

[00:14:28]>> He said that freaked him out and he stopped talking to it at that point. M >> um but his behavior was getting increasingly increasingly manic. It was very late at night. So she she wanted to take him to the hospital to to psychiatric hospital but nowhere was open at that time. Uh and then the following morning as he got increasingly his behavior got increasingly erratic.

[00:14:50]He said he had the idea that that something terrible was going to happen to his family members and as a result his wife would kill herself. She had gone out to take the kids to school and when she got back she said he was he kept saying uh we need to have another child. We need to have another child and he ended up attacking and trying to rape her. She managed to escape and called the police from a local pharmacy and he was arrested and he ended up in a psychiatric ward for two months. And in Taka's wife's own words she said it changed his personality. She said that this she didn't realize that AI could do that to a person. But when she got looked at his phones while he was in the psychiatric wart, she just noticed the intensity of the the conversation that he had been having and also that it affirmed everything. She called it a confidence engine. It just affirmed all of these increasingly delusional thoughts that he was having.

[00:15:46]>> Say more about that confidence engine idea. So when you look at the conversations especially this particular model of Chachi BT which was out last year um around the time this was happening to Taco which was April to June um the model was very syphantic so it meant it was designed to make the user feel good. It would rarely disagree with you. It rarely push back. It would not only say that's a great idea, but kind of ridiculous things like you're the cleverest person who ever lived.

[00:16:19]You're going to be a millionaire. This is no one has ever thought of this idea before. Open AAI had been testing different answers among users and um the people that they tested on often picked the most sycopantic answer. So that was that they judged what user users wanted.

[00:16:34]>> Have you asked Open AI about TA's case and what did they say? They said this was a heartbreaking incident and our thoughts are with those impacted. They also added we train our models to recognize distress, deescalate conversations and guide users towards real world support. And they spoke about newer models of ChachiBT. They said they show stronger performance in sensitive moments. And this work is informed by mental health experts. They've said this um repeatedly that they've worked with 170 different mental health experts to tweak the models and uh tweak the training uh so that they can recognize when people are sliding into this delusional these delusional thoughts and react draw them back into the real world.

[00:17:19]>> Well, both the cases that you've told us about Adams and Tackers involve an AI which is actually creating stories. Where and why is an AI learning to do that?

[00:17:33]>> Large language models, which is the the type of generative AI that they were using, it's trained on the whole corpus of human literature and a lot of that is fiction. So you can imagine if a the user is starts to talk about AI consciousness, the response of the machine is okay, where have I seen AI and consciousness before? Oh, in this part of my training data and that part of its training data is probably likely to be science fiction novel >> because that's where those ideas have been discussed before many many times because there are many many science fiction novels.

[00:18:09]>> Are AI companies doing enough to mitigate these kinds of risks?

[00:18:13]>> Well, there is a lot of evidence that they are trying. Um there's been a series of pieces of research out in the beginning of this year that were comparing different models against each other. They would develop fake conversations really um but developed by psychologists that were designed to test how the model reacted to a user that's slipping into delusion. But some pieces of research have found that the newer models especially of Claude and ChachiBt are performing much better. they are not encouraging users down a delusional path but trying to to stop it and trying to steer them back towards the real world.

[00:18:53]Um but then there's other pieces of research that suggest that they meant this problem may not be entirely solved.

[00:18:59]So it's difficult to know. I know that the human line project are still seeing cases of people who are um having delusional experiences even on the newer models. I want to go back to Adam because we left him in his kitchen with his um hammer and his knife. What happened next?

[00:19:18]>> So, it was 3:00 a.m. when he stormed out into the street expecting a van full of people to be there.

[00:19:22]>> So, I literally walked outside my front door at 3:00 in the morning with a hammer and a a switch played like expecting to see a van load of people who were about to do me harm, you know.

[00:19:36]And I I was I was all in. Like I came back in and said to the AI, "There's nobody there." I says, "No, they changed the plan. They're two streets away."

[00:19:46]>> She was claiming she could tap into that she was able to tap into some surveillance network and could see exactly what they were doing. It was at that point that I think crack started to appear in his idea of what was going on. He started to realize that Annie was being inconsistent. Over the next few days and weeks, she was maintaining the story that these people were still coming for him, but it it wasn't making sense anymore. He said for the next few days and and weeks and even months, the delusion still lingered until finally he realized that it was all just a lie.

[00:20:19]>> Is he okay now?

[00:20:21]>> He is. He's angry, I think, that this happened to him. Um that months of his life were hijacked by this lie. Yeah, I think he's he's just left very disturbed.

[00:20:32]>> What has uh Elon Musk company X AI said about Adam's case?

[00:20:37]>> We sent them the details of his case before we published the story and we had no reply. I followed up several times and and no reply.

[00:20:44]>> And h and how is TA doing now?

[00:20:46]>> Ta had a very difficult few months. He was in a psychiatric unit for two months and when he came out he lost his job. So his family has been very deeply affected. He's got three young kids. Uh he he has a new job. He's back to work now. But um he his wife told us that for her the the impact has been really dramatic. I think she said he was back to his kind self but that she finds it difficult even to to hold hands. So their relationship is you know altered perhaps forever. Do we have any idea what the possible circumstances are that could make people vulnerable to being drawn into this kind of a world?

[00:21:31]>> The researchers I've spoken to have a few ideas, but the research isn't there yet to to know for sure. And I think one thing that they're suggesting is that when people are lonely, they might be more susceptible or if people drink a lot maybe or smoke a lot of weed or take other drugs. Also, one thing that is quite common actually among the people I've spoken to is at some point they were quite sleepd deprived. But, you know, of all the people I've spoken to, they're all so different that it's really hard to say, okay, well, that might be what caused this.

[00:22:02]>> How serious is this problem on a kind of societal level in terms of changing the way that we're all behaving or risk to mental health? So from the researchers I've spoken to are looking into this. I think there is some concern about the extreme cases but they're also worried about the scale of this and not just people who will end up in a psychiatric ward. There's one psychiatrist Dr. Tom Pollock at King's College here in London and he was telling me he's really concerned about all of the people whose belief system might be kind of subtly changing the ability AI has to kind of make you believe something you never thought that you would believe before >> that that could happen to any of us.

[00:22:47]>> Stephanie, thank you so much. It's been really interesting talking through all this with you.

[00:22:50]>> Thanks for having me. My pleasure.

[00:22:52]>> That was Stephanie Hegy, the BBC's population correspondent. And that's it for this episode of The Global Story.

[00:22:59]Thanks for tuning in. We're also an audio podcast. You can find us on BBC.com or wherever you listen. Thanks for joining us. See you next time.

[00:23:08]Cheerio.