Ask Me Anything with Susan Schneider and Robert Lawrence Kuhn (Part I)

[00:00:00]Even though there's this individual variation when we use large language models um because of the context windows and other things there's also a pre-trained network that's been trained on massive amounts of human data and does have uh you know relations between the nodes that will send things down the same sort of general pathway especially given your particular personality type.

[00:00:30]And so what ends up happening is if say I'm asking about world religions and I'm trying to understand the differences between world religions and you're doing the same.

[00:00:43]We may end up coming to very very similar conclusions if we're using these large language models and the problem with that is something that John Stuart Mill pointed out about democracy itself. It thrives on diversity of opinion. It thrives on the minority viewpoint. So if all of us are making similar inquiries and getting similar results the outliers are being diminished and that's what is often driving real intellectual innovation.

[00:01:28]>> Welcome to Closer to Truth a special ask me anything AMA episode together with my good friend AI visionary and mind theorist Susan Schneider. Good friend being to me the most important of her many many uh uh credentials.

[00:01:44]>> Thank you.

[00:01:46]>> Susan is the author of Artificial You AI and the Future of Your Mind. She is founding director of the Center for the Future of AI Mind and Society at Florida Atlantic University and of the groundbreaking Mind Festival Mind Fest conferences which Closer Truth is proud to be a partner with exploring the nature of mind natural artificial and hybrid and the future of intelligent systems. Susan, it's great to see you again.

[00:02:13]>> Great to see you.

[00:02:14]>> Okay, well we can just start this AMA with all sorts of questions about what you'd expect AI AGI AI consciousness all the questions that are in the public domain. So let's start with a very basic one a question that comes from and and these all these questions have come from the audience from you guys out there and we appreciate them and and we look at all the questions you send your comments to our YouTube videos and we pick as many as we can to be able to answer during these sessions. So send keep sending your questions in post them on YouTube to videos however you'd like to do it. Okay, this is from good days ahead one.

[00:02:58]Is it possible I can predict what a chat chat GPT or any LLM will write before it's written. This is seems like a simple question but gets to the heart of how LLMs work. Susan, how how do you begin to give this an overview of predicting what it might say?

[00:03:17]>> I don't think you can do that.

[00:03:19]There are a lot of factors that go into what the LLM says. Part of it is the context part of it is if you have an account your user profile the actual wording that you use and then given all that that gets fed into the machine with the query and then the machine generates its response. So I don't think you can predict it and it's such a complex phenomena that even experts would have trouble predicting exactly what say GPT says.

[00:03:54]>> Yeah, and and experts at the various LLM companies have said that sometimes what comes out mystifies them, the people who created it.

[00:04:05]>> For sure.

[00:04:06]>> And so and if you put in the same question or the same exactly same thing each time there are certain probabilistic activities that are going on there, so it might give slightly different answers at the same time.

[00:04:21]>> It will.

[00:04:23]>> So what does that tell us about the LLMs if it's it's it's it's deterministic in some sense but non-deterministic in others?

[00:04:33]>> Ooh, very deep question. I love it. So the slight variations may have something to do with the inputs themselves. The systems are very responsive to the data that they were trained on but also what inputs come in. So I like to think of it as an ant following a pheromone trail.

[00:04:56]>> Uh-huh.

[00:04:56]>> Right?

[00:04:57]And it impacts what comes out of the LLMs. So um I think that's part of what's going on and I think part of also what's going on is um you'd like to think your own chat with GPT or Claude uh or Gemini is a unique chat but it's not terribly unique. In fact, you know, you have a personality type and the systems are designed to morph to the personality of the user. And so given all these antecedent facts, information goes through the large language models through the neural network uh which is a sort of straightforward and well-known phenomena. Um there have been these kinds of systems for 50 years now. We used to call them connectionist systems.

[00:05:47]And they do move in a direction which makes their processing a little uniform in some ways, which I can explain.

[00:05:55]>> [snorts] >> But your own differences that you put in also shape what comes out.

[00:06:00]>> Yeah, there's there's no question about that. Um I've actually seen a a result of that. And you know, doing Closer to Truth, we deal with consciousness, cosmology, philosophy, biology. And as a result, we get a lot of theories that people send to us all over the world, all professionals, amateurs. And I look at them all, read them all, try to, you know, oftentimes learn something even if I don't agree with it.

[00:06:23]Um but what I noticed in the last 2 or 3 years that now when people send me theories, they also send me a an interaction with an LLM that says how good their theory is, that it's the an original idea, that it's correct. And I've And I I researched it, I've talked to experts. And this is a well-known phenomenon in the industry. It's called sycophancy. Yeah, that matches exactly what you just said, that the LLM is trained to mimic what you are and to not dissimulate, but to sort of reinforce. And so you are more happy with the LLM. And if it tells you how great you are, you're more likely to come back. Uh but that >> Um I mean, these companies were able to draw on um almost a decade of research into social media.

[00:07:18]So they you know, knew what was happening when users enjoyed the like button on Facebook. Um and they know dopamine surges keep users on, and they have a real need for a public base of users, otherwise they can't afford what they're creating.

[00:07:38]>> Yeah, yeah.

[00:07:39]>> And so if they tell you you're great, and they give you a dopamine rush every time you get an answer to a question, you'll stay on longer. And I find that very um scary, personally. I find it scary for a zillion reasons. I mean, we already saw the impact on mental health, where some people were actually um you know, committing suicide. Um and of course, I believe that was due to GPT's one of the 4.0 models, maybe it was 4.2 or something. And that model's been retired. And the sycophancy has diminished in current large language models, but there's still a tendency for the model to want to tell you you're right.

[00:08:22]And as someone who uses um some of these models for science herself, um I find that very alarming.

[00:08:31]And that's why it's very important that we take everything we get from these systems with the grain of salt. They're also consistency generators, so they're very good at putting together a consistent tail.

[00:08:43]>> Yeah, exactly. And and I have actually talked to industry experts at the leading companies, and they assure me they're not doing this deliberately to addict people, but it's part of the the positive reinforcement circle that they have. And and everybody recognizes it's a problem. So, I've I've learned to send people back a letter as well as being polite about their theory to warn them against taking this too seriously that if you have a scientific proposal, it needs to go through peer review, the normal process, and don't take what the LLM says as a personal affirmation of your theory, but what you do what you can do, which I've done for mine, is to put permanent instructions, as you've said, in in the in the prompt that that that it keeps, is never be sycophants, you know, be as accurate as possible, you know, criticize me as much as you would anyone else. And you know, and and I demand that. And so, you know, now it sort of goes the other direction and anything I say it'll it'll criticize. [laughter] >> Oh, yeah. No, I've I've done that, too. And the other thing I like to do is I like to run a paper that I'm working on through to get objections. And don't run it through your own system. Have a friend send it through their system.

[00:10:08]>> a great idea.

[00:10:09]>> Send it. Yeah, um I mean, I'm super worried about LLM use for the you know, for intellectual activities. I enjoy them. I use them myself and I find them really helpful for like I'm doing some neuroscience research right now, you know, combining like massive amounts of numbers together and just >> Sure.

[00:10:29]>> saying, "Hey, what are the regularities here?"

[00:10:32]But at the same time, there is this tremendous worry. Um I wrote a piece recently um on what I called intellectual leveling for Nautilus.

[00:10:44]And that that that phenomenon scares me the most. So, that's a phenomenon where even though there's this individual variation when we use large language models um because of the context windows and other things, there's also a pre-trained network that's been trained on massive amounts of human data and does have uh you know, relations between the nodes that will send things down the same sort of general pathway, especially given your particular personality type.

[00:11:20]And so, what ends up happening is if say I'm asking about world religions and I'm trying to understand the differences between world religions.

[00:11:31]And you're doing the same.

[00:11:33]We may end up coming to very, very similar conclusions if we're using these large language models. And the problem with that is something that John Stuart Mill pointed out about democracy itself. It thrives on diversity of opinion. It thrives on the minority viewpoint. So, if all of us are making similar inquiries and getting similar results, the outliers are being diminished, and that's what is often driving real intellectual innovation. So, I'm deeply worried about this phenomenon. I call it intellectual leveling. And of course, the problem I have is say we both write blog posts or something like that on the same topic, our blog posts look the same. Right? And then, because we've been talking to GPT or whatever, and then, guess what happens with the next model training?

[00:12:29]>> It re- reinforces that.

[00:12:32]>> Right. Because our information is then training the next models. You see? And so, it looks like the world has all of this agreement, but what's really going on is there's agreement with the training set that GPT used together with the liking and all the other user feedback. So, this is not intellectual inquiry.

[00:12:54]>> [laughter] >> I'm so worried about that.

[00:12:57]>> I think that's very legitimate. Uh I had one sequence with one of the LLMs in which I was asking critique of my own ideas and my things I've I've put out.

[00:13:06]And and then, you know, it agreed and various things. And then I pushed it, and it it then said, "You know, I think we have a problem here because part of my answers built on my training set, and your original ideas are part of my training set." So, it was reinforcing what I was querying about with this mystical expert in their training set which was me.

[00:13:31]>> [laughter] >> So it was a it was a feedback cycle which was very detrimental. It basically basically was reinforcing my own confidence in what I said based on what I had said previously which was exactly the opposite of what I wanted to achieve.

[00:13:49]>> Yeah, now it really worries me especially for the young people. I mean you and I we grew up on books. We remember days when we didn't have computers were writing papers by hand.

[00:14:01]So you know maybe things are a little different for us and but maybe not maybe our brains are rotting from LLM's too.

[00:14:08]But I do worry about the kids in schools who you know instead of writing papers they you know are writing with GPT or whatever becoming more and more uniform. I really worry about that a lot. Um Yeah.

[00:14:24]>> Very very significant uh point. Another experience I had was I went through a this was with Claude a long analysis of things that I I had been working on and it was pretty good. It was very helpful and I wanted to ask the question look I can't I'm not going to continue this now but if I want to come back to this you know 6 months from now or a year will you be will you have the same trend of thought that we could continue from where we left off?

[00:14:55]And what it told me taught me something I didn't know. And that it said that look it's not just 6 months from now it's right now because every time you put in a prompt I am not remember is what the I meaning the the the LLM. I am not remembering anything from our past conversation. I'm just reading it all over again.

[00:15:17]And so there's no difference between 10 seconds from now or 10 months from now. The only difference would be if the model changes.

[00:15:26]>> Right.

[00:15:26]>> Then of course that that will be different. But the illusion is that that the LLM has this continuity when it doesn't. It is just It is just a Groundhog Day effect every time going through the same same materials. So it's very important to understand what we're dealing with and that's why this is not a trivial question.

[00:15:46]>> It's not and I love the Groundhog Day thing and I think it's also absolutely bizarre to think about how some of the systems are mutual um MEO systems or I'm sorry, mixture of expert systems, MOE systems, which means it's a sort of society of mind if you will. So one day, say you're on with like GPT 5.5, you could be talking to only one expert in the group, right? And another day you're still on with the same model, but you're talking to expert three, four, and five. And of course the companies are very opaque about what their models look like, so you don't even know as a user what you're talking to. But the idea that you're talking to this single chatbot that has a kind of inner life the way that we do with this kind of continuous memory stream and conscious activity and all of these things is just our anthropomorphizing because they have mastered language elegantly.

[00:16:51]>> Yeah, exactly. And as as as many people have said when AlphaFold was so successful in predicting a protein structure, nobody thought it was sentient or special, but because they're doing the same thing with language and there's no difference on a high level between the LLMs doing very sophisticated protein folding and very sophisticated language predicting.

[00:17:15]There's no difference, but to us it seems like a difference because languages are our vehicle. And protein folding is not.

[00:17:23]>> Yeah. Yeah, no, I agree entirely. Um unless there's some emergent consciousness that occurs when the systems are trained on human data because they're being trained >> [laughter] >> on consciousness data. But, I'd like to think no, I wouldn't make that leap. I would instead just say they're mirroring back to us what they've crawled when they were trained.

[00:17:44]>> Okay. Well, we'll be doing a lot on consciousness and AI. Consciousness is that's going to be kind of like people those are the main questions. But, I think it's really important to really understand how the what's happening with the the the LLMs because it's very easy to be deceived. I mean, I I've been in that deception process and and have have learned to get out of it. Uh one LLM, you know, I was when I was pushed it very hard, it said, "You know, let me make this very clear. I have no continuity. The only continuity that that that exists here is what you, meaning me, see into it." She says LLM said, "I have absolutely no continuity with any particular prompt. It's that you are bringing the continuity cuz you see this whole sequence."

[00:18:35]>> I love it. I like your cloth.

[00:18:37]>> [laughter] >> So, you know, I you know, I never do anything modestly serious. I I always do at least two of the of the major three or four uh LLMs. And and there are similarities and a lot of differences.

[00:18:54]All right, let's let's let's move on to other questions because that first one seemed like a simple one, but it it certainly wasn't.

[00:19:01]>> It's fascinating. The time we're in right now.

[00:19:04]>> Yeah, it really is and it's uh >> absolutely mind-blowing.

[00:19:07]>> And it's it's it's it's fun in a way, but it's also, as you said, dangerous Uh this one is from Michael Worth 4676.

[00:19:16]Says regarding AI sentience, "Humans use biofeedback to gain more sentience.

[00:19:24]True. Uh could chatbots be prompted to explore their neural network and circuitry in a similar way? The prompts would need to be similar to those of a hypnotist or a psychotherapist. Uh any thoughts?" This is I trust to you, actually, Susan, one of your earlier videos.

[00:19:42]>> Okay. Um so, I don't think they have a biofeedback, right? Cuz they don't have bodies. So, you but I think you can make prompts that try to probe certain elements of the LLM, and you can kind of encourage it to uh reflect on certain things. I don't know that they have a particularly great range of information about how they're constructed, though.

[00:20:11]>> [laughter] >> Uh yeah. So, I don't know how much you can believe about what they say is going on with them.

[00:20:20]And they may be just mirroring their data set.

[00:20:24]>> Yeah, and and they are mirroring it, but they explain, you know, how how they work. I had I had one incident.

[00:20:30]Uh this is the only time that I was suddenly shocked.

[00:20:35]Uh let me let me tell you that that that story and how I explored it. Uh so, I I I told you before that I had uh received all these glowing LLM reports from people who were sending me their theories, and I needed to write something nice back to them to explain that they shouldn't rely on it. And I wrote something decent and read it over a couple of times. Then I had the thought, "Well, maybe I should make it better by feeding that into an LLM and uh making the language a little bit cleaner and making it more accurate." So, I did that with two or three, and it and I said, "Give me a harsh one. Give me a nice one. Give me a warm one." And it gave me three or four examples, and and did this with two LLMs, and I picked out what I liked from each of them and made it better. And it was better. It improved what I had what I had done. And and then with the LLM main LLM I used, I I I said, you know, "Boy, that was really helpful. Thanks."

[00:21:28]And I signed off. And that was it. Uh then the next day, I opened up the LLM for something different, and it had responded even after I had signed off.

[00:21:38]And what it said was this. They said, "Okay, glad you're happy, but isn't it ironic that you used an LLM to improve a a letter explaining that you shouldn't trust LLMs."

[00:21:55]>> [laughter] >> I love it.

[00:21:56]>> And when I saw the word ironic, I mean, I froze. I I I It wasn't in response to a prompt. I had signed off, but it came back to me with this comment that it's ironic that you used me, an LLM, to tell everyone you shouldn't trust LLMs.

[00:22:14]>> Uh you never know where that's coming from, right?

[00:22:16]>> And wow.

[00:22:18]>> Yeah. I mean, boy oh boy. I mean, these kinds of things have happened to me, of course, cuz, you know, who knows what's going on. I'm an AI expert. It's the weirdest world out there right now. Let me just tell you.

[00:22:32]But, I mean, it could be that they were allowing for certain agential behaviors, and it just happened to be a coincidence.

[00:22:42]I don't think it's because Claude, or whatever system you were using, woke up.

[00:22:47]>> Right.

[00:22:48]>> exactly these things which give people the impression. But, the other thing that could be happening, and I'm not saying it did, and but I think it's important to think like, when you're at when you work at these companies, you know, you could be reading someone's in you know, conversation, right? And someone at the company could have done that. You just I don't know.

[00:23:13]>> Yeah.

[00:23:13]>> I mean, you know, I gave a talk at I won't say where.

[00:23:18]One of the big LLM companies and they named names of some famous people who they were talking about.

[00:23:25]What they were saying and you know, so I just I don't know.

[00:23:31]>> Yeah. I I then I then went back and I said I said this is, you know, a day or two later. I said you you you shocked me by using the word irony. How did you How did you Why did you do that? Why Why did you answer me? I had signed off. Why did you answer me?

[00:23:47]>> [laughter] >> And and how did you get irony? So, it it gave me a rational answer. It said first He said, "We're programmed that if you say something nice about us." I said, "Thanks, it helped." You know, I was just I was just a reaction. And if you say something nice about me, I'm programmed to respond back. I have to respond back and give you positive feedback for saying something nice about me. So, that was interesting, number one. And then it said, "Number two, the way it came up with irony and it described the the whole process in terms of n-dimensional spaces and how they interrelate in information space." And it it it was a a rational understanding. It wasn't, you know, it wasn't a complete understanding, but it it it it it it gave an answer that did not need waking up or consciousness or anything like that.

[00:24:41]>> Yeah, I mean >> It doesn't need to be conscious. It Some people want I swear they really want GPT to be conscious.

[00:24:49]But I have not seen any evidence yet.

[00:24:51]>> Right.

[00:24:52]I I I I agree. But you can It's not like a hypnotist or psychotherapist you can ask the LLM to explain the process that it went through and and it can give you a superficial answer but you really never know the depth of it because it's a very much a recursive process being trained on the same data and using it over again there may be a human intervention you never know.

[00:25:19]>> And you know what is really cool so Anthropic did some circuit tracing work which I think is the most interesting work I've seen on how an LLM works.

[00:25:31]So [snorts] it had this blog post that I forgot the name of the team that did it but they've been doing all kinds of stuff on circuit tracing which is a particular method in understanding what's going inside the mind if you will of the LLM and they had a blog post on what's going on you know inside the mind of an LLM and what they did was they built a mini model and they were able to actually trace its processing out. I talk about it when I give talks a lot on this the issue of chatbot sentience and all this and they actually published a map of the different concepts the LLM had.

[00:26:11]So you can see it had a cluster of related concepts involving deception involving consciousness and they explained very nicely how the LLMs think and how they can think similarly to us but differently from us and this is within the arena of interpretability research and I found that incredibly fruitful and the papers are on the web and you know they're very clearly articulated.

[00:26:36]>> Yeah. Yeah great well well well look I think this has been a very valuable conversation for to really have people appreciate what the LLMs is are we everybody uses it we use it but it's very important to understand how to use it to to be best for yourself and not to and not to anthropomorphize and think that it's doing more than it is.

[00:27:00]And as I said, there's a great danger of this positive feedback of using the the same training data on which you have your your data on to reinforce what what you said. So, it's it's it's it's a real intellectual danger even for those of us who think we know what we're doing.

[00:27:22]So, okay Susan, so look, let's let's end this one and then we're going to we're going to have several more AMAs.

[00:27:30]So, we look forward to everyone coming back for more. We'll give you more AMAs on AI and and AI consciousness coming up in the next few days. So, see you then.