IWMT and Human Consciousness Hypothesis - Adam Safron, Victoria Klimaj and Zahra Sheikhbahaee

[00:00:03]So, I will try to explain a theory of consciousness that I developed over a number of years called integrable modeling theory, which is a synthetic approach of trying to bring together multiple theories and cross-reference them and see if uh a whole can be made that's greater than the sum of its parts.

[00:00:24]Um in the position paper, I suggest that there's uh convergence between integrable modeling theory and the human consciousness hypothesis. And I'll try to touch on that uh with the time I have.

[00:00:36]Um so Yeah, so the goal is to make headway on the enduring problems of consciousness and preferably uh all of them.

[00:00:45]And how to do this? Well, all these different theories of consciousness and you know, you know, people might think, you know, we need to find like, you know, the one true theory and that's a that's a good goal. Um but my approach has been basically assuming that you know, all the theories have something valuable to teach me, basically. Um you know, different degrees, you know, some some theories I take more of a page from than others like but I've really never found a theory of consciousness I don't like.

[00:01:12]Like, like, even micro-tubules, like, I don't believe that like you're having like physical quantum computation, but I think there is actually something to like the quantum intuition.

[00:01:21]But so the idea is that, you know, all these uh brilliant and creative people who've been trying to understand consciousness, they might be understanding uh different parts of this uh nominal elephant. This is uh parable of the blind sages and the elephant. You know, one grabs the tail and says it's like a snake, the other grabs the leg and says it's like a tree, the other grabs the trunk and says it's you know, like something else. And uh they're all correct and they're all like really, you know, onto something. You know, the the the tail person is a really good tail person. The leg person is a really good leg person. When you bring it together and if they can communicate, you might get a whole a greater understanding. And this is both my approach to try to understand consciousness and also I think this is part of what consciousness does.

[00:02:04]Is that it basically gives you this kind of informational synergy from a bunch of very imperfect processes and then lets you, if you're lucky, uh bring forth a world from a point of view.

[00:02:17]So, in approaching consciousness and try to integrate across these theories, um you know, sometimes uh you know, in the earlier session, like you know, are we even using words in the same way?

[00:02:28]So, like I so I uh personally uh uh people have mixed feelings, but I like Ned Block's distinction between phenomenal consciousness and conscious access. I think this is valuable. So, phenomenal consciousness and experience, uh what it is like or what it feels like, um your subjectivity, or conscious access, your awareness of your experience, your ability to have knowledge of your experience or to manipulate or report on what's in your experience. Uh these seem to me to be different things, and different theories address these in different ways.

[00:02:57]And I think a satisfying theory of consciousness uh sh- address them both.

[00:03:02]And then some.

[00:03:04]Uh so, I I mostly focus in in a global modeling theory on uh So, you know, I I bring in um a number of theories uh to different degrees, like you know, consistent with current processing theory, attention schema theory, uh higher-order thought even, but I focus on global workspace theory and IIT as like uh the main sources of inspiration.

[00:03:24]This doesn't mean like I take wholesale all of their claims, but I take them as guides to um parts of the elephant. So, global neural workspace theory, you know, we're starting from function, uh this idea of a blackboard architecture and then bars, so you know, these different specialists uh processes uh which are good at doing their thing, and they can communicate via the shared blackboard where they can exchange information and become mutually legible to one another.

[00:03:50]And so, in the form of the global neural theory as uh developed by Dehaene, it's the idea of this workspace which tends to center on these rich club hubs of the brain where brains connect and where they're they're connected to the overall system establishes and they're very central to the overall network establish overall small world networks. They talk to everybody. They also talk very intensely to themselves and interestingly take up seems about 50% of resting brain metabolism. They might be doing something important. But it is if you can get access to this like inner clique of cool kids who you can get famous in the brain where you get access to workspace and then your information is made globally available. And I put global in scare quotes because how much What do you mean by global? Like the whole thing? Like, you know, so And so, but with respect to the workspace theory, is this a theory of phenomenal consciousness or conscious access?

[00:04:50]Usually it's actually a theory of conscious access. But with respect to the IWMT though, I treat it as both. I take like the original Baars version and I say you can take you can understand a workspace functionally as something like a qualia generator.

[00:05:04]And then, but also you can have this other broader idea of a workspace where you get into things like access and report and really uh higher order knowledge and that this is also uh is another kind of workspace.

[00:05:18]There's multiple kinds of workspaces.

[00:05:20]Uh in this figure here from this uh 2000 uh 20 paper with some of Dehaene's colleagues, you see like this information kind of coming in uh this bottleneck, right? Like funnels in. Here it is like getting into this rich club.

[00:05:34]And from there, things will talk to each other and then they broadcast it out and there's like a ignition. Let's say you get into this concentrated portion.

[00:05:40]A lot of signaling broadcast.

[00:05:43]But why should there be something that it feels like to be a workspace?

[00:05:48]Um Wow, I skipped over integrated information theory. Back. So, integrated information theory I'm so as opposed to well, workspace theory, IIT starts from phenomenology. Um it says, "Okay, what are the properties that characterize all your experiences?" Uh they have intrinsic existence or like they exist unto themselves. They have their their own perspective and their uh they're they're self-referencing.

[00:06:11]They're they're compositional. They have parts. The way the parts come together makes them informative.

[00:06:16]Uh there are particularities of the composition. They're integrated and they're they're the composition things come together.

[00:06:23]And they're exclusive. Some things are in and some things are out. And so with respect to IIT, you know, there's a lot of different um qualms have been made about it. Um you know, I think they kind of beg the question about what's physical and what's real. But um you know, a photodiode that's conscious really? Um is this a a jointly sufficient set of axioms for like if you have these things, do you know this conscious or are they necessary? So, I think that fact that they start from phenomenology and move forward as carefully as they can and try to map basically what ought to characterize consciousness onto mechanistic processes um I take as a valuable guide.

[00:07:04]So, uh so to try to bring them together, uh I use the uh free energy principle and active inference framework which we just heard about. Um and this particular this idea like the Bayesian brainer, your perception is a kind of inference to the best guess as to the causes of your sensation. So, um Anil Seth and I think it was the uh Chris Frith first called it like consciousness is a kind of grounded hallucination.

[00:07:27]Um or the further back, Helmholtz, um objects are always imagined as being present in the field of vision as it have to be there in order to produce the same impressions on the nervous mechanism. So, uh you know, if inference is a perception to the best guess, that means sometimes you can guess wrong. And so some uh reason you would find us to be compelling is visual illusions. That's like an example of your guesses like it's a good guess, but it's a wrong guess. You led down a garden path to a misleading percept that doesn't reflect reality.

[00:07:56]But the percepts never reflect reality in a Bayesian brain. All models are wrong, some are useful. It's always an approximation.

[00:08:03]Um I guess you know I'm just also if you just think of your senses like you actually have acuity, you know, for vision. Like a thumbnail held at arms length, you're moving around a few times a second. That's all your senses got, but it doesn't seem that way. It seems like I'm filling in more complete with I'm I'm inferring more than I'm getting from my sensation.

[00:08:24]So I guess I I first started thinking in these terms like like I know when I heard about like with with deep learning the way so much like like models like convolutional neural networks how they like they like similar to like the visual system, but then also like when like generative adversarial adversarial networks and like kind of came on the scene or like like the like deep dream, I was like hmm uh this idea of like a generative model, is this something like what we're looking for as a bridging principle between uh implement between basically mechanism and experience. You know, a lot of devils in the details here, but something along those lines, but instead of like filling in pixel arrays, what if you could have a model that's filling in all of your different sensory modalities and all their particular combinations, you know, your your touch, your sight, your sound, um and and to whatever modal characters there iteratively to create like a stream of experience. So, how could maybe that work?

[00:09:24]Yeah, so um integrating from information theory, global workspace theory, can they be brought together? I think they can. Uh you know, the theorists do not think so, but I think they can. If you you relax some of their assumptions, the question is in the process of relaxing assumptions you break the theory. I don't think so. I think you know there's like value to something that's called something like weak IIT.

[00:09:44]Um as I was saying before, workspace can be a multiple kinds of workspaces.

[00:09:49]Um but bring these together and integrate world model theory and may need to add some additional elements to it like things like uh some of the other papers I talk about attention schema theory like you can get the decent recurrent processing or Okay, so there's a lot of things you would essentially get there but these theories that seem to be conflicting aren't necessarily conflicting I would argue and that they're actually talking about different things at different explanatory levels.

[00:10:12]So now GWT is actually more of a theory of conscious access and IIT is more of a theory of phenomenology. And so when they have this like adversary collaboration and they're saying like oh is it the front of the brain or the back of the brain?

[00:10:22]Maybe they're both right. Maybe the back of the brain is something like a qualia generator that IIT uses and maybe for you to be aware of it and to report it and manipulate it, you need like the frontal lobes and like the loops of the striatum to get the mix and then you actually have something like conscious access.

[00:10:35]Maybe they're both right but they're just using words differently. If they kind of slow down a bit and listen to each other and they the blind sages could talk, maybe they could realize that instead of like spending much money and like getting angry. So Okay, so real quickly uh world models.

[00:10:49]Uh these are increasingly found in machine learning.

[00:10:52]Uh Hannes Schimduber has great paper 2018.

[00:10:57]It's Here's a bottleneck, you know.

[00:11:00]Here's a lot of really bad theory.

[00:11:02]Here's another bottleneck. You basically train a network to compress its information through an information bottleneck or compress its data through an information bottleneck and then reconstruct using a minimize discrepancy between the input and the output.

[00:11:14]And then when you train the network how to do this trick of distilling and expanding you can then um you can both use this to allow it to infer likely patterns of sense data given incomplete inputs and you can also once you train it up run it in offline mode. So, you can basically take this trained up network and loop it on itself and you have something like imagination.

[00:11:39]So, like Yeah, so this is like the the world model of dreaming or imagining. And this is um highly useful for um let's say uh planning what you might do in terms of like, you know, if you're like a driverless car, like, you know, should I change lanes? You like a roll up into the future what might happen. Or if you're like a fish swimming in the ocean, should I go up behind this coral reef? I don't know. Somebody eat me or not? You imagine it. And so, uh Donald called these uh Pompeian creatures whose hypotheses can guide our status. This would be like a functional role of consciousness is this counterfactual information. And Rueda has like written on this uh very compellingly on his counterfactual uh theory. And also, uh a theory of information generation.

[00:12:26]We're not going to go over this, but basically this autoencoding idea could be mapped on predictive processing. You kind of fold the article uh fold the article o- over and and then like have it each level have this like hierarchy of beliefs you're trying to basically predict the lower level and updating via the prediction errors. And basically, when you do this thing, you get a uh powerful self-supervising system that has all sorts of interesting properties and cortex might work this way.

[00:12:50]Um and so, the global modeling theory says, "Okay, so have different sensory modalities and do this like predictive processing autoencoding thing. And these different autoencoders, you stitch them together with a shared a bit more space.

[00:13:01]Um and that they're through their hub portions, which in the brain might be something like the rich club, they're passing messages back and forth. And uh this basically allows them, these different uh sages on the elephants, to bring their information together to get this informational synergy. And then, get enough of this information together, you might be able to get something like uh experience world.

[00:13:23]Okay, so in a nutshell, uh every model modeling theory says that Ah, almost done.

[00:13:30]Consciousness is what it feels like to be the function of a general model of an embodied agent's inferring a sense world.

[00:13:36]Phenomenal experience entailing world models need to have spatial, temporal, and causal coherence.

[00:13:42]And I took this from Kant and also basically I got the idea from my friend Ben who studied global workspace theory and and Kant and uh is a good friend and uh good person.

[00:13:54]Um who's no longer with us.

[00:13:56]Um And also in relation to Kant, this is the idea of like a transcendental unity of apperception or basically the self has a kind of binding function. The unity of self helps to basically give a unity of experience.

[00:14:11]Okay, so a lot of moving parts. Uh this is really an advertisement. Uh please read the papers. Um I'm going to skip over this. Uh but but yeah, basically uh item 3 really focuses on again, this idea of self-organizing harmonic modes and this relates to this chord and arpeggio and also there's this great paper from Yosha, this conscious conductor, where basically thinking of the mind in musical terms, um I really kind of go crazy with it. And so yeah, so oscillating systems, things like neurons, they can um tend to sync up and their ability to sync up and create order is very important for the functioning and also it just happens spontaneously. Um and but functionally uh this is thought to basically enable uh Pascal Fries calls it communication through coherence. You know, so for neurons, for their information to uh sum or then decay, you have to basically um bring them and align their activity in time. And if you can give them coherence in time, then instead of having just like a cacophony of chaos and things just going all cross purposes, you can get um basically uh signals that can coherently and stably propagate through a network.

[00:15:22]Um, I take this uh idea from the cell annulus with connectome harmonics and I generalize it. So, basically you Chladni plates are basically the harmonic function is where vector field description of a thing, the place where the vector goes in and out and there's zero net flux, that's the harmonic solution. And so if I the Chladni plate with the sand is like vibrating on this plate and where basically the sand is going in and out and at the same rate, accumulates and then that's the shape you have like low frequencies and high frequencies and you have nested structure of low within high.

[00:15:52]And so say like, okay, self-organizing harmonic modes is what I call this generalized generalization of cell annulus with connectome harmonics.

[00:16:02]I say that basically you can think of complex information as these secret complexes. Some of them can function as workspaces of varying degrees of globality and what they're doing computationally if you communicate coherence is establishing a joint belief for whatever is in the synchronization manifold of the harmonic.

[00:16:18]I try to map this on the different frequencies of the brain to the different functions and their potential functions for consciousness. I make a lot of hay out of alpha rhythms as potentially allowing you to have a kind of minimal embodied selfhood and a view on the world. Alpha frequency has high power from like these posterior medial areas of posterior cingulate where basically it's um talking to all the different modalities so it knows all the stages and certain alpha but it also gets intense information via the pain circuit from the vestibular apparatus, stretch receptors from the neck. Um, and basically you have your entire sensorium pegged to an egocentric reference frame and I think this might be important for basically giving you a coherent experience of the world with a lived body at center or the lived body when it's posterior medial cortices would be something more like the lateral parietal. But basically that's what I'm looking for. Uh yeah, so bring so different kinds of machine learning analogies for what cortex might I'd doing. Probably need to update this in a language of transformers or Francis and others, but let's go forward. Um, you know, uh it's faster uh things nested faster smaller things nested within larger slower things. A hierarchy of rhythms and a hierarchy of representation.

[00:17:35]And I think from this you can get something like a composition of experience.

[00:17:39]Draw these comics, I call them modal frame sequences cuz I'm embarrassed to say comics, but basically I think gold would be like just like you know, like take the aspect of experience and describe it like in the particular modal character and the different things there'll be different frequency bands reflecting this deep temporal hierarchy of this nested harmonics. And I say composing um because and I mean that seriously in terms of I think actually technically speaking the best way of thinking of this is in terms of music. Like these are harmonic functions. Like just technically it's it's it is music. Um, okay, so along and the short of it uh the goal is basically to take the core aspects of experience. I call this a Marian phenomenology and Neil Seth calls it computational phenomenology.

[00:18:22]His name is better. Basically take experience seriously, try to describe it in its texture, and do this Marian stack of function uh implements mechanism implementation and some kind of algorithmic bridge.

[00:18:34]And then using the idea would be then for the algorithmic bridging principle some kind of hybrid machine learning architecture description of the brain.

[00:18:43]Um would be and in its functioning you would be able to find the uh computational object that is consciousness.

[00:18:52]And so this full stack basically gets you to do a kind of the blind men and the sages thing of you can it's like a crossword puzzle where where you have ignorance in one you can look to others and I have this story in the functional level and this story in the implementation level. I know a little bit here here. What might this apply to?

[00:19:06]Finishing up. Uh so but uh I actually think um the I think most many people here will be sympathetic to this that very well fleshed out theory of consciousness is basically pseudo code for AGI. It basically is describing an architecture that can do the full range of things that makes biological systems so remarkably intelligent.

[00:19:29]I don't have it but I think maybe the community will get there soon enough.

[00:19:33]And part of the reason I believe in such a convergence is basically human conscious hypothesis.

[00:19:39]I find each of the principles to be highly resonant or consonant with what I came I thought of that at the DMT and so it seems like my names are kind of coming at similar ideas of like consciousness has a very important function for learning. It involves coherence maximization via second order perception.

[00:19:57]Yes. Okay, so finally basically I'll leave you with a coming back to the point about music.

[00:20:05]Yoshua's like conscious conductor piece.

[00:20:08]I think it's a beautiful idea. I actually think it's a precise idea. I actually think that actually it is exactly like conducting.

[00:20:16]That in terms of what's happening and basically minds are music. And so coming back to like chords or serpegio, I lean a little bit on the side of chords in terms of harmonic modes. Again, these are standard descriptions. Although I feel like if you zoom in on them, they're actually composed of standing waves. And it's a matter of like high or coarse grained or the granularity and like what's an observer and like the information where it's relevant to whom.

[00:20:41]So you know, is it a standing wave or a traveling wave? Is it a chord or serpegio?

[00:20:45]We should talk about this later. So thank you so much for your time and attention.

[00:20:50]>> Thank you, Adam.

[00:20:52]Okay, Q&A. We can take some questions.

[00:20:55]Who has a question to start us off with?

[00:20:59]Maybe on coherence and maybe at meta level of the theories around them.

[00:21:12]>> Yes, back here.

[00:21:18]>> Thank you. Um I think I'm I'm really liking all the music analogies that are going on today.

[00:21:24]Um and I guess my question is I wonder if you've kind of approached the problem uh from like a signal processing kind of uh perspective because you're talking a lot about uh harmonics and resonance and all this kind of stuff.

[00:21:38]Um so I wonder if uh we've thought about it from a a signal processing kind of perspective and what uh what we can learn from thinking about it in that way.

[00:21:50]>> I don't know if this is exactly getting what you're saying but um I think abstractly you can potentially think of like part of the coral cortical hierarchy as doing in the brain generally is something kind of like multi-resolution wavelet analysis.

[00:22:05]Where it's like you have you know a hierarchy that like the world has this hierarchical nested structure of uh bigger slower things within them there's uh bigger and more slowly evolving things which are composed of uh smaller more fastly moving things. And uh but then the brain itself is capable of having this via its hierarchical um resonant structure of its harmonics. And that this gives you um Northoff has um talked about temporal spatial alignment theory. But basically um this then can give you a kind of a homomorphism or maybe like a diffeomorphism between like what's within and without in the representation. Because the one other interesting thing about signal processing is uh this idea of like a shared workspace and the passing of messages uh across the latent space. Uh I I suggested this uh could thought of as as the harmonics are forming um that it could be thought of as actually uh turbo coding. Where it's this idea in telecommunications where if um basically uh you can approach the Shannon limit for uh passing information through a system using noisy channels by having a bunch of imperfect systems, but then if you and their latent spaces have them loop and they are literally saying like, "Okay, I think this, you think this, I think this." They can converge on efficiently converge on a maximal output story as to what's happening and I suggest this is basically a kind of conscious and kind of turbo code. And so, uh by having the stages talk to one another, this gives you informational synergy and greater efficiency as you're doing this data fusion thing. And this might also have some like connection to things like the Jeff proposal in terms of like, you know, you're you're not in the feature space, but you're like kicking it back and forth in the latent space as we're making our predictions.

[00:23:43]>> Mhm.