Semantic Chemistry

Added: 2026-05-12

[00:00:01]I want to tell you a bit about the concept I've been cooking up called semantic chemistry. And uh you know, what's cool is these days I can have an interesting AI concept like this and then actually think about you know, vibe coding it or mostly vibe coding it on the hyperon system in the in the near term rather than just speculating and uh ideating about it like the path from concept to realization is not that long anymore although tuning the initial implemented version and fiddling with it and getting it to actually do useful things uh still takes some time and some some human intelligence which is uh fun and reassuring I I enjoyed these last few years when the human brain uh still has a purpose for for creating AGI, right? Um Semantic chemistry came out of some design work I was doing on algorithmic chemistry for for hyper, right? So algorithmic chemistry is about making a sort of program soup where you have a bunch of little computer programs and each program sort of takes one program as an input, makes another program as an output programs can also catalyze other programs, program transformation activity, right? So and instead of a primordial soup of chemicals transforming chemicals and other chemicals and chemicals catalyzing these relationships. You have a primordial algorithm soup of algorithms creating other algorithms creating other algorithms and you know, just like in a primordial soup that we get life is through autocatalytic sets, right? So, sets of chemicals where every chemical in the set is produced by other chemicals in the in the set. In a in a similar way you have autocatalytic program sets where every program in the set is created by other programs in the set applied to get other programs in the in in in the set and that that's algorithmic chemistry. You know, I was experimenting with that in the mid-90s in the early versions of Haskell and I we just didn't have the the RAM or the fast enough compilers and interpreters to to do what we to do what we wanted back then. I mean now with the Hyperon and and Morgan and all that, I mean you can you can design quite efficient implementations for a for for all these things and that has a lot of different applications, but one and I mean self-modifying cognitive code can pop out of this.

[00:03:45]You can use this sort of approach for evolutionary programming plus plus trying to learn creative programs fulfilling given specifications. But what occurred to me recently is we can apply this to natural language data also. So there you're chemical rules are transformations between semantic graphs and other semantic graphs, right? And some of these could be could be pretty simple. So you you you you can you can take any logical implication or probably most interestingly any what we're calling intentional implication. Intentional with an S. It's like a uh implication that the properties of one thing certainly imply imply the the properties of of another thing. So you you you can you can say for for example that if uh if someone has a great deal of power over themselves, this may imply they can figure out how to exert a great deal of power over others. And this this isn't a necessary logical implication, but it's a sort of implication in terms of properties, right? And so you have a whole bunch of intentional implications like this and each one is like mapping a semantic graph into another semantic graph.

[00:05:48]and say if it's about companies or something, we we could say, you know, companies with a lot of young smart employees in areas with modern culture may be able to pivot rapidly when situations change.

[00:06:11]Again, this is a not a definite logical implication, but it's a an implication in terms of properties which may have a high strength of implication based on on a body of observed data, right? So, you you take a big soup of these intentional implications and the idea is you just let them act on each other and transform each other and see what pops out. And the the idea is if you do this enough, you take all these implications and let them transform each other and transform some some data about some situation, what you get is an autocatalytic set of natural language implications and this this is interesting both conceptually and mathematically, right? Like on the on the math side, one interesting thing is what you're not doing. Like you're you're not trying to keep track of where evidence comes from super super carefully. I mean, you can still track it, but you're not trying to avoid double counting of evidence the way that you are in normal PLN reasoning or base net reasoning or or something. You're just saying let's do, you know, wacky semantic transformations all over the place and just do it and then see what results come out. And the definition of success is you get a system of concepts or concept networks that all produce each other. Now, you can you can then sculpt this afterwards if you want to, right? So, if you're trying to evolve another catalytic set that does something in particular, say that explains, you know, why certain people get a certain kind of disease or or or that that gives a, you know, a strong motivation for a certain character in a story.

[00:08:34]You can then look at the autocatalytic network that you evolved and how it helps serve a certain goal.

[00:08:43]And you can then prune down that network. You can You can look at each thing in that network and say, "Well, does this really need to be in there? Does this really need to be in there?" And then you're saying like, "What are the semantic chemicals in your semantic chemical soup that really are critical to make the given reaction network happen, right?" And that's that's the analog of doing what I've called causal coding in a neural learning context, right? Like in a in neural net learning, what I've seen is you can take predictive coding and you can sort of upgrade it to causal coding. So, instead of just looking at whether one neuron predicts what another neuron's going to do, you look at look we can look at whether one neuron causes another neuron to do something.

[00:09:33]And this this can give you networks that generalize better uh than predictive coding do continual learning better than predictive coding and predictive coding can turn can do these things you know when properly configured better than standard back propagation though there's still some scaling problems to be to be worked out. So just as you can upgrade predictive coding into >> [panting] >> causal coding, you can upgrade a semantic chemistry network into a causal semantic chemistry network by pruning out the semantic reactants that aren't really necessary to produce the effect you're looking for from the uh the autocatalytic set, right? And that's the That's Yeah, that's kind of the math side of it. But if you're if you're thinking about it in more concrete terms, I mean I'm I'm reminded of uh an essay written by the late great science fiction writer Philip K. Dick whose two-year-old uh messianic character Sophia helped inspire the name of the Sophia robot that I I helped with way back when then Philip K. Dick had an essay called How to Build a Universe that doesn't fall apart a couple days later or something like that. And that's that's something everyone who's written fiction, but especially fantasy or science fiction, has dealt with.

[00:11:39]Or anyone who's designed a a complex video game, like an MMOG or something.

[00:11:47]World building, right? Like how how do you make up a world where all the pieces fit together? And that's a that's very much an autocatalytic set type of thing, right? Where you you want each piece of the world to be kind of implicit in and implied by other pieces of the world as they fit together, like in in Star Wars you know, the Empire is how it is because the Jedi are how they are. And the the robots in Star Wars are as they are, which is without superhuman intelligence, because that's that's what fits in with all the rest of the technology there, right? And the the lightsabers are what they are because they fill the role that's implied by all the other pieces in that fictional universe. Like it there's no rule from the outside regarding how the fictional universe has to be.

[00:13:00]But each piece of the fictional universe follows naturally from the other pieces of the fictional universe, right? So that's like an autocatalytic conceptual set of fictional constructs. And scientific theories are like that also.

[00:13:19]If you look at, for example, the philosophy of science of a Imre Lakatos or Paul Feyerabend from the middle of the last century, I mean, they were looking at the way each scientific theory kind of hangs together in its own way with its own integrity, right? So, I mean, yes, there's a probabilistic implication aspect to science where you want you have a certain agreed upon data set that everyone in the scientific community accepts as kind of a real thing.

[00:14:06]And then, you you want to have an explanation of what's in this data set that's explains the data in a probabilistic way, but is reasonably simple and concise by the standards of that community. So, you you want that, but then there's going to be a lot of ways to do that. And what happens when a scientific theory emerges is that you have a series of concepts, each of which is defined in terms of the other concepts in that network, and it all it all hangs together like in Newtonian mechanics, I mean, you have force, you have mass, you have inertia, and then you have charge of electrons, and these different concepts form an interdefined network where each one sort of fits in a slot defined by the others.

[00:15:05]Then you get something like quantum theory or general relativity.

[00:15:10]Each of these is a different network of concepts. They all kind of interdefine each other. And what what Feyerabend notices is that and Thomas Kuhn with his notion of paradigm shifts noticed when you have a leap from one scientific theory to another like classical to quantum mechanics or or you know pre genetics based biology to genetics based biology.

[00:15:45]When you have a leap like that part of the reason it's so sudden is that you're moving from one set of interdefined concepts, one autocatalytic set of concepts to another set of interdefined concepts, another autocatalytic concept set. So, it's not just mapping each term in the previous theory to corresponding term in the next theory like what what mass means in classical mechanics is defined by the relation to all the other concepts in classical mechanics.

[00:16:28]What mass means in general relativity is defined by its connection with all the other concepts in in general relativity and that's really it's really about systems like one system being replaced by another system in the formation of the new scientific theory is like the crystallization of an autocatalytic set or a or a life form out of the sort of shifting reactions and combinations of the the different ideas that are bouncing around. And you know a fictional universe the ultimate fitness function is is it entertaining to the the reader or first of all the author then the reader.

[00:17:17]Scientific theory, you want to explain data, right?

[00:17:22]So, there is a sort of forcing function that guides the formation of the network, and then sculpts and prunes down the network once it's there.

[00:17:38]But, the fitness function is not ruthlessly driving the creative process, right?

[00:17:47]The creative process is this process of autopoiesis, self- formation, self-organization, and the fitness function guides that formation, and then prunes what was formed, right? And this uh these are high-end examples, but on the other hand, not that high-end, right? Like, if you watch the imaginary games of young children, and I see this all the time with my my 5-year-old and my 8-year-old kids playing with their friends. I mean, kids making up their own imaginary games. I mean, this is it's the same kind of world-building process that you have in the formation of a scientific theory, or the creation of a fictional world in a novel, or a video game. It's just a it's more fasteningly dynamic, because the kids will change their imaginary world as they go.

[00:18:53]And one of the thing that is beautiful is when you have two kids who are they're really good friends with each other, they're vibing, they're they're on the same wavelength, right?

[00:19:05]And then then they're able to rebuild their imaginary worlds as as they roll along. And you could you could actually imagine human relationships this way generically, right? Like every every married couple is in the end constructing a sort of fictional world. They're constructing a narrative of themselves.

[00:19:33]They're constructing a narrative of their lives and their relationships to other people. Each piece of this narrative connects with each other piece, and they're they're building and evolving it together. And you know, this is how belief systems work. This is how self systems work. Each of our own selves is an autocatalytic set similarly. Like each each belief that we have about ourselves is connected to and produced by other beliefs that we have about about our own selves. And if you if you go back to I guess my childhood, the early '70s, the early humanistic psychologist R. D.

[00:20:27]Laing had a beautiful book called Knots, which is somewhere halfway between poetry and psychoanalysis, right? And he he he looked at these self-reinforcing systems that comprise our own our own selves and our and our relationships. Like, "Well, I I hate him because he hates me. He hates me because I hate him. I I love hate her because she she love hates me." And and you have much more complex knots and and tangles defining selves and others and their interrelationships. So, I think this process, you know, it's far and wide among science and and self and fiction and and creative play. But, if we go back to the algorithmic level, what this is about is sort of intentional implications between semantic graphs sort of intercreating each other in, you know, vibrating, flourishing, self- modifying, autocatalytic networks. And if you if you look at this from a nitty-gritty algorithmic view, what you have is just a different inference control scheme. Right? So, I mean, you can have an inference control scheme which is very rigorous and structured, like you have some premises, you have some conclusions that you want want to find the shortest path from the premises to the conclusions with the, you know, minimum amount of double counting of evidence and that the highest confidence and the least number of inference steps and so forth. And that that's often what you need, that's often the right thing to do. But, there's also an occasion for an inference control process that's much looser than that, right? And that's and what what it's trying to do is just form a network of conclusions that all follow from each other, like guided and and modulated by some external requirement but not ruthlessly dictated by it and that's Anyway, that's uh semantic chemistry. I will uh I'll link in the description of this video a technical write-up on this that I flogged out of GPT-5 this morning.

[00:23:29]But uh what I've tried to do here is give the conceptual uh motivation for working out these details. Of course, the the fun part will be actually actually implementing it and uh seeing what it can do in the in the hyperon system in a practical context.