Compilerrrrrrr

Added: 2026-05-26

[00:03:26]So, that's not optimal.

[00:03:29]Um so, I'll put those in and make sure it still runs.

[00:03:36]Okay, so um We don't exactly have a plug-in for the visualizer separately, but we we'll deal with that later. Okay.

[00:04:30]Um if visualize memory debugger in it visualize memory debugger. All right, so Actually, where does that get called from?

[00:05:17]Okay.

[00:05:25]We need this separate simpler in it function to in it the mutexes that happen um that guard a couple of data structures we use.

[00:05:45]These need to be set up before any alloc or free is called, which is why this is factored out. Okay.

[00:05:59]So, we're going to just in it some mutexes.

[00:06:04]We're going to We're going to just say uh We're going to just do that.

[00:06:15]And then here we're going to do that.

[00:06:41]So, sometimes Dude, Twitch chat I just I don't know what to do about Twitch chat, guys.

[00:07:00]I don't know what to do about Twitch chat. Hyped for the game? Well, I'm glad you're hyped for the game. I am hyped for the game.

[00:07:07]I've been working on it for a real long time.

[00:07:10]So, it's going to be good to have it out.

[00:07:23]Ignore the randos. I know, dude.

[00:07:26]It's it's an important survival thing.

[00:07:31]Okay.

[00:07:33]So, we have these add free this frame and add alloc this frame.

[00:07:47]All right, so Okay, actually, you know what?

[00:08:39]Maybe we don't need any of this [ __ ] that I just typed.

[00:08:43]Cuz we should use the same mutex.

[00:08:46]Right?

[00:08:48]Um Can we assert that a mutex is held?

[00:09:10]I don't think so.

[00:09:31]Okay, this load thread group thing can probably get simplified out.

[00:09:39]Maybe not.

[00:09:40]I don't know.

[00:09:41]Maybe.

[00:10:00]>> Okay.

[00:10:02]I think I'm going to make some tea.

[00:10:05]But I think we just Let me just see where this is called.

[00:10:16]Add at allocation this frame that's in um here.

[00:10:29]So, when we do this Yeah, so we're inside address table mutex. I would like to be able to assert Well, let's check for the other call. So uh Yeah, this is also okay.

[00:10:46]So, actually Dude, I need to Hope I didn't have any changes.

[00:11:14]Um So, I think we just need to do this. So The problem is down here.

[00:11:48]These this frame tables.

[00:12:14]Yeah.

[00:12:16]Okay.

[00:12:24]So This is all copied out of the report.

[00:12:57]Okay.

[00:13:07]When accessing Alex this frame and freeze this frame we must be holding address table mutex since these could be modified by uh alloc re-realloc, etc., which can be called on many threads.

[00:13:46]We would prefer not to be doing this much.

[00:13:52]Um We would prefer not to be doing much work inside this mutex, so we actually take it twice.

[00:14:16]Okay, so we're going to do this.

[00:14:19]We're going to go bytes total is an int.

[00:14:23]We're going to assign down there, and then here we're going to do that.

[00:14:36]So, the thing about mutexes is there's at least two major problems that happens if you hold them for too long, right?

[00:14:43]One is just things get log-jammed cuz someone's waiting a long time for this mutex while you're doing stuff, and someone else is waiting a long time, and someone else is waiting a long time on that guy, and then nobody can run, and then the whole point of being parallel is so that people can run. So, that's not good.

[00:15:03]But the second issue is um Well just the more complicated of code that you call while holding the mutex, the more you don't really know what's happening.

[00:15:25]And like you could be grabbing other mutexes while you're holding this one.

[00:15:30]And you could have like mutex inversion and stuff. Which by the way, this would be a good thing to do sometime. We'll definitely do this after the game ships, but maybe before the game ships. Like for our default mutex implementation there's all sorts of things that you probably want.

[00:15:51]Um So, for example, you want to be able to ask if you're holding it right now. Or you want to be able to assert that you're holding it right now.

[00:15:59]Um But you also want to detect mutex inversion and stuff, and I think I kind of want to figure out a good version of that to put in by default.

[00:16:23]So, for example There's There's one way of preventing mutex inversion, which is just uh you know, every utex mutex has a static priority number, and when you go to lock the mutex you can only lock things at that priority uh and below, right?

[00:16:47]Or above, whichever way you're counting, right? And then um And then if you ever don't do that that's a problem. Now, that's That's actually uh more um more serious or that's more constraining than you would actually need to be correct, but it's like it's this point of usefulness, right?

[00:17:22]Do we handle asserts gracefully? What?

[00:17:25]What do you mean?

[00:17:29]First of all, you control what happens when an assertion fails.

[00:17:33]But secondly, what do you want us to do for an assertion failure?

[00:17:40]Is the compiler going to ship before or after game release? Uh after, but hopefully not that long after.

[00:17:45]The reason is we just uh we don't want the compiler to distract from the game release, and we want to give out a version of the game source code when we give out the compiler. So, you know.

[00:18:00]Both of those Yeah. All right, let me get my tea. Back in a second. Any other technical questions that people have?

[00:19:36]Okay.

[00:19:39]I'm back.

[00:19:47]I got some tea.

[00:20:56]>> I don't know what the cleaners did with my tea tray. Oh, there it is.

[00:21:00]I found it right as I was leaving.

[00:21:11]Are they nice to use in C asserts? Yeah.

[00:21:16]Of course they are, but you control what happens. You You can override the hook of what happens when an assertion fails to do whatever you want, just as you would expect.

[00:21:30]Like it's not that big of a deal.

[00:21:34]Um What kind of mutexes? Well, uh it's operating system dependent. We just use whatever is the operating system's preferred mutex.

[00:21:59]All right.

[00:22:01]Uh I forget.

[00:22:06]Um I guess we could run this on Sokoban, but usually we would like a simpler test.

[00:22:14]Um Yeah, I guess we can we can do this.

[00:22:24]All right. Uh Well, is there a test in here or is there?

[00:22:42]There's an examples. Oh, there's a tests.

[00:22:46]This probably tests everything.

[00:22:53]Okay, well, that's great.

[00:22:56]Examples, do we have a All right.

[00:23:07]Okay.

[00:23:09]So, this doesn't do the visualizer, but we didn't break [clears throat] compilation there.

[00:23:15]Uh which is great. Okay. So, we're going to run this bad boy.

[00:23:22]And um I forget.

[00:23:29]We have uh There's a way to turn this on.

[00:23:43]Memory view.

[00:23:48]All right.

[00:24:06]So, we'll see what happens.

[00:24:27]We're building a bunch of shaders.

[00:24:32]Okay.

[00:24:42]So, all right, some stuff's happening.

[00:24:46]We only uh only only using 384 megs. Let me pour myself some tea.

[00:24:54]So, you'll see first of all, uh let me go to a simple level where it's easy to understand. Well, and also that's using real graphics. I'm going to give >> So, a big piece of my >> you'll see that we're not doing a lot of memory spam from frame to frame.

[00:25:17]I mean, you know.

[00:25:20]Which video games should not freaking do, like when we resize the window, we did a little bit. Like we've got some stuff when major things happen, but like you see the numbers bumping up to like 32K.

[00:25:33]That's pretty small for a modern video game. And we could get rid of that if it were a priority. But the point being, like if you put a freaking, you know, I don't even know, Unity game in here, you would be seeing some stuff right now.

[00:25:51]Um So, we could do this and right, I can run around. I mean, it's working, obviously.

[00:26:01]For some definition of working.

[00:26:05]Okay. Stop recording.

[00:26:08]And then uh we can just inspect the frames, right? So, let's let's do frame 1337.

[00:26:21]All right.

[00:26:26]Um yeah, we need 2K on that frame.

[00:26:33]Transition barrier. This is graphics stuff.

[00:26:37]Uh I don't know why I don't know why we're doing that exactly, but it's something we could get rid of.

[00:26:51]Like we've probably got an array of how many of these there are, and on this frame the array grew, but it's not that big of an allocation, which make which makes me just think, just like make a big array that can expand, but like usually doesn't.

[00:27:09]Right?

[00:27:10]So, this is easy to get rid of, I think.

[00:27:13]Um Anyhow, I mean, it's working.

[00:27:21]Let's look at where our memory comes from.

[00:27:24]Allocate geometry, okay.

[00:27:27]That makes sense. So, you know, we have like 78 megabytes of meshes.

[00:27:35]Um at least. There might be other call stacks. There probably are other call stacks that do, but we've got like 78 megs of meshes.

[00:27:44]Um the freaking animation data structures, which is >> [sighs and gasps] >> It's ridiculous that they're this big.

[00:27:53]It's maybe I'll change it before ship, maybe not. Okay.

[00:27:58]Um Shader material table What is this?

[00:28:07]Update material How do we have 25 megabytes of materials?

[00:28:14]Okay, this is a hash table that holds materials, I guess. Okay, so that's questionable.

[00:28:31]This is not the entire memory footprint of the game, by the way.

[00:28:39]Cuz the vast majority of the memory is texture maps, but they don't go through allocations here, which is why you don't see them.

[00:28:47]Um They just go straight to the graphics driver.

[00:28:55]I guess.

[00:28:59]Yeah, I don't know.

[00:29:02]I don't know.

[00:29:10]What's the back of the envelope for what goes into 78 megabytes of meshes? Seems like a lot. I mean, it's only 7.8 megabyte of meshes per year we worked on the game.

[00:29:24]That seem like a lot?

[00:29:45]Doesn't seem like a lot to me.

[00:29:47]When you put it that way.

[00:30:10]This is still more animation player crap, dude. Animation player is really spammy.

[00:30:18]This is loading entities.

[00:30:27]Yeah, you can you can invest to McGate all the things.

[00:30:31]Anyway, we put this in and it appears to be working.

[00:30:38]So I'm going to run the tests.

[00:30:42]Oh.

[00:30:46]I What happened there is I just alt F4 the game.

[00:30:52]What is this assertion?

[00:30:59]Data is equal to message. Oh, cuz we failed to write though.

[00:31:08]I mean are we did Well, actually I don't know. I don't know what happened there.

[00:31:18]The problem is because someone [ __ ] up our thing and I'm still angry about it. I can't attach from that point.

[00:31:27]Let's see if that happens again.

[00:31:32]I don't think that that had anything to do with the change that we made. I suspect that that would have been happening already.

[00:31:43]But let's see let's see if we can reproduce that.

[00:31:51]Tables grew between the two mutex locks.

[00:31:54]Um That's a good point. Okay, let's think about that.

[00:32:00]I don't I don't think that that's it's it's just happening when I exit the game interestingly, but it's a good point.

[00:32:10]Um Maybe we're not >> [sighs and gasps] >> Okay.

[00:32:18]This is safe in the following sense. If the tables grow between the two mutex locks, they're only going to grow. Like we know that we don't shrink the table and so we're looking at safe entries. We just might not actually catch all the entries and that's bad. It's a good point cuz when you grow the table you rehash and all this stuff. So I either we hold the mutex for a long ass time.

[00:32:51]Um Which I don't think that's causing that assert by the way.

[00:33:14]It's just It is a good point the twitch chat brings up here.

[00:33:23]It is a good point.

[00:33:28]Just don't exit the game. Yeah.

[00:33:31]Yeah.

[00:33:36]Okay.

[00:33:41]Address table mutex.

[00:33:45]I mean if we're Unity game engine we just grab the address table mutex the whole time.

[00:33:54]I just would prefer not to do that. We could copy this array while holding the mutex the first time and then copy it over later.

[00:34:23]I feel like that might be our best option.

[00:34:36]Slug is public domain now. I saw that.

[00:34:38]That's pretty cool.

[00:34:42]I've never actually looked at the algorithm so I don't know if I would prefer to use it over just what we do now. Although there are benefits.

[00:34:54]Right? So if you felt the code complexity was good.

[00:35:00]You know, since we have since we have a an intellectually well-to-do beta program full of smart people.

[00:35:10]I imagine somebody will just port it at some point.

[00:35:18]I do not have time to do so.

[00:35:21]I might do so at another time just for the benefit of everyone who uses the compiler, but this is not the right time to do it on the year when we are shipping a game plus the compiler plus the engine.

[00:35:38]It's just a lot.

[00:35:52]Will you see people implementing it in a day or so?

[00:35:57]I mean I guess.

[00:36:05]If it's that easy how is it patented?

[00:36:13]So with slug is it is there still a license?

[00:36:22]Can you license an algorithm like that?

[00:36:25]I doubt it, right? Like Need a license the library itself.

[00:36:30]Right? But if you did your own port Right. So if it's if it's your source code. Okay, there is a gray area though.

[00:36:45]Here's the problem.

[00:36:47]The problem is most people who do this kind of thing are not very careful about it legally speaking. I mean of course the laws vary around the world anyway, but like if you make your own version of it by looking at the original source code and then you decline to license the original library um they could sue you for that.

[00:37:26]Right?

[00:37:27]So it has to be like a clean room implementation from just the algorithm description not looking at the code.

[00:37:43]He published reference shaders. Yeah, but are they are the are the reference shaders in the public domain?

[00:37:50]Or are they you know, whatever.

[00:37:53]You know what I'm saying.

[00:37:55]You don't think the original source code is available. Okay, I see.

[00:37:59]Okay.

[00:38:00]Then that's not an issue.

[00:38:13]All right.

[00:38:14]What do we do about this?

[00:38:18]Um Here is the thing.

[00:38:46]I think I I don't remember, but I think this is everything in the heap right here.

[00:38:53]And of course if you were being careful about this and trying to keep utilization minimal you would just send what happened this frame.

[00:39:10]And you would rely on this the client to put the information together over time, right?

[00:39:21]But we don't want to do that cuz at least for now if you send it this way and they don't have to put that information together over time. It's just like easier to scrub and stuff cuz all the data is right there without you having to reconstruct it.

[00:39:34]Long-term though we might want to do something different. Okay, but for now this is the problem is I suspect this is a lot of stuff.

[00:39:44]Right? And so um So, we don't want to hold this mutex for this long. And so, the alternative is to copy this thing. All right?

[00:39:59]So, let's look at this.

[00:40:01]Um Okay, this also, by the way, is uh not good.

[00:40:12]Okay, so uh I should have should be more careful.

[00:40:22]Okay.

[00:40:26]So, we're going to assign these while holding the mutex.

[00:40:33]And then assign it into the data.

[00:40:44]Let's check to see if there's anything else. Alex, this. Nope, not till down here.

[00:40:52]Oh, and then, of course, Okay.

[00:41:12]All right.

[00:41:14]So, let's do this. Uh bytes for mutex controlled data.

[00:41:31]Right? And that's going to be It's going to be this stuff.

[00:41:53]All right. So, now we're going to need to do this copy.

[00:41:59]Okay? And then down here, we're going to go um mem copy data um uh mutex controlled data copy, right? Um.data mutex controlled data copy. count. All right.

[00:42:31]And we're going to go data plus equals this.

[00:42:36]All right.

[00:42:40]And then that should come out the same.

[00:42:44]Uh we don't ac- Okay, well, we don't actually need this.

[00:42:53]Okay. So, um here we're going to go Uh what what do I call this? Control mutex controlled data copy.

[00:43:16]Mutex controlled data copy if dot uh allocated is less than um bytes for mutex controlled data.

[00:43:34]We don't want to do this very often.

[00:43:39]Uh array reserve controlled data copy here.

[00:43:49]That finger.

[00:43:51]So, that makes sure we're at least as big. And now we're going to go mutex controlled data copy. count equals the the this.

[00:44:03]Dude, I was traveling for 2 weeks and just typing on a laptop keyboard the whole time, and it's hard to get used to this. All right. So, um data is um mutex controlled data copy.data.

[00:44:33]And then we cursor it forward, and we're going to assert data is equal to mutex controlled data copy.data plus mutex controlled data copy. count. All right.

[00:44:46]Um so, what is this?

[00:44:51]It's going to be uh one of these.

[00:45:02]Okay.

[00:45:09]I'm going to rewrite this comment. We'd prefer not to be doing much inside this mutex.

[00:45:14]But but report.sorted summaries could be quite large.

[00:45:28]Um You know, writer reserve message bytes total.

[00:45:36]Yeah. So, that's fine.

[00:45:39]Okay.

[00:45:44]Could be quite large.

[00:45:50]So, we copy the per frame data, which is hopefully pretty small, so that we don't have to hold the mutex that whole time.

[00:46:06]There could be some other creative solutions to the problem.

[00:46:13]For example, we only need to know the byte total because we are sending a single message containing all the data.

[00:46:25]But if we broke the mutex controlled data into a separate message, we could uh perhaps uh >> [laughter] >> avoid some of this entanglement.

[00:46:57]But then you'd have to be guaranteed that the message allocation is fast.

[00:47:07]Message target memory.

[00:47:09]Okay.

[00:47:11]Okay, so um I mean, I just typed some stuff.

[00:47:25]Right?

[00:47:26]We probably made some mistakes there.

[00:47:29]Who knows?

[00:47:31]It compiled the first time, which of course means we one-shotted it, obviously.

[00:47:47]Okay, I messed that up. 233.

[00:47:59]Okay, this is weird. So, Alex's frame out locks this frame. count times that freeze this frame. count. Bytes for mutex controlled data.

[00:48:13]So, this is Did I mess something up here?

[00:48:29]This looks correct to me.

[00:48:38]>> [snorts] >> I guess we will look in the debugger.

[00:49:16]Dude, we have multiple instances.

[00:49:19]Our cloud credits are just climbing due to all the instances we are running.

[00:49:49]>> Okay. Boom, right away.

[00:49:54]Vis update.

[00:49:59]Data.

[00:50:01]Commander data mutex controlled data copy.

[00:50:08]Okay, well, it is a startup frame.

[00:50:11]So, admittedly, it's probably big. Okay, so um I mean, we honestly probably shouldn't be allocating and freeing that much stuff at startup, so that's something we could look at.

[00:50:35]Um Okay, is the mutex even doing its job?

[00:50:43]No.

[00:50:47]Okay, guys.

[00:50:50]This is what I'm saying about like I want to add the ability to assert that the mutex is held. Look at this.

[00:50:56]Look at this.

[00:50:58]We assign it up here.

[00:51:01]We're supposedly inside the mutex, but by the time we get down here, uh it's not. Now, this could actually be us.

[00:51:23]But, I don't think so, cuz we don't we wouldn't free anything on the first frame.

[00:51:53]Hello.

[00:51:56]Raiding party of 12 people. How you doing?

[00:52:02]How you doing? All right.

[00:52:06]Well, let's go back and check that real fast. I I feel like this might be us.

[00:52:15]But, I don't think an array reserve would do four allocations in one free, but it might if that includes our internal structures.

[00:52:24]So, I mean, honestly, we could just do this.

[00:52:29]All right. I don't know if this re- redoes correctly, so we're just going to restart that every time.

[00:52:35]So, we're just going to set a breakpoint there.

[00:52:45]Okay.

[00:52:47]Allocates frame freezes frame 9288.

[00:52:55]So, these numbers match.

[00:52:57]Boom.

[00:52:58]Okay, wait. That that those line numbers.

[00:53:02]Okay.

[00:53:04]It looks like this counts us.

[00:53:12]Which is a problem.

[00:53:26]It is a problem. We're going to maybe step down in that, but why?

[00:53:37]Maybe I just didn't recompile enough times.

[00:53:48]Okay.

[00:53:52]So, 92842511.

[00:53:58]Step. Okay, we actually match now.

[00:54:00]So, we're going here.

[00:54:02]We're going here.

[00:54:05]That's just a macro on arrays. So, we're calling realloc.

[00:54:09]Calling the overriding allocator proc, which is a debugging procedure. We're calling the uh actual core allocator to get the memory. I'm going to skip over that.

[00:54:27]And then we're calling record new memory.

[00:54:32]Um Okay.

[00:54:39]Is that a macro? Let's memory debugger report new What?

[00:54:52]What?

[00:54:54]What?

[00:55:10]Oh, that's overriding allocators record new Okay.

[00:55:14]Okay. So, this is all So, overriding allocator has all its own [ __ ] Because when you free something, we need to know how big it was. So, this is a thing that just puts trash bytes into your memory when you free it so that you know you freed it, right?

[00:55:40]In debug builds. All right. So, this is where we are looking at now.

[00:55:46]We lock the address table mutex. We find We put that allocation in our address table, which is another bookkeep.

[00:56:08]So, we're doing tracing.

[00:56:12]Okay.

[00:56:13]Now, add an allocation if we're visualizing the memory debugger, which we are.

[00:56:23]Okay.

[00:56:38]So, we're going to go table add.

[00:56:47]Okay. Table's fine.

[00:56:51]Let's take a look at it.

[00:56:55]Slots filled 9284.

[00:56:58]Allocated 16384.

[00:57:01]16384 *.7 is 11. So, it's a while before this table would grow.

[00:57:12]So, we're not really going to allocate anything there.

[00:57:16]Okay.

[00:57:19]Except right.

[00:57:26]So, we are right here adding an allocation for this frame.

[00:57:35]Um Okay.

[00:58:00]So, this is happening because I kind of made an oopsie.

[00:58:07]And the oopsie is as follows.

[00:58:10]Uh we shouldn't be using the overriding allocator anyway. We might still need to deal with this separately, but when you're writing a memory debugging system like this, you want to be um insulated from program state.

[00:58:24]And so, um in general, uh we explicitly assign the allocator, right? Let me look for that happening.

[00:58:43]Uh in the init function, for example. Uh let's look at stack trace table. Stack trace table.

[00:58:52]Yeah, we're pushing context-based default allocator.

[00:58:56]This is what we have chosen to do here, right? And the reason we're explicitly pushing an allocator is because we're being called from a program that could have the allocator set to anything, right?

[00:59:08]That's also true here.

[00:59:10]So, when we are here, um uh mutex controlled we want to be using our own allocator um like this.

[00:59:41]Now, that might not solve the problem.

[00:59:44]Uh but it will reduce the problem because part of it was that other overriding allocator doing thing, which is why I was surprised cuz in my mind I was like wait, we're just Okay, I forgot to start the thing.

[00:59:58]We're just doing the default allocator.

[00:59:59]No. No, we were not doing the default allocator.

[01:00:21]Oh, are we we're breakpoint? I was sitting there waiting for something to happen.

[01:00:26]I'm like, oh, we're breakpointed. Okay, so now Okay, well, we still this is still counting as one, right?

[01:00:39]The problem is the problem is we're reporting this.

[01:00:47]Do we I mean, do we like reflexively, recursively, self-referentially account for this extra allocation by doing it again?

[01:01:02]I guess so. I mean, this is this is a little bit crazy balls, but um cuz we have to have the memory for it, right?

[01:01:34]Um Right?

[01:01:59]So, we're like, okay, there's actually going to be one more allocation than we wanted.

[01:02:06]Now, this is a weird thing. I mean, maybe your memory debugger should just be stealth and not do any of this stuff.

[01:02:19]But right now, that's not how it is.

[01:02:22]Uh Although, actually, maybe it could be.

[01:02:26]Hold on.

[01:02:30]I don't remember what policy decision we made on whether this is what we want to be doing for everything inside memory debugger.

[01:02:40]What's it called?

[01:02:45]context.

[01:02:47]inside memory debugger.

[01:02:50]Shh.

[01:02:52]False. Okay. Actually, actually, um I think the problem is we actually should be doing the following.

[01:03:24]Although, actually, in when when is this set?

[01:03:33]When we make the leak report, when we shut down, when we init.

[01:03:42]Yeah.

[01:03:44]All right.

[01:03:51]Let's just do this here.

[01:03:53]So, we're going to undo that.

[01:04:10]And we're going to do this.

[01:04:17]Cuz the thing is the reason I say policy decision is because on the one hand, you might think, oh, I really want to know about all the memory that my memory debugger is using cuz I want to trace literally everything, right?

[01:04:30]And it's like, yeah, but on the other hand, that's also just noise that keeps you from seeing your actual program's behavior and your shipping program isn't going to have the memory debugger in it, right?

[01:04:44]So, um All right, that still [ __ ] did it.

[01:04:52]What is going on?

[01:05:09]Okay.

[01:05:11]So, I mean, I I guess we're calling into the memory debugger somewhere else.

[01:05:35]And the pro See, the problem is this inside memory debugger is meant to be it's meant to be a leaf operation.

[01:05:49]Like a Leaf Erikson operation.

[01:05:51]Wait, did we not Oh, wait.

[01:05:58]Let's just recompile just to make sure.

[01:05:59]I think I did.

[01:06:33]Okay.

[01:06:36]So, I think what's going to happen is we're going to go we're going to do an allocation and it's going to get shut off, right?

[01:07:01]Yeah.

[01:07:03]So, oh, when we make the leak report, right?

[01:07:38]But I feel like that's that's not even necessarily Well, we'll see.

[01:07:56]Okay, so let's go back to this scenario on here.

[01:08:11]Hey, look, it stayed it stayed the same.

[01:08:15]Okay.

[01:08:16]It's tricky. I never quite remember because I don't work on this code that much. I never quite remember exactly what's going on, but I believe that when that value is set, we do record the allocation now that it happened.

[01:08:35]Because if you free that memory later from something, we need to know that it's a valid free or whatever. I don't know.

[01:08:44]I don't know, but we don't we don't put it in this thing.

[01:08:51]I don't think. We can look into that some other time.

[01:08:55]The point being this All right, so this is going to be thousands for now, and then we get down here.

[01:09:08]I'll just step it.

[01:09:10]Boom. Boom, it worked.

[01:09:13]Worked the first time, baby.

[01:09:17]For some definition of first time.

[01:09:20]All right, so Hey.

[01:09:25]It's working. All right. So, that was just all to fix that. I feel like we got to run the tests and take the W, but we still may have this assert when I quit the game. Let's see what happens.

[01:09:37]No. Okay. So, actually, I guess it was just when you close the game, there's more intensive uh there's more intensive memory activity, so you're more likely to hit that race condition. We should test a few more times, etc. But, let's uh Um we set inside memory debugger so that we don't mess up the accounting that the user sees.

[01:10:16]And also, so that the report size stays stable while we are trying to transmit it.

[01:10:28]Um Okay. However, however, make leak report sets this to false assuming that the caller is not inside the memory debugger.

[01:10:51]Mayhap we should do this in a more formal way, but for now, we just set it back to true.

[01:11:04]Keep in mind, there is a defer active that will set it to false still when we close the scope.

[01:11:16]Okay.

[01:11:18]Don't you wish code you were reading had nice comments?

[01:11:29]Okay.

[01:11:41]This is one of the reasons.

[01:11:56]Okay.

[01:11:59]Okay.

[01:12:13]Cool.

[01:12:16]Shall we Let's make a release build of the game, which due to LLVM general as bloat, will take a long time.

[01:12:28]Uh but we'll make it with the memory view enabled, right?

[01:12:33]So, that's just going to stress it out more.

[01:12:42]Is it accurate to say that a hybrid programming paradigm is the best way to manage complexity once the core problem is solved?

[01:12:51]I don't So, the examples given being procedural or OOP, data oriented.

[01:12:57]I mean, both OOP and data oriented are procedural, so I don't get that first of all, but secondly, I mean, I don't use object oriented for basically anything at this point, right?

[01:13:12]I use uh I do use a one layer deep hierarchy of things for like entities and sometimes for other things, but they don't like have methods and stuff, right? So, like entities in this game do not tick themselves.

[01:13:40]That's literally not anywhere in the game at all.

[01:13:45]So, not really. I mean, I would say what you said is true in the sense of like if you really work hard at programming and figure out what really works for the kind of problems you're going to solve, it probably doesn't match anything that anyone teaches you in college exactly and may in smaller parts resemble some of those things, right?

[01:14:09]But, like hybrid gives more credit to various things than they maybe deserve.

[01:14:18]Like, oh, your choice is to hybridize by taking different It's like, no. It's It's dude, it's code, man.

[01:14:27]You write the code that works and it works.

[01:14:32]You know what I'm talking about?

[01:14:43]Okay.

[01:14:46]So, All right. I mean, we're running around.

[01:15:11]I mean, I don't All right.

[01:15:17]Boom.

[01:15:25]What did we free?

[01:15:27]Input per frame event update window.

[01:15:30]Interesting. All right.

[01:15:42]I'm confused because Oh, we're looking at one frame.

[01:15:48]I was like, why? Because I was clicking on the wrong thing, that's why. All right. So, what here? Send key event.

[01:15:58]That's probably Yeah. So, this is trimmed so that you don't see everything all the way down, but um array add like That's a known leaf, we call it, right? So, you know, this calls Alec down and so we stop there.

[01:16:22]Uh making a bitmap.

[01:16:27]Yeah, you know, we could get rid of all this if we did some slug going on.

[01:16:34]Oh my god, the animation system, dude.

[01:16:39]All right. Array resize load all paint data.

[01:16:45]Load level and then general.

[01:16:48]Yep.

[01:17:06]I mean, this all looks reasonable.

[01:17:28]So, here the the transaction system does Alec and free.

[01:17:33]And that's something we could clean up in the end cuz it's not doing anything really complicated, but that is something that happens right now.

[01:17:41]Um Scan one entity. So, down here, right?

[01:17:47]We see both So, this was freed this frame, but we also see that it was when it was at or We don't see what frame number it was allocated. That could be a useful addition, but we show you where it was allocated.

[01:18:08]Anyway, let's run the general compiler test testerinos.

[01:18:23]Just to make sure we didn't break nothing there.

[01:18:25]And uh let's make sure we didn't leave a no check in.

[01:18:30]And then, I think I think that's the main thing we wanted to do tonight. We're going to ship another beta soon.

[01:18:37]Um Oh, I didn't need another editor. We have one.

[01:18:47]Please.

[01:18:55]Um basic um memory debugger fixed a race condition in the visualizer.

[01:19:14]Um modified uh uh uh Wait, what what would I say?

[01:19:22]Um, took extra steps to ensure that allocations internal to the visualizer do not pollute the user's memory report.

[01:19:40]Did we do anything else?

[01:19:43]Hey, look, it all passed. Um, So, we did that. We did this.

[01:20:07]I don't like it when the lines of code go up by that much, but yeah.

[01:20:12]Okay, now that's that's pretty much what we did.

[01:20:16]Dude, we went from 300 lines to 370 lines.

[01:20:22]Uh, but some of that's comments, so whatever.

[01:20:27]Whatever.

[01:20:32]Whatever.

[01:20:43]Okay.

[01:20:43]Cool. Um, I will reply internally where you cannot see to a team email.

[01:20:51]And I will say, um, I supposedly fixed this just now.

[01:21:05]Please test.

[01:21:07]Keep in mind, uh, uh, that said, we don't have any kind of ASan equivalent at this time, so or sorry, TSAN.

[01:21:26]TSAN equivalent, so it's hard to know for sure.

[01:21:35]Da da da da da da. All right.

[01:21:39]So, dude, TSAN, I keep thinking once in a while we should implement our own TSAN and make it built-in and really easy to use.

[01:21:46]And then I go every time I go read the TSAN paper, I have no idea what they're talking about cuz they start out by just saying, "Hey, they do the mission statement, which is like, of course, it's a good idea to detect these bugs and stuff.

[01:22:06]And then they're just like, "And we do some ad hoc stuff, and we detect some threads to And it's really weird because it's like I was always under the impression that it was some kind of like actually accurate race detector, and it's not. It's just like, "Well, we did some stuff, and we kind of detect some things.

[01:22:30]The end."

[01:22:33]It's pretty weird.

[01:22:48]Any technical questions that anyone has?

[01:22:58]Can we please get a no AI discussion?

[01:23:00]The problem is, if I ban the string AI, which I would love to do, I think it'll hit any word with the substring AI in it, right?

[01:23:50]Space AI space.

[01:23:59]AI Pacino.

[01:24:23]All right, are there any actual serious questions before I go out and have fun at the dance?

[01:24:36]How do you get out of beginner hell?

[01:24:40]Have felt like it for 2 years now, can't seem to improve. Okay.

[01:24:43]Um, what What do you mean? Like, what is What is happening to convince you that you're not improving? Like, what's an example?

[01:24:59]Cuz it's hard to answer that question without more concrete information.

[01:25:28]What TypeScript is to JavaScript? No, that's a completely inaccurate.

[01:25:34]Uh, completely inaccurate.

[01:25:51]Will PhilC actually work and make C memory safe? I don't know anything about PhilC.

[01:26:21]You want to work on a project and keep running into problems.

[01:26:25]You cannot seem to solve and struggle to find solutions. Okay, let's talk a Pick a problem. Let's talk about it right now.

[01:26:33]What's an example of something you couldn't solve?

[01:26:37]One specific problem.

[01:26:39]Because maybe what's going on Like, successful programming involves a certain way of breaking down problems into easier problems. And if you don't get that, everything's going to be hard forever, right?

[01:26:58]So, type in the problem, and we can look.

[01:27:26]You want us to develop a Sims-like game?

[01:27:28]You know, back in like 1998, I wanted to do a game vaguely like that, but it wasn't really Sims. It was a little different.

[01:27:43]But, you know what? We didn't make that game.

[01:27:48]>> [snorts] >> You can't think of anything specific, but memory allocation issues has been a recent problem.

[01:27:58]What kind of memory allocation issues?

[01:28:01]Like, do you mean like you're freeing something multiple times and it's a bad time?

[01:28:10]Or do you mean like you're using something after it was freed?

[01:28:15]Or what?

[01:28:17]What what kind of issues?

[01:28:39]The Sims, but it's survival genre. There are several games that fit that description.

[01:29:12]How do I differentiate between ideas and figure out where there's potential? I've given so many talks about that.

[01:29:20]What do you mean?

[01:29:38]You got wrecked by hash table pointers not being stable. What Well, that's I mean I would assume there's even a comment in the file that says that.

[01:29:51]Right?

[01:29:56]Maybe there isn't.

[01:30:03]Oh, no, cuz this has a giant comment about how it works.

[01:30:10]Yeah, there's no pointer stability in a hash table, obviously.

[01:30:26]How do you tackle physics-based algorithms? For example, 2D orbital mechanics.

[01:30:35]Uh you didn't mention an algorithm.

[01:30:39]What do you mean physics-based algorithms?

[01:30:46]If you mean how do you implement physics for a game that has things that orbit in 2D, first you implement the very simple version, which is you use the stuff you learned in high school physics and do a step-by-step simulator.

[01:31:07]And then eventually you discover that you need to do some things to get a little more accurate than that.

[01:31:13]So, like you at least upgrade to midpoint method, but you you probably eventually do like a more powerful integrator, and that gives you uh the opportunity to learn about integrators, and so forth.

[01:31:28]But, that's not an algorithm.

[01:31:33]You also got wrecked by this. Guys, get good.

[01:31:43]It is It is a little bit easy to forget about these things sometimes, right?

[01:31:51]I mean, if this were the 1980s and you were Microsoft, you could name every single variable in your program by whether the pointer is stable or not.

[01:32:04]Right?

[01:32:07]Um and then the Rust version of that is like that's literally the lifetime value, right?

[01:32:14]But, you know, weak pointer gang, I I don't understand why anybody likes weak pointers, but whatever.

[01:33:14]They're like they're such a pain in the ass implementation-wise, unless I mean, unless you do the generational index thing. That's like the only But, like why? It's just not good.

[01:33:43]It's not a good way to do things.

[01:33:48]Because like if your weak pointer is null now and you wanted to use it, what are you going to do?

[01:33:57]Like either you didn't want to use it and and therefore you shouldn't be holding a pointer, or you want to use it.

[01:34:06]But, but now you're like, "Oh [ __ ] the thing went away.

[01:34:10]What do I do?"

[01:34:27]Must have missed the talk where I talk about how to select what to convince the people to work on. I mean, >> [clears throat] >> that comes up in like to some degree in literally every talk.

[01:34:40]Like why do I do these talks? It's because these things are interesting to me.

[01:34:45]Why are these things interesting to me?

[01:34:48]Well, because Wondering what I think about stackful, stackless. I don't give a [ __ ] about coroutines. I don't I don't know why people like them.

[01:35:10]All HTTP calls and database calls block the thread. Yeah, well just have a thread do that.

[01:35:18]I don't like Like it doesn't take much to start a thread and give it a function and say do this function, right? That's the thing.

[01:35:39]And so, the reason I don't care much about coroutines is because they take all this extra infrastructure to give you like a small amount of syntactic sugar on top of that. Now, this becomes a little bit different if you have a different thesis about the language. Like So, for example, Go has a thesis that it makes sense to have a 57 quintillion coroutines, most of which are asleep, all right? But, I don't think that's true. That doesn't make much sense to me at all.

[01:36:14]So, like I Why can't you just have a function and say, "Job system, please do this."?

[01:36:31]You know?

[01:36:35]How do you design a satisfying ending for the game? You know, the answer to that is different for every game.

[01:36:42]How do you handle recursive dependencies?

[01:36:45]I mean, in this language they're just like automatically handled. I don't I don't know what the problem is.

[01:36:55]Is the metaprogramming system basically just macro maxing? No.

[01:36:59]No.

[01:37:01]What are you talking about?

[01:37:04]That's That's actually an offensive question the way that you phrased that, and it's like so clueless about how the language works, you get a temporary ban for that.

[01:37:15]Don't ask intentionally aggravating questions when you don't even know what you're talking about. Holy [ __ ] It's just not polite.

[01:37:25]It's not polite.

[01:37:47]You could do async to async? I mean, I don't think So, here's the thing.

[01:37:56]You never really want to do anything totally async when you're reading a web page anyway, right? It's like because if you're going to somewhere to request some data, it's cuz you want to use the data in a second, right?

[01:38:11]It's just you might want to do three things at the same time because the web is all messed up that way. So, yeah, you just do the three things at the same time, and then you wait until they're all done, and then you go, right? Like that's fine. That's That's a multithreaded program thing.

[01:38:31]I just I don't really understand what coroutines really gives you apart from oh, I'm in a super high-level programming language, and so I can pretend I that I have this certain infrastructure, right? But, this isn't a super high-level programming language in that way. I mean, it's very high-level for a systems language, but it is a systems language.

[01:38:54]So, you know.

[01:39:20]>> I mean co-routines and the the other reason why I say that I don't think much of them is just they're like a lot of things in programming languages and frameworks and abstractions and all this stuff, right?

[01:39:35]There's this very, very common pattern.

[01:39:39]And the pattern is this thing looks tidy and convenient and neat for small, easy problems.

[01:39:50]And then doesn't help you much for difficult, gnarly problems.

[01:39:56]That's like 98% of language features and abstractions and all that, especially in the past 20 years.

[01:40:05]And the problem is things that are easy are already easy.

[01:40:10]Why do you want to put Why do you want to spend the complexity budget of your system on making things that are easy 20% easier?

[01:40:20]I mean, maybe maybe if everything you do is easy, then it makes sense to accelerate that by 20%, but that's usually not how it is. It's usually you blast through some easy stuff and then you hit one of the hard things and the hard thing is pretty hard.

[01:40:33]And like, every all the design that I do is to help with hard things.

[01:40:40]Because that's the limiting factor on what we're able to accomplish is how many hard things are we able to do?

[01:40:48]And co-routines is one of these things that helps you with easy things and I just I don't care about that.

[01:40:59]The problem is you block the thread holding the connection. What?

[01:41:11]You No.

[01:41:13]What? No.

[01:41:21]I mean, unless you want to, no, you don't have to block the thread holding the connection.

[01:41:39]Implementation-wise, it's potentially more performant than a gazillion OS threads if you have a very, very large number of them.

[01:41:46]But nobody does.

[01:41:48]Nobody does. Like the example that I just gave you.

[01:41:54]I mean, again, there's there's two separate worlds, right?

[01:41:58]There's the web world where people think about concurrency.

[01:42:02]But concurrency, as I always say, is kind of a weasel word.

[01:42:05]In the game world and in the other high-performance application world, you care about parallelism.

[01:42:12]Parallelism is a stronger word than concurrency cuz it means you're actually doing things at the same time.

[01:42:18]Concurrency means the things may be overlap somewhat. It's a very weak word.

[01:42:27]Or it you know, the things maybe are unordered.

[01:42:33]Right? Like what?

[01:42:35]Um so parallelism means oh, we're actually using the hardware that we have to get the task done more quickly. And if you want parallelism, then the number of threads that you have should be related to the number of cores that you have.

[01:42:52]Okay? And there's no reason to create threads for everything like in the web example that I just said.

[01:43:00]You just write a five-line function or whatever that gets the page and puts the result into a known place and you give that to the job system and that fires and and then it's done. You don't create a thread for that, right?

[01:43:19]You don't want to create a thread for that because you don't want tons of things running all the time, right? So One of the things he was trying to implement is to park jobs that are past a predetermined deadline.

[01:43:54]Wouldn't co-routines help?

[01:43:56]Since Windows dropped user mode scheduling blah blah blah. I don't know anything about user mode scheduling in Windows anyway. Um but wouldn't co-routines help there? I mean you could do that with co-routines, but you can also do it without co-routines.

[01:44:17]Like Okay.

[01:44:20]There's one very fundamental thing about programming in systems languages or many languages, but you know, like functional style languages give you primitives to deal with the following thing that I'm about to say.

[01:44:37]Uh when you're programming in system languages especially there's this thing where your control flow is interleaved with like the stack, right? There's this implicit data structure called the stack that has to be there for you to run.

[01:45:00]You know, and as you call functions and return from them, the stack bloops up and down and all that, right?

[01:45:06]And then the thing about what you're talking about, about suspending some code, naively in a systems language you're like, oh, I don't know how to do this cuz I called all these functions and like I'm just on the stack somewhere and I don't know what to do, right?

[01:45:24]But there are two basic ways to handle it.

[01:45:27]One is on most systems languages, we actually don't provide a fundamental way of doing this, but you could implement it yourself, but you know, in C back in the day, you would use setjmp and longjmp and figure out the range of the stack and you would just copy the stack if you wanted to do that, which is maybe expensive back in the day, but these days is very cheap cuz stacks are not that big.

[01:45:58]Um you could do that, right?

[01:46:03]Uh but but I wouldn't prefer to do it that way anyway. What I would do is um just have So so the the way to become more powerful in situations like that is you just disentangle the state of the job that you're trying to do from the stack, right? And that just means you have a data structure that represents the task you're trying to do.

[01:46:29]You have a function that just iterates over the upcoming tasks and looks at them and makes progress on them, right?

[01:46:37]And once you centralize it in that function um you uh you know you can just yield if something is taking too long and whatever and go to the next one.

[01:46:52]That's kind of like writing your own scheduler, but it's tremendously, tremendously, tremendously simpler because it's only a simple set of well-defined things that you know everything about, right? Now, of course that by itself implies some constraints on the code. So you can't call something that blocks indefinitely because you wouldn't be able to yield it, right?

[01:47:16]But like you know it's fine. Just don't do that.

[01:47:29]Concurrency equals using same core.

[01:47:31]That's not true.

[01:47:33]No, concurrency is not even that well defined.

[01:47:36]But two things on different cores are concurrent for sure.

[01:47:42]It's just concurrency like I said, it's just a weaker word than parallelism.

[01:47:58]I'm saying a timeout fires. No, not necessarily.

[01:48:02]The what I was outlining is more of a cooperative thing. Um like you're talking about with co-routines, but co-routines implies a lot of things that are not really necessary if you just need to solve this problem within your own system. Like the thing is, a co-routine abstraction is it's like built for customers of this language feature and so it does a lot of things to handle a lot of things that people need to do in various cases that you don't necessarily need to do.

[01:48:37]Right?

[01:48:42]Oh, you meant to reply to the earlier question. I see.

[01:48:54]All right. Anyway it is time for me to head out. Thanks, everybody.

[01:49:01]Uh let's raid somebody since um since it's a while, sometimes the rating is grim, and we don't know how to do it.

[01:49:17]Um >> [snorts] >> Who shall we raid?

[01:49:30]Who shall we raid?

[01:50:04]Travis Roman.

[01:50:06]Okay, we'll do that. I hope it's good.

[01:50:10]Thanks, everybody.

[01:50:16]And uh see you later. I'm back from traveling, so we might do more streams.

[01:50:21]Who knows?

[01:50:24]Who knows?