building my own OS in c++ (week 121, stack guard pages)

Added: 2026-05-25

[00:00:01]All right, week 121.

[00:00:05]What is up everyone? Week 121 of building my own operating system. Uh, it's officially summer time, so we're doing a bit of a summary stream.

[00:00:16]Um, yeah, hope everyone is doing well.

[00:00:20]Let me just set up a little bit. Um yeah, so a couple things planned for today.

[00:00:33]Um mostly still continuing to work on this scheduleuler stuff.

[00:00:39]Um so let's just zoom into let's just see what's what's going on.

[00:00:49]So um it is Week 100 21 it's May 24th.

[00:01:00]Um lately we were working on some smart pointer stuff that is mostly finished. We were working on refactoring how the register state is stored in tasks. whether to store it inside the task strct or instead whether to store it on the stack.

[00:01:29]Um, and then last time we did a little bit of sketching, a possible refactoring for the scheduleuler state task and whether we could use a a variant type for that. And I think the conclusion was that it's not really worth it.

[00:01:56]It sort of was kind of cool um to explicitly make the state sort of like an object that could have state associated just with one particular state, but it wasn't really worth it. It sort of cluttered things up. This the stood variant type is a little bit ugly.

[00:02:25]to use.

[00:02:28]Um, and yeah, I think it wasn't quite worth it for me. So, um, I'm just going to sketch out a couple ideas for today. Also, welcome to everyone joining the chat right now.

[00:02:54]Feel free to say hello, drop a comment.

[00:03:00]Um, yeah, I'm just going to slowly get into it. I'm just going to sketch out a couple things um to start could start with today. Just some brainstorming about what I could do. So I had this idea refactor um move object for setting up task stack into the setor constructor.

[00:03:27]That could be an interesting idea.

[00:03:35]Yo, what's up? What's up, future? What's up, Lane?

[00:03:40]Looks like you're on vacation or on a secret mission. I'm on both.

[00:03:47]Summertime.

[00:03:49]Yeah. 2 a.m. Oh man. I'm going to guess actually where would that even be? Is that uh maybe maybe North America or something like that. Maybe west I'm guess like west coast or something like that. That's cool. Thanks for popping in the chat. Pacific Northwest.

[00:04:13]Yo, what's up?

[00:04:15]Garantana Grant Offn. What is up? Oh, Oregon. Repping in Oregon. That's cool, man. I've been to Seattle before, but I never went to Oregon.

[00:04:28]I went to Washington State, but never went to Oregon. So, nice.

[00:04:36]Well, cool. Everyone feel free to drop in how you're doing. Give a little check in. Also feel for you to share anything you're working on lately. Off and on with the test one for sure.

[00:04:48]Nice.

[00:04:51]Drop a test two just for fun.

[00:04:58]North Macedonia. Cool.

[00:05:01]I've been to North Macedonia. Well, good friend of mine's from North Macedonia.

[00:05:07]Um, yeah. I was in Scopia for a little bit.

[00:05:12]Yeah, beautiful, beautiful city. Uh, yeah. Cool. Waking up tomorrow live notification. Oh, too kind of off none.

[00:05:21]Also, waking up. Y'all, you were grinding late last night or something.

[00:05:29]Uh, yeah. All right, let's talk about what I'm going to do today.

[00:05:34]Um, I have some other ideas. And so factor the scheduleuler in general. Try to get rid of the we rid of the weird bridge. There's this weird bridge function that I have.

[00:05:54]Um what's up Abdullah? Nice to see you.

[00:06:01]Lastly, the the bigger topic is um revisiting um task stacks and guard pages.

[00:06:18]Do it properly.

[00:06:21]This this topic is something that goes back a couple weeks. I mean, the last time I was working on it, I think maybe over here. No, even before maybe even pre-week 115. And so, that's something I still have in mind. It's a bit It's a bit of a bigger thing. So, I think I think I'm going to just try to go with these two things for today. Um, you know, or actually, you know, I think I'm just going to dive into the day topic to be honest. I I don't have the longest amount of time to stream today and I want to like if I spend time on this sort of littleer thing that doesn't really matter. I'm not going to have enough time to really get into this one. So, um I'm just going to propose that we just do that.

[00:07:20]All right. So, um, let me just, yeah, let's just get into get into things. So, um, we're building building and I really don't have a great un a sense of what the current state of the build is, though. So, I think we'll just need to spend a little bit of time um just trying to understand what was even happening because I was doing a bunch of stuff and so and I I was just going kind of fast with my commits and so they're all whip.

[00:07:55]I'm going to clean that all up later, but um so here's where I was working on the smart pointer stuff and that's unrelated to what I'm talking about today. So that's things should be working there. Then this was a random random cleanup.

[00:08:21]This is like a mix of a couple of things. Unfortunately, this is like this is okay. This will be a good opportunity.

[00:08:32]I'll just Yo, what's up VA? Thank you, man.

[00:08:36]Yeah, it's that time of year. That time of year to break out break out the the clothing.

[00:08:48]All right. Um, so unfortunately this is a mix of quite a few different things.

[00:08:53]And just real quick, I can just show you how I Let's say I wanted to clean this up because this this um some of this stuff is not really related task exit.

[00:09:16]Okay.

[00:09:22]So, this is another wonder if I can just move this down and okay in a way I don't uh feel like cleaning up.

[00:09:35]Sub salvation.

[00:09:43]Fix this up.

[00:09:46]Okay, then here's where I start getting into this thing I was trying to do. Let me just let me just revisit and give an overview of the topic. And so what are we talking about? Um, we're talking about task stacks, guard pages, and doing it properly. Meaning, so um, as usual, we're going to sketch out a bit of a memory diagram.

[00:10:09]And oh, wait, I drew a frame by accident. As usual, we have a memory diagram.

[00:10:15]And as usual, we're going to have a stack.

[00:10:21]Um, and it's going to be the this is going to be a stack for a task. And I want to have, you know, I actually I drew this out before. Um, there's it already exists in somewhere up here. Um maybe down here.

[00:10:56]Yes, here.

[00:11:00]So we have tasks in the OS.

[00:11:05]Um we each task has a stack and the lazy way I was doing it is um I allocate some physical memory some memory for the stack but I don't do anything particularly special to map that into the address space. I just use this kind of fallback mechanism called the fs map which is a mapping of all physical memory. So any any possible physical address is just automatically available in the fizz map and that's very easy very simple very lazy but has down has a lot of downsides mostly in terms of safety and so in this setup I don't have um I don't have guard pages because the entirety of the fizz map is read writable and it's also dangerous because um if I if the stack does overflow, meaning kind of like underflow, but like if if if the stack pointer goes off the end of the stack, there's going to be no guard page.

[00:12:17]And not only is there no guard page, there is possibly something else just randomly allocated in that memory below it and that's going to get corrupted very easily. And so the idea is to bring this one step higher in terms of quality.

[00:12:33]Um where we are actually going to map these into the address space somehow. And not only we're going to do that, we're going to um add a guard page for each one of these things.

[00:13:00]And the question is how exactly to do that. Um, currently I do not have anything that allocates out virtual memory really.

[00:13:15]Um, if I did, I could kind I could basically sort of use that, but that's not really the approach I'm going for with my OS, though. I'm trying not I'm trying to avoid dynamic allocation.

[00:13:31]So, I'm trying to prefer static allocation because that's more interesting to me to explore.

[00:13:43]And I I I had had some unsuccessful attempts at this actually. Um so and we can even see this in the history.

[00:14:00]This is a clean up. I think this is the main commit we can look into.

[00:14:08]Um, Okay, I think I have a better uh grounding now. Just trying to understand what I was trying to do some experiments. Basically, I had this approach at solving this where I don't even have my editor open. Okay.

[00:15:23]Um, so refresh on theuler.

[00:15:29]And why are there some squiggly lines here?

[00:15:39]Okay, cool.

[00:15:43]Okay, the refresher on the scheduleuler.

[00:15:47]So I have um the idle task.

[00:16:01]It has a pool of one stack for itself for for some reason for for certain reasons. Um and I also have all the other tasks which are stored in this task list which is this ETL list.

[00:16:22]Um then I also have this pool of stacks which the tasks allocate from.

[00:16:33]What up soul collector is it better than C or C++?

[00:16:38]Um, probably rust is probably the answer. Um, it's it's it's unfortunately it's not super simple. People have a lot of different opinions on this. This is hotly debated.

[00:16:49]Um, and it depends on a lot of questions like whether you're trying to write a production OS or maybe just an experimental OS or what your goals are even to be honest. So, I have to say it's not the easiest uh simplest answer.

[00:17:10]But here you can see okay here you can so the task um pool is used right here in the create task method where we create a task. Um, what I was trying to do is I had this idea to unmap the guard pages from the pool.

[00:17:38]Um so the idea was to sort of take this pool and the thing the idea behind a pool is it's um a region of memory with space for a certain number of of copies of a particular um structure.

[00:18:02]So it's not really a general purpose memory allocator. It's an allocator for a very specific type of object that you want to get from the pool. In this case, it's our task stack, which I am choosing to model as a strct currently. Um, maybe kind of a weird idea, but I'm doing it that way.

[00:18:23]Um, and so these in memory, these objects are all laid out right next to each other. And because I am directly embedding this into theuler object, this is going to be part of the kernel's virtual memory area which is illustrated basically right here. The kernel is mapped into the top 2 gigabytes of virtual memory. And um I'm going with the static allocation approach. I'm using basically this global kernel object that's like a global that's literally a global object as my singular piece of global state and I'm just putting like everything uh into that.

[00:19:14]Um did you go to college? Yes. Yes. Soul culture I did. Um I have a bachelor's in computer engineering.

[00:19:28]Um right and so the idea is that okay we have this pool of stacks inside the address space and it's it's illustrated right here so inside the kernel these stacks and the idea is to on boot up of the OS do some initialization where we go through each of these stacks and unmap them.

[00:20:07]Um, yeah, where can I learn? Absolutely. I got you. I have a resources page explicitly for this question. It has a bunch of links to courses and stuff like this which you can use as a starting point.

[00:20:23]So, let me just I just need to think this through. So, we So, the idea was that we have this pool of objects. I can go through and unmap the ignore this metadata page. That's that was sort of a weird idea I had at the end of stream, but but let's just um Okay, time for a 20 second break. Be right back.

[00:21:07]Right. So the thinking, the theory was that I could pre-process this pool and I could um let's pretend this is not there. The guard page is the first member of the structure. So I could go through the pool and I could unmap the first the guard page from all of the stacks.

[00:21:31]And the thinking was if I do it this way then for the rest of the runtime of the kernel I can just um I don't need to worry about this. The garbages are just there. They're unmapped and um and they just simply sit there unmapped. And so unmapped portions of the kernel's address space the the problem with this approach, right? The problem with this approach is it actually just doesn't work. Um it is and that's what we were seeing in I think it was week 116 I wanted to say. Yes, that's in week 116.

[00:22:31]We investigated this. I tried this and what we discovered is that when a object in the pool is um is is basically unallocated and so it's empty basically and ready to be used.

[00:22:52]The pool actually stores metadata in the first at the start of the object and this is used to store a free list to internally implement the pool allocator.

[00:23:09]And this is incompatible with my scheme to try to unmap all of all of the the guard pages because those come first in this task stack object. And so the guard page occupies the same memory as the metadata that ETL needs. Uh, ETL is this library I'm using for data structures and this is a conflict. Um, yeah, Funk channel, what is up? Yeah, of course I remember you. Thanks for popping in again.

[00:23:55]Nice to see you.

[00:23:58]Yeah. And so the guard page conflicts with the metadata that ETL stores in the first part um yeah and so this approach just simply didn't work. Um, and that's I'm I'm sort of see and that's what we're seeing here in this history where I um was attempting to implement this approach.

[00:24:34]So we can just sort of walk through a little bit. And so my and in order to implement this approach, it was actually kind of involved um because the idea was I I'm trying to go through all the stacks in the pool and unmap the first one. But then how do I actually do that?

[00:24:58]What I need to do is I need to actually allocate every single stack in the pool and so I can access it and unmap it. Um, in order to do that, I actually need to exhaust the pool and like basically empty the pool. Because otherwise, if I allocate one thing from the pool, unmap the stack, and then release it back to the pool. The next time I try to allocate from the pool, I'll just get that same stack object back and it will already have its guard page unmapped.

[00:25:37]And so in order to make sure I to really go through the pool like this um I need to exhaust the pool. I mean another possible implementation is to make assumptions about the memory layout of the pool and just assume that the pool is a bunch of objects that are contiguous in memory which might have made things simpler but it's a little bit um less sound of an approach. And so the the approach I had is I need to um I need to exhaust the pool which means I need to be allocating every single thing. In order to do that I need to have I need to keep track of all of the stack pointers that I get back so that I can release them all at the end.

[00:26:40]But then the challenge is actually how do I actually keep track of all those pointers? Um this is this is potentially a lot of pointers. you know, I I might I'm aiming to have hundreds or thousands of stacks on the system and I need to have some kind of container, some kind of dynamic array or container that's going to store all these pointers to all the stacks. And again, I'm not I don't really have dynamic memory allocation. I don't have um an easy way to make a vector, for example. And so the workaround that I ended up using was I one step back. So I don't have these dynamic arrays where I can just kind of append to the end and they grow automatically cuz I'm again I'm using more of a static allocation approach.

[00:27:30]What I do have via ETL is is these kind of static arrays or static vectors where it's like it's a it's a dynamic array with a fixed capacity that is pre-allocated.

[00:27:46]This is all a long story that sort of doesn't matter. But the way I ended up going is I ended up implementing a function that alloc allocates a range of physical memory. I'll just show you the code um on map guard pages. So the approach was okay. Okay, I have my static vector type. I'm going to compute the amount of memory memory I need to um hold a vector that can store all of the tasks.

[00:28:29]So I'm storing one pointer to a stack for every task. I compute the amount of memory I need. And then I allocate that range of of physical memory.

[00:28:38]I locate that address in the fizz map and then I um I construct the vector in place into that memory. So that's going to give me uh a container I can use to store the pointers as I exhaust the pool.

[00:28:56]So now I have this ve vector container.

[00:29:00]Um and and then I have this loop while the pool is not full which is sort of that was a little bit confusing to me. I remember when I implemented this but a full pool is one where there's nothing more you can get from it. So um every slot is filled. While the pool is not full, I allocate a stack. I unmap the guard page and then I record the pointer um of the thing that I allocated. And then with this loop, I will exhaust the pool on map all the guard pages. And then at the very end, I can have a loop that loops over this pointers vector and releases all of those things back to the pool. Then I release the physical memory. And so the idea was that um I would be a I'll be able to use this to actually implement my my goal of exhausting the pool and pre-undmapping all of the stacks and then but that failed because of this metadata thing that I talked about. Um and so the whole point of saying that is like that was broken so I need a new plan. I have an idea and I'll sketch that out now. So um so I'll just summarize this. And so um so let me summarize problem. I want to have task stacks and guard guard pages. I don't want them to be in the fizz map.

[00:30:36]Uh they should be in the normal kernel virtual memory area.

[00:30:43]um idea um in pre-process the ta the task stack will unmap all the guard pages up front so I never have to worry about that again.

[00:31:06]Um problem But the problem is um can't unmap the guard page for the objects in a pool because because ETL needs that to be writable to store metadata for unused objects.

[00:31:42]This was also annoying and involved um and required exhausting. And the pool was also um cumbersome and required a container for all pointers uh to store pointers to all stacks.

[00:32:12]Okay, so that's all problems.

[00:32:15]This was totally broken or this totally didn't work.

[00:32:26]Okay.

[00:32:32]So, idea number two.

[00:32:35]Um, idea number two.

[00:32:39]So, the idea is we actually sort of need to implement a a bit of a custom allocator, which could be kind of fun.

[00:32:45]Um, okay. Let me check in. Let me check in on the chat.

[00:32:54]Yo, what's up, Bisha? So, Ragghav, bro, why didn't you build it in Rust?

[00:33:01]I'm going to hit you with the rust command.

[00:33:04]Um, there's a lot of reasons why I didn't use Rust. And I think actually, yeah, Nav really got the answer. Um, the answer is that I I know C++ very well and I'm very comfortable with it and I don't know Rust at all. So, I'm just focusing on learning one thing, one hard thing at a time to not overwhelm myself.

[00:33:28]Use as logic when you need speed or that's what I hear from Google. All right. All right.

[00:33:34]What's up, Hap Gamer? 120 weeks. Yes, sir.

[00:33:39]Yes. 120 weeks.

[00:33:44]Make a game compatible. Sounds hard.

[00:33:52]How did coding change you as a person?

[00:33:53]That's deep. That's deep, man. Also, what's up, Prawn Rain in the in the Twitch chat? Everyone say hello to Prawn Rain. We don't get I don't get too many people in the Twitch chat, so I'm really happy to see someone popping in on Twitch.

[00:34:10]Idea two, um, we store stacks somewhere in an array so that it's safe to unmap the guard pages.

[00:34:29]Um and then separately off to the side keep track what is used used or free.

[00:34:43]Um can I use a bit map allocator?

[00:34:47]The bit map allocator is is becoming my go-to type. It's really simple to use.

[00:34:54]It's pretty simple to implement and is is I think it's a good um good choice for this use case.

[00:35:06]What's up, Pron? Pron is this open source? Um maybe someday, but currently it is just my private little coding project.

[00:35:17]Yeah, that's where that stands currently. Okay. Okay. So, the idea is to have we're going to have um how do I sketch this out? We're going to have let's say these are the we're going to have a bunch of stacks allocated and then off to the side we're going to have a bit map allocator that keeps track of um which you know this which of these are free for example or which are used and it can be used to um allocate stacks in a way that doesn't that allows the the guard page to be unmapped and doesn't conflict with that and in a way that doesn't store any metadata directly in the object and so we're storing the we're storing the metadata off to the side in the bitmap allocator.

[00:36:11]So yeah, what's up?

[00:36:18]Doing well. Doing well.

[00:36:22]Um, all right.

[00:36:34]Oh, Shrea Scar, thank you for the correction.

[00:36:39]All right, let's try let's start implementing this. I'm going to I'm going to try to get to a sane place before I start implementing this crazy physical memory stuff. I think that was around here. Here, I'm going to make a new branch. I'm just going to copy the branch name. I'm going to come here, make a new branch called dev 2.

[00:37:00]I'm going to build um Shriascar asks, is this a course?

[00:37:11]No, I would not say this is a course. I would say we're just uh this is just a video of me working.

[00:37:21]Yeah. And and just showing how I do uh my operating system development.

[00:37:31]Um but um but if you're interested in a course, I'd be happy to talk about that. Feel free to I don't know. Yeah. What would you be interested in learning from a course from me, if anything at all? I'm I'm actually super curious to to learn about that.

[00:37:51]All right. So, my last question before I run the kernel is I actually want to see what the kernel is configured to even do.

[00:38:06]Um, what's up, Naughty? Why are you using an Apple keyboard not mechanical?

[00:38:17]That's a good question. I think I like how small the Apple keyboard is.

[00:38:25]Um, that might be the main reason. I just I just I like how small and minimable the Apple keyboard is. And I like that it's nice and wireless and works well with my computer.

[00:38:38]Okay, so we're configured to run a a single task. And I think um some somehow my claim D keeps getting messed up.

[00:38:47]Okay. Um so okay cool. We are currently configured to run one task which immediately exits.

[00:38:58]Let's just confirm that's what what's happens.

[00:39:03]Okay. And it exits. And now we're getting an assertion failure saying the idle task is exiting. So let's let's just get ourselves into a sane state and then we can begin idle task. Idle task should not exit. So I think I I put this in there for debugging, but really the idle task is I mean in in the current form it shouldn't even this doesn't even make any sense because there's no preemptive multitasking and so nothing's going to happen. Um, that'll task runs when nothing else is runnable. And, um, in the absence of interrupts and preeemption to like make things runnable, um, doesn't make sense for the idle task to spin yielding because there's no possibility for any other task to become runnable. And so, so in practice, right now, that idle task just needs to panic.

[00:40:11]Okay. And that's okay. So, now we're at a good state.

[00:40:15]Yeah.

[00:40:24]Um All right. And now whip. Um get to same state again.

[00:40:36]Uh don't judge me on my commit history.

[00:40:40]Still in progress. Still still working.

[00:40:42]So, what's up, Roit? Yoyo, GitHub.

[00:40:50]Let's see. Is there a GitHub coming, man?

[00:40:58]Just curious what it says.

[00:41:01]Join the newsletter for updates on the future GitHub release. I see. Yeah.

[00:41:05]Unfortunately, there's no GitHub for this right now.

[00:41:12]But if you're interested, join the newsletter.

[00:41:18]Okay. So, let's start implementing this. Okay. So, so so back to our design. So, there these two things. Let's let's do the first one.

[00:41:27]Nice and easy. We're going to we need to um task stack pool.

[00:41:37]We need to go here first off.

[00:41:39]Yeah, first off, let's see.

[00:41:44]Um, it might be nice to implement a custom type for this. Let's um I'm just going to I'm just going to sketch this out. I'm going to call it task stack pool.

[00:42:11]and task stack pool.

[00:42:19]And I'm going to snooze the inline suggestions for 5 minutes.

[00:42:24]And let's see. So, I'm going to need a few things. I'm going to need a array. I'm going to need the task stack header. So, I'm going to start out. Um, I'm also going to need some constants. Yeah, that's kind of annoying. It's annoying that these constants um or do I actually No, I don't. Um, I can make this a template.

[00:42:55]Uh, no, but it's a a size T. Um, uh, what's the It's like numbum tasks.

[00:43:08]call I don't know t num task um so array um for task stack t num tasks m task stacks Damn, Ibrahim got censored by the bot.

[00:43:43]I'm sorry. I don't particularly think that was a banable offense, bro. I'm I'm going to repost that message.

[00:43:55]I don't know why I feel a guy as scary and skill as a Don't be scared.

[00:44:14]No, I feel though C++ is kind of a crazy language.

[00:44:21]All right, so we're going to have um so again just to just to sketch this uh copy file, copy the name. So, um, task stack pool, right? And so, the goal is to get rid of this line and instead we're going to have task stack pool m task stack pool.

[00:44:46]That's the idea. And to get this, we're going to need to add the I can't see. Um, I'm going to add the include right task stackpool.hpp which is over here elbot.

[00:45:12]What's up sedarth? What do I need to do to make such a project? Like read an OS book first. What to even start?

[00:45:21]What's up sedarth? So, I'm going to hit you with my resources page. I'm also going to say I think for someone in your position, what you need, what you should do is take advantage of all of the free coursework that's available online.

[00:45:37]We also just hit the 20 second break.

[00:45:40]So, I'll be back. Feel free to join me for a stand up and stretch.

[00:46:01]Yeah. Anyway, um yes, it so basically I think there's a lot of free coursework available.

[00:46:09]Um, recently I've been recommending Harvard CS61 to people and Afnobenzy. What is up? What's up, my friend? Very unusual time I made. Yes, I'm sorry. I had to move it a bit earlier today um for some plans I have in the afternoon.

[00:46:32]Um, I still wanted to get a stream in.

[00:46:35]So, but you're still we still got about um let's say 45 minutes left and so I think yeah, I'm glad you're here and you have the beach vibes for sure.

[00:46:48]Okay, so let's keep going. So, the idea is we're going to have our stack pool.

[00:46:52]Um, and actually, you know what what I'm realizing is there's even um a pre-preparation that I want to do um even before I do this. And so, let's look at our let's talk about our bitmap frame allocator. I already have um so the this is the physical memory allocator in my OS. It's called it's called bitmap frame allocator. Frame referring to physical frames of memory.

[00:47:27]That's the terminology I'm using. And it's underneath it's using this thing.

[00:47:35]It's using this type called a frig bit set which is this library I'm using for some data structures.

[00:47:42]And and the interesting thing about this is that it is basically exactly what I want for my task pool. The only unfortunate thing is that it is hardcoded for this um physical memory allocation use case.

[00:48:10]And so what would actually make sense to do as a pre-preparation for implementing my custom task pool um snooze inline suggestions for bits set allocator goes here. Um so as a pre-preparation what I can do is I can extract a generic bit mapap allocator type out of this. I can reimplement the bit map frame allocator on top of that generic bit mapap allocator and then I can use the generic bit mapap allocator for my task stack pool. So, I'm going to venture to say that we should actually go with that.

[00:49:10]All right. Um, cool. So, let's let's see. What do I want? I want to probably dash these changes.

[00:49:30]Um that that work. Yep, that worked. Okay, cool. So let's uh let's let's just go through with this.

[00:49:53]Um so I'm going to duplicate this file.

[00:49:57]Call it bit map allocator. It's going to be it's this the idea is it's this generic type. Um get rid of some stuff here.

[00:50:12]What do we need? Um yeah, this is going to simplify things. So I'm again I'm going to make this a template type or no size t um t bit set at size.

[00:50:42]I yeah I don't know my my naming is TBD on this. I want to make it clear that this is a template argument. So that's why I have it like this. And so t bit set size.

[00:51:01]Um and this is interesting. We're no longer doing anything with memory maps.

[00:51:08]Um, and I think the type is just the um index.

[00:51:39]Right.

[00:51:43]And so I Let me because this is a template I'm going to copy some code.

[00:52:01]copying pretty much a lot of this stuff really.

[00:52:07]Um, let me see index.

[00:52:26]Okay.

[00:52:38]Okay.

[00:52:43]Um, Benzina, what are you working on today specifically? So, the goal today specifically is working on um task stacks and guard pages and doing it properly.

[00:53:06]That's that's really the goal for all my tasks running on the system. I want to have a stack obviously, but also a guard page beneath it. I want these things to not be in the fmap region. They should be in normal kernel virtual memory.

[00:53:23]There were a lot of problems with my first attempt at doing this which I did in week 116 115 maybe um kind of explained here. And then now I'm implementing my second try uh which is a different approach that should not have the issues that the first one had right and so so you were not using stack for user space you decided earlier. So, um, user space is is really not in the picture yet.

[00:54:07]It's really not in any way in the picture for now. Um, it'll come later, but none of the choices I'm making really um affect the user space work right now. So, it's kind of separate, I think.

[00:54:30]Okay. And so what's up with with this?

[00:54:32]So how the bit map frame allocator initializes is itself in initializes itself based off of the memory map.

[00:54:44]Um this but this is all pretty specific functionality that is tailored to the frame allocator use case. And so for a again we're trying to make a generic bitmap allocator right without any particular semantics associated with the allocator.

[00:55:11]So, I actually don't think any of this is relevant. And so, we don't I don't need this.

[00:55:21]Don't need this.

[00:55:24]Um, and then uh I might not even need this.

[00:55:39]Yeah.

[00:55:41]So this is approximately what's compiler support.

[00:55:49]Ah okay types for uh actually yeah do we do I need this adder you know I'm not really sure I don't think the memory types are used.

[00:56:09]We don't use that header. So, so this is ah uhhuh okay interesting. So let's think so allocate and we find the first index and we mark it used and I think we just uh return the index and this is this what type is this first it's an N64 is that RZY How long have you been programming?

[00:56:54]Pretty long time. Um I don't know like 13 years, 14 years, something like that. Yeah, pretty long time.

[00:57:13]Okay, so let's just make sure this is right. So, all find the first set because um How long will you do programming? I see.

[00:57:52]Okay, let me just This is always the tricky part. Um for defy defaults to zero.

[00:58:09]Interesting. Okay. So for the for the frame. Okay, frig defaults everything to zero.

[00:58:27]And for the frame allocator use case, the idea was to have everything be unused or everything be used actually.

[00:58:40]um and selectively mark parts of the address space as um brie for use.

[00:59:08]Mhm. Okay. So, that is actually interesting. It's It's also a bit problematic.

[00:59:31]Um, all right. Thanks, Aphnoenzene.

[00:59:37]Have fun at work.

[00:59:39]catch you later peace.

[00:59:42]Okay, let me just type this out. So the bit map is interesting and so the bitmap allocator. So we have the bitmap frame allocator frame allocator um used for fizz mem allocation defaults uh defaults every thing to used because it wants only to selectively enable parts of the address space based on the memory.

[01:00:18]map.

[01:00:20]Um, but for a generic bit map allocator, we want to default everything to Bree, which is sort of the more natural semantics of an allocator.

[01:00:40]You create it, everything is free by default, and then you can allocate from that. Um, and so these and so the the default state of the allocator is the opposite. Um, this wants a bit map allocator that that defaults everything to used.

[01:01:09]Yeah, this wants the opposite.

[01:01:17]Um, and the question is how do I how or or rather wait a sec. Um, the gen generic bit mapap allocator but then also the task stack pool bit map allocator. These are sort of the more typical case and they're going to want to default everything to free. And the question is for the generic bitmap allocator which needs to serve both of these purposes.

[01:01:46]How do we actually architect that in a way that can that can support both of these use cases and there are some different possibilities um support these use cases.

[01:02:14]The simplest and kind of stupidest one is add a bool to the seour.

[01:02:25]Um, for example, bool default to used to all used or something like this.

[01:02:41]And then this thing can pass. one thing that's going to pass the other one.

[01:02:47]Um, that's one way to do it. You know, people talk about how bad it is to have bool arguments for things.

[01:02:56]So, what would other ideas be? Um, dedicated classes for each. That also sounds kind of weird to me to be honest.

[01:03:11]Um, frankly, that doesn't sound like better to me than having a bull ctor. So, I'm just going to go with the bull for now, I think. Um, so in this case, we actually do need appracops.

[01:03:38]Yeah.

[01:03:43]Uh yeah.

[01:03:46]So the idea is we add a bool argument and then um I'm going to add back this init bit set because we're going to need that or init bit set.

[01:04:08]It's going to be private.

[01:04:10]Wait a minute. Bit set.

[01:04:16]Ah, it actually sort of knew. Oh, okay. So, it's going to depend on the bool. Um, let's check out what this thing provides, though.

[01:04:35]Yeah, this sets everything to zero.

[01:05:20]Yeah.

[01:05:22]Um, so basically the idea is I want to have a bull and depending on the bool we're going to I think I don't even want this comment really. Um, but you know, I want to I want to make sure I really understand this set.

[01:05:54]The C tour is Yeah, you can see it here. It sets everything to zero.

[01:06:06]And the code is is sort of Mhm.

[01:06:22]It's designed with the physical memory allocator in mind.

[01:06:29]Okay. Basically the idea is um Yeah.

[01:06:46]And the AI is going to kind of do it.

[01:06:48]We're going we want to have a bool. Um, and we're going to pass it in. And then based off of the bool, we're either going to do nothing and leave it all as zeros, meaning it's all used, or we're going to go through and set everything to one.

[01:07:08]And um All right, be right back for the 20 second break.

[01:07:51]Wait, is there a set?

[01:08:03]So you can just set the entire thing.

[01:08:04]That's actually exactly what we need. So yeah, reset sets every uh reset sets everything to zero.

[01:08:12]Set sets everything to one it seems like. So So actually yeah, it's kind of cool. So um default to free.

[01:08:42]I feel like that should be the default.

[01:08:45]Or maybe I want default to default to maybe it makes sense to have two constructors actually. There's like the default one that just does the normal thing that you want to do. And then I think I sort of like that one better.

[01:09:15]Um, default.

[01:09:38]Sorry, I think that was a stupid suggestion.

[01:09:41]Um, I think actually it's more like factory functions or something like that.

[01:09:47]I think I'm not going to focus too much more on this. This is that's more of a smaller detail. Um, but this is this is enough for now just to get going.

[01:10:01]So, cool. So, we have a our bitmap allocator.

[01:10:37]So now the idea is to implement I'm just going to actually do a stage.

[01:10:45]So now I'm I'm going to lean a bit more on the AI for this I think. Um, and so I want to say we want to just please reimplement the bit map frame allocator using the bit map allocator.

[01:11:02]Um, which is which is frankly a really like mechanical wrapping kind of change. And so I'm gonna go with the I don't have good I don't have advanced models apparently. So I'm gonna go with the claude.

[01:11:24]And let's just check out Wow. tools plant.

[01:11:35]I want to reimplement a bit uh map frame allocator using the new bit map generic bit map allocator.

[01:11:49]Uh, I just want to see it it do this should. So, basically all it's going to do is going to replace this member with a bit map allocator and it's going to make all these functions just sort of like wrappers over the bitmap allocator that converts these size T's into the fizz adder fader. So, I'm just have it uh make plan.

[01:12:10]Let's check that out.

[01:12:19]I think I should be able to do this pretty quickly. It's just it's just like a couple files.

[01:12:25]Um yeah, in the meantime I'm going to I haven't um a host test.

[01:12:44]Okay, test unit tests are still passing.

[01:13:04]If that takes too long, I'll just do it myself. But um basically the idea is this just this just becomes a call to m bitmap allocator alloc and then uh context.ext text. Interesting.

[01:14:18]actually. I mean, I think yeah, it's found a small oversight didn't have allocator HPPP. Um, where did it let me use my brain? So bit allocator just this type has all these bits and its job is to scan the bit set looking for the particular first bit that has a one basically and return that and then mark it and so isn't that what it does and first that this is actually kind of an interesting thing. If index is greater than or equal to Oh, wait. Yeah, that's just the error handling market used and return it. Yeah, right.

[01:15:31]So that's the basic operation of the bitmap allocator.

[01:15:48]Mhm. Frame word frames isn't really the right word here.

[01:15:53]So let me just let's just read this plan.

[01:15:58]Um it's free already asserts then marks much worse ends all bits start used.

[01:16:18]Okay.

[01:16:19]include the header, replace the member with the false, right? Because we want it to default to used.

[01:16:46]Then we can remove is used mark. Yeah, all of these can be removed. These things still need to stay here.

[01:17:02]Yeah. And we call um we simply call the outlook and then convert this to is pretty easy.

[01:17:15]Just call the free delegate.

[01:17:19]Uhhuh.

[01:17:29]Right. So then it's talking about um I see that's that was the issue. And in order to customize, right? in order to initial in initialize the frame um for the frame use case it wants to go through and kind of tweak things up front which requires mark ah but but it tries marking things as free.

[01:18:14]Um, so but I can't do that cuz now they're cuz now the bit map allocator type doesn't expose. So what what's the plan here?

[01:18:30]Replace.

[01:18:40]Right.

[01:18:41]But then um it can just use but right it could just it can just use the free API.

[01:18:51]Um wait I'm not looking at the wrong thing. bit mapap allocator hppp. So it can just use the free API obviously to mark things as free.

[01:19:03]So all right. So that totally works.

[01:19:22]Yeah, I think this plan is a good plan.

[01:19:42]I was going to say uh do it.

[01:19:52]Okay, cool. So, so the agent is kicking off this kind of uh mechanical refactor. That's why I'm using the agent because this is not really super interesting work and also I want a chance to continue to get better at using agents and also um frankly it can just do things very fast.

[01:20:21]I mean this whole thing like took a little bit of a long time. I was thinking and and stuff, but um uh it's sort of doing things a bit wrong.

[01:20:50]I wonder how I guide it. Anyway, let's just let's just take a look what it does. what it did and see if that matches our expectations. And so again, this is bitmap allocator is the generic type that we just added.

[01:21:02]Bitmap frame allocator is the physical memory allocator. And this thing goes this all the stuff goes away. And this is the key part. It's now implemented using a bit map allocator.

[01:21:16]Um configured to false and implementation wise, it's now it's just wrapping um the ALOC and it's getting an optional index and if there was a value, it just converts that um to a frame address um by uh shifting the bits cuz this It's basically a frame number. Um, converts that to a frame address by shifting the bits. You can see that here.

[01:22:00]It's just a shift left by K page bits.

[01:22:04]Cool. So, that looks good. And over here on the free side, this is this is pretty simple, too. We just pass uh the frame number into the free over here.

[01:22:20]Likewise, we delegate this.

[01:22:26]Um, you delegate this and then Yeah. And this is this is interesting that we even have this because it because the whole thing gets uh initialized to used anyway.

[01:22:53]But this line is there just in case um the memory map actually had actually specified that the first frame was usable. We still want to not use that. So um yeah and then let's talk about this. This is the one kind of interesting part here. So now we're in the init usable region function and we want to go through here and interpret the memory map. And so for every region that the memory map says is usable, we want to go through every frame in that region, mark it as free.

[01:23:40]And that's what I was doing here, right? And the reason it's done this way is because there's actually I mean wait couldn't it just um always allocate here I'm going to ask like an alternative implementation is rather than having this if here instead of mark used can't ah I see um you can't just call Alec because Alec like um will allocate an arbitrary thing. We want to specifically target the zero um the zeroth bit and force it to be zero.

[01:24:47]And this is how we we do this. So that makes sense.

[01:24:52]So that's looking pretty good. So now the moment of truth is we can build um the colonel no longer builds. We can build. Okay. Things just don't build. Um because missing assert.

[01:25:17]So now we can so I just built um both and now we can run the unit tests and verify hopefully that the bit map allocator bitmap frame allocator is still fully functional as a type and it seems like it is because we have um frame we have have this pretty comprehensive set of tests for the bit map frame allocator which are coming which are exactly for this reason we're trying to re refactor and reimplement the bitmap frame allocator. And we don't want to we don't want to break things in the process. To illustrate the point, we can try to break something like let's just break let's just trivially break um this code by always returning null opt from allocation. So so nothing can ever be allocated.

[01:26:09]Um hopefully the unit tests catch that and And they do.

[01:26:26]And interestingly, there was a SEG fault.

[01:26:29]So that is actually very unexpected.

[01:26:35]Take a screenshot of that. Um, but yeah.

[01:26:49]Um yeah, but that just goes to show that um the unit tests help catch the issues.

[01:26:56]And so a couple um kind of tasks are emerging from this work. And so um it's generic bit map allocator add tests for that.

[01:27:11]In addition to tests for frame allocator question mark, there's always this question of how much test duplication do you need? So um maybe if we have very comprehensive tests for the generic allocator, we don't necessarily need the full set of comprehensive tests for the frame than a frame allocator because we can assume that a lot of that is covered already by the the tests for the generic one. In in that case, we can trim down the bitmap frame allocators tests to be more for the specific details of what actually makes the bit map frame allocator unique, which are um this behavior around initializing itself based off of the memory map, for example.

[01:28:03]Let me check the chat quickly.

[01:28:06]Um, what's up, blood man? What is the OS's name? There is no name at the moment. It's unnamed.

[01:28:16]I'm planning to name it when I feel like it's ready to have a name. I don't I don't think it's ready to have a name yet. So, what's up, soul collector?

[01:28:24]Thoughts on AI and the future of coding, learning to code in specific? I mean, I think AI is very useful for coding or software engineering. Um, I think it's not going to totally erase all humans from this work, but it will change things a lot. I think it's going to collapse sort of like maybe more the kind of software engineering work that um is more mechanical and it's going to empower people that really know what they're doing to to achieve a lot more. So, quick 20 second break.

[01:29:24]MLB Sands, what's up? Which Linux distribution are you using? I'm using Mac OS as my Linux distribution.

[01:29:36]Okay.

[01:29:38]Um, so some work. I'm just going to capture this work. So what we have working is we have the generic bit map allocator ported the frame allocator to use it as pass. Good. Um this is sort of a bit of a future thing and just um yeah so I'll make a commit um reimplement bit map frame allocator using bit mapap allocator.

[01:30:23]Um, strictly speaking, I could even split this out into um, add bitmap allocator.

[01:30:34]So now we have now in the history we have this commit that just adds the bitmap allocator and theoretically it would even have tests in this commit and then in the next commit we have this one commit that ports things over to use the bitmap allocator.

[01:30:49]And so now we can finally get to the the the original goal here um which is to start implementing this um task pool and that means I need to go and pop my stash.

[01:31:10]So now let's talk about the task stack pool HP. So now that this is our custom type and so now that we finally have this um bit map allocator type which is over here we can actually use it. So bit map allocator exactly um exactly and so I think the AI knows what we want to do and it even knows we want to set it to true. So we want we want to set this to true because we want to default all of these to be free and available for allocation.

[01:31:50]What's up everyone?

[01:31:52]The OS name someone just asked this.

[01:31:54]There is no name currently. It is unnamed until I feel like it deserves a name.

[01:32:08]What about sockets? Yeah, that's in the future. That's that's probably in the pretty far future.

[01:32:14]Yeah, we're still working on on really the core system fundamentals right now.

[01:32:21]Um, okay. So, this is kind of interesting.

[01:32:26]Let's let's uh let's see. And I'll even snooze the inline suggestions for five minutes so we can do this out. So um so the basic idea is we want to have some kind of alloc function and free right um yeah I mean naming is is TBD ETL uses this create terminology.

[01:32:57]um doesn't honestly doesn't really matter.

[01:33:04]Um and so the basic implementation here is just going to be like um yeah and so Alec and we're going to return some task stack pointer possibly doesn't really matter. Um and index equals m bitmap allocal.

[01:33:34]Um and return if o index then we're going to task stacks index we're going to get the pointer otherwise return no pointer.

[01:34:05]And for free, we're going to take T and um interesting.

[01:34:24]Uh the distance is that a thing that I can use?

[01:34:30]Um, I mean one one thing I just want to do I just need want to do is I want to assert that um I want to assert that the stack pointer is actually within the array. So I want to assert that the stack is greater than or equal to mtask begin.

[01:35:07]I want to assert that it's less than m task stacks size.

[01:35:22]I also want to assert that it is is aligned.

[01:35:32]Um, okay. Page size and I sort of suspect this won't compile and pointer T.

[01:35:49]And this should be called P stack.

[01:35:57]Can you build an OS with assembly only?

[01:36:00]Uh yeah, sure. It it might be kind of hard depending on how complicated you want your OS to be, but you yeah, people do that for for a long time before there were programming languages.

[01:36:13]And so we're trying to free this thing.

[01:36:15]So I'm I'm I'm being very in um I'm interrogating this pointer and like really doing a lot of checks to make sure it's um where I want it to be. So for example, just so the test begin is like the the start pointer of of the first element.

[01:36:32]Um and it needs to be at least that. And then but then also let's say there's two um there's two there's two things you know we have to begin plus two is going to is going to lead to the first out of bounds that's it needs to be less than that and then in that case we can get the index and then um bit mapap allocator free index and I sort of anticipate that Uh with the dev docs distance, it returns a difference type.

[01:37:23]So we might get some warnings about the types. Um, so but this is approximately it. So we don't need a couple. We don't need this.

[01:37:41]We don't need any of this. Actually, we're not using any of this. We're not using anything x86. We're not using v address space.

[01:37:50]Um, we're not using login. We're not using badge.

[01:37:56]So this is our custom stack task stack pool implemented using an array and a bit mapap allocator. Kind of cool.

[01:38:09]Um now we can even see if it compiles.

[01:38:21]Um and that will probably be the last thing we do today. I need to get going.

[01:38:24]Um, right. And so what is this squiggly line about? Do I task stack pool? Right, we just find Oh, it's a because it's a template. And so we need to um there we go.

[01:38:55]So now we've um kind of sketched out we're going to replace the ETL pool with our custom task stack pool and I'm just curious to build the target and see does the kernel build and it kernel does not build because I have commented the line out. So now let's uh let's try one more time.

[01:39:24]Okay. And then now it's uh it's complaining that this member is same. I'm going to call it two just for the moment.

[01:39:34]And it compiles. That's pretty cool.

[01:39:39]Um so at least instantiating this template is compiling. You know, I actually just this is is like totally unre related, but I wanted to um refactor this thing to be numbum um free.

[01:40:00]Let's call it numbum num free because it's because this is because it's not appropriate to use the frames terminology in this type because this this is the generic bit map allocator.

[01:40:13]And I will actually, you know, it's just a random kind of moment, but I want to um just uh amend that into where it needs to go.

[01:40:25]Uh there we go. Cool.

[01:40:31]All right. And so I think that is approximately where we will need to leave it for today. Um, I'll just call this custom pool allocator or task stacks.

[01:40:48]Um, to be unmapped from each stack.

[01:41:08]Okay, cool.

[01:41:13]And there are maybe some ideas like like what if this returned a smart pointer?

[01:41:29]Smart smart pointer.

[01:41:33]That could be an idea.

[01:41:37]Um, yeah. All right, I got to go, guys. I got to go, but it's great seeing you all. Um, what's up, Andy? Would you say an average takes take a person? How about sort of mini O project? Um, I mean I think I mean it's everything's just so depends. But I think the main thing I want to say is that OSS can be much smaller and much simpler than what you might imagine. You know, you don't have to go with an incredibly complicated Linux pix compatible like preemptive kernel threads like crazy OS. You can build a tiny OS in two months for sure. Yeah.

[01:42:21]Um, or what's up new day using I'm using Yes. What's up, Sami? I'm using CMake. What's up, Yash?

[01:42:30]Um, check out my resources. There's a link um in the chat. What's up, MV Drama?

[01:42:38]It's going pretty good, Eric. Have a good night. Thank you very much, my friend. I'm going to hit the one hit the 100 a couple times. Um, thank you everyone. Feel free to join the Discord if you want to chat some more. I got to get going. Um, and I'll be back next week and we'll continue right where we left off right here. Um, my last ask is if you're watching this right now, if you like the content, please subscribe to my channel. That's all I ask. Please subscribe to my channel. Thank you very much. I'll catch you later.