FPS From Scratch (C++26 & OpenGL4.6) // Custom Profile Tooling pt.3

Added: 2026-05-28

[00:00:17]It's hot. It's too hot.

[00:00:21]It's too hot to be under a bright key light in a tiny room.

[00:00:31]But here we are. And I guess I guess threading and custom profiling is the best thing to do in this absolutely outrageous weather. Mysterious. Hello AC. Hello Connor. Hello Har. Hello Celestial. Hello V. Hello.

[00:01:08]Yes, Harand and Quo have been volunttoled to help out with minor administrative duties when AC is not around.

[00:01:22]uh Julian KK and I'm very thankful for them to uh help out a little bit.

[00:01:28]Everyone here is normally pretty well behaved, so I expect they'll have they'll have hopefully nothing to do other than just general heckle me from the back, which is fine, but it's nice to know. Uh takes a bit of pressure off AC as well. He doesn't always have to be around.

[00:01:46]Jesus Christ. Right. Um, what am I doing with my life?

[00:01:57]They've each split into their own separate domains. And the only reason they're separated is because I could not figure out what their respective names are on the platform they don't normally on. So, it's definitely a me error, but maybe I should keep keep the administrative powers segregated as much as possible.

[00:02:16]Is it possible to get the GPU usage of the process as in like as in part of this profiling? I don't know. I mean, what do you mean by GPU usage? Like what's what what metric are you talking about?

[00:02:36]It's just to keep the rowdy people at bay.

[00:02:43]You have no power over here.

[00:02:48]Right. Enough gassing. Let's get on with it.

[00:02:53]Right.

[00:02:56]What were we doing last time? We're building out our own custom threading primitive.

[00:03:02]Uh, yep. WSL minw. Uh, there's there's a docker file somewhere which does it all for you or does it all for me, I should say.

[00:03:14]Um so we recreated basically how stood thread just thread works with the passing of the arguments you're guaranteed to get a copy of them. Um we used the new C++ 23 decay copy functionality which was nice. We shoved it in a pupil. We whacked it in a unique pointer. We did windows create thread.

[00:03:36]We hoofed it over to a trampoline. The trampoline then recreated that unit pointer so we're all safe from tickly boo. We then did a double still apply to invoke dance to get everything going to the thread and we now have our own threading primitive which is nice.

[00:03:58]Uh I'm kind of getting used to the reflection syntax now. I don't think it's quite as weird as when I first saw it. It's just a It's just a splice, mate.

[00:04:14]Uh my thoughts on competitive programming are if you enjoy it, you should do it. I think ultimately what it teaches you to do is write really [ __ ] code really fast. And that's not in any way a disparaging or negative comment. I think that is that is not teaching you software engineering. It's teaching you to crank out C++ as fast as possible. If you enjoy it, I think that's fine. I am not convinced on the effect that has on your sort of employability. Some places might like it, in which case do your research and if if a place puts a lot of effort on that, then sure, do it. But I would treat it more as a hobby than anything else.

[00:04:52]Betho, uh, the dot dot dot means it's a variic template. Uh so where's a good example of this? So here this means that it can take any number of template arguments and this here unpacks it. So this basically says substitute this expression for every single one of these which has these types. So it's basically how I invoke f with all the arguments.

[00:05:20]So this bas this all this line basically does is call f with all the arguments you passed in and then I slip a little sneaky stop token in there.

[00:05:32]Right. Okay. Let's get on with it. And then cheeky fix that. Okay.

[00:05:41]Okay. So then we did our manual stack grabbing capturing which was kind of fun. Um, so we now can grab a stack trace of a thread. And this this code is actually really really interesting. I was quite pleased with this in the end.

[00:05:54]Like it nearly my brain nearly melted and dribbled out my ears from having write to write this in the heat. But here we are. So it uses the new C++ 6 in place vector which is a cross between an array and a vector. So you can push back and pop back, but it has a fixed upper bound which means it's stack allocated.

[00:06:11]So you kind of get the best of both worlds. So that means I I can have up to 100 things in here, but the amount I have is kind of relevant is only really dependent on how many I push to it. We suspend a thread and then we enter the danger zone because we don't know what state the thread was in when we suspended it. It might hold a lock. It might be doing some memory allocation.

[00:06:26]We don't know. So we can't do much. We can do kind of maths. That's about it.

[00:06:32]Um, we open the thread. I don't actually know if this is safe to call in a suspended thread. I do think we may have to call this outside, but so far it's been okay.

[00:06:42]So we get that should probably be outside.

[00:06:49]Oh well like what are you going to do?

[00:06:51]Uh then we get the thread context. So we basically say given the suspended thread give me the context which basically includes all the registers. We then see if we've got the rip and the risp. We uh stick those in the call stack using the very easy to remember try push back which will fail if you can't push it back as in we've filled up the capacity.

[00:07:12]So then we return false this immediately invoke lambda all the C++ keywords today. Then we do the riddle lookup function entry. So this basically says given given a instruction pointer like tell me what function that is which it does um handles reasonably elegantly leaf nodes uh i.e. those that don't have a function epilog and prologue otherwise it does unwinding. So it basically says given this function who calls that function and this is how we walk up the call stack and basically populate that and then we just hoof it out at the end. I'm hoping we get nrbo from this because otherwise you're returning otherwise quite a big copy but I'm kind of reasonably confident that we'll get that.

[00:07:56]Yeah, you can find the source code.

[00:08:00]Okay. So now the next thing to do and this this this this works right. We get tests. Lovely. Beautiful. The best tests in the world. Uh red tests.

[00:08:12]So using lambda soup we can force like a multi deep call stack and we can basically hey get make sure we get at least four elements in the call stack when we get that. So that's that's kind of pretty pretty cool. So the next thing to but this just gives us a bundle of void pointers right. So, the next thing to do today is to resolve them. I didn't I didn't don't think I did anything off stream other than um I think I tidied up some of this. I think I moved this handler up to the constructor and then I just reset it in. It was just a small tidy up. I don't think I did anything else.

[00:08:44]Um how do you think how do I how do you think about stood array versus stood in place? How do I personally think about them or like what do I think about them?

[00:08:55]I'm unsure of the question. like how do I decide which one to use or what do I just think of the the merits between the two uh s requires items a default? Yes, but the the thing that right so the the thing here right is that um it's kind of part of the interface right I don't know so I could have just used an array but I would have to return back something that tells me how many I've put into that array right because I don't I don't know how many stack frame when from the suspended thread when I generate the stack trace I do not know how big there's no way I can know a priori how big that's going to be so I have to guess at some upper bound and then I'm going to fill it and then I'm going to have to say the first n are part of the stack trace. Now I could just set the rest to null pointer cuz null pointer is never going to be uh a valid function address and call it a day. Um I could return I don't know like a pair or a structure or a tuple which basically says the stood array and then an index or a cursor into it but it seems like I for free I kind of get all of that with this new in place vector. And like also and I think this is important I've not used in place vector before and I really wanted to use it. So maybe there's a little bit self-indulgence, but it was kind of a fun fun excuse to use a new primitive. Um, but yeah, obviously I can't use vector because I cannot do in the danger zone. I cannot do any allocations. Allocation is out the window.

[00:10:21]Um, so I use the code on So I think today might be a bit of an exposition day cuz it's so bloody hot. I'm going to put off writing code for as long as possible.

[00:10:30]This just means it accesses it from the global namespace. So anything any of the the C API, anything that brought in, I can access immediately from this. And I go through I've gone through this quite a few times before, but I'm happy to go through it again. Um has two advantages.

[00:10:41]One, as soon as I type it, it opens up autocomplete. So now I can start looking for stuff like that, which is nice. Uh and the other advantage is it's to me, and this is a personal opinion, it is a visual cue that this is code I didn't write. So anything that comes from a third party or the platform, I use this.

[00:10:56]And then at a glance I can be like right and obviously I would never write open thread like that because I would use snake case but you know what I mean.

[00:11:06]Yes. So in place vector is what you get if you take stood array and stood vector and smash them together at high speed.

[00:11:11]Right. It's literally the it's kind of it's the cross of both of them. Right.

[00:11:14]So you get a fixed upper capacity stack allocated but you can push back and pop back from it as if it were a vector. So it's just nice. It kind of ties it all up. And then again like I will treat going over the end of this with try push back as a error and then that means I probably just need to increase this number right which we can just recompile for.

[00:11:34]Um, but I think I said I don't know if I've still got it up.

[00:11:40]Um, in place spectra I think is an excellent example of design by committee because it's got now notice as well there's no there's no allocator template to write. So it's stack allocated but it's got uh you've got m you so let's go for push.

[00:12:02]So you've got push back try push back unchecked push back right so push back is now the equivalent of is is a safe is push back is now safe in this so it will throw an exception if you try and push past the end of it try push back will tell you whether it happened or not and then unchecked is just that trust me bro trust me man trust me trust me guy trust me I know what I'm doing um and it will just it's undefined behavior if you push past the end of capacity it's it's designed So that you only pay for what you use, right? So push back now in in kind of I guess in more modern C++ should be most people reach for push back first and they wanted in place vector to be have push back be the safe option because memory safety is a huge concern at the moment. So if you push back and you can't don't have enough capacity for it then it will throw out out of range exception but not everyone wants that not everyone needs that. So you have the kind of escape patch here to get the the old push back um uh functionality. This is a wide contract and this is a narrow contract if you like those kind of terms basically. So you get the best of both worlds and then you can query it. I'm not that upset about that design. I don't think it's too bad.

[00:13:24]Exactly. Yeah.

[00:13:26]Also, memory safety may not be I'm going to be careful how I phrase this. May not be a concern for your app if it's not processing user data. If it's not public facing, if it's okay to crash, you know, kind of read, it depends on your situation. Now, now you can pick and choose, right?

[00:13:46]Right. Okay. So I'm going to try and do manual symbol resolution now at least in the win3. So I got bundle of void pointers. So now I want to turn these into strings. So strings that represent the function name the the name of the function that was that that instruction pointer points to. Um why would it be okay to crash? It depends what you're doing. Right? It's just a um if you uh don't think a good example now if you've got some backend service that you know is entirely in an environment that you control for example maybe it's better to fail file I mean technically it would invoke UB so it's not it's not guaranteed to crash um I guess it really depends on the situation right but the whole point in C++ is not forcing any choice down you it's giving you a sensible default now which is push back and given you the option to escape from that if you if you have kind of deemed it worth doing. So, right. Okay. Um, so I think so the reason, one of the reasons I did all this gubbins here, right, is well, one, it's fun. Okay. Hopefully educational.

[00:15:10]two um stood stack trace which basically solves all this problem for us has kind of two flaws in the sense in that one it allocates internally takes an allocator for example so you cannot do it from a suspend thread uh and the other even if you pass in a custom allocator is um it does all the simple resolution at the point where you call it mysterious has got a comment you have tell me where where do you want it quick have a think while I while I pat on uh append star star star to danger zone.

[00:15:42]Is that it? Is that what you want?

[00:15:46]Or is that being blocked out and you're actually doing some sort of what threeletter word that uh Twitch doesn't like?

[00:15:55]Oh, I see. Okay, fine. Hang on, hang on.

[00:15:58]Uh, can you danger zone? Give me a second.

[00:16:05]official video.

[00:16:12]Okay, I'm going to put it at the end just because I want I really I really do want that to be the first thing that you see.

[00:16:24]Hopefully that's okay with you.

[00:16:29]Okay, now that I fulfilled my obligation as a streamer.

[00:16:35]Okay. Right. So, anyway, what was I chat? What was I prattling about? Oh, yeah. Stat trace. It's not suitable for what we want to do. It does symbol resolution which is slow and I need this to be as fast as possible because the thread is suspended and I don't want to hold the thread make the thread suspended for too long because in doing a sampling profiler, you kind of want to jam those threads on super super quick and just like pull out the information.

[00:16:55]I don't know where to do symbol resolution because it's a bit of a pain in Windows. like Windows does give you the ability to do it, but it involves having some initialization that you have to do kind of basically once per process. So, it's going to have to be some sort of static, but it doesn't have any any really state other than that.

[00:17:09]So, I think it could just be a util function and we kind of bury the misery of the implementation within it. I don't think I have a concurrency utils yet.

[00:17:17]So, let's do source concurren. Is it concurrency or is it just a general utils?

[00:17:30]I mean, I I use Lance as a daily driver at work. I wouldn't say I was an expert in it, but I kind of I know how to CD.

[00:17:38]Uh, is there a concurrency thing or is it just a more core util?

[00:17:51]Uh, someone uh Zun Zuin probably apologies if I butchered that. Thank you very much for the prime sub. Right.

[00:18:01]Debug utils. I don't have a debug directory. I don't really want to have that within it. Uh uh the majority of my role is teaching mentoring junior engineering talent.

[00:18:13]Uh I'm going to do it in concurrency because it's kind of part of that and I don't re no can right I'm making absolutely zero process here so I'm just going to stick it in utils and worry about it later. The grab bag of all the misery. Um, so this cannot be constra. I'm going to keep this in the Oh, do I even have a CP file for this?

[00:18:44]It could just be its own.

[00:18:47]I changed my mind again. I forgot I had a utils source utils uh resolve symbol.h H.

[00:19:00]Oh, we're hype training.

[00:19:03]Ah, that's the the biggest the biggest part of RE is to find an excuse not to do it.

[00:19:17]Uh, okay. So, we're going to do we're gonna do it can't be constraining Jesus become a member of the channel.

[00:19:31]Thank you very much. Very much appreciated. There's a there's a new video you can go and watch now.

[00:19:37]Okay. Auto uh resolve symbols. Okay. Uh and this is going to return it's not return void but I need to bring in some headers first.

[00:19:47]Let's let's go with span. Let's go with vector. Let's go with string.

[00:19:59]Very kind of you. Thank you very much.

[00:20:01]Right. So, we're going to take in a sto span of a cons void star addresses.

[00:20:10]No, no, no, no, no. call stack because again like it's it's an implementation detail that I'm storing it in an in place vector. As long as it's in a continuous array, I don't actually care. So we should be able to take a I'm I really hope in C++ 26 stood in place vector will implicitly convert to a span. And I really hope that span covers just the bits you've pushed in it. But I guess we'll find out because I've never done it before.

[00:20:42]Yeah, if Nightbot is causing you grief, I apologize.

[00:20:49]I should probably I should probably rain that in. Well, I want to use Firebot, but it's just not ready for not ready for prime time yet. Uh, what am I doing?

[00:20:56]I don't I don't want I only want to I only want to declare it, not define it.

[00:21:00]Right. And this is going to return a stood vector uh stood string. Okay, this seems this seems like a reasonable interface. Okay, give me given some number of const void pointers, please resolve them all to a string.

[00:21:20]Yes, it should take in a range and um in place vector will be a range and I guess the range will be between the stuff that you've only inserted into it. So, we'll see. Okay, that was easy. I got actually do it now. uh e E source utils resolve symbols cppin include utils resolve symbols name space stick that in there right uh you resolve symbols going to want these these uh so good I included them. So good I included them twice. Uh okay. And then e source util cm make uh m n o pqrs.

[00:22:25]Okay. We're ready to do some coding now.

[00:22:27]Whoa. Gifted six tier one subs. Thank you very much. How very kind of you. an anonymous gifter. Well, isn't isn't your name very apt?

[00:22:39]Uh I I have like unfortunately I don't have like the time to look as at modern systems languages as much as I'd like.

[00:22:47]So my opinion on like Rust and Zig and Odin is is basically I don't know them.

[00:22:53]I know there's a lot of people in the Discord who like Zigg and I think people are doing good stuff with it. My understanding it's kind of like a a more modern C. It doesn't sound like something I would personally use because I want RAI. I want exceptions. I want basically everything that C++ gives me.

[00:23:10]That doesn't mean doesn't work for you.

[00:23:16]Right.

[00:23:19]Right. I've now got to do some Windows misery. So, let's let's look at some docs because it's always easier to look at the docs than it is to actually do any code. Uh sim initialize.

[00:23:28]I'm I'm pretty sure there's some Windows horror in this. Like some some real Windows goodies.

[00:23:38]AC still been here for 12 21 months.

[00:23:44]Okay. Initializes the symbol handler for the process. So, Windows can kind of smash into your process all the goubbins it needs for resolving symbols. Okay.

[00:23:53]And you have to call this function. I'm pretty sure you call it once kind of per process, but you can call it multiple times for the process because look at look at all this look at all this gubbins here.

[00:24:07]A handle that identifies the caller. So you can have multiple symbol resolvers in your process. Okay, but you have to register with it a handle that kind of uniquely identifies it. Value should be unique and non-zero. But it need not be a process handle. Like literally any any handle, mate. Any handle. This is very strange. Not I've never seen any other Windows code kind of do this. However, if you do use a process handle, be sure to use the correct handle. If the application is a debugger, use the process handle of the process being debugged. We are not a debugger. Don't worry about that. Do not use the handle returned by get current process. The handle use must be unique to avoid sharing a session with another component and using what what is this doing internally? Right? can have unexpected results when multiple are attempting to use debug help to inspect the current. I think what happens is that this initializes a bunch of state and is keyed right it's probably some sort of hash table or something on on this. I think that's what it's doing. So if you give it multiple handles you will not necessarily initialize a new one. You just potentially get someone else's. And again like you might be thinking well who the hell is going to be calling this right? But if I attach a debugger to this process, like it could very well be doing this. So we do have to kind of be a little bit careful.

[00:25:31]Uh and then like cannot be null. Okay.

[00:25:35]And then user search path blah don't care. I think that be null. And then f invade process. Ah now isn't that a great argument? If this value is true, enumerates the loaded modules for the process and effectively calls the sim load module 64 for each module. So this we want this to be true because this will load all the symbols for all the loaded modules.

[00:26:00]So sim function is used to initialize the symbol handler process in the context of symbol handler. It's a convenient object to use when collecting symbol information. That's what we want. We just want a convenient object for collecting symbol information. Like I don't know which more you want from me.

[00:26:12]Usually symbol handers used by debuggers and other tools that need low symbols for a process being debugged. So normally you're on the outside looking in, right? And you be like smash this into that process and then let me kind of like extract the symbols from it. But we are within the process cuz we're doing something a little bit strange.

[00:26:25]Uh must be the same value passed to all other symbol handle functions. So again like we have to there is a kind of a little bit state here because we will have to whatever handle we register with it we will have to then pass to all other future functions. So maybe it ends up being a class. I don't know. I don't know. Also I don't know how thread safe this is. When you finish using simple table call function request for passes process should not pull it again unless it so a process that calls it should you simply need to read module listah all debug functions such a single thread it therefore calls from more than one thread to this function will likely result in unexpected behavior or memory corruption. Right. So we're going to need to mutex the [ __ ] out of this.

[00:27:10]Thanks for the streams. No worries. I'm glad I'm glad I'm providing some value.

[00:27:15]To avoid this course only when your process starts and simply only when your press ends. So this sounds like a static to me.

[00:27:23]And there's a unic code version if you want it as right.

[00:27:28]There's a lot there's an awful lot to unpack there just for like getting this goubbins going. So let's wait.

[00:27:39]All right.

[00:27:49]It's true. And you could And you know what? And I think you're totally right.

[00:27:54]Like as long as it's not common, it's normally fine.

[00:27:59]Oh, mate. Win sock.

[00:28:01]Now, there's going to be a future future change not for this project, but future.

[00:28:05]Make sure you get your headers in the right order because Windsoft does not like it if your uh hash included headers are in the wrong order.

[00:28:15]Right, let's do hash include utils auto release. Okay.

[00:28:36]I'm going to probably want then auto auto tricky tricky tricky. Okay, let's do let's do something auto sim handle equals UFPS auto release handle void pointer. Right. And then we can put in the null handle and then auto h uh was it sim clean sim cleaner. All right. I'm going to need a header file for this. I need the debug the debug help one. Probably going to need Windows as well. Okay. Right. You have to bear with me for this cuz I'm going to have to I as with most my streams didn't really plan a lot before. I just had a hope and a dream and a good idea.

[00:29:50]So, you will have to watch me figure this out live. Uh, debug help. Okay, fine. So, this is going to be sim cleanup H.

[00:30:11]static G. Okay.

[00:30:28]So that's probably going to be something like what we want. So static we're going to going to call this once on do I want to use an immediately invoke lambda to make sure this is definitely definitely initialized decel type on void star type of what void star is already a Uh, I think I might actually want something like that. Right. Make sure this gets because then I can do some logic in here.

[00:31:35]I don't know why it doesn't like the void star. Definitely done this before, right?

[00:31:44]Uh oh, it's got Oh, it's a value, not a type. Sorry.

[00:31:54]Yes, you're right.

[00:31:57]Uh, have I forgot to All right.

[00:32:01]Right. It's a little bit of a little bit of token soup at the moment. Yes. Yes.

[00:32:05]Yes. You're right. Sorry. Right. Okay.

[00:32:07]So, what I want to do here is I basically want to do I'm going to do what it told me not to do, but then I'm going to fix it. So const auto auto h equals get current process right that will give me an handle uh and then I can do hang. Uh, so that's going to be H, right? So I'm going to do Sim clean up H, right?

[00:33:16]But before this I need to do if uh what's the return value of this? It returns a big uppercase bool. So if the function receives the return value is true return value is false. Okay. So if uh hang there in the end we will get there in the end. Utils error error. Right, let's do insure uh sim initial bit worrying I can't find it but we'll figure that out in a minute. H uh null pointer and then true uh ensure equals true failed to initialize sim sim handler sim symbol handler.

[00:34:09]Okay.

[00:34:14]So that we get there then we'll construct that. That's fine.

[00:34:25]But before that we need to do I think we're going to need to do so that is use current process handle right um before this we need to do cons auto h equals let's do duplicate handle right oh my goodness that did not there's that many flags to duplicate a handle cuz that will give us a new a unique handle, right? Which is definitely not the current process. So that's fine. Uh right. So in H source process handle in H source handle suppose those could be the same target process handle a handle to the process that receive the duplicate there LP. Oh, was that an outpoint? No target process. I mean, they can be the same. Yeah. Right.

[00:35:22]Desired access. Do I care the access request handle duplicate same access? Let's do that.

[00:35:31]Uh, inherited handle by this handle inheritable tr inherited by new process. I don't think I care.

[00:35:38]Probably made that false. Don't really want anyone getting hold of this so they don't have to. And then this occurs regardless of any duplicate.

[00:35:46]Close source. Closes the source. No, don't don't do that.

[00:35:51]Right.

[00:35:54]Let's give this a go. We're going to do current process handle. Current process handle.

[00:36:02]Current process handle. All sounds good to me. Uh, and then we're going to do out a pointer to a variable that receives handle. Handle is handle but does not return.

[00:36:17]What is this?

[00:36:20]If the target handle, so the out parameter is null, the function duplicates handle, but it does not return it to you. And that is fine. This behavior exists only for backwards compatibility with previous versions of function. You should not use this feature as you will lose system resources. I'm glad that's another foot gun waiting for us.

[00:36:39]I guess current process does return a magical value which is just null, right?

[00:36:44]I feel whilst you are probably technically correct, I feel this semantically kind of shows what I'm trying to do better.

[00:36:55]So I'm going to leave it. But I do understand your point. Okay. So this is that. So I've had to actually this is what does this return?

[00:37:01]The function. So if this function return succeeds, it returns true. If this function succeeds, it returns a nonzero value. Those are different.

[00:37:20]That should be as well.

[00:37:24]Okay. So, we're going to do auto h equals let's do handle, right? Then this will be H. And then desired access is duplicate.

[00:37:48]Same access.

[00:37:50]Do I need a different header for these?

[00:37:57]Yep. Winds die should be fine for us.

[00:38:00]Okay. Right. Duplicate. Close.

[00:38:03]uh terminal h target process handle that's fine desired access inherit handle false and then DW options I think I want that to be a big fat zero and then that ensure that is not equal to zero failed to duplicate handle Uh, pretty sure that's pretty sure that's right.

[00:38:41]Oh, yes. Yeah. Yeah. No, I'm I'm I'm on it. I'm on it. We won't forget to close.

[00:38:45]Okay. So, if that's fine, and then I guess the issue now is that if this throws, we will technically leak this handle. So, I need to get this.

[00:38:56]I've got like a weird double state thing now, haven't I?

[00:39:04]Uh, we immediately stick that in there and then do close handle H.

[00:39:28]And then we can initialize H.

[00:39:35]I guess the only issue here is that if this throws, then this will get cleaned up and it will technically kind of clean up something that's not been. But I think we're out of luck anyway at that point.

[00:39:44]So I'm not too fussed.

[00:39:51]I think that might be what's that extra at home.

[00:40:07]Yeah, exactly. What do we think? We think you think that might give us the handle that we can then use and then that will automatically get initialized for us during static initialization.

[00:40:18]Uh so then we're kind of ready to go, right? Because then we use uh Uh, Sim get name.

[00:40:36]What's the What's the magical rune I want?

[00:40:42]Uh, Sim from Adra. That's That's the badger, obviously. All right.

[00:40:52]All right. Now, now we're getting there.

[00:40:57]Uh, yes, basically there's no there's no OpenG on 4.6, but it's perfectly good enough for for our needs. Okay. Retrieve symbol information for the specified address. This sounds like what we want, right?

[00:41:10]There's and there's absolute Windows horror in here as well. So, just just just just you wait. A handle to a process. The handle must have been previously passed to the SIM initialized function. So, this is going to be our global handle, right? a duplicated handle that's all been initialized at the wizoo right the address for which the symbol symbol should be located the address does not does not have to be on a symbol boundary but it will be because the way we unwound manually unwound the stack so that's fine so we just this is where we slap in the void pointer right displacement the displacement from the beginning of the symbol or zero we can leave that zero I don't think we need to care about that we can I think we might have to get it doesn't say whether it can be null or not which is always or a pointer to an integer And it says it can be zero. Does that mean it has to be a valid pointer pointing to zero or the pointer itself can be zero? That is I'm going to assume it has to be a valid pointer because it normally does say null. But you can there is an ambiguity there. Okay. A pointer to a symbol info structure that provides information about the symbol. Okay. So this is what we're going to get out. The symbol name is variable. The symbol name is variable length. Therefore, this buffer must be large enough to hold the name stored at the end of symbol infrastructure. Be sure to set the max name L member to the number of bytes reserved for the name.

[00:42:23]Of course, this is a C API, right? There is no stood vector, right? It's going to give you a strct. Okay?

[00:42:33]And that strct has to be able to store a variable length string, which kind of sounds impossible, right? But if we look at the bottom here, it's a one byte char.

[00:42:49]So what it means is you overallocate and just tuck a little bit of extra space a little bit of extra space on the end and then this because this this will decay to a pointer right and this give you the pointer of a variable size array of max name len right now quiz time size of strct which is a very common Windows indium right the first member of a strct is size right which you have to manually fill it in yes so yeah we've got buffer overflow to go with memory leak so it's it's absolutely fine this is this is this is pretty this is not uncommon Windows programming yeah I mean there really is no other choice with the C API right you just have to you have to It either allocates for you and then you have to release it or you just kind of give it the space. I'd rather it was a out point, a pointer and then a size rather than just kind of tacking on the end of a strct cuz then we're going to have to do some weird almost certainly going to have to reinterpret cast which is going to technically be UB but until we get start lifetime ads but there's not really a lot I can do about that.

[00:44:06]Well, what do you think? Take take a guess.

[00:44:10]The size of the strct. Does this include the extra space you've put on it or not?

[00:44:27]Correct. It does not.

[00:44:29]This has to be size of the strct, right?

[00:44:32]Size of the structure this member must be must be set to size of info. Right?

[00:44:37]Note the total size of the data is the size of the strct plus the max name length minus one * the size of tar. The reason to subtract one is that the first character of the name is already accounted for. So it already knows there's at least one character there. So just just just make sure you take off one.

[00:44:57]This is just some of my favorite like windos documentation. this. It's just like it's it's just like so we've now got a memory leak, a buffer overflow, and an off by one error all just like waiting for us. Just just just waiting.

[00:45:19]All right. And I think the rest of this is kind of I don't think I care about this. All gets filled in, right? So all we have to do is make sure we set the max name Len and then count. What's max name Len going to be?

[00:45:30]The size of the name buffer in characters. If this meu the name right also like and like like and I think this is very important to to point out none of this has mentioned null terminating characters apart from like I've just noticed it here the length of name in characters not including the null terminating character so only right down here don't worry about the null terminator so we will get back a basically We'll get back a not guaranteed null terminated string.

[00:46:15]Yes. Right. Let's let's let's write some of this. Let's let's let's begin our let's begin our journey. Okay. So auto symbols equals sto vector sto string return symbols. So far so good.

[00:46:47]Well, no, I don't think it is not terminated. I think that's the point, right? Is is any not saying that it's I don't think anywhere uh it just says the length of naming characters not I see what you mean the length of the naming character is not including the null terminating character which implies the null terminating character is there right yeah you're right I think I see what you mean now I missed that just going to give it like a thousand characters and not worry about it.

[00:47:30]We just give it a we give it a big old string.

[00:47:38]You're right. Okay, fine. So, right, we just return that, mate. I just couldn't resolve them. Also, it did say something about mangling in there. So, I think we might have to handle name mangling separately. I think you have to set some magical flag options somewhere.

[00:48:03]Okay. Right. So, let's let's call this.

[00:48:07]So, let's do mate.

[00:48:12]Is it Is this the most horrific range we're ever going to write?

[00:48:22]I think I think it is. I think I think it can only be a horrific range because because you know what? You know what I need on top of all this like all these sharp edges around memory leaks and memory issues? What I what what really complements that, right? What the perfect pairing for that is is really long template errors.

[00:48:52]I'm going to try and do it all as one.

[00:48:55]Right. return.

[00:48:57]Cool stack third views transform con uh let's do auto address. I suppose actually I should do e because it's consistent with everything else I've written even though it's not actually particularly good code.

[00:49:20]stood ranges to stood vector.

[00:49:30]Hello.

[00:49:37]Uh have I Oh yeah, failed at the first hurdle. Thank you. Right. Okay. So, this is basically it. Right. It's just and then like this implementation deliberately left blank.

[00:49:51]The implementation is trivial and left up to the reader. Okay. So here we're going to get the address. So we're going to need to do I probably don't need to. I suppose it's going to be stack allocated, isn't it?

[00:50:13]So, let's do auto symbol buffer equals stored array of stored bite size of What was it? It's going to need to be at least as big as a symbol info, right? Plus like what? Like 200 256.

[00:50:55]How big how big do we think a symbol is going to be? 500. Should we do 512?

[00:50:58]Treat ourselves. Stack memory.

[00:51:12]Okay, so we got the symbol buffer, right?

[00:51:15]We now need to I really is the temporary I'm trying to think about the materialization rules cuz I could just reinterpret cast that back but it does mean I would have had pointed to an object that's not been initialized and even in the C++ 23 changes I think that's still technically you be everyone start But I'm as uh can I do auto symbol equals new uh symbol buffer uh symbol info.

[00:52:16]Can I placement new construct my symbol info into that buffer? I think I think I can. I think that's legit.

[00:52:23]Of course, I technically be accessing stuff out out of bounds at the end, which does violate provenence, but like there's only there's going to be UB in here somewhere, right?

[00:52:37]At least now I have a legit symbol. So, I should be able to do symbol dot uh no autocomplete. It's given up.

[00:52:49]Uh size of I'm not convinced. So rare I do placement new. Is that is that the right runes up?

[00:53:12]I can't remember if it goes new placement arg type initializer. Yeah, that's fine.

[00:53:28]3 4 provides addition arguments to the allocation function see placement new new buff t fine and that should zero initialize it right yes you're right that's a good point I guess if I mem copy it out of here into a string then that will not violate the provenence of this pointer which is technically only valid up until the end of this object.

[00:54:01]Okay. So then symbol uh hang where we at? We're now max name len max name len.

[00:54:15]Is this not the name of the buffer in characters?

[00:54:20]the size of the name buffer. So the size of the buffer is 512, right? This is already already don't care if it's magic number. So let's do static conra uh max uh auto max string length equals 512 u. All right. Okay.

[00:54:41]Let's do max string length.

[00:54:47]Uh this is a max strong length and the size of the name buffer.

[00:55:04][ __ ] it is [ __ ] sake.

[00:55:11]Yes.

[00:55:13]Or we could subtract one from here. I don't know which I don't know what's better.

[00:55:19]Ah, thanks.

[00:55:23]I kind of hate both of them.

[00:55:26]Also, I'm absolutely just assuming that tar is size of char size is is one plus one for the bite.

[00:55:37]Well, I'm not guaranteed. I'm still not convinced I'm going to get an all byes.

[00:55:41]So, I'm just going to extract the actual length I get out of it. Right. Let's call this horror and see if let's see see what happens.

[00:55:48]So we should now be able to do uh ensure do I swear if I can't resolve it or do I just keep going and do my just do my darnest.

[00:56:10]If the function sees the function fails, it just says that Oh, also I've got to remember that I've got to put in some locks as well at some point.

[00:56:19]I I want to see if it's working or not.

[00:56:20]So, I think what I'm going to do is I'm going to I'm going to throw, right? And then if it's a total disaster and some things can't be resolved, then we'll go back and retrofit it. But I think I should start off assuming assuming failure and seeking success, right? Sim from Adra. All right. And this is going to be my G. Oh, Clang D is absolutely just given up. It's that I'm not doing any of What are you doing? So, what are you doing, mate? None of this. Uh G Sim handle.

[00:56:58]Uh and then E. I might need to reinterpret retin reinate cast dword 64 like a really big double word not just a regular double word but a really big one e uh I'm assume I'm assume null pointer it does say out and optional and Then oh no it's a pointer symbol right all right and then return if the function succeeds the return value is true so if true uh fail to resolve symbol e It's actually not going be too bad arrange this I don't think.

[00:58:08]And then I'm just going to do right.

[00:58:09]Right. So assuming that passed, I can then do return stood string of right uh symbol buffer dot data plus size of symbol info minus one minus one plus symbol name length name len. Yeah.

[00:59:13]Just have to stare at this a bit, convince myself it's right.

[00:59:25]Getting Charles back. You can't just put that in the stood string. Why isn't that There's no way Charles go. Don't they go in a stood stream?

[00:59:36]because it would have the C API which can do whatever it wants because it's not my API and it can do whatever the Windows can do whatever UB it wants internally is going to blow past this pointer and fill this array up right no I don't think it's with Charles I think it's no no if you look here it's definitely it's definitely Charles right so I'm going to say construct me a string between two iterators pointer is iterator iterator. The first iterator being the start of this buffer plus the size of the whole the symbol thing minus one because it's one char at the end.

[01:00:10]And then do the same thing there, but also then to the end of the name.

[01:00:21]Yeah, I think I'm not not I think I think I'm going to get Well, we'll soon find out cuz it's sort of if it's sort of bloody no bites then I got it wrong.

[01:00:28]Um what do you think? Like it's actually not too bad.

[01:00:37]Yeah, I don't think it's which I think I think we're fine.

[01:00:47]Oh, hang on. Wait a minute. Also, right.

[01:00:49]Okay, fine. And then we need to do hash include utils. This is another reason why uh we need to do this separately is static uh auto glock equals UFPS lock.

[01:01:16]Uh, and then we're going to do con auto lock equals third go lock glo.

[01:01:36]Okay.

[01:01:50]So this is now thread safe. It is hopefully memory safe. Question mark.

[01:01:56]And it's definitely kind of type safe.

[01:01:58]We're going to get out of extra string.

[01:01:59]I think we just have to just call this in a test and just see what happens.

[01:02:02]We're already an hour in.

[01:02:06]Right.

[01:02:08]I'm going to take this code.

[01:02:11]Uh, I guess utils should probably have a separate test, but I don't think I have any utils tests. I don't really I think I think I think I get away with just testing this here. Right.

[01:02:30]In fact, let's just slap it in here because it's all part of the same test, right?

[01:02:38]uh utils.

[01:02:47]So, we're going to get the stack track.

[01:02:49]So, const auto auto. Let's just check we got them first. Then we're going to do con auto symbol sim balls equals uh UFPS resolve symbols stack trace.

[01:03:12]Interesting.

[01:03:25]Now, next video I talk a lot about UCS2 and the history of Windows and UTF6 and UTF8. So, that's that'll be exciting for you.

[01:03:35]Uh yeah, you can convert between there's the multi to wide bite whatever it is Windows function which let you in fact actually We've already done this before, right?

[01:03:53]Uh spring.

[01:03:57]I've got in here somewhere unless I don't need it anymore. Is it in text utils? Yeah, text widen. Here you go. You can use multibite to wide char multib. And then this is the classic double call. Get the size and then convert it. So, we do have this. If we do need to convert it, then I've already got the the the util functions for this.

[01:04:26]Wasn't it nearly standardized? Wasn't it like, no, I think I don't think it was I think it was I think there might have been a paper for it. Uh oh no there's there's this isn't there there's this in it there's wing convert right uh performance conversion between byte string and wstring the standard pass somewhere in here it basically says don't use it uh I mean it's deprecated now I I think on Windows the implementation of this always resulted in a memory leak which is why they ended up removing it because no one used it everyone just used the platform one there's definitely a reason why this got deprecated did.

[01:05:17]Uh, but yeah, all this was removed and stuff, I'm pretty sure. And I don't think they could change it cuz I think it would break break an ABI or something.

[01:05:30]Right. Let's just see if this builds cuz I got a horrible feeling it's not going to.

[01:05:42]Oh, couldn't convert. blah blah blah blah blah blah blah blah. Don't care about you. We'll get to you later.

[01:05:49]Oh, I got a template error here cuz you can see you can see it trying loads of different types which normally means uh template error. Oh my goodness.

[01:05:57]This I'm going to roll off the end of T-Max in a minute.

[01:06:05]I've got a template error and a header file somewhere.

[01:06:13]Did I? Okay, because this is I'm going have to fix. Tell me where this error is, please, and I will fix it cuz I cannot stand looking at this any longer.

[01:06:24]Where in my code?

[01:06:31]Just give me give me a line in my code.

[01:06:33]Just tell me just tell me tell me where. Tell me where my code.

[01:06:52]I just do not care for any of this.

[01:07:04]I think right. Should we just see if we can see see there ourselves?

[01:07:08]Uh symbol result symbol.

[01:07:16]Oh yes.

[01:07:20]Good spot.

[01:07:41]Resolve symbol. Here we go. Substitution F. Yes, I can tell it's substitution f.

[01:07:46]Can you tell me where there 14 14 candles right type void for anonymous okay it doesn't like this oh mate you got to return it haven't Right.

[01:08:35]Still doesn't like this.

[01:08:38]Hi word. Hello I got RAM. I got RAM spare. Mate, you also have an error in the file you crawl resolve symbol from.

[01:08:57]I don't know why it doesn't like this.

[01:09:08]Anyone see what I've done wrong here?

[01:09:16]Actually, I suppose let's just So, it likes the function call.

[01:09:30]Have I got a type issue somewhere?

[01:09:32]What's duplicate handle?

[01:09:43]returns a ball. Okay.

[01:09:47]But then it says the return value is non zero obviously.

[01:09:58]Oh do I bet you need windows winds first otherwise it's not going to understand any of the types.

[01:10:08]Classic. There we go.

[01:10:12]That's a little bit better.

[01:10:15]At least now I feel like I can kind of uh still got all this goubbins.

[01:10:21]Have I gone past?

[01:10:27]Uh, okay. Fine. Maybe not. Uh, what is it trying to do? Resolve symbols ranges operator. It doesn't like my chain here.

[01:10:39]Resolves 81. Oh, some of my code is hell novel.

[01:10:46]It still doesn't like this. I'm just going to comment that out for a second. I just want to just see if I can kind of get through some of these error messages to actually useful.

[01:11:04]So, what was on 81?

[01:11:17]I mean, it shouldn't matter in this translation unit, right?

[01:11:25]Uh, and I don't think I've got it included in the header file. No.

[01:11:38]Let's see if we can.

[01:12:05]It's almost like I What the hell is that?

[01:12:12]That's not good.

[01:12:14]Uh wind leaning mean should be part of the cake.

[01:12:27]Uh interesting. It's not. I don't think it's going to make a difference. I mean, I'm have to put it in there anyway. That should be part of the cmake.

[01:12:48]That's no minmax. Yes. Well done. Okay.

[01:12:54]Could not convert stack trace from con to stood span. Interesting. I wonder why that is cuz it definitely should be convoid.

[01:13:08]And then you got all this horror as well. I really don't there's some include misery going on here cuz look none of this. Have I just formulated this wrong? Is my is my Oh.

[01:13:45]Okay, now I've got all the unused variables. Okay, fine. I think it might have been that and the rest of this I wonder if the rest of these now stem from the fact that it doesn't like passing in this range.

[01:14:20]Let's go to the tests.

[01:14:23]Okay. And let's do the span const void star.

[01:14:36]It's just with all these like when you write a bunch of code and then go compiling all this. You just got to kind of peel back it bit by bit. Come on.

[01:14:45]Right. What are you What are you doing?

[01:14:58]I don't understand why it doesn't like my pipe.

[01:15:03]It definitely doesn't like it doesn't like this. Like what? What have I I apologize. I'm being really stupid here.

[01:15:25]Take the call stack.

[01:15:33]Yeah.

[01:15:36]Yeah. But it needs a pipe, right?

[01:15:37]Because then I need to pipe it into stood ranges to stood vector, right?

[01:15:40]That's how I You take span which is in and of itself a range. You transform that range into a range over stood string and then yeah so this this is how you chain them together right call stack goes into stood transform then goes into stood ranges too is it cause the range if it doesn't it shouldn't do because I mean you could try we could try removing the const it'd be really lame because I should be able to bear in mind this is taking it by value right so it doesn't I'm not mutating this in any way cuz I'm just going to a fresh copy of that pointer.

[01:16:22]Uh yeah. So this returns the for the transform, right? So yeah. Yeah. See, so I'm doing that effectively in a loop, right?

[01:16:32]I just I'm missing something really obvious here.

[01:16:42]It shouldn't do. It should be able to deduce it. But we can do the we can give it an explicit return type, right?

[01:16:59]No, because insure throws and throws is allowed to kind of not return on that path, right?

[01:17:09]Oh, here you go. Right, we've got some other errors now. What's this?

[01:17:14]expectable primary expression. Ah, okay.

[01:17:16]Fine. Yes. Yes. Yes. Yes. Yes. Yes. I see it. So, that should be zero initialized.

[01:17:24]Okay. Right. We're starting to unlock some more errors now.

[01:17:36]So, it doesn't like my operator new now, which is fine. It doesn't have to.

[01:17:43]No match function to operate a new.

[01:17:47]Oh, cuz it needs data.

[01:17:58]That's right. We just peel off layer by layer. We'll figure it out bit by bit.

[01:18:01]Okay. Right. Unused variable. Get current process handle. Right. Let's go up and fix this now.

[01:18:14]Uh GCC trunk.

[01:18:21]Okay. Link error. Link. Linker error.

[01:18:24]That's fine. I bet.

[01:18:26]Remember Window 2 DOS is is sometimes helpful. If you go to the bottom, it' be like make sure you link against this.

[01:18:33]And I want one of them.

[01:18:40]Uh, where do I do my linking? Target link libraries. I will have one of those, please.

[01:18:47]Think I might spaced too much.

[01:18:56]Probably private.

[01:19:04]Uh, what?

[01:19:24]Oh, is it case dependent?

[01:19:36]Okay, fine.

[01:19:38]Failed. So, our static it ran and then failed to initialize symbol handler.

[01:19:48]Okay, fine. We're getting somewhere. We are getting somewhere.

[01:20:01]some Windows [ __ ] I'm unaware of.

[01:20:05]What game am I making? Oh, mate. What a horrible What a horrible time to ask that question when I'm on like a really complex side quest at the moment.

[01:20:17]This is This is the game, right? This is the game.

[01:20:25]Oh, is it not going to Oh, it's not there. Oh, interesting.

[01:20:31]What?

[01:20:39]Oh, there. Sorry.

[01:20:53]Uh, yeah. Okay, I'll get there. I'll get angling. I'll get there.

[01:20:59]Uh, Windows error 183. Windows errors.

[01:21:10]Cannot create a file when that file already exists.

[01:21:24]I mean, of course it exists, mate. I'm duplicating here. I What What more do you want from me?

[01:21:35]And it's a handle, not a file.

[01:21:43]What's duplic must have the I mean I assume I have my own process dup handle right duplicated.

[01:22:04]Oh, uh, yes, you may very well be. They passed in zero, right? So, so they've gone d out handle.

[01:22:42]Oh, have I got these the wrong way around? Look at look at they've got duplicate same access then false then zero. I think I I may I may just have I may have just ruined wrong.

[01:23:09]There we go. Right. Right. See if we right now. We got to go back to work out why I formulate tests.

[01:23:20]Thread pool thread tests. Why can I not pass in stack trace?

[01:23:40]Yeah, but that data won't work because it won't be able to convert it to a span because we won't know how big the span is. I could construct a span from data and size, but I feel like this should Well, this is it. I'm just trying to find that out.

[01:24:32]So you should, especially with vector, you can you can add const to the inner type and that's fine, but you can't remove const.

[01:24:46]Yeah, I'm just Yeah. Okay, fine. Well, let's let's knobble this a bit. So, let's just take that off, right? Uh, and then render. Uh, nope. Don't look at that file. Uh like I should I should be explicit with my with my arguments, right? Like I'm not going to mutate here. I should kind of not provide that option.

[01:25:27]Okay. So it is a const issue.

[01:25:29]Interesting.

[01:25:34]And then I threw fail to resolve symbol. Interesting.

[01:25:40]Okay. Okay. Okay.

[01:25:45]I guess let's leave it as non-con at the moment. It's a little bit annoying.

[01:25:55]I have bigger fish to fry though at the moment.

[01:26:06]I don't know if this sets get last error.

[01:26:11]Uh sim from address. Sim from address.

[01:26:16]If the function return retrieve the extended call get last error. Okay, fine. Nice. Tight.

[01:26:35]Oh, mate. Isn't that Isn't that success?

[01:26:42]Isn't Isn't Isn't error code zero success in Windows Land?

[01:26:55]Okay, fine.

[01:26:57]Ah, yes. My my favorite error error success.

[01:27:05]We have indeed failed successfully. Have I from address the function? If the function receives the return value is true. If the function fails, the return value is false. To retrieve extended error information, call get last error.

[01:27:18]Oh, is it doing that thing where it's evaluating it before it's called it?

[01:27:22]We've had this issue before, haven't we?

[01:27:24]Cuz it will evaluate the arguments right to left, which means the other one above probably wasn't that error message.

[01:27:40]I should I should figure out a better way of doing that with insure like it should have something like insur wind windows or something which will capture that for you after it's after it's done it.

[01:28:19]I feel like I could I I could fix that with templates. That's definitely something I can fix with templates.

[01:28:26]Like all my problems and fix with more templates and ranges.

[01:28:34]Yeah, he could stick in a lander. Uh, where? Cuz it's still that the the insure thing doesn't really support that. Anyway, come back to 487.

[01:28:48]487 attempt to access invalid address.

[01:29:05]I guess we would just let that pass through if that I don't understand why that address would no longer be valid.

[01:30:00]the thread closing after the trace. No, because like threads don't Oh, be careful what I say here. Like spinning up a thread doesn't necessarily mean more code is mapped into the process, right? Like when you're spinning up a thread, you're pointing it at uh the the code section or the text section or something that's already mapped in. So, unless there's something weird with like DL like unload, I don't think it would remove that.

[01:30:32]Okay. Right. Well, that obviously passes cuz I don't do anything with it. So, let's go red tests. Let's do displacement being null pointer. So, it just I I think what we need to do is just do console. Uh console too.

[01:31:05]Let's see what it does.

[01:31:15]So we get use thread start and then we get then we get the init thunk which is basically how Windows starts off your thread, right? And then sweep fuckle.

[01:31:28]Um okay, let's check that. So for displacement then well this is what we're doing for the entire stream right so where do we where was the zero thing I was no point to that so we're going to do uh auto displacement equals dword 64 and then we're going to do fn displacement like Oh, I've not set the symbol options yet.

[01:32:10]Good point. Uh I was cuz you can use that to demangle stuff. Let me let me have a quick look. Set uh sim set options. Sim Let's just see if this makes change like difference first. But yes, you might be right. I might have not set all the magical runes up to make this work.

[01:32:31]Okay, fix it.

[01:32:37]Sim set options. So this is another function you can use and this basically says given this selects the option mask.

[01:32:43]So the symbol options zero is fair. So we can have allow absolute symbols allow zero enables user symbols that do not have an address filters out symbols that do not have an address. Auto public case insensitive debug deferred load symb.

[01:33:01]want the fastest most efficient way to use a symbol handler.

[01:33:05]Uh uh that is a good point. I don't know.

[01:33:13]Oh, I think it might be via dwarf, which is going to be fun cuz we don't have the PDB to pull the symbols out of. I hadn't considered that.

[01:33:33]Trixie Trixy Trixie.

[01:33:37]Uh enable code view. Is that an option here?

[01:33:53]No. GC requires uh you have to use the duty to do it.

[01:33:59]Uh I don't think we get PDB if I'm being honest. Uh find build type name uh star PDB.

[01:34:12]No.

[01:34:16]Does code view generate PDB for you?

[01:34:46]Uh let's have a look. Right. Okay. Let's Okay, one thing at a time. Someone said I need to set this option. So, let me just do that quickly.

[01:35:05]So after we've done this, so unname I think is the one that unmangles the names.

[01:35:18]Yeah, this this we could we could very well be at a crossroads here.

[01:35:30]It's fine. We'll sort we'll get it sorted. We'll get it sorted.

[01:35:34]Uh, okay. So, e uh CMake source compile options uh dash e code view test make Uh I'm I have public private ones.

[01:36:16]Uh that's in Oh, I don't want it there.

[01:36:18]I don't want it in I guess there. Let's just see if that does anything.

[01:36:23]So there's no harm having it in both.

[01:36:24]Just see if it does something.

[01:36:27]Oh, here you go. public code. Yeah, that's what I wanted.

[01:36:42]Uh, ice or is that nice?

[01:36:51]Doesn't like that.

[01:36:56]Uh, we could just clean it.

[01:37:16]Quite a lot of the fun stuff I want to do really hinges on this working.

[01:37:30]Yeah, I know, right? Uh, apparently code view should work from GCC 14 onwards and with MinGW according to this rando stack overflow post f overflow post I found.

[01:38:01]No, Clang does not. I'm afraid it does not have reflection. There is the there is the PDB. There is a tool which will extract PDB from the dwarf information.

[01:38:10]I've used it before. Uh written by the D community.

[01:38:16]Um, so it might need uh Yeah. B.

[01:38:32]It sure does not like that.

[01:38:37]This is where I can't now get it back to what I had before.

[01:38:50]Yeah, but because I because I build the world, I can't not have it do as I'd have to separate asset power to a separate library and then static link it. Uh oh, doesn't that happen anyway?

[01:39:04]Yeah. So, I wonder why it was doing it on that. Maybe I do need it as the private flags. Okay, fine. Let's just try on the limb. Let's just try G. What was it called? Uh G code view.

[01:39:16]Yeah, it'll be a little bit annoying, but we'll we'll we'll work it out. We'll work it out.

[01:39:25]No, it's it could be because I'm using trunk and not 16.1.

[01:39:33]This could just be a regression in GCC.

[01:39:39]I could rebuild GCC, but then we'd all have to watch me build that.

[01:39:45]Okay, let's get that building while I have a think.

[01:39:53]Uh, okay. So, it's a regression in GCC then.

[01:40:08]Uh there is the CV2 PD. So there's this right I think it's this converter it's a D code you to PDB files right conversion of dwarf information to PTB file that great thanks Uh, CV2P.

[01:40:52]Do I have it?

[01:40:58]Can I just get a binary of it?

[01:41:04]All right, give me one second.

[01:41:09]is a separate terminal just off the screen. temp. Uh, w get that gtrx.

[01:41:20]Uh, CV CD CV.

[01:41:30]Okay. temp. Save it to PD CV, right? So, let's do temp cv2p cvb2pd 64.xe XC.

[01:42:12]Okay. Okay. Okay.

[01:42:18]So let's do build tests debug unit test.exe XC and then do build test debug unit test PDB can't create file So I using those right. So exe file. Oh am I supposed to do I just give it the ex file? It figures it out.

[01:43:05]Is this is this a red herring?

[01:43:16]Uh I mean I've got a PDB file now.

[01:43:31]MSV program database. I mean, is it That's it.

[01:43:55]That's it. I've done it. I've solved it.

[01:43:56]I've cracked it. I've only I've only got to use a utility written in D to extract out.

[01:44:04]Oh, how the [ __ ] am I going to integrate that into my build system? Oh, make sure you got this rando rando ex do I build it? Do I do I now make having a decompiler a dependency of building my game?

[01:44:23]I reckon I I reckon I W I reckon I use CMake to pull it down and then unpack it and then make it executable.

[01:44:32]I'm going have to think real hard about that one.

[01:44:42]Yeah. But also like if people want to obviously they can't run it with the assets, but if you want to pull this down and build it and play around with it and see how it works, like I want it to be reasonably reproducible on another system. You know what I mean? Without too much horror. Like things should check out and build is my is my view.

[01:45:06]And the only way to build this at the moment is with GCM and GW because there's just no way to do it without I'm I I'm going to make that a off stream side quest because I cannot [ __ ] to do it. Now, what I want to do is have a play around with this. But look at that. Like you can see the the and on these are the lambdas. You can see how they're they're nested as well, which is quite interesting. Um, and they're all demangled as well, right? Thread trampoline. That's nice. That's nice.

[01:45:41]Uh, you're welcome.

[01:45:43]I don't know how entertaining this is at the moment, but uh here we are. Test.

[01:45:50]Right, let's get this commented and I'll think really hard about this later.

[01:46:03]Assert through uh ranges all of uh symbols predicate is uh I want to use the projection for this don't I? So the predicate is auto e return e and then the projection is auto e return not stood ranges empty e.

[01:46:51]Is that confusing enough?

[01:47:12]actual false expected true does that mean? So what I'm trying to do is is I want in a single line to basically say hey can you tell tell can you assert that everything in here is not empty.

[01:47:23]So this should only return true if everything here is true.

[01:47:28]You have you get a projection which allows you to basically do the transform for you.

[01:47:35]What I don't what I can't tell is either the data is wrong and the test is supposed to fail or my test is borked. I think I know which one's most likely to be.

[01:47:46]Checks if predicate returns false.

[01:47:50]Uh, I don't want to use the projection, do I? What a fool. What a fool. What am I doing?

[01:47:57]I can just return const auto auto e and then this can just be that.

[01:48:15]Uh, I don't know if I want to test this actually because is this going to fail on the CI without having to pull out all those symbols and stuff?

[01:48:30]Also, is one of them empty? You know what? I've seen it work. I've seen it work.

[01:48:42]Right. AC, let me let me let me spell it out for you.

[01:48:47]I've written my own stack walker and symbol resolver code for reasons.

[01:48:57]The problem is because I'm compiling with GCCM in GW, I don't have a PDB file. All the debug information is embedded as in the dwarf format directly within the executable alla Linux.

[01:49:08]The Windows APIs for doing the symbol resolution assumes you have a PDB file.

[01:49:17]I can use a third party external tool which will convert the dwarf information into the PDB file.

[01:49:27]That tool is written in D. So if I want to bundle this all up to just work from checkout from main I either need to build that tool in D which means I'm assume you have a decompiler or somehow materialize it out of thin air.

[01:49:50]Interesting.

[01:49:53]I think I'm just going to wait for uh for GC to fix that.

[01:50:00]Right. Okay.

[01:50:03]This feels like a commit.

[01:50:12]It's basically just all of it, isn't it?

[01:50:14]So, okay. Uh, resolve press symbols, right?

[01:50:26]Yes. It's something, right? Okay. That was longer than I expected. What the what we next need to do, I might start just scoping out the next bit. We're probably not have time to do it all now because I am bloody hot, is we need to track this.

[01:50:45]So let's let's let's start doing this, right? So in thread pool, thread pool is kind of like the in to everything, right? Everything should only you should only be constructing threads. Well, you should not be creating threads, right?

[01:50:59]You should use the thread pool. And in fact, I'm going to posit that to create threads after the game started outside the thread pool is effectually not supported and you're kind of on your own within my within my paradigm, my universe. What we want, right, is a secret thread. Don't tell anyone.

[01:51:25]It's just thread and stop tokens. Sweet.

[01:51:26]Let's hope you don't need anything more than that cuz you'll quite the journey if you decide to recreate it.

[01:51:34]Right.

[01:51:35]So, this will this will when the thread pool kicks off create the profile thread. That's a secret thread. You won't know about that sit in the background. And this is what's going to jam the other ones on and off. We can make it configurable. We can make it go through the options or the config or anything like that. So, we could maybe have it off for release builds or maybe on for certain release builds if we want to see what's going on, yada yada yada.

[01:51:56]For now, just spin that up and get on with it. Okay, so let's do and I'm probably going to want I'm not a massive fan of uh not a massive fan of private members, but here we are.

[01:52:44]And here we can do log. log info starting profile thread just useful to see.

[01:52:57]Uh so what I'm going to do is I'm going to at the end of all this right so workers uh profile thread I think it's default constructible now. Oh it's not okay fine.

[01:53:14]Uh, I really want to wait.

[01:53:22]So annoying. Uh, I really want it to be instantiated after the workers, but I guess actually no. I'm going to cheat.

[01:53:30]I'm going to cheat.

[01:53:32]Am I going to cheat? Uh this profile worker.

[01:53:49]Uh oh. I'm going to need a stop token, aren't I?

[01:54:10]Oh, I need a name. That's fine.

[01:54:14]Uh, profile profile thread.

[01:54:22]Uh, and that's going to want uh stop token. I guess I better pass that in there.

[01:54:43]I don't actually I think it's a mistake to stood move this because they are cheap to copy anyway. But I will do it for consistency.

[01:54:54]Uh, okay. Can I not capture this from here? What are you chatting about?

[01:55:10]Oh, profile worker.

[01:55:16]Okay. And then that's fine because then we can do thread pool stop token. Stop token and back to thread pool and then do prof. Nope. No, don't do that. Jesus.

[01:55:35]Stop. Stop. Stop. Help.

[01:55:47]uh use parameter stop token. That's fine because then we're going to do while stop token dot stop requested.

[01:56:01]Keep looking around.

[01:56:05]Uh I'm going to be like uh stood threaded this thread sleep for 500 milliseconds which is gross but I'm pretty sure the threads would have started by then so uh using name place.

[01:56:41]Uh, Luki, thank you very much. How very kind of you. And yes, I will definitely have a drink after this.

[01:56:48]So, thread does do all this for you.

[01:56:50]Absolutely right. But what it doesn't do is allow me to do funky stuff like suspend the thread and grab the stack trace. You can technically do it with native handle.

[01:56:59]However, because I've basically made my life extra difficult by using miningw and GCC, which will use the Pthread implementation of threads, um use the Windows implementation of Pthreads, which means you'll get back a Pread handle, not a Windows handle. And I'm sure you can then resolve it to that. But by that point, I was like, let's just write ourselves. It's a bit of fun as well. Bit of fun.

[01:57:24]Uh okay. And then so what this is going to do, this is going to do for con uh for auto worker in workers because now what I can do is basically do I'm going to do worker dot stack trace. Right? I'm just going to discard it, but I want actually go through the motions of calling it.

[01:58:02]And then we're going to do how often what's a good frequency for like profiling every millisecond.

[01:58:16]5 milliseconds.

[01:58:21]This is Yes. This is the This is This is what the last two streams have been building up to.

[01:58:26]Uh cuz AI would do this pretty quickly and that'd be really boring cuz I actually enjoy programming and this is fun to me. Keeps my brain sharp.

[01:58:43]Well, for a profile on the nanoc scale maybe I don't know like if this is this is going to be this is the balancing act, right? Because you have to decide between like the more frequently you sample the more accurate results will be but the slower the more impact you will have on the runtime of the program 10 kHz. So what so we're thinking micros what's what's 10 kHz in so that's 10 so one over 10,000.

[01:59:17]So is that 10 is that 10 micros 1 millies?

[01:59:27]1 mill is 10 hertz. Okay, let's give it a go. Let's do one. Let's go for broke.

[01:59:32]Obviously, this is this is this is not going to be totally accurate because I'm not accounting for the time consumed in doing this.

[01:59:48]Uh, does sleep before have that? I thought it'd be whatever the implementation is cuz I can I mean I can always just do I guess sleep, right?

[01:59:59]I wonder if it's better I wonder if it's better to actually go through the Windows API.

[02:00:20]Um, I think we'll just go for this first and just see if we can get results out, right? And then we can always like tweak them as we go along. But yeah, and I just wanted I just wanted to see if this like absolutely BS the the game right time being period end for this. Uh I will have to look uh time period begin.

[02:00:56]I'm not sure I'm familiar with that.

[02:01:00]Regress a minimum resolution for periodic timers. Oh, okay. Fine. Minimum timer is milliseconds for that.

[02:01:07]call time function after she finishes.

[02:01:09]Okay, fine. We can look at that later.

[02:01:10]That sounds that sounds like an attraction I need to build. Um, but for now, let's just see if this all goes together. Uh, so the last thing I need to do is wire up this get I need to be able to where do I where do I kill everything stopping threads? So, we also want to do also you have to remember as well this doesn't include the main thread.

[02:01:41]So we need to do profiler thread dot request stop and profile after this we will then need to do so the issue here and this is kind of what I was alluding to and the other reason I want to do my own threading implementation I'm not sure I have time to do this now is this needs to also be able to construct from an existing handle. I need to be able to wrap the current the main thread in this so I can then request it to stop.

[02:02:30]So this so this will this this should run if it if it runs this will run pretty smoothly. Uh so there are some drivers for OpenGL that require you to execute off the main thread. So spinning up main into a new thread and then issuing OpenGL calls could have some issues. I think they're quite old and probably not a modern hardware, but it's something I just want to just there's a whole class of problems I'd like to avoid.

[02:03:04]Oh [ __ ] Sorry, I forgot I did make clean. So all the resources have gone. Hang on a second. I rebuild all those and then I can start looking at this threading guns.

[02:03:20]So thread H. So, I this is going to be a little bit gnarly this uh because we're going to want to have a new constructor which takes a existing thread handle.

[02:03:48]Uh, and then handle's going to want to have that handle, but I don't think we we're just going to take a copy of it. So, we don't actually need to do anything with it.

[02:04:08]So, we can just close that.

[02:04:19]Is suspend really necessary? Yeah, because you can't be I don't think you can be call get context on something when it's running.

[02:04:37]Uh there's a docker file which will do all this for you by the way and you can just I have a top level make file which drives it and then you just run the docker file and that's all on git. Um uh okay the issue here is I won't have a stop s this this this basically becomes like a read only thread. Do you know what I mean? As in like a thread view onto something else and I can't interact and I can't so request stop is kind of meaningless. I suppose it doesn't actually matter, right?

[02:05:13]I'm actually okay.

[02:05:22]Maybe let's make run.

[02:05:29]Oh, but I think it be a noop effectively, right? as in like I think we function or not because the thread object would contain a stop token but I would never have passed that stop token into the main thread. So it'll just signal a stop token and then no one will ever listen to it.

[02:05:48]Okay, so it looks the game run seems fine.

[02:05:56]Again, I'm not hammering the main thread on and off at the moment.

[02:06:02]So let's now go to thread pool and have thread main thread.

[02:06:23]So where was that? It was main thread main thread and then get current thread because we're assuming that this is going to be called from the main thread, right? You the main thread initializes the thread pool.

[02:07:00]Let's see see what happens, I guess. So, we don't need to request the don't request the main one to stop, but it's not going to do anything anyway, right?

[02:07:07]And then we'll go down to the profiler and then we'll be like uh here we will do that's the worker ignore me. Uh oh no prof work sorry then here we will do main thread dostack trace.

[02:07:30]Yeah. Sorry. It is the game's looking pretty good. Uh, it's just that we've not really been looking at recently because I've been fuffing around on this like massive side quest, which I am really enjoying, by the way. But I'm having absolute grand time doing this.

[02:07:46]And I hope that if I'm enjoying it, then that means you guys enjoy it. And that seems pretty bloom. Bloom is blooming.

[02:08:02]We've got our pulsing light. Uh, if we go down the hallway, we should have the flickering light.

[02:08:10]There we go.

[02:08:22]Did the bit rate get lowered? I don't know. Yeah, we're still we're still humming along quite nicely.

[02:08:30]So, I'm kind of deadlock.

[02:08:44]Uh, right.

[02:08:48]It's going to be this, isn't it?

[02:08:58]Do I have collision? That is that is that's next. That's [ __ ] fan. It's coming soon, mate. Soon after I finish the side track. So So side quest time.

[02:09:08]We got to wire up the this. So it does after after it's done all this accumulation. It needs to it needs to build out a tracker, right? So we need to produce a hash of each stack trace, right? And then each time we see a stack trace, we can increment that by one. uh per thread as well. So we need to keep a a thread indexed counter or hashmap right of where the hash key is the stack trace and then wire that up into the debug renderer which I absolutely hate because it's the worst bit of code I've written on this entire project. Wire that up into that and then we want to be able to see like in real time what thread is executing what. Right? And at that point I think we can probably leave it there because I think that's kind of good enough to get going.

[02:09:58]We could take this exact code and lift and shift that into the memory profile we've already got so we could see which threads allocating, but I don't think we need to do that yet. But that's future work.

[02:10:08]Then after that, we need to I want to do some super quick be a stream uh three streams later do a quick bit of OpenGL profiling as well using the OpenGL begin timer stuff just for each of the part render passes and then after all that we can do physics. So, it's like three I think I said this whole thing be like three streams. This is the third stream.

[02:10:29]So, there's probably another three streams to go.

[02:10:32]Uh, it's going to be Jolt.

[02:10:40]Uh, no. I don't care about build size.

[02:10:41]Like, I I used to say that storage is cheap. It's not at the moment, but like hard drive space is is is fine. I'm not too worried about that. Look, we'll get there. Right. I got to figure out why this is deadlocking at the moment.

[02:10:54]super quick. Let's just see if we Let's just run this under a debugger and see what's going on.

[02:11:03]We used Jolt in the last project, so I've got some experience with it.

[02:11:10]Look, we'll get there. It's not.

[02:11:15]Uh, it's all about it's about the friends we make on the way.

[02:11:27]All right. So, that's deadlocked. So, if I pause this, so this pause thing, this is under the hood. This is what Windows is doing. Basically, what we did, right, with suspend thread and then the symbol resolution. This is what's going on here. Uh, I mean, obviously it's using GDB, so it's passing the dwarf format, but yada yada yada.

[02:11:43]Um, so it's in the thread destructor auto release.

[02:11:59]Interesting. Wait for single object infinite. Why is it failing to Yeah, I think well because we've overloaded new, right? We can just we can just grab a stack trace every time someone calls new, right?

[02:12:30]I'm not convinced I'm not, to be honest.

[02:12:32]I'm not convinced. I'm not. But I don't really understand why.

[02:12:40]Oh, what a great what a great point.

[02:12:42]What is the name of the thread?

[02:12:46]Profile profiler thread.

[02:12:56]What's me profiler thread doing? Can I Oh, can I set the Right. Hang on. side quest.

[02:13:03]Uh do quest look.

[02:13:20]Don't leave me alone at the top.

[02:13:24]Uh right. Uh so in the once we've constructed this right we should be able to then after this should be able to do I don't really want to do anything at all. This has been set thread name. Oh, does that not exist?

[02:13:58]Has it got some?

[02:14:12]So, I want I should be able to get the the debug to display the thread name.

[02:14:21]set for a description.

[02:14:30]Set thread description uh new thread uh name. Sea straight might build that and just see if that's I'm hoping white. Why? Why would you do that to me?

[02:15:02]Why would you make it wide? I bet there's no I bet there's no asky version of it.

[02:15:08]All this and it won't work. Right. Hang on. Hang on. Hang on. Hang on. Hang on a second.

[02:15:27]Uh, sorry. Text widen. Uh, name Okay. Right. Fine.

[02:15:51]Uh be interested to know if VS Code is smart enough to pick this up. Uh uh it's not. I've already seen it's not.

[02:16:01]What a [ __ ] waste of time that was.

[02:16:03]Apologies. Um, pause. Uh, oh, I know. I've got I mean, that is the one I care about, but why does it not set the names for any of the others?

[02:16:28]Anyway, what's what's this thread doing?

[02:16:37]I'm really worried it's stuck there.

[02:16:39]Have I just gotten unlucky? And this is No, this should be like worker thread whatever.

[02:17:03]That's really concerning that it's stuck there. Can I step over this?

[02:17:12]No, it's it's definitely stuck there.

[02:17:23]Yeah, I wonder if we need to wait.

[02:17:26]I bet we need to wait for the profiler to stop first before we signal the others to finish.

[02:17:37]Uh yeah, now they get their own name there.

[02:17:47]So I don't understand why those are not being set.

[02:17:53]Uh how do you know how to quickly separate your engine, etc. Uh, uh, I don't I can't make up going along, but I'm also not making an engine. I'm making a game. So, I'm kind of happy for the code to be a little bit messy because it just gets us towards making a game. Um, I need like weight.

[02:18:21]I'm not profiling the profiler thread.

[02:18:23]Like that's definitely not at least I don't think I am.

[02:18:30]No.

[02:18:32]Uh right. There's kind of there's basically two solutions to this.

[02:18:38]One I add a join method which blocks and waits for it to finish.

[02:18:43]The other is that I basically make this a unique pointer and then I can force I can just release it or reset it.

[02:18:56]I don't know which is better. I don't really like either of them. Ah, I tell you what actually I can do. I tell you what makes what I think makes slightly more sense is let's make this optional, right? Because we might in the future not have this.

[02:19:23]All right.

[02:19:36]And then we can do request stop. And then we do profiler thread oop thread dot reset which will cause the destructor to be called which will which internally does the join.

[02:20:08]I don't think I'm totally against that.

[02:20:32]Not quite.

[02:20:38]Just reset. Not if this contains value. Destroy that value. is if calling the destructor.

[02:20:50]Yeah. So I don't understand why that doesn't work because that will call the destructor of that and then the thread destructor will call handle destructor which we call wait for single object which we'll call close handle right and then this will have caused stop source and request sorry we'll call request stop which we'll come back around and say this. Do we see this?

[02:21:41]I've already got him going, mate. It's there. I need to change the default color of this. I've just not had a chance yet cuz it's some Windows horror you have to do. Right. So let's break out of that.

[02:21:55]Uh no it should it should look at the stop token right.

[02:22:01]So main thread pool stopping threads.

[02:22:08]I don't get that which is interesting.

[02:22:14]I get this.

[02:22:18]See, look. It should work off this stop token here.

[02:22:26]All right, I have to leave this here and look at this another time because I am my brain is absolutely dead in this heat. So, start of uh profiling loop.

[02:22:45]Uh what we on stream 36 still the githole.

[02:22:59]Yeah, I still think we made quite a lot of pro progress there. I think I will try by Sunday, although I'm not sure if I'll have time to integrate the PDB converter into the build system, but it might just be that for this we just run it manually when we need to. Um, hang on. Let me begin myself. Right.

[02:23:23]Uh, before everyone disappears off, I'm going to see if there's anyone doing any C++ things. Uh, other than that, uh, new video soon. Like real soon. Like like sooner than you might think. Well, maybe not that soon. Um, and second no, no one really C++.

[02:24:07]What a shame.

[02:24:22]Oh, I really hate this tool.

[02:24:25]I really struggling to No, never mind. Right. Anyway, uh what was I saying? Oh, yeah. Um I normally do, so it's available for members now. I normally do members get it 24 hours earlier. However, I uploaded it Tuesday, which meant I was streaming today and I didn't want to release it at the same time as I was streaming cuz I wouldn't really I'd be distracted by it.

[02:24:50]And then I'm actually busy tomorrow. So, Friday, it'll be Friday. New video Friday.

[02:24:59]Uh, no. Stop tokens copyable. It's designed to be cheaply copied. Um, we also stood move in anyway, so it wouldn't have that issue. Um, right, that's all I've got. Thank you very much for keeping me company. Very much appreciated as always. Uh, I'm very grateful that you give up some of your time to watch me flail around with C++.

[02:25:18]Uh, I think we got quite a lot done. I'm quite pleased with the progress. We'll make some more progress next time we stream. Uh, and other than that, see you all and on.