Lexer | Writing a Compiler in C++ | risc-v toolchain | Day 1

Added: 2026-05-15

[00:00:13]All right.

[00:00:16]All right. Let me just test the audio real quick.

[00:00:26]All right. Yeah. I hope everything's good.

[00:00:42]Let's see.

[00:00:48]Yeah, I think everything's looking good here. Uh, let me just finish setting up stuff.

[00:00:54]Music's good. Yeah, I think everything's good.

[00:00:59]Just start.

[00:01:02]One second.

[00:01:28]Yo, what's up, EXO? How you doing, bro?

[00:01:30]Or RNG, is that like your new game? Is that your new game thing that you mentioned?

[00:01:41]Okay, I think the audio is fine.

[00:01:45]Actually, the audio is good, right, for the stream. All right. Uh yeah. So we're going to start making a compiler. Making a good compiler for our Risk 5 emulator.

[00:02:00]So the emulator is already done. We finished that last stream.

[00:02:05]Playing games on my Roku TV. Well, games I remember playing games on the Roku TV.

[00:02:11]Mostly like Angry Birds. I remember Angry Birds was on there. The OG Angry Birds that was on Roku TV. That's what I played.

[00:02:18]back then.

[00:02:20]All right, audio's good. All right, perfect. All right. Okay. Uh, so the emulator is done from last time. Now we got to start on the compiler and the project's on GitHub. I actually made the project more structured. I did CMake. I learned CMake and all that stuff to actually make the build properly. So yeah, it's on GitHub.

[00:02:39]I'll link in the description. Actually, is the link even updated properly? I think it should still redirect to the main one. I think it still should redirect properly.

[00:02:47]So that's fine. All right. Uh yeah, guess we can start by making a file. So we're working on the lecture today. So let's just call it lexer.cp.

[00:03:02]And also the header file elixir.h.

[00:03:12]There's a lot of knockoff games. So mostly I got Pac-Man, Pixel Dash, and Fruit Ninja. Nice. Nice.

[00:03:18]I remember Fruit Ninja.

[00:03:22]Okay.

[00:03:26]Yeah, I guess we could start. Okay.

[00:03:31]So, the main point of the lexer is to basically take these raw lexims, which are the individual words, and parse them into tokens. Actually make them into tokens.

[00:03:45]So that's what we need to do.

[00:03:50]So like for example, if you have like int x= 3, it splits that up into different like seams and then identifies them into the proper tokens that they are. All right. So first we need to get our enumeration here. So basically setting up our keywords basically.

[00:04:13]Uh, I'm going to say call once. That's usually what I do for every header file.

[00:04:21]Yeah, private once.

[00:04:27]And I have this goofy game called Mr. Snake 2. What's What's that about? Mr. Snake 2.

[00:04:35]Yeah. All right. So, we have.h.

[00:04:38]I'll add header files as I need as I go through it.

[00:04:42]Uh, I'm going to close these ones. We don't need these for a long time later. So, yeah, we'll just have these set up like this. All right. So, lexer.h.

[00:04:56]Basically, we need to set our different enum. So, let's just call this like enum tokens essentially or yeah, tokens or token something like that. And then we can set our different type. I'm just going to say token.

[00:05:15]This is only a single one. So let's just call, you know, token.

[00:05:21]And we want to set like our different types of tokens. I know the the book that I was reading did not only like the different keywords but also like literals or any any punctuation that was involved in there.

[00:05:42]So I guess yeah we can kind of just kind of just set it up.

[00:05:48]I guess let's just start with um let's just start with uh let's start with keyboards first.

[00:06:08]So let's think some keywords that we might include in this compiler to the language. Uh well I do plan on Okay, so let's just kind of map it out real quick. Let me just like map it up. So I kind of do want to cover uh like control flow loops uh basic data types.

[00:06:43]Yeah, probably basic data types. What else? Um like logical operators.

[00:07:06]Huh. Um what else? Oh yeah, maybe probably like a probably functions as well. Probably functions. So, I could probably just have these to find um yeah, I can't really think of anything else. So, for now, we'll just define a couple. So, let's just start with some keywords.

[00:07:43]Uh so some keywords might be like if else like else if uh you can't make it like that. See like this. Okay.

[00:08:00]Possibly something like while four probably like print as well.

[00:08:25]Well, for functions, we don't really need to we don't really type function unless it's like yeah, unless some specific language, but for ours, I don't think we need to type function.

[00:08:43]Yeah, let's just um Okay, so that's for control flow loops.

[00:08:55]Uh some other keywords I guess we can say like int float void char uh I'll say double I'm going to just specify now I'm watching Paddington. Isn't Isn't that like that one like raccoon or whatever that something like that? I forgot.

[00:09:40]Bear or something. You don't know that one animated thing.

[00:09:45]Uh this can also be keywords but this can be like control slash uh types as well, right?

[00:10:26]Uh you can say like literals and then have I don't know like string that's like a number itself uh identifier like the identifier of the thing.

[00:11:00]Uh what else?

[00:11:07]I guess uh I don't know what do they call like punctuation uh punctuation slash I just call like single single characters.

[00:11:41]And this would be stuff like like a comma semicolon plus minus like the asterisk star slash modulus operator is a and uh left parenthesis, right parenthesis, left brace, and right brace. Stuff like that. It's very simple. Just a set of these.

[00:12:32]What else am I missing?

[00:12:41]Let's call this one ops or just call it operations. I guess operations are also there as well. Uh so this is kind of it's kind of hard to like separate. This is just for like comments, but I guess people understand anyway. That's fine. I'm just going to bring these over. Then I'm going to call this operations and just move these over.

[00:13:18]All right. Yeah, that's good.

[00:13:34]And we could also say like Uh, exclamation equal sign.

[00:14:14]Make equal equal sign.

[00:14:23]than sign less than sign greater than or equal to less than or equal to.

[00:14:36]Yeah, I think that's good.

[00:14:47]Yeah, I think that's pretty good.

[00:15:15]Did you know there's a developer mode in Roku? But if you use it in TV micro cuz it's not for regular users. What is the developer mode? Like what does that even mean?

[00:15:26]Like the code on it or like just in general for like Roku developers?

[00:15:39]And I guess we also really want we want our file token.

[00:15:53]And I guess that that's pretty good.

[00:15:55]That's we'll probably add some more that I'm not remembering. But I guess this is like a good starting point for the enomes.

[00:16:23]Okay, let's let's keep going.

[00:16:27]So, we did get our enumerations where we type in enumeration.

[00:16:33]need to be able to track errors by telling users where it occurred. Okay.

[00:16:37]Yeah, that's true. Uh what else?

[00:16:43]Yeah. So, this is where we need to actually set the token. So, I think uh I'm going to call this token type cuz it's not exactly a token itself. Token type. Let's call it token type.

[00:16:58]And oops. Um each token is grouped into four different things. So we have the token type as we said earlier. So it would be one of these things. Uh the actual lexim which is like the actual the word that we're using literal value. So this is like any optional value. So you have like I don't know three or like any random string literal and then the line number where you had the token at. And the line number is mainly just for like error handling. So when we get to the error part, error handling part, we can do that. Um, yeah, we can set up the strct right now. So we're going to need a strct that handles that has four of those.

[00:17:45]So now this would be the actual token.

[00:17:48]Okay. So let me see exactly what.

[00:17:51]Okay. So token type, right? So we already have token type. So, we just need to pass in a token type and call that type.

[00:18:04]It gives you access to other things. I don't want to try cuz I don't know what will happen to my TV. So, I might have to search a photo of what it looks like.

[00:18:10]Seems interesting. I've never heard about that. Seems interesting. Exo. All right. So, token type will just be a string. So, just a string and like seam. So that's like the actual word of it. Oh, we need to include string on the top. So we can use that. All right. So did string like same. What else? A literal value. So there's like a couple values that we can use. So uh based on the book, there's like a couple values you could need. Usually what people do for very early compilers is that they pass in they pass in double for any type of integer literal or any any double literal float or anything like that. So we want to we want to pass in a couple things. So we want ST variant. Uh sorry variant not bad stood variant. We need to pass it at the top. That's why.

[00:19:24]All right. So stood variant.

[00:19:28]And we need a couple things. So we can get um so we can get a double value.

[00:19:42]We could do a double value or we could separate them into floats. Uh what people usually do is just spread it out into double and any type of number will just be passed as double. But let me see uh let me just write out the way. So we can either do so we can pass in like this.

[00:20:06]This is essentially just a nil value.

[00:20:12]I'll send you a photo of what it looks like. I don't know if it's real or not.

[00:20:16]Just saying. I'll DM. Be right back. All right. Bet. All right. Bet. See you soon.

[00:20:22]All right. Uh yeah. So no value or we could do like we can do stuff like this. So your float and this will be like our literal value.

[00:20:51]Something like this. I would want to say that this is fine. We could also probably do something like like a stiff variant. Same same thing with the normal state, but we'll pass a double instead and then have string and then have that be.

[00:21:14]Uh, it's just a matter of design choice.

[00:21:16]I guess we can do the first one though.

[00:21:20]Um, well, hold up. I'm going to comment stuff for now. I'll ask Claude actually what should I what is a better design pattern to do there I've seen many ways like that I've seen many different ways where I can do that okay then we have the of course the line number that's just going to be so just call that like in line let's uh let's pass our constructor so let's say to type uh the different types it's going Token type. Oh, not token type. It's going to be token. Token type. T type.

[00:22:14]Uh, it's just going to be the same.

[00:22:23]Lex the variance I'm just going to copy this okay it's going to be literal value and and and we're essentially setting up the constructor here. So, I'm going to say uh type uh Lexi Lex and uh literal literal and then line line. So, this just basically sets up the constructor for how this should look like each time.

[00:23:16]So, it should be like that. Oh, yeah, that's fine.

[00:23:20]Okay, so the constructor's filled and that's basically the strruct we need. So now we have the enumeration for the token type and then we have the specific strct that gets that type and then actually sets up how the token will be.

[00:23:33]Uh yeah, we'll check the design choice so far. Uh I'll check in DMs.

[00:23:43]That's interesting.

[00:23:45]Development application allows developers to test their own applications. Oh, you can like actually add your own applications into here and test it out.

[00:23:54]That's interesting. Pretty interesting.

[00:23:58]Okay. Uh, we'll check. Um, have this.

[00:24:08]I was wondering about varants. I am aware some developers use double to represent any integer any number.

[00:24:33]Uh I'm not exactly sure but uh give me some insight. Just give me some insight.

[00:24:49]Oh yeah. Uh I'm forgetting something.

[00:24:53]Uh for our literal value and also our variance, we want to close the move on them so we can actually copy them.

[00:25:00]We can move them instead of copying them. It'll be much more It'll be much better.

[00:25:07]Uh not the legend. Sorry.

[00:25:14]Oh, wait. No. Yeah. Yeah, it is alexium.

[00:25:17]Sorry. The string is alexium and the variance is literal. Yeah. Yeah.

[00:25:23]Okay. So, that's good. So, now our constructor is properly built.

[00:25:34]Uh oh, he did find an error here. Oh, that's accidentally a lower case. Good catch. I didn't even notice that. Okay.

[00:25:44]Uh yeah. Okay.

[00:25:53]See double maps to that which you haven't implemented.

[00:25:57]Uh that's probably better choice for mine. I'm going to remove double. We haven't done double. I don't want to do double. Maybe in the future. Who knows?

[00:26:06]That means I'd have to go back to the emulator and implement double, which shouldn't be that long, but it's like it's kind of tedious. So, I'm just going to skip that for now cuz I'm already familiar with that whole process. So, I just want to kind of learn some new stuff like this. I'll get back to it somewhat. Maybe though.

[00:26:23]Maybe. Who knows? We'll see. All right.

[00:26:26]So, instruct and yeah. So, I guess we'll keep it like this. Uh Cloud thinks it's a good idea to just keep it like this. Uh so, we don't need this.

[00:26:39]Let's see. He doesn't actually have a LS keyword. It's Elsif Parsons and two separate token.

[00:26:46]Oh, is that the case? Okay.

[00:26:48]I didn't I was actually thinking like when I wrote like uh Elsif here, I didn't know if they combine these two or if it's a separate, but I guess they do combine it. Is that actually like a thing? Like is that okay if it works?

[00:27:27]exactly. Oh, it's not a keyword. Okay. I thought it was just a keyword in general. Like it's just one full keyword.

[00:27:37]else takes one statement else if that happens to be another if to be honest I did not even know this to be honest I did not even know this but I guess that's interesting I guess we don't need else then we'll just keep it as if else while and for probably not going to add like do while or anything right now we'll just keep it like that I don't want over complicate. We're just going to take a subset of C and compile a subset of C.

[00:28:12]That should be that should be fine.

[00:28:17]Uh let's also give him my new strct.

[00:28:23]I wanted to know. I also modified my strct to use semantics for and so we don't need catch. Yeah, I'm aware. That's good.

[00:28:49]What else?

[00:28:55]Uh what else? Uh one small thing should also be taken by the valley so the colors decide to move in.

[00:29:07]Yeah, they are. It's already correct like that. It's already correct.

[00:29:15]Yeah.

[00:29:20]Um, yeah, I guess that's fine.

[00:29:35]Uh, if anything comes up, we'll move on.

[00:29:37]Um, let's do our main file now. I believe that's next. I'm going to do the scanning later to actually set up the scanner class and everything.

[00:29:51]But for now, we can.

[00:29:57]We can kind of just set it up.

[00:30:03]Set up the program itself. All right.

[00:30:05]So, I do have some notes here that I wrote.

[00:30:10]It's probably going to help me get through all this. So, okay. So, setting up our main.

[00:30:28]This is going to be quite interesting to see how we set this up. So, um, yeah, I don't know if we should include Let's see.

[00:30:50]We need to set uh set up some functions in here. So let's see.

[00:30:57]Yeah, so we need to handle a couple things here. So we need command line handling, file reading. We need to set up a ripple and also error codes.

[00:31:06]Okay, we can do that.

[00:31:11]We can do that. So command line handling So we need to uh find a way to handle these separate files itself.

[00:31:33]Okay. I have a feeling we're going to need so many. So let's just do the normal includes we used to do since we're handling files. I believe we're going to need um S3 frame.

[00:31:48]And I believe when I was writing the notes, it was also like um there's also one other include stood o stream. What does that include again? Let me see. Uh stood o stream.

[00:32:06]Let me go check that out again.

[00:32:10]define and header by uh estream. Okay, I believe we also need that. That's what was written in my notes.

[00:32:19]All right, so we have that.

[00:32:22]All right, let's just go through this slowly by looking through my notes here.

[00:32:27]Let's see.

[00:32:32]All right, so all right, so we need to cover these four tasks. command line file rebel and error codes. All right, so we need to know when errors occur. So having a shared state that can flag the error as true. So I guess we just set a we'll just set a global variable here just call it b error equals uh we should set as false for now.

[00:33:02]Uh what else am I not? So yeah, we need to okay handle file. We need to handle files.

[00:33:14]This is kind of similar to how we did in our I believe it was the interpreter CP. I wrote this to actually open a file, read it properly. So yeah, we can kind of do something similar to that in here.

[00:33:33]Um, let's first set up uh since we're reading the individual characters of a source, we the source will actually be the actual like lexium that we're taking the whole source.

[00:33:58]So let's just set up like a little we're not going to actually implement the function but actually just like a floor decoration.

[00:34:06]So basically just running to get that file and we don't uh we don't want to make a full we don't want to make a full copy. So I'm just going to say constant string reference to the source.

[00:34:26]So whatever like source the person passes in into whatever it is the command line or the ripple or anything that should be there.

[00:34:45]Uh we need to set up a couple of couple of things. So okay first handling files.

[00:34:52]So we want to actually have a way to run these files.

[00:34:57]So, let's call them file.

[00:35:02]And the path is usually just the stitch string. Uh, let me see how I did that in here. Believe it's just a constant string reference. Yeah. And then the file name. So, we're probably just going to do the same thing here.

[00:35:15]So constant string reference to uh five.

[00:35:27]Um okay.

[00:35:30]So we need to first read the file, right? So we're going to be using that if string that if stream uh can use the if stream to read that file.

[00:35:44]Yes. So we want to we want to read that file uh going to call this file name or yeah I think that's fine. Well it's not really a file name.

[00:36:10]path is maybe a better name for that.

[00:36:18]So we're opening the file from the file path that we got and we need to also check for errors check for check to make sure the file opened properly. So if not file, so if it didn't open properly, uh we can just pass in like an error handling thing here. So just say canot file and pass in the file name. So file path the file path.

[00:36:58]So the specific file path that was entered in itself and then we can give like an exit code as well.

[00:37:16]Okay, that's good.

[00:37:20]Uh what else is there? So handle the files. We also need to read the whole file and that's where uh stood old string stream comes from. Okay. uh like a string builds are stored up in a string. You need to copy the entire content of the file into the buffer.

[00:37:35]Then read then return that build string.

[00:37:37]Let's also check for errors.

[00:37:45]Okay, so we had that first string. So instead of o string stream let's see.

[00:38:13]I believe we should use this It's actually construct that value itself.

[00:38:34]So call it buffer.

[00:38:43]Uh file. Yeah. File is the right one.

[00:38:45]file and then calling that um the member function there and that should essentially copy the entire content of the file into the buffer and then we need to return it as well after uh let me see returns the underlying raw string device object.

[00:39:10]Okay, it's a function, right? So, it should be like that.

[00:39:22]Uh, then we can also this is kind of just a source. So, that's we need to run it as well. And the source that we're taking will be uh the buffer uh we want to call stir buffer or wait uh got to set the content. So yeah, uh buffer dots there.

[00:40:08]Uh what else? It must also check for any errors. So okay, we already have our error as error here. The error flag that we set. So if um I mean I guess if not error well no you need it to be if if it is an error. So if it's an error it won't it just kind of won't like work here. So let's just call like ST exit or let's just give it a different exit code than the top one.

[00:41:00]Okay, that should be reading that should be reading the file properly now. So that's the way to handle files. If I do believe in the book, that's how they kind of handle it. The book is the book is Java. So it's not going to apply directly.

[00:41:17]So I had to do some research on how to read these files and open these files properly. But yeah, the book is Java, so it's kind of difficult to read for me to be honest. But yeah, we're getting by. All right, so we have the run file. Uh let's see what else is in my notes.

[00:41:37]Yeah, we need to do the reppel file now.

[00:41:38]So the ripple is just going to be running prompts. It's not going to be taking any files in. So, we don't need to open any files and read. We just need to take in allow the user to actually pass in what they want to pass in for the ripple. Kind of just like how a Python ripple would work.

[00:41:57]So I mean kind of just something like this where it has like the ability for you to actually write what you want to implement here.

[00:42:13]So we need to somehow make that as well in here just allow basic rubble. So we're going to need a separate function here for that. So, let's call it run prompt.

[00:42:30]Okay. So, we're not taking a file path.

[00:42:32]What What do we take? Do we take anything in at all?

[00:42:38]Uh, I mean, not really. If we're just running the prompt, we if we're running the prompt, then we just allow the user to type. We don't need to pass anything from the user itself. We don't need to get anything from the user.

[00:42:52]Uh let me read this. So let me reread my notes real quick. So user can enter the ripple type. So yeah, if they type print three, then it allow them to print that value. And there needs to be an infinite loop. So unless the user actually exits the rebel, it will just be infinite loop. So kind of like a wild true loop there.

[00:43:24]Uh then you can run the line type of user.

[00:43:30]Okay.

[00:43:32]End of file. See when the input ends.

[00:43:35]Okay. That's like in like the command prompt I believe if I was reading that right. Like in command prompts when you type in control Z in the operation you exit out. So then you can run the line type of the user then reset errors.

[00:43:46]Okay.

[00:44:01]So, let's see. Uh, I close the thing.

[00:44:06]I just want to kind of like see as a reference real quick.

[00:44:13]Where was that one thing that we had?

[00:44:17]Well, I guess it doesn't have that. So, okay.

[00:44:33]Well, I guess we want the user the user input. So, let's say um let's just call user input. So that's the input that the user gives to us for that.

[00:44:55]And then kind of just like a a wild true loop.

[00:45:09]Uh, I believe like the the ripple there had like is there like anything like a C ripple or something? Let me see.

[00:45:20]Is that is that even a thing? Let me see.

[00:45:27]So I guess yeah. Okay. So like that.

[00:45:31]So, uh, so kind of just like this extraction just this sign.

[00:45:44]So, we can start off like that.

[00:45:49]Um, well, we don't want to make a new line there since it's going to be on the same line. Okay. So, uh to allow one full line from keyboard input to be read.

[00:46:21]So kind of like a stood seen I believe.

[00:46:24]Wait. Um, isn't there one where you can just get just one line and to process a stream line by line? I guess that's what we need. Uh, damn.

[00:46:35]What was it like? Let me go back to learn P for a second. I believe it was in here somewhere.

[00:46:44]Uh, I don't even know where it would be to be honest.

[00:46:52]Uh, did I write it anywhere in my notes at all?

[00:47:06]I mean, it should be in like s string stuff like that.

[00:47:10]Uh, let me let me check. Maybe it's in here.

[00:47:14]Get mine. No, it's not in here.

[00:47:16]If I can't find it, I'm just going to go to the the reference page.

[00:47:22]Okay, here it is. Yeah. To read a full line of input into a string, you're better off using this. Okay.

[00:47:41]Uh so I guess Okay. So you pass in stick to get line and then you pass in uh what is uh what is to the west again.

[00:47:58]Oh white space. Okay.

[00:48:05]Yeah, we probably want that too.

[00:48:08]So, actually, let me go back down here.

[00:48:22]Uh, hopefully audio on stream is okay.

[00:48:32]Okay. So, reading the blue one full line of keyboard input.

[00:48:38]Okay. So, I mean I guess if it's if it's not a line, if there's no user input there, Oh, what do you want Uh yeah, I mean that's okay. That's also fine. We can keep that. So it's it was like this. So WS and then I believe it was in the parenthesis or no it's not in the parenthesis. It's just um that's the first argument and then the second argument is the actual user input. So the user input that we have So, so for not getting any user input, it's going to basically get be the end of the line there.

[00:50:10]So, we kind of just break out break out of this loop.

[00:50:17]But if if we don't break out, we can just run the line. So basically the same thing. We're running the source here.

[00:50:23]Kind of the same thing as we did up here. It's actually run the source. Uh not line run user input.

[00:50:33]We'll be running the user input. Uh it said something about resetting errors as well.

[00:50:40]So I guess uh that's just setting the error back to false, right?

[00:50:51]Okay.

[00:50:54]Yeah. So, we handled we handle files, handle the ripple here.

[00:51:02]What else?

[00:51:05]We need to do error handling after too.

[00:51:16]Um, yeah, I'm going to check before we write in uh main the main. I also have the notes straight into main. I'm going to check with Claude though to see what I wrote is actually good enough.

[00:51:29]Let's let's uh let's take a quick second.

[00:51:35]Uh this is for file handling and bre handling.

[00:51:43]Uh see if I implement it correctly. I believe this is the correct implementation. Let's see if uh I maybe messed up something.

[00:51:54]Uh let's see.

[00:52:02]Yeah, I figured that was right. Uh, global error. Let's see.

[00:52:08]It's going to get harder to manage as the compiler grows. You'll eventually have a lecture error, parser error.

[00:52:12]Okay.

[00:52:17]Stern pattern is an error reporter object or even a simple strct you pass around.

[00:52:23]Okay.

[00:52:25]Yeah, we can write that. We can we can write something like that.

[00:52:30]Oh, let's Yeah. Okay. If that's the only problem, we can we can reset that. Make it not a global. Uh let's do in main though.

[00:52:40]All right. This is the important part.

[00:52:42]So, for something like this, having in main, you need uh I was really confused at this when I first saw this, but you're going to need your need RC. need a char pointer for the arg v. This is your argument count and this is the array of the arguments that you have. So essentially if someone passed in like C uh test C, the argument count is two. So C and then test C are different arguments. And then the array of these two arguments right here. So argument zero is C. And then your actual file that you're testing will be the the first the second element. Sorry, not the first element, the second element.

[00:53:25]And you can't have any arguments passed to. And if it's just one argument, then we're not passing in a file. It's probably going to be like the ripple prompt that we had up here.

[00:53:38]The file handling's for two arguments.

[00:53:41]And then ripple, we don't need any files. So we're just passing in one argument. So all right.

[00:53:50]So let's see.

[00:53:57]So yeah. So uh any argument pass? So we need to we need to validate this. So if our argument count is over two.

[00:54:18]I guess we can just error error check that.

[00:54:27]I'm just going to type this for now. Too many arguments.

[00:54:42]I said too many arguments here. Um, what else?

[00:54:54]Oh, dang. I realized we have a good amount of people watching.

[00:55:01]Uh, so arguments is greater than two is that if our argument is just one, it'll be the run prompt. If our arguments is equal to two, then it'll be the file handling. So let's uh let's write that. So else uh what's going on here?

[00:55:20]So else uh else if we need to else 16 this will check if RXT is equal to two.

[00:55:36]So we want that and then we also want else for any other type of argument. So one argument zero arguments.

[00:55:49]So yeah so the argument count is two.

[00:55:53]Uh, so I guess we just call the function run five, right?

[00:56:02]And we want to call we want to call the the first element of the argument array because that's going to be the file we pass in. So not the first element, the second element. Sorry, it's it's yeah, you know what I mean.

[00:56:18]Uh so we want to call in arg of I believe that's that should work.

[00:56:33]Yeah cuz running the file path and that should handle that.

[00:56:38]Else uh this will be run prompt. So doesn't need any think to pass. It should just be run prompt. We don't need to pass anything in. So yeah, run prompt. Uh oh, semicolon. Missed that. Uh yeah.

[00:57:11]Think that's it for main.

[00:57:18]I think that is it. So let's just recap on what we have before.

[00:57:23]Uh so I started off by setting up this lexer header file. So we got the token type. So whatever the type of the token is and then we set up the strct to actually set up what the token would be.

[00:57:34]So it's type the actual raw word of it, the variant like the literal what it could be and the line number it occurs on. We set the constructor. Uh, oh yeah, we need to revamp this.

[00:57:47]We set the run, we set the run function.

[00:57:50]We haven't actually implemented it yet, but we will.

[00:57:53]This is for file handling. This is for the ripple. And this is for um this for main to actually handle everything.

[00:58:06]So yeah, that's a good amount we've done so far. I think I'm I'm still good to go for a bit more.

[00:58:13]I think we can still see. Yeah, we need to fix this. Uh let's not make it a global.

[00:58:20]Let's actually make it within a structure function.

[00:58:33]Uh yeah. So, let's let's make a destruct.

[00:58:50]So, we're going to put that in here.

[00:58:56]So, the actual value and then we can set up like a function here.

[00:59:02]uh we can have like a generic way of handling files. I believe like in the book in the book it had like a generic way of handling these errors and then also like a more sophisticated version.

[00:59:13]So we could run something like void and call it error.

[00:59:44]Let's call this hazard just in case.

[00:59:50]And let's change something here.

[00:59:55]Uh let's what should I call this trip? I guess just error handling I guess for now. Names can change. That's fine. Uh so void error. So what exactly do we want in the air to pass through?

[01:00:12]Uh, I believe in the book. I want to check the book real quick. Uh, where was it that they did that?

[01:00:27]Yeah, in the book. Okay, so they pass in a line like the actual implementation like what was wrong and the message and the actual string message. I believe that I wrote that in my notes too. Yeah.

[01:00:42]So it would be something like that. So line error at whatever here. So like whatever you mistyped or miswrote and then the explanation.

[01:00:56]So yeah. Okay, that's good.

[01:01:01]Yeah.

[01:01:16]What's up, Lucy? How you doing?

[01:01:19]All right. Line um the actual like wear part. So, I'm going to I'm going to kind of follow them. So, they made this report an error. So, that's how they handled it.

[01:01:38]I'm going to kind of follow them on that real quick for this part.

[01:01:46]That's great. I'm doing good. I'm doing good. Just writing this compiler.

[01:01:51]I finished the emulator the other day.

[01:01:53]So now I'm working on the second part of the project which is the compiler to put it all together. Okay. Uh line uh the wear would just be like a string, right? So uh let's make it a string reference. a constant string reference.

[01:02:25]Same thing with the message. So con string reference message and I believe uh yeah kind of just setting up so in Java they have the system. I don't know that about that but uh C++ we have stair so we can just use stair and kind of just like the same as that uh let me just see the formatting I wrote how I wanted it to be so line uh okay so line is going to be like that uh line.

[01:03:25]I'm not going to lie, C++ like the way to write out these strings are kind of annoying.

[01:03:34]They confuse you a bit, but yeah. Okay.

[01:03:35]So, that put space and then the specific error. So, wear. So, we want the wear here.

[01:03:47]where at that specific one will be the um wait hold up. Um, I wrote it wrong.

[01:04:19]Air at and then this is going to be the layer.

[01:04:24]Okay.

[01:04:26]All right. Error at.

[01:04:36]Okay. So, error at where I actually hate strings, bro. I actually hate output in C++. All right. Okay. Uh the where. So, that's there. And then whatever the value is, this will be the message, the message of our error. So, message and yeah that should be good.

[01:05:06]Yeah. So this will now handle errors and we can also have like a more simplified version of that kind of like how the book did. So we can call that error and that will just be like a line and the message.

[01:05:31]I believe that's how they did. Yeah.

[01:05:42]Okay.

[01:05:47]To be honest, I don't see a reason of splitting, but maybe I'll figure that out later on why to have a more lessened version of that.

[01:06:01]Uh yeah, same thing pretty much.

[01:06:14]Or I guess we can just call report.

[01:06:17]We can call it report and then say the line. So so this will basically call the message.

[01:06:29]Okay. Now I understand why now. I understand why. Yeah. This will basically call that function. Yeah.

[01:06:34]That's why the they did that in the book. Yeah, that makes sense now.

[01:06:42]So passing in those three arguments.

[01:06:46]All right. Error handling should be good.

[01:06:55]Um H I'm not actually sure what's going on here.

[01:07:15]Why that's a problem.

[01:07:22]Natic memor reference material to specific object.

[01:07:37]Uh, to be honest, I'm not actually sure.

[01:07:46]But I guess I can ask CL for a second.

[01:07:51]Uh, yeah.

[01:07:55]Yeah, I'll ask for a second. So the air is like number reference must be rel to specific object.

[01:08:24]Yeah, it must be relative.

[01:08:39]Uh, let's see. What's the problem?

[01:09:02]So, what's going on? Uh, so you can't access it. Okay.

[01:09:16]Oh, you're right. Oh, I totally forgot.

[01:09:18]That's such a that's such a bad thing to forget.

[01:09:26]So I guess in like outside of the functions we have we have to actually actually like make actually initialize the str. So I totally forgot. I was wondering why we couldn't because I've been able to do that for my other projects. I totally just forgot there.

[01:09:45]Yeah. So we want to actually initialize the strct properly and any other like any other things that we need to do there have to get that and I guess it's in the same file so it should be able to see it right or it's a problem there's still a problem here.

[01:10:21]Oh, okay.

[01:10:31]Okay. So, we need to actually add that as a we need to actually add that as a parameter in every single thing. now to actually be able to use it.

[01:10:53]So I guess then in any places that we called it, we need to actually like get that in properly. Uh and then also in here, right?

[01:11:21]And then make that I think he did that every single place, right? Okay. Every single place.

[01:11:31]So then uh I guess we need to make changes.

[01:11:39]So this will take it air handling. Well, wait. No, it wouldn't take it air handling.

[01:11:48]Um, it's still a bit confusing here.

[01:12:11]It's still a bit confusing on what exactly What exactly went wrong here? So, okay.

[01:12:21]So, they're Okay. So, what are we doing?

[01:12:23]So, we're running. Okay. Let me go to this part. So, the if part, right? So, the if part Okay. Yeah, they're accessing. I totally Yeah. Okay, that makes sense. Yeah.

[01:12:37]then accessing that um member.

[01:12:51]Yo, what's up, Muhammad? Uh, I'm making a compiler right now.

[01:12:56]I have my CPU emulator here already made. Now, I'm making the compiler for that. It's going to take a a subset of C and compile that through the virtual CPU that I made.

[01:13:10]It's the first day of the compiler, so I'm just trying to figure things out as I go.

[01:13:20]Yeah.

[01:13:22]Okay. And I mean to Okay. So, every every single part I need to change now.

[01:13:26]So, like that.

[01:13:44]And we also need to access it like how we usually do.

[01:13:58]Let me make a change. It's just me.

[01:14:07]The error handling accessing that accessing the error for that.

[01:14:22]And then both of these should also be changed. So I guess just error handling just to pass it through.

[01:14:36]Yes, that's a compiler.

[01:14:39]Okay, I think we did it. I think this is good now.

[01:14:44]Small little roadblock we had there, but we figured it out. Well, Claude helped me figure that part out. But yeah, everything else we did properly by ourselves.

[01:14:57]Yeah, I just totally forgot about that one part for the strct.

[01:15:04]I'll definitely uh review.

[01:15:09]Uh but yeah, that part should be done now. So, let's review. We have our ination token type.

[01:15:22]So whatever like the type of the token will be then we actually need the strct for the token to actually make the token. So our type the actual word which is the lexim and whatever type the literal could be and also the line number where that is declared right and then we have error handling here I just revamped that right now this is to actually run any source that we're taking in either the command line or the ripple and this is the file handling so if a user passes a file through in the command line they will be it'll be taking the file and running opening it and then running it and this is for the ripple for the Crepel. So basically a user can pass in can basically pass in whatever they want. So if they want to print three or make a while loop or anything. So it's kind of just like the Python ripple kind of like that. And this is the main function. So I actually get everything running. So that works for that. Okay.

[01:16:31]What's next? Um I guess we have to make the scanner class.

[01:16:41]We should actually be scanning now.

[01:16:51]So we can actually set up our scanner class. Let me try doing that right now.

[01:16:55]So, uh I guess we we can set a back in lexir uh going to have to pro I haven't gone through the scanning part properly. So, we're probably going to take a look here as we go. kind of like uh kind of like convert that from converted from Java on over uh where is it actually? Wait, did I go too far? Yeah, here. So, they actually set up the scanner like this in Java.

[01:17:40]So, okay. Okay. So, they're saying a a string for the source.

[01:18:04]So, let's set our let's set our private private members in here.

[01:18:12]So, string source I believe. Is it uh is it from is it before? I'm pretty sure it's like this tab like private members with the underscore. I think it's after. I think it's after.

[01:18:37]Uh I know in Python it's before, but I think in C++ it's after if I remember correctly. Okay. So the string source And I guess they Okay, so they passed in a list of tokens.

[01:18:58]Well, let me try and think why they would do that. So, I mean, I guess yeah, they need to pass in a list of tokens. So like if they have in x= 3 and they pass in the whole list of these individual tokens to actually scan through to actually make it valid. So that makes sense. So I guess we can here we can include vector and write that out.

[01:19:46]kind of the same like how they did. So setting a token.

[01:19:50]So list of tokens and we can also call that token. So okay that's pretty good.

[01:20:07]What else did they end up doing? So, let's see.

[01:20:13]Let's see the actual logic.

[01:20:18]Yeah, I'm using a vector. Sorry, I'm using a vector.

[01:20:30]Let's create a simple string. We have a list ready to fill the tokens we're going to generate.

[01:20:34]That does that. Looks like the Okay.

[01:20:47]So, it looks like they're actually going through it. They're setting kind of like a a starting point and also an ending point to actually go through the tokens, the individual tokens.

[01:21:12]and start. Okay, so they have a they'll have an end they'll have an end like a current value like a current place in the token and also a start value. So okay I guess yeah we can set these variables here.

[01:22:00]Um, okay.

[01:22:05]So, I guess kind of like a line. Okay.

[01:22:06]The line number as well that they have.

[01:22:09]Yeah, that's important that we need that. That was in our strct that we had earlier for the line. So, uh, you can't really have, yeah, so that's why they put one. You can't really have lines here. So, one just initializing that. So, uh, is there an equivalent of that in is there equivalent of that in C++? Does that end that function or is that something they had to implement or is that C++?

[01:23:10]No, that's something they implemented.

[01:23:11]So, we can implement that.

[01:23:16]Okay. All right. These should be all our private member variables. Uh yeah, let's go. Let's go for it. So guess we need to set our constructor now. So let's do that in here.

[01:23:40]So it's taking the variables So we'll take that variable. So the source that we get and we'll tie that in uh we can call like uh to move on it. So just like how we did earlier st move so we can just move the string instead of making a copy of it. So the source uh yeah and then the constructor will have the vector in it. So that the link went and yeah called tokens.

[01:24:45]So yeah, that should be good for that.

[01:24:49]Good for the constructor. Um, and now we actually need to implement.

[01:24:54]So, let me just quickly try and understand. So, they're starting at the start of the token and they're going through the token and whenever they hit the end of it, that's when they stop. So, okay, that's interesting.

[01:25:31]Okay. Um, so we need to know if that be right. So it's going to be a blue.

[01:25:54]And we want to be returning whether or not it's true. So, I guess when the the current th I guess when the current value is greater or uh would it be greater than equal to?

[01:26:27]I guess um yeah greater than or equal to the size of that vector that we're having. So or not necessarily the vector because I guess it would be yeah it would be the source right because it's not necessarily the vector because that's the vector of all the tokens we need the source and we need the actual source so whenever hits the end of one source the token one token then we can move on to the next one.

[01:27:24]So yeah. So basically just returning the size of this I guess. But uh we would probably need to cast that.

[01:27:37]All right. Is that even possible?

[01:27:44]Wait.

[01:27:46]Um I guess that's fine, right?

[01:27:57]I guess that would be that would work.

[01:28:11]And then we need to keep going for it.

[01:28:13]So let's just see how they're doing.

[01:28:19]So they have they have different functions. So they have the advanced function, add a token.

[01:28:28]Okay, that's interesting. Okay, let's try and implement that in here.

[01:29:03]Okay. So for advancing I want to return like the specific char of that source and keep moving forward.

[01:29:16]So, I guess it would be current that current value incremented by one essentially. So now this will pass that value back to keep moving forward.

[01:29:56]That's fine. We we'll check it all with Claude anyways at the end to make sure I'm actually doing this right.

[01:30:06]And I believe that add token scan token. Add token.

[01:30:22]What other implementations?

[01:30:35]How long have I been streaming? Let me see.

[01:30:40]30. All right. Maybe like five or 10 more minutes and I might end the stream.

[01:30:50]Okay. Uh Start field points to the first character and the lexim being scanned.

[01:31:11]Current point current points at the character currently being considered. Line field tracks with source line current is on.

[01:31:18]So we can produce tokens and know the location. Okay.

[01:31:25]Okay.

[01:31:29]Interesting.

[01:31:58]These are for scanning the specific tokens. Okay.

[01:32:08]Okay. Okay.

[01:32:11]Char advance. We just implement the char advance add token.

[01:32:36]Okay, that's how they're reporting back the error. Okay, that's interesting. Okay. Um operator division characters need a special handling because comments begin to slash two. Okay, got another helper function. Char peak.

[01:33:19]So, it only looks at the current unconsumed character. We have one character up. Look ahead.

[01:33:34]Only peak one or two characters ahead.

[01:33:41]Okay.

[01:33:50]And I guess that's that's pretty easy to find. So start peeking at the value.

[01:34:16]It's also probably a good idea to make these little con.

[01:34:23]It's probably a good idea to make them all const.

[01:34:30]Oh, we can't make this one const.

[01:34:38]Uh, so we're just peing at the current value, right? The current we're peing at the current value. So, so as long as we're not at the end, if that's the case, we'll just return it.

[01:35:22]I mean, I guess return like a I guess return like the null terminator, right? I assume that's what they probably do in C++ uh in C and C++ as well. But yeah, so at the end it's just going to be the returning the null terminator and otherwise we just peek at the valley.

[01:36:05]instead of just returning that value. Uh yeah. Okay, that's good.

[01:36:16]But I believe we also should have like a another one as well. But that's to peak the next value cuz the I remember it did say in the book that we need they can peak two Uh we need to also check if it's at the end. So, so if our current value right if our current value if we increment that by one right So if that's greater than or equal to the actual uh source kind of just similar to how we did there our source size.

[01:37:31]If that's the case you're going to need to just return out which will be an alter just how we did before.

[01:37:45]Uh and H.

[01:37:51]Yeah. So, if that's not the case, then we can peek ahead. So, let's see. Um I guess it would just be this value but just plus one instead.

[01:38:19]So we do have the peak next functions now. Peek and peek next functions.

[01:38:35]Okay.

[01:38:53]Now they're handling longer legs. So, okay, that's interesting.

[01:39:23]uh these will be implemented out of the class anyways to actually implement all of these but uh I'm just looking to make all the helper functions but yeah okay so we would need to make this helper function here too uh let me see exactly let me see exactly why we need to do this though Um look at the second character, right?

[01:39:59]Okay. Yeah, since these characters are taking multiple multiple characters just like we had uh those would fall under some of these ones like exclamation equal equal equal. So they look at both both characters to actually implement. So that makes sense.

[01:40:29]Okay. So, okay. So, for their specific case and then they're adding another token for that.

[01:40:38]Okay. So, this one's exclamation mark.

[01:40:41]If they're adding the token and they're matching this value, it's going to be not equal to and then this will just be not okay.

[01:40:56]And then let me look at the helper function how it's how they did it. All right. So I assume it should be pretty similar to do C++ expected.

[01:41:12]Yeah. So they're taking that value that they're expecting.

[01:41:19]Yeah. So if they're moving forward, if they're moving to the next one, if it's at the end, it's going to be Yeah. So, okay. So, if they're moving, okay, so that at the end there will turn false.

[01:41:43]That would mean that it would end up being one of these ones.

[01:41:48]Yeah, I think I understand.

[01:41:51]Let me try and implement. I understand it. Uh, okay.

[01:42:02]I believe it was a boo, right? So, we need to see if it's true, right? You put a Yeah, it was a bull.

[01:42:13]So, I need to see if there's a match or not. So, I expected and if we are at the end just be uh I guess just between otherwise it's too true.

[01:43:10]But also I guess we need to also check one more thing. We need to make sure the token matches right. So Uh so kind of just like how we did there.

[01:43:40]So the token that we're moving forward.

[01:43:51]Uh no. Um what am I what am I exactly trying to do? uh not source size we already we're already checking that it's source dot um the the current value that's what I'm trying to see so I guess it's going to be current if I can type right now all right so current that's not equal to the expected value that we gave and we'll return false.

[01:44:49]And we also want to comment this this value up after that's done.

[01:44:58]So we can allow it to actually add it.

[01:45:00]So yeah, that should be that should be good.

[01:45:15]Okay, they did it in two checks. They did two checks. I think this is fine though. It's not a big problem. Uh I accidentally opened up Opera. Sorry about that.

[01:45:26]Going to close that. All right. Uh yeah, I think all the helper function should be I think I looked over helper functions good enough.

[01:45:40]I think that should be good.

[01:45:45]Yeah, we've been streaming for a long time. I might and also stream here. We got quite a bit done. It started slowing down at the end a bit. I do need to read more about this stuff. I do understand it, but uh I need to read more about it.

[01:45:58]But yeah, at the beginning we were kind of fast through going through the whole main function with their handling, file handling, ripple handling. We did that pretty fast and pretty well. Same with the enumeration. The strct got a bit slow once we got to the class, but that's fine. It's our It's the first time, so that's not a problem. Yeah, I guess I'll continue it off tomorrow after I do some more research about it.

[01:46:24]Yeah, we'll continue up tomorrow, guys.