The scanning phase (lexing) of a compiler converts source code into a sequence of tokens by recognizing keywords, operators, literals, and symbols. This process involves defining token types (such as operators like +, -, *, /, ==, !=, and keywords like if, else, true, false), creating a scanner class that reads the source file character by character, and implementing functions like advance() to move through the source and peek() to look ahead without consuming characters. The scanner then uses a switch-case mechanism to identify each token type and push it into a token vector, enabling the compiler to understand the structure of the source code before parsing.
Deep Dive
Prerequisite Knowledge
- No data available.
Where to go next
- No data available.
Deep Dive
🔴 Live | Building C++ Compiler from scratch Again!!! | S02 Part 1Added:
Okay. So, hello guys. We are live now and today I'm going to continue with a project which I was doing already. Uh but it will be a newer and better version of a compiler. Uh I'm going to make a C++ compiler today and I will try my best. So today's topic will be scanning inside compilers which means I'm going to scan tokens and here are my notes that I made. Um currently I'm following a book uh which is called crafting interpreters.
So that is the book that I'm following and uh I will just continue with the project of my compiler.
So let's go.
So the first thing that I'm going to do is to make a main file of my C++ a main file. Let me increase the size of the font. Yeah, good. A main file. Main CP.
And then I'm going to do some basic thing. Include your IO stream and in the main because I'm also going to pass arguments. So I'm also going to mention that thing.
The first thing that I need is um int a r gc then int uh it will be int will be gap gap pointer a r gv if I'm not wrong.
if I'm not wrong.
And I'm just going to check out which will be our normal hello world.
Hello world.
I also need to return something. Return zero.
will be it will be it will be RGB but uh but uh but hey this is is the basic code so there's the most basic think okay what I'm going to do is I'm going to make a a z file so that I can run my code I'm also going to make a a new file which will be uh test cdp it will be simple code return zero Is this going to work? I think not.
But let's just see.
Um, we have main.
We have main. We have main. And then I'm just going to give it some another extension like any random extension which will be CCC or I can just call it CC still C++.
So this will be the extension that I'm going to use for now. I'm just going to make it as a text file, not a CPV file to keep it simple.
And the next thing that I need is a z file.
A z file which will be which will be run z. And I'm just going to simply pin boot.
What was the What was the What was the What was the was it?
It was been boot.
stupid boot and then then then then the command that I'm going to use first of all I need a path absolute path absolute path which will be the last dot and I have documents documents then I have GitHub then I have CPP compiler this will be my absolute path I'm going to use this path I I'm not sure about this syntax if I'm correct or not let's just And the command that I'm going to run is plus which will be the main file.
Oh, I'm going to run the S command.
We need to cd Yeah, this space is unnecessary.
And the main CP then I'm going to output a main file.
Yes, main file will be good.
Will be good.
And uh next thing is uh to run the main file main CP. No, it will be mainly main file. But I'm also going to pass a argument which will be the file name. It will be test.ox test is a keyword I guess.
Test C. So I'm going to keep it test C.
If you read the test and in this is I need to do CD then to the ABS path.
with the dollar sign.
I hope this is going to work.
It will be ch mode x and the file name which will be run.z run.z.
Yep.
So I have this s file.
Let's try running it.
Z a bad interpreter. Boot loose file directory. What did I what I did wrong?
Bin bash. Bin bash. It's not good.
It's bin bash.
Run it again. And it's hello world.
Okay. So, I have this file which is going to run my main.
That's good. That is good. That is good.
I have a main file.
So the first thing that I need is a token and the token type for my scanning phase.
Yep, that's what I need.
Is the So let's continue. I'm going to make a token enum enum class token which will um I'm just going to give it enum token token type. Yes, it will be a token type.
So we have a lot of type of tokens and I need to name them all.
Just one little thing that I need to do.
What I'm going to do is add a boolean.
Let's do it. Start a pool.
Which is your most favorite language?
Is it C++?
Is it Java?
Is this Python Rust or I will add JavaScript.
Okay, let's go.
I was doing this thing. I was adding some token types. So the first one this will this will be of type uh uh left pair, left, left pair and right barrel right par.
Then we have right barrel, right bar, right barrel.
Then we have what? What? What? Um, left brace.
Left brace.
Right brace.
Uh, left.
What do we call this bracket?
Left squad.
This is S Q U A O E bracket and again right square bracket Q U A bracket.
So I have these Okay, so I have these parentheses, bracket, braces, whatever they are called. And the next thing that I need is some different kind and like a dot a dot comma.
And what do we have?
We have hyper We have hyphen, we have dash, dash, and what are the different types that I can add? Um, I can add them later. So I'm going to add a integer character or string into literal in lit character string in lit and and and and what I can have I can have boolean boolean Then I have then I have what?
Literal.
Literal.
Is that a right thing?
I'm not so sure.
these type of tokens and I have I have semicolon semicolon also I need to add operators which will be plus minus multiply then divide plus minus Multiply divide and and and what more a modulus modulus operator is there any other thing that I I'm not covering that I have not covered divide divide divide -7 slash it will be slash not divide slash and and and also one more operator it will be multiply I will just call it star because it is not multiply it's star these are the operators and and and made that exclamation.
So I'm just going to add that exclamation type which will be band blank. Then also bang equals which means not equals equals then I'm going to add greater than or greater equal greater equal.
After this I have less less equal equal Q a l after this I'm also going to add equal and with equal I'm going to add a equal equal equal then again a equal type equal equal equal Hello. Hello Dian. Hello.
Okay, I have equal equal.
So I have all type of parenthesis and I have dot comma- also I need some kind of keywords which will be which will be which will be and keyword and no no and is not a keyword. If plus we have we have we have if we have else we have and typing code in big 26. Yes, I know.
I know it's a crime to write code by your own hand.
Um if else um I also need to have those true false value.
True false for my for my boolean operator not operator boolean keyword.
Then I need a variable like I need int and and and I need a do I need a string?
No, I don't need a string.
What I need is character intu.
I guess this much is enough. As much as there are already a lot of token types I'm going to go with token now token token token token token token token.
Oops. Oops. Oops. Oops. Oops.
This file is getting big big already.
So I have a token.
First of all, I'm going to add a adder which should be token type and token type.
Token type then I'm also going to add value of this token do to get it level right now.
What project did you do to get the level you are right now? To be honest, I haven't done any anything great like uh I have not done any big projects. So, compiler is really not a magic. It's not that complicated stuff. I'm just trying to make a basic compiler, you know, for learning purpose, just for a better understanding of code. That's what I'm trying to do.
int uh let's just I'm only going to take int value for now I can do it later only a int value how can I take multiple value first of all I need a token type there are two things here a line number yes a line number will be line number in line I have a value graduated or person I am already graduated in 2025 pxs string lets me string can I use a string value string.
Okay, I will try to keep value in strings. That will be easy.
I can also hold int values and I can convert them later. That's okay.
The string is better for me.
I have line number. I have token type.
What more do I need?
I don't think I need any more other than this.
So, let's just go to the main function.
Now, what I need is I will input a file and I'm going to check the arguments.
If the oxy oxy is not equals to not equals to two I'm going to throw three out will be all good all good But I will get see wrong wrong uh wrong or or or I'm just going to write wrong arguments.
wrong arguments and and and just as much as only for me now and I'm going to return it was uh it was it was it was error exit field.
Yeah. So if I have a a RGC then what I'm going to do is I'm going to take the file name.
So what I need is uh #incclude f string comma file just going to read the file. So this if if if uh not if If 7 out of thousand very good year certification the first one congratulations why congratulations sir notification frame file which will be my argument your GB your GB of one. Yes.
CLD2 and associate both clear.
So how did you prepare for your AWS certification from like what did you try from online resources purchasing some course or just YouTube stream file my bad my bad stream file stream file I'm going to read the file yeah let's just check if the code is working till Now I'm going to save the file.
I'll try running it. Runs is Yeah, code is working.
No issue.
So what I need is if not a file then I'm going to throw a error. not error, but I'm just going to tell him that I don't get any file.
Uh, wrong file.
Wrong file.
And then back slashing.
I'm going to return again. Return exit failure.
Is this going to be exit failure?
Yes, I'm just going to return exit failure.
So I'm going to return this. I also need a variable which can hold the uh wool file content in a string.
For that what we have is uh string string. Yes.
I got YouTube videos and created my own web app where I put my sample questions file practice platform. Oh, that's that's great. That's great.
So, the next thing that I need is video.
After this, what I need was spring string string string uh string and and n is there something that I need to Yeah, I need to add the file the header file.
Include include include include.
Oops. My bad.
Include include s stream. Yeah, direct file.
So the next thing that I need is file strr left shift operator. After base what was the function? It was uh file dot str buffer p.
Nope.
It was uh read buffer rd buffer. Yes it was rd buffer.
So I can try printing this file now.
uh see out it will be file str let's just try printing this if there's some kind of issue do I need to do something string I cannot directly print out this file file reader I cannot to directly read this file for one week but worth the trouble.
Yeah, definitely worth it. Yeah, if you are getting a certification in AWS AWS AWS is already in high demand. So, it's beneficial.
It's my 10 port. What should I start with?
You want to start C in CS? You want to start some kind of language or something?
Do you have an algorithm of flowchart or you're rebelling it? Now I'm I'm following a book which is called um uh it was crafting interpreters.
I'm following that book and according to that I'm trying to make this one. So it is only the first part of my compiler which is the lexing part the token which in which I'm going to make tokens and then in the next stream I will try to make some parsing like a or anything but I need to read that post here file.
Why I cannot print this?
What do I need to do?
Do I need to convert it to string? Is this going to work? I don't think so.
Oops.
Is this going to work? Okay, it worked.
Yeah, I only need to convert it to string.
It worked.
You worked with F or M? No, I don't. I have never worked with any F or M.
Start with C and basic component. It's better to understand basics. Don't jump to Python. Yeah, I will I will recommend learning like u starting with your starting your journey with C or C++. Um it can be a little bit hard. Not little bit, it will be definitely hard but it's definitely worth the time you will give it.
Uh directly starting with Python I don't recommend. Yep. Starting with Python is most dumbest thing I have ever seen.
file studio.
Okay, so I have all the file content.
Now what I need to do is to make a variable uh variable variable variable.
Where do I make it?
I just need to pass the I just need to pass the strings to the class.
Okay. So, I'm going to make a scanner class.
Scanner class.
Is that okay?
Yeah, I'm going to make it.
So, I'm going to make a header file first, which will be If you are really starting with C or C++ that's best please do not start with Python. If you are a student go with C++ you have plenty of time to you know learn many other languages. You can learn Python later but not in your initial days.
is the best for basic on C and C++.
Okay.
So, so so so okay. Okay, I was trying to make a header file which will be of scanner class scanner.h.
Let's keep the name small scanner.h and a scanner scanner CPV file.
I'm going to keep it inside the scanner folder.
Oh, where's my scanner? Move, move, move, move, move, move, move.
What is the move?
What is the command for moving a file inside? new shift scanner.
CP uh scanner dot h I'm just going to delete this file. Yep. Delete this one also. Delete this. Just going to keep these two. Then we'll be define my class which will be class scanner scanner and public and then private private private private private so inside private I'm going to add my add my source string which will be of type string string uh source string.
Yeah, I will keep it source string. And then I also need a Yeah, I get it.
Yes. So, I thought I would get in touch with Python. I don't know why people hyped. Uh Python is not overhyped. uh you know the currently anything you do in your AI stuff whenever you do some kind of AI stuff you always do it in Python which is the best language for AI you know it is the I guess it is the only language currently which is being used for AI most of the time but uh you know if you're starting your CS journey you need to learn fundamentals first which uh only C and C++ I guess I can teach you Even Java is good, but I will still recommend C or C++. You know that pointer concept is really good.
Is going to make you cry, but still.
Okay.
So, I have a source string. What I need is a vector.
A vector for tokens.
Tokens. Tokens. token of type token.
Oops, oops. Wrong syntax.
It will be of type token.
Yep, I'm not wrong. It will be of type token.
Okay, I'm going to make one more header file.
One more shall be of type uh of type of type token token.h.
Yeah.
What I'm going to do is I'm going to move this code inside the file.
It's token enough.
Come on. Get select.
Open it.
Ragma ragma once.
I've never used this but ragma once.
I hope this syntax is not wrong.
How to use pragma once? How to use use pragma once I do C++ mobiles.
declaration class goes here. Okay.
So that is correct.
That is correct.
I have correct implementation of these.
And the next thing that we need is Do we need to do J E or C?
J E or C is not compulsory but if you want a good college you definitely need to do it.
Generally I think it will be better to see if you're really good. Yeah, if you are really good at studies just go for J. Don't waste your time. Just prepare for it if you're really good.
Let's do it. Fine.
Where will be my token?
I don't think I need any token inside this. Yeah, but I have some kind of syntax error which is st string.
Okay, so I need to include a string also.
String.
Is there any other thing? No. All error gone done.
Now this is the scanner class.
I need to include first of all I'm going to say pragma once pragma once.
And the next thing is um what can be the next great thing?
I need a vector and string also.
Yep. Uh include vector include string and in the public function. I'm going to make those uh what were those functions?
It was advanc peak ahead.
Speak ahead. Yes.
To speak ahead.
Well, personally in my life, I have never given J E or C. Never never tried study but don't take it.
Yes. Preparing for a G leaves impact on you. It makes you smarter. You're right.
I agree with that.
I don't give C. I don't give C.
So the public function that I need is a string string. It was advanced.
First thing that I need is a scanner.
What do we call this thing? Constructor.
Yeah. I'm going to pass the string.
it will be source string.
And what I'm going to do is I'm going to move.
So see the syntax going to use this.
I'm just going to go with simple syntax.
It will be a copy and then yeah I should not use any reference but a copy src see oops it will be token I also need to include the token file include my token file which will be uh scanner then this token.h Edit this can do this. Yes, I can issue.
There is some kind of issue.
Define.
I don't know. My back hurts when I open VS Code. Why is that?
Why does your back hurts?
So I have a scanner. I need a void.
Will it be a void?
It's not going to be void.
Going to be a string character. I will give optional.
optional options.
So optional city optional the type will be but yeah type and for this uh I need to advance advance advance advance.
I used to make cheese for CS2 and sell them to random Russian. Seriously.
I used to play CS:GO. It's been like five years.
Same even I play I regularly play CS GO. Uh I mean CS2.
It's CS2 now.
Advance.
Yes. Advance.
So, what do we need in advance? We don't need anything.
I am mold. What is your age?
You have to change CS2.
Yeah, movements are like more natural.
That jiggle physics, you know, that's becoming more and more real. But, uh, that update with the magazine that [ __ ] the whole game.
Bro, continue. I'll see you later.
Okay. Okay, buddy. Okay.
You are 21. I am 24. You know, still younger. You are still younger. Improve.
You still have time.
So the next thing that I need is standard optional type data type but but it will be a peak which will be a similar version of advanced but not moving the index. ind.
Okay. So also while moving the index uh it need I need variables to keep track of line.
Yes, I need it.
I need variables to keep track of line 15. Damn.
What do we have? Uh we have errors. We have peak. We have peak after. Yeah.
There's one more function that I need is optional of type. Yeah. But peak after peak after after you know I will name it peak next. That sounds much better.
Pick next. Uh, do I need any other thing?
It will be a peak next.
Is there any other function that I need?
I have all the required function.
One more function that I need is Y type.
Y will it be? Yeah. What app which is going to tokenize tokenize tokenize I will just give the name tokenize and it will take a source string. Do I need a source string? No, I don't because it's already inside the class.
So what do I need?
Do I need those variables? Where is that? Where is that?
Uh it will be web.
You missed Y tab. Uh, what does that mean? Y tab. I don't get it. Y tab.
Y tab. Y tab.
Oops. Oops.
Nice.
We need to pass anything.
Let's just implement these first use of undeclared identifier srcstrl.
Undeclared identifier private.
It's private. It's private.
Okay. I guess I Okay. Okay. Okay.
This is the error.
Missing return type for function scanner.
Do you mean constructor scanner? Yes. I Okay. I I added extra and that was the mistake.
Now it is okay. Now it's okay. It's okay.
Okay. Bye-bye.
Thanks. Do you work for any company?
Well, currently I'm not working for any company. I did one year internship uh in a startup as a back-end engineer.
Currently I'm looking for some new job.
I'm not employed right now.
Yeah, it is okay now.
Just give me a moment.
So, all good. Let's continue.
Where do you study? Um, if you're asking about uh college or my current u degree, anything that I'm doing, currently I'm not uh studying. I did my bet already in 2025.
It's private. Private private private private.
Let's go with this C.
Okay.
Z file. Close. Main file.
Scanner. H. I'm just going to copy these functions.
So the first thing that I need to add is the header file which will be include uh scanner.h.
Yes, this is the file.
Now I also need to tell that it is of scanner class scope resolution.
Scanner class scope resolution. Same thing here.
scan a class and the implementation that I need is here.
I'm going to implement these functions.
So this is the last function of tokenize.
Okay, bro. Bye.
Okay. So, uh the first function is advance. I'm going to keep it down.
I will implement this later. The first thing that we have is a peak if uh I also need a size which is see oops we are putting the multiple functions that I need to implement. Now I'm getting confused.
Function join is is at end.
Where can I implement that function?
It's a C++ compiler for C++.
Yeah, there are more functions that I need to implement.
After knowing a bit of C language, I'm going to do some C++. Yeah, you can definitely do C++ after C.
What stupidity was I trying to do?
and took current line.
I need these functions private uh this variable private which will be int line int line and uh in start and also need a inurren type current.
Yes.
Is it going to be private?
Yes.
Are you or by Welcome to the stream. Yeah.
For series of Black Mirror.
Black Mirror.
Black Robots. same level here.
They both are same level here.
The best thing about Black Mirror is that uh all episodes are different. All episodes.
Okay. Aish you change your name. Okay.
Okay. Well, got it. Got it. Got it, buddy.
I have line start and current.
Okay, one more function which will be of type boolean.
It will be of type bool is at end.
This will be my function and I'm implement this inside this class.
FIFA mobiles favorite mobile.
this function.
Now thing that I check is return if the src is the current current current is greater than equal to src what was that src src string the strategy compiler so the strategy that I'm um the thing is that I'm following a book which is called crafting compilers.
I heard it is one of the best books. So I'm trying to um go according to that book how it implements things and how it does things. So I'm trying to follow that book and today I'm just going to I'm just going to you know do the first part of the compiler which is scanning which is the most basic and first initial part just to generate some tokens from a simple text of you know simple string which I can get from a file.
For now I'm just uh you know just uh reading this line you know return zero.
I'm not going to return zero. I'm going to return like 23. I'm going to yeah this is the line that I need to you know scan and create its token.
That's what I'm trying to you know achieve today.
But no, they aren't going.
Um I know one of my friend is in PUBG sports. Yeah. Um he go he goes to the events and you know he get sponsorship and some money from you know all this stuff. That's how he gets money most of the time from people gambling. Strange [ __ ] G S I26.
Then I would need a team. Are you an exos user?
No, I'm not a Nexos user. No. No, I'm not an Nexos user, but I am a Linux user.
I think you I think you mean because of my name, you know, Leo Nix. Not that has nothing to do with Nexos Lu Football State level. Uh LPU supports all of the sports. You know LPU I would say if anyone want to play sports then LPU is one of the best college. One of the best, not the best but one of the best for sports people. You know, students, they get a lot of opportunity to play in many, you know, in many games and LPU really supports really is really supports their students who play sports and get them medals. And uh for gold medalist that wants medal for college uh you know they have free host, they have free study, everything is free for them.
Black America episode is biggest.
Have you watched that episode of Black Mirror? Yes, free bro. Free. You need to earn college. Sorry. Sorry. You don't need to earn college. You need to earn medal for college. and college is going to provide you free living you know yeah that's that's what LPU do and many students just go in L help you for that reason a lot of most of the people most uh I would not say most but uh yeah many many of the students which are in sports they're generally not from you know richer background not from very financially strong background and it's one of the best that lovely can do for them brainstorm.
What are you trying to make?
Um if the current is greater than src str length hing here for example if my current length is undeclared identifier string we cannot use it like There's some level of stupidity that I'm doing. Uh is it because of this scanner colon?
Okay, it's going to work now.
Is it going to work?
Nope.
What is the issue?
I guess because of these functions.
So, we don't have access to current. We don't have access to current.
Int current. C U R R E N T. It's definitely some kind of spelling mistake.
Is it an ur?
So what is the issue?
Return current.
It should not be the case.
This is the camera.
It's nine lakh if I'm not wrong. One, two, three, four, four zeros. That is nine lakh.
Are you serious? 9.3 lakh 9 lakh people getting laid.
Okay. So this is the news after 9. It's 92,000. It's not 9 lakh.
six, I guess. Uh, okay. Overall, two years.
Yeah, that that may be possible.
That may be possible.
cheese string as different uh different kind of symbol R Do I need to include include string? Do I need to include string?
Do I is also not working? And I don't know why it should work. This [ __ ] should work.
Yeah. Fallen. Fallen.
It's going to work now.
Is that an out of line definition? Is at does not match any declaration in scanner.
Does not match any scanner.
How is that possible?
Total 203.
Okay.
But best thing is do coding you know AI is doing that already for us.
If you want to code that is only for your own understanding not for company anymore is attend. Uh why why this I guess I forgot to save these things.
Is it why?
Okay. So I guess these it is okay. Now I just need to remove this scanner.
Now what is the issue? Can you please tell me string sourc still not working?
Still not working. Why? Why? Why? Why?
Red definition of a string has a different kind of symbol. Why do we confused now?
Yeah.
Oh, where do we where do we I have a class class called scanner inh file and in C++ I need to implement it's uh what is that thing uh I need to implement constructor.
How do I do this?
I have scanner.
What about this file? scanner.h.
Okay, I did it already.
It's good good up till now but now when I do this inside scanner scanner entire then what is wrong?
I need a scanner. Scanner, scanner, scanner. Yeah.
Okay. All error are gone which is good.
The piecing model is possible.
Stupid. Okay.
Machine learning totally different.
So I don't want to pursue it.
and develop.
Scientist Research and development.
Hello bro. Hello Ain. Welcome to the stream. research and development.
But as a develop exactly I'm back. Do you remember me? I mean, I think it's been many days to be honest. I I don't really remember.
I'm not lie.
Exactly. By if you want to syntax. Exactly. Move the move. Move in C++.
Move in C++.
How do we use it? How do we use it? How do we use it? SD move.
Where to run?
Operating system.
Operating system.
stability.
Yeah. Yeah, I remember.
But I think it's been many days since we joined last stream. Yeah.
H how do you move? You know what? I I'm just simply going to do it like this.
S OC string uh equals to src and it is done.
Just remove this [ __ ] Oh. Oh.
Yeah. Ubuntu is really great. You know the best distro I would say for a beginner is Ubuntu and uh if you really love to you know explore more about Linux. So you can try different different flavors on your virtual machine or you can just you know partition a 70 80 GB of partition take out death part and you can keep on installing and trying different different flavors but for the stability it's always file test. I'm just going to close it.
scanner.h token.h Don't need it. Don't need it.
Now let's take okay arch is great.
Shut up you arch user.
We by the way I use arch portfolio website.
Yes, I installed it on my all three PC.
Seriously, you don't want Windows anymore?
Uh, you know, you should also have Windows side by side because there are some application or some games that you may need to, you know, run in Windows.
Sometimes in Linux, it just get hectic.
Let me show you my portfolio.
portfolio.
Just look at this.
Um, it's a very basic portfolio, you know, very simple and minimal.
Just to show what who I am and what I have done, my GitHub is dead right now.
I made a dual boot of window. That's great. That's great. Dual boot of window and Ubuntu. That's great.
I would also recommend you to use Art Linux. Yeah.
And just give it a try. But don't use it as your daily dro.
Ubuntu is your daily W. Remember that.
Remember that. Do not forget that.
But Mac OS is love.
Mac OS So requirement.
Sure, please.
It will hurt me.
Linux is for games. I played hundreds of games on Linux.
N Yeah, you're right. You know I have tried uh what was the game? Um it was Rainbow Six Siege. I have tried Rainbow Six Siege on my Linux in this Ubuntu.
You know this. Yeah, this Ubuntu in which I'm working. I have stream installed. I can show you. You know I do play Taken. Yeah, Tekken works fine.
It's work really great. Taken is really good on on Linux. But the Rainbow Six Siege was having some kind of issue when I was going in you know lobby took uh there was that section where you can you know you can apply skins of your characters. So I was doing that and it always crashes. It always crashes never worked. That was the issue code for code forces.
I've only tried one question on code forces.
If you're talking about code forces you know RE9 what is RE9?
What is RE9?
What is R9? Sh. What is this?
If if if if the current current current which will be sourc current string current is not equal to is if is at n is at 10 then I'm going to Hello. What can I return?
Options. I forgot. I forgot options.
What can I do if I don't find to it was something like null or something.
Null opt. Yes. Yes. Yes. This was the thing. It was return null opt. Null opt.
Yeah. And else if it's No, not else but is it going to be else?
Okay. Resident Evil 9. Yeah.
works well on my system on Ubuntu or you know Windows Resident Evil 9 Lux. So, how is it going? It's Is it working fine?
And what is your PC specs? You are running Resident Evil 9 on Linux. I guess that requires a good PC because Linux is not uh as good as Windows.
21 votes C++ or Python favorite least favorite JavaScript as expected JavaScript language.
Mac 12, I guess.
I don't remember null pointer. If it is not null, then it's definitely going to be a vector.
What is it going to be return?
It's going to return src string which will be current.
It should run return current and same thing I'm going to implement in slide peak next I will just implement this later I'm going to comment this patent Why cannot I use it directly?
Why I cannot use this directory?
If it end about me.
This could be the day I die for you.
No, I have never tried Zik Never.
Okay. Okay. Okay.
But why languages.
That's the main question.
Uh so what is the current situation of Zen market?
Are people using it or not?
Let's if attend use of undeclared if attend if attend what we declared it above this now it's working no no I write If at end it is is at end.
My bad. My bad. My bad.
Same thing I'm going to implement for advance but I'm also going to consume the character.
What I'm going to do is I'm going to increment the current. Increment the current.
The same thing I'm going to return it and no output.
Okay, all good. All good. All good.
Now there's some issue with this code.
Okay, both functions are done. Now the function that I need is tokenize.
No jobs in Zigg.
Replace scanner tokenize tokenize function handling.
line.
Advance peak.
Advance line number.
If the character is consume consume is that let's go back if the character is here.
If the character is how can I check this?
C str current current will be equals equals back slashn and what was the back slash zero back slash zero null character programming program blah blah blah blah I just need to check for back slashing Is backslash in a character or string?
8 83 I guess 800 something Ready.
Are you eating?
Inspire new My bad.
Sing character.
New line.
It is a character. Oh, character single single single line plus string.
or you can reverse engineer you can do some productive all know like reverse engineer uh I don't think I'm that good like uh I can start doing reverse engineering directly reading assembly and start doing reverse engineer I don't think attack level and I'm not trying to go for you know that um reverse engineer hacking and cracking stuff. No, not into that but I'm trying to what I'm trying to do is not to do not to go into that line. I'm just trying to go with a simple basic compiler just for a better understanding of code better understanding of C++ and many more things.
I have tried uh like backtracking. I tried reading a little bit about backtracking. I've also tried this IDA.
I've also tried that software. What was the name? GDX. And there was one more.
It was Kra something. I don't exactly remember. I have even used that software. I have tried but I am not into that you know. I'll just stick with my compiler.
I'm good with I'm good with it.
Is this a Linux resto? Yes, it is a Linux restore. It's Ubuntu.
So I think I'm happy with these two functions which are peak and advance.
I'm going to implement that error thing later. I can do that later.
Error handling can be done later. So thing that I need is tokens.
Then let's go with the tokenize and take it without a slots.
Thank you. Thank you. Thank you. Yeah, I am trying it by myself. You know, if I get stuck in something, I don't know about I'm not very good with C++. So I just go to Google and you know Google some stuff and try to understand what does that mean or is there any different way to do that. That's what I'm trying to do just for a better understanding.
There's not something like I'm going to release a whole new compiler that's nice to have source I've sourced a string and I'm going to run it till it's choose Ubuntu.
Um I have tried many different uh you know u Linux distros like Mint Ar Linux and Kali and different but the most stable one and most peaceful you know it's Ubuntu. I've tried many different but I'm back to Ubuntu. Started with Ubuntu and I'm back to Ubuntu. It's my favorite alltime favorite. I can do anything with Ubuntu which I can do with any other dro what I need is while while while see string length is start start and string but I don't need this start or do I need Starting current.
I'm stuck.
I need scanner class. The scan class tokens. Yes. Source.
Uh the source. Okay.
Scan tokens while not particular current scan tokens.
number one.
That is a good way. I can use a scan token function separately and and and token scan token.
Very good.
So what I'm going to do is uh scan to scan. Oh my bad.
It will be of type wide because I only need to Right.
and open field tokens definition class.
Not while but the character which will be current update current index.
Then I need to check which token is that token which will be switch statement.
Now if I'm if if if uh CR see it will pointer advance ADB Ace C advance advance advance ADB NC what is I do advance undeclared. I do advance.
Why is that?
Why undeclared?
Click next.
You have advance.
Okay. Okay. Okay.
Because resolution All good.
If the conversion exactly if the C is is space which will be C I'm going to skip it.
Skip it.
Advance.
Advance.
Advance. Advance.
Already past. Then I'm going to continue.
But I need to run this inside token.
Okay. Nice.
Length length length. If it is greater than or equal to current is equal to click button.
Okay then.
No doubt. No doubt here.
If the current which is my index which is my index line third once string going.
Advances check again. Enter Y current is greater than equal to source sourcing. Yeah.
And open.
file manually.
This set and get Good to go.
If if it is legit.
Which project if web development mostly Z.
Am I right?
Linux and tokens does not match obviously.
I guess I'll save tokens tokens. Tokens tokens development.
Never tried it.
But try confuse error handling.
Uh okay.
Object type.
Hi Java.
Scanner class.
Scanner class.
Scanning.
switch case.
But, but scan tokens scan token advanc match function match function once again consume the current character. That's what we are looking up there.
Okay.
Whatever digits book.
programming language 01.
Starting 01 Mhm. Longer.
Long.
We're still using operator. Okay.
case match here while comments another helper.
Okay. Direct last end of file return.
Literally Black Christmas.
Top cap is over here.
White Christmas.
Another helper. Look ahead. Look ahead.
Look.
Sub. Sub everything is done within one function between those switch cases.
Can I do this? Can I do this? Can I? Can I? Let's take example. If I have a advanced function is this piece. No, no, no, no, no, no.
Not is this piece? But what I'm going to do is I am going to check it via switch.
Switch switch.
It may be possible here.
food.
Second No memories name of the professor.
Switch switch. Let's see.
these keys that I'm going to check is the first thing is star which will be um of token type. So what I'm going to do is token push back. Uh, I'm going to push back compiler C++ compens and token token type which will be token type it was star yes it was star and the next thing that I need to pass is empty value because there is no value yeah feed lm feed pred.
Black and paste for push back. Um start special which will be then minus.
Python language career options.
Python JavaScript and Java market.
Are you a college student?
depressing depressing common people.
Common people if I'm not wrong.
Common people. Oops. Black common people.
Season depressing episode.
Start C++ beginners easy no doubt. or C++ 2 or Assembly.
No doubt C++ favorite language 29 29y Python C++ 30 43 for Python. C++art chart. basuto.
My bad.
Season 1 episode 3 I don't remember entire history of you. Oh my Okay.
Like Tech, computer science.
AI possible.
Fore degree.
But mostly EC is still better.
Mostly mostly just because three years BCA BCA Okay.
It was plus minus plus modus.
That's good.
So the thing is after this I'm going to do break.
Yep.
Break over here. And after this I have plus value empty.
Then we have break.
pressure.
Interview Sure.
Sure. Question B CSC.
Let's get back to work.
Yes.
And we have this. We have this. We have this. We have this.
So we have plus minus it was minus uh then we have 25 BT complete and one year experience.
plus minus plus minus or we have integer. We have character string int.
Then we have bang bang equals equals left.
Okay. do not commot.
G.
participation opportunity.
So gap interviewerap.
reason Minus minus - dot E Then we have comma will be comma.
Oops. Oops. Oops.
Then we have a semicolon. Semicolon.
Okay. So the next thing dot comma semicolon class if else true false and care then we have left single character left right left brace right I'm going to remove these two also because I don't need it not right now put is I'm going to add four cases of 1 2 1 2 3 4 and in these cases switch syntax issue no case statement is not not in switch statement. Oh.
to copy this.
Copy. Then we paste. Uh paste. Paste.
Then we have parenthesis. This one this one left. Then we have right brace semicolon.
Oops. down.
Left base.
Right face.
Then we have let will not be bra but it will be Sorry.
Different different hackathons participate differently.
eligibility.
Sorry to cover operators and then different case.
Comment.
Comment support.
Support.
Do that.
Good night.
Good night. Good night.
See you buddy. See you.
Okay. So let's push now. It will be case. C case will be of type slash. If it is a slash key then what we are going to do is we need to check we need to check check what while they peak next.
Peak next. Next function next will be next and comment. Oops.
Delete.
Delete.
DC and and actually next function does not return a value. I don't care.
What I need is if is at end. So what I'm going to do is return while end of line but maybe possible.
/ next start.
Peak. Next.
Next.
Next.
This a fall back ending when we have peak next.
Next.
Next.
And what we need is that next.
Need add token.
would be good.
String string this was for string is for this.
What about literal which is explained our skin is getting smart. It can handle fairly frequent code like this.
What we need is next Okay. Okay. Safety.
Check if string length current + 1 is equal to not is equal to but is greater than is equal to.
This is length then that means it is a uh we cannot we cannot we cannot pick next this then what we going to do is return null option then that means that current + one so we can return.
We can return Cy string shall be of print + one print + one and uh no this much is enough. We can pick next without any issue.
What's the is now it's semicolon.
Yes, we can pick next. Now if the case is this if the case is this going to add break if the if the peak next is also slash peak next peak Next one pointer is also going to be a slash then it will be a comment.
Yeah, it will be a comment.
And what we are going to do is until we find a enter which is backslashing. I'm going to just continue and skip.
Yeah, I'm going to skip it.
Come on. Y. Come on.
is ticket. Too good. Too good.
Okay. Okay.
continue while um it will be advanc while until I get a backlash n that is for sure advance Advance.
This will be the command line to traverse.
I can just continue and after continue with advance it will automatically it will automatically increase the um the current the current line the current pointer which if it is not uh pick next case if it is not a PKX case that means it is a it is a token token of slash so what I'm going to do is first of all I need to make a function type star type star no I don't need a function I'm just going We'll keep it like this and and and will be else it is going to be a it will be a token dot push back of type token and This was the case which was of token type token type of slash. Yes. Slash token type slash.
Oopsie. Caps lock on slash and the value will be this empty value. The keys keys keys is okay. Now uh I just need to add a semicolon.
Okay. After adding this semicolon, the next thing that I need to do is I've handled this case I also need to handle but these the greater than less than. Yeah also equals to where is equals to equals to is not here and that is good. That's good. That is good. So the next case will be next case will be of type equal. If it is of equal type what I'm going to do is I'm going to do the match I need to write a match function a match function for this which will be a boolean type boolean type match match match character match here will be my function and inside this file for example if it is equals to I need to match what I'm going to do match it with the next one after consuming it pick next will not be this I'm already I'm already doing your advance that means I'm getting the first character which is star I am proceeding then I need to check peak this is confusing Dig it. Let's dig it. Let's dig it.
There was sword and identifier. No, no, no. Not now. Not now.
That's the case of exclusion.
Also needless match. It is match.
We don't need peak next. We need match.
What match is going to do is match it with the previous character.
If I'm not wrong, it is going to match it with the previous character, not the next one or the current one.
Is it? And it's going to return false character at current not equal to expected expected.
The character that we are matching if it is not equal to expected we are returning false. If it is expected true it's a it's a simple function.
So you can make this simple function of type um bool. Yeah, boolean was the type and we need scanner colon match. We are going to match See how is the poll going on. There are 37 votes and most of the votes are in favor of Python.
Most of you people like Python and not C++ also JavaScript is only 11% also Java.
Damn. Damn.
That's good. That's good. That's good.
Let's go. Let's go. Let's go. Let's go.
Let's go for match. I'm going to pass the character character of the expected test name if it is not the if it if is at end then I'm going to return false already return false and if the uh src the str current position if it is not equal to this expected then I'm going to return false return false for this also else what I'm going to do is I'm just going to return the uh Uh it will be current++ also return true.
Let's check it again.
If we are at the end uh return false definitely if we are source current expected if we return false it is good this function is going to work fine now we yeah there's um then we need to do the match Um we're going to match if match if match occurred which will be of type equals to yes that means it is equals to it is a comparator comparator operator then if it is not then what I'm going to do is else.
Uh if it is not then we're going to add it to the tokens tokens push back which will be your flag token and I'm going to tell him that oops see I deleted uh push back uh token then what I'm going to tell this If token type is token type is equals equal and the value is nothing.
Expected this uh after this semicolon then we have a break.
break also need a parenthesis and going to remove this.
There's no issue.
This is a comparator.
Then the tokens got pushed back to tokens.
Tokens I say tokens token token token token type token type of uh it was equals equals equals equal the value will be value will be nothing.
This is the case that I handle for equals. Now I also need to handle this for greater than and less than. I'm just going to copy this one and paste and paste again.
Good thing is if we have this, this and this we need to match this uh greater equals it will be will be less equals. It was this is for less less equals. So I did something wrong.
Oopsie.
Let's go. Let's go. Let's go.
It was equals uh equals equals equals uh less equal else it is going to be a just a less sign.
It will be less only.
And what is the issue with this code? It needs a semicolon. Semicolon here.
What is the next thing?
live just whatever finish.
Okay.
Push back matchals toals check.
Okay.
One more case.
Then I need this for zang.
Bang bang bang. It will be bang equal.
It's going to be bang equal else it is going to be a band.
Yeah, a simple band.
It will be bang or it will be equal. It will be less. It will be greater.
Greater greater.
Why that? Greater else it is going to be a greater.
So many cases we have handled.
There are so many cases with the help of switch case. It just looks good. It looks good.
If it was supposed to suppose it was uh if else case then it it would have looked very ugly.
What is the case?
I'm going to keep it at The last side the cases that I have handle is plus minus slash I have bang bang equal greater greater equal less less equal equal equal equal equal all these cases have been handled then we have left per right all the braces these cases have been handled Okay. So I also need for dot comma and semicolon. Semicolon is done. I need one comma and dot uh dot comma semicolon. Okay, these are done. These are also done. What I need is these one.
I need to parse the strings.
I need to pass a strings.
This digit. This digit. This digit.
Something is wrong. Bank equal column value. Oh, bang equals.
Is there something wrong?
Bang equals. It is going to be equal not equals.
Now it is okay.
The next error that we have is of type match. Scanner match.
Where is gonna match boolean match care?
It is match care. My bad.
What I'm going to do is just replace this one.
So most of the errors are fixed now and we have match match match too many arguments to function call expected zero because I have not given any parameter here which will be care of type expected expected type.
So what is the next issue? We have return type return issue because of semicolon missing fixed.
So the whole file is fixed. It's working. That is great. It's great. It's great.
I guess I need to test the file.
Where is my file? Where is my file?
which is this DC sc I'm just going to check these will be comments comments in my file and comments in my file.
So we have characters now we have parenthesis we have different different characters which is star exclamation and also curly braces.
Let's check um let's try for what was that slash and then we can have a space. I'm going to add some uh what was the operator? It was minus.
Then we have plus again.
I need to check if I am able to pass these.
Is my scanner going to pass these characters? We are going to add in token. Scan tokens.
We can scan tokens. Um what the current length Okay, then we are good to go. I just need to call the function in this function scan tokens that is scan token and this is scan tokens.
So inside scan tokens src it was it was scan token.
I don't need to pass anything while the current is greater than equal to src string.length.
Um so I'm just going to scan.
Is there any other thing that I can do?
I think not.
Let's just try compiling this file.
Comment in my file. What is this?
Okay, it is because of the main file because I have a stdc out. I did it wrong.
I need to include the scanner file in this file.
I need to include this file include which will be of scanner scanner scanner.h not edge but cpp.
Next, we need H file.
It's going to be inside this. Wrong, wrong, wrong, wrong, wrong, wrong. My bad.
Now, I need to scanner. Scanner.h.
What I need is a uh object for my scanner class.
I'm currently I have not uh you know I have not uh studied up to the parsing part. I have only understand how to do the scanning and how to create tokens, how to make tokens. I'm not sure about the parsing algorithm right now.
Now currently I'm following a book which is called crafting interpreters and I am uh trying to understand all these things that this book have explained and I'm trying to implement those things inside my code.
Here things are explained in Java in the Java language and I am trying in C++ because I like C++ more.
uh currently I'm not sure about parsing algorithm which algorithm I'm going to use So what I need is scanner get scan object and let's give it scan scanner.
We need to pass the um we will pass the file string. Yep.
File. It will be of stype.
I hope there is no issue. But there is some kind of issue.
I need a pointer. Yeah.
And then what I can do is the next step that I need to do is scan tokens.
This was the function that I need a peak. Next peak uh I use peak next year I can use match match match of this type it gets a match.
Now I get a very strong feeling that if I am going to run my file I'm going to have some runtime errors.
Let's just see scan um of scan tokens.
Is this going to work?
Is this going to work?
Try running map file.
I knew it.
What is this? No me no such file director and you can reference to scanner scanner this I forgot which file do we need to implement it wash or ctp it wasp okay code is compiling without any issue the whole code is getting comp compiled without any issue without any runtime issue without any compilation issue.
Now what I need to do is I need to check tokens.
But the worst part is how can I print all the vector of token type.
This is the type of tokens.
We have tokens. How many tokens do we have? What I can do is I can directly at least one one small thing that I can do is I can ask for scan tokens. It's only going to scan. I cannot access those variables.
What I can do is I can add a function.
Of course I can small function just for uh debugging purpose which will be which will be which will be um void. Yeah, it will be void and DVG DVG DVG.
What is the time?
Okay, it's so already getting late.
I guess I have done enough for today.
Able to make those tokens.
And so the first part of the series is done.
Today I was also only going to cover the the the lexing part uh to which is based on how to make tokens from the file that I have given as a input and I guess I'm done with this.
Okay. So I'm going to continue this in the next stream.
The part for today is done.
Anything anything left? No, I don't think anything is left. So, thanks for watching the stream, guys, and thanks for liking, thanks for subscribing, and thanks a lot for everything. So, let's see you guys in the next stream.
Okay, bye-bye.
Um if you guys got any question any question to ask just uh drop comment in my videos uh in any video drop me a comment I'm going to answer your question. If you guys have any question related to you know any computer science or career just anything okay see you guys in the next stream. Bye-bye. Tada.
Related Videos
Agentforce NOW AMA: Build with React and Salesforce Multi-Framework
SalesforceDevs
490 views•2026-05-28
How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust
aiDotEngineer
450 views•2026-05-28
WEB TECHNOLOGIES UNIT-2 | Degree 4th sem BCOM Computers web technologies unit-2 full explanation💯✅
LearnwithSahera
1K views•2026-05-29
More tests are always better? How to use AI to identify tests that bring little value
Alliance4Qualification
335 views•2026-05-29
Search Algorithms Explained in 60 Seconds! 🤖💨
samarthtuliofficial
218 views•2026-06-01
People of Game of Thrones using JavaScript DOM
AltCampus
296 views•2026-05-30
Introduction to Problem Solving Part - 1 | Lecture 1 | Intermediate DSA
ascensionix
107 views•2026-05-29
🚀 BCS613C Compiler Design | Module 1 to 5 Schema Evaluation 🔥 | VTU 6th Sem 💯 #VTU #bcs613c #exam
Pranavaa-y4y
104 views•2026-06-02











