PyTorch Transformer: Part 4

Added: 2026-05-31

[00:00:05]Hey, how's it going everyone? Welcome on in.

[00:00:08]>> Happy Thursday already. We're getting close to the weekend. We're going to be continuing our transformer today. We've got really close. We basically ran into a bug yesterday where we messed up some things in our dictionary creation. Our tokenizer, our tokenizer got broken. We fixed it. And now we're going to be doing the training. And then we're going to adding positional encodings. And I think that will basically mean that we're done with that. Wait. I think if we get to it, if we get to it, we can make it work. All right. Let me see here. Let me do a quick little share button on our Discord.

[00:00:47]Over to main. Here we go. Oo, nice patent. Oh, we a patent pilot quoting idea. No priority. Does anybody have advice or suggestions on me? Uh this is a good idea. Someone mentioned previously that legal zoom would be an option. Legal zoom may be a good place to start for patent. All right.

[00:01:12]Let's see here at everyone. There we go.

[00:01:18]>> All right.

[00:01:20]Okay. Let's get started with some coding. Ah, this is my favorite part.

[00:01:25]All right.

[00:01:28]accelerator.

[00:01:30]>> Here we go.

[00:01:32]>> Okay, so we've got Where do we go here on? So, we basically we need to do this.

[00:01:36]We should get some more data. Upgrading the dictionary support to betterment. We don't need to worry about that right now. I don't think we don't I think we're pretty fine without it. Training data for generator based on our input. Wait, hold on. Training data generate based on our input. Oh, right, right, right.

[00:01:55]Okay. Yeah. Yeah. Hey, Seek the Vault.

[00:01:57]Good to see you. Welcome on in. Happy Thursday. We're working on our transformer. We basically mostly got it done. We just got a few things to do.

[00:02:06]We've got a few items left. We did their masking. We did our tokenizer. We did the the the multi head. We did the target parameters. We built the dictionary, the tokenizer, which we obviously this is kind of the same thing. And this is where this is where we had our bug yesterday. That's where we had a problem yesterday. We fixed that. This is an identical item. We duplicated that. Okay. So, we need to get our training data. And then we need to do a positional encoding.

[00:02:39]So, let me do positional.

[00:02:43]This is the one I'm excited about because I think we can do a pretty good job on this one. Hey, Just Human.

[00:02:48]Greetings. Good to see you. Welcome on in. Happy Thursday. Hope you have a good week so far. positional encoding.

[00:02:55]All right, we're going to do defaf in it. We're just going to copy and paste this for the most part. And we don't need we basically need is it what is our we need to match dims. I'm just going to call it dims cuz this is what I've been tracking. I've been tracking I'm gonna be calling it dims basically.

[00:03:15]>> There we go. Okay, now I'm curious. Uh what are we doing? Stephen here. Hey, how's it going there Deepac? Good to see you. Happy Thursday. We're building our transformer. We're building our trans.

[00:03:26]We're going to make Chad GPT. That's what we're doing. Although, we're going to make like a little mini Jet GPT cuz I don't want to like just spend a lot of money on training a large a super big model. We're going to train like a mini model, like a little mini model, and see if we can get it to do something very specific, right? So, that'll be the plan for that. Once we have completed with the initial transformer. So this is our model right here that we wrote earlier this week. We did this model right here. Really simple straightforward. This layer specifically I would like to implement myself.

[00:04:02]This is a nice layer that is pre-built for us in PyTorch. The good part is I think we can build our own. I think we could cuz the part's pretty straightforward I think. Okay. So, and we don't we don't have to do like a whole bunch of like optimization things like for the number of heads, we don't really need to split the dimensions to parallelize the processing, right? We only need an encoder. We don't need it.

[00:04:26]Well, we only need a decoder. We don't need encoder or one way or the other. I we only need one. Although, it doesn't really matter because it's pretty much identical architecture.

[00:04:34]It's just how you stack them. And we could add a dropout. We don't need to worry about that too much. So, your implementing attention is the key paper.

[00:04:43]Yeah. Look at where we got it right here. Attention's all you need. Look it.

[00:04:47]This is it. That's it. You called it.

[00:04:50]You called it. That's great. In a simple manner. Yep. We can We can simplify it.

[00:04:56]Exactly. How's it going there, Horser Hobbit? Welcome on in. Thank you for saying hello. Good to have you here.

[00:05:02]Happy Thursday. We're building Attention's all you need today. That's what we're doing today. We're building this right here. This like nice diagram.

[00:05:12]It's a nice one. So, we've got quite a few things to do in order to achieve our objective. I think we could I think we can do it, though. I'm pretty I'm pretty positive.

[00:05:22]>> I want to do this. I think we could do positional encoding today. We might spend the entire day on this because it's quite a bit. This could be quite a bit cuz there's a lot to consider here.

[00:05:32]>> You're no scholar, but even heard about this paper. YOU DID? NO WAY. ALSO, hi Steven. Good to have you here. Hope you're having a great week so far. We're almost to the weekend, you guys. Should I use Premiere Pro or Cap Cut?

[00:05:46]>> Cap Cut is free, isn't it?

[00:05:49]>> Why wouldn't you just use Cap Cut?

[00:05:51]Right. There's Cap Cap Cut's free.

[00:05:54]>> Premiere Pro is going to give you a lot more features, though. And it's going to have a lot more major industry thing.

[00:05:59]Hey, thank you for the hearts.

[00:06:00]Appreciate it. Hey, look at that. I see little floaty heart icons there. Yep. I see them throwing up. Thank you so much.

[00:06:06]I appreciate it. I can see when you guys do that. Yes. That's really exciting. So excited for you. All right, more power to you, sir. Thank you. Fantastic. All right, this is great, you guys.

[00:06:17]Appreciate it. I like that. All right, so we're going to do some positional encoding and then we need to do the training itself, which I think we can almost do. We're like really close. We need our criterion, which we can put here, right? Can Can I just do that? So, we got our criterion right here. I think so. I think so. Thank you for all the hearts. Appreciate it.

[00:06:38]Okay. Thanks, man. Man, you made my Oh, yeah. made it clear. Love from India.

[00:06:43]Hey, Ankit, that sounds great. All right. Happy to hear it. That's fantastic. Thank you. Thank you for hanging out here. Okay, so we've got our criterion. Let me see.

[00:06:57]Let me change the music real quick.

[00:06:59]Okay, there we go. Let me see if we can run this Python transformer.

[00:07:06]Okay, you got a sub, man. Thanks, NK.

[00:07:08]Oh, that's great. Hey, you joined the right channel for software engineering.

[00:07:12]What we're doing here is we code by hand on our keyboard. We don't use those AIs for that much because we want to learn.

[00:07:18]This is learning. You got to make sure not to have someone else do the work for you. Otherwise, right, you won't learn anything. Oh, hi Mr. Bumpy Pickle. Good to see you, Steven and France. Good to have you here. Hope you had a great week so far. Like and subscribe or else I see that.

[00:07:37]Thank you, Mr. Bumpy Pickle.

[00:07:40]I appreciate it. I appreciate it. Torch has no module cross entropy loss. Yeah, it does. Wait, is it torch inn?

[00:07:48]I don't think it's in, right? It's not in. It's cross entropy loss, right? Oh.

[00:07:53]Oh. Oh, wait. No way. Really? Hold on.

[00:07:55]Wait, wait, wait, wait. I feel like I should know that. Python. Let's see.

[00:07:59]Import torch. We need We need to be like you, dude. You made my day. Oh, and kit.

[00:08:05]Whoa. Hey, that's great feedback. Thank you. Appreciate it. That's so nice for you to say. Hey, Krishna, good to see you. Hello, bro. Happy Thursday. Hope you're having a great week. Today, we're programming and we're doing the best thing, the best thing ever.

[00:08:19]Torch.n.cross entropy loss. Hey, it's right there. All right, perfect. Perfect. Okay, so it really is only available in the neural network. Okay, sounds good to me. All right, let's rerun that and see where it's going. Hey, Sergio, welcome on in.

[00:08:39]Good to have you here. Happy Thursday, Sergio. Hope you're having a good week as well. All right, so I want to let's see if we can get one loss. Let's see if we can do a loss. I don't know how we're going to do it. I don't know how. What we need to do is we need our target data. We need our target data.

[00:09:01]So, we did an input, right? Where's our input at? Where' we do our output model transform model train and then we got our output. Here we go. Here we go. This right here. Okay.

[00:09:13]Let me move this here. So we got optimizer. Then we're going to do through epochs.

[00:09:19]So this is going to be our input and output data.

[00:09:23]Bro, what are you? Uh oh. What are you man? What are you man? Wait Krishna, what do you mean by that? I'm a software engineer. Wait. Uh, even more. If you guys don't know, CTO at PubNub, we raised 134 million series 1 billion devices and we connect a lot of devices, you guys.

[00:09:43]He's a goat. Hey, Mr. Bumpy Pickle.

[00:09:45]Thank you. Appreciate it. Good to have you guys here. All right, so this is what we're going to do today.

[00:09:55]We're going to be optimizing our model, which means we're training the model.

[00:09:59]And we're going to do it for so many epochs. How many epochs? I don't know.

[00:10:03]Uh, epochs equals, we'll do like 10. No, we'll do 100 cuz we have not that much data. For epoch in range epochs. Hey, squirrel. Hey, squirrel squad. Hey, it's good to see you. Welcome on back. Good to have you here.

[00:10:20]Happy Thursday. For epoch in range epoches, we're going to say print epoch.

[00:10:30]All right.

[00:10:32]Then I want to run this through on the output. Here we go. Print the output shape words equals dictionary decode output. We don't need this right now.

[00:10:49]And I'm going to indent it anyway. So here, this is what this is the this is the this is the magic shot right here.

[00:10:55]This is the magic right here you guys. This is the magic.

[00:10:59]This is what we want to make it work.

[00:11:01]Sorry, bro. Typing error. I'm asking you, what are you doing today? Oh, what are you making? We are building chat GPT is what we're making. You've probably used an AI to do some work for you before, have you? You've probably everyone's done it now. So, we're going to be building our own transformer, our own language model, you guys. That's what we're currently doing. And our language model is only going to know a few things. It's it's going to know just a few things. Just a few. And we'll be we'll we'll make it we'll make it work is what we want to do. I want to learn the algorithm. That's what we are doing.

[00:11:37]For the most part, I understand about 95% of this for for example the nuances between encoding and decoding here and what the difference is. So your input and output you need to put both in at the same time. I'm not I it that I have that part is weird. I get it. I understand. I just don't fully get it.

[00:12:03]Hey Twilight, good to see you. Don't forget to drink water, bro. Oh, we got the water right here. Thank you for reminding me. Apparently, it's important. It is cuz you can have problems if you don't. I tried not drinking water for a while.

[00:12:17]Have you tried that? You can do it, but you you It'll be hard.

[00:12:25]And it was a mistake. Whoops. Make sure to drink your water.

[00:12:30]Okay, here we go. So this is probably going to fail because the training data is is just words and the output is going to be numbers. So hey Mark Lemon, good to see you. Thank you for clicking the hi button and welcome here the Thursday.

[00:12:49]Happy Thursday.

[00:12:51]Good to see you.

[00:12:53]All right, lost criterion training data we need. So here's the problem. The model in my model allows me to input full on text data. So full-on text data, right?

[00:13:11]The problem with that is that I don't have a way to capture the correct encoding essentially, right? The the correct embeddings, the vectorzation part, the part where we turn the words into numbers.

[00:13:29]This is where I need help with really quick.

[00:13:33]If I'm going to be passing data through, I don't know what the output's going to be. So if our output is this, which are pro probabilities, if there are probabilities of which number it's going to be, we need to capture that.

[00:13:53]We need to capture that in a way so that way we can we can train what the actual the actual expected output is going to be. Right?

[00:14:03]So that that's what that's what I need that's what I need to fix right here.

[00:14:07]This this one little part it's like one variable right here is going to be a little bit of effort.

[00:14:16]Just a smidgen of effort.

[00:14:19]So how do we want to go about that is my question. Let's think about that. So it's words. We got to convert it into tokens.

[00:14:28]And then once we convert it to tokens, we need to turn it into probabilities.

[00:14:35]But the good news is the probabilities are going to be one hot, right? They're guarantees.

[00:14:41]They're full-on guarantees.

[00:14:45]So it's more code than I have written right now. So we need to capture this convert it into a So we need to we need to one hot this essentially right here.

[00:14:57]This needs we need to one hot on that.

[00:14:59]Where am I going to put that? How do I want to do that? So I I probably want to put it in the model the dictionary. I have a decoder here. So what if I have So I've got my normalize tokenize. Yes.

[00:15:13]Oh, this actually I already have it. I have tokenizer here and then it needs to go to one hot after that. So I need to one hot my tokenizer.

[00:15:22]All right. So defaf one hot self, tokens. So it has to be tokens.

[00:15:33]Okay. Then I need to convert that into one hot. Now I think torch has a built-in onehot. We could build this ourselves. So basically we do torch.0 zeros, create all zeros, and then we update the index where the token is the value at that index.

[00:15:51]Hello. Hello. Hey, Bogoteers. Good to see you. Happy Thursday. Welcome on in.

[00:15:55]Hope you're having a great day so far and a great week. It's almost the weekend. All right. So, we could I want I want to see if I can use a native pie torch. All right. Pie torch one hot. Is there Is there a one hot? There is right here. There's the one hot right here.

[00:16:13]Hey, look at that number of classes. OH, IT'S ALREADY READY for us to use. Hey, that's what I like right there.

[00:16:22]All right. Good to hear it. Good to hear it.

[00:16:26]So, we can just do functional one hot and then we build. We We already have the tokens. We already have the tokens.

[00:16:37]So, we we should just be able to do this function real quick.

[00:16:41]This might not have been as hard as I thought it was going to be. We We could do this. So, we could do this. So, return torch.nfunctional one. Was it one hot one_h hot tokens?

[00:16:57]Is that it? Is that really all it was?

[00:17:00]You think?

[00:17:04]All right, let's see. So, we can't do that yet. So let's do target uh target uh labels target. How do we say this? If this is this is the this is the answer. This is what we're looking for. How about if we say targets t target targets equals training data and then we say dictionary oneh hot like that. Is that going to work? You think that'll work?

[00:17:34]They ripped out the entire crypto API that my entire Linux kernel patch was based on. Oh, Bonzupi. That's right. You mentioned that yesterday. They did. So, my sanity took another heavy hit. Oh, no. That's terrible. Wait, but why?

[00:17:52]Everyone wants the cryptography library to be there. There's a lot of standard cryptography libraries. Why did they remove it, you guys? Hey, Ronnie, good to see you. Ronnie in one, welcome on in. Happy Thursday. How are you getting the subtitles on YouTube live? Isn't that great? Right there. Technology I wrote many years ago right here. You can grab this yourself. It's for free. Free forever, you guys. Free forever. It's a common question everyone always asks.

[00:18:18]So, I have the tab pinned. The tab is pinned. Also, it's here in our Discord channel. So, if I go to link share, I believe it will be around here somewhere. Here it is right here. So, if you go on our link share channel, it'll be right there on Discord, you guys.

[00:18:38]No, yesterday I found the update that just refactored the network stack API.

[00:18:42]Ah, got it. That's what I was complaining about yesterday. Okay, got it. Got it. And so now they've officially removed the crypto API for your entire Linux kernel patch. Oh, that's terrible. Bunupi, you found an update where they just ripped out the crypto API and replaced it with a new one. A well, okay, that makes sense. I understand they What was I'm actually kind of curious. What did they choose? I know myself some crypto. I know a little bit of crypto. When I say crypto, I mean cryptography, right? I mean, obvious and then bitcoins or other things. What you think of like cryptocurrency. In this case, we're talking about cryptography, right?

[00:19:21]It's a misnomer. Did you know there is no encryption?

[00:19:26]There's no encryption in cryptocurrency.

[00:19:29]It's that Wait, you're That's counterintuitive. There's signatures.

[00:19:33]That's why it's called crypto. There's signing, cryptographic signing, but there's no encryption, per se. If you didn't know, now you do.

[00:19:45]Now you do. All right. So, is this one not going to work? Print. I want to print the targets. Targets.

[00:19:54]Targets. Whoa. Target. There we go. All right, we got our targets. Print targets. Is that going to do the trick?

[00:20:02]I don't know. We're gonna find out. Here we go. Here we go.

[00:20:06]Uh, tokens. Oh, it doesn't like that.

[00:20:08]Why not? Uh, argument input one must be a tensor, not a string. Oh, wait. Oh, right, right, right, right, right. Oh, right, right. Okay, we have to go a step further here. Uh, we need to say, see, not tokens. Let's say let's see see words tokens equals self dot do we normalize and tokenize right that's what we got to do we do normal and tokenize do we automatically normalize wouldn't that be better if we did right I think that would be better if we did subn words so for each word for each input sentence today. Oh, they removed the monolithic PCBC crypto API from cryptography subsystem.

[00:20:59]And for the RXRPC subsystem, they implemented a simplified version of the same cipher in a way that breaks everything I did. Oh, they could have kept the API compatible. They could have for that. They could have just replaced the underlying cipher if they wanted to.

[00:21:16]So many selfves. I know. What does it do? So the self just refers to the classes instance. So you see here we've got a dictionary class definition. We can make an instance of it which just means like creating a copy of it kind of cuz this is kind of like a template.

[00:21:33]This is like saying hey this is a code structure template let's create a copy of it in memory. So I create a copy of it in memory right here calling it a dictionary right. So I create a new dictionary and all the self is going to reference the dictionary's instance itself, right? So it means this dictionary right here. That's what that means.

[00:21:57]So in a way you're starting over. Hey, oh well your implementation simpler now that they have a crypto library that is uh local to the network stack that you're working on. So okay, well that's that's like silver lining, right? Where it's just pretty good. is a little good.

[00:22:14]Okay, thanks. All right. Hey, how's it going there? Uh, Vinnet Vinetkumar. Hey, Vinnetkumar, thanks for saying hello.

[00:22:21]Good to have you here. Happy Thursday.

[00:22:23]We're building technology. Check it out.

[00:22:25]It's so great. This is my favorite thing. It's my favorite thing, you guys.

[00:22:30]All right. Also, for those of you who are new here, we do have a hackathon going on right now. Visit our Discord and the hackathon channel right here. Go to blocks.ai, AI create an agent and then submit it to the Discord and you'll be entered to win.

[00:22:46]You'll be entered to win. Oh, Vinnetkumar abs. Yep. Glad to have you here. All right, so we need to finish writing our one hot with the words. We're going to say tokens.normalize norm and then we're going to pass in my words. And then the tokenize is not is still words, but it's normalized words.

[00:23:12]So, what I think I want to do is simplify this code really quick because I'm always normalizing before I'm tokenizing. And I think that's necessary. I think it's necessary, you guys. Which PC are you using? Oh, I've got your answer right here. Are you ready? Check this out. You name Darwin right here. What does that mean? I'm running MacOSS. I'm running Mac OS. Yes, this is a Mac. You got it. It's a Mac, you guys. We're doing a Mac here, which is very Unix based. That means that I get to have all of the wonderful terminal interfaces that I want without having to have any emulation layer, basically. That's what that means.

[00:23:54]That's what that means.

[00:23:57]So, what do you how about you? What do you use? What do you guys use? Hey, Oxno. Good to see you. WHAT'S UP?

[00:24:03]WELCOME on in. Good to see you back.

[00:24:04]Happy Thursday. We're getting close to the weekend. All right, you guys. We're going to curious about this. Let's take a look. What OS do you use? Question mark. Do you go You go Windows. Wait, I got to spell that right. Wind. Yeah, that's I spel I spelled that right.

[00:24:20]Windows. You Mac. Let's add some options here. We go Linux, right? Or other slash Arch. I don't know.

[00:24:34]like I don't know maybe maybe like other I don't know what other it's just let's see what you guys say I'm curious about it I'm thinking Windows is going to be number one I think Windows might be number one it is I I'd be curious to see if anyone's running a Mac all right Darwin all right so I want to update this here to do I'm going to double it's okay to double normalize actually it's perfectly fine so I'm going to say words equals self.norm words. All right. So, now we need to optimize. Now that I've normalized that, we only need to say tokenize.

[00:25:13]Tokens. Tokens. Boom. Done. Did we fix it? Yeah. I think so.

[00:25:17]You're good. Currently, use the old Intel Mac. Oh, wa. No way. That's crazy.

[00:25:24]Hey, check this out. I've got that right here. Wait, wait, wait. I'll show you.

[00:25:31]I'm not using it right now, but it's the old Intel Mac right here. The old Intel Mac. It's Intel. It's got an Intel right here.

[00:25:42]I got it. It's just sitting over here on the other desk.

[00:25:48]But we got it. It's a 2019. It's a 2019.

[00:25:52]It's the one with the Mac with the Intel inside. Okay.

[00:25:57]All right. So we normalize here on our tokenizer. I also want to see if we do any other norms here. We don't need to do norms anymore. So what we do instead, we just remove the norm. We get rid of the norm as well. And boom. Simplified.

[00:26:12]Done. Easy. Easy.

[00:26:15]Finally, someone else. Ah, right. I know. It's crazy. I mean, it is like a seven-year-old system. Although at the same time, that's still really powerful.

[00:26:25]Same as 2019. A. All right. Very nice.

[00:26:29]That's awesome.

[00:26:32]That is amazing. So, we uh we do need to do this, but we're going to deal with that later. Okay. And then uh transformer. We already got this. We're using a swish glue, which is pretty neat. Where is this? All right. So, I need to see my normalizer here. Word list normalizer. Okay. So, we do need to have it as a separate function, which is nice. And we got to keep that there. Okay. Bonzupi. Oh my lord. They bricked the protocol. Oh, what? No. Wait. Why' they do that?

[00:27:03]That's crazy. Don't break the protocol.

[00:27:05]That causes problems. That causes problems. Why would you do this?

[00:27:12]Got to brick. Bricking is not a Hey, Lun. Good to see you. Welcome on in.

[00:27:17]Happy Thursday. We're almost to the weekend, loons. We almost made it.

[00:27:22]Did you submit your your hackathon project yet?

[00:27:25]You know, I haven't seen too many being submitted. You might be the only one.

[00:27:30]You might win first place.

[00:27:33]Ben Kater, good to see you. Welcome on in. Happy to have you here. Happy Thursday. It's Welcome back. Ben Kater's here, everyone. All right, so I think I got everything I want here. Let me switch up the music really quick. Let's go to the next song. Let's try this. Is this a good song?

[00:27:53]This is a good song.

[00:27:57]I think we'll listen. We'll try it.

[00:27:58]We'll try it. You just installed Chinese Linux on a VM. What is Chinese Linux?

[00:28:04]What is that? I don't know what that is.

[00:28:06]What could that possibly be? I'm curious about it.

[00:28:09]Let me know.

[00:28:12]What could it be? Let me see if I can scroll here. All right. Perfect. Okay.

[00:28:16]Think of switching to framework 13. Ooh, nice. Sadly, they don't deliver to Saudi. Oh, no. Oh, well, you could probably have it like delivered through some sort of like hopping or something.

[00:28:30]It's really cool UI. Oh, is it? Does it really? Oh, does it? Okay. Deep in Linux. Oh, is it deep in Linux? Is that what that is?

[00:28:40]All right. Normalize. Normalize. We're looking good there. Okay. I think we're in the right spot now.

[00:28:47]I think we are. All right. Things are looking good here. Let me see over here real quick.

[00:28:55]This just smidgen. There we go. Okay.

[00:28:59]All right. Looking good now.

[00:29:04]Oh, Clint OS. Or is it Klein OS? Let's try this. C K Y Kalen. Kalen Linux. Is it Linux Windows clone? Oh, wa. No way. That's crazy. Look at that. It's Linux. Wait, wait, wait, wait. How are we?

[00:29:23]>> I don't want Wait, wait, wait, wait. Uh, is there just images? So, wait. It's basically Windows, but it's Linux. Is that what that is? It looks like the Windows interface, but it's running Linux.

[00:29:36]Never heard of that one. How is it?

[00:29:40]I can ask their support. They don't recommend it because it might break while shipping. Oh, okay. Got it. Yeah. Yeah, that would be that would be terrible.

[00:29:53]Go to try deepen. All right, try deepen Linux.

[00:30:00]Oh, hey, it's all it's very similar. It looks like a Mac kind of kind of got that Mac type interface. All right, so you've got Oh, neat. So deep in Linux kind of like userfriendly Mac or you could go with the other kind of Linux which looks like a Windows.

[00:30:20]Perfect. Then you never have to worry about anything else.

[00:30:27]I bet it phones home to the CCP. Yep. Uh it probably lose. Oh, Ki Linux Ted. Yep.

[00:30:34]Kali Linux for the penetration testers.

[00:30:38]The pen testers. You guys, you got your Kali Linux here that's got all the pre- nicely builtin utilities for testing security. Kali Linux. It was featured on a popular TV show called Mr. Robot. They had Kali Linux on there.

[00:30:57]Omari. Oh, wait. Is the Go distribution.

[00:31:00]Is it Omari? Wait, is Omari like Omar?

[00:31:03]There's Omar. Omari versus Omari. Is that separate?

[00:31:10]Omarky. Omarky. So, is that just a misspelling? Mr. Robot's cool. It was a great show, right? I like that show.

[00:31:19]Thank you for clicking the hide button, Linds. I think you meant Omari, right?

[00:31:25]Black Arc is better than Oh, wait.

[00:31:27]Bonzupi. I haven't heard about that one.

[00:31:28]Uh, Black Arch. Let's see here. Ooh, we've got an upgraded version, you guys.

[00:31:35]This reminds me of the early days. Oh, look at that. This reminds me of like the 2000s era. Wow. This this aesthetic like way back 26 years ago.

[00:31:50]No, you you misspelled it. All right.

[00:31:52]You misspelled it. Okay. I was wondering. I'm like, is there something?

[00:31:54]Is there a new one? Cuz you never know.

[00:31:56]There might be a new one. There might be a new one, you guys. Fedora. Great. I love that. I like I like Fedora. That's the yum, right?

[00:32:05]Fedora is uh for for the package manager.

[00:32:10]Fedora package manager is yum, right?

[00:32:14]Oh, wait. Oh, it is yum. Yeah, yum in Fedora.

[00:32:21]Okay, let's keep on going.

[00:32:26]Okay, how's it going there? We we a gamer. Hey, we a gamer. Welcome on in.

[00:32:32]Happy Thursday. Hope you're having a good week so far. Thank for clicking the high button. Okay, I want to see if our one hot worked now. Let's see if our one hot worked. Okay, did we make it work? Oh, look at that.

[00:32:46]It worked. Hey, we're on the right track now. Now we can do some loss detection.

[00:32:53]Oh, looking good. Yep, that's it right there. That's the correct answer.

[00:32:58]However, it doesn't have Oh, no. No, it has the right one. Okay, so we got it.

[00:33:01]We got it. Okay. All right, we're good.

[00:33:03]We can start training our model now.

[00:33:06]It's going to be great.

[00:33:08]Good. Pen trashing testing. DRO. Also, Parrot. I've heard about Parrot. I heard about Parrot. It's been a long time.

[00:33:14]Parrot OS.

[00:33:17]Parrot security. Yes, I've heard about this. In fact, we've been to this website because I've already zoomed in on it.

[00:33:25]I remember this OS. I've never used it.

[00:33:29]It never came up. I always went to use different distros. Ah, there's different ones, you guys. You can also bootstrap the the repo with vanilla arc Linux. So, you can keep your regular daily driver set up and install the pentest tools you need for whatever you're doing. That's right. You can just install the pentest tools directly in any any of your Linux.

[00:33:49]You don't need to install any of the dros that come pre-installed with the with the pentest tools. It is nice though to have a pentest USB, right? So you have a full ISO and then you flash your USB and you just plug it in.

[00:34:03]Klein 11 is paid just like Windows. You got to get a license key, but it isn't necessary. And there are Open Klein, too. Oh, you can get Open Klein. Hey, I like the free option. Very nice. PNG Hunter. Uh oh, PNG Thunder. Hey, PNG Thunder. You got yourself a file extension. That's an image. An image Thunder. Good. Thank you for clicking the high button there.

[00:34:27]Have you been following your content for a few days? Hey, great. And man, you're very underrated. Well, thank you. Your content is JavaScript.

[00:34:36]Wait, wait, wait, wait, wait. I'm pretty sure JS has a different a different meaning in that context. I'm guessing that is a positive. I think it's a positive compliment. Thank you. Your cunning is gold. Thank you. That's so nice for you to say. I appreciate it.

[00:34:50]We're building ourselves a chat GPT today, you guys. We just figured out our one hot as you can see here on the screen. That's exactly what I want right there. That's exactly what Now that it's one hot, we can train our model cuz we got the logs. We or the logits. They're called logs. I called them logs when I didn't know that they were called logits, right? So logits. Let's see. How do you pronounce?

[00:35:15]Let's see. I'll show you. All right.

[00:35:16]Let's do pronunciation. Pronuniation.

[00:35:24]Logits.

[00:35:26]Logits. Oh, so it's kind of like a log.

[00:35:30]Oh, wait. Yeah, log like lodge. Oh, logits. Wait, ski lodge. Wait, no, wait.

[00:35:37]How do you guys pronounce this? I'm so confused.

[00:35:41]Use the soft G sound. Uh, rhymes with the word like gadgets, logets, widgets, logits, logits, logets, logets. All right, there we go. You tried installing Metas-ploit on Ubuntu.

[00:35:59]No, wait, you have it. Metas-ploit is like a really nice pre-built package that includes a whole bunch of pin testing tools all right there. It's a really good one. You use Ventoy on a USB stick which contains digital forensics distributions. Oh, nice. I haven't heard about that one. Ventoy OS. Is it Ventoy? What is this?

[00:36:22]Open source software tool created bootable USB drive with the ISO.

[00:36:28]You don't need any format on the disc over. You just drag and drop the files on USB and then they then boot them directly. You can copy many files, different folders.

[00:36:39]Vento will give you a boot menu. Select them. You can browse and you can boot them directly. Oh, that's great. I've never used this. I didn't know about it.

[00:36:48]A new bootable USB solution. Hey, very nice. Dylan, welcome on in. Thank you for saying hello. Good to have you here.

[00:36:54]Happy Thursday. We're talking about a whole bunch of logic, some pen testing, different operating systems, right? And AI. It's way too complicated for you to to say pronunciation.

[00:37:11]Log logets. It's logit. But you know, if anyone says it differently, like logits, like the word log, then I think everyone kind of knows what you're saying. I think they do.

[00:37:24]Ventoy is a bootloader, I believe. Uh, so it's a bootloadader and then you can put whatever files you want into it which is kind of nice which means you can create your own custom your own custom which is nice only the tools you use.

[00:37:40]Okay, get status get add get commit. We want to say I know I don't I want to say that we added train. See trainable target. See if that works. Push. Get push origin main. Okay.

[00:38:02]We might need it cuz I remember we we're missing a piece. Let me see this to-do item. All right. So, there might be a to-do item here. To-do. You might want to add the web search plugin to your Zshell. Search the browser direct from the terminal.

[00:38:18]Oo, I think I might want to do that cuz I keep popping in and out between Chrome.

[00:38:24]It would be nice if I didn't have to, right to do No, no, no, no, no, no, no. Wait, wait, wait. I want to There we go.

[00:38:40]Oxnol. I kind of want to do that. web search plugin into your Zshell.

[00:38:45]What would that look like? H.

[00:38:50]Okay. Okay. I'll think about it. I'll think about it. Do you have a preference? Do you have a specific one that you use?

[00:38:59]I want to use like a lightweight one.

[00:39:01]Thank you for the hundreds. Appreciate it. Thanks for clicking 100 over there.

[00:39:05]Whoever is doing it, I don't know cuz I can't tell. I can only see when the hundreds float to the screen.

[00:39:11]We need to we might need to split. So we need to trim start and end between the target and training. Right? So we need to trim this and training. There we go.

[00:39:34]Okay. That's a todo item. Let me put this at the top.

[00:39:38]There we go.

[00:39:40]Okay, we don't need to do it right now, I don't think. I think we'll probably be perfectly fine. No, it's actually named web search literally. Oh, it is really.

[00:39:49]All right, let me learn more about this web search.

[00:39:54]Uh, ZSH H.

[00:39:59]The term web search and the context typically refers to the web search plugin. Oh my Zsh plugin that allows you to run web search directly in your terminal by opening your default browser with a search query. Oh, okay. So it does it still pops me out.

[00:40:18]It still pops me out. So I mean it won't stay in my terminal is what this is. So it won't let's see allow me to stay in terminal or it only load browser open I'll say open browser it would be nice to be able to stay in the terminal only opens your browser okay yeah cannot display results directly inside my terminal which is what I want you know I could I could just use anti-gravity for that right so I could pop out over Here I could say cd empty and I do agy and then I could just run search queries right here, right?

[00:41:04]I could just do that as well cuz that's essentially Google's AI and then I'd stay in my terminal and I could just be doing that. I should get used to that. The only problem is it's it's not that fast. Yep. It's still people with all the two. Yeah, exactly.

[00:41:19]That's what I want. I want to stay in my terminal. It's my favorite. Doesn't it make you like excited to like stay in your terminal? I think it just is very it's very like heartwarming. It is probably because I've been doing this for so many years now. It's it's my IDE of choice. Okay. So, if I was going to say web search zsh, see it takes a little while.

[00:41:45]It just takes a while to load. and see if I had done that in Chrome it would have it would have given me the answer already and see it's just taking so long.

[00:41:59]All right. It did it did come up with some something though. Here using Oh my plugin easy if using you already using the Okay. Yeah.

[00:42:08]Here. Let's let's let's take it let's take a look at it though.

[00:42:12]Let's let's just at least dump it jump in here real quick. Do we have an example?

[00:42:18]La.

[00:42:21]I get it. I get it. Okay. Oh, there's a lot more to this. Okay. It's doing more than I want it to do. I just want a basic command.

[00:42:29]I could write my own. I could write my own right now. Why? Why? You don't need this. You guys want to build our own shell script real quick? You want to see how this actually works? I think we could do this.

[00:42:45]I mean, this is this is just a bit too much. If it's just going to open up a browser window, then why what what's the point?

[00:42:53]You should write a plugin for that.

[00:42:54]Parsing HTML in your terminal. Sheesh.

[00:42:56]Oh, yeah. That's a good point. Yeah. Uh, yeah, that would be pretty nice. I think we could probably do it though. I think we probably could.

[00:43:07]Okay.

[00:43:08]Oh, so we say Google web search. Okay.

[00:43:13]Yeah. So, I could just have a terminal command called Google that runs an open command, right? So, we could do that. Why don't we do that really quick, you guys? So, look. I mean, it's not hard. Look at this.

[00:43:31]Question mark search equals Q equals.

[00:43:36]How hard is that? How hard is that? It can't be that hard, right? Here, watch this. Open.

[00:43:44]And we say oxnull-r.

[00:43:49]Ready? Enter. Oh, wait. That should have worked.

[00:43:54]Uh, how about we put that in quotes?

[00:43:58]There we go. See, right there. Right.

[00:44:01]It's like that right there.

[00:44:05]Hey, how's it going there? 11 Malcolm, good to see you. Which free paid researchers are the best for to become AI machine learning engineer? All right.

[00:44:13]My recommendation is that you find all the bad terrible tutorials and all the good tutorials in a self-discovery adventure on your on your own. And this is important because you need to find the struggle and the failures in order to really learn. If you find all of the answers, you're not going to know the most important part, the part that lets you fail. Your failures are what solidifies your learning. If you just get the easy answers, you're not going to get the greatest possible learning journey. You're not It's not going to work. Are you hiring? Hey, yes, I am.

[00:44:51]Theo is light. Good to see you again, Theo. We are hiring. Yep. Just jump on over here. Go to pubnub.com. Scroll down to careers. Click on the careers link and then we zoom in. We are currently hiring some software roles and some looks like corporate council role as well. Senior solutions architect and a senior software engineer for AI and large language model systems.

[00:45:14]Damn, don't expose like that. Oh, wait.

[00:45:18]It did. Wait, I didn't. Google knows. Google knows already. I didn't know. Oxno. I didn't know that was going to happen. I was just giving you an example. So, like look, we could create our own alias, right? Watch this. Uh, I don't here. I don't know if it's okay for me to open my ZSH real quick, though. What we could do though is we can just type an alias in here though.

[00:45:40]Alias Google equals and then we can do a command, right? And then we do like money one, right? We do money one. Is that it?

[00:45:52]Omar. Hey Omar. Awesome. I'm drastically underqualified. Hey Pro swing my the best advice that any someone gave me. Swing for the fences.

[00:46:05]There you go. It's a term that means that take the shot even if you don't qualify. And if you fail, that's fine.

[00:46:12]Just swing. Swing. Swing for the fences.

[00:46:15]Love your show. I could watch it all day. Hey, Theos Light, that's great to hear. Thank you. Good feedback. Uh, we'll keep doing this, you guys. We're live every day now, even the weekends.

[00:46:26]Just kidding. It's okay. Hey, good.

[00:46:28]Good. I was worried for a second. I was like, "Oh, no.

[00:46:32]I'm a network ops guy." Ooh, nice. So you know about like network masks and cers right with the for slash32 to get that cider like rocks like there's nothing it has to be this IP address and it can be no others right what is slash32 you mean guys all right so we're going to say open wait did I already copy paste this yeah open and then we're going to put money one right there I'm going to put this in quotes I believe this should do the trick maybe uh I don't know. I might Hold on. Let's see here. How do we do an alias?

[00:47:10]Alias. Alias. Alias. Did we already close it? Not this one. Up here. Here we go. Let's see. URL. Yeah. See, look at that right here. We're We're doing that right now. That's exactly what that is.

[00:47:26]Look at that here in Turkey. Oh, I see. Wait. You see? What do you see?

[00:47:34]All right, let's learn networking together, you guys. This is pretty cool.

[00:47:41]See, I told you. Look how easy that is right there. It's so easy, you guys. And then you do money one, right?

[00:47:50]for the well the only problem is I need all the monies cuz if I if I want to type in naturally like say Google blah blah blah blah blah blah with spaces then I need I need to do more than just money one but for now let's do money one alias Google hold on let's see hold on a second alias next yeah here we go H how do I want to do this? Okay. See, the only thing is I think it makes sense to do we just do this? Do we put it in quotes like that and then we say money one? Will that work?

[00:48:39]Google help.

[00:48:42]Help does not exist. Google.

[00:48:45]Oh, that did something I wasn't expecting.

[00:48:52]What do you think about the theory about recruiters or hiring managers interviewing candidates at the first stage then leaving them hanging while they finish their batch of interviews already in let's see basically you get the interview and wait till they finish who's interviewing but that results you have hearing back.

[00:49:12]So yeah, they should it's what would be nice is if they just let you know upfront where you are. If they take a long time to respond that that's terrible, right? It's terrible. It's not good.

[00:49:27]Also, I have wanted to know what is the difference between the role of machine learning engineer and AI engineer or are they the same? I think they're basically the same. They're the same, right? Cuz you're using machine learning to train AI, right? You use machine learning algorithms to make AI. AI is the outcome. This is the way I define it.

[00:49:46]Everyone defines it differently. I think everything's machine learning algorithms and that creates the model weights and parameters, right? The biases. This gives you the output of AI, artificial intelligence. Now other people might say machine learning is like low rank AI. So it's the machine and then AI is deep learning.

[00:50:13]That's that's what other people classified this, right? Which is a different it's a different dimension.

[00:50:18]So that's kind of how I think of it.

[00:50:21]There are lots of free materials out there. Network Chuck. Oh, Network Chuck.

[00:50:24]Yeah, good channel. Learn the basics.

[00:50:28]Why don't you check the plug-in source code and get an idea? Oh, right, right, right, Oxnal, that's a good idea. We could do that. It's pretty neat. Check this out. Um, I do have this. My Google command does work. Google, it works right there. It just It's just missing the query part. Google. Boom. Google like overbooking. Ah, Ray Silvers is exactly like overbooking, right? It's exactly like overbooking.

[00:50:58]See, where was it? Here.

[00:51:00]Uh, here's the source code. All right.

[00:51:04]Alias. Alias.

[00:51:07]See here. So, I want to do an alias that allows me to pass in a variable instantiation.

[00:51:17]That's what I want.

[00:51:20]Oh, there's too many folders to really look. There's too many folders. All right. Let's just do All right. So, uh, alias here. I'm just going to copy and paste what I wrote so that way I can get this really quick cuz this isn't our main objective.

[00:51:35]Let's see. There's a problem though.

[00:51:41]Google open. Yes, that's exactly what I want. Yes.

[00:51:45]Uh, la join all the arguments with a Yes, exactly. It already knows. It already knows.

[00:51:57]So, how do So, wait, wait, wait, wait, wait.

[00:52:04]How to alias all on one line. Let's see if that works.

[00:52:14]You've been most mostly making webcrawler cos light data compressors and analyzers using Python and Ubuntu Windows Linux sub subsystem. Wait w is that Windows Linux subsystem or is that a different thing? The virtual environment.

[00:52:31]Are there more opportunities in cyber security or AI engineering? There's both. They're both right. I think more probably right now definitely spent in AI. There's more spend in AI than anything else right now. So if you're looking if you're looking AI is the answer. Hey, here we go. All right. So we could have written this ourselves.

[00:52:52]However, uh let's just do this. All right. So alias Google Google search.

[00:52:57]Boom. All right. Google, we'll say help me learn about torch. change function right there. Boom. Quick and easy. Look at that.

[00:53:13]I like that. That's pretty nice. I think I might want to save that. I think I might want to save it.

[00:53:19]Theo's light. Yeah, correct. All right.

[00:53:21]Good to hear it. Hey, Peace Lord. Hello.

[00:53:23]Hello. Hello. Welcome on in. Happy Thursday. Good to have you here. What are you working on today? 11 Malcolm. We are creating an alias really quick that I'm going to save because I really like this alias into my terminal. So that way I can now easily Google from my terminal window and not have to pop open another browser. It just does that for me directly in my terminal window which is pretty nice. I think it's pretty nice.

[00:53:45]Here by I'm I'm going to do something off screen real quick. Zshrc I'm not going to share. All right, here we go. We got an alias.

[00:53:56]Okay, I can I can show you guys.

[00:54:00]This is what I'm doing right here. All right, so paste. We added a new alias.

[00:54:05]Google search right there. Right there.

[00:54:07]Google search right there. Close. Offscreen editing.

[00:54:13]Perfect. Oh, nice. No plugin needed.

[00:54:16]Stephen the rescue. Hey, Oxno. That's because of you. You inspired it. You inspired it. Thank you. Thank you very much. Happy Thursday. All right.

[00:54:25]Why would you spend two seconds opening a browser tab when you can automate it in 30 days? Average engineer, right? I know, right?

[00:54:34]We can automate it. We We can save We can save time. It'll be better. It'll be better this way. It'll be better. I promise.

[00:54:43]That's pretty funny.

[00:54:46]Oh, hey.

[00:54:48]I didn't see you. Do we read Twitch comments also or just YouTube ones? Uh, Silver Ray, I just saw your message over there. It was hiding. It was hiding in the corner. Uh, if I see it, I'll say it.

[00:55:00]Good. Good to see you. Okay.

[00:55:06]All right. So, where are we at now?

[00:55:07]Where are we at here? Let me uh get back to our our business over here. Get all the things in the right spot. Perfect.

[00:55:14]Okay. Good. Good. Good.

[00:55:16]All good. Here. All right. Good to hear it on YouTube. Yeah. Ray Silvers. Sounds good to me. Look more energetic today.

[00:55:23]Peace, Lord. We're having a good day. We had a uh a lot of good healthy food and we're also doing things that we love doing, which is softer computer stuffy things, right? I love it.

[00:55:35]I love it. Oxnal, thanks. Yeah, it's because of you. All right, so hey, I love Liberty Justice. Thank you for the high five. Appreciate it. That was What' you say? Oh, you said what? Uh, high five. High five. Thank you. My gratitudes to you. That's very nice of you. You're I love Liberty Justice. I've been thinking about you. I have off stream. I'm like that. I love Liberty Justice. Always sending over the wonderful the wonderful super chats.

[00:56:04]Likewise, man. First time watching the stream, not clipped videos. Hey. All right. Good to have you here. Thank you.

[00:56:10]Yep. We're trying to get our our Twitch up here. So, we got our Twitch up and running because we get started late on Twitch. We got started late on Twitch.

[00:56:18]Do you have a system SOC team? Let's see. SOC team or incident response in your Yes, we do. Yeah, we call them the We call them Well, they're SRRES is what we call them. Call them SRRES. Site reliability engineering team. S team. That's what we do.

[00:56:44]Broccoli. Peace. Lord. Exactly.

[00:56:46]Broccoli. Yeah. Have you guys eaten your broccoli today? I have. I've eaten mine.

[00:56:52]Google. Let's see. Broccoli.

[00:56:57]Hey, there it shows. See how fast that is? It's instantaneous.

[00:57:02]It's instant.

[00:57:04]That's why I like it. That's why I like it.

[00:57:08]All right. All right. Let's get back to it. So, where did we leave off? So, we're doing We're doing the self attention.

[00:57:16]Yeah, nice. Watching this while applying to jobs on LinkedIn. It's horrible. Oh, race slers. You know what? That is a pretty tough situation. It is. Peace, Lord. Nice. Good to hear it. Okay, let's get back into our coding, you guys. Now that we've got our new fancy Google CLI alias, thank you for the hundreds.

[00:57:36]Appreciate it.

[00:57:38]All right, we need to do a new poll. Let's see here. All right, let's pull the world.

[00:57:46]Let's pull the world. Broccoli or uh animal product. Wait, no. That That's too That's too conceited. I can't do that to you guys. I can't do that to you guys. Are you being eating healthy? No, I'm not going to ask that question. I'm not going to do it. That was mean. I shouldn't do that. I'm sorry.

[00:58:10]All right, let's get back to our coding.

[00:58:12]is going to be like, "Are you guys eating properly?" No, don't worry about it. It's fine. You do whatever you want.

[00:58:18]It's It's whatever you want to do. All right, let's look at Let's look at the poll really quick. I'm kind of curious to see what the actual results of the poll are. So, most of y'all use Windows, some of y'all use Linux, and then Mac. Wow, more Linux than Mac.

[00:58:33]Impressive. Other other basically can go into like the the Unix Linux type family. So, if you combine all these up, it's just a little bit more on the alternative operating system, right? 3% extra on the alternative operating systems. That's impressive. I was not expecting that.

[00:58:55]Hey, how's it going there? Uh, Baron, wait.

[00:58:59]Barrento, thank you for subscribing.

[00:59:01]Appreciate it. Welcome on in. You join the right channel for software engineering. Today, we're building chat GPT Linux equals architecture. Uh or the ark as in Omar. Omar dogu. The ark. The ark. Nice. Different names for the cyber security team. Yes, exactly.

[00:59:20]You got it, Axel. I mean, I will take meat over broccoli any day, anytime, anywhere. Oxnal. I figured as much. I figured. Hey, Peter Parker. Good to see you. Welcome on in. Happy Thursday. Good to have you here. Did we do this correctly, you guys? the desktop. I the transformer. Okay. Did we make it work?

[00:59:42]Python transformer. Nope. No. No. Nope.

[00:59:46]Python transformer. All right.

[00:59:50]So far so good. Did we get our criterion?

[00:59:54]Did we do the criterion? No, we didn't.

[00:59:56]Okay. Let's do print loss.

[01:00:03]Here we go, guys. Moment of truth. Wait, wait, wait, wait, wait. Uh, it's got to be targets.

[01:00:11]Targets.

[01:00:13]There we go. Okay.

[01:00:16]Is it going to work? Hey, loons. By the way, I'm back. All right. Good to have you back, Lons.

[01:00:21]When someone tries to roast me by saying uh you suck, you all I always tell them, tell me what I did wrong so I can be better person. Oh, that's a good that's a good approach. That's a good approach.

[01:00:36]You're doing it right. You're doing it right. Look for that positivity. I recommend it. Baronto definitely sounds like a dro name. Oh, it kind of does.

[01:00:45]Like a Linux dro. It sure does.

[01:00:48]Thanks. Sorry for English levels intermediate. Oh, good. Omar. Oh, good.

[01:00:52]Just happy you're here. You just learned how to see or how to use the visual line feature in Vim yesterday. Oh, yes.

[01:01:00]Visual line. Isn't that great? Now that you got visual line, you want to do visual block, right? Visual block. Can do visual block. I like that one. That's a good one. All right. You got to do a little bit of visual blocking right there. Just a smidgen of visual block.

[01:01:16]Got visual line. And you've got visual mode, which is line, which is like string mode.

[01:01:23]Three visual modes. Line and block. and whatever this is just regular visual mode. Wait, what's that? How I know right buni? Isn't that great? It's pretty cool. It's so shift V. So you see on my screen here, you can see the keyboard at the very bottom at the bottom of the screen. I'm holding shift and pressing V. Block mode is controlV.

[01:01:49]Visual block mode, right?VRLV is the way to go. There you go. Shift control I. Shift control I.

[01:02:02]I just That doesn't do anything for me.

[01:02:07]Uh yeah. Yeah, you guys. That's a good one. V to get visual block mode. I like it. I like visual block mode. It's one of my favorites. I you actually visual line mode is the best though by far.

[01:02:18]Visual line number one. Hello. I just got back. Good news. We've passed the 50% thesis defense. Oh, wait. Really?

[01:02:27]Got busy coating our 3D segmentation using a new new net of the human firmer and wait hold on and proximal tabia. I don't know what that is. That sounds like something biology related, right?

[01:02:42]Human biology.

[01:02:44]Is that what that is? I don't recognize those words. I know computers. Biology though, it's a different story. I've known visual for quite some time. Yeah.

[01:02:54]And then get your visual block mode right there.

[01:02:58]Always revert to nano as your default cuz I always forget to handle. Yeah, I I see what you mean there, Oxnal. All good. It's all good. Use what you know.

[01:03:09]All right. Moment of truth, you guys.

[01:03:10]Will it work? Will our criteria work? I think it might. Oh, no. Exploit voting point type target for class probability.

[01:03:18]Oh, right, right, right. Okay. One hot.

[01:03:22]All right. So, can we say d type equals we say torch.flat.

[01:03:30]Can we do that?

[01:03:32]No. Doesn't like d type. All right.

[01:03:35]We've got a solution for that. Two.

[01:03:39]See if that works there. Yeah. Wait.

[01:03:41]Ignore index is not supported for floating point target. No. No. Really?

[01:03:48]Oh, okay.

[01:03:53]That's unfair. That's unfair.

[01:03:57]Stephen, why don't you use Visual Studio Code? Because I like myself, my Vim editor, and it makes me happy. You know what? I've got your answer. Use the IDE that makes you happy. There you go.

[01:04:10]Easy, easy answer. Basically, you created a model with the new net model.

[01:04:15]Then it segments the CT image. So you create like blocks. You take the large image and you chunk the block. You chunk it and then you convert it into a 3D for visualization.

[01:04:28]Oh, got it. Okay. So are you making 3D models using this approach? 2D images into 3D 3D models. Is that what you're doing?

[01:04:40]What's the thing you do when you select multiple lines and you prepend all of them at once?

[01:04:47]Oh, right. Yeah. Okay. So, this is what I do. I don't know if there's other ways to do it. This is how I do it. So, if I want to comment a bunch of lines out in Bim.

[01:04:55]Check this out, you guys. Wait, who is that? Who's that? Uh, Mong.

[01:05:01]Hey, Mong. Thank you for joining.

[01:05:03]Appreciate it. Thank you for your subscription. Welcome on in. Happy Thursday. you are joined the right channel for software engineering. We're about to share a trick in Vim if you use the Vim text editor. Thank for the party poppers. Check this out you guys. All right, so hold hold shift to plus V to get visual line mode and then I press col/arro slash uh octothorpg.

[01:05:29]Done.

[01:05:31]Is that how you do multiline comment?

[01:05:34]That's how I do it.

[01:05:36]Uh, did you did you catch that?

[01:05:39]Hey, how's it going there? Unknown.

[01:05:41]Unknown. Hey, Unknownverse. Thank you for clicking the hi button. Good to have you here. Happy Thursday. Think I saw you do that once for a second. You did, Bonzupi. You did. You also use Flask.

[01:05:53]Hey, for backend. Flask or 3JS for the front end? Yes. JavaScript for it uses WebDL. It uses WebGL. It's a nice little higher level programming interface for WebGL. It's pretty great.

[01:06:07]It's been around for decades.

[01:06:10]Our thesis advisor asked me to convert everything to Rust. Oh, what? Really? No way. That's actually really powerful.

[01:06:17]Rust is great. Although for a project, I don't, you know, I don't know if you necessarily need to use Rust for a project. Will AI bubble burst anytime soon? I think the AI bubble will just keep on like like flapping around. It's going to flap.

[01:06:33]It's going to flap around. You're you're using it. I'm using it every day.

[01:06:37]Everyone on the planet who is doing productive desk work is using it for the most part, right? So, it's it's here to No, it's here to stay. It is it's just a tool. Think of it as a tool. It's like a really good tool. It's not going to replace anyone. It's going to increase productivity and the number of jobs.

[01:06:56]It's just there's a little bit of turbulence right now.

[01:07:02]Oh, one of your thesis members also joined the Discord group, too. Hey, that sounds great. Fantastic.

[01:07:09]What is just the regular said syntax?

[01:07:12]Oh, right. It's pretty close. It's pretty close. It's similar to said reg x. Wait, what? You commented out that with macro you can just use GC after selecting something. GC.

[01:07:26]Wait, I haven't done that. You said GC.

[01:07:29]Wait, what is this? So, highlight and you say GC. That doesn't do anything for me. GC doesn't do anything. I'm pressing it right now. It's pressing nothing. You can see it on my screen doing nothing.

[01:07:41]Yeah. So, what I do is I highlight the code that I want. I press colon s/c carrot/ octaport slower slashg. That's what I do. I know it's weird. Is there a better way? Do you guys know of a better way to comment out lines and oneliner?

[01:07:59]You guys know have you ever seen a rocket engine designed by AI and its prototype is running? I've not I've not seen an AI proto I have seen an AI write a schematic for specific components and them being used in say like an airplane and they are improved.

[01:08:24]Hi sir unknown uh unknown verse. How's it going there? You want to make a website but I don't know coding only know HTML and CSS basics is can you make it using AI? Yeah you can. It does a really good job. The first prototype.

[01:08:37]Oh, wait. Peace lord. I have definitely not seen that. I haven't. Has someone done it yet? Has someone done this?

[01:08:46]Boom. Comment. Easy.

[01:08:50]Okay. So, let's continue on downward.

[01:08:53]We've got a problem here.

[01:08:55]It doesn't it So, does our output our outputs float? And it needs to stay float. Our targets are float, but it doesn't support padding indexes, which is annoying.

[01:09:10]So, I have to remove this sadly. I'm sorry, you guys. I'm sorry. I know it's sad for it to go. I know we're all feeling sad about it. We had We can't ignore the index zero right now, which is very sad. Don't worry. We're going to put it in a a little a little place right here for now to restore this. We will. So I need to say res restore ignore index, right? We got to restore it.

[01:09:38]Do you see the discord in link share?

[01:09:41]Oh, wait. Oh, hey Kyle. Good to see you.

[01:09:43]Welcome on in. Happy Thursday. Large language models are the tool. Consistent and efficient workflows are the bottleneck. Currently, they are. You got it. these large English models, they can achieve what we want them to achieve as long as you get them in the right spot, right? Thank you for all the reactions.

[01:10:02]Appreciate it you guys. You think your thesis member asked in Discord for the other project about the distill classifier for our natural language processing project? Hey, there we go.

[01:10:16]Distbert you can do that. So yes, NLP you want to use embedding model and it's really great because you don't need to train anything. It's just a classifier at that point. It works so good. That's the power of embedding. Everybody's sleeping on embedding models. They think, "Oh, it's all for vector and retrieval augmented generation, right? With the AI, we're going to rag. We got to rag it all." No, it's way better than that. It's way better.

[01:10:45]You use lazy vim, but it's a package called comment inbim. Okay, got it.

[01:10:49]Okay.

[01:10:51]Got it. Got it. Got it.

[01:10:54]We'll check it out. Thanks. All right.

[01:10:56]Link share.

[01:10:58]Oh, Lun. Um, yeah. What's going on here?

[01:11:02]It looks like you've installed an operating system that has a default language set to some sort of Mandarin.

[01:11:10]Hey, SSH3 Arlock. Good to see you. Hey, bro. What are you doing? Oh, we How are mate? We're doing great. Thank you.

[01:11:15]We're building ourselves our transformer. Have you used chat GPT?

[01:11:19]We're building one. Yes, we are. We are.

[01:11:22]We're building it. Small distributed language models future. Yeah, I think so. Because you can use them for very specific tasks. And if you get them really good at a very specific task, you can use energy efficient, highly accurate AI.

[01:11:37]Doesn't that sound better? Yeah.

[01:11:41]Oh before that we'll have to check it out. Sir is there any tool like in N8 which to help make AI workflow? There is lang chain, right? You could do that or you could do, you know, just Python script, right? You could do that, too.

[01:12:01]That's what I do. I do Python script.

[01:12:03]Just easy Python script.

[01:12:06]That'll work. All right. So, before that message. Okay. Uh oh. Hey. Wouldn't give this to my worst enemy. What do you got here? Low-level programming sandbox inspired by early game consoles. In this fantasy console, you program an assembly language. Oh no. With full control over execution, rendering your games and other creations to a retro monochrome screen.

[01:12:29]Steam. It's a Steam game, you guys.

[01:12:34]No way. It's a programming game.

[01:12:37]Oh, that's really cool.

[01:12:43]Ne, how do you say that? Neimonov.

[01:12:47]Neimonyimov.

[01:12:48]Neimonyimov. Okay, pretty nice. So, you can learn yourself some programming in assembly. Hey, there we go. Hey, Nea, welcome on in. Good to see you. Oh, Nea, you might have an answer for me. Although, this will just be off the top of your head. I don't know. It's a very silly. It's very silly. So, we've got a fun challenge right here. I'm using cross entropy loss that ignores the padding index. However, we got this error that says that when I'm using a float on my logit on my output from my criterion that I cannot use uh integers. I have to use floats which makes sense because I'm comparing floats. However, when I get my output, it's going to be in floats. It's just going to be floats.

[01:13:36]That's what it's going to be. A whole bunch of floats.

[01:13:39]So for now I just disabled the index ignoring for the loss and I it's probably fine.

[01:13:48]However, my question is if we have this here. So here's our criterion, right? Our criterion is a cross entropy loss. I would like to keep ignore index.

[01:13:59]However, I'm sure there's a way to do it. I just haven't really thought of it.

[01:14:05]Which software do you use for employee productivity management? Jira out. We use We use Atlassian tools. No, it's definitely not fine to disable it. It's not fine. It's not. Well, we Okay. I'm pretty sure it'll still work though. AI very flexible.

[01:14:23]Very flexible. It'll work. Uh though it's not fine. I I get it. All right.

[01:14:28]So, we we want to bring it back.

[01:14:30]However, if we do, it's going to say nope.

[01:14:35]will not likey this Google dude. Oh, wait. Oh, Google this. I suppose we could obviously we could search around for this. We could.

[01:14:45]I mean, the the whole point though is that we learn it. We don't want the easy answer. We want the hard answer. Test results are in the most recent patch.

[01:14:54]Oh, the subsystem definitely broke. Oh, okay. Got it. Got it. Got it. Got it.

[01:14:59]Got it. I see what you're talking about there. Thank you for all the likes, you guys. I appreciate it.

[01:15:06]You also need a proper mask on attention layer. Oh, good news. We did. We did. We got the mask. We add the mask. We added it. We got the mask right there. We did it on both the question and the answer.

[01:15:18]I I know we probably only need it on the answer. We just put it on the question as well.

[01:15:24]SSH, how's it going? Hey, do you prefer Python JavaScript over the C? I like all three of them as the first language. Oh, I would prefer Python first because C is going to give you a whole bunch of roadblocks with the syntax and it's also doesn't have any built-in functions and so you have to build everything yourself. I'd say Python is great as an introduction to programming because you're going to get some of the foundational basics of variables, flow control, loops, if statements, those sorts of things, right? Things that make a programming language, what do you call that? uh a programming language touring complete.

[01:16:03]Right? So if you get those foundational knowledges, then you can jump down to C where you get closer to the metal. It's going to require you to learn more of data structures and algorithms cuz you have to build everything yourself. You got to build it all. Why is your learning rate at uh instead of 0.1? I don't know. Let's change it. LR equals there. Boom. How about that? All right.

[01:16:25]Now it's at 1%. Do you like that? Is that better? Is that better? Oh, Trello.

[01:16:31]Oh, yeah. I know. I know Trello. We use I think Trello was acquired by Atlassian, right?

[01:16:38]Hey, we we fixed it. We fixed it. All right, you guys getting caught up on the chats here. Google stepping on your toes over here. Uh, is it the ignore index is the one hot of what to ignore.

[01:16:52]So, you can also just implement manually. Oh, it's a negative log. Okay.

[01:17:01]Okay.

[01:17:03]No, no. This dude from Google is stepping on your toes. All right. Bonsy B. Right. Right. Right. I've been learning a lot about Transformers the last few days. Thanks to Steven stream.

[01:17:11]Hey, great. Been very informative. I appreciate it, Kyle. Thank you so much.

[01:17:15]There's It's They sound complex and there are some nuances that are weird.

[01:17:20]Don't get me wrong. They're like, "Why is that happening over here? Why why are we doing it like that? The good news is we're figuring it out. Messing up your work and breaking user space up on Zupi.

[01:17:32]So just do negative log P correct sum.

[01:17:36]Oh, okay. All right. All right. Okay.

[01:17:41]Nea, you asked. We shall do it. Also, I see Trello is owned by Atlassian. Hey, they did acquire it. All right. Trello is now Atlassian tool. Also, I have a tip. You can also make your learning rate increase every epoch and then you can stop training on the epoch where it doesn't improve anymore. Hey, variable learning rate, right? You can get some momentum in there with your optimizer.

[01:18:09]We can add some momentum.

[01:18:12]Correct. Would not make masks. So, it ignores masks easily. Okay.

[01:18:18]All right. Thank you very much, Neva.

[01:18:20]Appreciate it. So let's change our criterion.

[01:18:24]How do we do this? All right. So if we're going to do negative log P correct the correct answers and then wait correct sum.

[01:18:42]Okay. So this would ignore the zeros.

[01:18:45]Oh this is going to snag. So we take our output which is P. Is P the output?

[01:18:55]Is P the output? So we do torch torch.log output.

[01:19:03]We say negative. And then we do our targets like that.

[01:19:11]Something like that.

[01:19:13]Uh there is a lot of devs having actual freakouts over AI. uh they are because it's a big change to their daily cycles, right? It's a big change and like I used to code and I used to have this passion for my craftsmanship, right? It was so great. I have code comments. I have my classical architecture and I've got my design patterns that make my code look beautiful. You don't need that anymore.

[01:19:41]The AI writes it for you now. So code craftsmanship is no longer a discipline.

[01:19:47]Stephen seems to be the only engineer online taking it optimistically. Yeah, it because it makes it makes my day better. I can do a lot more. I can do so much more. I don't really have to care.

[01:19:59]The sad thing is I do kind of miss code craftsmanship. The good news is I'm getting my my my appetite satisfied by live stream coding with you guys.

[01:20:11]Most of them have courses and content.

[01:20:14]They are losing money. Oh, right.

[01:20:18]There are content creators who are teaching programming and now the world doesn't need that as much because AI can write it all for them. I know that kind of counts as us here, doesn't it? It does. I mean, there are still people who want to learn, I think. Right. Maybe I want to learn. I want to learn.

[01:20:44]Oh, Muhammad. Hey, how's going over there? You're a Java developer or overwhelming by the data structures and algorithms I love building, but my job requires DSA. Not many people can say that. Most people just say they import libraries and their day is over. Yeah.

[01:20:58]Use this library in your code. Boom.

[01:21:00]Done. Easy. You just made an executive decision to use a library. DSA in your day-to-day. That's pretty special.

[01:21:09]What's your opinion? Thank you very much for your time. Yeah, I would say you're lucky because a lot of people would really like that job to use DSA in their day-to-day.

[01:21:21]Yes, Stephen. Now you just need to sum it or mean it some.

[01:21:26]We do some sum there. All right, let's see what it looks like. Yeah. Okay, so I want to say my loss equals let's print our loss. All right, Nea.

[01:21:37]All right, here we go. Here we go.

[01:21:41]All right. Tensor used as MC's must be long in type or bite. Oh, they aren't.

[01:21:47]Oh, right. All right. We We can fix that. We can fix it. One hot.

[01:21:54]We just get rid of the this here.

[01:21:58]Boom. Easy. Boom. Hey, I got an an uh-oh. Uh-oh. Neva, I got a not a number. I got a not a number. Mean probably more correct. Okay, I'll try the mean. I'll try the average mean squared error, right? But you're doing a negative log mean code craftsmanship is still important even though AI makes automatically it can can't beat the creativity of the human mind.

[01:22:26]Right now it is. You're right. The human mind can get real crazy. It can get real crazy.

[01:22:33]So, for the mask tokens, you need to ignore them.

[01:22:37]And we do that.

[01:22:42]I've got the mask. However, Oh, wait.

[01:22:45]The mask tokens. Okay. So, you mean the pad tokens? The padding.

[01:22:52]So, I already set my pad tokens to zero.

[01:22:56]Uh, so it'll be the first element. So, we need to eliminate any pad tokens is what you're saying.

[01:23:05]One hot is likely wrong. Also, you should be able to index it by an integer.

[01:23:10]Okay. Well, I'm getting the the if there's another approach, let me know. I'm I actually I'm not familiar.

[01:23:20]So, let me see. All right. So, let's do a mean here. Average. All right. See output. It's still a problem.

[01:23:31]So, we've got not a number still.

[01:23:34]So, what if we got rid of the negative here? All right, let's try that.

[01:23:39]Not still a problem. Okay.

[01:23:43]Torch log outputs. H.

[01:23:48]All right. What if we just did the here? Let's just see if let's see see if the log is causing an issue.

[01:23:57]Oh, look at that.

[01:24:00]Oh, I see.

[01:24:02]Aha.

[01:24:05]Okay. Okay.

[01:24:07]That's bad. Pad token should not be zero.

[01:24:10]Why? Pad token should be totally you can put them wherever you want them to be.

[01:24:14]Right.

[01:24:16]If your pad is zero, then your max token index is one over the largest index of the vector.

[01:24:25]So here, this is my these are my special tokens.

[01:24:30]Pad, start, end, and unknown. Oh, Stephen, you didn't softmax. Oh, well, I sure didn't.

[01:24:38]I sure didn't.

[01:24:41]Let's soft max it. All right. All right.

[01:24:44]We'll do ourselves some soft maxing. You You noticed. You noticed. We didn't softax. self do.Soft equals torch softmax. Can we do that? Is that right?

[01:24:54]Python import torch torch soft max. Is that thing? No. Maybe it's one word. Not like that. Okay. Is it torchin?

[01:25:08]There's a rand. There it is. Right there. Okay.

[01:25:13]There's our soft max. Now we will say out equals self.soft Soft out. Done.

[01:25:19]Soft maxed complete. We added it, Neba.

[01:25:24]There we go. Oh, much better. Hey, thank you. Thank you very much. Do you stream all day? Hey, Gooseman. How's it going there, Gooseman? Good to see you.

[01:25:35]Happy Thursday. I stream only for a little while each day because I have work that I got to do on a regular basis. How many hours? We still have time, though. We're still going to hang out here for a little while. If you want to avoid all the softmax log, you can use a dedicated function to avoid all that. But you know what? I think we're okay. I think we're okay. I think we're doing good now. So, let's see if we can do the rest of this. Here we go. See if we got it good. Got it good. Hey, look it right there. We got ourselves some loss. We got ourselves a little bit of loss right there. Does it need to be negative, though?

[01:26:13]Does it need to be negative?

[01:26:17]I don't. Oh, we we do. We have to negate it. There we go. There we go. So that way. Ah, much better. Okay.

[01:26:25]You want to avoid softmech log so you can Okay. You'll hang out till I fall asleep. All right. Sounds good. Hey, good news. We're We're here for a little while, so you have plenty of time. 3:00 a.m. in the Philippines. Oh, wow. That is very late. Are you doing the hackathon? Oh yeah, Neva, are you doing the hackathon? We got our hackathon, you guys. Hey, look at here. For those of you who are here, we join our Discord.

[01:26:50]We've got a hackathon occurring. Just go to Discordsai.com, follow the online instructions, build an agent, and then submit it to the Discord channel and you get entered to win.

[01:27:05]This is how you do cross entropy. Is it really? Is it really that simple?

[01:27:10]Is it really? Is that really how simple it is? Nea, you made my day. Oh, that's so great, Nea. Thank you. Oh, that is very satisfying. Keep coming. It's so good, Nea. Thank you so much. All right, just what you're doing is fantastic. I love it. I love it. I appreciate so much. Thank you.

[01:27:33]So, I have a good idea for it. I'm not a good project manager. This is a PM thing. Okay. All right. Good to hear.

[01:27:42]Good to hear you guys. Build your agent.

[01:27:46]Build your agent. Submit it. And you get to enter into the hackathon for prizes.

[01:27:52]This is so cool. This is so cool right here. It's so good.

[01:27:58]All right. Now we can start optimizing our model. You guys ready? Cross domain synthesis. Wait, wait, wait. What is What is that? I I don't You mean turning from one discipline to another discipline?

[01:28:13]Oh, what is that?

[01:28:18]Google it. Okay, we're googling. Hey, we can use our new Google thing, right? All right. Google crossdomain synthesis.

[01:28:30]All right. the process of generating new data, knowledge or solutions by bridging the gap between different fields or data environments. I want an example. Give me an example.

[01:28:42]What is an example of that? 0.

[01:28:46]Okay. Utilize across artificial intelligence, cyber security, and strategic problem solving to adapt, augment, and transfer insights from a source to a target. Well, I mean, that's very abstract. I need more. I need an example. Simple example.

[01:29:03]voluntarily subscribed to the newsletter. Uh voluntarily.

[01:29:08]Uh yes, I am voluntarily subscribed to one newsletter. It's called the the hacker newsletter.

[01:29:18]However, I've been ignoring it recently.

[01:29:22]That's a good that's a good poll, Lon.

[01:29:24]That's a good poll. See, are you V? Wait, how do you spell that?

[01:29:31]Let's see. Fuel unarily subscribed to a newsletter.

[01:29:40]Yes or no? All right, let's find out.

[01:29:44]Good poll. Good poll, loons. Good poll.

[01:29:48]How do you balance your time with work and study? Can't do that. Oh, it's pretty tough. I pretty much plan play all day and code a bit. Oh, well, what I do is I do all of my prep on the weekends for the most part, like food prep, chores, and everything like that.

[01:30:07]I do all that work on the weekend. And then I also stream on the weekends as well now. So, I've added that into my schedule. And then I just play a little bit less Minecraft.

[01:30:17]That's what I do.

[01:30:19]Leonardo da Vinci studied autonomy for drawing. Let's see here.

[01:30:24]Where was that? A real world example of cross domain synthesis is using autonomous car technology to improve medical imaging. Oh, okay. Well, self-driving cars, hospital radiology. I don't know how that would possibly work.

[01:30:43]Self-driving cars use computer vision algorithm to detect moving pedestrians lane markers in real time. Scientists transferred this exact algorithm structures to the medical field. Instead of looking for pedestrians on a street, the AI now scans MRIs to identify moving and growing tumors. Oh, hey. All right.

[01:31:01]Got it. All right, Kyle. Thank you. I understand it now. Advances in neurology make AI better. Hey. Yeah, there we go.

[01:31:11]Neural network was used modeling how the eyes work. Nice. Hey, Billy Eusic. Hey, how's it going? Welcome on in. Happy Thursday. What's the new Grock build best for? I don't know. I don't know.

[01:31:25]Does anyone know?

[01:31:28]I've never heard about the cross domain synthesis. Same here. I mean, I I feel like I've run into the concept before in AI specifically. Can we do Jeepa next time using PyTorch framework? Wait, what is Ja? Jeepa PyTorch. What is that? What is Jeepa?

[01:31:47]VJ Jeepa method for self-supervised learning.

[01:31:52]That's a good idea. Divine whisperer.

[01:31:56]Let's add that. Let's add Jeepa. So, this is Is this VJA?

[01:32:02]Is that what that is?

[01:32:05]Let's see. Or is this a different kind of Jeepa? Hands-on Jeepa building selfsupervised vision models.

[01:32:13]Okay.

[01:32:15]Jeppa building self-supervised vision models that work.

[01:32:20]Oh, new paper to read. Hey, Yep. new paper. Nice. All right, I'll add it to our our ideas or stream ideas. Here we go. New post. Here we go. It's called VJ Jeepa.

[01:32:34]And that is There we go. And that would be Python and AI post. There we go.

[01:32:44]There we go. Hold on one second. I heard my calendar, you guys. I heard my calendar. I have to check.

[01:32:58]Uh, okay.

[01:33:02]All right.

[01:33:04]I might have to leave in 10 minutes because I've got a meeting that just suddenly magically appeared out of nowhere. Okay.

[01:33:11]Divine Whisper is the Jeppa language.

[01:33:14]Oh, then translate to there are too funny. Uh, wait, their translations are too funny. Are they? Wait, you love the ch Oh, really? The ch Yeah, Mandarin, you guys. Oops. I accidentally clicked on the wrong thing in the poll. I meant to click yes. Oh, Bonzupi. All right, no worries. No worries.

[01:33:31]Most of their explainer articles remove details are sometimes just wrong. Okay.

[01:33:37]Look for it on archive. Always best to implement the papers directly. Okay, Jeepa archive.

[01:33:48]Let's see if we can get the archive.

[01:33:50]Here we go. Found it.

[01:33:53]Thank you, Nea. Hey, J bro. Good to see you. Welcome on in.

[01:33:57]Jea, is there the Leon's baby? Oh, Lun's baby. Okay, got it. Is there a way to add a comment? Jeepa, here we go. All right, there it is.

[01:34:10]Okay, future stream idea recorded.

[01:34:14]Thank you very much, you guys. All right, so let's see if we can do some optimization. Yeah, loss.backward.

[01:34:25]Is it backwards or back? I think it's just backward. One word.

[01:34:30]Hi all. Hey J, bro. Good to have you here.

[01:34:34]And a lot of good papers are just readable as is. Yeah. All right.

[01:34:42]Yane uh Yan Lacunin is a scientist.

[01:34:47]Lacun Lun. Got it. Got it. Got it. I figured as much. I figured I figured as much.

[01:34:56]Is that Google Collab thing good? I tried building an agent in it. I think you burnt something. Oh, did you J bro?

[01:35:05]It works pretty well. It's free. It's free and they give you hardware compute.

[01:35:10]Stephen, that one hot on target is wrong. Is it really?

[01:35:15]What What would it be?

[01:35:19]I I understand that it's wrong. I don't know what else to make it though cuz it's I'm looking for the token output, right?

[01:35:26]So, I got my logits. I did my transform from my uh logit logits, right? Or logits, however you say. I converted them to logats to target the output of token that I'm searching for. Then I do an argmax to find the right token. And since yes, for sure 100% just training one is fine.

[01:35:49]I don't know about that cuz my training data is a bunch of words.

[01:35:58]So I don't think that'll work very nicely.

[01:36:02]Hi from India. Hey, how's it going?

[01:36:05]Welcome on in Dpack.

[01:36:07]Raj Dpack, thank you for joining in here, guys.

[01:36:12]So, yeah. So, that is a bunch of number.

[01:36:14]That's a bunch of words, right? It's a whole bunch of words.

[01:36:19]It's pronounced the same as the word logic. Just replace the C with a at with a t. Oh, log log it. Oh, log it. Yeah. See, I thought it was logit. I thought it was logit. I was looking that up earlier today. We were looking that up and I thought it was logit. Like the word log, like you're logging data to a disk or to a, you know, uh, some sort of central logging repository. You're logging the data. It's a logit. All right. All right. Bonzupi, I'm going to revert back to log it cuz that's what I thought it was.

[01:36:56]Rad jeep. Oh, Rajep. Oh, Rajep. Okay.

[01:36:59]Kundar. Kundo. Oh, Kundu. Kundu. Kundu.

[01:37:02]Okay.

[01:37:05]The index of the proper token to output as an in as an integer, not as a one hot.

[01:37:13]I suppose we could do that as well, which is perfectly fine. I could do that. I could do that, too. All right.

[01:37:24]All right. So, let me instead of doing one hot target, all we have to do is tokenize it.

[01:37:32]We do dictionary tokenize.

[01:37:36]There you go. All right. Are you sure, Nea? Are you sure? Are you sure? Before we do this. Okay.

[01:37:44]All right. We're going to run it. Okay.

[01:37:46]Here's the output. That's what we're looking for right there.

[01:37:50]The proper input to a one hot would be your correct input.

[01:37:54]Would be your correct input. Yes.

[01:37:58]Yes. That's what That's what that is.

[01:37:59]Yeah. So, so I've got my the actual answers, right? So, here uh and my output my model automatically converts words into tokens and then it runs them through an embedding, right? So, it normalizes. So, the model's forward function automatically does this. So, it tokenizes and then it runs it through the embedding which converts them into indexes, right?

[01:38:25]So that part's done there and we can tokenize again. However, the output is here. So comparing this target to this output feels weird, right?

[01:38:43]So that's why I did a one hot cuz this is we want to select the probability. Yes. Yes, we do.

[01:38:55]Uh uh. Okay. Oh, I see what you mean.

[01:39:03]Okay. All right. All right. We'll try it out. We'll try it out. Let's see if that works. Let's Let's find out. Let's find out. Loss.

[01:39:13]Here we go. And then we're going to print the loss. All right. Let's try index out of bounds for dimension zero with size seven. I was wondering. See, that's what that's I I wanted. We needed to select the correct the the law. We we can select it through argmax. I get the right token on the Oh, we should probably arg good though. Do we still want to do it that way? You can also one you can also one hot zero all the elements that are pad and then multiply by the vector to get as input.

[01:40:02]Okay.

[01:40:04]All right. I'll have to think about that. I'll have to think about that.

[01:40:09]You're stringing the pot now. Wait, Bonzupi guys, if it's GIF or GIF. Oh, yeah. You're stir You're stirring the pot. You're stirring the pot. GIF or GIF, right? GIF or GIF.

[01:40:24]I also like another way is the target is a one hot targets pad token equals zero.

[01:40:33]M Oh, okay. Got it. All right. Yeah, I need I need to think about that. I'll need to think about it. So, I do see a possibility here. I could grab the ARG max and then compare the tokens directly and use that as a loss.

[01:40:54]Froms times targets.

[01:40:57]Oo, I'm going to copy that. All right, so I I Give me one second here. Give me one second.

[01:41:06]Um, a a magic meeting just showed up out of nowhere on my calendar. So, I might have to go to that here in a second. So, give me give me one second. All right, I'll say over one moment. All right. Uno momento.

[01:41:21]All right.

[01:41:22]Uh, that won't work either, will it? So, who who who did this? Who did it?

[01:41:32]Uh, interesting.

[01:41:37]Oh, that's what I thought. That's what I thought. Okay.

[01:41:40]Argmax is wrong for several reasons. The major one is it's not differentiable.

[01:41:44]So, I wouldn't have no grads.

[01:41:46]Uh, okay. All right. Uh, it sounds good to me.

[01:41:51]So then, well, as you can see, what's clearly wrong right here? Right. So, I've got my tokens and then I've got my softmaxed Uh, logits right here. My softmax logits. This could have been an email, Stephen, 10 minutes later. I know, right? It could have been an email.

[01:42:12]It could have been.

[01:42:16]It could have been an email, you guys.

[01:42:25]All right, let's keep going. Let's keep going.

[01:42:29]Okay. So the reason why So the reason why I did the one hot was to be able to say what the target output is trying to achieve to predict the next token.

[01:42:40]Now I'm sure there's a lot of different ways to do it. However, very obviously this won't work because it's not going to be selecting the correct value.

[01:42:52]Meanings can be productive when you have a great communication style. All right.

[01:42:55]It could be, which is what Steven said.

[01:42:57]Makes you uh Yeah. makes you hireable.

[01:43:03]That is a very good That's a good point.

[01:43:05]You Exactly. It's a very good point.

[01:43:07]Hey, Brandon, welcome on in. Good to see you. Your stream on your phone died. Oh, BT. What's BT?

[01:43:14]Wait, what's BT? Hold on. I feel like I should know what that means. You should be able to make your targets that are batch target. Oh.

[01:43:24]Oh, it's the Okay. So then I just dot Wait, does it unsqueeze or squeeze?

[01:43:32]Unsqueeze. No, it's squeeze. It's squeeze. S q U E E Z E. Can we say zero?

[01:43:39]Can we do that?

[01:43:43]Dim zero.

[01:43:46]No. Okay. So 7 by4 versus H. Oh, wait. DIM one. That's what I want. Let's try that.

[01:44:02]Okay, let me see this real quick. Let me do targets dots squeeze sq e dim equals 1. We'll do that there instead.

[01:44:18]Okay.

[01:44:23]Uh then use the target to index the probabilities that are batch token W where W is the word token.

[01:44:32]I know if we watch Steven stream I get smarter. Hey Brandon, that's nice for you to say. Thank you. You can tell when my kernel is almost done compiling by the sound of the CPU fans. Yeah, it gets toasty, right? It gets really toasty.

[01:44:47]You compile this thing like 30 300 times in a week. That's a lot of compilation.

[01:44:50]Your you CPU is getting all the work out right now. What's the shape of target before you do anything? And the input, what's the shape? Okay, so here's the here's the output, right? Which we're going to compare the loss to 7x4.

[01:45:04]And the target we will get the shape of here in a second. Uh it's basically 1 by 7. It's 1x 7.

[01:45:14]So see yeah this is output.

[01:45:18]So we'll see we'll do this really quick.

[01:45:20]So we'll say output shape.

[01:45:24]There we go.

[01:45:27]Output shape.

[01:45:29]I think you have a bug around the size of the output layer. Like basically some offset need to be added for to sub the from the target. Okay. See yeah it's I got you. I got you.

[01:45:41]When do I I I see I see what you mean.

[01:45:43]Yeah it makes sense. Hey Stephen, do you manage to choose your current career?

[01:45:47]How did I manage? I wanted to be a game developer. I started making games and then I found out that writing enterprise software was immediately more valuable because you get you get revenue immediately.

[01:46:02]You've been incisive for months between cyber security and backend development devops. So I would say cyber security is going to be a really critical item and that role is very necessary still backend and devops uh I would choose which one's going to make you happy that's actually one of the good which regardless of what the role is which one's going to make you more happy I think that's a important one which one do you want to do when I do something on your laptop it burns you oh do you hold your you put your laptop on your lap Yes, you have a bug in your data preparation. Stephen, you yeah are defining getting enough target tokens. I think my my my target is looking good.

[01:46:46]So I I like I like my So here here's my tokens, right? My to my dictionary.

[01:46:53]Let me let me uh hide this stuff for a second. Let's hide this. Okay. And then I will print to you Let's go to our model forward.

[01:47:07]Okay.

[01:47:09]So, here's the features and labels, right? So, I'll print to you the labels.

[01:47:13]So, I will say print the labels.

[01:47:20]So, this is the labels. Labels.

[01:47:25]I think this is correct. I feel like this part is correct. So, here. So, here's my labels. Looks fine to me.

[01:47:31]Let's hit the embedding. We got seven, right? Easy, easy tensor, right?

[01:47:37]So, I've got basically a sentence and a bunch of tokens. Every word gets its own line on the embedding, right? So, that part I'm pretty confident about. That part makes sense.

[01:47:52]You're not in microwave trouble again.

[01:47:53]What's beeping? I don't know. What is beeping? Yeah, I love three so much yet I'm still indecisive. Senko. Ah, that's tough. That's tough, right? Because you got three good options. Well, here's the deal. Check this out. You don't always have to stick with one. You could try the one that you think is the best first and then continue down that path. And if you want to change your mind later, it might not be as bad as you think to change it later.

[01:48:21]Bespoke AI. Yeah, we got these AIs. You never know. You never know. Like if you get the shape you have, the outputs are batch token and weights and targets that are tokens instead of BT. Well, yeah.

[01:48:36]Right, right, right, right. Um, oh, do we need to on our t here? I've never done this before. However, Nea, would we be able to convert our our would we run it through embedding here? All right.

[01:48:54]So, maybe this is the answer. I don't know if that would work, though. I don't see how that would work, though. I think we could try it. Basically, we're going to make them the same shape. We'll make them the same shape. Want me to set my clock? Buy the Panasonic inverter. All right, enough about boring microwave ovens. Oh, wait. Panasonic.

[01:49:16]That sounds like a good one. Is it cold out there? You're always in a jacket.

[01:49:20]Oh, yeah. I always have a hoodie on.

[01:49:22]Yep. Oh, he's got the hoodie on. Senko, you got it.

[01:49:27]It's not cold. It's actually really nice. I just It's comfortable.

[01:49:34]Yeah, it's pretty easy to change career later, I think. So, Senko, I'm not sure if this word is vectoring. Oh, yeah.

[01:49:40]Vectorizing. Yeah, we're vectoring. Yep, we are. That's what we're doing. Every bat should have a stream of tokens. Yes.

[01:49:47]So, we got that.

[01:49:51]I like seam only take part of the need targets. The thing is when uh for the embedding can I use the embedding to encode it feels it feels wrong to use the embedding for the targets as well right for the actual answers. Do we want to I think that would work. It just feels feels weird because you're training the embedding.

[01:50:20]So you want the exact answers, right? Don't you want to target towards the one?

[01:50:28]You should contribute to open source via comments. Yeah. Oh, yes. That's the way to do it. You got it. Basically, targets need to be 2D and actual 2D. Okay. All right. Sounds good to me.

[01:50:42]Who really care about comment PRs? Oh, it's just a way. Not not they don't.

[01:50:48]See, the good news is they see that all you're doing is adding documentation to the code. Well, it really depends though. Really, what you should be doing is building docs for the code or some way around that, right? Maybe if you find a line of code that might be helpful. It's just it's just a soft way to get into being a contributor to open source. It's not always going to work.

[01:51:13]You're hiring out your company. Yes, Ray Silvers. Yes, we are. What's the company's name again? It's called PubNub. PUB and UB.

[01:51:22]There you go. And we've got ourselves the answer right here. We are hiring for two specific engineering roles right now. Senior software engineering, artificial intelligence, large language model systems, and a senior solutions architect.

[01:51:37]Hey, that Vasper, welcome on in. Good to see you. Good to have you here. Happy Thursday.

[01:51:44]targets before the one hot needs to be 2D.

[01:51:52]Got it. Okay. Seems like you have a target before it one hits one D instead of two. So basically the data is corrupt. Corrupt. Well, it's it's the right data for sure. It's just we need to put it in the right shape.

[01:52:11]Right. So if our outputs are are logat which are going to be the next token right it's the next token what's the next word basically what's the next word and there probabilities of what the next word are right you see here on the screen we've got the probabilities of the next word so in this case the next word looks like it's going to be whatever this position is right here like for example maybe oh no it's going to be this one oh and it got it right interesting well actually it says this one uh but it's got it's getting close. So, we just need to match this. And so, my idea was why not tell it what the exact token should be, right?

[01:52:54]The exact token. The total size is too small.

[01:52:59]Well, is it though? Because look, my my my data size is very small. Look at this. This is my data size right here.

[01:53:08]So, I think it looks about right.

[01:53:10]There's only like 15 words there.

[01:53:13]There's only 15 words. So, there should only be 15 positions. Well, actually, there's 14 words, right? So, there's only 14 positions. So, that looks good to me, right?

[01:53:24]Oh, Lun. All right. Good to have you here. Thank you for joining on in. How do you get users for a platform like UGC content? How did the platform take off?

[01:53:33]A bit of history lesson, if you may.

[01:53:36]Sure, absolutely. I'll tell you. I'll give you the answer.

[01:53:43]Can you tell me what the post one hot shape is currently?

[01:53:49]Uh the post one hot shape will just be the tokens themselves. Right. So here I will share it to you. Here we go. The tokeniz. Here we go. And the shape.shape.

[01:54:05]I so I I'm we I can get it to be whatever like it's not not a challenge here. Hey mom, I'm famous in Blum's code. Hey, yes, Kyle.

[01:54:17]Wait. Oh, what happened? That should have worked. Uh why did that not work properly?

[01:54:22]Uh tokenize. Oh, wait. Let's close that out real quick. Okay, that shape. Oh, it's not a function.

[01:54:33]Here we go. Okay.

[01:54:36]So yeah, so it's seven tokens and seven tokens and then we need to convert this.

[01:54:42]That's why I one hoted it is so we need to know the exact answer. So if this is the if this is what the model returned its answer as and the target tokens or the actual answers are these tokens here.

[01:55:00]See you next stream, Brandon. All right, sounds good. Thank you for joining in.

[01:55:03]All right, we got a story to tell you guys. It's not a very long one. So, what happens uh what happens at Discord?

[01:55:11]No. So, this is wrong. Post one hot should be 3D.

[01:55:21]All right. Here. Get status. Get add here. Get commit am uh partway deciding.

[01:55:31]Let's see. part way with loss back prop push. All right, I will send you the link so you can just see the code directly. Uh, and then cuz I don't think we're far off. I think we're like one or two lines from getting it to work.

[01:55:48]Okay, let me see here. Story story. Hey, don't go anywhere. We're going to tell you about the UGC discussion here real quick.

[01:55:56]All right. Uh, raise silvers, we will get you. So, GitHub GitHub Steven repositories uh transformer.

[01:56:09]Here we go.

[01:56:12]Copy link.

[01:56:14]Easy. Oh, what? I I made a That's weird.

[01:56:17]Okay, here you go. All right, Neva, I'll put this on link share. Link share at getting better. There you go. Over there on link share.

[01:56:29]I think we're really close. I think we're really close.

[01:56:34]Technommancy. Hey, Mike. Hey, Mike.

[01:56:36]Mike. I don't know about technommancy.

[01:56:39]What is it? What is technommancy?

[01:56:42]You have batch dimensions, right? I don't have batch dimensions. I have not done batch dimensions yet. I know that sounds weird. I know it sounds weird. I don't have batch dimensions yet. I think the data propagation on the group. You have the target labels. Okay, I'm here.

[01:56:58]No worries. I'm battling job applications on LinkedIn. That's right.

[01:57:02]Okay. So, let's jump back to your your request here. How do you get users for platforms that has UGC content? So, what is UGC content first?

[01:57:11]So, let me look that up really quick.

[01:57:14]Let's see. Google UGC content.

[01:57:20]All right. What is that? User generated content is a brand. Oh, got it. Okay.

[01:57:25]So, you've got basically a Tik Tok or an Instagram. How do you acquire users for that platform? It's actually pretty not that difficult. So, you launch it in the app store. That's going to get you initial discovery. You launch it in media, right? So, you make news story coverage wherever you can get your your your main story released, right? So, if you if you built a new Tik Tok, right?

[01:57:49]You build a brand new Tik Tok and you want to market it to the world. You'll then release it in places where other users are already existing. It's called OPA. It's the number one strategy. You can either pay for OPA or you can create an engaging story that is compelling for someone to author and release on their news outlet. Right? So there's another way. You can also add integrations. So you can integrate with other businesses or companies.

[01:58:17]Right? That's another good way to do it.

[01:58:19]That's fantastic. That's actually what we did. So, we don't have UTC business.

[01:58:25]We have a platform for communication, an API company called PubNub. An API company called PubNub that allows communication for in-app experiences.

[01:58:35]Things like chat, user chat, multiplayer games, tele medicine, when you talk to the doctor and patient, on demand delivery when you order food and you see the car on the map approaching you.

[01:58:46]That's what we do. We integrate with all the different developer frameworks as part of our OPA approach, other people's audiences.

[01:58:57]And we also have something called the loss leader where we give away a free tier that allows anyone to start without a credit card. So free forever, we have a loss leader. So that's another approach. So OPA and loss leaders, loss leaders are a lot of fun. They're my favorite because you get to build something amazing and give it away for free and attract new users. So you might have some sort of advanced AI feature in your app and you don't charge for it, right? Also for UGC businesses in general, most of the users are going to be free and you sell them you you make money through ads, right? So UGC based businesses will be ad driven.

[01:59:40]All right, there you go. Let me know if you have any questions on the matter.

[01:59:45]Let me scroll up here. Stephen, why do you guys use PubNub? Need ML engineers?

[01:59:50]Uh, we've got a lot of data going through our system right now. We need things like NLP and moderation, uh, classification. So, so common standard stuff. Also, we have a code platform that allows you to mutate data through the network called PubNub functions. PubMed functions allow you to intercept messages, do some JavaScript work on them and then let the message pass on to the receiver. This happens in the infrastructure at the edge. So, it's edge computing. We offer this to our customers and they are also asking for AI features that can be added into the mutation and manipulation of those messages. So, that's where that's where we do it if that makes sense.

[02:00:36]All right. Missed some of your chats, you guys. Oh, wait. That's so weird. It threw me off so bad. Sorry. Just assumed you did batches. Hey. Oh, good. No worries, Neva. No worries. We We You want Batches are ideal. You want to do batches. I just didn't start with it. We can add it in. I was going to add it in later.

[02:00:57]Let's see.

[02:00:59]All right. So, raceovers, let me know if that made sense. Good. I'm glad you don't know about it. I like you even better now. Oh, Mike Mike Jr. All right, that's great to hear. Yeah. Yeah, I technommancy. I've not heard about it.

[02:01:14]Technommancy.

[02:01:16]It's coding rituals into software for all types of purposes. Technommancy.

[02:01:20]All right. I'm glad you don't practice it. I don't. I do not practice technommancy.

[02:01:25]Kyle is listening. Who is handling your database? We handle our own databases.

[02:01:30]We've got a lot of different ones. We've got Postgress, of course. We've got Cassandra, we've got Reddus, we've got uh a couple others I'm thinking of that I can't remember right now. And we run them and operate them in Amazon's infrastructure.

[02:01:49]I want to make money through transactions. Hey. Yeah, that was a good one. Transactional revenue. I like that's a good one.

[02:01:57]Your company seemed like UGC kind of. I have an event platform that I need users to become hosts in order to attract more users like Meetup hopefully better over time. Yeah, that's a good way to do it too. Raise silvers. You got it. You got it. Marketing without money is hard. It is. It's called growth hacking. It's like called growth hacking. It basically growing without a heavy investment.

[02:02:25]Thank you for your viewpoints. You're welcome.

[02:02:29]Do you need a dude who knows crypto?

[02:02:31]Because Bunzupi might be someone I think I'm accidentally becoming expert in cryptography. Bunzi, that's how it happens. You become an expert. You've become an expert.

[02:02:44]Could you guys do edge functions as Python? Oh, we are working on it, NeA.

[02:02:49]We are. Python makes more sense for machine learning. We are right now it's JavaScript. We are working on it. We are working on it. It's going to happen. We think this year. We think this year. So, yes, Nea. Exactly.

[02:03:03]Just got to put boots on the floor, grab attention little by little. Yep. That's how it works. And then you get the snowball rolling. And it gets a little bit bigger and a little faster and a little bigger and faster eventually.

[02:03:20]Probably going to go give some away some of your features for free and call it cost of customer. Yes. Wait. Uh cost of customer acquisition or something like that. Uh customer cost of acquis what is CAC always? It's like cost of acquisition, right? CAC term busyiness.

[02:03:40]Uh customer acquisition cost. Ah see right. Cost of customer acquisition is the way I say it. Customer acquisition cost. CAC.

[02:03:51]Hey RXT RO. How's it going then? Oh, was it? Uh, Ruxro, hello. I love your informational videos. Hey, great to hear on Instagram. Oh, nice. You come from Instagram. All right. It's great. Just starting to learn Python. Very nice.

[02:04:07]Very nice. It's so neat to see other people like coming across from different platforms. Yeah. From Instagram. Hey, how's it going over there? That's awesome. Hey, you're you caught a live stream. This is where the Instagram videos come from is this video.

[02:04:24]That's great. That's so great.

[02:04:28]Okay, let's see. So, I want to Where are we at? Here. Where are we at? Where did we leave up on this? Here we go. Here we go. Okay.

[02:04:40]So, I can tokenize. I'm going to go back to one hot really quick and I'm going to try to test this to see if this works here. Blah blah blah. We're going to get a loss going. We got a loss right here.

[02:04:49]Let's print our loss. We're going to do it backward and then we're going to do an optimization step. Oh, how is your subtitle so fast and accurate? Hey, right. Isn't it really good? It's so good. I'm glad you asked. We just happen to have a pinned a pinned tab in our Chrome browser here. If you want, it's free and available right here. It's on our Discord. If you're interested, go under the link share. Scroll up here.

[02:05:13]You'll find it. You'll find it right here.

[02:05:17]OBS is OBS subtitles for YouTube and Twitch. Free, fast, instantaneous. It's so good. It's good right here. Yep.

[02:05:30]Right there. And it's also powered by my communication technology, PubNub. It's powered by and my obviously, as you saw, it's really fast. It's a really fast technology. PubNub right here. See, for example, you can kind of think of as as a chat. We've got speech to text, which goes through your microphone, gets translated to text, transmits over my technology pubnob to this tab that I've got. I've got a I've got a tab over here, right here.

[02:06:00]This tab right here, and it then transmits that tab voice into OBS. And I've got an OBS overlay.

[02:06:11]That's how it works. And it's free. It's open source available for you.

[02:06:19]All right. So, let's get back to I want to see if we can make this work really well. So, I do have a We have 6 minutes left, you guys. So, 6 minutes. Will we make it in 6 minutes? Will we make it in 6 minutes?

[02:06:31]There's the loss. All right. Let's do the step. Let's do the step. Optimizer step. Is this right? Is that Yep. Okay.

[02:06:38]So, we got our optimizer there. Let's try it. Here we go. Here we go, you guys. This is uh 100.

[02:06:47]All right. Oh, wo wo wo. We're printing too much stuff out here. Let's print.

[02:06:52]Not that. Not that.

[02:06:56]And we only lost. Okay. Here we go. All right. 16 minutes left, guys. I know, right?

[02:07:01]Wait. Oh, it's still Wait, did I not do that right? Hold on.

[02:07:06]Where'd this go? Where' this go? It should only print the It should only print loss. Oh, hold on. I see. I see. I see. I have another print up up here somewhere. There we go. That's what I was looking for. Okay.

[02:07:21]Oh, the loss is going the wrong direction. Uh-oh. Whoops.

[02:07:26]All right.

[02:07:29]Okay. All right. Well, that definitely didn't work.

[02:07:34]Okay. Oh, maybe maybe we need to lower lower the learning rate here. There we go. Let's lower that learning rate a smidgen.

[02:07:43]Hey, that's what I'm looking for. All right. All right. All right. Let's get rid of the epoch here. And let's increase the learning rate a smidge.

[02:07:52]Okay.

[02:07:54]Oh, it's going the wrong direction.

[02:07:56]Let's give it a few more epochs.

[02:07:59]Let's go. Let's just There. Let's reduce it again. Zero. There we go. Extra zeros. Extra zeros. So, I'm probably going to give away most of my features for free. Okay. Yeah, I read that one already. Uh, let's see here. Would it be great if you ever need a React developer? I'll be on the lookout for a job posting race. All right. We'll let you know. Absolutely. We'll let you know. Okay. So, my model didn't learn. All right. So, we need to make some adjustments. We got to make some adjustments, you guys.

[02:08:33]Oh, hey. Kyle just said race overs.

[02:08:37]Kyle's interested.

[02:08:40]Kyle said maybe, maybe, maybe.

[02:08:43]Okay, so that didn't learn. Let's increase it a smidge.

[02:08:48]All right.

[02:08:51]All right. So, yeah, it's not learning.

[02:08:54]It's not not It's not working. Okay. So, we have to come up with a different approach. We got to do something different, you guys. I think that'll be a plan for tomorrow. Oh, tell me more.

[02:09:04]Ah, tell me more. Hey, Kyle. Very nice.

[02:09:08]Very nice.

[02:09:10]What if we don't do this? Yeah, we don't do that. Then it just goes uh in the wrong direction. Okay.

[02:09:19]Yeah, that's not good. Oh. Oh, it's not good. Okay.

[02:09:27]Okay. Okay. So, what if we did some try that? Oh, wow. Oh, it's Oh, it's not not happy now. It's not happy.

[02:09:42]That is not happy there. Okay, let's try this. Okay, nice. Y'all are doing business. You're doing business in the chat. Also using a cursor. Stephen, I see your pink cursor.

[02:10:02]Yeah, I do. I've got a cursor. Yeah, this is just my the location of my my cursor. Yeah, you a smear cursor. I don't know what a smear cursor is, Stephen. Some just changes with the scale of the loss.

[02:10:19]Yeah, I figured I figured I know. I was I was messing around. I was just I was playing around. I was I didn't actually break it really. That's a good point.

[02:10:25]All right, that's a good point. So, the loss could have actually just worked anyway, even though it was very big loss. Let's try it again.

[02:10:32]Okay.

[02:10:34]Smear cursor is a package for Neovim.

[02:10:37]Oh, got it. Okay, so it's going the wrong direction.

[02:10:47]Yeah, it's still it's still it's still struggling. Let's try this.

[02:10:54]Yeah. Then it fails. It fails.

[02:10:59]There we go. There we go.

[02:11:03]I want to see what the smear cursor is.

[02:11:05]All right. So, Google, you call it smear cursor in BIM.

[02:11:13]Here we go. Let's take a look. What does that look like?

[02:11:17]Let's play the video.

[02:11:20]Oh, hey, that's fancy. Yeah, I don't have that.

[02:11:26]I do like it. I think that's pretty fancy. I do not have it.

[02:11:32]Is there it? It might look I mean I am I'm moving pretty fast on the screen, so it might look a little smeary, but it's not.

[02:11:47]Is it against the rules? Really? Wait.

[02:11:50]Uh, wait. Am I bad if it is, Kyle? I don't think so. I don't think there's any bad situations here.

[02:11:59]Oh, we do have 1 minute left, you guys.

[02:12:01]We got to go. We got to go. All right, everyone. Thank you so much for joining today. Had a lot of fun. We built a little bit more on our model. We We are trying to get the optimizer working. And the good news is we made some progress.

[02:12:12]And Nea gave us the lost criterion. So that way we got our model at the beginning getting actually running through the back propagation. The only thing is it's not learning anything. So what we have to do tomorrow is figure out why. Let's figure out why tomorrow.

[02:12:27]We'll beginning our training process for our our chat GPT model, right? We're building a transformer. That will be the plan for tomorrow. Then we will get that working. I'll add some uh positional encodings as well. I think that will be the plan. Bun zupy. All right, see you later everybody. See you Kyle.

[02:12:48]See you Nea. See you Ray Silvers. Good to have you guys here. Good night, J bro. Thank you everybody. We'll be back tomorrow with more AI, more transformers. We're going to build our GPT, you guys. Bye, man. Have a nice day. You too. Ray Silvers. Oh, Sergio, have a good rest of your day. Bye, everybody. And of course, I love Liberty Justice. If you're still here, thank you. Thank you so much uh for all the donations. Peace out, dude. RXT. All right. Thank you guys. See you tomorrow.

[02:13:19]Bye everybody. Have a good rest of your day.