DL Gen AI - Live Session | Quiz1 Revision coding

[00:00:00]and I'll start uh your questions and based on those questions you'll be uh I'll I'll give some time to answer those questions then we'll discuss the solution.

[00:00:18]So you have um first question here. Uh what I will try to do I will try to paste in the text uh format so that uh becomes okay. So this is code only.

[00:00:43]Okay. So this is what is going to be the output of this uh print statement here.

[00:00:49]So that is your uh first question.

[00:00:56]So what is this print statement is going to output.

[00:01:42]So are you expecting to tell us or should we write into the chat? Uh sorry.

[00:01:47]Yeah. So that is fine. You can write in the chat. So this t dot n dimension will be two right and then shape is going to be 2 + 3 and then the number of elements. So this t dot uh num l this is the this this function counts the number of elements right so the number of elements uh in this tensor are uh six right so this is going to be the six so it's 2 into 2a 3 sorry 2a 2 into 3 you can say shape is 2a 3 and then t dot n is going to be six okay so I'll give this second question.

[00:02:36]Now here you have to tell the data type.

[00:02:42]So this what are the print statements I'm going to uh output.

[00:03:03]Yes. So, one is going to be float and one is going to be uh int, right? So, first one is going to be float and then the second one is going to be in. Okay. Now, here you have to tell uh what should we write? Can you >> So, for the second question, can you run the code once?

[00:03:24]So you have to run this also.

[00:03:30]There will be many questions where it is not possible to run the code. Okay. So I'm running it because for these questions it is possible but then we will be avoiding running the code for all the questions.

[00:03:50]So here in the third question you have x you have w. So to calculate the result you have to figure out what should be right what kind of thing should be written here in this uh blank line.

[00:05:02]Goodbye.

[00:05:30]Sorry, I was not speaking. Sorry. So this float is something that we have to write here to match the data type, right?

[00:05:37]So not the transpose here because one is integer, right? X is in integer, W is in float.

[00:05:45]So first we have to convert them.

[00:05:49]So make sure that you very be careful in certain situations such questions can be asked. So what if we are multiplying or taking the dotproduct of a matrix and a vector uh and if uh the data type is not same then we have to do transpose and float both.

[00:06:10]>> Yeah. First we have to convert it to float then we can do the uh transpose because transpose does not require anything else right transpose is just for that particular matrix. there is no uh there is no other variable that is going to be involved for the transpose right so transpose you can do without root also but then when you do some multiplication then you have to make sure that the dimension is uh not the dimension the the type is matching >> okay >> okay so one more uh now here you have to tell what is going to be the shape for 12 2 + 12 2 + 12 right so that's yeah that's correct so it's going to be 2 + 12 right because you are fixing that I want two rows and then based on all the other elements uh adjust them somehow right that the total number of elements are same.

[00:07:25]So this adjustment because there are total 24 elements. So now two rows we have fixed. So we have to get 12 columns to make the uh calculation be equal to 24. Right? So this is 1 2 3 4 question.

[00:07:41]Right? So give the question number five.

[00:08:03]Now you have to tell here what is going to be the output of these print statements.

[00:08:29]So first will be five.

[00:08:41]Now your voice is breaking it. Somebody >> five and then five comma 1.

[00:08:49]five and five comma 1. Yeah, you understood this right? So when you do not give any parameter inside you directly write squeeze it is going to remove all the all those uh dimensions where which has size one. Right? So here this has size one and then this has size one. Right? So these two dimensions are going to be removed and if you write zero then it will not remove all one size dimensions. it is only only going to remove the size of so if at zeroth position the size is one then only it is going to be removed right but so if I write so if the zero means this position right and it saw that okay there is uh size one at this position then it is uh it has removed so the output uh came out to be 5a 1 >> sir if if if it is the index you are seeing like this then if I do square squeeze of two then it will do it remove >> it will not do uh it will not do anyh yes for two it is going to remove the last one so it will be 1a 5 correct >> okay >> but if I write one it is not going to do anything why because this is not one so nothing will happen you get all uh everything same >> sir can you explain one second Yeah. So what happened with the squeeze?

[00:10:17]Uh so squeeze just takes uh the dimension which has size one and removes them. So if we don't specify at what uh location we want to check the size for example here I'm writing one. I'm specifying that go to the location one.

[00:10:35]If that has size one then remove otherwise do not remove. But if we don't specify anything it is going to see all those positions where the size is one and remove all of them. So here it has so after this uh execution the output is only five. That means this this one has also been removed and this one also uh has been removed. But when I specify a particular position then it says that I only going to do perform the action action at this particular location not all the locations.

[00:11:08]Right? So if I write here zero that means I'm only going to check this first position and if the first position has size one then I'm going to remove it otherwise I will keep it. So zero means first position. So at zero so it has one here. So when I run it it is going to remove this and you will get the size 5a 1 only. So that is what is coming here.

[00:11:33]If I give one that means this position and this position uh does not have size one. So it is going to keep everything same as it is. So it is keeping everything same as it is.

[00:11:47]>> So uh I could not hear your voice is not clear but I hope it is clear. So I'm moving to question number uh six now.

[00:12:04]Okay. So you have to find the output of this X grad here.

[00:13:14]So 26.

[00:13:16]>> Yeah, correct. So it's going to be 26, right? If I run it, it should give the answer 26.

[00:13:26]Okay. So no need to explain this, right?

[00:13:29]Because it's just exactly the gradient calculation of this equation here. So it's just written in the code format.

[00:13:45]Okay, one more similar question.

[00:13:57]>> Hello sir.

[00:13:58]>> Uh yes >> if we if we if we give a require grade is supposed to be false then what will happen?

[00:14:04]>> This will th error right? Because then you cannot do this.

[00:14:10]>> Yeah.

[00:14:16]>> So in examination only that answer will you have to type or that options will be given.

[00:14:22]>> It can be options given also it can be nity type also based on the type of question.

[00:14:30]Okay. Like for example, given a code snippet, you have to find the number of parameters that are there. Right? So then you will be typing the answer.

[00:14:45]>> So answer is 12.

[00:14:48]>> Take some time. Yeah. So can somebody explain how it is 12?

[00:14:53]>> The gradients accumulate and keep adding for three rounds.

[00:14:58]>> Correct. Correct. Correct. So that is why we write uh for example when we do optimizer.0 grad right we have to do that otherwise uh in training we have seen that we have to write gradient of zero grad right so that the this accumulation can be cancelled out.

[00:15:17]Okay. So we have this um so this this uh the gradient is going to be uh 2x right and at x equal to 2 you have the gradient coming out as four right 2x but x equals to two so this is four but this is going to happen for how many times?

[00:15:37]Three times right because this loop is going to run for sorry uh yeah three times right. So every time the same thing will happen and every time you are going to get the gradient four and since we are not making the gradient go zero right so it is going to keep adding and then the answer will become 12 >> sir what is the code to make it zero is it step or >> uh zero grat right we have seen that >> okay yeah >> in the training okay uh the source was question number seven.

[00:16:22]Now here you have to so these are the kind of abbreviation that okay this is line one this is line two this is line three four and five right so you have 1 2 3 uh so this is not this so you have 1 2 and then this is three this is four and this is five so you have to show that or you have to tell which line will throw the error should make it go here.

[00:16:54]This is actually this line.

[00:17:01]This is this line.

[00:17:07]This the last line.

[00:17:13]So, which line is going to throw the error?

[00:17:20]So line two.

[00:17:23]Okay.

[00:17:35]>> Sir, sorry I came late. Can you please explain the about two one tens that tens 26 and tens?

[00:17:44]Sorry, what what did you ask >> the about two quotes?

[00:17:49]>> About two quotes. Uh okay, I will explain. Let's um let this first uh be done then we'll I I will explain those.

[00:17:57]>> Okay. Okay.

[00:18:16]Okay. So here um giving the answer. So it is going to be line five, right? Why line five? Because um you have this C that you are initializing that is inside this with loop, right? Uh with tall short inference mode. Inference mode means there is no uh gradient tracking that is going to happen. So the gradient require gradient gradient is false for this C. So you don't have any information about the gradient to calculate for C. So when you write C dot backward it's going to throw error it is the error.

[00:18:53]>> I have a question on this. You're doing C do.backward outside of that inference loop inference uh mode right? So uh what what is it that it does not track inside that uh that it uh you know gives an error when you do C dot backward like outside of it?

[00:19:13]>> Sorry I did not understand. So you're saying that should I keep it inside this within loop?

[00:19:19]>> Yeah. So for example say on line two B is equal to A into 3 right which means that there is some data being captured there because of which B dot backward works. M >> but when you do it C is equal to A into 5.

[00:19:32]>> No B has B the backward is working because A has this gradient right? A has grad required true.

[00:19:39]>> Okay.

[00:19:40]>> So because of this flow B will also have the uh because B has also initialized. B is just like a function for uh so B has been initialized in the kind of you can understand that it is kind of a same environment as A >> but here C has been initialized in different environment. the environment was that I don't want so it is like it is going to block >> the gradients over there so basically do gradient equal to false is it when C is initialized >> correct correct correct correct because this is kind of a uh one different environment is going to be created >> got it okay >> sir here uh in the second line the data type are not same we did something similar in question two and that gave the error >> here we are Actually uh what we are doing is it's kind of a simple uh only kind of integer applic or kind of a algebra that is we are doing not kind of a matrix multiplication or some list uh operation. So for that it is going to automatically uh make it uh easy. So it's kind of for example if I write here um B right. So one second.

[00:20:58]Yeah. So this is C six dot right. So it's anyway making it float because my A was float. So these are kind of some simple operations where it can automatically say that okay uh there is no problem because one is float. When I multiply a float with integer I'll consider B also becoming as float. But when we do some operation with matrix and all there it becomes kind of more careful that are you sure that so instead of directly or inbuilt uh from itself converting one into different converting one type uh changing one type because of one operation it just says that are you sure that you want to change the operation. So they want to change the data type. So it just becomes because of more careful becoming more careful we just so they have decided not to do for this matrix or list multiplication. So that's why there it throws error and here it is kind of safe to do because it is just simple algebra operation. So it kind of becomes safe to do but as I said also in initial time that always try to make so in practice always try to be aware that uh or be aware and take always flow data type because it is easy to work with flow data type >> or avoid some unwanted errors.

[00:22:16]>> Here line five gave the error because of torso inference mode. And what about torch eval >> that is same right? Even in if you go in torch.e also you are saying that your so there we don't say torch.e we say model.l so it's like >> it's not like so there is no environment you are setting you are just making your uh the difference is you are making your model uh so your model has various uh things uh with it. For example, if you say model train, right? So there are various functionalities that py says that okay now if my model is in the training phase give model some of these additional tools so that it can perform training well but then those tools going to cost you computation also. So for example if there is uh you are working with large GPU right and then you write model eval then your GPU load will decrease because it is not in the training mode. So there are the tools that are available to model has been reduced or kind of has been shut down.

[00:23:16]So model eval will make uh the load on the load of working be less. So something like that. So that's the difference that's about the model. This is about the setting up the environment that whatever happens with this dot inference mode all everything will not have this. Uh so it's like so when you say request grad equals to true it's kind of a there is a graph there is a gradient flow that you have to track that that this this tracking happens within this pytor right so this tracking is also kind of a very big thing that you have to so the pytor has to make sure that okay from where all the directions this gradient is flowing and it has to keep track of all those directions all all that gradient flow so that when it does the uh back propagation it can uh it it so that that's why it is able to do that back propagation. So it's a very hectic task actually to do right to track all of these uh gradient flow. So when you do toss inference mode you are completely avoiding that extra uh kind of this huge task. So that's why this is the difference.

[00:24:21]>> So can we also use torchnograd?

[00:24:26]>> Um I think we can use but let's see.

[00:24:31]But is it go does it go with the environment? So with tosh no grat I'm not sure but let's see.

[00:24:40]Yeah I think it works like this also.

[00:24:43]Right. So if I remove this and then I print B I don't think I should get any error. Yeah.

[00:24:54]Okay. So let's move to next question.

[00:24:58]uh >> okay now so I will not have the so we will not be able to run the code but that should be fine right because this >> I just explain the lines no need to run the code >> yes yes yes so I will be putting this in the text >> um yes >> sir >> yes Yes, >> in the previous question like we printed the A after the B dot backward right.

[00:25:36]So a should get b should get differentiated and the output should be uh only two right sorry >> output should be three now >> no so b is uh a into three right so after differentiating only three will be remaining so after b dot backward will be three Um you're saying that B dot backward should come out three.

[00:26:14]Okay. How to >> both output as six.

[00:26:21]>> No it is right. I mean two has a this thing.

[00:26:25]>> Okay. What he's saying is if three differentiated it should be three. Okay.

[00:26:31]>> Okay. I think this so I have not written it correctly. I'm still printing B only.

[00:26:35]It's grad is what should I print?

[00:26:39]Sorry. Uh what is this?

[00:26:46]Yeah. So it is coming none right because it becomes constant there.

[00:26:53]So it's not but as a value it should be three.

[00:26:57]No, then I should. So, for example, if I have uh B equals to two, right?

[00:27:13]And if I write B dot grad, what should I get? Okay, I said print it.

[00:27:38]uh pen here. Okay. This is B is constant >> element zero of tensor does not require that. Okay. Okay. So I have to make it grad also.

[00:28:03]>> So can you scroll down? I think the answer six is correct. Right? If it's just please scroll down.

[00:28:09]>> H yeah here it is coming one right? Ah so why it is coming?

[00:28:14]>> B is A into three. So B will get initialized as six which is a constant.

[00:28:19]So now if you do a backward it will stay a constant which is six.

[00:28:24]>> No then answer is coming as none here right?

[00:28:28]>> Then here also it should be one as we saw in previous the derivative should come as zero.

[00:28:35]>> Yeah for that it none also makes >> yeah that yeah exactly it could be because of that. Mhm. But here it is coming one. So yeah, I mean let's uh okay let's let's move this uh we'll we'll discuss this later. I'll see what's the point here. Okay. So we'll come to this later. So where we were Yeah. So in this question >> no no sir sorry I'll just one more point to add there's a warning there which says that that grad attribute of the tensor. No, if you scroll down, it says of a tensor that is not the leaf tensor is being accessed. So maybe it is accessing the uh the gradient of a and not b is it possible because you have written b as a.

[00:29:20]>> Mhm. Could be correct. Yeah. So it's saying that the grad attribute of a tensor that is not a leaf tensor is being accessed. Yeah. So uh it's grad attribute won't be populated during autograd. If you inh field field to be populated for a non-leaf tensor you start retain gra something something I think yeah something much deeper I also >> sure about that >> um no no but we are actually trying to find for this B only so that's why I'm not writing a anyway so we are moving this so this was anyway not the discussion we'll come back to this we'll see this what can be done so okay so this was the question, right? So the point here was that if we initialize some uh variable inside this uh in inside this environment, then we cannot take the gradient for that, right? Even though this environment even though that variable involves some other variable which has the gradient true. Still the variable that has been initialized within this environment will not be able to uh have the gradient uh available. So that is the idea for this.

[00:30:33]>> See your question six because the function that you have given is y is equal to something but then you have asked for x.g where it is coming from maybe here also it has to be a do.

[00:30:50]>> Ah okay okay. Yeah. So we are kind of tracking the gradient of uh okay correct correct. So if I write here um yeah so what okay here you are saying right when we write >> B we should write A correct correct let me see >> um okay we again going back yeah fine we'll come back to this later anyway so I think that's even beyond this discussion so what I was >> uh trying to test is different >> so how many questions are we like trying to practice just today >> there are many questions that's why I'm very going very quickly >> move on yeah please >> yeah yeah so that's why once we done with that then we can again come back here and see okay so what about this question so here again it's simple you have to fill these uh empty lines >> it's zero grad and then backward and then step >> optimizer dot step sorry optimizer dot zero grad.

[00:32:01]>> Yeah. So that is zero grad and then loss what about the loss >> dot backward?

[00:32:08]>> Yeah. Loss dot backward and then optimizer.step right?

[00:32:13]>> Yes.

[00:32:13]>> Okay. Okay. So another question this this can be pasted here I think.

[00:32:26]Yeah. So can leave this. Um so what can you do here? So what are so you have three four uh lines that you have to fill.

[00:32:45]First one is a super. Second line is NN dotparameters and third is parameters and fourth one is uh forward.

[00:32:57]>> Yes. So fourth one is define forward.

[00:33:05]>> Okay. So what what you are saying what should go here?

[00:33:10]That's super >> super something super >> super what? So anything else or just super here?

[00:33:31]>> Class class name. No sir after the bracket. After bracket super of linear model.init in it.

[00:33:43]>> Okay. So, this is uh this thing and then here what are you saying?

[00:33:49]>> Parameters.

[00:33:52]>> Okay. And what about the last line?

[00:33:55]>> Forward.

[00:33:56]>> Forward.

[00:33:59]>> So, it will be parameter here also parameter, right?

[00:34:02]>> Yes sir.

[00:34:03]>> Yes.

[00:34:06]and it will be forward.

[00:34:10]Okay. Yeah. So that is fine. Okay. We'll go to the next question.

[00:34:16]And now you have to So I'm asking the question again. It's just the code is different. I mean code is also going to do the same. What will be the output of this this code here? It's related to the previous question.

[00:34:32]>> Hello sir. In the super in the super calling we will not pass model. No two >> uh sorry what are you saying?

[00:34:43]>> Uh we can pass in the super as a parameter linear model the self class.

[00:34:50]>> Yeah yeah yeah but let's yeah so let it be like this only here and uh so for such questions you will be anyway getting the uh options. So there will be no problem that there right because how else we can ask then right.

[00:35:06]So what can you do? So what is the output for this uh which is related to the uh this this uh so linear model is the class that we have defined here in this section. So what is the output of this print statement?

[00:35:25]It will be two two.

[00:35:32]Uh any anyone else?

[00:35:39]Okay. Yes, everyone is telling it's two.

[00:35:42]Okay. So now I'll put as text. Now in this which uh method is the correct method to call uh or yeah to run the model >> out one equal to >> model that's like >> out one right so out one is the correct way right because we have discussed that also I think in session that why we cannot directly call forward right because it is inbuilt and it automatically calls forward when we when we run this linear uh where yeah here >> I think when we say model x >> we can call this other one also but it won't inherit the superass uh >> correct correct correct yeah so that's what we discussed in the model uh in in previous session so when you call like this it just anyway you are running this forward uh uh this function only.

[00:36:47]Okay. So yeah, so that is the correct way.

[00:36:52]Now for this question, uh you have to find the output for this print statement and for this print statement.

[00:37:38]Somebody I think typing then if you type and run it then it's not going to be helpful for the exam.

[00:37:45]Try doing it uh without printing the output.

[00:37:52]360 60 it's saying that this is 50 and okay this is 50 and this is 16 comma 50A 3 I don't know we'll see uh later any any other uh answer there are two print statement not just one. If you are saying just 50 then like there should be two print state uh two outputs one coming from this print one coming from this print.

[00:38:47]So sometimes I get confused with this torch random and the Python random.

[00:38:53]So we say 50 comma 3 is it like 50 random numbers or like 50 from range 50 to range something.

[00:39:04]>> No. So this random is all about the range. So random means you have the data has been generated from the standard normal distribution. Here you are specifying the shape.

[00:39:15]So 50 it's like 50 rows you have and three columns you have.

[00:39:21]>> Okay. Directly the matrix is generated.

[00:39:24]>> Correct. Correct.

[00:39:28]>> With the normal distribution zero mean and one.

[00:39:32]>> Yeah. Standard normal. Yes. Zero mean and one variance >> for integers. Is it directly rand like that?

[00:39:43]Uh no it's not about integer it's rand n is the n stand for the normal so I'm not sure about the integer I think we can assign the data type but then it doesn't go I don't think you can make it integer here right because then in a standard normal distribution you will have values anywhere from most values are like close to zero so not sure how to make it integer within that like how to generate integer numbers So you have for you have other uh functions for that right for example arrange you can call to create uh l space is also there right to create some integer values even lin space will it depends what uh space you are uh putting or how many numbers you want to generate but here I don't think you can uh make the numbers or you can generate integer numbers So what will be the output? Uh four batches. Uh okay. And then the shape is 16, 3.

[00:40:58]Yeah, that's correct answer. So you have four batches. Why? Because Okay. So first of all, you have to understand this this part here, right?

[00:41:10]This part is saying that you are giving the data and then the batch size equals to 16.

[00:41:16]That means each batch has 16 data points. How many data points you have?

[00:41:21]You have 50 uh data points, right? So in uh in three batches you have 48 data points. Then you need fourth batch also to accommodate those two extra data points. So total four batches will be required.

[00:41:40]So that's why this four and then this we have used this also in some of the live sessions where we are just making this DL as iterable and then we are uh kind of uh accessing its uh one batch.

[00:41:56]So this is XB comma YB. So it's like the data and the label of so that is the batch that we have right. And then x dot shape is going to give the so it's kind of for one batch uh for the for one batch the shape of the training data. So each batch has kind of so this is for the first batch only when you write next iter so you have made a data table and then you're saying that next just like the start uh you're starting from the first batch so the first batch will give you 16 uh data points.

[00:42:29]>> So a quick question on this. So the the the first uh line right totrand 50 comma that it produces 50 rows and three columns technically right like three features and 50 rows.

[00:42:40]>> So I understand the print DL where you're so you're dividing it into batch 50 into batches of 16. So you might you'll end up with four batches right that's why it's four >> XB.shape. So the next item should give you the first batch and that batch will have uh okay that batch will have uh 16 uh rows >> first four batches first three batches will have 16 data points >> 16 rows and three all right okay >> so if you put next uh thrice or twice uh the batch size will change it when it go to the fourth batch >> yeah yeah then it will come out to be true >> 2a 3 >> so what This y be whole here.

[00:43:25]>> So in that case uh next should be three times because this is taking the zero batch.

[00:43:32]>> Correct. Correct.

[00:43:33]>> Yeah. Somebody else also asked something.

[00:43:36]>> Yeah. So what what will be the value of y here?

[00:43:40]>> YB you tell what will be the size here.

[00:43:43]If I write print yb dot shape what should come here?

[00:43:49]>> 16 comma 1.

[00:43:51]>> Right. So the number of data points. So here it is just the batch right? So it's number of data points here and here also the same because these are the label for these data points only right.

[00:44:00]So it's just that since this is label you have the uh the feature is just one you can say or the column is just one.

[00:44:09]>> Got it. Thank you.

[00:44:11]>> Okay. Now this is another question. Um so what will be the output for all these print statements?

[00:44:39]error. It will give error.

[00:44:51]>> So >> let's say does not give error. Let's say it is not going to give error. Then what will be done?

[00:44:57]>> So will you be sharing this collab with us?

[00:45:00]>> Yeah. I will share.

[00:45:08]>> So 3.5 first.

[00:45:28]I don't think it is going to be two.

[00:45:33]No, I don't think it is going to be two.

[00:45:36]Think of this dimension thing uh carefully.

[00:45:40]>> The first dimension >> 2 3 4 2 3 4 >> 2 3 4 Okay. How how come? 2 2 3 4 >> mean and the close direction.

[00:45:54]>> So two two is the mean of what? What are the values of >> uh 2.5 3.5 and 4.5?

[00:46:01]>> Yeah. So it is going to be 2.5 3.5 and 4.5 right?

[00:46:05]>> Yeah. Yeah. Yeah.

[00:46:10]>> Yes. So what is happening here when you write dimension equals to zero? What happens? you are kind of averaging your columns right like column wise average you are trying to find and since the shape is 2a 3 so you have data in what uh I think I should print this also first so your x will be in this in this manner right so now what is happening here when you take dimension zero you are averaging this 1 and 4. So 1 + 4 / 2 that is 2.5.

[00:46:52]2 + 5 / 2 it is 3.5.

[00:46:57]3 + 6 / 2 is 4.5. When you when you make it dimension equals to 1, you are averaging in this manner. So this is 6 / 3, right? That is 2. You average these values row wise. So that is 15 / 3 that is five.

[00:47:19]>> So but normally dimension dim means rows.

[00:47:27]>> Yes. So but it is like you are collapsing the row here. Yeah. You you kind of say that you are so it's kind of a row wise also you can say right it's just that you're taking the average across the rows right in dimension zero.

[00:47:40]It's just that I might have used different wording. So you take one then you take four it's like across rows you are going right here if you say you say across columns you are going >> okay one more question >> quick question would we have to deal with more than two dimensions here like what if it's a as an additional channel like you know >> and then I also might not be aware of how to even Okay, >> how those go? Yeah, we have to see the out. I don't think that much complicated code also going to be asked, right?

[00:48:19]Because this is basic. So, we will not test your calculation in let's say 10 dimensions is just going to be this way.

[00:48:28]>> Okay. So, what about this uh what will be the output for this?

[00:48:34]>> So, we already have it in the last question.

[00:48:37]>> It will throw error.

[00:48:40]it it will throw the error because mean read need float and it is >> correct. So we discussed this in uh first session. So when you calculate mean yeah you divide by certain value.

[00:48:51]So when you divide >> then this output uh division output expects float right because you may not get integer value after that can be asked here we to have float value. So here we are not giving any float value. So this is kind of a trick question that can be asked here. We have root float here to make things work.

[00:49:14]Yeah, >> trick question. I mean the the data type of X is uh is integer. Okay. So but when you're doing X dot mean, >> it is not storing it in back in X anywhere, right? It is just printing it.

[00:49:27]So why doesn't it just >> then it is calculating it. So it so you are not storing in in in in the back end. There must be some intermediate steps where it is getting stored and then the calculation is happening right because every time when you do so it's kind of a read and write operation if you want to think from the back end side right so when you do read and write operations you are kind of a storing then you are fetching kind of reading it right then you are writing means you are updating some memory location so there is some storage anyway that is happening otherwise it will be difficult right how will you calculate it so it's just that you are not storing it in your uh in in your created variable.

[00:50:10]>> So what if we give at uh at last we give intstead of dot float that >> uh then that's the problem right because this mean calculation will be problematic right because you cannot you're forcing it to be so anyway it is int only right so now I'm not writing in any way it is int so then also this this error is coming right you can make this go in like this that is other thing right that is you are saying that I made I keep it float sorry but here at the end I want to print integer only so that is fine but then with the int values you divide those integer values that means it is expecting now float because the value is not going to be integer after the division So that's why this error is coming.

[00:51:06]>> So so even if instead of int print statement we write float it will still give an error.

[00:51:14]>> Uh no it is float only. So should not give error.

[00:51:19]>> Oh because mean is calculated first.

[00:51:24]Okay.

[00:51:25]>> So it is just changing. If it is float it is anyway going to be a float. If it was int then then it would become float.

[00:51:33]For example, you are saying I make keep it int and then again I make it float. Right?

[00:51:39]That's also going to be fine. I guess right >> because it was first int and then you this integer value you have converted into float. So what I read is like is that it stores the floating point numbers in separately in the GPUs that's why it Maybe the GPU type. So we are anyway not going for the GPU here. It is normal calculation happening. Uh so might be different in that sense not specifically. So we are anyway not going into GPU memory right >> even in CPUs it says anyways something like that it gave.

[00:52:23]>> Yeah. Okay. Okay. Okay. But anyway there is some kind of so the point is this x dot mean when you calculate this mean will become so after this calculation the output will become flow but what you have given is so your kind of uh so this is for the pytorch uh something related to pytorch only. So PyTorch has made these things very clear or strict that if you have integer and then after the multip so in in normal Python you can do all that right in normal Python you have this integer values you divide it with something and then it automatically going to convert to float when you get the value right because you can you can say B equals to A plus B divide by 2 and A and B are integer but then B will become automatically float without throwing any error right because that is the Python handles all that but the tensor the PyTorch has made it very strict because they want to make sure that you are all you are really uh like make the difference so that the further calculations or further operations be very clear. So this is something related to pytorch only.

[00:53:33]>> So but we are not giving it as tensor we just giving a range >> this torch we have here right? So it's PyTorch. It's coming from the PyTorch.

[00:53:42]>> Aange will create a tensor. No.

[00:53:44]>> Yeah.

[00:53:47]>> So does a range uh return as tensor?

[00:53:51]>> Yeah, it just so it's coming from PyTorch H. Fine. Right.

[00:54:06]>> Yes sir.

[00:54:08]>> Okay. So let's move to next question.

[00:54:09]Now you have to find the print statement of this All right. So this is again so this star is for the element wise multiplication right. So that will not >> yeah so this will not change the uh shape right this is going to change the shape according to how this multiplication is happening. So a is yeah so a is 3a 4 this is 4a 3. So it will become 3a 3 right? So the second operation is going to be uh the size is going to be 3a 3. For the last operation it's not possible to do multiplication like that. Right? So it is going to throw error right.

[00:55:40]Okay. Uh so you so you can understand the pattern here also right. So such questions can be easily asked and uh you can also be okay with such questions thinking that you don't really if you have done enough coding you will be aware of the output without even being able to run the code right >> so but uh there is another method inside torch itself right for multiplication is it uh mul or mt Matt Mal.

[00:56:16]>> Yeah. Matt Mull. Yeah. Right. Yeah. So, Matt Mull is there.

[00:56:20]>> It is Matt M.

[00:56:21]>> Yeah.

[00:56:22]>> Okay.

[00:56:24]>> Okay. So, for here you have let's say uh uh so what you have to do here you have to complete the classifier here. Okay. So this classifier and you have 28 + 28 grayscale image 10 classes one hidden layer of 256. I will write this down one second. Let me just copy paste it.

[00:56:57]So this is what you have to do. Complete this MLP for 28 cross 28 gray scale images.

[00:57:04]Um 10 classes, one hidden layer of 256 neurons.

[00:57:22]So first is nn dot view.

[00:57:26]Uh next is uh uh 1 + 20 1 + 10.

[00:57:33]>> You just type it in the chat now that >> yeah because here everyone is trying trying to answer. So you can type in the chat.

[00:59:17]So I think we can discuss now enough people have answered in the chat.

[00:59:23]Um okay okay yeah so what was this so flatten and then you have uh 784 from uh from 784 to 256 256 to 108 I think that is fine right so you here you just need to flatten it right so you will write just n dot uh flatten yeah so this goes here and then in features are going to be uh whatever the output comes comes here right so whatever the number of uh the number that is here so it is going to be when you flatten this 28 + 28 you'll get 784 right so here it is going to be 784 and then out feature it should be 256 because one hidden layer of 256 now here you have 256 because output of the earlier layer was 256 and then out feature should be 10 here right so this is uh these are the values So it is not more like such questions I don't know how can be asked but uh these are also kind of important questions which will give you idea about the >> so one general question here uh these variable names self.flatten selflinear uh layer 1 these can be named anything right? Yes. Yes. Yes. Yeah. This is you uh giving some name to store the uh values.

[01:00:53]Okay. So I think uh the same question like similar question just that few things will be different.

[01:01:12]So here okay avoid this error because anyway it is not supposed to be run.

[01:01:19]So predict the say the shape of each mark point for input X of shape this.

[01:01:27]So you have to predict the shape of A, shape of B, shape of C and shape of D.

[01:01:59]and then you can have this uh layer. So these layers are related to this question here. So when you calculate this layer one, it is this layer one here. So this this all okay I should put this here inside this forward.

[01:02:16]Okay. So now you have this way.

[01:02:34]So this 128 is the batch size.

[01:02:38]>> That is Yeah, it is batch size. Make sure you know this thing. It's going to be batch size, right?

[01:02:46]>> Batch size, channel and height and width.

[01:02:48]>> Correct.

[01:02:49]>> And this is the uh universal format.

[01:02:53]>> Yes. Yes. But mostly in questions we will be uh so it will be specified also.

[01:03:00]If not then you can go with this. Yeah.

[01:03:13]Yeah. So here the uh shape will be what should you do you want to take time or should we discuss?

[01:03:24]>> We can discuss sir the same question.

[01:03:28]>> Okay. So here we batch size is not going to go anywhere right. So 128 comma something is going to be there since we are flattening then it should be >> yeah but 128 comma 784 that is going to be the shape here right.

[01:03:44]Uh what about shape here?

[01:03:48]128 comma >> yeah 128 comma sorry >> 256 >> 256 >> correct 256 right and then in the relu case what will happen >> it's the same >> same same size >> so it is going to be same here and here again so what will happen here >> 128 comma So it's a little difficult to visualize because doesn't it uh doesn't don't we feed one feature at a time like 784 vectors one at a time to x1 here like 2x >> no that is I think I will not be going in the detail we have discussed that in previous live session so what happen is it goes batch wise so all these yeah so for example when you are flattening you are flattening one output one input But it is kind of stored it has happened for 128 inputs one time. So it's kind of a batch. So it's so that's why this calculation batch wise calculation happens. So everything so these operations will happen for one input but it will happen for each batch. So for uh each input inside a batch.

[01:05:07]>> So that is why >> one quick question uh so when when it when it flattens the first first uh batch element to 784 right? M >> does it not directly pass that to the second layer or does it wait for it to flatten all the 128? Uh >> so that actually happens parallelly.

[01:05:25]Okay. So that is why we even do this patch wise so that we can go we can do the things parallelly. If we are not doing then anyway so each operation will happen and then it will pass to the next layer. Practically speaking, the layer 2 is not going to wait for 128 uh flattened uh inputs, right? It is just going to take the first 784 one and it is going to process it then the next 784 and it is going to process it and so on.

[01:05:51]>> Ah yes yes correct correct. So that is why we we are able to even do the uh training very uh very fast.

[01:05:59]>> Okay. Okay.

[01:06:03]>> Okay. So >> sir >> uh can you can you explain again what he said?

[01:06:10]>> So it is like when we process the data right in in GPU when we try to uh do the calculation batch wise. So this batch wise so all these calculations batch wise they happen parally. So that's why so the idea is just the to create this uh parallel calculation possible we use GPU and then we send this batch wise uh data to it and then it calculates all those uh for example loss that you are calculating that is for the batch one batch you are calculating loss right and then the back propagation that is happening that is also happening batch wise not one data per like not the loss for the each data not the gradient we are not taking based on the loss of one data but we are taking gradient based on the loss of full batch.

[01:06:54]So that is all of this. So here when this x has gone and this x1 has been uh initiated. So this x1 will store the flattened vectors on 128 flattened vectors. So that's why this size is coming like this.

[01:07:13]Here it is 128 flattened uh 128 passing through layer 1. And then the output because out channel was uh 256 then uh the out channel here becomes uh shape 256.

[01:07:32]Okay.

[01:07:34]Um let me see if this question can be asked.

[01:07:46]Okay, I'll anyway ask this question.

[01:07:52]Can be many solutions here. Not very a specific question for the exam point of view. But what will be what can you put here and here?

[01:08:22]So this predict uh okay so this predicted means what you have I'll I'll wait second >> one second just one second I have to give the question also then it will be understood.

[01:08:49]Yes. So this is the question uh fill in the blank to convert logits to shape 128 10 into predicted class indices criterion.

[01:09:17]uh try to write in the uh in the chat so that everyone everyone else also can so without even hearing the answer or Anything.

[01:09:57]You can tell just the function and then the dimension number here.

[01:10:04]So it's going to be arg max here somebody has wrote right. So here it will come arg max and here dimension will be what?

[01:10:12]Zero or one?

[01:10:18]>> Should be zero, right? Because >> Oh, it should be one because we are doing row wise.

[01:10:28]>> Correct. Right. So, it is not about zero. Zero will you are going across the data points, right? But that is not what you are doing. You want to go across the number of labels for each data point, right? So you have 10 labels in each data point. So across those labels you want to find the highest value right and then that indices will be given by argmax h >> so here the outputs is basically the uh >> logics you can say.

[01:11:05]>> Yeah logics. Yeah.

[01:11:06]>> So so dimension one I didn't understand there's only one dimension right? No, this is the shape will be like this, right? Even though these are logits, these are logs for each >> saying 128, 10. Okay.

[01:11:19]>> Yeah, this is the shape.

[01:11:30]Okay. So once this is uh finished then we will try to so it's kind of you have to identify some bugs that are there in the code.

[01:12:28]So whatever you find the answer make sure you just write the reasons in the chat instead of uh telling them loud.

[01:12:53]So are these questions are helpful for you to also understand the code but uh or or not? So I just want to have some idea so that if not really helpful we can change something.

[01:13:06]>> So this is helpful.

[01:13:08]Okay.

[01:13:30]I think many things are missing. So you guys are not able to give correct answers so far. So you try to think this is the validate that means this is the validation function.

[01:13:48]So this is the task of validation.

[01:13:53]So one bug I'm telling you that has been told right. So this model train is a bug right because inside the validation we don't do model.rain train we do model model eval correct. Now find the second B correct now I think you understood right so loss dot backwards so here we are not back propagating anything right because not calculating the loss u uh gradient here because it's the validation so in validation we don't write this correct so these are the two problems Fine. Let's move to the uh next question.

[01:14:52]It's fairly simple but I think let's discuss that.

[01:15:09]So you have to first tell what we are what we import also and then how do we split the data based on uh given condition. That's it.

[01:15:41]One thing I missed here. Think I should write you want to split randomly, right?

[01:15:47]So something that short should be written to make it more clear.

[01:16:28]So we want to split them randomly.

[01:16:30]We split randomly.

[01:17:12]Um yeah so random split is the correct thing that we want to import and then uh 0.8 I think Karthik Kumar you have written 0.8 that we cannot write here because we have to strictly follow this uh this criteria here.

[01:17:32]So here we have random split right and then we'll be using that here and then here what we have to write is so these are two things. So this is how we split the data. So it's going to be 54 training and then six uh for the validation.

[01:18:02]Not so important for the exam point of view but again it just will be okay for you to know this.

[01:18:22]Okay. So there is one question I think we I should have asked this earlier.

[01:18:29]forgot to ask. Uh let me keep it there.

[01:18:35]We'll revisit that this structure here.

[01:18:39]So we want to ask this question here.

[01:18:43]What will be the output for this?

[01:18:48]Don't need to give exact. Like like answer may vary here and there but what what is going to happen or like you don't need to follow certain pattern to give the answer. You can just give the answer based on in in any in any uh order.

[01:19:14]So this name I'm not so concerned about.

[01:19:18]Uh what you should cue correctly is uh this param dot size and then param.tre require gra 784 true 256 true and 10 true >> do we do we add I I don't know whether this is correct and the second question that comes to my mind is do we also add the bias to this >> yes sorry >> the name will be weight bias Yes, that should be like that, right?

[01:20:33]>> Correct. Correct.

[01:20:36]Um, okay. This this Okay, we have not initialized.

[01:20:47]Let me run this again. and all right. So now you have bias and then require grad will be true for all and this is the size for the bias. This is the size for the weight. This is the size for the bias.

[01:21:21]Again bias will be. So default bias is true.

[01:21:27]So make sure that you keep that also in mind.

[01:21:47]Okay. Now here one more thing what should be written here? So it's so these questions will have very structured options also available. I'm not able to give the options here but that's fine. I believe here you have to find what should come here.

[01:22:17]So complete the normalization uh transform for single channel image.

[01:22:22]Yeah. So this two tensor is very important right. It should come always uh before the normalization step. So that's why >> so two tensor will scale the input and this will normalize it right and uh >> no no it is going to convert your image uh pixels into tensor >> but it also scales it right. It scales between 0 and one something like that.

[01:22:49]>> Yeah, I I think um but not sure actually. Uh I think it scales but anyway the here the idea is this normalization should come first. Uh oh sorry this two tensor should come first before the normalization.

[01:23:19]So it is I think kind of scales.

[01:23:23]Yeah. But not sure. I think it makes the values go in 0 to one something like that. Right. So yeah that is uh that is there >> and this 0.5 0.5 it normalizes it between the range of minus1 and one is it?

[01:23:46]>> No. So it's like a standard normalization. Now you have mean zero and standard deviation one after this this step.

[01:23:53]>> Okay. Okay.

[01:23:54]>> Yeah. So values can be anything but when you calculate the mean it will be 0 and one >> correct >> sorry zero mean and standard deviation one >> and what does this 0.5 and 0.5 signify like in the parameters >> uh this is yeah so this is kind of a for it's kind of you are saying that this mean uh for the for the data that you have so it's kind of a single channel data right so for that particular channel the current mean of the data is 0.5 and then the standard ard deviation is also 0.5. So with for for the data of 0.5 mean and 0.5 standard deviation you are converting you are doing this normalization to make it mean zero and standard deviation one.

[01:24:38]>> Okay.

[01:24:39]But what is the >> yeah it differs. Yeah it differs. Then you have to give that value. There are some standard data sets for that those values are very even famous. you will directly uh get those values like for the mean of um uh mnest data you will directly find those means and you can put them otherwise you can calculate them also because you have the data right so it's just got >> one line of code to calculate the data >> okay so now let's move to week three part where we will be dealing with this CNN and everything so so far if you have not noticed then we week week two is all of this. So such questions you can expect uh in quiz also >> if they ask like what is the definition of compose and they give options what is like a definition of composing >> so it's kind of anyway so you have this kind of uh so this compose also there in in skal right so it's same definition here you want to perform these steps sequentially right so something like if option can be something like they perform perform these steps sequentially or they perform these steps parallelly.

[01:25:54]Right? So there you have to be very careful when you see which option is correct. Right? Because here this this this actions performed sequentially means first this code will be executed and on the output of this code then this will be this operation will be executed.

[01:26:12]So that is why this that is what this compos.

[01:26:17]So this options you can be careful with what type like how uh in what manner they are giving the options. For example with in one option they are saying it is parallel operation in one operation they are saying sequential operation. So just make sure you >> what will be the answer transform dot uh >> two right. So it's >> Oh yes.

[01:26:47]Okay. So now um right we three and four we'll be covering here.

[01:26:59]So here you have to find out this shape.

[01:27:41]14 is the shape. Uh I don't think 14 will be the shape.

[01:27:46]Shape means you have to give exact detailed value not just one output.

[01:27:50]Right? You must understand the output of shape also. Otherwise, let's say some option has values like one whole value.

[01:27:58]Then how will you understand 1 + 3 + 3 + 16?

[01:28:13]Okay, let's see. I don't know. Let's see.

[01:28:46]Yeah. So now you are able to don't know whether you have run the code or not because earlier nobody was able to answer this. Do not run the code again. I will say if if you are doing it.

[01:29:02]Okay.

[01:29:04]Yeah. So this going to be 8a 16a 28a 28 right. Because 8 is going to be always there because it's the batch size.

[01:29:14]So even if you run uh like at the end whatever has happened it has happened for all the eight data points right? So eight will always going to become going to come first. Now this out channel is 16. That means you have 16 kernel kernels that that are operating on your image right 16 different kernels. So in channel so here this is the channel right? So in channel was one out channel 16 then 16 will come here right and then the kernel size has three with padding one. If you calculate the output uh dimension or output size it will not change right? it will be 28 only.

[01:29:57]So you can calculate for height because for width also it will be same. So just calculate calculate for 1 and it will be 28. So 28 28.

[01:30:08]Make sure you remember the formula for that. Right? Can somebody tell what is the formula?

[01:30:15]So input minus uh uh input plus two padding minus kernel upon stride uh this is uh floor function plus one >> uh so floor of your uh for example height or width whatever you want to take right so floor of let's say I'm taking height so floor of height plus 2 into p minus kernel size divide by s and then plus one yeah Okay.

[01:30:46]Yes. So, make sure that you remember this formula.

[01:30:56]So, what will happen if you keep the if you make the padding zero here? Can somebody tell again the output uh shape keeping the padding zero >> 26?

[01:31:20]>> Yeah, it will become uh the height and width will become 26, right?

[01:31:31]Okay, one more simple thing but just that just to be sure that you are aware of it. How many learnable parameters are here in this in this?

[01:31:45]So I'm just learnable parameters I'm asking just writing just this 48.

[01:32:11]>> Uh, color writing 144. Somebody wrote 160.

[01:32:20]Is it 144 or 160? So one of this is true 160 right because the bias also you have to add. So you guys are forgetting the bias.

[01:32:36]So with each of these output channel you have one bias. 16 biases are also there that you have to ask uh you have to add.

[01:32:44]Yes. Because it is default the bias is true.

[01:32:52]Okay.

[01:33:02]Okay. I think that is fine. I'll give another question.

[01:33:08]>> So for uh you asked the question just for this particular layer, right?

[01:33:14]>> So then how is it 14 160?

[01:33:18]Um so for each kernel uh so uh one kernel will have nine parameters right because it is 3 + 3.

[01:33:27]>> Yes.

[01:33:28]>> And you have total 16 different kernels right that is so that will add to 144.

[01:33:37]Then each kernel has one bias also. So that is okay. Now here you have to predict the what what will be the print statement here.

[01:34:46]Correct. So it is um 8A 32A 7A 7 not 6A 6. So it is just with stride two and you have kernel size two then it is kind of your so this is standard thing right when you have kernel size and stride with same number it's kind of going to have your uh your input dimension >> for pool you have to take the input channel itself number of channels >> sorry I did not understand >> for pooling operation We will take the input channels as the number of channels >> not the output one. Normally for cono layer we will take the output channel as the channel in the shape.

[01:35:38]>> Ah correct correct correct.

[01:35:39]>> For pooling we have to take that input channel itself.

[01:35:43]>> Yes. Yes. So pooling will be performed in that uh the channel that you are getting because there is so output channel will be what you are going to produce right after that. So it's not predicted first. It's it's decided by the kernel size and stride.

[01:35:59]>> So the formula remains the same is it even for this like uh it's 304.

[01:36:03]>> Correct. Correct. Yeah. That that formula of that floor value right. So that will be >> so in this case do I have to like the I mean if I do that I get six and if I did not add the bias term which is plus one but do we have >> no how are you getting six? You are adding plus one at the end or not? Six will be the answer of that floor value right?

[01:36:22]>> Yeah. Yeah. So okay I have to add plus one here as well.

[01:36:25]>> Yeah. Yeah that's also in the formula right plus one >> and what is what does it like in the for the convolution layer it signified that it's a bias term right?

[01:36:34]>> So max pool has no nothing to learn right. So that's why there is no bias we are putting here.

[01:36:40]>> Okay but still that plus one will give the correct >> that is for the formula. Yeah. So even though you try to so formula is so that plus one is not signifying for the bias in the formula.

[01:36:53]Okay.

[01:36:54]>> Yeah, bias is different thing.

[01:36:57]Bias is not added inside the kernel. So, kernel size has 3 + 3. It has nine elements. It does not mean that one of that element is bias. So, bias is separate.

[01:37:07]>> Got it. Okay.

[01:37:08]>> Yeah.

[01:37:14]Okay. So, similar questions we have also seen. Let me just make this also for convolutional operations.

[01:37:28]So you have to give the shape at each step.

[01:37:37]Yes. So here you have to trace uh you have to trace the shapes through the full forward pass for input of this size of this shape.

[01:37:54]So this con one was con 2D 1A 16A 3 with padding 1. And this con 2 was con 2D with 16A 32A 3 with padding one.

[01:38:20]>> So what do we have to do the size of the input or the output?

[01:38:24]>> For each of this A B Yeah. So after for example what will the shape of X this here? So and then what is the shape of this X in the second line? What is the shape of X in the third line? That is what you need to answer.

[01:39:08]The stride is one. Is it?

[01:39:13]>> Stride is uh stride. Yeah. Take stride one.

[01:39:43]stride for the corn layer. Right? For the corn layer you can take stride one default.

[01:40:23]For the last to pull, what is the size for last pool, right? That is what you are asking.

[01:40:34]>> Yes, that is what I'm asking.

[01:40:37]>> So, are you not able to get or you're confused with the question?

[01:40:41]>> Oh, no. The first pool you have given max pool 2d 2a 2.

[01:40:46]>> Uh, okay. Take the same. Take the same.

[01:40:48]>> Okay.

[01:41:23]Yeah, I think has given the correct answer for all all these steps.

[01:41:30]So for a it will become um 64, 16. So 64 is going to be same everywhere, right? So let's uh talk about the other things. So it's going to be 16 because of the out channel and then 28A 28 because padding is 1 and kernel size is three. ReLU will not do anything. Then again the shape of B here will be 60 64 comma 16A 28A 28. Now you have pooling layer right? So and the pooling is of 2A 2 and it is going to have your uh size.

[01:42:05]So 64A 16A 14A 14 28 comma 28 will become 14A 14 and then again you have >> max pooling layer how can you apply the same formula and get the answer because uh max pooling is doing something else.

[01:42:26]>> No when you apply kernel also you kind of add everything and at the end you get one value right with one kernel operation.

[01:42:35]The same max pooling is also kind of going to give you one value only. It's just that the operation there was different and here it is just take the maximum.

[01:42:43]>> So it's going 28 - 2 which is 26 by 2 which is 13 + 1 which is 14.

[01:42:50]>> Correct?

[01:42:53]If it is if you you are confused with this operation then you just do one take one dummy write some uh let's say 10 + 10 or even 5 + 5 uh this image grid and take 3 + 3 kernel and then take some 3 + 3 pooling also and then see what is happening right with some stride and some padding. Okay about the syntax for this max poolool 2D uh first is input channel and second is output is it?

[01:43:25]No here what we have is so it's kind of uh when you say 2a 2 right so it is kind of telling you the size and then what so kernel size and stride so it's uh >> and >> yeah there is no output input we decide here max pooling is just like we have let's say 10 image and I want to put some some kernel size with some stride on top of it to reduce the dimension or kind of to aggregate the information.

[01:44:05]Okay, I think this is fine, right? Um, moving to next question.

[01:44:12]So for this same only, so you have this.

[01:44:14]Okay, G. Yeah, let's discuss about the G.

[01:44:18]So what will happen in in G?

[01:44:26]Sorry, but I'm saying G. Okay, I have not written G here the uh the last what will be what will be the size of this?

[01:44:44]>> So it be 64 and then rest of the dimensions multiplied like 32 into 7 into 7.

[01:44:50]>> Correct. Correct. Right. So it's going to be somebody wrote. Yeah. 64, 1568.

[01:45:00]Understood? Right. So it's like you have 64 coming out as the batch size and then everything else is getting multiplied.

[01:45:08]So like everything else is getting flattened out. So just same as flattening, you are flattening the image now here because now you want to pass them to some fully connected layers.

[01:45:21]So can we give three times like minus one again three dimension tensor in that >> uh then also you have to put minus one right because everything whatever you have you are you are just going to flatten >> yeah if you give 2 * minus1 in that and we make a three-dimension tensor So we'll again uh what will get multiplied for the third >> um I'm not understanding so what you actually so okay I'm I'm not able to understand the question can you specify a bit more about what you are saying so here I will uh show so this f you have here right? So what will be the size of or the shape of f here?

[01:46:26]It's 16 7 >> uh into 32 right comma 7A 7 right so you have this uh uh so you this 32 is your uh channel and this is your height and width right so what you are doing here is you are saying that just keep the x dot size0 will give you 64 right >> okay 64 16 3277 I think >> yeah okay 16 sorry 16. Yeah.

[01:46:57]>> Then 32 right after that.

[01:47:00]>> Um, one second.

[01:47:02]>> 64 size.

[01:47:04]>> No, it's 64. 3277 only. Correct. It's >> 32. Okay.

[01:47:09]>> Yeah. So, what will happen here? This uh 64 is going to be this. So, this x dot size of zero. If you extract, so let's say this is your x now, right?

[01:47:19]And now you extract x dot size of zero, you are getting 64. This will come here.

[01:47:23]Minus one means whatever all the elements were there or whatever all the dimensions were there you just multiply all those dimensions.

[01:47:35]So that is what is going to happen here.

[01:47:37]Not sorry not not the dimensions like the the values that are there you have to multiply them.

[01:47:48]So it's like 32 uh 7A 7 you it is a tensor 3D tensor and you have flattened all of that and then it has so this this 3D tensor has values of 32 into 7 into 7 that many elements it has. Now all those elements will be in just let's say one list.

[01:48:07]So that is what this view will do.

[01:48:13]Okay. So now the shape here will become uh 64 comma 15. What was the answer? 1568.

[01:48:22]Right? Yeah.

[01:48:24]So this 1568. So now you have each data point has 1568 features you can say.

[01:48:34]Hello sir. X dot size zero. What what will give? This will extract this.

[01:48:46]So you want your uh you want your batch size to be same, right? So 64 batch size should be here always.

[01:48:58]Okay. So I think this is the same code.

[01:49:06]Yeah. So one more uh on this same setup that we have now we want to go to fully connected layer and what should be the output here? Sorry, what should be the value here?

[01:49:20]This is the first uh fully connected layer that we are adding after all these operations.

[01:49:28]after all these operations.

[01:49:32]So what should be the value here?

[01:49:35]>> It's 1568, right? Like 64 is a bad size, >> correct? So that is going to be so here um no this is not this 64 is not bad size. Okay. So this value will be 1568 but this linear you're saying that this is the hidden uh layer which with 64.

[01:49:58]No, I was in the G you have 64a 1568 right 64 over there is the bat size okay >> correct >> okay now I'll have one more question so this one is not clear uh this tenant linear which you mentioned How did we get this figure? I mean not sure.

[01:50:30]>> So this is so I said as I said this is going to be added right after this.

[01:50:35]>> Okay. So what is happening here? You have um so this is this has given the shape this right.

[01:50:44]>> Yeah.

[01:50:44]>> Now this uh this will be the input for this uh first uh fully connected layer.

[01:50:51]So when this is going to be input I want this many neurons to be present. Right.

[01:50:56]Yeah. Okay.

[01:50:57]>> So then because this should match. So it's kind of this is output and then this input. So output of the previous layer must match with the input of the next layer. So that is why this >> so 64 is the output of this layer. A linear layer.

[01:51:10]>> Correct. Correct. Correct. Yeah.

[01:51:12]>> That will be >> it can be. Yeah. Here it can be anything. I can write 128 also. But this must be same.

[01:51:20]>> Thank you sir.

[01:51:22]Okay.

[01:51:28]Okay. So, yeah. Now, some of the kind of memorization type of questions. Yeah.

[01:51:34]>> Yeah. I didn't understand the relevance of this uh this self. FC1. Why why have we run this like >> why are we adding fully connected layer, right?

[01:51:44]>> I Yeah, I mean why this suddenly one state one separate cell for that? I didn't understand.

[01:51:50]>> No, no, that is not sorry. I'm just I just because it's I have treated this uh treated it as different question. So that's why I have added it. You can write it here.

[01:51:59]>> Okay. Okay. Sorry.

[01:52:07]>> Okay. So now uh it just kind of a memorization thing.

[01:52:13]What type of uh what do I ask? So what type of feature it is going to detect?

[01:52:21]>> Vertical edge >> right? So it is going to detect vertical edge right. So because this is very famous thing and just added this to okay so most of the things we have >> um sorry >> the last one what you said can you please repeat it?

[01:52:43]The last one I said I just I mean I for for this explanation or for for this >> the last one the last one >> um this is the vertical edge detector feature. So when you apply this kernel uh to a image it is going to give you vertical edge. So what wherever the vertical edge will be present that will get highlighted.

[01:53:11]What is vertical edge?

[01:53:14]>> Vertical edge is like for example you have uh so first of all you have to understand the edge in terms of the image right? So in in in terms of the pixel values. So the edge is where this so let's not talk about vertical or horizontal. What is edge? Right in terms of pixel values. So the edge is where you have at one point there's a sharp uh jump or there's a sharp difference between pixel values. So that's the edge.

[01:53:42]>> Now if that sharp difference of the pixel values happening vertically like you you uh so let's say you have two columns right or two rods vertically standing can think of that and then there is a one rod has all the pixel values very high rod has all pixel values very low. So that's kind of you are creating one uh and they are very they are like joined with each other those two rods and that is like vertical edge you have.

[01:54:10]>> Okay sir. Thank you.

[01:54:12]>> Yeah.

[01:54:18]Okay. So I think >> can you share the collab?

[01:54:23]>> Yeah yeah I will be sharing one second.

[01:54:24]few more questions uh then we will be I'll be sharing the code.

[01:54:31]>> So from week four uh what type of questions will be there from week four?

[01:54:37]>> Uh week four there are not lot of things to do. So that's why I have not taken many questions from week four.

[01:54:46]Um but let me take some one second.

[01:54:58]For example, one question is simple like this. Um, we have discussed that also in last session.

[01:55:07]Okay. So, this is the question it's going. I think I should.

[01:55:19]Yeah. So, this is the question you have.

[01:55:21]uh you have complete the classifier head uh replacement for a threecl class problem uh on a backbone whose feature vector is 1280 dimensional.

[01:55:37]So isn't this straightforward like yeah it is 1280 and three correct right so it's going to be 1280 3 so in features are going to be 1280 and out features are going to be three >> sir I don't understand can you please explain >> so here what what I'm asking here is uh so you have certain so you have uh scraped the or you have uh you are accessing the classifier part of some uh pre-trained model and in that you want to so for example that particular classifier has uh feature vector of 1280 dimension so backbone or in more better way I can give this to you say 1280 now I'm asking what should be output feature so that you can replace this classifier for your three classification problem or I can give values here also right let's say I'm giving value here 100,000 Now I ask you what should be changed uh to make this classifier work for your problem statement where you have three class classification problem. So you only want to change output features and you want to make it three. We have done the coding also for that right in in week four transfer learning live live coding session.

[01:57:02]>> Hello sir.

[01:57:04]>> Yes. Uh sir if you want sir in the place of a classifier if you want to solve a regression problem then the output layer what we have to change >> in regression then you want to just predict one value right so then there there can be multiple things that you want to change first of all here out features instead of uh 1,000 or anything we just want one right because it is just one value that you want to it's one continuous value that you want to output right >> yes >> and then it it should be one and then The loss also changes. Many things will change there, right? Because then you cannot use the loss function that we are using for classification, right?

[01:57:42]>> Yes. So activation function will also change. And so or sir what we have to change in the output layer sir only >> output layer you make it one >> one. Okay.

[01:57:59]>> Okay. So I will give one more uh question here.

[01:58:14]So very I think this is again kind of question which is so whenever you find a code very uh for example you are reading some code in in exam and finding it very like no problem it seems right then you must focus on the wording so like there must be some problem with the wording here. So one such question uh can be this.

[01:58:46]>> So you have to identify what is wrong in this >> audible.

[01:58:52]>> H yes you're audible but your voice is very low.

[01:59:00]Am I audible now?

[01:59:09]>> Uh yes. Yes.

[01:59:10]>> Augmentation part is wrong.

[01:59:13]>> Yeah. One second. Let me >> my actually I got disconnected several times while we were discussing the question. I think um the convolutional layer and the uh Reu one. Can you scroll a little bit?

[01:59:27]>> I have a doubt. Yeah. This one.

[01:59:29]>> This one. Yes. So here for finding out the output size what would be the kernel here is it mentioned I'm just confused.

[01:59:40]>> Um so here uh this is the kernel. So I'm not writing kernel but this is the flow that happens. So here if such questions comes in exam you won't be seeing this way of writing things. Kernel will be specified with you that the kernel has this size stride has this value padding has this value. Right? So when you when we write code if we can write the code like this so it is automatically that okay first argument is going to be in channel uh the second argument is going to be out channel then comes the kernel size so if you even if so it's kind of a short hand of uh writing code so that's why >> okay thank you >> yeah so here it is the what's the problem here >> so the augmentation is not done in test right it's done in train >> correct correct so right so these are the problem problematic thing here so the augmentation we don't do in the test data set so this is kind of wrong for the test data uh for the test point of view okay so few more question let me see if I can get So in this case like the two tensor right should it can it anywhere at in this sequence or does it always have to be after you do the augmentation and then you do convert it to a >> so anything that you want to do with the date with the pixel values before that you have to make two tensor like some algebraic operation where you want to do before that you have to make first you have to convert into tensor then do some algebraic operation >> so but technically is not like rotation and all also changes the pixels right >> um but we are not kind of transforming them right we just like um the same values are only going to be there so I think no operation no like algebraic operation we cannot say there is something that is happening here right it's like shifting the values the same values let's in of >> okay >> or like the location is shifting kind of >> so one doubt I had with respect to the previous question so model classifier uh in that torchen dropout dropout no the come down >> okay here >> yeah the one previous to this. Yeah. So in this uh the dropout doesn't come in ANN, right?

[02:02:31]>> Um no, you can't have there also. Yeah, it comes there also >> it does.

[02:02:40]>> Yes. Yes.

[02:02:42]>> Okay. Because I think I I heard somewhere in the lecture somewhere saying that saying that dropouts are there only in the CNN and not in the NNS.

[02:02:51]>> Okay. No, no, no. There also.

[02:02:53]>> No, no, no. It is only in the C. Only in the not in the CNN only you can apply dropouts for the neurons.

[02:03:10]>> Oh, >> correct. Because that is Yeah, because that will go with the neurons like you have hidden layers and then there are neurons and you can drop them in. In CNN you CNN means what? Just the kernel operation, right? The kernel is there and you are doing the convolution operation with uh with kernel and image right so there what are you dropping nothing >> okay there there is already there's lot of sparity is it that's >> yeah correct correct correct in the lecture professor told dropout won't come in the validation and constip it will come only in the train training part correct correct correct correct okay ma'am thank you ma'am thank you sir Okay. So I think I'm not finding any other good question. So this is uh all.

[02:03:57]So try to practice such more questions.

[02:04:00]You can find them from the PQ also. So I think we have covered uh most of them.

[02:04:08]All right. So all the best for the exam. So I'm closing the screen now.

[02:04:16]>> So could you share the link to the >> Yeah. Yeah. One second.

[02:04:23]>> So the notebook for the previous session uh is available on the portal.

[02:04:28]>> It is available on portal. You have some G sheet link right? Uh where the resource gets uh updated.

[02:04:37]>> So can someone let me uh >> okay I'll be sharing here also. One second.

[02:04:55]Thank you.

[02:05:05]This is the collab room.

[02:05:14]So I think we have solved some uh 40 plus questions.

[02:05:22]Not very uh involved question but this kind of questions you can practice more.

[02:05:29]>> Sir I have a question for week four right? Uh do we have to practice like some of the code for segmentation and all of that?

[02:05:39]>> No, no, no. Segmentation part you can omit.

[02:05:43]>> Okay.

[02:05:45]So the Google sheet which is shared by Priti and Kush uh in the chat that is for uh TA sessions.

[02:05:52]>> No, it has instructor sessions also.

[02:05:55]>> It has instructor sessions also.

[02:05:58]>> Okay, I'll have a look. Thank you. Yeah, but make sure that you are aware of the kind of the structure of this uh pre-trained models discussed in week four, right? Or or not the structure means you don't have to like understand every every layer of it. But then there are certain very like exact difference from one architecture to another architecture, right? So make sure that you are aware of that.

[02:06:27]>> Okay.

[02:06:29]So the segmentation next week so segmentation yeah I'll be taking it uh differently but there is a reason that I have made segmentation uh I have not discussed segmentation there I'll be announcing for that after the quiz >> okay so I'm stopping this stream thank you sir