SMAA Assignment 2 Support Session Part1

Added: 2026-05-11

[00:00:19]This is the support session for assignment two for small. Um I'm going to walk you through the assignment and discuss what is expected um and how to approach it. Okay. So let's begin. I'm going to share my screen.

[00:00:57]All right. Okay. So, I won't go over question one. It's purely theoretical.

[00:01:05]Um and I don't think it's unfamiliar to you. Okay. So you should be fine for that. Question two um your uh assessed on logistic regression. So how to go about um let me just share an alternative data set so I can just walk you through what the question is all about. Okay. So you need to um create a model specifically a logistic regression model to predict your y okay given a certain um variable x. In the example that I'm showing you on the right, what I have here is your X variable is your income.

[00:01:55]Say it's in 10,000. So the first person um the salary is 1.8 * 10,000. That's their salary. The next person 1.89 * 10,000. But you don't have to worry about the 10,000s. It's just the the values in A that you actually need to be focused on. So you're going to use the values for your x or in your a column to predict uh your y variable. Now in my example, my y variable is whether they will purchase a product. So zero means they won't purchase it and one means they will purchase a product. Okay? So I'm going to use income to predict that.

[00:02:35]Okay? To predict whether it's a zero or a one. So what you need to do is you you're essentially creating a linear model, right? So your first step is to create a equation beta plus beta 1 * x. Now your beta not and beta 1 is something that you can just set to whatever value you want right initially. So you need to initialize it. So as you see in here, what I've done is I just created some values. All right. Um what values it is you know it's not too important. You can use your discretion uh because you're going to change this.

[00:03:21]Okay? And you should come to the same values in the end no matter what values you initialize it to. Okay? So there's my beta not my beta 1. All right? As I'm showing in my equation on the left. So this equation let's say it's equal to let's say it's equal to L. All right we'll call this L. This is actually your logit function. All right. So what I'm doing here this beta plus beta 1 * X that is my column C. So this is my L.

[00:03:53]All right. And all I'm doing is is I'm plugging in all my x values that is your a values into this equation. Right? Now you know what your beta 1 is and your beta not because you said it here. So all you got to do is compute that. All right? So if you look at my logic calculation you see it takes into account beta not plus beta 1 times your x value. All right your column A. And that value is what you find in column C.

[00:04:23]And you would do the same thing for every row. Okay? You keep your beta kn and beta 1 fixed. And the only thing that should be varying is your x value.

[00:04:33]Right? So you'll have your logit values in column C. All right? So this is your column C.

[00:04:41]All right? Now you you can follow my guideline or you can do it your own way.

[00:04:45]It's really up to you. But um this is just just one way to do it. Okay. All right. So that's your step one. The next for column D, what you want to do is you're going to do another calculation.

[00:04:57]It's going to be 1 / 1 + e to the minus l. Now the l is what you got from column C. So you're just going to feed that value into this equation here. All right. And this this next equation this is these are your probabilities. So this is your probability and I'm going to make it my column D.

[00:05:20]All right. And that's what I'm pretty much doing here. All right, the E, remember in your calculator you'd see E.

[00:05:27]In Excel, it's the function exp. Okay, so you would type E equals to exp. And that's the same thing as E. So if you say um say I wanted E to the minus 5 like that in Excel, it'll be equals to exp brackets minus 5.

[00:05:48]All right, that's how you did in Excel.

[00:05:49]Right? Make a note of that. Okay? So that's what I'm doing here in this column. All right? So you'll do the same thing for all the rows. And now you got all your probabilities. What you're saying here in these probabilities is you're saying for someone who has a income of 1.8. All right. The probability of purchasing the product is.17.

[00:06:12]All right. Okay. Lastly, in column E, okay, you're going to calculate the log likelihood. Okay, and this would have been covered with you in your session uh your master class session, but I'll just quickly show you essentially what your log likelihood is. Remember your log likelihood. The equation for that is simply just your natural log oops natural log ln of your values that you have in column D. All right. So let me just say this the values in column D.

[00:06:50]Let's just call it P because these are your probabilities. So you're just taking the log of whatever values you have in column D. Log of P. But just just to be careful right here, you're only going to take log of P, right? When when your Y value is one.

[00:07:11]So if you so if you see here, I'm going to use logic to say only when my Y values that is for values in B only when it's in one, then take L of P otherwise I want to take L of 1 minus P.

[00:07:30]otherwise or you can say y equals to zero it's the same thing as saying otherwise because y can only take on zero and one right it's the same thing so if you're using an if statement then your first condition will be this and then this is your else right so you have to use an if statement to code this logic okay so if your target value is one then take the log of column C right otherwise take the log log of 1 minus C and that's your um log likelihood. All right. And now what you need to do is you need to take the total log likelihood and you're going to use this to actually optimize your beta values.

[00:08:14]So essentially what your beta your best beta values are, it's the values for beta KN and beta 1 that maximize the sum of your column E. that is the sum of your log likelihood and you need to use solver to do that. All right, so what you're trying to say is if you had to use solver, right? You're saying uh if it pops up ah there it goes on the other screen. Okay, what you're saying is I want my total lo likelihood to be maximized. So I'll click my maximize by changing my beta one and beta one. All right, you can untclick this make unconstrained variables. All right, you don't need that. And then you'll just solve it. All right, and then that'll give you your best beta not beta 1 values. Okay, now once you've got that, now you can make predictions. Along comes a new point. Someone with a salary of 3.2.

[00:09:17]Okay, you want to work out what they ultimately what their probability is of purchasing your product. That is what's the probability of Y equals 1. So your step basically is just to follow what you've done here above. You've already done the work. So all you got to do is kind of just repeat the calculation for the new point. You need to get the logit and then you need the equivalent probability value which is using the formula that you used in column D here.

[00:09:45]That will give you the probability. So that's saying the probability of this person with a salary of 3.2 buying the product is 24%. All right, that's how you work out the probabilities. And then what does beta 1 mean? So remember if you look at our equation here on the left, beta 1 is the slope. All right. But the slope tells you how a change in your x variable is going to affect the change in your y variable. So in other words, in our case, if you look at our optimized beta value, beta one, the slope, it's here. I'll just make it red so it draws your attention to it.

[00:10:31]Right? It's 3. So what we're saying is um as income increases so my x variable which is income right as it increases by one unit then the log odds okay log odds of purchasing increases by.3 this value here. All right.

[00:10:53]Okay. Um, so that's question two. Pretty straightforward. All right. Um, let's move on to question three. All right.

[00:11:06]Question three. Um, so it's not too much. I'm not going to do the calculations here. It's left to you to actually do this. But essentially, you have precision and you have recall. All right. Now these metrics are used for when you want to focus on false positive or false negatives. All right. Um and you need to figure out which metric is it precision or recall that focuses on false positives and uh which one focuses on false negatives. All right.

[00:11:43]You'd have to do the calculations for recall and precision. you need to uh determine the formula for that. Okay, I'm not going to give that to you, but it's easily obtainable. So, you want to get the formula for recall, sorry, right, for recall. And then you need the formula for precision.

[00:12:07]And these formulas are going to use the your true negatives, your true positives. It's going to use your false positives and your false negatives.

[00:12:17]These values which are given to you in this equation and your question paper.

[00:12:21]So I'll just go back to it. So have some context and see here they give you the uh true positives, false negatives, etc. And they give you two models. So you're going to have to work precision and recall out for both of these models. All right. And um uh once you have those values, you can compare the each model to each other.

[00:12:46]All right? So maybe neural network has a higher recall than logistic regression and what does that mean? Okay, you need to interpret that. So coming back to here what you also want to know is okay so once you've identified whether recall focuses on minimizing false positives or whether it minimizes false negatives once you determine that then you need to figure out when would false positives be important to minimize. So you need to apply your mindset and intuition to this. So for instance, look at false positives, right? When would you want to minimize that? When does that become more important than false negatives?

[00:13:28]Think about a patient that maybe goes for um uh scanning uh for breast cancer. Now, as you can imagine, if the result is positive, you know, that can lead to quite intensive uh treatment and surgery. So you want to be absolutely sure that when it is a positive, it actually is a positive. So you don't really want it to be a false positive, right? So that example you need to apply your mind to other examples as well, right? So um provide examples when you want to minimize false negatives and false positives and which metric is it precision or recall that you would use to determine that. Right? Provide examples, right?

[00:14:11]Um okay, so that's question three.

[00:14:15]Let's move on to question four.

[00:14:20]Okay.

[00:14:22]So here given a linear SVM with a normal vector w= to 2 and minus one. Compute the margin width. Right now I understand that you haven't covered this uh as yet.

[00:14:34]So what I'm going to do is I'm going to uh explain this at a high level. Um so you should still be able to answer it even though you haven't covered it as yet but this should help you. All right.

[00:14:47]So I know this diagram on the right is quite busy. Um but I will explain it to you. So ignore all the writing. So I just want to check that I'm recording again. Yeah. All right. So ignore all the writing. Okay. What I want you want you to focus on is look at the red squares and the blue circles. Now those shapes represent two different classes.

[00:15:13]Now maybe the the blue shape represents I'm sorry the red squares represent um clients that bought the product and then the blue circles of the clients that did not buy the product. So what SVM tries to do is it tries to create these margins and what I mean by these margins I'm talking about these dash black lines.

[00:15:36]These parallel dash black lines separate the two classes. So the blue dots and the red squares and you want the the distance the vertical distance not vertical I'll say perpendicular distance between these dash lines to be as large as possible. In other words this green arrow SVM tries to find these black lines black dash lines as well as maximizing the distance between them shown by this green line. And when when it can do that then it means it can effectively predict these classes because it can separate them. That's essentially what SVM does. Right? So when you asked to work out the margin width, what you're actually asked to work out is the size of this green arrow, the length of this green arrow.

[00:16:27]Okay? Now to do that, okay, there's a specific formula you would use to do that and I'll talk you through it. and just share my whiteboard here.

[00:16:44]Sorry.

[00:16:50]It's frozen.

[00:16:54]I'm just going to pause it until I fix this.

[00:17:17]I can't. Sorry.