Tutorial 7

[00:00:02]Okay, so let's get started. The agenda for today's session is just statistical inference. I have two notebooks over here and then I have a set of quick practice questions on some important concepts in statistical inference.

[00:00:21]Starting with the law of averages. Um, for the law of averages, we have uh a famous problem, the Monty Hall problem.

[00:00:30]This was already covered in class. I'm just quickly going to run you through this visualization to show you how this works. Even before we look at the Monty Hall problem, let's recall what the law of averages is. So, the law of averages refer um refers to and it only applies to chance experiments. So it says if a chance experiment is repeated many times independently and under the same conditions then times repeat the proportion of times that the event occurs will get closer to the theoretical probability of the event.

[00:01:06]For instance if you toss a coin the probability of getting heads is equal to 50%.

[00:01:13]But if you toss a coin 10 times it's possible out of 10 times head but that doesn't mean head possibility 90%.

[00:01:24]However the law of averages under the law of averages if you toss it like 10,000 times to per the number of times that you get head for will get closer and closer to 50%. It will get it will be closer to 5,000. It's highly unlikely 10,000 miss 9,000 times head with that. Let's recall the Monty Hall problem really quick. So this is based on um an American television game. You are given three doors and between one of the three doors there is a car. Between the other two doors there are goats.

[00:02:02]You're supposed to pick one. The host knows kids door.

[00:02:07]So, um either you're going to pick a door with the car behind it or you're going to pick a door with the goat behind it. Either way, without opening the door that you have picked, the host will open one of the two remaining doors. The thing is though that the host will definitely open a door.

[00:02:27]After that, two doors remain closed.

[00:02:29]There is this one door select and then there is this other door host.

[00:02:36]So the host will give you a choice.

[00:02:39]Either you stick with the door that you picked initially or you switch the door.

[00:02:44]And the question is is switching your doors going to increase the probability of you winning. The answer is that it does improve the probability of you winning because under standard probability assumptions probability was 1 / 3 probability one of the two doors was 2 over 3. So that means that the door that the host didn't open has a has a total probability of 2 over three of having the car behind it. Which means that switching your door is going to double your chances of winning. I hope that this makes sense. Are there any questions?

[00:03:39]>> The thing I do have a query here.

[00:03:42]>> Yeah. Yeah. Go on. Uh once the door is open, why is our sample space still three? Why is it not two?

[00:03:49]>> Oh no, no sample space effect.

[00:03:53]The the thing is initially let me just show you over here as well.

[00:04:04]So initially three doors her door probability was equal. It was 1 over3 right?

[00:04:15]Just copy this value.

[00:04:22]Suppose this is the door that you had selected. Okay. Which means remaining two doors probability was equal to 2 over3 2 over 3 number fixed because this is a percentage.

[00:04:40]There is a 33% chance that the car is behind the door that you picked. And there is a 67% chance that the car is behind one of these two doors.

[00:05:00]probability.

[00:05:03]You know this information which means remaining probability associated.

[00:05:14]Does that make sense?

[00:05:18]>> Yes.

[00:05:21]>> Okay. All right.

[00:05:23]So the thing is though theoretically if you switch the door to in about six to seven out of those 10 games you should win but this may or may not be true. You may win in all of them or you may not win in any one of them because 10 is a very small number of games. But if you play the game a thousand times, then you're very likely to win in about 670 of those games.

[00:05:57]Uh which is why we have this simulation over here. The code is not for you to be concerned about at all. Uh game simulate and you have an interactive plot that is given to you number of trials set. So suppose that this is what I did. I set the number of trials to 10. So x-axis number of trials and yaxis I have the probability of winning.

[00:06:28]Um you see as I am increasing the number of trials your dotted line say marked this is the theoretical probability. So um the blue line indicates a switching strategy for switching the theoretical probability of winning is around 67. for staying with the same choice of door the theoretical probability of winning is about 33%.

[00:06:53]But um probabilities calculate given the number of trials that I'm running they are still deviating from my theoretical probabilities when I increase the number of trials and suppose my number of trials about 3,100 now you will see um I have actually gotten really really close to the theoretical probabilities now or increase to it will feel like I am exactly at the right theoretical probability.

[00:07:28]Are there any questions so far?

[00:07:34]Okay. Well, we're going to move on towards hypothesis testing then. I'm going to reiterate the number of steps over here as well using an example that you've already covered in class. uh and well we're going to look at the same example in code. Then hopefully we will be able to reiterate most concepts in um hypothesis testing before we move on to a few practice questions.

[00:08:02]So let's quickly recall model. So we're going to work with Mendel's model of flowers. Mendel was an Austrian monk who is widely recognized as the father of modern field of genetics. He was performing large scale experiments to come up with the fundamental law of genetics. Mendel experiment and Mendela model.

[00:08:25]So Mendel's model was there there were different varieties of pea flowers.

[00:08:31]Either pee flowers were going to uh be purple in color or they were going to be white in color. So mental model assume regardless of anything else regardless of any other conditions when you plant a flower there is a 75% chance that it's going to turn out to have purple flowers and there is a 25% chance that it's going to turn out to have white flowers uh Mendelka model.

[00:09:05]Now you want to assess this model and how you assess a model is steps.

[00:09:12]The very first step is that you come up with a statistic that helps you decide whether your data supports the model or an alternative view of the world. Um so it means test statistic select which will either favor my model or it will be against my model. this statistic is going to help me take my decision which means that this statistic has to be relevant for me. Um usually a hypothesis draw there is the null hypothesis which in experiments would assume the would assume no effect. Null by default assumes no effect. In this case it assumes that the model's assumption are true. The alternative assumes an alternative view of the world. So in an experiment etc it would assume that there is an effect for instance a drug effect measure on a patient. So my null hypothesis would be that this drug does not impact recovery and my alternative hypothesis could be that this drug does impact recovery or that it increases the chances of recovery however you state it. In this case, my null hypothesis is that my data is coming from the model and alternatively the data is not coming from the model.

[00:10:39]So this is what we're trying to understand. The statistic that you come up with helps you decide data.

[00:10:51]So anyway, this is how you come up with a statistic. Just give me a second.

[00:11:08]Yeah, here we go. So when you're selecting a statistic, you want to ensure values statistic and there are several valid statistics for the same scenario.

[00:11:23]We have accepted several in the past in exams as well. So first you see values statistic that will make you lean towards the null hypothesis and what values are going to make you lean towards the alternative. Your answer should actually be just high for the alternative alternative.

[00:11:52]Once we choose a statistic, well before we actually look at the next step, let me show you the statistic that we are choosing over here without reading the rest of the scenario given. So your statistic that is the absolute value of the difference between the proportion of purple flowering plants and 75. Why?

[00:12:15]because 75 was the um probability as per Mendel's model of um well purple flowering plants it means thousand plants grow according to Mendel's model according to the null 75% of them should have been purple actual proportion purple flowering plants absolute model against the alternative. Hence, we have selected this statistic. Are there any questions so far?

[00:13:06]Okay. All right. So once you select a test statistic, you simulate the statistic under the assumptions of the model. This means that you're going to generate lots of data for the statistic.

[00:13:18]Um and we have seen data generating algorithms in the past as well. It's data generating algorithm simple. Uh hard iteration is test statistic compute and then you can plot a histogram. So I can quickly show you the code as well so that you know what is happening at the same time. The code is nothing that you are supposed to remember.

[00:13:46]So I have an array over here just three times purple store and then once I have stored white.

[00:13:54]This is a function that just computes my test statistic and returns it.

[00:14:02]sample. I just count the number of zeros and then I divide that by the um sorry I count the number of nonzeros where the sample is purple. So I'm just counting the number of times that purple appears in my sample and I divide that with the length of the sample is purple proportion. Then I return the absolute difference of that and 75.

[00:14:26]Uh we can ignore this for now. I will circle back to this.

[00:14:31]So anyway, the test statistic define we're going to move on from this. Now I am defining a certain number of repetitions. If I've specified 10,000 I want to generate 10,000 samples sample, I'm going to have a certain number of total plants. So what happened was in actual life grow 929 plants he grew 929 plants and the statistic that he observed came out to be this much. So it's actually very close to zero. We're going to circle back to the observed statistic for now since 929 plants grow. So the repetitions 10,000 and her single repetition 929 plants key random values generate from purple and white and then for every single sample we compute the statistic or statistic store array sample stats. So this is what we're doing. uh each sample contains 929 plants. Her sample test statistic her sample generated here randomly based on this proportion because um I'm trying to generate data under the null distribution right now and I do this 10,000 times to get a distributions.

[00:16:08]See, so essentially array length it's going to be 10,000 and I have the test statistics against my 10,000 samples in this array. I will convert this into a data frame and then for my data frame I will well sort and now I'm creating a histogram.

[00:16:34]So this is um how my histogram comes out to be. Essentially this is the distribution of my model is red line.

[00:16:44]But are there any questions so far?

[00:16:49]No ma'am.

[00:16:51]>> Okay. All right. Perfect. So um >> um I just wanted to ask um four values and then you have mapped those values or those uh or that proportion to 10,000 samples. No, no.

[00:17:07]I'll tell you what I did.

[00:17:16]But I use the generator on the basis of this array. Like I'll tell you what key.

[00:17:26]If these four were written on chits of paper and I put them in a hat and I ask you to randomly take one out and read what it says, what are the chances that it says purple?

[00:17:37]>> Oh, okay. Okay.

[00:17:38]>> Three out of four, right? Yeah. Yeah.

[00:17:40]Right. Okay.

[00:17:40]>> Yeah. So, um 929 I will sample with replacement.

[00:17:49]Okay.

[00:17:50]>> Okay. Okay. Got it. So 929 values you know in 929 values against value test statistic repeat to get 10,000 values of the statistic. Got it.

[00:18:15]Uh yeah yeah got it.

[00:18:18]>> Perfect. Okay. Are there any other questions?

[00:18:22]Okay. So one important thing to be mindful of over here is none of this data was actually real. Uh model key assumptions under I just generated you can say synthetic data. None of this data is real. Right now in real life what happened was the mental grew 929 plants 929 test statistic compute curly this is called the observed test statistic and this is what Mendel's observed test statistic came out to be 088805 line plot. So this red dotted line represents what? It represents the statistic that Mendel observed.

[00:19:21]So first you come up with the statistic that helps you decide whether the data supports the model or an alternative view of the world. Statistic select higher values of it support the null support the alternative. Then you simulate the statistic under the assumptions of the model which we did.

[00:19:40]Then you draw a histogram of these simulated values. This is the model's prediction for how the output should come out to be histogram.

[00:19:50]Then you compute the observed statistic from like the actual real sample that was used in the study. The observed statistic.

[00:19:59]Then what you do is that you compare this value this dotted line value with your distribution with your histogram in order to decide uh whether your data is actually coming from the model or not.

[00:20:14]Either you will support the model or you will not support the model. Um in terms of statistics this translates to either you reject the null hypothesis or you fail to reject the null hypothesis.

[00:20:27]All right.

[00:20:29]So, um, however, the thing is red line drawn here from this line. How on earth do I know just by looking at this?

[00:20:41]So, um, in our lecture, the intuition that we look at and the intuition that we build on is you define a certain threshold accept. Yeah, I will fail to reject the null. So this is a certain threshold that you define as part of the design of your study.

[00:21:14]um distribution it should sum up to 100%. It's a distribution. So um the thing is I can say histogram like last 5% value reject 5% against line and I draw a line at 0.03 03 for instance. So it would mean 5% area. Let me see if I No, I didn't draw it over here but I will quickly draw it over here for your information.

[00:22:08]Okay, this is terrible. What did I Oh, that's what I did wrong.

[00:22:16]So if this was my histogram or set% and this is 10% highlighted does that make sense? So the thing is define as part of my study. So this is called alpha. Alpha is also basically called the significance level.

[00:23:09]All right. Significance level alpha.

[00:23:12]So significance level define it's very easy to interpret as well actually I mean it's exactly the way that it um fail to reject this is what you are defining. So if essentially it means%ific this value right so value this is I'm just going to label this over here as well. These are some terminologies that are often confusing for students up until the very end. So I want to spend some time on this. This is the critical value. The critical value is just the value of the um can someone tell me x-axis in fact.

[00:24:26]So guys this is very simple x-axis to test it at sticky values.

[00:24:32]test statistic values on the xaxis. So the critical value is the value of the test statistic that corresponds to your preset significance level. Does that make sense?

[00:24:50]Awesome.

[00:24:52]So um significance level define common levels that we define are 10% 5% 1% and significance levels you know if it is if it's on this side of the critical value. So it means that it is far from the center of your null which means that you have a lot of evidence against your null. Right? So um if your observed test statistic is greater than the critical value then you reject the null hypothesis. Similarly if it's um less than the critical value then you fail to reject the null. Are there any questions up until now?

[00:25:54]>> I just want to ask if uh we have the probabilities on the y-axis and test statistic on the x-axis. Also, you said that if the observed test uh statistic is uh after the critical value, we either reject the null hypothesis or we fail to reject the null hypothesis. So do we like set this already that uh uh I mean um for example we have u uh this is the histogram for the null hypothesis.

[00:26:24]So mean our null our observed statistic should be towards the right or like clarify first like your very first question if I got it right you wanted to understand the thing000 trials.

[00:26:56]All right.

[00:27:01]On the yaxis and let me show you on the sorry on the x-axis and on the yaxis you have the number of trials.

[00:27:15]Make sense?

[00:27:18]>> Oh, >> this is how this is how histograms work for distributions.

[00:27:23]>> Oh, so >> second question >> um I wanted to ask aa. So basically we will for in order to draw a histogram we need to take samples from 10 20 and we need to exponentially increase our sample. You need to take you need to have lots of samples in order to make a histogram. 1020,000 values data, right?

[00:28:20]model assumptions 10,000 samples in real life 929 plants actually purple flower actual test statisticulate.

[00:28:44]So that is the red line. That's the observed statistic.

[00:28:49]Okay. Okay. Got it. So our observed statistic should be uh after the critical value.

[00:28:57]No, not not that it should be um observed statist.

[00:29:21]So critical value this this is basically like a threshold threshold value to reject the null or threshold value then you fail to reject the null. Make sense?

[00:29:41]>> Yeah. Yeah, it does. Okay, great.

[00:29:44]>> Perfect. So, and pink line observed value the pink line so you observe value critical value say on the right if it's greater than the critical value then I will reject the null it's in the rejection region opposite side I would have failed to reject the null make sense or critical value define critical value preset significance level preset the significance level tells you what the critical value is going to be because the significance value is just the area that you are defining area tail rejection region the significance level tells you region and onwards reject make sense. So this yellow region suppose 10%. So it means yellow highlighted area 10% area of the histogram under the histogram curve. So the green value, the critical value is just the value that is corresponding to that 10%.

[00:31:18]Um all right, on that note, let's circle back to this graph alpha.

[00:31:27]Um and that takes me to one uh last very important terminology before we move on to questions as well and that is p value. Um p value can be a little bit tricky but as long as concepts are clear to you um if you understand what the significance value what the critical value and what the observed statistic mean so you're in a good position to understand what the p value is. P value means the observed significance level.

[00:32:00]This is the formal definition. It is the chance under the null. Under the null means assuming that the null is true. So the p value is the chance under the null that the test statistic is equal to the value that was observed in the data or is even further in the direction of the alternative. This is the formal definition of the p value and if I show you through the graph it's even simpler.

[00:32:26]So you statistic observe key statistic um area right under the null graph. That is the p value. This area so I'm just going to highlight this over here as well.

[00:32:52]This is the p value. That is why it's also called the observed test statistic.

[00:33:00]P value also called Observe test statistic is sorry not observe test statistic what did I do here observed significance level so observed significance level actual significance level define 10% 15% whatsoever P value is the significance level that you actually observed that tells you how significant your results are.

[00:33:40]Um and area in this case significance level 10 15% to evidently it shows value.

[00:33:50]So it means k um if the p value is less than alpha then you reject the null which is kind of the same thing as saying that if the critical value sorry if the observed statistic is greater than the critical value. Does that make sense? Are there any questions? I'm actually going to pause on this graph for like a minute and I will let you guys go over this.

[00:34:16]Uh I am just confused between the observed test statistic and the observed significance level because uh it's like it it has a very overlapping effect.

[00:34:26]Yeah.

[00:34:36]Xaxis under this value make sense. So value highlighted area.

[00:34:56]>> So is it the same thing then?

[00:35:00]>> No. No, not at all.

[00:35:04]Um, oh okay. So test statistic is the line.

[00:35:09]It is the particular like a an value on the x-axis.

[00:35:13]>> Exactly.

[00:35:14]>> And then uh p value is actually the area after that line.

[00:35:19]>> Absolutely. You got it completely right.

[00:35:22]>> Okay. Okay. Okay.

[00:35:24]So we'll compare the area to the significance level.

[00:35:27]>> Huh. Exactly. Up. Um is observed significance level. significance level compare P value essentially comparison. Okay. Okay. Got it. Got it.

[00:35:49]>> I will still give it like a few more seconds. Still go over this. I want this is really important.

[00:36:14]Okay. Are there any questions before we move forward?

[00:36:26]Awesome. We're going to move on then.

[00:36:28]So, red line area under this histogram what I will do samples the value was greater than or equal to the observed statistic and then I divide that with the total number of samples which was 10,000. So um in fact see sample starts greater than or equal to observed statistic divided by repetitions. So this is what my p value comes out to be 54% 54.0 06% is a very high p value in fact.

[00:37:30]Um so um this is what I've written down on a concluding note. Okay. If the significance level is set at 10% for instance, then you will fail to reject the null hypothesis based on the observed test statistic because 10%ific level value.

[00:38:00]Are there any questions before we move further?

[00:38:06]>> Oh, no. It's good.

[00:38:08]>> Awesome. We're gonna look at um two important questions.

[00:38:14]Hold on a second. Okay. It's area past exams, I suppose. And I want to iterate these through a question now. And for those of you watching, you should definitely pause and try this out on your own as well. For those of you here, I will pause and give you guys some time to go through this too. So, a study evaluates the effect of a new sleep enhancing therapy to improve sleep quality in adults. The null hypothesis states that well actually alternative you should be able to do that as well. The null means no effect.

[00:38:56]So the null hypothesis mean states that the new sleep therapy has no effect on sleep quality. Um the alternative states that the new sleep therapy improves improves sleep quality. It's that the mean sleep quality index of adults receiving the therapy is the same as the general population.

[00:39:19]Alternatively, alternative hypothesis here, the mean sleep quality index of adults receiving the therapy is higher than the general population mean. So, uh this really means null or alternative several ways very often with that. Let's look at this. We are given two distributions instead of just one for this question. Um you can assume data generate whatsoever either way doesn't matter to us. Blue curve that is data that you have generated under the null hypothesis red line curve that indicates the data that you have generated under the alternative hypothesis is critical value be plotted here the critical value is 58.22 22. Similarly, you are given this table of values as well. So, I will just pause over here for about a minute and I want you to just uh look at this table, identify what these values are representing you probabilities table and look at the curves and understand them well. Then we will look at the questions.

[00:41:11]Okay. Are there any questions?

[00:41:18]Okay, since there are no questions to questions first, what is the significance level alpha? Okay. Um I do have I think one or two people in the meeting still. So if one of you could tell me too what is alpha, what is the significance level?

[00:41:40]The answer is given to you in the table.

[00:42:02]No one. All right. So what you're doing is let's recall once as well in fact significance level area under the null curve that is towards the right of the critical value right or critical value 58.

[00:42:26]So you green highlighted here this is alpha in fact because this is the area under the null blue curve um where the value is greater than the critical value. So is table actually to value 05 probability that x is greater than 58 which was the critical value under the null hypothesis 05. So this is um going to come out to be 5% or 0.05. Are there any questions?

[00:43:05]Okay. Next you are required to calculate the p value if the observed test statistic is 60. So we're told observe statistic 60 or p valueulate.

[00:43:18]Again I will pause over here for a few seconds.

[00:43:45]Will it be 0.03?

[00:43:48]>> Perfect. Excellent. That's what I was looking for. It will be 0.03. 03 because um let's quickly recall looking at this P value statistic right under the null.

[00:44:02]So observe statistic plot but if you had to plot it this is where it would be or it's right area under the null which would be this area basically. So you given as 0.03 03. Are there any questions?

[00:44:23]No questions. So 3% or 0.03.

[00:44:28]So would you reject the null or not?

[00:44:40]I will pause over here for a few seconds.

[00:44:43]Would you reject the null or not?

[00:44:45]Observe statistic 60 here critical value 58 here significance level is 5% and the p value is 3%.

[00:45:09]um >> yes >> our p value is less than the significance level. So I think we well >> yeah yeah you should be more confident in your answer.

[00:45:27]You will reject the null possible explanations since um the p value explanation is less than alpha which is the significance level. Okay. Or you could also say that the observed test statistic is greater than the critical value. So you no explanations valid in fact. Okay. Are there any questions?

[00:46:05]Great. Is opposite to fail to reject the null. So for the very last part of this question, you have to estimate the power of this study. But power is something that I have not iterated as yet. So I will um quickly walk you over power as well. Before I walk you over power, in fact, I just want to um help you recall two different types of errors.

[00:46:32]Yeah, table. I'm quickly going to reiterate this.

[00:46:39]What happens is in actual life either the null is true or the alternative is true. Right? And one of the two things that you will do is either you will reject the null or you will not reject the null. This doesn't mean you're going to be right every single time just like ML model binary classification task.

[00:47:03]So either the model will correctly predict the positive class or it will correctly predict the negative class or the other two possibilities are that it falsely predicts the positive class or it falsely predicts the negative class which are false positives and false negatives respectively.

[00:47:23]Sometimes you can reject the null even though the null is true. All right. uh in which case you are doing a type one error. This is a type one error. It's also a false sorry it's also a false positive. Alternatively that the alternative is true but your test doesn't reject the null in which case you do a type two error. So this is exactly what's stated over here as well.

[00:47:54]Type one error means rejecting a true null hypothesis. Type two error means failing to reject the null when it is actually false. So I'm going to pause over here for one minute on account of the aan and you can just go over this particular slide in the meantime. Okay.

[00:49:54]Okay. All right. So to quickly reiterate, a type one error corresponds to rejecting a true null hypothesis and a type two error corresponds to failing to reject the null when the alternative is true. Type one on your graph. You should be able to um identify values type one and type two error probabilities identify.

[00:50:26]So a type one error means reject.

[00:50:31]Why don't I just copy this here? In fact, why don't I just write this down here?

[00:50:47]Rejecting the null falsely when it is actually true. What is the probability of this happening? So null hypothesis under who? Uh where is the area in which I reject the null?

[00:51:14]reject area over here.

[00:51:23]What percent was this area?

[00:51:28]0.05.

[00:51:29]>> Yeah, correct. Thank you. This was 5%.

[00:51:32]0.05.

[00:51:33]So you know what in real life is like I'm going to reject the null right that is the area under the null where I am rejecting the null. So the probability that my value under the null falls in this value sorry falls in this range is 5%. So your type one error probability in this case exactly 5% in fact so type one error probability we can conclude significance level alpha equal. Does that make sense? Are there any questions?

[00:52:31]So identifying this area of type one error is actually um fairly straightforward still with that let's move on towards the type two error and let's quickly recall type two error failing to reject the null when it is actually false reject.

[00:52:55]So let me just write down write this down as well. I'm actually still building towards power. So um failing to reject the null when it is actually false. Yeah, the alternative is true.

[00:53:12]Probability.

[00:53:14]In fact, we haven't reached this one yet. Actually is graph.

[00:53:20]Okay, perhaps we could calculate it. So failing to reject the null falsely when it is true. Failing to reject the null when it is true falsely baby.

[00:53:35]Okay. What is the area in which you fail to reject the null? This is very straightforward discuss you fail to reject the null. Correct. um left value to I can look at the area under the alternative curve which essentially means area >> 0.95 hold on I haven't gotten there yet um okay I will double check the value as well.

[00:54:22]Not not sure to be honest. I will have to check. So yellow area highlighted represent the probability of a type two error happening.

[00:54:38]area I failed to reject the null and let's note students blueve the blue curve assumes that the null is true alternative truth or you failed to reject the null. So you yellow highlighted area here this represents the probability of a type two error because this is the area under the alternative curve in which you are failing to reject the null. So um let's see yeah the probability that x is less than this critical value under the alternative. So under the alternative the probability that x is less than the critical value is actually 85. Did you say 85 or did you say 95? Hey.

[00:55:29]>> Uh, no. I said 95, but now I get it.

[00:55:32]Why?

[00:55:33]>> Okay. All right. Are you sure you get it?

[00:55:36]>> Uh, yeah. Because it's not the null graph that we will >> Yeah, absolutely. This is actually a very common mistake as well. Um, type two errors assume alternative truth.

[00:55:48]Type one error probability actually 85% under the alternative X is less than the critical value.

[00:56:08]Um now what have I tried to build to over here? I'm statistical power about so I will quickly iterate this over here. So statistical power is simply the power to avoid a type two error. Um it is the power to avoid a type two error which simply means your probability of type two error here. The statistical power is simply equal to 1 minus this.

[00:56:37]So it's very simple once you understand the concept of a type two error then it's really straightforward power is equal to 1 minus the probability of a type two error and sinceulate type two error probability that is actually equal to 85%.

[00:57:04]So this means your power will come out to be what 15%, which is a very low level of power actually. But are there any questions so far?

[00:57:20]Okay, logically what could I do to increase this power? Um you null or alternative curves? This is something for you to think about as well. I will not go over all of this content from the lecture. inves the center of the alternative this center this distance actually represents the uh effect size which is one of the main ingredients of power over here. So effect size effect size is actually in simple words it corresponds to effect treatment on the outcome. So statistical power the effect size is something that you can't practically influence. For instance, if you're observing effect size but you can't enforce it. Um one other thing is sample size sample size increase your curves or narrow and the narrower that these curves are the lesser area is going to fall over here and in turn your power is going to increase. So sample size increase power increase.

[00:59:07]Um and then there is the variance. I will let you go over the rest of power yourself as well. But are there any questions before we move on though? I will just write this down over here. Are there any questions?

[00:59:27]>> No.

[00:59:29]>> Okay. All right. Perfect.

[00:59:31]So this is the last question that I have for today. Um one question from bootstrap sampling from a past exam to iterate just a couple of small things.

[00:59:43]So you are investigating whether background music negatively affects exam performance or not. To assess this you conducted an observational study by analyzing the exam scores of two groups of five students with similar characteristics. One group studied with background music. This is your treatment group. And then the other group studied without without background music. This is your control group. Each exam score is out of 100 or up scores given here for the treatment and the control groups. To validate your hypothesis, you conducted a bootstrap analysis.

[01:00:20]What exactly is bootstrap sampling?

[01:00:23]bootstrap reample with replacement.

[01:00:30]Um you do that in order to increase your data because sometimes mis samples nal from nature is difficult for you.

[01:00:41]Sometimes it's infeasible altogether.

[01:00:44]Sometimes it's not possible. Sometimes it's just too expensive. Um however bootstrap sampling important original sample several times. So your original sample itself must be um well it must not be biased. It must be well representative of the population and your original sample must be large enough. original matter, right?

[01:01:20]10,000 then my would make sense to me because I'm trying to increase my data. The second thing is replacement which means size original sample same.

[01:01:43]So treatment groups bootstrap sample it is possible that the sample could look like this.

[01:01:57]All right. So numbers repeated but size exactly original sample.

[01:02:05]So anyway this is what I have obtained um three times bootstrap samples. I took three bootstrap samples or three numbers sample mean. I did the same for the treatment group and for the control group. Um, so before we look at the rest, state the null and the alternative hypothesis for this experiment. I will pause over here for a few seconds and I want you to think about this.

[01:03:06]Okay. So, what is the null? What is the alternative?

[01:03:15]Okay. So, it's really simple.

[01:03:16]Investigate background music effect on exam performance. So null assumes no effect which means that the null is that background music does not affect exam performance.

[01:03:36]The alternative is that background music up in this case in fact negative impact measure. So you would say that negatively impacts exam performance in which case null that it does not impact exam performance or it positively impacts exam performance. So this really depends on your question.

[01:04:03]Even if you wrote down that the null is K the background music has no impact on exam performance and the alternative is that it does have an impact on exam performance that would that would still have been acceptable I suppose. So the sample means calculate original and treatment groups. I've already written these down over here because it's simple enough to compute in five numbers mean.

[01:04:28]Uh next, which of the two test statistics tistic and zstistic is more appropriate for testing your null hypothesis justify your answer? So these are different tests that you have looked at. T test, permutation test and Z test or detailed course of permutation test cover. But for the sake of uh one example you can see over here um permutation test assumption T and Z test assumptions. The important thing to note over here is T test wants a small sample size and Z test the sample size has to be large. In our case, sample size five, which is very small, which is why we're choosing a t a t stat.

[01:05:19]Lastly, a colleague suggests that the variance of the sample means for both groups appears too high. She proposes increasing the size of each bootstrap sample to address this issue. Do you agree with her approach? Provide proper reasoning for your answer.

[01:05:40]We do not agree with her approach because like I mentioned at the very beginning, this is a trick question.

[01:05:46]Bootstrap sampling. You have to ensure that the size of every resample is the same as the size of your original sample to size increase in size increase. You are still sampling from those same five values. So it shouldn't make any difference. That's actually going to be it from my side for today. Are there any questions?

[01:06:16]>> Um, no.

[01:06:18]>> Okay. All right. Perfect, guys. All the best with your exam, too. Allah.