Performance Concepts in Machine Learning

[00:00:00]Hello everyone, welcome to NESU Academy.

[00:00:03]In this lecture that is performance concepts, we are going to understand the machine learning terminologies related to performance of a machine learning model. So the topic for this lecture is performance concepts and in this lecture we are going to study about total six performance concepts in machine learning. Let's get started.

[00:00:26]The first performance concept is performance evaluation metric.

[00:00:32]Performance evaluation metric is a quantitative measure used to assess the performance of a machine learning model.

[00:00:40]What does that mean? It simply means that performance evaluation metric is a quantitative measure or I can say a numerical indicator that is used to assess or test or evaluate the final performance of a machine learning model.

[00:00:59]So in the previous lecture I have explained you that after the training of a machine learning model we evaluate its performance on unseen data. that evaluation or testing of a machine learning model can be done by a numerical tool that is a numerical measurement tool. That tool is known as performance evaluation metric. So it is basically a metric or a measurement tool that is used for evaluating the performance of a machine learning model.

[00:01:31]for example, accuracy, precision, recall, mean squared error, even mean absolute error, F1 score and many others. Now you already know that what is accuracy? Accuracy is out of the total number of classifications, how many classifications are correct? That is accuracy. Like in case of spam email detection, accuracy can be the performance evaluation metric. Higher is the accuracy, better will be the performance of the machine learning model. Similarly, there is mean squared error. It is also an example of performance evaluation metric. And as you can see, mean squared error is a type of error, which means if the value of mean squared error is less, then the performance of machine learning model will be better. Similarly, precision, recall, F1 score, mean absolute error all are the examples of performance evaluation metric. And we will study about all the performance evaluation metrics in our upcoming lectures. In this way, we are done with our first performance concept that is performance evaluation metric. The second performance concept is overfitting. So overfitting is a condition that arises when the machine learning model learns the training data too closely including noise and irrelevant patterns resulting in poor performance on unseen data. What does that mean? It simply means that overfitting occurs when the machine learning model learns the training data too closely. too closely means that instead of understanding the actual patterns, features and relationships behind the data, the machine learning model memorizes the training data such that it also learns the noises that is unwanted data and irrelevant patterns, the patterns that are not even useful which results in poor performance of the machine learning model on unseen data.

[00:03:43]So the model performs very well on the training data because it has memorized the training data but it will perform very poor on the unseen data that is testing data. For example, if the training accuracy is 99%.

[00:04:00]And testing accuracy is 67%.

[00:04:03]It is the case of overfitting. Why?

[00:04:07]Because model performs very well on the training data because it has memorized the training data. It has not even understood the actual patterns, features and relationships of that data. It has just memorized the whole data. That is why the training accuracy is very nice.

[00:04:23]But when it is evaluated on unseen data that is testing, its accuracy is very bad. That is 67%. So this is known as overfitting.

[00:04:34]Now let me explain overfitting with a very nice real life example. Suppose you have an exam tomorrow and today you are practicing the same set of questions again and again again and again. Then it will be like that you have memorized those questions. You have memorized those questions too closely such that you also learned the full stops, the commas, even the question numbers that are like noises or irrelevant patterns that will result in poor performance in final examination because you have memorized the data without even understanding the proper concepts and patterns behind those problems. And that is why if the same type of questions will appear in exam then you will have a very nice performance. But if unseen questions will appear in exam then you will have a very poor performance. That is why the training accuracy is very nice but the testing accuracy is very bad. This is overfitting. Now let's understand what is underfitting.

[00:05:39]Underfitting occurs when the machine learning model fails to learn the underlying patterns in the data resulting in poor performance on both training as well as testing data. So underfitting is a condition that arises when the machine learning model is not able to learn the patterns, features and relationships in the training data which result in the poor performance on both training as well as testing data. It is like you haven't even practiced a single question and you have appeared in your final examination. Then if the same type of practice problems will appear in your exam then also you will have a poor performance. And even if the unseen question comes then also you will have a poor performance because you are not able to learn the concepts features and patterns behind those questions. That is why in case of underfitting the training accuracy and testing accuracy are very poor. For example, here training accuracy is 58%. And testing accuracy is 49%. That means on training as well as testing the machine learning model will have a very poor performance. This is called underfitting. Overfitting and underfitting are the most frequently used terms in machine learning. So I hope you now have a good understanding of both the terminologies overfitting as well as underfitting. Now let's move towards our fourth term that is bias.

[00:07:16]Now what is English meaning of bias?

[00:07:19]Bias means having an interest or liking in one particular thing over the others.

[00:07:25]This is the English meaning of bias. Now let's understand what is bias in machine learning.

[00:07:33]The model makes very simple assumptions about the data because of which it cannot understand the real patterns.

[00:07:42]What does that mean? It simply means that if bias is high then the machine learning model makes very simple assumptions about the data that we have provided to it. The model becomes too simple that it is not able to handle the complex relationships in the data. And because of this it cannot understand the real patterns, features and relationships in the data. This is known as bias. Now let's understand bias by taking an example. Consider the example of house price prediction system. Now the prices of the house are dependent on many features like size of the house, age of the house, number of bedrooms in the house, location of the house and so on.

[00:08:32]If the model becomes too simple such that it predicts the price of the house only on the basis of size of the house ignoring all the other features then this is known as high bias because the model is having a bias towards only one feature that is size of the house. This means that model makes very simple assumptions about the data. It is ignoring all the other features and considering only the size of the house for predicting the price of the house.

[00:09:05]This is known as bias and it will result in poor performance of model on both the data training data as well as testing data because the model is not able to learn the complex patterns the features the relationships of that data and that is why it will have a poor performance on training as well as testing and the condition in which the machine learning model performs poor in both training as well as testing data That is known as underfitting. Which means if bias is high, it will lead to underfitting.

[00:09:43]Now let's understand what is variance.

[00:09:47]The model learns the training data too closely and becomes sensitive to small changes in data. So I hope you can relate this definition with overfitting.

[00:09:58]If the variance of the machine learning model is high, it means that the model learns the training data too closely such that it also learns the noises and irrelevant patterns of the data and becomes sensitive to small changes in data. Even if we did small changes in training data, it will have a very poor performance. And that is why high variance leads to overfitting because it performs very well on training data but it performs very poor on testing data.

[00:10:31]This is variance. Now let's understand bias variance tradeoff. We have just understood that high bias leads to underfitting and high variance leads to overfitting. So what will you prefer in the machine learning model? Obviously low bias and low variance so that there is no underfitting and no overfitting.

[00:10:56]So ideally a machine learning model with low bias and low variance is preferred so that there is no underfitting and there is no overfitting.

[00:11:07]But practically it is not possible. In practice reducing bias often increases variance and reducing variance often increases bias. That means practically if we reduce bias it will increase the variance and if we reduce the variance it will increase the bias. That means we need to find a balance of bias and variance and that is known as bias variance trade-off. So bias variance trade-off is the process of finding a balance between bias and variance to achieve the best model performance. So bias variance trade-off is a process of finding a state of balance or equilibrium between bias and variance so that bias and variance will have the least possible values so that there is no underfitting and no overfitting and our machine learning model achieves its best performance. This is known as bias variance trade-off. I hope you have understood all the six terms.

[00:12:14]performance evaluation metric, overfitting, underfitting, bias, variance and bias variance trade-off.

[00:12:23]These are the performance concepts in machine learning. And in this way we are done with our topic that is performance concepts in machine learning. This was all from my side for this lecture. Thank you and I'll see you in the next lecture.

[00:12:43]>> [music]