Part 1: Linear Model - No Data Split Example

[00:00:00]in this video I'm going to do a demo on how to do regression using slackit learn which is a machine learning library in Python so regression involves having a dependent variable that is continuous now there is a term called Target variable in machine learning and that's the same thing as a dependent variable and a regression model we're going to be focusing on a Target variable dependent variable that is continuous we're not going to be utilizing classification but just know that if your dependent variable is categorical then you would use a different type of technique and you would use a this would be considered a classification problem rather than a regression problem all right so that means the data that we're going to be working with is going to be numerical it's going to be probably involved some decimal points and we'll take a look at that in a little bit right so categorical is a little bit different it could be numerical but it's it's a discrete it could be zero or one or it's a categorical could be things such as um a binary variable such as smoker no smoker male female Etc all right so we're going to first start off developing a model using uh just one independent variable and one target variable now we know that the target variable is considered to be a dependent variable the independent variable in machine learning is the name for that is features so you'll hear the term features which would be the independent variable and then Target would be the dependent variable we're going to start off with a simple linear regression and I'm going to show you what the data looks like I just want to show you first all the different import statements that I have here in the cell we're familiar with these three already these three are the new ones related to scikit-learn and we're not going to just yet make use of this particular object since I get learn but we will be making use of this right now and then later on I will demonstrate what this is all about so in in this first statement we are importing something called train test split and that involves this is an algorithm that will split the data into two different sets the train set in a test set so I'll explain a little bit more about that later on um and then we have the linear regression so this is basically a function that we're going to be making use of that's coming out of the scikit-learn library and then we have another type of um object which is cross valve score has to deal with cross validation where you you can get a more generalized model more reliable model and we'll talk about that last okay now examining the the data that we're going to be working with and this is uh advertising data I'm going to run this cell so you can take a look at this uh well first I should go ahead and run these import statements I get these libraries down and then here we have the data frame uh now the data frame is going to involve four different data columns and we are focusing on one target variable which is sales this is going to be again the dependent variable and then the features are going to be either TV radio newspaper or all of them and we're going to start off with again a simple linear regression so that means we're going to have one target variable one uh one target variable and then one feature okay If You observe the data the data points are in fact continuous the target variable values are continuous these are numerical containing some also data points so this involves regression okay now before I continue let me just show you this link the content that I'm using is coming from this website with a little bit of additional content coming from me so feel free to go to this website and read through what's going on I'm also going to provide this link for your review all right now we have the advertising data on the data frame and just to make sense of this data we are dealing with 200 rows so that means we're dealing in the context of this data we're dealing with 200 different markets and the Assumption here is there is a company that is spending on different type of media streams so you have television advertising you have radio advertising you have newspaper advertising and these are uh to somehow predict the number of sales units we don't know exactly what kind of product but let's just go ahead and make sure that we understand that cells and the values that you see here just represent the units of some sort of product um and now if we pay attention to the first record then what we're looking at is if a company spends and since this right here says 230.1 uh it's important for us to understand that we're talking about in thousands so if a company spends 230 000 on television advertising um then that would mean it could result to possibly resolve to uh 22.1 which would in this case we would interpret this as 22 100 units okay um same thing for radio let's say this is about 30 37 800 uh advertising through radio right and this would be 69 200 advertising in newspaper now the model that we're working with is one of these could possibly predict what's going on here and the company managed to collect this data over time and see exactly what happened during a different during a certain period where they spent this much on television this much on radio this much on a newspaper and then somehow resulted in this number of units being sold right now we're dealing with 200 records here all right now what we're gonna look at here is we are gonna first inspect the data frame right um and the thing that I'm verbally talking about is also outlined here in text and I'm going to jump through this code cell right here and start inspecting the data frame and understanding it we know that we already have a view of the data frame right here where we have the first five records and the last five records that I'm gonna use a method called info to give me some metadata about this data frame we're dealing with these columns TV radio newspaper cells we're dealing with 200 observations nothing is blank I'm going to say no no nothing's blank here and then the data types for each of the data columns are floats so that's pretty straightforward now part of understanding data is also visualizing it and let's go ahead and see an example of how we could visualize this data there are different functions and methods that could be used different visualizations that could be used and what I'll just do here is I'm going to demonstrate something called a pair plot that's coming from Seaborn so if I write sns.pair plot and this function contains a bunch of different arguments that we can include but what I'm going to make use of is the X bars and Y bars and then I'm also have to indicate the data frame that we're using and things like that now if I write DF that it could be a positional argument and leave that in just like that but then I'm going to call V bars and open up a list and then here what I'll do is I'm going to include the different type of data columns that I want to have in my visual in my visualization so I have TV radio and then I got newspaper right that takes care of that argument then I'm going to call On yvars and Here what I'll do is Select cells that means that the X uh access is gonna represent TV radio newspaper and then the y-axis is going to be cells but basically what this is going to do is it's going to project three different charts and let's go ahead and take a look and see how that shows up and here are these Scatter Plots right so what this pair plot is doing is it's projecting these three Scatter Plots for each of the features so for we're going to say TV is a feature in sales is a Target and you can see exactly how things are looking in terms of the pattern right same thing for radio so we have radio in the second uh scatter plot shown right here and then we also have the y-axis being cells newspaper and we can tell that TV has a pretty good pattern going upwards so there seems to be some sort of linear relationship or at least a stronger linear relationship than the other variables that we see here these other features that we see here now um I'm going to do another pair plot I'll just go ahead and do that right here but I'm not going to put any sort of additional arguments other than does the data frame itself so pair blocks CF and I'm going to go ahead and just run that right it's going to go ahead and generate a pair grid right and this right here is giving us how each one of the variables or each one of the data columns pair up with another data column right now whereas here I was being specific in terms of what I want to see in the x-axis and what I want to see in the y-axis and if I don't if I'm not specific and just put DF then it's going to go ahead and look at all of the data columns that contain numerical data and pair those up in different combinations right now you can see the the diagonal or these are histograms and these are univari analysis paying attention to the distribution of that particular data column so if we look at the top left this represents the histogram for TP data and then the next one is a histogram for Radio Data but then the Scatter Plots are the pairs right so it's if you look at the first row here uh the first scatter plot is looking at TV data along with radio and showing what kind of uh plot is going on here when you combine the two and then we see this one right here on the on the very far top right where we got that pattern going on that we saw earlier so we know that this is probably TV in sales so if we go down we know yep that is in fact the TV in cells it seems to be a good pattern going on here when it comes down to an increase of uh spending when for television is going to increase sales units that's what we're seeing here pretty strong pattern other ones are scattered but we can still possibly make use of this in our model but let's just go ahead right now and just continue on so this is a very good quick visual to get you understanding the data and see what kind of patterns are going on through each one of the data cons in a data frame now if I use df.core this is going to provide me with a correlation Matrix and essentially what this is is it is a numerical version of what we're seeing here up top with these pair grids but we're looking at them from a numerical standpoint other than the uh diagonal values which are just ones all across right just because when you compare a TV with TV it's going to have a correlation coefficient of one but you can just ignore those and then start kind of combining see exactly what other type of correlation values that you have so for example television to radio right now that we're saying that there's a correlation of 0.054 which is a pretty weak correlation and we know that if we observe the plots right we observe the plot for TV and radio which is this one right here it's not really showing us a very clear pattern in any sort of Direction hence the weak correlation value that we see here same thing with newspaper and TV right so it's a weak correlation but then TV with sales we get a 0.90 right so we know that in correlation uh we're looking correlation uh is everything from either positive one to negative one or negative one to positive one anything in between the closer the value is to one and it's absolute then the stronger the relationship here we happen to have a 0.90 and it's positive it's close to one so we know that's a strong relationship right and if you were to compare this right here we know this is 0.05 close to zero pretty weak relationship here now if this were a negative 0.90 that would still be a strong correlation but in the opposite direction would be a negative correlation this is a positive correlation meaning that as one variable increases in its value the other variable also increases so the more TV advertising expense the more the sales units uh so that we can say that based off of this correlation that there is some sort of Association there right and um that's pretty much what I want to cover in relation to the correlation Matrix so we can kind of get an idea of which variables are associated strongly correlated strongly with other variables now what I'm going to be doing is I'm going to focus on cells as being the Target and I want to know how uh how these features will how they'll help predict cells right now since I'm doing a simple linear regression I'm just going to focus my attention on TV for now being the feature that's going to predict this target so going into number four uh is consensual stuff uh related to what we're doing some questions that could be asked related to this data uh is there a relationship between ads and cells well we're gonna we're gonna find out we're gonna focus on television ads and cells how strong is your relationship we were able to do that through a correlation Matrix um and we want to know which ad types contribute to cell so we do have television we have newspaper we have Radio Data we can see how each one of these measure out and so on right now we're gonna deal with a simple linear regression meaning that we're dealing with one target in one feature right the target is going to be y right dependent variable the feature is going to be X the independent variable and then we're going to have the y-intercept which is beta sub zero even though these are not shown as sub beta Sub Zero then we have beta sub one which is related to the feature and then together when you put them in these coefficients together that's what is a model so we're creating this model that's going to help us predict cells assuming that the model is okay and we're going to figure out and see if the model that we're coming up with is safe enough for us to say okay we can predict sales okay so let's go ahead and move on to this first part where it says estimating learning model coefficients and we're going to be making use of scikit learn uh some of the tools in this Library and then we'll kind of look at some of the methods and some of the values and just get understanding what's going on right now so in this this approach what I'm going to be doing is really not involved in machine learning just yet up until the next part where I'm going to talk about how to uh how to train the model and then test it out so there's a little bit of a process involved we're going to start off with uh this part right here where we're going to we're going to put the data into the respected variables now we said X is going to represent the feature Y is going to represent the target so let's go ahead and start doing that all right what I'll do is create a variable called X and this is going to be a capital x I'm going to just keep I'm going to keep with the best practice approach um so typically in Python when you have variables you don't capitalize variables but in the in this situation doing a second learning doing analytics capital x is something that is commonly practiced that represents the feature or features depending on the model that you're trying to do now I am going to use the data frame and I need to I need to go ahead and um make this into a data frame where it contains a column and this is important and I'm going to explain this uh more in detail in a little bit but one way I can do this and under different approaches to to this but one way that you could do this is by simply opening opening up two square brackets and putting in the feature that you want to use in your model so I'm going to use television and then from there I'm just going to go ahead and run it and you can see that it executed I'm going to open up a blink cell and I want to open up I want to look at the X variable that I just created right and you can take a look at a sample of this right here so it grabs the television data now if you pay attention to this part right here it's saying that we have 200 rows so 200 records and then one columns right so basically just one column related to television data now this right here is acceptable as an acceptable format and scikit-learn if I didn't use double brackets so for example if I just used one bracket I would still be able to run this without a problem but let's take a look at a sample of this you know it's it's different right now it's showing us a series of data it doesn't give us the information in terms of how many records and how many columns but what I can do is say shape and this is an attribute and it gives me the shape of the uh that data this Tuple is letting me know that I have 200 rows 200 observations 200 records and there's a comma with nothing in it meaning that there is no column right and that's because it's a series now if I want this model to work uh I need to make sure that I follow the instructions and scikit-learn documentation and it says that in order for us to create a feature variable an X variable that X variable has to contain a column even if it's just one column that means I would want to use I can use again this approach to double bracket approach right and now I ran it when I do X shape you can see that says 200 comma one so I know the fact that my data is in a column uh it will work without a problem right um I I do demonstrate how to do this using the the reshape approach and you can use that or you could use this technique and that will suffice for example let me just go ahead and see if I can demonstrate this uh quickly if I have TV right now this is a single bracket that means it's a series but it's not really there's no column involved it's just a one-dimensional array of data if I dot values dot reshape and then do negative one one right this is another approach where and let me go ahead and run into cell you can see now it's also saying 200 comma 1 because I'm using the reshape method I'm putting negative one comma one meaning that this is kind of a standard way of saying negative one is keep the same number of uh rows but include one column right so this is the same thing as doing something like this right it will both of these will create one column now I'm going to go ahead and just comment out this one I'll keep this one I'll probably have to revisit this later on and I'll explain why later but let me just run that run the shape just make sure it's showing there good to go all right now I managed to get my my uh feature right which is going to be television uh advertising expenses but I also need to get my Target so the target I'm going to go ahead and say here it's going to be sales but I want to make sure that I spelled this out all right so it's case sensitive I want to make sure it's exactly we're in the way it is in the data frame now this is going to be acceptable uh where I don't have to put double brackets so I don't have to use this uh the Target variable could be in any shape so it could be a one-dimensional array of data or it can have just be um you can contain this one column of data so this is going to be just fine it's the Target that is necessary to have a column at least one column of data inside of it all right now I can go ahead and close off of that and now let's go ahead and I'm not quite done I'm trying to jump ahead here I need to go ahead and pay attention to this part so I split up the data into these respective variables and now I need to instantiate and fit so I'm standing in the sense I need to instantiate the model that I'm trying to create and that's going to require me to use the linear regression object up top that I showed you earlier we're going to be making use of this right here this is a function that I'm going to make use so I've already ran this so I don't have to run it again but let me go ahead and go down right here and to instantiate a a model I'm going to go ahead and create a variable I'm going to call this variable LM for linear model and I'm going to hit the equal sign and then here's the function name and I'm going to make sure I put some parentheses right there at the end so this is what we mean by instantiate the model this is a function that contains a lot of predefined code that we don't have to worry about but it's basically the algorithm for a regression model and we're not going to have to go through the details of all of that and I just know that this once I have this set up then the linear regression function is going to uh return some information and it's going to be placed in the LM variable and I'm going to refer to my Ln variable and this LM variable is going to be capable of holding different methods and one of the methods is called fit I'm going to use the fit and put in the data that I want to use for my model so I have the X data and I'm going to hit comma and then I have the Y data so I have my feature which is a TV data I didn't have my target which is the sales data right so that is the fifth approach you're always going to fit with the train data I haven't yet talked about that but we will cover that in the next part there's really no training test data right now right now we're just putting all the data that's in the data frame and fitting it and seeing how things fare now uh this is create we've instanted the model and we've now created the model using the data available and we can go ahead and print out the coefficients related to that now one of those things is uh there's a coefficient um there's an attribute I should say it's called co-f underscore and that's going to give me the coefficient of the uh coefficient value of the uh feature right now there's another one lm.intercept and that is intercept underscore there that's going to print out now I want to I want to be clear and what's what I'm just going to write uh coefficient and then right here I'm going to say intercept and that should actually do it all right so let me go ahead and run everything I finally ran this code and the model was instantiated and then I used the fit method and put the data inside it went ahead and created the model and then based off of that model it printed out what the coefficient is for the feature and then what the y-intercept is and this is the data right now you can disregard like the format of how it Returns the data but we know that this is the coefficient value for so this is the coefficient value related to the television variable right that feature and then this is the y-intercept that we would include as part of the model now the way that we would work on assuming that this is a model that is sufficient and we'll look at how we know this in a little bit we have enough information to use if we were to have another value maybe in a different time period that the company is going to use of how much television expense um television advertising expense are going to have and what what it could possibly result to in terms of sales units right and we can go ahead and do that using this type of approach that we see right here here it says let's say that there was a new market where the television advertising spends uh spent spending was fifty thousand okay what would we predict for these cells in that market you can see exactly what's going on right here So based off of our predictions we know what beta0 is which is a y-intercept that we came up with and we also have beta 1 which happens to be the coefficient for the television X is going to be whatever we want to test it with we're going to test it with 50 000 since our data is in the scale uh uh where it's it's shortened out we're going to make sure that we put the proper type of value it's within the scale of the data right now I can simply just copy this example let's see here um we know that I'm going to copy this right here and I'm going to just do this manually where I have the intercept is this value and I'm going to add that to the coefficients right and I want to multiply that times x so X in this case is going to be 50 000 but I'm going to put 50 rather than 50 000 because of how the data Frame data is presented right I'm going to stick with that scale and just put 50. so I'll go ahead and delete this now and run it I end up getting 9.748 right so this is the result this is how many units sales units that were predicted based off of the model that we created right so it's in space off of 50 000 in TV advertising expenses another way you can write this is using the the different attributes now remember there's a there's the lm.coeb right underscore and then we also have the uh that's that's a coefficient but we also have LM intercepts underscore right and now if I wanted to use this let's see here I'm going to say it is positive so I'm going to say Plus and then right here times fifty thousand right I'm going to run that oh I do get an uh an error right here and that's because it says LM oops it should have been Ln dot intercept there you go and here's the results the next one is this cell right here it's going to generate the same thing but a different way where we're going to be making use of the predict method it's already written out for us uh if you're going to be using the predict method let's say for example we have another um a new piece of data where we know exactly how much was spent for a given month we can enter that value and it will predict so if I run this I end up getting this results right which is the same thing that I was able to get in these previous two examples so the predict method is useful but now just be careful because the predict method has to include double brackets and then the value in order for it to work if you don't you just have one right you get a value error so it has to accept it in a way where it's a column of data and that's why we're putting this double bracket right here now you notice that I do have a warning and it's saying that X does not does not have valid feature names um and that is that has to deal with how I split the data originally and for example I'm going to open up a a cell above this and show you the uh particularly I want to show you the X variable that was split so the X variable that was split if you recall it's a one column of data so this is a data frame one column of data now because I use the double bracket it went ahead and included the name of the feature which is television and that is kind of throwing off a warning uh when I'm trying to do some predictions off of maybe a different type of value for television expense to get that taken care of what I can do is when I split the data so let me just delete this I'm going to go back up particularly this part right here right I have these double brackets but if I write dot values then it will instead create an array of values and not include the feature name so that means I'm not going to have that warning anymore right so let me before I hit run to this cell let me open up another cell right X you can see how that looks like right but now I'm going to go back to the cell I didn't change dot values attribute I'm going to run it and now same type of results uh we we're going to go ahead and move into the cell run it and now it's an array rather than a data frame but it's an array that contains a column you can see how we have this double bracket notation right here so if I include the dot values it's going to make this error go away now when I run this it is removed right that's how you could clean out that warning it's not really an error it's just a it's a warning right an error would not really return anything and let you kind of proceed all right now we can play around numbers let's say the company ended up spending let's say 75 000 right so then I can put 75 right here and this is how much it would generate uh in terms of sales units I would generate 11 000 uh 11 000 1300 units or eleven thousand one hundred three right that's I think the better way to interpret that right I'm going to go ahead and put 50 and there we have it now when it comes down to plotting the results uh we could plot this using different different uh libraries and functions let me just go ahead and just do a quick DF just to kind of inspect the data right here and understand what I'm working with and I want to plot uh the what I'm going to do is I'm going to I'm going to plot this I'm going to use sns.lm plots and I'm going to include the x is going to be TV and the Y is going to be cells the data that's coming from the data frame variable and I'm going to run it now when I run this it's going to show me the data that I split right now if you recall I split the um I spite of into X and Y right um and I'm just going to go ahead and just use its original um names and the original data comes from the data frame but you can see right here that it went ahead and plotted all the data but it also provided me with the regression line all right now moving on to this next part if we wanted to know how well the data how well our model is then we have to look at the r squared of the regression model and one way we can do that is using the score method now remember when it comes down to r squared usually what we are thinking about is a value from zero to one the closer the value is to one then the better the model the stronger the model the closer it is to zero but then the less you know the weaker the model happens to be now to figure out the model now when we're saying model we're talking about this simple linear regression model that was just created we said there's only one feature which is the television expenses that's predicting sales units that's what we're saying here um and then we kind of tested it out with like fifty thousand dollar TV advertising expenses and and saw how it kind of predicted the sales units but uh that we really wouldn't want to do any sort of predictions until we know exactly how well the model is doing and that's through the r squared right now r squared is a proportion of the variance explained right and we're talking about how well the feature the independent variable that the television expenses how well it does in explaining the dependent variable right now the method for this would be score so you would want to use the LM that was instantiated and this can this LM contains the model that was created with the data so I'm going to refer to that and then say dot score and then enter two arguments the feature would be the first one to hit comma followed by the Target when we run this we get this score okay so this is uh pretty close to one it's a pretty strong model right so we're setting we're pretty much saying that 81 of the variance is being explained by the uh by the feature which is the television advertising expenses so that it's able to explain 81 percent of the variance of uh the dependent variable right which is sales units all right now looks like it's a pretty strong model could we improve it possibly we'll take a look at another example later on we can maybe include an additional feature so instead of just television maybe what if we were to also include like radio along with TV and how well would it how would the model be in terms of that would it be stronger and if it is stronger than maybe I can use that to predict rather than just Television right all right now moving into just kind of understanding what ended up happening um in the back end right now remember we split the data into X and Y here's how X looks like you see that this is an array of data and remember it's because now I changed it from the data frame to an array um and then we can see how y looks like let me go ahead and do that let me just type in y that's cells right this is a series of data and now if we wanted to view the like the predicted values meaning that we know we do have a model that can predict cells so anytime we put in a value for television advertising expenses right it's going to go ahead and return a value in terms of sales units predicting sales units right and that what I want to do here is I'm going to call the LM model that was generated and I'm going to use the predict method and I'm going to put the value X now remember X is a an array that looks like this with a column format if I were to run this it will go ahead and place each of the values in X into the model and it's going to return a it's going to return a predicted value of cells and units right so that means if we look at the very first value in x right this is the very first value of TV advertising expenses and this is the returned value right here 19.73 is the predicted value if we were to spending two hundred thirty thousand dollars in TV expenses so essentially what I just did right here by placing the X it went ahead and looked at each of the values inside the X uh variable and in in that particular order and then started predicting what those values happen to be right now to kind of get a full picture of this in a data frame what I'm going to do here is I have the original data frame shown right here but now I'm going to create a variable called predicted uh cells right and so this is going to be another data column in that data frame and this code that I used right here I'm going to copy that and I'm going to put it next to the assignment operator then I also just want to show the data frames I'll just write DF and run that out right that means this new column right here is the predicted cells units when the television advertising expenses was entered into the that prediction model right it generated that radio and newspaper is not something that we're paying attention to right so you can just ignore newspaper a radio newspaper data columns we're really paying attention to this feature television the actual cells and then the predicted cells and if you go through the actual cells each record right actual sales predicted sales actual sales predicted sales and so on you'll see that that they don't vary too much and that's because we had a strong correlation between TV and cells we have a pretty strong model based off of the r squared now if r squared if our r squared was not 80 let's say maybe it was 20 then you'll probably compare the actual sales and predicted sales and they would vary pretty far right so that's what's going on we created this model with one feature one Target and then we went ahead and was predicted we just test it out by putting all of the actual television expenses in the model and see what it predicted and we can compare that to the actual sales amount so here this is the target value that actually occurred and then here is the predicted value from the model all right so now I'm just going to run this data frame see how that looks like again just to refer to it but I'm going to create the visualization of the prediction model and this is going to be built using a matplotlib so what I'll do here is I'm going to call PLT dot uh let's go ahead and do a DOT scatter and I'm going to put in X and Y right and these represent the actual the actual cells right it's not anything predicted these are the actual cells when I do that here is that pattern that we've seen before and now in addition to that I'm going to also include another uh chart and it's going to be plt.plot and I'm going to write X for my x-axis data point and then for the Y I'm going to use that lm.predict and I'm going to put X right now if you recall I already have a variable that contains that right so I could probably call that instead which is d f dot I'm going to use a DOT notation here what did I ended up calling predicted cells I can say predicted underscore cells right and let me see if this is going to go through yep you can see the now the line is showing right there but I want to I'm going to give that a different color so let me go ahead and say color people's red there you go we got our model our meteor model shown right here in red the actual data points shown in blue not going to include some additional information just to understand what's going on so if we say plt.title I'll say simple linear regression model and then I'll do a p ltix label say this is TV spending we've got PLT y labeled and that would be sales for if it's a unit cells and um let's see I'm gonna go ahead and run that right now we have an example of this model I'm going to go ahead and do the ps3. show so just remove that data that was shown on both but this is the visualization of this model that we created so we see the observed value and then the prediction line as well all right so that is the first approach it doesn't really involve splitting data into different training and test sets we just made use of all of the data that is available to us but in the next part I am going to demonstrate the process of splitting the data first into two different sets the train and a test set and then based off of that train the model using the train set and then once the model is created I can let that model I'll feed that model new data coming from the test set and see how well that model is doing and that would be the better approach because here using all of the data we don't know how it's going to really how this model is going to do unless we end up collecting another set of data and then feeding it in and seeing if the model is in fact something that was reliable or not right so to avoid any sort of issues training the model using a train set and then test it with another set is the better approach and I'll explain how to go about that next