Visualize Linear Regression in 3D

Added: 2026-06-03

[00:00:00]Let's revisit our example of employee salaries. Obviously, in the real world, salaries depend on many other factors.

[00:00:08]For example, we can consider the number of years of formal education. To account for this extra dimension, instead of fitting a straight line, we will fit a plane. What this means for our equation is that we must add another variable.

[00:00:26]Now to solve linear regression we need to minimize the average squared error.

[00:00:31]Remember that each error is the difference between the predicted salary and the actual one. We take the average across all data points. By the way, you have probably noticed that we are back to the one-dimensional case. This is because it lends itself better to visualization.

[00:00:49]To find the minimum of this function, you can use the algorithm called gradient descent. The idea is simple.

[00:00:56]You start at a random point and then you keep moving in the direction that decreases the function the fastest. This direction is given by the negative of the gradient which is a vector of all partial derivatives of the function.