More on Linear Regression

Linear Regression

In basic linear regression, we loop over a number of candidate lines for the fit and grade them by a measure of how closely they fit the data; the line with the best grade is the winner, and this line is the linear regression line for that data. The value used for this grade is the sum of the squares of the residuals between each data point and the line height at that point. The best grade is then the line with the least sum of squares. So for our dataset y(x), we can describe the series of candidate lines by their values a and b in the standard line equation y = ax + b, and think of a 3D plot of sum of squares R versus a and b, with some minimum R_o as seen below:

Linear regression with a straight line

R_o is the least sum of squares and hence the best line fit, so we use its associated a and b values to obtain our line.

The general model form is:

The method of least squares is used in the estimation of the parameters

What should Linear Regression be used for?

Problems for which the relationships between variables are linear.
Linear regression works very well for finding linear associations between or regularities within a set of patterns.

Return to top