Regression #2: Ordinary Least Squares (OLS)

OLS minimizes the residual sum of squares (RSS). RSS is the sum of each squared residual (residual = the observed Y minus the predicted "on the line" Y). Also, about the OLS: the average residual is always zero, and the line passes through the point (average X, average Y)

Category:

Education

Tags:

• Finance
• Quant
• Econometrics
• Statistics
• Regression

[Cyphlix]

Great explanation! can anyone tell me why bother squaring the terms at all? couldnt you also find the minimised residual sum without squaring?

[bionicturtledotcom]

@Cyphlix Thanks, appreciated. You can generate a line based only on minimum differences, those estimators (slope, intercept) are not "wrong." But the OLS estimators, penalizing distance by squaring, posses desirable properties (technically, they are BLUE: unbiased and efficient).

[LiveIgnition]

This was the best explanation I seen for linear regression by OLS. Very well explained by someone who can clearly see the big picture. Thank you!

[bionicturtledotcom]

@LiveIgnition thank you for the kind feedback, really appreciated. David

[ODannyBoi]

I dont get it... doesn't excel do all of this for you by doing a regression output and using the coefficients?

[bionicturtledotcom]

@ODannyBoi some people like to know how the coefficients are calculated in order to better understand their limitations and how better to apply and interpret them. Folks who skip this are the ones who tend to use them incorrectly, IMO

[mrbrdo]

Very clear explanation! Exactly what I needed, thank you!

[sachin01663]

Perfect and very helpful

[sereneallitia]

This is great!! Thank you so much for this excellent tutorial...you don;t happen to have one on partial least squares, do you? :-)