What a weighted least squares regression and when is it used?
1 Answer
Weighted Least Squares as name indicate is a mechanism for accounting for the impact of the each date point by weighting each data point by a proper amount of influence over the parameter estimate. You can interpret WLS as an extension of OLS that still leverages the "Least Square" method for optimizing the fit but in so doing does not assume a constant Standard Deviation of the error term.
Explanation:
One of the assumption in Ordinary Least Square (OLS) is the Standard Deviation of the error term is constant over all values of the predictor or explanatory variables. Put another way there is an assumption that each data point has equal impact or results in equal information to the underlying process. There are many cases this not the case and a procedure that treats all of the data equally would give less precisely measured points more influence than they should have and would give highly precise points too little influence. Clearly we can see for this cases the OLS is quite inadequate, so we seek a Least Square model that account for the disparate impact of the data points. Weighted Least Squares as name indicate is a mechanism for accounting for the impact of the each date point by weighting each data point by a proper amount of influence over the parameter estimate. This method still leverages the Least Square approach for optimizing the fit but in so doing does not assume a constant Standard Deviation of the error term.
Weighted Least Squares like all Least Squares is an efficient method using inversely proportional weights to the variance. It shares the ability to provide different types of easily interpretable statistical intervals for estimation, prediction, calibration and optimization. The main advantage that weighted least squares is the ability to handle regression situations in which the data points are of varying quality.
The biggest disadvantage of weighted least squares, is in the process of getting or estimating the weights. While the theoretical construct assumes that the weights are known, this is not the case in practice. Typically, in a real application the weighted are estimated, and the effect of the estimates may have significant impact on your model. Some empirical evidence shows that variations in the weights due to estimation do not often affect a regression analysis or its interpretation. However, when the weights are estimated from small sample of data, the results of an analysis can be poor.
