Linear least squares regression: a different view

Abstract

The main result of this paper is filling an existing gap between the theory of least squares regression and the solution of linear systems of equations. A linear least squares regression problem with p-parameters over n cases is converted, via non-orthogonal transformations, into a k-parameter regression problem through the origin on n - p + k cases, and p - k equations in diagonal form with p - k unknowns, 0 < k < p. As a consequence of this result: (i) tests and confidence intervals can be easily obtained for any subset of the parameters of the model; (ii) the regression problem can be converted into p-univariate regression problems through the origin based on (n - p + 1) cases only; (iii) one may conclude that we can talk about the influence of the observations on any subset of the least squares estimates; (iv) the PC user may provide solutions to regression problems of higher dimension than the ones previously handled.