Parameter Calculation by Least Squares Minimization

The method of least squares consists of minimizing

with respect to β. Setting θ = Xβ, we minimize:

subject to:

Let

be the least squares estimate of β. The fitted regression

is denoted by:

The elements of

are called the residuals. The value of:

is called the residual sum of squares. The matrix:

which is the regression matrix without the first column of 1s, is called the predictor data matrix.

Model Variance

The variance of the model is defined to be the variance of ε. The statistic:

is an unbiased estimator of this variance.