3.2.3 Numerical Aspects

The solution of a least-squares problem directly from (3.13) is rather susceptible to roundoff error. An alternative, and preferred, technique is Singular Value Decomposition (SVD) [42] of the matrix $\mathcal{A} = \left ( \mathcal{Z}^{\cal T} \mathcal{Z} \right )^{-1} \mathcal{Z}^{\cal T}$.

However, in many cases the normal equations are very close to singular. A zero pivot element may be encountered during the solution of the linear equations, in which case one does not get any solution at all.

Any matrix $\mathcal{A} \in \mathbb{R}^{m \times n}$ whose number of rows m is greater than or equal to its number of columns n can be written as: the product of an m x n column-orthogonal matrix $\mathcal{U}$, an n x n diagonal matrix $\mathcal{W}$ with positive or zero elements (the singular values), and the transpose of an n x n orthogonal matrix $\mathcal{V}$.

$$\mathcal{A} = \mathcal{U} \cdot \mathcal{W} \cdot \mathcal{V}^{\cal T}$$ (3.15)

$$\mathcal{A} \cdot \vec{x} = \vec{b}$$ (3.16)

The solution of the linear problem can be calculated by

$$\vec{x} = \mathcal{V} \cdot \mathop{\rm diag}(1 / w_j) \cdot \mathcal{U}^{\cal T} \cdot \vec{b} .$$ (3.17)

SVD's only significant disadvantage is that it requires an extra array of the size n x m to store the whole design matrix, and that it can be significantly slower than solving the normal equations.

For solving the least-squares problem (3.12)

$$\mathcal{A} \mathcal{Z}^{\cal T} \mathcal{Z}$$ = (3.18)

$$\vec{b} \mathcal{Z}^{\cal T} \vec{y}$$ = (3.19)

and the result $\vec{x}$ is the parameter vector of the model function $\vec{a}$.

In the previous section the fit of one response over an experimental region is described. In practical applications several responses have to fitted. As the set of input parameters is identical for all the extracted responses only the vector $\vec{b}$ changes and $\mathcal{A}$ is unchanged so the singular values only have to be calculates once.