Levenberg-Marquardt Optimizer

The Levenberg-Marquardt [96] optimizer (LMMIN) uses a so-called trust region method where the search direction $\vec{p}$ is a combination of steepest descent and the Newton direction. The search direction is defined by:

$$\vec{p}\,({\boldmath J}(\vec{x})^\mathcal{T}{\boldmath J}(\vec{x})+\lambda {\boldmath I}) = -{\boldmath J}(\vec{x})^\mathcal{T}\vec{f}(\vec{x})$$

where ${\boldmath I}$ denotes the unity matrix, $\lambda \ge 0$ the Marquardt parameter, and ${\boldmath J}(\vec{x})$ the Jacobian matrix:

$${\boldmath J}(\vec{x}) = \left( \begin{array}{ccc} \frac{\par... ... \frac{\partial f_m(\vec{x})}{\partial x_n} \end{array}\right)$$

The value of $\lambda$ is adjusted based on the last evaluation. For a value of $\lambda=0$ the direction $\vec{p}$ results in the Newton direction, whereas for $\lambda=\infty$ the direction $\vec{p}$ is common to the steepest descent method.

The implementation used in the presented examples is based on the MINPACK project [97,98].

2003-03-27