4.3.2 Levenberg-Marquardt Algorithm

The Levenberg-Marquardt algorithm [30] is an efficient method to solve nonlinear least-squares problems. This algorithm uses a trust region method described in Section 4.1.8. In this method the search direction p is defined by the equation

$$(J(\vec{x})^T J(\vec{x}) + \lambda I) p = -J(\vec{x})^T \vec{f}(\vec{x})$$ (4.41)

where I denotes the unity matrix, and $\lambda \ge 0$ the Marquardt Parameter.

Depending on the result of the previous evaluation the value of $\lambda$ is adjusted. A strategy is to decrease the value of $\lambda$ if the actual reduction of the target function is in the range of the expected value and to increase $\lambda$ if it is smaller.

As described in Section 4.1.8 for $\lambda = 0$ the solution of (4.26) is simply the Newton direction, and for $\lambda \rightarrow \infty$ the step $\vec{p}$ becomes parallel to the steepest-descent direction.