4.1.3 Steepest-descent

The modeling of the target function can be done by a linear approximation of $f(\vec{x}_0)$ based on a Taylor-series expansion of first order

$$f(\vec{x}_0 + \vec{p}) \approx f(\vec{x}_0) + ( {\mathop{\nabla }\nolimits f(\vec{x}_0)} ) ^{\cal T} \vec{p}$$ (4.6)

where $\vec{p}$ denotes a small step away from the current point $\vec{x}$. A reduction of the function can be achieved by a step $\vec{p}$ where $({\mathop{\nabla }\nolimits f(\vec{x})})^{\cal T} \vec{p}$ is a negative number with a large absolute value. The direction can be found by solving the following minimization problem

$$\min_{\vec{p} \in \mathbb{R}^{n}} \frac{({\mathop{\nabla }... ...ec{x}_0)})^{\cal T} \vec{p}} {\vert\vert\vec{p}\vert\vert _2}$$ (4.7)

which gives direction $\vec{p}$

$$\vec{p} = - {\mathop{\nabla }\nolimits f(\vec{x}_0)}$$ (4.8)

along the negative gradient of the target function f.