A.1 Convexity of Sets and Domains

A subset $S \subset X$ is defined to be convex if for arbitrarily chosen points $x,y \in S$ the connection line between these two points

$$[x,y] := (1-\xi) x + \xi y : \xi \in [0,1]\} %\nonumber$$ (A.1)

is completely included in $S$  [219,348]. Therefore, the implication

$$x,y \in S \:\:\:\Rightarrow \:\:\:[x,y] \subseteq S$$ (A.2)

can be applied. Figure A.1 shows two typical parameter domains. The left domain in this figure is convex but the second one has obviously not a convex nature.

Figure A.1: Convexity of domains

For unconstraint parameters of an optimization problem the parameter domain is convex. If non-linear constraint functions are applied to this problem the parameter space might be become non-convex. If penalty or barrier functions are used instead of constraint functions the objective function becomes more complex but the lemmas for the convergence of these optimization problems can be again applied as for unconstraint optimization [219,220].