\( \)
Gradient descent is a very popular method for finding the maximum/ minimum point of a given function. It’s very simple yet powerful but may trap in the local minima. Here I try to find the minimum of the following function:
$$ z= -( 4 \times e^{- ( (x-4)^2 +(y-4)^2 ) }+ 2 \times e^{- ( (x-2)^2 +(y-2)^2 ) } )$$
Here I have solved this function with Limited-Memory CMA-ES (LM-CMA-ES) and as you can see it didn’t trap in the local minima:
Here if we start at $$x=4$$ and $$y=2.2$$ we will trap in the local minima
if try different start point, for example, $$x=3.5$$ and $$y=2.2$$ we will find the minima:
code available at my GitHub.
