Gradient descent method for finding the minimum

\( \)
Gradient descent is a very popular method for finding the maximum/ minimum point of a given function. It’s very simple yet powerful but may trap in the local minima. Here I try to find the minimum of the following function:

$$ z= -( 4 \times e^{- ( (x-4)^2 +(y-4)^2 ) }+ 2 \times e^{- ( (x-2)^2 +(y-2)^2 ) } )$$

Here I have  solved this function with Limited-Memory CMA-ES (LM-CMA-ES)  and as you can see it didn’t trap in the local minima:

Here if we start at $$x=4$$ and  $$y=2.2$$ we will trap in the local minima

if try different start point, for example, $$x=3.5$$ and  $$y=2.2$$ we will find the minima:

code available at my GitHub.