fmincon solution different for different start values

Hello everyone,

i have a question regarding the optimization tool fmincon.

In order to check my results i have run my optimizatio with different start values.

While the output of the function (here LCOE) is really close, values at which the output ist produced differ.

I am not sure if its normal or if i did anything wrong.

For additional information. I am optimizing the four values (Cut Off, Threshold, Tank and Cap_EL) in order to achive the lowest LCOE, Levelized Cost of Electricty.

Here is the code of my optimization:

clear all, close all

%Startwerte für die Optimierung
Start_CO = [ 5 15 30]; 
Start_TH = [15 30 50];
Start_Tank = [ 2500 10000 20000];
Start_Cap_EL = [5 10 25];

%Gesammelte Startwerte zum Übergeben
Start = [Start_CO; Start_TH; Start_Tank; Start_Cap_EL];

%Lower and Upper Bound [Cut off Threshold Tanksize Cap_EL]
LB = [0 0 0 5];
UB = [50 100 40000 100];


for i = 1:3
    rng default 
    
    opts = optimoptions(@fmincon,'Algorithm','sqp');
    problem = createOptimProblem('fmincon','objective',...
        @main_S1,'x0',Start(:,i),'lb',LB,'ub',UB,'options',opts);
    gs = GlobalSearch;
    [x,f] = run(gs,problem)
    
    LCOE(i) = f;
    Cutoff (i) = x(1);
    Threshold(i) = x(2);
    Tank(i) = x(3);
    Cap(i) = x(4);
end 

Thanks alot in advance and greeting from germany!

Answer 1

[Matt J]

While the output of the function (here LCOE) is really close, values at which the output ist produced differ.

That means they are all equivalent solutions, right? If you have multiple equivalent solutions, it is reasonable that GlobalSearch can reach different ones from different start points.

[Isabell Hetmann]

I should have shared the output for the three different starting points, sorry,

The LCOE, which is calculated in main_S1 is the same. But the values of the other variables are different. For Cut Off Threshold and Cap_EL, they are almost the same, very close. But the Tank size, varies alot, which I find suprising and I´m not sure if its do to a mistake.

Solution

LCOE 249.03 249.65 249.05

Cut Off 67 64 67

Threshold 0 14 0

Tank 18351 37244 13658

Cap_EL 38 39 38

[Matt J]

It doesn't change my answer. The value of LCOE is what decides whether the other variables are solutions. Since the optimal LCOE is achieved in all cases, the values of the other variables are optimal in all cases. You have multiple optimal solutions.

[Isabell Hetmann]

Okay, thanks alot Matt. Then I have to think about how to handel it for my thesis.

[Matt J]

You're quite welcome, but if this resolves your question, please Accept-click the answer.

Answer 2

[John D'Errico]

There are many scenarios where a solver can produce different solutions for different start values. First, know that fmincon is not a global solver. It stops when it gets to a local minimum. If the problem has multiple local minima, then fmincon does not care. It stops when it cannot see any place better to search from where it is. Remember that a solver is not looking at the entire function, and knowing what the shape is. For example:

F1 = @(x) (x+3).^2.*(x-2).^2;
fplot(F1)

When you startt a solver near the solution at x==-3, fmincon will be happy to report a minimum found at x==-3. Likewise, it will be happy to report a solution at x==2. Which solution is returned will be based on the basin of attraction for that solution. Thiink of a basin of attraction as the set of points, wherein if you start the solver there, you will find a solution at one corresponding point. Here there are two basins of attraction, and two solutions. (To be mathematically pedantic, there are more than that because at infinitely many start points, working in double precision arithmetic most solvers will fail to converge to anything. Think about what happens if you started the solver at x=1e1000. Evaluating the objective function there produces an overflow at inf.)

Next, you may have a problem where the objective function has a flat valley. For example, consider the simple objective function of two variables:

F2 = @(x1,x2) (x1-x2).^2;
fsurf(F2)

Here as along as x1==x2, this problem has a minimum at 0, all solutions alopng that valley are equally good. Starting at any arbitrary point can yield a different solution.

obj = @(X) F2(X(1),X(2));
[X,fval,exitflag] = fmincon(obj,[4 5])

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

X = 1×2

4.5000 4.5000

fval = 1.3878e-17

exitflag = 1

[X,fval,exitflag] = fmincon(obj,[-3,7])

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

X = 1×2

2.0000 2.0000

fval = 1.1601e-15

exitflag = 1

fval was effectively zero to within floating point trash, and the fmincon default tolerances at each solution. In each case, a solver will find a corresponding solution. fmincon cannot find a better place to look, so it stops. But we know from how the function was constructed that ANY solution where x1==x2 is equally good. Again, solvers are not mathematicians, with some gods eye view of the problem. They are just tools that use your objective function as a black box. They send values in, and test to see the result that comes back from the black box of an objective as they were given. Then based on assumptions about the objective function like differentiability, etc., they decide where to look next. The metaphor I always use is to compare a solver to a blind person, walking around the surface with only a cane to guide them about how to move downhill.

[Isabell Hetmann]

Thanks John for this very elaborate answere, could you recommend a solver for a global minima?