Fmincon convergence

Problem description

I am fitting experimental data with well-knowed mathematical model, minimizing the root mean square error (my objective function) with fmincon command. I noticed that Octave does not converge at the optimal solution at the first shot (I know the expected values of the model parameters thanks to an optimization in Excel and in Matlab).
I have overcome the problem running a large number of simulations (e.g. 100+) with random initial conditions but included in the boundary range and extraprolating the solution with the best goodness (lowest fval). So, I obtain the expected results.
However, it is not clear to me why the fmincon command in Octave does not converge at the optimum values with only one attempt, while the same script runned in Matlab does.
According to this, I report the result of the 100 simulations for the two cases and as you can easily see in the following graph, the fval value for the code run in Matlab always remains constant (each simulation converges) while Octave presents many simulations with results far from the optimal point.

Similar problem was also reported here, but there was no solution

My system

  • OS: Windows 10 Pro
  • Octave version: Version 6.1.0
  • Installation method: e.g. Downloaded and installed “octave-6.1.0-w64-installer.exe” from https://www.octave.org/download.html

Thank you.

Thank you for your detailed analysis of your problem. Unfortunately, your observations do not really point to something Octave can improve :sweat:

As in your other reported page, where no solution could be given. Can you post us some example code that does not converge properly in Octave? :slightly_smiling_face:

Thank you @siko1056 for your reply.

In attached you can find an example.

Data.mat (1.0 MB)
Fit_script.m (1.3 KB)

Hi @siko1056,

are there any news regarding the example posted?

BR,

Thank you for the example data. fmincon from the optim-package seems to use an interior-point algorithm lm_feasible by default. First I thought it is a tolerance problem, 1e-4 by default. But decreasing it to eps does not help with the strong convergence dependence on the initial guess (blue graph).

I tried Octave’s builtin sqp-funtion and it seems to converge more reliably for your case (red graph).

clc; close all; clear all; format shortEng;
set(0,'DefaultFigureWindowStyle','docked');

load('Data.mat');
pkg load optim;

% Matlab converges at optimal solution (SuperOptimVal = [582.39, 688.86, 0.4688])
% with one attempt:

%for n = 1

% To get the correct solution with Octave I have to run 100 simulations with 
% random initial guess and extrapolate the simulation with lower fval:

for n = 1:100

lb = [400,   600,   0.3]';
ub = [600,   800,   0.6]';

% Initial conditions for model parameters:
% 100 different values for each simulation in the boundary range 
InitPoints = lb+rand(3,1).*[ub-lb];

function obj = objFun(Coeff, x_experimental, y_experimental)
  y_model = Coeff(1) + Coeff(2)*x_experimental.^Coeff(3);
  obj = sum((y_model-y_experimental).^2);
  %disp([obj, NaN, Coeff'])
end

options.TolFun = eps();
fun = @(Coeff) objFun(Coeff, x_experimental, y_experimental);
[CoeffOpt(n,:), fval(n,:), ExitFlag, OutInfo] = fmincon(fun, InitPoints, [],[],[],[],lb,ub,[],options);
[x, fval_sqp(n,:), info, iter, nf, lambda] = sqp (InitPoints, fun,[],[],lb,ub,[],[]);

end

SuperOptimVal = CoeffOpt(find(fval==min(fval)),:);
[min(fval) NaN, SuperOptimVal]

figure;
plot([1:n],fval,'b',[1:n],fval_sqp,'r'); grid on;
xlabel('Objective function'); ylabel('Optimization number');
ylim([0 6e8])

Thank you for your exhaustive reply.
Initially, I tried to use an sqp algorithm, always with fmincon command, without success.
After you answered me, I expected that doing the comparison between fmincon (with sqp algorithm) and Octave’s builtin sqp-function suggested by you, I would obtain the same result, as written below:

MySettings = optimset;
MySettings.Algorithm = 'sqp';
MySettings.TolFun = eps();

fun = @(Coeff) objFun(Coeff, x_experimental, y_experimental);
[CoeffOpt(n,:), fval(n,:), ExitFlag, OutInfo] = fmincon(fun, InitPoints, [],[],[],[],lb,ub, [], MySettings);
[x, fval_sqp(n,:), info, iter, nf, lambda] = sqp (InitPoints, fun,[],[],lb,ub,[],[]);

But the following resulting graph shows that it’s not the same thing.

I accepted your answer as solution but I want ask you what is the difference between the two methods. I apologize for this question, but I’m new for the Octave environment and there is no sqp function in Matlab.

In Octave or in Matlab, why did it not work for you?

This is “true”, the literal sqp function does not exist in Matlab, but a sequential quadratic programming (SQP) algorithm can be chosen via the options of fmincon, e.g.

options.Algorithm = 'sqp';
% ...
x = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,options)

For the details of Matlab’s SQP implementation, see Constrained Nonlinear Optimization Algorithms - MATLAB & Simulink.

Also fmincon of the optim-package offers other algorithms, but I cannot find literature references. You can simply try them out, study the code for references and implementation details, or email the package maintainer to get some implementation details :slightly_smiling_face:

At least for Octave’s sqp the results seem to be the same on both our machines.

Regarding the optim package fmincon default algorithm, it is strongly depended on the initial value with your given data. As the initial values are pseudo-random with probably a different seed value, both our experimental outputs are very likely to be different :sweat: