Test step sizes and parameter scaling for LSQNONLIN (and the Levenberg Marquardt algorithm)
17 views (last 30 days)
I have a multi-parameter and multi-objective optimisation problem for which I am trying to use LSQNONLIN.
1) On its initial iterations it appears that LSQNONLIN attempts to vary the parameters by as little as 1e-8. Is this correct? E.g. for parameters [1 1 1] it will investigate [1+1e-8 1 1] and then [1 1+1e-8 1} etc. For my problem these step-sizes hardly make a difference, thus wasting a lot of time, or causing it to make a poor initial evaluation. Is there any way to increase/control this step size?
2) What is the best way to scale the parameters. In my case I have parameters having an effect like p1*x^p2 so p1 has a linear effect while p2 causes a lot of non-linearity. At the moment I was scaling the parameters to all equal 1 i.e. normalize them. However in reality they are actually say p1=1e-4 and p2=12.
3) If I normalize the initial parameter set P by dividing by P itself I obtain ones (provided the initial set does not contain zeros). If I instead divide by (P*10) or (P*100) or (P*1000) I am effectively increasing the effect of the small step-size 1e-8. Is this a good approach to simply scale the initial parameter set closer to the step-size?
Alan Weiss on 12 Jan 2015
The initial perturbations are lsqnonlin attempting to estimate the Jacobian of the objective. You can control the gradient estimation as explained here.
MATLAB mathematical toolbox documentation