I am trying to verify my Jacobian calculation for use in lsqcurvefit using the DerivativeCheck option. I typically find that the check fails the 1e-6 tolerance, but barely,
Objective function derivatives:
Maximum relative difference between user-supplied
and finite-difference derivatives = 1.0823e-06.
User-supplied derivative element (27,2): -0.0547181
Finite-difference derivative element (27,2): -0.0547171
Wondering how genuine these failures were, I coded my own finite differencer and found that all elements of my finite-difference Jacobian Jn(i,j) and analytical Jacobian Ja(i,j) could be made to agree well for some (i,j)-dependent choice of the finite differencing stepsize, delta(i,j).
It occurs to me, however, that Optimization Toolbox finite differencers probably use only a j-dependent stepsize in its normal mode, rather than an (i,j)-dependent one. Otherwise, the objective function would have to be called numel(Jn) times instead of only size(Jn,2) times. On the other hand, maybe the Toolbox finite differencer doesn't operate in normal mode when DerivativeCheck is 'on'.
My question is whether the Toolbox always uses only a j-dependent delta in its finite differencing? If this is the case, it might be impossible in some cases to ever pass the DerivativeCheck tolerance. The example below attempts to illustrate this. It plots the worst error in the Jacobian as a function of stepsize delta. The resulting plot shows that the 1e-6 tolerance threshold is never achieved over a broad choice of delta.
f = @(x) [1000+0.9*x ; cos(100*x)];
Error(:,i) = abs( (f(delta(i))-f(0))/delta(i) - Ja);
ylabel 'Worst Error'