Checking Validity of Gradients or Jacobians
Check Gradient or Jacobian in Objective Function
Many solvers allow you to supply a function that calculates first derivatives (gradients or Jacobians) of objective or constraint functions. You can check whether the derivatives calculated by your function match finite-difference approximations. This check can help you diagnose whether your derivative function is correct.
If a component of the gradient function is less than
1, “match” means the absolute difference of the gradient function and the finite-difference approximation of that component is less than
Otherwise, “match” means that the relative difference is less than
CheckGradients option causes the solver to check the
supplied derivative against a finite-difference approximation at just one point. If
the finite-difference and supplied derivatives do not match, the solver errors. If
the derivatives match to within
1e-6, the solver reports the
calculated differences, and continues iterating without further derivative checks.
Solvers check the match at a point that is a small random perturbation of the
x0, modified to be within any bounds. Solvers do
not include the computations for
CheckGradients in the function
count; see Iterations and Function Counts.
How to Check Derivatives
At the MATLAB® command line:
optimoptions. Make sure your objective or constraint functions supply the appropriate derivatives.
Central finite differences are more accurate than the default forward finite
differences. To use central finite differences at the MATLAB command line, set
FiniteDifferenceType option to
Example: Checking Derivatives of Objective and Constraint Functions
Objective and Constraint Functions
Consider the problem of minimizing the Rosenbrock function within the unit
disk as described in Constrained Nonlinear Problem Using Optimize Live Editor Task or Solver. The
rosenboth function calculates the objective function
and its gradient:
function [f g H] = rosenboth(x) f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2; if nargout > 1 g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1)); 200*(x(2)-x(1)^2)]; if nargout > 2 H = [1200*x(1)^2-400*x(2)+2, -400*x(1); -400*x(1), 200]; end end
rosenboth calculates the Hessian, too, but this example
does not use the Hessian.
unitdisk2 function correctly calculates the constraint
function and its gradient:
function [c,ceq,gc,gceq] = unitdisk2(x) c = x(1)^2 + x(2)^2 - 1; ceq = [ ]; if nargout > 2 gc = [2*x(1);2*x(2)]; gceq = ; end
unitdiskb function incorrectly calculates gradient of
the constraint function:
function [c ceq gc gceq] = unitdiskb(x) c = x(1)^2 + x(2)^2 - 1; ceq = [ ]; if nargout > 2 gc = [x(1);x(2)]; % Gradient incorrect: off by a factor of 2 gceq = ; end
Checking Derivatives at the Command Line
Set the options to use the interior-point algorithm, gradient of objective and constraint functions, and the
% For reproducibility--CheckGradients randomly perturbs the initial point rng(0,'twister'); options = optimoptions(@fmincon,'Algorithm','interior-point',... 'CheckGradients',true,'SpecifyObjectiveGradient',true,'SpecifyConstraintGradient',true);
Solve the minimization with
fminconusing the erroneous
[x fval exitflag output] = fmincon(@rosenboth,... [-1;2],,,,,,,@unitdiskb,options); ____________________________________________________________ Derivative Check Information Objective function derivatives: Maximum relative difference between user-supplied and finite-difference derivatives = 1.84768e-008. Nonlinear inequality constraint derivatives: Maximum relative difference between user-supplied and finite-difference derivatives = 1. User-supplied constraint derivative element (2,1): 1.99838 Finite-difference constraint derivative element (2,1): 3.99675 ____________________________________________________________ Error using validateFirstDerivatives Derivative Check failed: User-supplied and forward finite-difference derivatives do not match within 1e-006 relative tolerance. Error in fmincon at 805 validateFirstDerivatives(funfcn,confcn,X, ...
The constraint function does not match the calculated gradient, encouraging you to check the function for an error.
unitdiskbconstraint function with
unitdisk2and run the minimization again:
[x fval exitflag output] = fmincon(@rosenboth,... [-1;2],,,,,,,@unitdisk2,options); ____________________________________________________________ Derivative Check Information Objective function derivatives: Maximum relative difference between user-supplied and finite-difference derivatives = 1.28553e-008. Nonlinear inequality constraint derivatives: Maximum relative difference between user-supplied and finite-difference derivatives = 1.46443e-008. Derivative Check successfully passed. ____________________________________________________________ Local minimum found that satisfies the constraints...