How to get Gradient from Network Created by Neural Network Toolbox?

How can the gradient of the performance function with respect to the weights and biases of a neural network created by the Neural Network Toolbox be obtained? I am looking for a function analogous to "getwb".

[x,t] = simplefit_dataset;
net   = feedforwardnet(10);
net   = train(net,x,t);
wb    = getwb(net);
gwb   = ????; % something analogous to getwb(net)
              % resulting in a 31x1 vector

Thanks, Ahmed

Answer 1

[Ahmed]

Actully, the function 'staticderivative' can do this for a static network. In general, however, it is likely that defaultderiv is the better choice.

 help defaultderiv
 help staticderiv

Answer 2

[Greg Heath]

[x,t] = simplefit_dataset;

net = fitnet; % No need for feedforwardnet

rng(0) % Convenient for duplicating the run

[ net tr ] = train(net,x,t);

wb = getwb(net);

tr = tr % No semicolon...Look at all of the goodies!

stopcrit = tr.stop

bestepoch = tr.best_epoch

gradient = tr.gradient; %complete history

Hope this helps

Thank you for formally accepting my answer

Greg ;

[Ahmed]

Thank you for your reply, Greg. The gradient in the "tr" structure is not the gradient of the performance with respect to the weights and biases. The vector contains one element per training iteration. It seems to be something like the gradient magnitude. The solution I am looking for would yield a vector of the same size as "wb", in this example 31x1.

[Greg Heath]

Either output wb every epoch or dig into the source code of trainlm.

[Ahmed]

'wb' would be the vector containing the current weight/bias coefficients and not the gradient of the performance.

[Greg Heath]

Yes. Then approximate the derivatives with finite differences. {Novel idea!)

[Zheng Chai]

Hi Greg, sorry to bother you but do you know how to get the weights and biases after each epoch? I mean record the w&b within training. You mentioned dig into the source code of trainlm, can you help me with where to insert my command in trainlm? I am a beginner and know little about matlab. Many thanks.

[Greg Heath]

Sorry, Each year it seems that MATLAB modifies code, presumably to make it better in some esoteric way.

However an annoying result is that the code has become more and more difficult for the average user to understand.

Greg

PS Try contacting MATLAB directly.

[Zheng Chai]

Thank you so much Greg. Indeed Matlab modified its code. I could not find a proper helper function in my R2013b, but after updating it to R2017b, I found it. Thank you again for your kind help.