Is there a way to extract partial derivatives of specific layers in deep learning toolbox?

Hi,

I asked this question last year, in which I would like to know if it is possible to extract partial derivatives involved in back propagation, for the parameters of layer so that I can use for other purpose. At that time, the latest MATLAB version is 2019b, and I was told in the above post that it is only possible when the final output y is a scalar, while my desired y can be a vector, matrix or even a tensor (e.g. reconstruction tasks).

Now, is it possible to extract the partial derivatives of layer in 2020b? Thanks.

Answer 1

[Amanpreetsingh Arora]

As mentioned in the answer to the question referred by you, the only way to find partial derivatives of a tensor is by looping over elements and calling "dlgradient" as "dlgradient" only supports scalar input for auto differentiation. However, I understand your concern that this will waste time recomputing overlapping traces. In order to improve time performance, you can call "dlgradient" with 'RetainData' parameter set to 'true' which will retain the intermediate computation of gradient and any subsequent calls to the function will reuse these intermediate computations. Refer to the following documentation for more information on this.

https://in.mathworks.com/help/deeplearning/ref/dlarray.dlgradient.html#mw_10cd0d63-9a6a-461e-a2f7-4a3596f62a92

[SC]

Thanks. I just submitted a new feature request. It is a useful feature for network surgery and it is available in Tensorflow as well. I hope I can use a more convenient partial derivative extraction function in Matlab in the future. Since the values are already stored in the model during the backward propagation, it shouldn't be difficult to allow the user to extract those values as there is not additional computation required.