Programmatically determine which Deep Learning layer properties contain learnables

In the Deep Learning Toolbox, there are a variety of layer object types. The display methods of these objects indicate which object properties contain learnable parameter data. For example, for LSTMLayer objects, the learnable parameters are stored in the properties "InputWeights", "RecurrentWeights", and "Bias" as shown below. My question is, is there a way, given a layer object, to programmatically determine the subset of its properties that are learnable?

layer = lstmLayer(100,'Name','lstm1')

layer = 
  LSTMLayer with properties:

                       Name: 'lstm1'
                 InputNames: {'in'}
                OutputNames: {'out'}
                  NumInputs: 1
                 NumOutputs: 1
             HasStateInputs: 0
            HasStateOutputs: 0

   Hyperparameters
                  InputSize: 'auto'
             NumHiddenUnits: 100
                 OutputMode: 'sequence'
    StateActivationFunction: 'tanh'
     GateActivationFunction: 'sigmoid'

   Learnable Parameters
               InputWeights: []
           RecurrentWeights: []
                       Bias: []

   State Parameters
                HiddenState: []
                  CellState: []

Use properties method to see a list of all properties.

Answer 1

[Pratyush Swain]

Hi Matt,

I understand you want to access the learnable properties of a deep learning layer object. There is no direct way fetch the trainable properties of a layer but a workaround can be to initialize a deep learning layer from the "lstm" layer object and then fetch its learnable parameters.

Please refer to the below example implementation:

% Define a lstm layer %
layer = lstmLayer(100,'Name','lstm1')

layer = 
  LSTMLayer with properties:

                       Name: 'lstm1'
                 InputNames: {'in'}
                OutputNames: {'out'}
                  NumInputs: 1
                 NumOutputs: 1
             HasStateInputs: 0
            HasStateOutputs: 0

   Hyperparameters
                  InputSize: 'auto'
             NumHiddenUnits: 100
                 OutputMode: 'sequence'
    StateActivationFunction: 'tanh'
     GateActivationFunction: 'sigmoid'

   Learnable Parameters
               InputWeights: []
           RecurrentWeights: []
                       Bias: []

   State Parameters
                HiddenState: []
                  CellState: []

Use properties method to see a list of all properties.

% Define a deep learning network %
net = dlnetwork(layer,Initialize=false);

% Retreive the learnable properties %
properties = net.Learnables;

% Display the learnable properties %
disp(properties)

     Layer         Parameter            Value    
    _______    __________________    ____________

    "lstm1"    "InputWeights"        {0×0 double}
    "lstm1"    "RecurrentWeights"    {0×0 double}
    "lstm1"    "Bias"                {0×0 double}

As we can observe, the "dlnetwork" forms a single layer network named as "lstm1" and we have successfully retreived its learnable properties as a table.

For more information , please refer to:

https://www.mathworks.com/help/deeplearning/ref/dlnetwork.initialize.html

Hope this helps.

[Matt J]

Perfect! Thanks.

[Matt J]

In order to use this technique, though, I must programmatically determine if the layer is a type of output layer, because dlnetworks may not contain output layers. Is there a way to detect that?