How to manually modify weights in a SeriesNetwork?

For some of my studies on deep neural nets, I need to change select weights in a (previously trained) SeriesNetwork by hand, and evaluate changes in the classification results. Ideally, I would like to do

 net.Layers(k).Weights = W;

where net is a SeriesNetwork (a class in the Neural Network Toolbox), k is an integer that indexes a fully connected layer, and W is a suitable array. I would then classify inputs with the net modified in this way.

However, the field net.Layers(k).Weights is read-only, so the instruction above will generate the following error message:

 You cannot set the read-only property 'Layers' of SeriesNetwork.

Is there some way to circumvent this restriction?

Here is what I tried, to no avail: First define an array of Layers by saying something like this:

 layers = [imageInputLayer([28 28 1])
    % Severa layers here
                   classificationLayer()];

then initialize the weights as desired, and finally make these layers into a SeriesNetwork with the class’s constructor:

 net = SeriesNetwork(layers);

While this does create a new SeriesNetwork, attempting to classify an input with

 y = classify(net, x);

where x is a suitable input results in the following error message:

 Error using nnet.internal.cnn.layer.FullyConnected/forwardPropagateSize (line 99)
 An input size for the layer must be defined in order to call forwardPropagateSize.

 Error in SeriesNetwork>iDetermineLayerOutputSize (line 417)
     inputSize = layers{i}.forwardPropagateSize(inputSize);

 Error in SeriesNetwork/get.OutputSize (line 80)
             val = iDetermineLayerOutputSize( internalLayers, outputLayerIdx );

 Error in SeriesNetwork/predict (line 185)
             Y = precision.cast( zeros([this.OutputSize dispatcher.NumObservations]) );

 Error in SeriesNetwork/classify (line 250)
             scores = this.predict( X, varargin{:} );

What is missing in my use of the constructor to make net a fully-fledged SeriesNetwork that can be used with classify?

Answer 1

[Carlo Tomasi]

See this Q/A .

Answer 2

[DL]

You can covert the trained NN to an object and it is a possible way, which I had test for it.

modify_able_NN = NN.saveobj;

% Changes made in NN

Modified_NN = NN.loadobj(modify_able_NN);

[qingqing]

Thanks for your comment.It helps a lot!

[Andrea Tagliabue]

The above code does not work for me. Trying to access the saveobj method in a shallow neural network returns the following errors: "Switch expression must be a scalar or a character vector". Any suggestion?

Code to reproduce:

myNN = feedforwardnet(10); 
modifiableNN= myNN.saveobj

SWITCH expression must be a scalar or a character vector.

Error in indexing (line 173)
switch (subs)

Answer 3

[Shashank]

Hi Carlo,

Please see the following image to understand how the weights of various layers are stored in MATLAB.

It is stored as a cell array of matrices and each matrix corresponds to a weight matrix of that layer. It is like a map from 1st to 2nd layer , 2nd to 3rd and so on. Hence only the corresponding indices have weight vectors.

The 1st layer is called the input layer and weights can be set by :

>>net.IW{1} = [1;2;3;4;5;6;7;8;9;10;11;10];

However, please train the network using the following function before modifying the weight vectors:

 >> net = trainscg(net,x,t); % where x and t are the input and output variables respectively.

For the succeeding layers, you can modify the weights using:

>> net.LW{2,1}=rand(128,12);

as shown in the screenshot.Please do not modify the empty vectors

Hope this helps.

- Shashank

[Carlo Tomasi]

Thank you, Shashank. However, your solution works for what Matlab calls ``shallow networks.'' In other words, the following needs to be true:

all(class(net) == 'network')

My question was about series networks, for which the following is true:

all(class(net) == 'SeriesNetwork')

Unfortunately, members of class SeriesNetwork are not structured as you say (they have no .IW or .LW fields, for instance).