how to combine/append 2 dagnetworks/LayerGraphs: Add layer graph or network to layer graph or network

10 views (last 30 days)
Hi,
I am trying to combine/append 2 complex dagnetworks. One is a downloaded pretrained CNN (googlenet). The other is a complex LSTM ([netLSTM,info] = trainNetwork(sequencesTrain,labelsTrain,lg,options)) which I have entered and trained.
I want to use the output of the CNN as the input to the LSTM, with approprite editing and connecting of the input and output layers.
>>netCNN=googlenet;
>>[netLSTM,info] = trainNetwork(sequencesTrain,labelsTrain,lg,options);
>>cnnLayers = layerGraph(netCNN);
>>lstmLayers = layerGraph(netLSTM);
When I use addLayers I get:
">> netest=addLayers(cnnLayers,lstmLayers)
Error using nnet.internal.cnn.util.validateLayersForLayerGraph (line 22)
Expected input to be one of these types:
nnet.cnn.layer.Layer
Instead its type was nnet.cnn.LayerGraph.
Error in nnet.cnn.LayerGraph>iValidateLayers (line 550)
larray = nnet.internal.cnn.util.validateLayersForLayerGraph(larray, existingLayers);
Error in nnet.cnn.LayerGraph/addLayers (line 184)
larray = iValidateLayers(larray, existingLayers);"
I have tried to use netest=addLayers(cnnLayers,lstmLayers.Layers) but that removes the branching that I have used in the lstmLayers dagnetwork/layerGraph. In addition the training weights and biases of the LSTM do not transfer.
In net2=addLayers(net1, larray) net1 may be a dagnetwork/layerGraph, but larray can only be layers connected sequentially, no branching allowed.
Thank you
  1 Comment
Peter
Peter on 16 Apr 2023
Example in R2022b (error message has changed)
layers = [
imageInputLayer([28 28 1])
convolution2dLayer(5,16,'Padding','same')
batchNormalizationLayer
reluLayer('Name','relu_1')
convolution2dLayer(3,32,'Padding','same','Stride',2)
batchNormalizationLayer
reluLayer
convolution2dLayer(3,32,'Padding','same')
batchNormalizationLayer
reluLayer
additionLayer(2,'Name','add')
averagePooling2dLayer(2,'Stride',2)
fullyConnectedLayer(10)
softmaxLayer
classificationLayer];
lgraph = layerGraph(layers);
skipConv = convolution2dLayer(1,32,'Stride',2,'Name','skipConv');
lgraph = addLayers(lgraph,skipConv);
lgraph = connectLayers(lgraph,'relu_1','skipConv');
lgraph = connectLayers(lgraph,'skipConv','add/in2');
figure
plot(lgraph);
cnn=googlenet;
cnnLayers=layerGraph(cnn);
lg2=addLayers(cnnLayers,lgraph)
Error using nnet.cnn.LayerGraph/addLayers
lg2=addLayers(cnnLayers,lgraph)
Invalid argument at position 2. Input must be an array of layers.

Sign in to comment.

Accepted Answer

Katja Mogalle
Katja Mogalle on 25 Apr 2023
The easiest way to combine complicated networks is to use the app "Deep Network Designer". You can find it in the apps gallery in the MATLAB desktop toolstrip or you just type the following command in the command window: deepNetworkDesigner. In the app, you can import networks from the workspace.
Note that currently, your pretrained CNN expects image data as input. The output of the network is plain features (no more spatial dimensions and no sequence/time dimensions). This might be problematic when you want to use an LSTM network on this output. However, Deep Network Designer let's you make changes to the network structure (e.g. swapping out input layers, deleting layers, and inserting layers) very easily.
I don't have a good understanding of what you're trying to achieve at the end so I can't provide more specific help at this point. But if you're intending to build a network for video processing, you could start by looking at this documentation example: https://uk.mathworks.com/help/deeplearning/ug/classify-videos-using-deep-learning.html

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!