Convolutional LSTM (C-LSTM) in MATLAB

120 views (last 30 days)
Jake
Jake on 9 Oct 2018
Edited: Dieter Mayer on 29 Aug 2022
I'd like to train a convolutional neural network with an LSTM layer on the end of it. Similar to what was done in:
  1. https://arxiv.org/pdf/1710.03804.pdf
  2. https://arxiv.org/pdf/1612.01079.pdf
Is this possible?

Answers (5)

Shounak Mitra
Shounak Mitra on 9 Oct 2018
Hi Jake,
Unfortunately, we do not directly support C-LSTM. We are working on it and it should be available soon.
-- Shounak
  7 Comments
Dieter Mayer
Dieter Mayer on 29 Aug 2022
Edited: Dieter Mayer on 29 Aug 2022
Hi David,
Thanks for your reply! Is this workflow shows a real convolution LSTM (LSTM carries out convolutional operations instead of matrix multiplication) and is not only implied to a input matrix, which is a result of a convolution net work applied before?
Sorry for asking that, I have to learn the syntax of using the deep learning toolbox, I am a beginner. The background is, that I will use such a Conv-LSTM to make precipitation forecasts for grids bases on precipitation radar inputs from several timesteps of the last minutes / hours as discussed in this paper publication

Sign in to comment.


Yi Wei
Yi Wei on 17 Dec 2019
Hi, can matlab support C-LSTM now?
  5 Comments
Ioana Cretu
Ioana Cretu on 18 May 2021
Hi! When I try to train the model I have this error:
Error using trainNetwork (line 170)
Invalid network.
Caused by:
Layer 'fold': Unconnected output. Each layer output must be connected to the input of another layer.
Detected unconnected outputs:
output 'miniBatchSize'
Layer 'unfold': Unconnected input. Each layer input must be connected to the output of another layer.
I connected the layers using this:
lgraph = layerGraph(Layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
What do you think the cause is?

Sign in to comment.


Chen
Chen on 25 Aug 2021
Please refer to this excellent example in:
It is possible to train the hybrid together.

Jonathan
Jonathan on 4 Aug 2022
inputSize = [28 28 1];
filterSize = 5;
numFilters = 20;
numHiddenUnits = 200;
numClasses = 10;
layers = [ ...
sequenceInputLayer(inputSize,'Name','input')
sequenceFoldingLayer('Name','fold')
convolution2dLayer(filterSize,numFilters,'Name','conv')
batchNormalizationLayer('Name','bn')
reluLayer('Name','relu')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm')
fullyConnectedLayer(numClasses, 'Name','fc')
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');

David Willingham
David Willingham on 26 Aug 2022
Updating this answer. This workflow has been supported since R2021. The following example illustrates how to combin CNN's with LSTM layers:
% Load data
[XTrain,YTrain] = japaneseVowelsTrainData;
% Define layers
layers = [ sequenceInputLayer(12,'Normalization','none', 'MinLength', 9);
convolution1dLayer(3, 16)
batchNormalizationLayer()
reluLayer()
maxPooling1dLayer(2)
convolution1dLayer(5, 32)
batchNormalizationLayer()
reluLayer()
averagePooling1dLayer(2)
lstmLayer(100, 'OutputMode', 'last')
fullyConnectedLayer(9)
softmaxLayer()
classificationLayer()];
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize',27, ...
'SequenceLength','longest');
% Train network
net = trainNetwork(XTrain,YTrain,layers,options);

Categories

Find more on Deep Learning with Time Series and Sequence Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!