Sequence to Sequence Classification with Deep Learning CNN+LSTM

I was looking through the possible implementation of sequence classification using deep-learning.

There are pllenty of example of LSTM/BILSTM implementations

A) https://it.mathworks.com/help/deeplearning/ug/sequence-to-sequence-classification-using-deep-learning.html

and 1D-Convolutional implementations of the problem.

B) https://it.mathworks.com/help/deeplearning/ug/sequence-to-sequence-classification-using-1-d-convolutions.html

My question is there is a way to combine the two solutions?

If for the first one the building of the net seems pretty immediate by stacking series of custom layers:

layers = [ ...

sequenceInputLayer(numFeatures)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(numClasses)

softmaxLayer

classificationLayer];

The convolution implementation seems indeed more complex, as it directly defines the various computational blocks.

Can i use a pre-defined convolution2Dlayer in the layers structure like in A) or do i have to go deeply in coding as described in B)?

Answer 1

[Srivardhan Gadila]

I think you can use the convolution2Dlayer with appropriate input arguments but make sure you use the sequenceFoldingLayer, sequenceUnfoldingLayer wherever necessary. Also refer to List of Deep Learning Layers.

[Mirko Job]

Thanks for the early response,

It indeed came with good news since i am actually trying to solve the problem using custom loop and dlarrays with not satisfying results. However it is not clear for me the need for sequenceFolding/UnfoldingLayer since i am working on accelerometry data and not images. As a first rude approach, starting from the convolutional block described in:

https://it.mathworks.com/help/deeplearning/ug/sequence-to-sequence-classification-using-1-d-convolutions.html

I would concatenate the convolutional2DLayer just after the sequenceInputLayer. Is there any implicit step that i lost in the workflow?

[Srivardhan Gadila]

Refer to the following MATLAB Answer: CNN code and Sequence Input Error