Training a dense layer along with an lstm layer

30 views (last 30 days)
I'm trying to set my initial hidden states in the lstm network using a dense layer but, I have been having problems with using multiple inputs in my network. The flow of the network looks something like below. The dense layer between static features (input) and init_hidden_states should be trainable. So far, I tried using a DAG network but, failed due to an error (If there is sequential input layer, there can't be any other input layer present). In my case the sequential input layer is used for time series input and I'm using an image input layer to input the static features. Any advice or example is appreciated. Thank you.

Accepted Answer

Asvin Kumar
Asvin Kumar on 9 Feb 2021
I suppose what you're asking for is to have an LSTM layer in which the initial values of hidden state is trainable. If that's the case, I'm afraid this is not currently possible. The initial value of hidden state of an LSTM is a property of the layer, the 'HiddenState' property, and it is a state parameter which is not learnable.
Refer the following doc page and the linked example for more details: HiddenState - LSTM Layer and LSTM Layer and Properties.
  1 Comment
Yildirim Kocoglu
Yildirim Kocoglu on 9 Feb 2021
Thank you for letting me know Asvin. Yes, that’s what I meant: for initial values of the hidden state to be trainable. I was also looking to find out how the dimensions of the initial hidden state is decided: if I have num_hidden_units=50 in lstm, I end up having initial hidden state with dimensions (50,1). I was wondering where this “1” is from? When does it change to “2” for instance (if it does)? Thank you again.

Sign in to comment.

More Answers (0)

Products


Release

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!