Deep Reinforcement Learning Toolbox with LSTM RNN
10 views (last 30 days)
Show older comments
Hi,
Currently I am using a LSTM RNN network in my TD3 agent model.
I am using a LSTM network architecture based on the DDPG TD3 the online help, as shown below
%Critic Network
rng(0) % fix the random seed
statePath1 = [
sequenceInputLayer(numObservations,'Normalization','none','Name','observation')
fullyConnectedLayer(400,'Name','CriticStateFC1')
reluLayer('Name','CriticStateRelu1')
fullyConnectedLayer(300,'Name','CriticStateFC2')
];
actionPath1 = [
sequenceInputLayer(numActions,'Normalization','none','Name','action')
fullyConnectedLayer(300,'Name','CriticActionFC1')
];
commonPath1 = [
additionLayer(2,'Name','add')
reluLayer('Name','CriticCommonRelu1')
lstmLayer(16,'OutputMode','sequence','Name','CriticLSTM');
fullyConnectedLayer(1,'Name','CriticOutput')
];
criticNet = layerGraph(statePath1);
criticNet = addLayers(criticNet,actionPath1);
criticNet = addLayers(criticNet,commonPath1);
criticNet = connectLayers(criticNet,'CriticStateFC2','add/in1');
criticNet = connectLayers(criticNet,'CriticActionFC1','add/in2');
%Actor Network
actorNet = [
sequenceInputLayer(numObservations,'Normalization','none','Name','observation')
fullyConnectedLayer(400,'Name','ActorFC1')
lstmLayer(8,'OutputMode','sequence','Name','ActorLSTM')
reluLayer('Name','ActorRelu1')
fullyConnectedLayer(300,'Name','ActorFC2')
reluLayer('Name','ActorRelu2')
fullyConnectedLayer(numActions,'Name','ActorFC3')
tanhLayer('Name','ActorTanh1')
];
My Mini Batch size is set to 200.
When my training begins the model runs very quickly, during the initial episodes, which is the same as the mini batch size of 200, then from episode 201 onward the model runs at a normal training pace and seems to start learning, this can be seen in the episode manager plot below.
My question is:
Is this initial training behavour normal for a LSTM RNN?
Cheers
Patrick
0 Comments
Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!