Reinforcement learning - DDPG - minibatch - Continuos action saturation
7 views (last 30 days)
Show older comments
what is the influence of the minibatch? since in this case its value determines the moment in which the continuous action signal begins to saturate (it only adopts the values of the limits-blue line). In the case in which the value of the minibatch is very large, the weights of the nets do not change over time. Action[-0.15 0.15],
I don't know why before the minibatch which is when it starts from random weights it does not saturate but it is the value of the minibatch that determines where the saturation begins
data:
GradientThreesHold: 1
Learnrate: 0.03
agentOpts.NoiseOptions.StandardDeviation = 0.001;
agentOpts.NoiseOptions.StandardDeviationDecayRate =0.00001;
minibatch: 100 --- x axis=500
0 Comments
Answers (1)
Yash
on 21 Feb 2024
The minibatch size in a neural network training process refers to the number of samples used in each iteration to update the weights of the network. A larger minibatch size can lead to slower weight updates and potentially slower convergence, while a smaller minibatch size can result in faster weight updates but with more noisy updates.
In your case, it seems that a larger minibatch size is causing the action signal to saturate earlier. This could be because with a larger minibatch size, the weight updates are slower, and therefore the network takes longer to adapt to the changing environment. As a result, the action signal reaches its limits ([-0.15, 0.15]) earlier. It may be worth experimenting with different minibatch sizes to find the optimal value for your specific case.
0 Comments
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!