Reinforcement Learning with Parallel Computing Query
5 views (last 30 days)
Show older comments
Hi All,
I am attempting to get parallel computing enabled when I train my RL agent in R2022a. Forgive the basic question regarding parallel computing,as this is my first attempt. My laptop is compatible with a HVIDIA GeForce RTX 3060. I seem to get an issue where the pool goes idle "IdleTimeout", and I had to "restart" pool on several occasions. I left it to run overnight and again it stalled and stopped training. I am not sure what is happening. Any help would be great. I have included some screen grabs below and the code in my RL script, GPU device properties, and the error showing the pool stopped. I did have episode manager open during the simulation, but the learning did not seem the sae as when I originally ran the training on the CPU. Plus in R2022a I am unable to stop training via episode manager.
Thanks in advance,
Patrick
trainingOpts.UseParallel = true;
trainingOpts.ParallelizationOptions.Mode = 'async';
trainingOpts.ParallelizationOptions.StepsUntilDataIsSent = 32;
trainingOpts.ParallelizationOptions.DataToSendFromWorkers = 'Experiences';
The "IdleTimeout" error is shown below:
Episode Manager:
1 Comment
Joss Knight
on 27 Jul 2022
Can you try increasing your pool IdleTimeout? Maybe this tool is spending a long time computing on the client and meanwhile your pool is idle.
Answers (2)
PB75
on 27 Jul 2022
2 Comments
Joss Knight
on 27 Jul 2022
Please try restarting MATLAB, also try converting your script to .m (using Save As) in case Live Script is causing the issue.
In the meantime, I am asking others for help.
Joss Knight
on 27 Jul 2022
By the way, it seems the Idle Timeout is occurring simply because your code errored during execution and stopped running. The error is displayed in the Live Script. Then the pool times out and the timeout message is displayed in the command window.
PB75
on 27 Jul 2022
3 Comments
Joss Knight
on 10 Sep 2022
I think this is too hard to debug with this little information. There is some sort of error happening during training and probably we would need to inspect the logs to determine what happened. Please open a support case.
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!