rlSimulationOptions
Options for simulating a reinforcement learning agent within an environment
Description
Use an rlSimulationOptions
object to specify simulation options for simulating a reinforcement learning agent within an
environment. To perform the simulation, use sim
.
For more information on agents training and simulation, see Train Reinforcement Learning Agents.
Creation
Description
returns the
default options for simulating a reinforcement learning environment against an agent. Use
simulation options to specify parameters about the simulation such as the maximum number
of steps to run per simulation and the number of simulations to run. After configuring the
options, use simOpts
= rlSimulationOptionssimOpts
as an input argument for sim
.
creates a simulation options set with the specified properties using one or more
name-value pair arguments.opt
= rlSimulationOptions(Name=Value
)
Properties
MaxSteps
— Number of steps to run the simulation
500 (default) | positive integer
Number of steps to run the simulation, specified as the comma-separated pair
consisting of 'MaxSteps'
and a positive integer. In general, you
define episode termination conditions in the environment. This value is the maximum
number of steps to run in the simulation if those termination conditions are not
met.
Example: MaxSteps=1000
NumSimulations
— Number of simulations
1 (default) | positive integer
Number of simulations to run, specified as the comma-separated pair consisting of
'NumSimulations'
and a positive integer. At the start of each
simulation, sim
resets the environment. You specify what happens on
environment reset when you create the environment. For instance, resetting the
environment at the start of each episode can include randomizing initial state values,
if you configure your environment to do so. In that case, running multiple simulations
allows you to validate performance of a trained agent over a range of initial
conditions.
Example: NumSimulations=10
StopOnError
— Stop simulation when error occurs
"on"
(default) | "off"
Stop simulation when an error occurs, specified as "off"
or
"on"
. When this option is "off"
, errors are
captured and returned in the SimulationInfo
output of
sim
, and simulation continues.
SimulationStorageType
— Storage type for environment data
"memory"
(default) | "file"
| "none"
Storage type for environment data, specified as "memory"
,
"file"
, or "none"
. This option specifies the
type of storage used for data generated during training or simulation by a Simulink® environment. Specifically, the software saves anything that appears as the
output of a sim
(Simulink) command.
Note that this option does not affect (and is not affected by) any option to save
agents during training specified within a training option object, or any data logged by
a FileLogger
or
MonitorLogger
object.
The default value is "memory"
, indicating that data is stored in an
internal memory variable. When you set this option to "file"
, data is
stored to disk, in MAT-files in the directory specified by the
SaveSimulationDirectory
property, and using the MAT-file
version specified by the SaveFileVersion
property. When you set
this option to "none"
, simulation data is not stored.
You can use this option to prevent out-of-memory issues during training or simulation.
Example: "none"
SaveSimulationDirectory
— Folder used to save environment data
"savedSims"
(default) | string | character vector
Folder used to save environment data, specified as a string or character vector. The
folder name can contain a full or relative path. When you set the
SimulationStorageType
property to "file"
,
the software saves data generated during training or simulation by a Simulink environment in MAT-files in this folder, using the MAT-file version
specified by the SaveFileVersion
property. If the folder does not
exist, the software creates it.
Example: "envSimData"
SaveFileVersion
— MAT-file version used to save environment data
"-v7"
(default) | "-v7.3"
| "-v6"
MAT-file version used to save environment data, specified as a string or character
vector. When you set the SimulationStorageType
property to
"file"
, the software saves data generated by a Simulink environment in MAT-files in the version specified by
SaveFileVersion
, in the folder specified by the
SaveSimulationDirectory
property. For more information, see
MAT-File Versions.
Example: Version="-v7.3"
UseParallel
— Flag for using parallel simulation
false
(default) | true
Flag for using parallel simulation, specified as a logical
.
Setting this option to true
configures the simulation to use parallel
processing to simulate the environment, thereby enabling usage of multiple cores,
processors, computer clusters or cloud resources to speed up simulation. To specify
options for parallel simulation, use the ParallelizationOptions
property.
Note that if you want to speed up deep neural network calculations (such as gradient
computation, parameter update and prediction) using a local GPU you do not need to set
UseParallel
to true. Instead, when creating your actor or critic
representation, use an rlRepresentationOptions
object in which the UseDevice
option is set to "gpu"
.
Using parallel computing or the GPU requires Parallel Computing Toolbox™ software. Using computer clusters or cloud resources additionally requires MATLAB® Parallel Server™.
For more information about training using multicore processors and GPUs, see Train Agents Using Parallel Computing and GPUs.
Example: UseParallel=true
ParallelizationOptions
— Options to control parallel simulation
ParallelSimulation
object
Parallelization options to control parallel simulation, specified as a
ParallelSimulation
object. For more information about training using
parallel computing, see Train Reinforcement Learning Agents.
The ParallelSimulation
object has the following properties, which
you can modify using dot notation after creating the
rlSimulationOptions
object.
WorkerRandomSeeds
— Randomizer initialization for workers
–1
(default) | –2
| vector
Randomizer initialization for workers, specified as one the following:
–1
— Assign a unique random seed to each worker. The value of the seed is the worker ID.–2
— Do not assign a random seed to the workers.Vector — Manually specify the random seed for each work. The number of elements in the vector must match the number of workers.
TransferBaseWorkspaceVariables
— Send model and workspace variables to parallel workers
"on"
(default) | "off"
Send model and workspace variables to parallel workers, specified as
"on"
or "off"
. When the option is
"on"
, the host sends the variables used in models and defined
in the base MATLAB workspace to the workers.
AttachedFiles
— Additional files to attach to the parallel pool
[]
(default) | string | string array
Additional files to attach to the parallel pool, specified as a string or string array.
SetupFcn
— Function to run before simulation starts
[]
(default) | function handle
Function to run on each worker before simulation starts, specified as a handle to a function having no input arguments. This function is run once per worker before simulation begins. Write this function to perform any processing that you need prior to simulation.
CleanupFcn
— Function to run after simulation ends
[]
(default) | function handle
Function to run on each worker after simulation ends, specified as a handle to a function having no input arguments. You can write this function to clean up the workspace or perform other processing after simulation terminates.
Object Functions
sim | Simulate trained reinforcement learning agents within specified environment |
Examples
Configure Options for Simulation
Create an options set for simulating a reinforcement learning environment. Set the number of steps to simulate to 1000, and configure the options to run three simulations.
You can set the options using Name,Value pairs when you create the options set. Any options that you do not explicitly set have their default values.
simOpts = rlSimulationOptions(... MaxSteps=1000,... NumSimulations=3)
simOpts = rlSimulationOptions with properties: MaxSteps: 1000 NumSimulations: 3 StopOnError: "on" SimulationStorageType: "memory" SaveSimulationDirectory: "savedSims" SaveFileVersion: "-v7" UseParallel: 0 ParallelizationOptions: [1x1 rl.option.ParallelSimulation]
Alternatively, create a default options set and use dot notation to change some of the values.
simOpts = rlSimulationOptions; simOpts.MaxSteps = 1000; simOpts.NumSimulations = 3; simOpts
simOpts = rlSimulationOptions with properties: MaxSteps: 1000 NumSimulations: 3 StopOnError: "on" SimulationStorageType: "memory" SaveSimulationDirectory: "savedSims" SaveFileVersion: "-v7" UseParallel: 0 ParallelizationOptions: [1x1 rl.option.ParallelSimulation]
Version History
Introduced in R2019a
See Also
Functions
Objects
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)