rlContinuousDeterministicActor
Deterministic actor with a continuous action space for reinforcement learning agents
Description
This object implements a function approximator to be used as a deterministic actor
within a reinforcement learning agent with a continuous action space. A continuous
deterministic actor takes an environment state as input and returns as output the action that
maximizes the expected discounted cumulative long-term reward, thereby implementing a
deterministic policy. After you create an rlContinuousDeterministicActor
object, use it to create a suitable agent, such as rlDDPGAgent
. For more
information on creating representations, see Create Policies and Value Functions.
Creation
Syntax
Description
creates a continuous deterministic actor object using the deep neural network
actor
= rlContinuousDeterministicActor(net
,observationInfo
,actionInfo
)net
as underlying approximator. For this actor,
actionInfo
must specify a continuous action space. The network
input layers are automatically associated with the environment observation channels
according to the dimension specifications in observationInfo
. The
network must have a single output layer with the same data type and dimensions as the
action specified in actionInfo
. This function sets the
ObservationInfo
and ActionInfo
properties of
actor
to the observationInfo
and
actionInfo
input arguments, respectively.
Note
actor
does not enforce constraints set by the action
specification; therefore, when using this actor, you must enforce action space
constraints within the environment.
specifies the names of the network input layers to be associated with the environment
observation channels. The function assigns, in sequential order, each environment
observation channel specified in actor
= rlContinuousDeterministicActor(net
,observationInfo
,actionInfo
,ObservationInputNames=netObsNames
)observationInfo
to the layer
specified by the corresponding name in the string array
netObsNames
. Therefore, the network input layers, ordered as the
names in netObsNames
, must have the same data type and dimensions
as the observation specifications, as ordered in
observationInfo
.
creates a continuous deterministic actor object using a custom basis function as
underlying approximator. The first input argument is a two-element cell array whose
first element is the handle actor
= rlContinuousDeterministicActor({basisFcn
,W0
},observationInfo
,actionInfo
)basisFcn
to a custom basis function and
whose second element is the initial weight vector W0
. This function
sets the ObservationInfo
and ActionInfo
properties of actor
to the observationInfo
and
actionInfo
input arguments, respectively.
specifies the device used to perform computational operations on the
actor
= rlContinuousDeterministicActor(___,UseDevice=useDevice
)actor
object, and sets the UseDevice
property of actor
to the useDevice
input
argument. You can use this syntax with any of the previous input-argument
combinations.
Input Arguments
Properties
Object Functions
rlDDPGAgent | Deep deterministic policy gradient (DDPG) reinforcement learning agent |
rlTD3Agent | Twin-delayed deep deterministic policy gradient reinforcement learning agent |
getAction | Obtain action from agent, actor, or policy object given environment observations |
evaluate | Evaluate function approximator object given observation (or observation-action) input data |
gradient | Evaluate gradient of function approximator object given observation and action input data |
accelerate | Option to accelerate computation of gradient for approximator object based on neural network |
getLearnableParameters | Obtain learnable parameter values from agent, function approximator, or policy object |
setLearnableParameters | Set learnable parameter values of agent, function approximator, or policy object |
setModel | Set function approximation model for actor or critic |
getModel | Get function approximator model from actor or critic |
Examples
Version History
Introduced in R2022a