rlPGAgent

Create policy gradient reinforcement learning agent

Syntax

agent = rlPGAgent(actor)
agent = rlPGAgent(actor,critic)
agent = rlPGAgent(___,opt)

Description

example

agent = rlPGAgent(actor) creates a PG agent with the specified actor network. By default, the UseBaseline property of the agent is false for this case. For more information on PG agents, see Policy Gradient Agents.

agent = rlPGAgent(actor,critic) creates a PG agent with the specified actor and critic networks. By default, the UseBaseline option is true for this case.

agent = rlPGAgent(___,opt) creates a PG agent using the specified agent options to override the agent defaults.

Examples

collapse all

Create an environment interface.

env = rlPredefinedEnv("DoubleIntegrator-Discrete");
obsInfo = getObservationInfo(env);
numObservations = obsInfo.Dimension(1);
actInfo = getActionInfo(env);
numActions = numel(actInfo);

Create a critic representation to use as a baseline.

baselineNetwork = [
    imageInputLayer([numObservations 1 1], 'Normalization', 'none', 'Name', 'state')
    fullyConnectedLayer(8, 'Name', 'BaselineFC')
    reluLayer('Name', 'CriticRelu1')
    fullyConnectedLayer(1, 'Name', 'BaselineFC2', 'BiasLearnRateFactor', 0)];
baselineOpts = rlRepresentationOptions('LearnRate',5e-3,'GradientThreshold',1);
baseline = rlRepresentation(baselineNetwork,baselineOpts,'Observation',{'state'},obsInfo);

Create an actor representation.

actorNetwork = [
    imageInputLayer([numObservations 1 1], 'Normalization', 'none', 'Name', 'state')
    fullyConnectedLayer(numActions, 'Name', 'action', 'BiasLearnRateFactor', 0)];
actorOpts = rlRepresentationOptions('LearnRate',5e-3,'GradientThreshold',1);
actor = rlRepresentation(actorNetwork,actorOpts,...
    'Observation',{'state'},obsInfo,'Action',{'action'},actInfo);

Specify agent options, and create a PG agent using the environment, actor, and critic.

agentOpts = rlPGAgentOptions(...
    'UseBaseline',true, ...
    'DiscountFactor', 0.99);
agent = rlPGAgent(actor,baseline,agentOpts);

Input Arguments

collapse all

Actor network representation, specified as an rlLayerRepresentation object created using rlRepresentation. For more information on creating actor representations, see Create Policy and Value Function Representations.

Critic network representation, specified as an rlLayerRepresentation object created using rlRepresentation. For more information on creating critic representations, see Create Policy and Value Function Representations.

Agent options, specified as an rlPGAgentOptions object.

Output Arguments

collapse all

PG agent, returned as an rlPGAgent object.

Introduced in R2019a