错误使用 rlDetermin​isticActor​Representa​tion。Obser​vation names must match the names of the deep neural network's input layers.

% Create Environment

env = MYEnv();

% Define State and Action Specifications

stateSpec = env.getObservationInfo();

actionSpec = env.getActionInfo();

stateName = stateSpec.Name;

% Create Actor Network

actorNetwork = [

featureInputLayer(stateSpec.Dimension(1), 'Name', stateName)

fullyConnectedLayer(400, 'Name', 'ActorHiddenLayer1')

reluLayer('Name', 'ActorReLU1')

fullyConnectedLayer(300, 'Name', 'ActorHiddenLayer2')

reluLayer('Name', 'ActorReLU2')

fullyConnectedLayer(actionSpec.Dimension(1), 'Name', 'ActorOutputLayer')

tanhLayer('Name', 'ActorTanh')

];

% Create Critic Network

criticNetwork = [

featureInputLayer(stateSpec.Dimension(1), 'Name', stateName)

fullyConnectedLayer(400, 'Name', 'CriticHiddenLayer1')

reluLayer('Name', 'CriticReLU1')

fullyConnectedLayer(300, 'Name', 'CriticHiddenLayer2')

reluLayer('Name', 'CriticReLU2')

fullyConnectedLayer(1, 'Name', 'CriticOutputLayer')

];

% Create Actor Representation

actorOpts = rlRepresentationOptions('LearnRate', actorLearningRate);

actor = rlDeterministicActorRepresentation(actorNetwork, stateSpec, actionSpec, actorOpts);

Answer 1

[Ronit]

Hello,

The error you're encountering is due to a mismatch between the observation names used in your environment's observation specification ‘stateSpec.Name’ and the names of the input layers in your deep neural network. The names must match exactly.

Here’s how you can resolve this issue:

1. Ensure that the ‘Name’ property of the ‘featureInputLayer’ in your actor and critic networks matches the Name property of ‘stateSpec’.
2. Verify that the ‘stateSpec.Name’ is correctly defined and matches the names used in your neural network.
3. Try adding a check to ensure that ‘stateName’ is a string. If ‘stateName’ is a cell array, it extracts the first element. This should ensure that the ‘Name’ property of the ‘featureInputLayer’ matches the Name property of ‘stateSpec’.

% Ensure stateName is a string
if iscell(stateName)
    stateName = stateName{1};
end

Although, ‘rlDeterministicActorRepresentation’ is not recommended anymore since R2022a, you can use ‘rlContinuousDeterministicActor’ instead.

For more details, please refer to the following documentations:

https://www.mathworks.com/help/reinforcement-learning/ref/rl.representation.rldeterministicactorrepresentation.html

https://www.mathworks.com/help/reinforcement-learning/ref/rl.function.rlcontinuousdeterministicactor.html

Hope it helps!

[张]

这些我都检查了，没问题，不知道哪里出了问题。

[Ronit]

Try and use 'rlContinuousDeterministicActor' instead of 'rlDeterministicActorRepresentation'.

[张]

我使用的是2021a版本的MATLAB