Valid indices are restricted in parfor loops
21 views (last 30 days)
Show older comments
Can anybody help me to understand what is wrong in the following code? Everything works until I add lines 24-27 with the for loop, where I get the "valid indices are restricted in parfor loops" for state and action variables. I have a lot about classification variables in parfor loops but I still haven't understood where I am wrong.
%% SARSA PARFOR TEST
alpha = 0.1;
epsilon = 0.05;
states = 1:1:100;
ls = length(states);
la = length(actions);
actions = [0 1 2];
num_agents = 10;
Q = zeros(ls,la,num_agents);
state = zeros(time_steps,num_agents);
action = zeros(time_steps,num_agents);
equalmax = zeros(time_steps,num_agents);
idx = zeros(time_steps,num_agents);
time_steps = 1000;
parfor k = 1:num_agents
state(1,k) = states(randi(ls),1);
if rand < epsilon
action(1,k) = actions(1,randi(la));
else
[maxq(1,k),action(1,k)] = max(Q(state(1,k),:,k));
equalmax(1,k) = find(Q(state(1,k),:,k) == maxq(1,k));
if numel(equalmax(1,k)) > 1
idx(1,k) = randi(numel(equalmax(1,k)));
action(1,k) = idx(1,k) -1;
end
end
for t = 2:time_steps
reward(t,k) = state(t-1,k)*action(t-1,k);
end
end
0 Comments
Accepted Answer
Matt J
on 18 Jun 2019
Edited: Matt J
on 18 Jun 2019
You need to get familiar with Sliced Variables and their restrictions. Basically, if you have an expression like state(expr,k) involving a sliced variable, then expr must remain the same if you index the same variable later in the loop. You violate this where you have state(1,k) and then later state(t-1,k). Moreover, t-1 is not a valid choice for expr. You cannot have complex expressions in the index that doesn't involve the loop variable. It has to be something like state(t,k).
Here is one way to get rid of the errors:
parfor k = 1:num_agents
for t = 1:time_steps-1
if t==1
state(t,k) = states(randi(ls),1);
if rand < epsilon
action(t,k) = actions(1,randi(la)); %#ok<*PFBNS>
else
[maxq(1,k),action(t,k)] = max(Q(state(t,k),:,k));
equalmax(1,k) = find(Q(state(t,k),:,k) == maxq(1,k));
if numel(equalmax(1,k)) > 1
idx(1,k) = randi(numel(equalmax(1,k)));
action(t,k) = idx(1,k) -1;
end
end
end
Reward_tmp(t,k) = state(t,k)*action(t,k);
end
end
reward(2:end,:)=Reward_tmp;
3 Comments
Matt J
on 4 Sep 2020
idx=sub2ind(size(Q), Q_IN(:,1),Q_IN(:,2),Q_IN(:,3) );
XCell=Q(idx);
IDentCell=IDent(idx);
parfor k=1:ZsplitNo*YsplitNo*XsplitNo
X=XCell{k};
partition1=1;
[Ident1] = Ex_PartitionScanner(partition1,X);
IDentCell{k}=Ident1;
p=p+1;
end
IDent(idx)=IDentCell;
More Answers (0)
See Also
Categories
Find more on Loops and Conditional Statements in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!