Some workers go idle in parfor creating a waste of computation time

Hi all,

I have a for loop of 90 iterations (array xx has 90 rows and say 20 colums - each row is an iteration). Each iteration evaluate a costly function. the evaluation time for some functions is around ~ 24 hours while for some others is around 10 hours.

I am using parfor with a pool of 20 workers. However, I observe that some of the workers go idel after finishing the iteration faster. this can create a large waster of time. How can I request parfor worker to immediately run a new iteration after it finishes its current evlauation?

myCluster = parcluster('local');
myCluster.NumWorkers = 20;
saveProfile(myCluster);
p=parpool(20, 'IdleTimeout', Inf);

opts = parforOptions(p, 'RangePartitionMethod', 'fixed', 'SubrangeSize', 1);
parfor (k=1:size(xx,1),opts)
    out(k,:)=f(xx(k,:), k);
end

Answer 1

[Edric Ellis]

The parfor implementation uses a number of different strategies to try to keep the workers busy. However, sometimes these might turn out not to work well for a particular problem. There is no way to explicitly request the workers start the next iteration - that is all handled automatically by the parfor implementation. I'm a bit surprised that you're seeing workers being idle (for a noticeable amount of time) and then later becoming busy. (Of course, at the end of the loop, there's always going to be a "tail" when each worker has finished its work). I wouldn't normally expect that to happen unless you've got a huge amount of data to transfer (and even then, the parfor implementation tries pretty hard to hide data transfers by running them concurrently with other stuff).

One alternative you might explore is to use parfeval rather than parfor. This uses a simpler scheduling mechanism than parfor, but it might work better for your situation. Briefly, each call to parfeval schedules a single remote function evaluation on a worker. You call fetchNext or fetchOutputs to collect the results. A bit like this:

% schedule work
for k = 1:size(xx,1)
    f(k) = parfeval(f, 1, xx(k,:), k); % 1 is the number of outputs you're requesting
end
% Collect results
for idx = 1:size(xx, 1)
    [k, data] = fetchNext(f);
    out(k,:) = data;
end

There's an introduction to parfeval here in the doc.