Clear Filters
Clear Filters

special case of using parallel computing toolbox in order to solve decomposed problems

1 view (last 30 days)
How can I have a for loop with workers of parallel computing doing "1-some functions with different input for each worker, 2-gather the result of iteration from workers 3-doing computation with the result of all workers, 4-update the global input to each worker until stopping criteria and so on?
pseudonym code can be like this:
a=cell(number of workers,1); %first input in cell
b=cell(number of workers,1); %first input in cell
for i=1:maxiter % a loop in the client
tell the workers to find "X" based on some identical function and computations by different inputs (the cells of a and b)
do a global computation to find z with the resut of X gathered form each worker and update the input parameters (a and b)
if fcn(a,b,X,z)<eps

Accepted Answer

Walter Roberson
Walter Roberson on 1 Nov 2020
perhaps use spmd. As each iteration calculates results, labSend to the other workers. You might need to barrier() to synchronize. Depending on how the results are combined it might make sense to select one lab (such as lab 1) to send all the results to and have it combine the results and send out the updated parameters; if you use labSendRecieve on all the labs except the coordinator then they willeeffectively block until they get a response without any barrier(). The tricky part might be ensuring that the coordinator has results from all of the other workers.
Bill Masters
Bill Masters on 7 Nov 2020
My purpose of do this is to assign the workers their own initial vector and then start the for loop seperately. in other words, I want the for loop to run in each worker with their own inputs. If I don't write the code like this, how can each worker call their own inputs?
Walter Roberson
Walter Roberson on 8 Nov 2020
%% problem definition (a simple problem to check if the code works)
close all;
for i=1:4
cc=[c{1}; c{2}; c{3}; c{4}];
for i=1:4
x00=[x0{1}; x0{2}; x0{3}; x0{4}];
N=4; %number of workers
A{1}=[2 3;1 5];
A{2}=[4 3;2 6];
A{3}=[5 2;3 7];
A{4}=[6 1;7 2];
AA=[A{1} A{2} A{3} A{4}];
for i=1:N
% Defining a sample problem finishes here
% ----------------------------------------------------------------
%% admm call (the code written to solve generated problem)
QUIET = 0;
MAX_ITER = 1000;
ABSTOL = 1e-4;
RELTOL = 1e-2;
%[m n] = size(A);
for i=1:4
for i=1:4
fprintf('%3s\t%10s\t%10s\t%10s\t%10s\t%10s\n', 'iter', ...
'r norm', 'eps pri', 's norm', 'eps dual', 'objective');
cw = c{labindex}; %define input parameters of workers
Aw = A{labindex}; %define input parameters of workers
bw = b{labindex}; %define input parameters of workers
xw = x{labindex}; %define input parameters of workers
uw = u{labindex}; %define input parameters of workers
for k = 1:MAX_ITER
% x-update
z_old = z; % the initial z vector that would be used in ahead lines of code
tmp = [ rho*eye(2), transpose(Aw); Aw, zeros(2) ] \ [ rho*(z - uw) - cw; bw ];
xw = tmp(1:n);
x_bar = gop(@plus, xw)/4; %x_bar the average of the xw that is calculated for each worker
u_bar = gop(@plus, u)/4; %u_bar the average of the uw that is calculated for each worker
x_hat = alpha*x_bar + (1 - alpha)*z_old; %x_hat variable calculated for each worker
z = subplus(x_bar + u_bar); % variable z must be calculated in each iteration by the results of workers
uw = uw + (x_hat - z); % the new amount of uw, calculated in this iteration usable in the rest of the iterations
history.objval(k) = gop(@plus, objective(cw, xw));
history.r_norm(k) = norm(x_bar - z);
history.s_norm(k) = norm(-rho*(z - zold));
history.eps_pri(k) = sqrt(2)*ABSTOL + RELTOL*max(norm(x_bar), norm(-z));
history.eps_dual(k)= sqrt(2)*ABSTOL + RELTOL*norm(rho*u_bar);
fprintf('%3d\t%10.4f\t%10.4f\t%10.4f\t%10.4f\t%10.2f\n', k, ...
history.r_norm(k), history.eps_pri(k), ...
history.s_norm(k), history.eps_dual(k), history.objval(k));
if (history.r_norm(k) < history.eps_pri(k) && ...
history.s_norm(k) < history.eps_dual(k))
%bring composite values into local workspace
%reverse loop order to get pre-allocation effects
%without needing to define structure ahead of time
for i = N : -1 : 1
xhats(i) = x_hat{i};
histories(i) = history{i};

Sign in to comment.

More Answers (1)

Raymond Norris
Raymond Norris on 30 Oct 2020
Edited: Raymond Norris on 30 Oct 2020
Hi Bill,
There might be a couple of options
  1. parfeval*
  2. createJob/createTask
parfor won't do the trick because it doesn't allow for early exit (i.e. if fcn<eps then stop). parfeval provides the ability to run a collection of 'futures'. Each future could be the same function with different input arguments or different functions entirely. Once the futures are submitted, loop through and query their results. If you've found your threshhold, delete the remaining futures.
I'm putting an * next to parfeval because I don't fully under how you'd update the input parameters to something that's already been submitted, which would could cause issues for parfeval. I think you're suggesting something like:
a=cell(number of workers,1); %first input in cell
b=cell(number of workers,1); %first input in cell
for i=1:maxiter % a loop in the client
X(i) = identical_fcn(a{i},b{i})
z = X ...
% Would updating a and b effect how you're calling identical_fcn at the next iteration?
a{i} = z ... update some element of a, based on z before calling the next iteration of the identical_fcn
b{i} = z ... ditto
if fcn(a,b,X,z)<eps
A bit more code would help refine this.
Raymond Norris
Raymond Norris on 1 Nov 2020
Hi Bill,
So I don't make any assumptions, a couple of suggestions/thoughts
  1. I think you're missing an end for the for loop (before the definition of the tmp function)
  2. the tmp function should be returning (I assume f) not tmp
  3. the tmp function doesn't use the input argument i
  4. the call to tmp in the parfeval only passes in MAX_ITER (which maps to i). Nowhere are any of the variables rho, etc. defined. Even if they are global they won't be passed to each of the workers (for starters, you’d need to declare them as global in tmp as well, but this still won’t work).
  5. parfeval and fetchNext are (typically) called from within different for loops, because fetchNext is blocking until MATLAB fetches the results. This, along with updating z_old, x_hat, and u, all indicate that you'll have issues parallelizing this. This is a cascading waterfall algorithm, u(k) is dependent on u(k-1).
Based on #5, perhaps the best way to think of parallezing this is if you need to run this entire block of code multiple times. Then each worker can be given the entire code you have listed above and you can run 4 of these (for example). They wouldn't be tied together, but you could run multiple sets, all at the same time.
But perhaps I've misconstrued some things here.
Bill Masters
Bill Masters on 3 Nov 2020
Hi Raymond,
thanks for your suggestions
I think using SPMD makes sense for my problem. I have written the code using spmd but yet I have errors. I would appreciate If you could check it in the below answer's comment.

Sign in to comment.


Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!