# special case of using parallel computing toolbox in order to solve decomposed problems

2 views (last 30 days)
Bill Masters on 30 Oct 2020
Commented: Walter Roberson on 8 Nov 2020
Hello,
How can I have a for loop with workers of parallel computing doing "1-some functions with different input for each worker, 2-gather the result of iteration from workers 3-doing computation with the result of all workers, 4-update the global input to each worker until stopping criteria and so on?
pseudonym code can be like this:
a=cell(number of workers,1); %first input in cell
b=cell(number of workers,1); %first input in cell
for i=1:maxiter % a loop in the client
tell the workers to find "X" based on some identical function and computations by different inputs (the cells of a and b)
do a global computation to find z with the resut of X gathered form each worker and update the input parameters (a and b)
if fcn(a,b,X,z)<eps
stop
end
end

Walter Roberson on 1 Nov 2020
perhaps use spmd. As each iteration calculates results, labSend to the other workers. You might need to barrier() to synchronize. Depending on how the results are combined it might make sense to select one lab (such as lab 1) to send all the results to and have it combine the results and send out the updated parameters; if you use labSendRecieve on all the labs except the coordinator then they willeeffectively block until they get a response without any barrier(). The tricky part might be ensuring that the coordinator has results from all of the other workers.
Walter Roberson on 8 Nov 2020
%% problem definition (a simple problem to check if the code works)
delete(gcp('nocreate'));
clear;
close all;
clc;
N=4;
c=cell(1,N);
for i=1:4
c{i}=rand(2,1)+0.5;
end
cc=[c{1}; c{2}; c{3}; c{4}];
x0=cell(1,N);
for i=1:4
x0{i}=randn(2,1);
end
x00=[x0{1}; x0{2}; x0{3}; x0{4}];
N=4; %number of workers
A=cell(1,N);
A{1}=[2 3;1 5];
A{2}=[4 3;2 6];
A{3}=[5 2;3 7];
A{4}=[6 1;7 2];
AA=[A{1} A{2} A{3} A{4}];
b=cell(1,N);
x0=cell(1,N);
bb=AA*x00;
for i=1:N
b{i}=bb/N;
end
% Defining a sample problem finishes here
% ----------------------------------------------------------------
%% admm call (the code written to solve generated problem)
rho=1.5;
alpha=1.5;
QUIET = 0;
MAX_ITER = 1000;
ABSTOL = 1e-4;
RELTOL = 1e-2;
%[m n] = size(A);
x=cell(4,1);
for i=1:4
x{i}=zeros(2,1);
end
z=zeros(2,1);
u=cell(N,1);
for i=1:4
u{i}=zeros(2,1);
end
if ~QUIET
fprintf('%3s\t%10s\t%10s\t%10s\t%10s\t%10s\n', 'iter', ...
'r norm', 'eps pri', 's norm', 'eps dual', 'objective');
end
parpool(4);
spmd
cw = c{labindex}; %define input parameters of workers
Aw = A{labindex}; %define input parameters of workers
bw = b{labindex}; %define input parameters of workers
xw = x{labindex}; %define input parameters of workers
uw = u{labindex}; %define input parameters of workers
for k = 1:MAX_ITER
% x-update
z_old = z; % the initial z vector that would be used in ahead lines of code
tmp = [ rho*eye(2), transpose(Aw); Aw, zeros(2) ] \ [ rho*(z - uw) - cw; bw ];
xw = tmp(1:n);
labBarrier;
x_bar = gop(@plus, xw)/4; %x_bar the average of the xw that is calculated for each worker
u_bar = gop(@plus, u)/4; %u_bar the average of the uw that is calculated for each worker
x_hat = alpha*x_bar + (1 - alpha)*z_old; %x_hat variable calculated for each worker
z = subplus(x_bar + u_bar); % variable z must be calculated in each iteration by the results of workers
uw = uw + (x_hat - z); % the new amount of uw, calculated in this iteration usable in the rest of the iterations
labBarrier;
history.objval(k) = gop(@plus, objective(cw, xw));
history.r_norm(k) = norm(x_bar - z);
history.s_norm(k) = norm(-rho*(z - zold));
history.eps_pri(k) = sqrt(2)*ABSTOL + RELTOL*max(norm(x_bar), norm(-z));
history.eps_dual(k)= sqrt(2)*ABSTOL + RELTOL*norm(rho*u_bar);
if ~QUIET
fprintf('%3d\t%10.4f\t%10.4f\t%10.4f\t%10.4f\t%10.2f\n', k, ...
history.r_norm(k), history.eps_pri(k), ...
history.s_norm(k), history.eps_dual(k), history.objval(k));
end
if (history.r_norm(k) < history.eps_pri(k) && ...
history.s_norm(k) < history.eps_dual(k))
break;
end
end
end
%bring composite values into local workspace
%reverse loop order to get pre-allocation effects
%without needing to define structure ahead of time
for i = N : -1 : 1
xhats(i) = x_hat{i};
histories(i) = history{i};
end

Raymond Norris on 30 Oct 2020
Edited: Raymond Norris on 30 Oct 2020
Hi Bill,
There might be a couple of options
1. parfeval*
parfor won't do the trick because it doesn't allow for early exit (i.e. if fcn<eps then stop). parfeval provides the ability to run a collection of 'futures'. Each future could be the same function with different input arguments or different functions entirely. Once the futures are submitted, loop through and query their results. If you've found your threshhold, delete the remaining futures.
I'm putting an * next to parfeval because I don't fully under how you'd update the input parameters to something that's already been submitted, which would could cause issues for parfeval. I think you're suggesting something like:
a=cell(number of workers,1); %first input in cell
b=cell(number of workers,1); %first input in cell
for i=1:maxiter % a loop in the client
X(i) = identical_fcn(a{i},b{i})
z = X ...
% Would updating a and b effect how you're calling identical_fcn at the next iteration?
a{i} = z ... update some element of a, based on z before calling the next iteration of the identical_fcn
b{i} = z ... ditto
if fcn(a,b,X,z)<eps
break
end
end
A bit more code would help refine this.
Raymond
Bill Masters on 3 Nov 2020
Hi Raymond,
I think using SPMD makes sense for my problem. I have written the code using spmd but yet I have errors. I would appreciate If you could check it in the below answer's comment.

### Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!