# special case of using parallel computing toolbox in order to solve decomposed problems

1 view (last 30 days)

Show older comments

Bill Masters
on 30 Oct 2020

Commented: Walter Roberson
on 8 Nov 2020

Hello,

How can I have a for loop with workers of parallel computing doing "1-some functions with different input for each worker, 2-gather the result of iteration from workers 3-doing computation with the result of all workers, 4-update the global input to each worker until stopping criteria and so on?

pseudonym code can be like this:

a=cell(number of workers,1); %first input in cell

b=cell(number of workers,1); %first input in cell

for i=1:maxiter % a loop in the client

tell the workers to find "X" based on some identical function and computations by different inputs (the cells of a and b)

do a global computation to find z with the resut of X gathered form each worker and update the input parameters (a and b)

if fcn(a,b,X,z)<eps

stop

end

end

##### 0 Comments

### Accepted Answer

Walter Roberson
on 1 Nov 2020

##### 5 Comments

Walter Roberson
on 8 Nov 2020

%% problem definition (a simple problem to check if the code works)

delete(gcp('nocreate'));

clear;

close all;

clc;

N=4;

c=cell(1,N);

for i=1:4

c{i}=rand(2,1)+0.5;

end

cc=[c{1}; c{2}; c{3}; c{4}];

x0=cell(1,N);

for i=1:4

x0{i}=randn(2,1);

end

x00=[x0{1}; x0{2}; x0{3}; x0{4}];

N=4; %number of workers

A=cell(1,N);

A{1}=[2 3;1 5];

A{2}=[4 3;2 6];

A{3}=[5 2;3 7];

A{4}=[6 1;7 2];

AA=[A{1} A{2} A{3} A{4}];

b=cell(1,N);

x0=cell(1,N);

bb=AA*x00;

for i=1:N

b{i}=bb/N;

end

% Defining a sample problem finishes here

% ----------------------------------------------------------------

%% admm call (the code written to solve generated problem)

rho=1.5;

alpha=1.5;

%% PADMM

QUIET = 0;

MAX_ITER = 1000;

ABSTOL = 1e-4;

RELTOL = 1e-2;

%[m n] = size(A);

x=cell(4,1);

for i=1:4

x{i}=zeros(2,1);

end

z=zeros(2,1);

u=cell(N,1);

for i=1:4

u{i}=zeros(2,1);

end

if ~QUIET

fprintf('%3s\t%10s\t%10s\t%10s\t%10s\t%10s\n', 'iter', ...

'r norm', 'eps pri', 's norm', 'eps dual', 'objective');

end

parpool(4);

spmd

cw = c{labindex}; %define input parameters of workers

Aw = A{labindex}; %define input parameters of workers

bw = b{labindex}; %define input parameters of workers

xw = x{labindex}; %define input parameters of workers

uw = u{labindex}; %define input parameters of workers

for k = 1:MAX_ITER

% x-update

z_old = z; % the initial z vector that would be used in ahead lines of code

tmp = [ rho*eye(2), transpose(Aw); Aw, zeros(2) ] \ [ rho*(z - uw) - cw; bw ];

xw = tmp(1:n);

labBarrier;

x_bar = gop(@plus, xw)/4; %x_bar the average of the xw that is calculated for each worker

u_bar = gop(@plus, u)/4; %u_bar the average of the uw that is calculated for each worker

x_hat = alpha*x_bar + (1 - alpha)*z_old; %x_hat variable calculated for each worker

z = subplus(x_bar + u_bar); % variable z must be calculated in each iteration by the results of workers

uw = uw + (x_hat - z); % the new amount of uw, calculated in this iteration usable in the rest of the iterations

labBarrier;

history.objval(k) = gop(@plus, objective(cw, xw));

history.r_norm(k) = norm(x_bar - z);

history.s_norm(k) = norm(-rho*(z - zold));

history.eps_pri(k) = sqrt(2)*ABSTOL + RELTOL*max(norm(x_bar), norm(-z));

history.eps_dual(k)= sqrt(2)*ABSTOL + RELTOL*norm(rho*u_bar);

if ~QUIET

fprintf('%3d\t%10.4f\t%10.4f\t%10.4f\t%10.4f\t%10.2f\n', k, ...

history.r_norm(k), history.eps_pri(k), ...

history.s_norm(k), history.eps_dual(k), history.objval(k));

end

if (history.r_norm(k) < history.eps_pri(k) && ...

history.s_norm(k) < history.eps_dual(k))

break;

end

end

end

%bring composite values into local workspace

%reverse loop order to get pre-allocation effects

%without needing to define structure ahead of time

for i = N : -1 : 1

xhats(i) = x_hat{i};

histories(i) = history{i};

end

### More Answers (1)

Raymond Norris
on 30 Oct 2020

Edited: Raymond Norris
on 30 Oct 2020

Hi Bill,

There might be a couple of options

- parfeval*
- createJob/createTask

parfor won't do the trick because it doesn't allow for early exit (i.e. if fcn<eps then stop). parfeval provides the ability to run a collection of 'futures'. Each future could be the same function with different input arguments or different functions entirely. Once the futures are submitted, loop through and query their results. If you've found your threshhold, delete the remaining futures.

I'm putting an * next to parfeval because I don't fully under how you'd update the input parameters to something that's already been submitted, which would could cause issues for parfeval. I think you're suggesting something like:

a=cell(number of workers,1); %first input in cell

b=cell(number of workers,1); %first input in cell

for i=1:maxiter % a loop in the client

X(i) = identical_fcn(a{i},b{i})

z = X ...

% Would updating a and b effect how you're calling identical_fcn at the next iteration?

a{i} = z ... update some element of a, based on z before calling the next iteration of the identical_fcn

b{i} = z ... ditto

if fcn(a,b,X,z)<eps

break

end

end

A bit more code would help refine this.

Raymond

##### 4 Comments

Raymond Norris
on 1 Nov 2020

Hi Bill,

So I don't make any assumptions, a couple of suggestions/thoughts

- I think you're missing an end for the for loop (before the definition of the tmp function)
- the tmp function should be returning (I assume f) not tmp
- the tmp function doesn't use the input argument i
- the call to tmp in the parfeval only passes in MAX_ITER (which maps to i). Nowhere are any of the variables rho, etc. defined. Even if they are global they won't be passed to each of the workers (for starters, you’d need to declare them as global in tmp as well, but this still won’t work).
- parfeval and fetchNext are (typically) called from within different for loops, because fetchNext is blocking until MATLAB fetches the results. This, along with updating z_old, x_hat, and u, all indicate that you'll have issues parallelizing this. This is a cascading waterfall algorithm, u(k) is dependent on u(k-1).

Based on #5, perhaps the best way to think of parallezing this is if you need to run this entire block of code multiple times. Then each worker can be given the entire code you have listed above and you can run 4 of these (for example). They wouldn't be tied together, but you could run multiple sets, all at the same time.

But perhaps I've misconstrued some things here.

Raymond

### See Also

### Categories

### Products

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!