How to input global variables in function that is in parloop?

4 views (last 30 days)
Hi.
I want to call some data ( = matrix) in function in parloop.
First time, I used global variable, but it is not worked in parloop.
How can I implement the code below without global variable?
The size of 'data1' is so large, so individually calling the data in function takes so much computing resources.
load('...\data1')
data1 = data1;
global data1
x = nchoosek([1:10], 2);
parfor i = 1:10
y(i) = myfunc(x(i, :));
end
% Function
function y = myfunc(x)
global data1
y = data1(x);
end

Accepted Answer

Raymond Norris
Raymond Norris on 4 May 2022
We can agree that if global variables were supported, you wouldn't modify data1 and have that reflected on the client -- that would violate being task independent.
Therefore, you ought to create/load the data on the worker. To avoid loading it each time, you could make the variable persistent, such as:
x = nchoosek(1:10, 2);
parfor i = 1:10
y(i) = myfunc(x(i, :));
end
% Function
function y = myfunc(x)
persistent data1
if isempty(data1)
data1 = load("data1");
end
y = data1(x);
end
Of course, the caveat is that wherever the workers are running, they need access to data1.mat If you think this workflow would work, I'm guessing if it's large, this is probably running on a cluster, correct? I can sketch out an example of how the workers can find the data on a cluster (it would be a requirement that MAT-file is on the cluster).
  1 Comment
Jungeon Kim
Jungeon Kim on 4 May 2022
Edited: Jungeon Kim on 4 May 2022
Thanks a lot!
It works well!
For answer your question, 'no'. I'm using workstation, so single computer is running.
Thank you so much for the consideration.

Sign in to comment.

More Answers (1)

Edric Ellis
Edric Ellis on 5 May 2022
Another option is to use parallel.pool.Constant to arrange for the data to be loaded and then accessed. This can be a bit simpler than writing your own function using persistent data (behind the scenes it is basically the same thing though). Using the "function handle" constructor avoids transferring the data, and gets each worker to call load for itself. Here's a simple example:
parpool("local");
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 2).
% This instructs each worker to load variable X from the file durer.mat and
% retain the value.
c = parallel.pool.Constant(@() load("durer.mat", "X"));
parfor i = 1:100
% Here's how you access the value - use "c.Value". The return from the
% "load" function is a struct, so we need to get the field "X" out of
% that struct
data = c.Value.X;
% Now we can operate on the data.
y(i) = sum(data(:,i));
end

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!