Parfor loop: how to keep the temporary variables ?

41 views (last 30 days)
V.D-C
V.D-C on 28 Apr 2020
Commented: V.D-C on 29 Apr 2020
Hello,
I am working with a time-consuming code and I want to put in parallel computing the independant functions I use.
I have the following script (only a fraction of it):
parfor i = 1:2
if i == 1
[T20cm_land, albedo_land, smb_land, swe_land, T1m_land, T0_land, abl_land,...
acc_land, runoff_land] = display_output_land(point_output_path_land, yr, mm, dd, date);
else
[T20cm_glacier, albedo_glacier, smb_glacier, swe_glacier, T1m_glacier, T0_glacier,...
abl_glacier, acc_glacier, runoff_glacier] = display_output_glacier(point_output_path_glacier, yr, mm, dd, date);
end
end
These lines are themselves part of a function. In it, I use the resulting variables (eg: albedo_land, albedo_glacier etc..) to make some calculations and plots. Originally I just ran the functions by writing :
[T20cm_land, albedo_land, smb_land, swe_land, T1m_land, T0_land, abl_land,...
acc_land, runoff_land] = display_output_land(point_output_path_land, yr, mm, dd, date);
[T20cm_glacier, albedo_glacier, smb_glacier, swe_glacier, T1m_glacier, T0_glacier,...
abl_glacier, acc_glacier, runoff_glacier] = display_output_glacier(point_output_path_glacier, yr, mm, dd, date);
But these two functions are independant and take around 30 minutes to run. Is there a way to save all the output variables from these 2 functions outside of the parfor loop, in order to use them later in the script ?
Have a nice day !

Accepted Answer

Edric Ellis
Edric Ellis on 29 Apr 2020
Output variables from parfor must be either sliced or reduction variables. So, you could adapt your code like so:
parfor i = 1:2
if i == 1
[T20cm{i}, albedo{i}, . . .] = display_output_land(. . .);
else
[T20cm{i}, albedo{i}, . . .] = display_output_glacier(. . .);
end
end
This converts all the outputs into sliced form.
An alternative given the fact that you really essentially have two completely separate function calls is to use parfeval instead, like this:
% Invoke both function evaluations in parallel.
fut_land = parfeval(@display_output_land, 9, point_output_path_land, yr, mm, dd, date);
fut_glacier = parfeval(@display_output_glacier, 9, point_output_path_glacier, yr, mm, dd, date);
% Collect results - this blocks until the evaluation is complete.
[T20cm_land, albedo_land, . . .] = fetchOutputs(fut_land);
[T20cm_glacier, albedo_glacier, . . .] = fetchOutputs(fut_glacier);
  3 Comments
V.D-C
V.D-C on 29 Apr 2020
Then I should change my code for your solution, I'm working on a cluster (even though the files are written in a different folder). Thank you for your advices !

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!