How can we handle the conditional use of a parfor loop with the eval function?

Hi,

I'd like to use a parfor loop conditionally, and to use a simple for loop on the same code block otherwise.

The eval functions reports the following error message:

Error: At least one END is missing: the statement may begin here.

when executing the following code:

% start for/parfor loop
if bool_parfor
    parpool(2);
    eval('parfor k=1:length(vect_tmp)'); % > error message
else
    eval('for k=1:length(vect_tmp)');
end
% code
...
% end for/parfor loop
if bool_parfor
    eval('end')
    p = gcp;
    delete(p);
else
    eval('end')
end

This topic has already been discussed in the following post, where the proposed solutions are to set the number of workers to 0 (it still slows down the process, so this is not an option), or to use two different blocks of code (or two functions), which is not efficient for development purposes.

https://www.mathworks.com/matlabcentral/answers/161551-is-it-possible-to-conditionally-switch-between-parallel-and-non-parallel-for-loops?s_tid=srchtitle

As a consequence, I don't understand why even in the last versions of Maltab the eval function returns an error. When just entering the command 'for a=1:3' in the command window, Matlab is indeed expected to run until a 'end' command is entered by the user. Is there a way to avoid this issue in the eval function? Is it possible to access/modify its source code?

Thanks for your view,

Thomas

Answer 1

[Walter Roberson]

This is not going to work. eval() does not just drop the text into the input buffer where it might be completed by later code after the eval. eval() runs a complete parsing session on code that must be complete in itself.

The effect is like running a script. If you run a script has several extra end statements, then those statements do not get buffered and act upon calling routine: you get a syntax error instead.

eval is built in. To see the source code you will need to get a job with Mathworks.

You cannot eval inside a parfor block. parfor needs to do static analysis to figure out what memory to transfer.

There are way way too many problems with the kind of semantics that you want. For example

if rand<.5
  eval('for i=1:10')
  if rand<. 5
    eval('for j=1:3') 
    eval('eval(''end'')')

How many iterations does this do? Is the code complete or not? The code might eval extra end statements so you cannot tell just by reading it where the blocks terminate under your proposal.

What is possible is to put sections of code into data files, and have other code that reads the data file, puts for or parfor as the first word and writes the results to a file. Then run the file as a script.

You will still run into some problems as there are differences in treatment of variables that also happen to be function names when you use scripts.

On the other hand, you can drop the entire function in as a data file and preprocess the entire function, at the point where you change bool_parfor

[tom3w]

Thank you Walter, it becomes clearer to me.

I think Mathworks should implement a variant version of eval as a complement (it can hence have another name), for which the code is not required to be 'complete in itself'. It would be extremely useful and much easier.

Regarding your proposal (putting part of code/a function into a data file, preprocessing and running it as a script), would you have a simple example or a link to share?

Nb: in your example showing a semantic issue, two separate eval('end') statements would indeed be needed I guess.

[tom3w]

I have asked Mathworks for a new function. Would anyone have a simple solution, I naturally remain interested. Thanks

[Walter Roberson]

function scriptname = preparefor(filename, bool_parfor)
    S = fileread(filename);
    forstrings = {'for', 'parfor'};
    forstring = forstrings{bool_parfor+1};
    S = regexprep(S, '\$FOR', forstring);
    
    scriptname = [tempfile '.m'];
    fid = fopen(scriptname, 'w');
    if bool_parfor
        fprintf('p = gcp(''nocreate''); if isempty(p); p = parpool(2); end;\n');
    end
    fwrite(fid, S);
    fprintf(fid, '\n');
    
    if bool_parfor
        fprintf('p = gcp(''nocreate''); if ~isempty(p); delete(p); end;\n');
    end
end

function main
   %stuff. 
   %eventually
   filename = 'blahblahblah.txt';
   bool_parfor = rand() > .7;
   scriptname = preparefor(filename, bool_parfor);
   cleanMe = onCleanup(@() delete(scriptname));
   
   run(scriptname);
   %more stuff
end

file blahblahblah.txt :

 % start for/parfor loop
 %the for / parfor does not need to be the first thing
$FOR k=1:length(vect_tmp)');

% code

end

[tom3w]

Great, I'll try this out; many thanks Walter.