how to run two scripts (one MEX function and one that calls upon a digital library) in parallel using parfor and parallel pools?

I am trying to achieve parallel exectution of two functions that would normally need two separate matlab(r2023a) windows open in order to successfuly execute. The PULM() function is a MEX funtion and DMD_Funtion is a matlab function that uses a library ('alp4395'). because of this, a threadpool does not work because 'Use of MEX functions is not supported on a thread-based worker.' The reason either a parallel processing function or 2 MATLAB windows are needed is because PULM() needs to initialize into its ready state then it waits for a trigger (meaning matlab will stay 'Busy' and wont execute any other commands until PULM is finished, which waits indefinitely for a trigger), which is the DMD_Function(), then finishes the rest of the PULM MEX functions to collect data and exit.

I have been trying to use parpool('Processes') instead, however it seems that it is unable to successfully 'exit gracefully' from the C code, probably due to timing issues with the trigger. PULM() needs several seconds to be in its 'Ready' state before the DMD_Funtion() is ran or else the PULM() will be waiting for triggers and basically never exit. I noticed that even with the Pause(5) or even 10 that I put in before the the DMD funtion is ran, all that happens is that both workers wait 5 seconds before running simultaneously instead of being delayed like they are supposed to. How do I delay the exectution of the DMD funtion? is it possible that i = 3 is running before i = 2 causing both of interations to pause before executing? Are there any other ways I could accomplish parallel processing in this way without the use of 2 MATLAB windows?

Initially I thought using 2 push buttons on a GUI would work because I believed that push buttons worked in parallel, but I soon realized they were executed sequentially. So I now have to use parallel toolbox functions so that one push button with execute everything in parallel.

Here's the code for the parfor:

parfor i=2:3
    if i==2
        
        %pause(1);
        %DMD_Function(patternNumber,AvergeNumber,SamplingRate,BinningNumber=768/SamplingRate) 768/32
        PULM(PonitNumber,NumberofRecording,AvergeNumber);
        %disp('run 1');
        disp('done 1')
        
        
    else
        pause(5);
        DMD_Function(patternNumber,AvergeNumber,SamplingRate,BinningNumber);
        
        disp('done 2')
        
        
    end
    
end

Note: 'done 1' does not display meaning it does not complete the MEX function but 'All records were saved to file' does display which is right before 'Exit Gracefully' for the Digitizer.

In a separate code I tried to why the pause was just making them both wait 5 seconds then simultaneously running instead of being delayed. Threadpools seem to do what I want, it executes i==2 and delays i == 3. I am still not sure as to why Processespools doesn't do the same thing. I even tried to stop the execution of the i==3 line by having it check if i == 2 wrote in a file, but it just ccomes up with an error. this is the reason why I was wondering if i==3 is being executed faster than i==2.

stateFile = 'state.txt';
parfor i = 2:3
    if i == 2
        %pause(5);
        disp('hi')
        
        fid = fopen(stateFile, 'w');
        fwrite(fid, '5', 'char');
        fclose(fid);
    else
        pause(5);
        if isfile(stateFile)
            fid = fopen(stateFile, 'r');
            state = fread(fid, '*char')';
            fclose(fid);
            % Check if the state value matches
            if strcmp(state, '5')
                disp('hello');
            end
        end
    end
end

[Daniel Bengtson]

Maybe this is an application where parfeval might work?

Parfeval

[Luiz]

fetchOutput returns:

' Use of function libisloaded is not supported on a thread-based worker.'

Answer 1

[Subhajyoti]

Hi @Luiz

To resolve the error, I've modularized the code into two functions and converted one into a MEX function. I've also added print statements for debugging to track the code execution flow. Here's the updated script:

stateFile = 'state.txt';
parfor i = 2:3
    if i == 2
        disp('------------');
        disp('Start MEX');
        disp('============');
        funcA_mex(stateFile);
        disp('============');
        disp('Done MEX');
        disp('------------');
    else
        funcB(stateFile);
    end
end

function funcB(filename)
    disp('pausing i==3')
    pause(20);
    disp('resuming i==3')
    if isfile(filename)
        fid = fopen(filename, 'r');
        state = fread(fid, '*char')';
        fclose(fid);
        % Check if the state value matches
        if strcmp(state, '5')
            disp('hello');
        end
    end
end

The following ‘funcA.m’ is used to generate ‘funcA_mex’ using MATLAB Coder:

function funcA(filename)
    disp('executing i==2')
    
    fid = fopen(filename, 'w');
    fwrite(fid, '5', 'char');
    fclose(fid);
    pause(4);
    disp('resuming i==2')
end

This generated the following output:

This setup shows that the MEX function execution wasn't paused by the pause function, allowing the script to complete parallel execution successfully.

Also, threading is not supported in External Language Interfaces. Using ‘parfeval’ in these contexts will result in errors. For more information, you can refer to the following MathWorks Documentation: https://www.mathworks.com/help/matlab/matlab_prog/run-functions-on-threads.html

Also, I suggest going through the following MathWorks documentation to learn more about Thread-Based v/s Process-Based Environments: https://www.mathworks.com/help/parallel-computing/choose-between-thread-based-and-process-based-environments.html