Communicating between parfor loop iterations

I am using a parfor loop (Matlab parallel tool box) as part of a multi-start technique to look for a global minimum to a derivative free optimization problem. The problem I am facing is that I need some small communication between the loop iterations, to determine if an iteration is converging to a local minimum that is higher than another so that it can cease what it is doing and move on to another starting point. I know that all of the iterations are independent, so I what I want is for each loop iteration to read/write to a common file so that at a certain point in the algorithm, the each individual loop will open the file, compare what it's function value is to what is already in the file, and either proceed or terminate. Ideally, the final result in the file will be the minimum point found from all of the iterations, but what I actually get is the best point of the last iteration to finish. It seems that each loop iteration is writing over the previous information, or that with every loop, the file re-initiallizes and is a blank canvas for the taking, so to speak.

(I have already built in a 'lock' type function that should prevent two loops from opening the file at the same time to eliminate any race conditions.)

Any help would be greatly appreciated and thanks in advance.

Answer 1

[Edric Ellis]

I don't think there's a reliable way to achieve this with PARFOR. I would be tempted to investigate SPMD which does allow communication. For example, you could do something like this:

    spmd
       while ~done
          myResult = fcn(...);

          % use GCAT to get results from all labs onto
          % each lab - after this, 'allResults' is a vector
          % of length NUMLABS containing all 'myResult' values
          allResults = gcat( myResult );
          if myResult == max( allResults ) % I am worst
             % do something else
          end

          % Note that all labs compute the same result
          % for 'done'
          done = any( abs( allResults ) < tolerance );
       end
    end

Answer 2

[Brian W]

Thanks Edric- I did get the PARFOR loop to do what I needed it to by implementing a 'lock' file that uses system commands to delete and write a file that exists outside of the specific Matlab program. This has taken care of any race conditions and allowed the individual workers to compare the specific values I am looking for without corrupting the files or unintentionally overwriting any data.

I did consider using the spmd code block, but with spmd I was going have to to program an independent scheduler, as the pieces of data to be evaluated numbers in the 1000's, and I am using the built-in scheduler in the PARALLEL TOOLBOX (I don't have the Matlab Distributed Computing Server, so I am only working off of 12 workers), and spmd will wait until it has enough workers to complete the task.

That said, I could have used a distributed array or something to parse the data, but the algorithm each worker is running has some early termination criteria built-in, so there is potential that if a worker were to evaluate a large amount of data points that triggered this early termination, it would finish its allotment of data before the others and sit idle while the others finished their respective portions of the larger data set.

Basically, my algorithm is somewhat chaotic in that the number if iterations it takes to converge to a solution will be different for different initial data points (seeds). Thus, I found that the PARFOR loop allowed me the most flexibility in initial point distribution across the workers.

This question sat unanswered for several days at least, so thanks again for taking the time to think about my problem and making some suggestions. It is appreciated.

Answer 3

[Eric]

Hi Brian,

I am struggling with a similar issue of desiring a very small amount of communication in a parfor loop. Could you share which system commands you used that were able to create lock files without any race conditions?

Thanks! Eric