How can implement parallel pipeline using 3 workers in one computer ?
1 view (last 30 days)
Show older comments
Dear everyone, Please I have a question:
I would like to used 3 workers (cores) to implement 3 functions in a parallel as pipeline, i.e. the first worker create the data, do some process(fun-add), and sends the results (x) to the first buffer/queue/matrix. Then, the second worker receives the data from the first buffer, do some process (fun-mult), and send the results (y) to the second buffer. The third worker receive the data, do some process (fun-double) and print the output(z). In the following attach the picture show the situation.
My question is that: How can I implement these three functions in parallel pipeline ?
Please, any idea how to solve this problem?
Kind regards Ammar
0 Comments
Answers (1)
Walter Roberson
on 19 Aug 2017
You could use spmd and labSend the data from one worker to another.
4 Comments
Walter Roberson
on 23 Aug 2017
In order to implement that by using memory instead of a file, you would have to use something like the routine named sharedmatrix from the File Exchange; however, I see a post that claims that routine might not work starting with R2017a. Also, you would have to change the source code for the sharedmatrix routine a little if you are using MS Windows.
There is no built-in routine for handling queues that are based on files. There are a few different approaches. https://www.mathworks.com/help/matlab/import_export/share-memory-between-applications.html shows one of them. That example is not reliable as a Queue, though, as it is not designed to allow there to be several entries in the queue.
It is a bit tricky to get file-based queues working properly; it is common to encounter "race conditions" unless you have operating system support for atomic operations, such as semaphore objects .
See Also
Categories
Find more on Asynchronous Parallel Programming in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!