What Is Parallel Computing?

R2026b

Parallel computing allows you to carry out many calculations simultaneously. Split large problems into smaller ones, which you can process at the same time.

The main reasons to consider parallel computing are to:

• Save time by distributing tasks and executing these simultaneously

• Solve big data problems by distributing data

• Take advantage of your desktop computer resources and scale up to clusters and cloud computing

This table lists some useful parallel computing terms and their definitions.

Term	Definition
Thread	Smallest set of instructions that a CPU can schedule and execute independently. A GPU, multiprocessor, or multicore computer can perform multithreading, or execute multiple threads simultaneously.
Process	Execution of an instance of a computer program by one or many threads. Each process has its own blocks of memory.
Node	Standalone computer containing one or more CPUs or GPUs. Nodes can be networked to form a cluster or supercomputer.
Cluster	Collection of interconnected computers that work together as a unified system to provide high-performance computing power for processing complex and data-intensive tasks.
Scalability	Increase in parallel speedup with the addition of more resources.
Batch	Off-load execution of a functional script to run in the background.

What tools do MATLAB^® and Parallel Computing Toolbox™ offer?

• MATLAB workers: MATLAB computational engines that run in the background without a graphical desktop. You use functions in the Parallel Computing Toolbox to automatically divide tasks and assign them to these workers to execute the computations in parallel. You can run local workers to take advantage of all the cores in your multicore desktop computer. You can also scale up to run your workers on a cluster of machines, using the MATLAB Parallel Server™. The MATLAB session you interact with is known as the MATLAB client. The client instructs the workers with parallel language functions.

• Parallel pool: a parallel pool of MATLAB workers created using parpool or functions with automatic parallel support. By default, parallel language functions automatically create a parallel pool for you when necessary. To learn more, see Run Code on Parallel Pools.

  For the default Processes profile, the default number of workers is one per physical CPU core using a single computational thread. This is because even though each physical core can have several virtual cores, the virtual cores share some resources, typically including a shared floating point unit (FPU). Most MATLAB computations use this unit because they are double-precision floating point. Restricting to one worker per physical core ensures that each worker has exclusive access to a floating point unit, which generally optimizes performance of computational code. If your code is not computationally intensive, for example, it is input/output (I/O) intensive, then consider using up to two workers per physical core. Running too many workers on too few resources may impact performance and stability of your machine.

• Speed up: Accelerate your code by running on multiple MATLAB workers or GPUs, for example, using parfor, parfeval, or gpuArray.

• Scale up your data: Partition your big data across multiple MATLAB workers and use interactive Big Data processing tools such as distributed and tall arrays, datastore, and mapreduce. To learn more, see Big Data Processing.

• Asynchronous processing: Use parfeval to execute a computing task in the background without waiting for it to complete.

• to offload your calculation to computer clusters or cloud computing facilities

• Scale up to clusters and clouds: If your computing task is too big or too slow for your local computer, you can use batch jobs to offload your calculation to a cluster onsite or in the cloud using MATLAB Parallel Server. For more information, see Clusters and Clouds.