gpuDevice
Query or select a GPU device
Description
A GPUDevice
object represents a graphic processing unit (GPU)
in your computer. You can use the GPU to run MATLAB® code that supports gpuArray
variables or execute CUDA® kernels using CUDAKernel
objects.
You can use a GPUDevice
object to inspect the properties of your GPU
device, reset the GPU device, or wait for your GPU to finish executing a computation. To
obtain a GPUDevice
object, use the gpuDevice
function.
You can also select or deselect your GPU device using the gpuDevice
function. If you have access to multiple GPUs, use the gpuDevice
function
to choose a specific GPU device on which to execute your code.
You do not need to use a GPUDevice
object to run functions on a GPU.
For more information on how to use GPU-enabled functions, see Run MATLAB Functions on a GPU.
Creation
Description
gpuDevice
displays the properties of the currently selected GPU
device. If there is no currently selected device, gpuDevice
selects the
default device without clearing it. Use this syntax when you want to inspect the
properties of your GPU device.
D = gpuDevice
returns a GPUDevice
object
representing the currently selected device. If there is no currently selected device,
gpuDevice
selects the default device and returns a
GPUDevice
object representing that device without clearing it.
D = gpuDevice(
selects the GPU device
specified by index ind
)ind
. If the specified GPU device is not supported,
an error occurs. This syntax resets the specified device and clears its memory, even if
the device is already currently selected (equivalent to the reset
function). All workspace variables representing
gpuArray
or CUDAKernel
variables are now invalid
and must be cleared from the workspace or redefined.
gpuDevice([])
, with an empty argument (as opposed to no
argument), deselects the GPU device and clears its memory of gpuArray
and CUDAKernel
variables. This syntax leaves no GPU device selected as
the current device.
Input Arguments
ind
— Index of the GPU device
integer
Index of the GPU device, specified as an integer in the range 1
to gpuDeviceCount
.
Example: gpuDevice(1);
Data Types: single
| double
| int8
| int16
| int32
| int64
| uint8
| uint16
| uint32
| uint64
Properties
Identity
Name
— Name of the GPU device
character array
This property is read-only.
Name of the GPU device, specified as a character array. The name assigned to the device is derived from the GPU device model.
Data Types: char
Index
— Index of the GPU device
integer
This property is read-only.
Index of the GPU device, specified as an integer in the range 1
to gpuDeviceCount
. Use this index to select a particular GPU
device.
Data Types: single
| double
| int8
| int16
| int32
| int64
| uint8
| uint16
| uint32
| uint64
UUID
— Universally unique identifier
character array
Since R2024a
This property is read-only.
Universally unique identifier (UUID) of the device, specified as a character array.
The UUID typically starts with 'GPU-'
and includes a
36-character hexadecimal sequence. You can use the UUID to distinguish otherwise
identical GPUs.
Data Types: char
State
DeviceSelected
— Flag for currently selected device
0
(false
) | 1
(true
)
This property is read-only.
Flag for currently selected device, specified by the logical values
0
(false
) or 1
(true
).
Data Types: logical
DeviceAvailable
— Flag for available device
0
(false
) | 1
(true
)
This property is read-only.
Flag for available device, specified by the logical values 0
(false
) or 1
(true
). This
property indicates whether the device is available for use in the current MATLAB session. Unsupported devices with a DeviceSupported
property of 0
(false
) are always unavailable. A
device can also be unavailable if its ComputeMode
property is set
to 'Exclusive thread'
, 'Exclusive process'
, or
'Prohibited'
.
Data Types: logical
DeviceSupported
— Flag for supported device
0
(false
) | 1
(true
)
This property is read-only.
Flag for supported device, specified by the logical values 0
(false
) or 1
(true
). Not
all devices are supported; for example, devices with insufficient
ComputeCapability
.
Data Types: logical
LastAccessed
— Date and time device was last accessed
datetime
array
Since R2024b
This property is read-only.
Date and time the device was last accessed by the current MATLAB session, specified as a scalar datetime
array. If the
device has not been accessed in this session, then LastAccessed
is Not-a-Time (NaT
).
Most uses of a GPU device in MATLAB update LastAccessed
, including:
Selecting a device using
gpuDevice
.Resetting a device using
reset
.Creating or using a
gpuArray
.
Querying properties of the currently selected device does not update
LastAccessed
.
Data Types: datetime
Memory
TotalMemory
— Total memory
scalar
This property is read-only.
Total memory (in bytes) on the device, specified as a scalar value.
Data Types: double
AvailableMemory
— Total memory available for data
scalar
This property is read-only.
Total memory (in bytes) available for data, specified as a scalar value. This property is available only for the currently selected device. This value can differ from the value reported by the NVIDIA® System Management Interface due to memory caching.
Data Types: double
CachePolicy
— Current caching policy of GPU device
'balanced'
| 'minimum'
| 'maximum'
Since R2023a
Caching policy of the GPU device, specified as 'balanced'
,
'minimum'
, or 'maximum'
. The caching policy
determines how much GPU memory can be cached to accelerate computation, specified as
one of the following values.
'minimum'
– The amount of memory that can be cached on the GPU device is minimal.'balanced'
– The amount of memory that can be cached on the GPU device is balanced. This policy provides a balance between GPU memory usage and computational performance.'maximum'
– The amount of memory that can be cached on the GPU device is limited only by the total memory of the device.
The default value is 'balanced'
for devices in
'Default'
or 'Prohibited'
compute mode and
'maximum'
for devices in 'Exclusive process'
compute mode. For more information on the compute mode property, see
ComputeMode
.
Note
Resetting the device using
reset
, clearing the device usinggpuDevice([])
, or selecting another device usinggpuDevice
resets the caching policy to the default policy.Saving and loading a MAT file containing a
GPUDevice
object does not preserve the caching policy.You cannot set the caching policy of a device that is not selected. For example, after storing a first
GPUDevice
object in an array and selecting another device, you cannot set the caching policy of the firstGPUDevice
object.
Data Types: char
| string
Driver
GraphicsDriverVersion
— Graphics driver version in use
character array
Since R2023a
This property is read-only.
Graphics driver version currently in use by the GPU device, specified as a character array.
Download the latest graphics driver for your GPU at NVIDIA Driver Downloads.
Data Types: char
DriverModel
— Operating model of graphics driver
'WDDM'
| 'TCC'
| 'N/A'
Since R2023a
This property is read-only.
Operating model of the graphics driver, specified as one of these values:
'WDDM'
– Use the display operating model.'TCC'
– Use the compute operating model.'TCC'
disables Windows® graphics and can improve the performance of large scale calculations.'N/A'
–'WDDM'
and'TCC'
are only available on Windows. On other operating systems the driver model is'N/A'
.
For more information about changing models and which GPU devices support
'TCC'
, see the NVIDIA documentation.
Data Types: char
ComputeMode
— Compute mode
'Default'
| 'Exclusive process'
| 'Prohibited'
This property is read-only.
Compute mode of the device, specified as one of the following values.
'Default' | The device is not restricted, and multiple applications can use it simultaneously. MATLAB can share the device with other applications, including other MATLAB sessions or workers. |
'Exclusive process' | Only one application at a time can use the device. While the device is selected in MATLAB, other applications cannot use it, including other MATLAB sessions or workers. |
'Prohibited' | The device cannot be used. |
For more information changing the compute mode of your GPU device, consult the NVIDIA documentation.
Data Types: char
KernelExecutionTimeout
— Flag for timeout for long-running kernels
0
(false
) | 1
(true
)
This property is read-only.
Flag for timeout for long-running kernels, specified as 0
(false
) or 1
(true
). If
KernelExecutionTimeout
is 1
(true
), then the operating system places an upper bound on the
time allowed for the CUDA kernel to execute. After this time, the CUDA driver times out the kernel and returns an error.
Data Types: logical
Capabilities
ComputeCapability
— Computational capability of the GPU device
character array
This property is read-only.
Computational capability of the GPU device, specified as a character array. To use
the selected GPU device in MATLAB, ComputeCapability
must meet the required
specification in GPU Computing Requirements.
Data Types: char
MultiprocessorCount
— Number of streaming multiprocessors
scalar
This property is read-only.
Number of streaming multiprocessors present on the device, specified as a scalar value.
ClockRateKHz
— Peak clock rate
scalar
This property is read-only.
Peak clock rate of the GPU in kHz, specified as a scalar value.
Data Types: double
SingleDoubleRatio
— Ratio of single- to double-precision FPUs
scalar
Since R2024b
This property is read-only.
Ratio of single- to double-precision floating point units (FPUs) on the device, specified as a scalar value.
The ratio indicates the single-precision processing power relative to the double-precision processing power of the device. Devices that are suitable for double-precision computations, such as solving linear systems, generally have a lower ratio. Devices that are suitable for single-precision computations, such as training deep neural networks and rendering graphics, generally have a larger ratio.
Data Types: double
Kernel Programming
MaxThreadsPerBlock
— Maximum supported number of threads per block
scalar
This property is read-only.
Maximum supported number of threads per block during CUDAKernel
execution, specified as a scalar value.
Example: 1024
Data Types: double
MaxShmemPerBlock
— Maximum supported amount of shared memory
scalar
This property is read-only.
Maximum supported amount of shared memory that a thread block can use during
CUDAKernel
execution, specified as a scalar value.
Data Types: double
MaxThreadBlockSize
— Maximum size in each dimension for thread block
vector
This property is read-only.
Maximum size in each dimension for thread block, specified as a vector. Each
dimension of a thread block must not exceed these dimensions. Also, the product of the
thread block size must not exceed MaxThreadsPerBlock
.
Example: [1024 1024 64]
Data Types: double
MaxGridSize
— Maximum size of grid of thread blocks
vector
This property is read-only.
Maximum size of grid of thread blocks, specified as a vector.
Example: [2.1475e+09 65535 65535]
Data Types: double
SIMDWidth
— Number of simultaneously executing threads
scalar
This property is read-only.
Number of simultaneously executing threads, specified as a scalar value.
Data Types: double
ToolkitVersion
— CUDA toolkit version
scalar
This property is read-only.
CUDA toolkit version used by the current release of MATLAB, specified as a scalar value.
Data Types: double
Object Functions
reset | Reset GPU device and clear its memory |
wait (GPUDevice) | Wait for GPU calculation to complete |
The following functions are also available:
parallel.gpu.GPUDevice.isAvailable(ind) | Returns logical 1 or true if the GPU
specified by index ind is supported and capable of being
selected. ind can be an integer or a vector of integers; the
default index is the current device. |
parallel.gpu.GPUDevice.getDevice(ind) | Returns a GPUDevice object without selecting it. |
For a complete list of functions, use the methods
function on the
GPUDevice
object:
methods('parallel.gpu.GPUDevice')
You can get help on any of the object functions with the following command:
help parallel.gpu.GPUDevice.functionname
where functionname
is the name of the function. For example, to
get help on isAvailable
, type:
help parallel.gpu.GPUDevice.isAvailable
Examples
Identify and Select a GPU Device
This example shows how to use gpuDevice
to identify and select which device you want to use.
To determine how many GPU devices are available in your computer, use the gpuDeviceCount
function.
gpuDeviceCount("available")
ans = 2
When there are multiple devices, the first is the default. You can examine its properties with the gpuDeviceTable
function to determine if that is the one you want to use.
gpuDeviceTable
ans=2×5 table
Index Name ComputeCapability DeviceAvailable DeviceSelected
_____ __________________ _________________ _______________ ______________
1 "NVIDIA RTX A5000" "8.6" true false
2 "Quadro P620" "6.1" true false
If the first device is the device you want to use, you can proceed. To run computations on the GPU, use gpuArray
enabled functions. For more information, see Run MATLAB Functions on a GPU.
To verify that MATLAB® can use your GPU, use the canUseGPU
function. The function returns 1
(true
) if there is a GPU available for computation and 0
(false
) otherwise.
canUseGPU
ans = logical
1
To diagnose an issue with your GPU setup, for example if canUseGPU
returns 0
(false
), use the validateGPU
function. Validating your GPU is optional.
validateGPU
# Beginning GPU validation # Performing system validation # CUDA-supported platform .................................................PASSED # CUDA-enabled graphics driver exists .....................................PASSED # Version: 537.70 # CUDA-enabled graphics driver load .......................................PASSED # CUDA environment variables ..............................................PASSED # CUDA device count .......................................................PASSED # Found 2 devices. # GPU libraries load ......................................................PASSED # # Performing device validation for device index 1 # Device exists ...........................................................PASSED # NVIDIA RTX A5000 # Device supported ........................................................PASSED # Device available ........................................................PASSED # Device is in 'Default' compute mode. # Device selectable .......................................................PASSED # Device memory allocation ................................................PASSED # Device kernel launch ....................................................PASSED # # Finished GPU validation with no failures.
To use another device, call gpuDevice
with the index of the other device.
gpuDevice(2)
ans = CUDADevice with properties: Name: 'Quadro P620' Index: 2 (of 2) ComputeCapability: '6.1' DriverModel: 'WDDM' TotalMemory: 2147352576 (2.15 GB) AvailableMemory: 1596066816 (1.60 GB) DeviceAvailable: true DeviceSelected: true Show all properties.
Alternatively, you can determine how many GPU devices are available, inspect some of their properties, and select a device to use from the MATLAB® desktop. On the Home tab, in the Environment area, select Parallel > Select GPU Environment.
Query Compute Capabilities
Create an object representing the default GPU device and query its compute capability.
D = gpuDevice; D.ComputeCapability
ans = '8.6'
Query the compute capabilities of all available GPU devices.
for idx = 1:gpuDeviceCount D = gpuDevice(idx); fprintf(1,"Device %i has ComputeCapability %s \n", ... D.Index,D.ComputeCapability) end
Device 1 has ComputeCapability 8.6 Device 2 has ComputeCapability 6.1
Compare the compute capabilities and availability of the GPU devices in your system using gpuDeviceTable
.
gpuDeviceTable
ans=2×5 table
Index Name ComputeCapability DeviceAvailable DeviceSelected
_____ __________________ _________________ _______________ ______________
1 "NVIDIA RTX A5000" "8.6" true false
2 "Quadro P620" "6.1" true true
Query and Change Caching Policy
Change the caching policy of your GPU.
Create an object representing the default GPU device.
D = gpuDevice
D = CUDADevice with properties: Name: 'NVIDIA RTX A5000' Index: 1 (of 2) ComputeCapability: '8.6' DriverModel: 'TCC' TotalMemory: 25544294400 (25.54 GB) AvailableMemory: 25120866304 (25.12 GB) DeviceAvailable: true DeviceSelected: true Show all properties.
Access the CachePolicy
property of the GPU device.
D.CachePolicy
ans = 'balanced'
Change the caching policy to allow the GPU to cache the maximum amount of memory for accelerating computation.
D.CachePolicy = "maximum";
D.CachePolicy
ans = 'maximum'
Reset the caching policy to the default policy by setting the property to []
.
D.CachePolicy = [];
Calling reset(D)
or selecting another device with gpuDevice
also resets the caching policy to its default value.
Use Multiple GPUs in Parallel Pool
If you have access to several GPUs, you can perform your calculations on multiple GPUs in parallel using a parallel pool.
To determine the number of GPUs that are available for use in MATLAB, use the gpuDeviceCount
function.
availableGPUs = gpuDeviceCount("available")
availableGPUs = 3
Start a parallel pool with as many workers as available GPUs. For best performance, MATLAB assigns a different GPU to each worker by default.
parpool("Processes",availableGPUs);
Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 3).
To identify which GPU each worker is using, call gpuDevice
inside an spmd
block. The spmd
block runs gpuDevice
on every worker.
spmd gpuDevice end
Use parallel language features, such as parfor
or parfeval
, to distribute your computations to workers in the parallel pool. If you use gpuArray
enabled functions in your computations, these functions run on the GPU of the worker. For more information, see Run MATLAB Functions on a GPU. For an example, see Run MATLAB Functions on Multiple GPUs.
When you are done with your computations, shut down the parallel pool. You can use the gcp
function to obtain the current parallel pool.
delete(gcp("nocreate"));
If you want to use a different choice of GPUs, then you can use gpuDevice
to select a particular GPU on each worker, using the GPU device index. You can obtain the index of each GPU device in your system using the gpuDeviceCount
function.
Suppose you have three GPUs available in your system, but you want to use only two for a computation. Obtain the indices of the devices.
[availableGPUs,gpuIndx] = gpuDeviceCount("available")
availableGPUs = 3
gpuIndx = 1×3
1 2 3
Define the indices of the devices you want to use.
useGPUs = [1 3];
Start your parallel pool. Use an spmd
block and gpuDevice
to associate each worker with one of the GPUs you want to use, using the device index. The spmdIndex
function identifies the index of each worker.
parpool("Processes",numel(useGPUs));
Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 2).
spmd gpuDevice(useGPUs(spmdIndex)); end
As a best practice, and for best performance, assign a different GPU to each worker.
When you are done with your computations, shut down the parallel pool.
delete(gcp("nocreate"));
Extended Capabilities
Thread-Based Environment
Run code in the background using MATLAB® backgroundPool
or accelerate code with Parallel Computing Toolbox™ ThreadPool
.
This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.
Version History
Introduced in R2010bR2024b: New Device Properties and Updated Display
Use the gpuDevice
function to inspect these new properties of your
GPU device:
LastAccessed
— the date and time the device was last accessed by the current MATLAB session.SingleDoubleRatio
— the ratio of single- to double-precision floating point units (FPUs) on the device.
Creating or querying a GPUDevice
object now displays only the
Name
, Index
,
ComputeCapability
, DriverModel
,
TotalMemory
, AvailableMemory
,
DeviceAvailable
, and DeviceSelected
properties.
To view all of the properties of a device, create or query a GPUDevice
object without suppressing output and click the Show all properties
link.
D = gpuDevice
D = CUDADevice with properties: Name: 'NVIDIA RTX A5000' Index: 1 (of 2) ComputeCapability: '8.6' DriverModel: 'TCC' TotalMemory: 25544294400 (25.54 GB) AvailableMemory: 25120866304 (25.12 GB) DeviceAvailable: true DeviceSelected: true Show all properties.
The SupportsDouble
, GPUOverlapsTransfers
, and
CanMapHostMemory
properties are no longer displayed but you can still
query these properties using dot notation. There are no plans to remove these
properties.
R2024a: Identify GPUs using their UUIDs
Inspect the universally unique identifier (UUID) of your GPU using the
UUID
property of a GPUDevice
object. You can use the
UUID to distinguish otherwise identical GPUs.
For example, you can inspect the UUID of two GPUs using the gpuDeviceTable
function.
gpuDeviceTable(["Index","Name","UUID"])
Index Name UUID _____ __________________ __________________________________________ 1 "NVIDIA RTX A5000" "GPU-957b509e-322a-ae88-59c8-b7435d0f98f4" 2 "NVIDIA RTX A5000" "GPU-6f3ad2c0-5ea1-b1a2-1dca-cd756d10dbc0" 3 "NVIDIA RTX A5000" "GPU-41c24f34-c915-919b-0bb3-20d07117e0ec" 4 "NVIDIA RTX A5000" "GPU-23ab01ce-2f42-2d0c-0f6b-db7ac8c10867"
Alternatively, you can select a GPU using the gpuDevice
function and query its UUID.
D = gpuDevice; D.UUID
'GPU-957b509e-28ca-ae88-59c8-b7435d0f98f4'
R2023a: Changes to Device Properties
GraphicsDriverVersion
property added to show the graphics driver version currently in use.DriverModel
property added to show the operating model of the graphics driver on Windows.CachePolicy
property added to allow changes to GPU memory caching policy.The
DriverVersion
property is no longer displayed by default but you can still query the property using dot notation. There are no plans to remove theDriverVersion
property.
See Also
gpuArray
| gpuDeviceTable
| gpuDeviceCount
| canUseGPU
| validateGPU
| arrayfun
| reset
| wait (GPUDevice)
| GPUDeviceManager
Topics
External Websites
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)