The utilization of GPU is low in deep learning

Hello, I’m using NVIDIA GeForce GTX 1080 Ti to run “single image super resolution network” code The utilization average for gpu was 25%, I added this code:

feature('GpuAllocPoolSizeKb', intmax('int32'));

And the utilization increased to 33%

Is there something more I can do to increase the utilization? Also, I don’t think MATLAB runs in the same speed of my GPU Note: my MATLAB version is R2019b, and I have student license

Answer 1

[Joss Knight]

Try following some of the advice in the following MATLAB Answer: https://uk.mathworks.com/matlabcentral/answers/463367-gpu-utilization-is-not-100

[Lamya Mohammad]

Thank you, but it didn’t help

The thing that I don’t understand is the utilization at the first epochs was 35% but it kept dropping till it reached 20%

[Joss Knight]

Given this and your opening post it sounds like memory pressure is an obstacle to having the GPU running kernels all the time. You might also try the additional feature command:

feature('GpuMaxWorkspaceSizeKb', intmax('int32'));

which may help. But ultimately you should be reducing the amount of memory your network needs to run, and better load-balancing the GPU work vs the CPU work: Reduce the MiniBatchSize options, and use the DispatchInBackground training option.

[Joss Knight]

Or if you're doing inference, use 'Acceleration', 'mex' if that's available to you.

You may also wish to ask yourself the question: why do I care about utilisation? Utilisation is a very rough indicator. It just says the GPU is running kernels it doesn't say whether those kernels are useful or have good occupancy. What really ought to be of interest to you is whether your code runs fast enough to be useful to you.

[Lamya Mohammad]

Thank you, I’ll try them for the next training. For the feature function, can I use it anywhere in the code?

[Joss Knight]

You can use it anywhere as long as it is called after the first command that does something on the GPU in a MATLAB session, such as calling gpuDevice or creating a gpuArray.