How to check if the parallel server GPU can handle a particular network

I am running through a hyperparameter sweep of different network parameters. These include the number of LSTM layers, the number of hidden units in each layer, as well as a shape parameter as I call it (have the number of hidden units increase, decrease, or remain the same with subsequent layers).

I fully expect some of the setting to be too large to use the gpu. And I see this with the error returned regarding gpu memory.

What I would like to do is the following. Have a function compute how much gpu memory a network will take by looking at the layers, and then update the training options to use the cpu if it is too large. See the pseudo-code below where 'i' represent one set of hyperparameters.

How can I do this on my machine? Also, how can I do this on MATLAB Parallel Server?

if GPUmemory<NETmemory(layers{i})

trainingoptions{i}= ... 'ExecutionEnvironment','cpu' ...

else

trainingoptions{i}= ... 'ExecutionEnvironment','gpu' ...

end