Why are my MATLAB Parallel Server worker licenses not being released after a job completes or why are my workers continually running when no job is submitted?

I have an MATLAB Parallel Server cluster using MATLAB Job Scheduler. After submitting a job and receiving the results, the MATLAB Parallel Server worker licenses stay checked out when they should have been returned. This results in a License Manager error -4 when another user attempts to submit to the cluster. How can I release these licenses back to the network license manager?

Answer 1

[MathWorks Support Team]

When using the MATLAB Job Scheduler, job information is stored on the worker nodes in the form various log files. These log files sometimes cause worker licenses to get stuck in the checked out state. This could result in a License Manager Error -4 which indicates no licenses are available for the workers because they are erroneously checked out. Since the log files are stored locally on the worker nodes, this issue can occur on all the worker nodes or only affect a few of the worker nodes. 

In order to resolve the issue please follow the steps below on each of the worker nodes. 

• Navigate to $MATLAB\toolbox\parallel\bin
• Run MJS stop -clean
• Run MJS start -clean 

Once finished, stop and then start the License Manager service. This process will restart the MATLAB Parallel Server services on each of the worker nodes and clear out any left over log files as well as reset the currently checked out licenses. Once completed, reconfigure the job scheduler and start the workers using the Admin Center. Proceed with use of the cluster. You should no longer have issues with stuck licenses. 

If you are still unable to resolve the issue by the steps above, contact the Installation and Licensing Team by creating a Service Request here:

https://www.mathworks.com/support/contact_us.html

NOTE: Starting in R2019a the following changes occurred:

• MATLAB Distributed Computing Server was renamed to MATLAB Parallel Server 
• mdce_def was renamed to mjs_def
• mdce binary was renamed to mjs
• mjs scripts are in $MATLAB/R20XXx/toolbox/distcomp/bin for R2019a and earlier