Experiment Manager

Design and run experiments to train and compare machine learning models

Since R2023a

Description

You can use the Experiment Manager app to create machine learning experiments to train models under different training conditions and compare the results. For example, you can use Experiment Manager to:

• Try a range of hyperparameter values using Bayesian optimization.

• Compare the results of using different data sets, preprocessing steps, or metrics.

To set up your experiment quickly, start by training a model in Classification Learner or Regression Learner. Then, export the model to Experiment Manager.

The Experiment Browser panel displays the hierarchy of experiments and results in a project. The icon next to the experiment name indicates its type.

• — Built-in training experiment that uses the trainnet training function

• — Custom training experiment that uses a custom training function

• — General-purpose experiment that uses a user-authored experiment function

This page contains information about using Experiment Manager with the Classification Learner and Regression Learner apps. For general information about using the app, see Experiment Manager. For information about built-in and custom training experiments, see Experiment Manager (Deep Learning Toolbox).

More

Required Products

• Use Deep Learning Toolbox™ to run built-in or custom training experiments for deep learning and to view confusion matrices for these experiments.

• Use Statistics and Machine Learning Toolbox™ to run custom training experiments for machine learning and experiments that use Bayesian optimization.

• Use Parallel Computing Toolbox™ to run multiple trials at the same time or a single trial on multiple GPUs, on a cluster, or in the cloud. For more information, see Run Experiments in Parallel (Deep Learning Toolbox).

• Use MATLAB^® Parallel Server™ to offload experiments as batch jobs in a remote cluster. For more information, see Offload Experiments as Batch Jobs to a Cluster (Deep Learning Toolbox).

×

Open the Experiment Manager App

• MATLAB Toolstrip: On the Apps tab, under MATLAB, click the Experiment Manager icon.

• MATLAB command prompt: Enter experimentManager.

For general information about using the app, see Experiment Manager.

Examples

Autogenerate Experiment From Classification or Regression Learner App

To set up your experiment quickly, start by training a model in Classification Learner or Regression Learner. Then, export the model to Experiment Manager.

Train a model in the Classification Learner or Regression Learner app. Then, with the model selected in the Models panel, click Export Model in the app toolstrip and select Create Experiment. The app autogenerates an experiment in Experiment Manager to tune the hyperparameters of the selected model.

After you click Create Experiment, use the dialog boxes to confirm the training data set filename and to choose a project for the experiment.

For more information, see Export Model from Classification Learner to Experiment Manager or Export Model from Regression Learner to Experiment Manager.

Related Examples

• Export Model from Classification Learner to Experiment Manager
• Tune Classification Model Using Experiment Manager
• Export Model from Regression Learner to Experiment Manager
• Tune Regression Model Using Experiment Manager

Version History

Introduced in R2023a

R2023b: App available in MATLAB

You can now use Experiment Manager in MATLAB, with or without Statistics and Machine Learning Toolbox. When you share your experiments with colleagues who do not have a Statistics and Machine Learning Toolbox license, they can open your experiments and access your results. Experiment Manager requires:

• Deep Learning Toolbox to run built-in or custom training experiments for deep learning and to view confusion matrices for these experiments

• Statistics and Machine Learning Toolbox to run custom training experiments for machine learning and experiments that use Bayesian optimization

• Parallel Computing Toolbox to run multiple trials at the same time or a single trial at a time on multiple GPUs, on a cluster, or in the cloud

• MATLAB Parallel Server to offload experiments as batch jobs in a remote cluster

R2023a: App available in Statistics and Machine Learning Toolbox

You can now use Experiment Manager when you have a Statistics and Machine Learning Toolbox license. In previous releases, you needed a Deep Learning Toolbox license to use the Experiment Manager app. For more information on how to use the app for deep learning workflows, see Manage Experiments (Deep Learning Toolbox).