Custom Training Loops

Customize deep learning training loops and loss functions for image networks

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Functions

expand all

Custom Training Loops

`dlnetwork`	Deep learning network for custom training loops
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops
`minibatchqueue`	Create mini-batches for deep learning
`dlarray`	Deep learning array for customization
`dlgradient`	Compute gradients for custom training loops using automatic differentiation
`dlfeval`	Evaluate deep learning model for custom training loops

Loss Operations

`crossentropy`	Cross-entropy loss for classification tasks
`l1loss`	L₁ loss for regression tasks
`l2loss`	L₂ loss for regression tasks
`huber`	Huber loss for regression tasks
`mse`	Half mean squared error

Deep Learning Operations

`dlconv`	Deep learning convolution
`dltranspconv`	Deep learning transposed convolution
`fullyconnect`	Sum all weighted input data and apply a bias
`batchnorm`	Normalize data across all observations for each channel independently
`crosschannelnorm`	Cross channel square-normalize using local responses
`groupnorm`	Normalize data across grouped subsets of channels for each observation independently
`instancenorm`	Normalize across each channel for each observation independently
`layernorm`	Normalize data across all channels for each observation independently
`avgpool`	Pool data to average values over spatial dimensions
`maxpool`	Pool data to maximum value
`maxunpool`	Unpool the output of a maximum pooling operation

Activations

`relu`	Apply rectified linear unit activation
`leakyrelu`	Apply leaky rectified linear unit activation
`gelu`	Apply Gaussian error linear unit (GELU) activation
`softmax`	Apply softmax activation to channel dimension
`sigmoid`	Apply sigmoid activation

Topics

Custom Training Loops

Train Deep Learning Model in MATLAB
Learn how to training deep learning models in MATLAB^®.
Define Custom Training Loops, Loss Functions, and Networks
Learn how to define and customize deep learning training loops, loss functions, and networks using automatic differentiation.
Train Network Using Custom Training Loop
This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.
Monitor Custom Training Loop Progress
Track and plot custom training loop progress.
Train Network with Multiple Outputs
This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.
Classify Videos Using Deep Learning with Custom Training Loop
This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.
Train Image Classification Network Robust to Adversarial Examples
This example shows how to train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.