Main Content

If Deep Learning Toolbox™ does not provide the layer you require for your classification or regression problem, then you can define your own custom layer. For a list of built-in layers, see List of Deep Learning Layers.

The example Define Custom Classification Output Layer shows how to define and create a custom classification output layer with sum of squares error (SSE) loss and goes through the following steps:

Name the layer – Give the layer a name so it can be used in MATLAB

^{®}.Declare the layer properties – Specify the properties of the layer.

Create a constructor function (optional) – Specify how to construct the layer and initialize its properties. If you do not specify a constructor function, then the software initializes the properties with

`''`

at creation.Create a forward loss function – Specify the loss between the predictions and the training targets.

Create a backward loss function (optional) – Specify the derivative of the loss with respect to the predictions. If you do not specify a backward loss function, then the forward loss function must support

`dlarray`

objects.

Creating a backward loss function is optional. If the forward loss function only uses
functions that support `dlarray`

objects, then software determines the
derivatives automatically using automatic differentiation. For a list of functions that
support `dlarray`

objects, see List of Functions with dlarray Support. If you want to use
functions that do not support `dlarray`

objects, or want to use a specific
algorithm for the backward loss function, then you can define a custom backward function
using this example as a guide.

The example Define Custom Classification Output Layer shows how to create a SSE classification layer.

A classification SSE layer computes the sum of squares error
loss for classification problems. SSE is an error measure between two continuous random variables. For predictions
*Y* and training targets *T*, the SSE loss between
*Y* and *T* is given by

$$L=\frac{1}{N}{\displaystyle \sum}_{n=1}^{N}\text{}{\displaystyle \sum}_{i=1}^{K}\text{}{({Y}_{ni}-{T}_{ni})}^{2},$$

where *N* is the number of observations and
*K* is the number of classes.

View the layer created in the example Define Custom Classification Output Layer. This layer does not
have a `backwardLoss`

function.

classdef sseClassificationLayer < nnet.layer.ClassificationLayer % Example custom classification layer with sum of squares error loss. methods function layer = sseClassificationLayer(name) % layer = sseClassificationLayer(name) creates a sum of squares % error classification layer and specifies the layer name. % Set layer name. layer.Name = name; % Set layer description. layer.Description = 'Sum of squares error'; end function loss = forwardLoss(layer, Y, T) % loss = forwardLoss(layer, Y, T) returns the SSE loss between % the predictions Y and the training targets T. % Calculate sum of squares. sumSquares = sum((Y-T).^2); % Take mean over mini-batch. N = size(Y,4); loss = sum(sumSquares)/N; end end end

Implement the `backwardLoss`

function that returns the derivatives of
the loss with respect to the input data and the learnable parameters.

The syntax for `backwardLoss`

is ```
dLdY
= backwardLoss(layer, Y, T)
```

. The input `Y`

contains the
predictions made by the network and `T`

contains the training targets. The
output `dLdY`

is the derivative of the loss with respect to the predictions
`Y`

. The output `dLdY`

must be the same size as the layer
input `Y`

.

The dimensions of `Y`

and `T`

are the same as the
inputs in `forwardLoss`

.

The derivative of the SSE loss with respect to the predictions *Y* is
given by

$$\frac{\delta L}{\delta {Y}_{i}}=\frac{2}{N}({Y}_{i}-{T}_{i}),$$

where *N* is the number of observations in the
input.

Create the backward loss function that returns these derivatives.

```
function dLdY = backwardLoss(layer, Y, T)
% dLdY = backwardLoss(layer, Y, T) returns the derivatives of
% the SSE loss with respect to the predictions Y.
N = size(Y,4);
dLdY = 2*(Y-T)/N;
end
```

View the completed layer class file.

classdef sseClassificationLayer < nnet.layer.ClassificationLayer % Example custom classification layer with sum of squares error loss. methods function layer = sseClassificationLayer(name) % layer = sseClassificationLayer(name) creates a sum of squares % error classification layer and specifies the layer name. % Set layer name. layer.Name = name; % Set layer description. layer.Description = 'Sum of squares error'; end function loss = forwardLoss(layer, Y, T) % loss = forwardLoss(layer, Y, T) returns the SSE loss between % the predictions Y and the training targets T. % Calculate sum of squares. sumSquares = sum((Y-T).^2); % Take mean over mini-batch. N = size(Y,4); loss = sum(sumSquares)/N; end function dLdY = backwardLoss(layer, Y, T) % dLdY = backwardLoss(layer, Y, T) returns the derivatives of % the SSE loss with respect to the predictions Y. N = size(Y,4); dLdY = 2*(Y-T)/N; end end end

If the layer forward functions fully support `dlarray`

objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type `gpuArray`

(Parallel Computing Toolbox).

Many MATLAB built-in functions support `gpuArray`

(Parallel Computing Toolbox) and `dlarray`

input arguments. For a list of
functions that support `dlarray`

objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a supported GPU device. For information on supported devices, see
GPU Support by Release (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).

- Define Custom Deep Learning Layers
- Define Custom Deep Learning Layer with Learnable Parameters
- Define Custom Deep Learning Layer with Multiple Inputs
- Define Nested Deep Learning Layer
- Define Custom Classification Output Layer
- Define Custom Regression Output Layer
- Specify Custom Layer Backward Function
- Check Custom Layer Validity
- List of Deep Learning Layers
- Deep Learning Tips and Tricks