softmaxLayer

Softmax layer

Description

A softmax layer applies a softmax function to the input.

Creation

Syntax

layer = softmaxLayer

layer = softmaxLayer('Name',Name)

Description

layer = softmaxLayer creates a softmax layer.

layer = softmaxLayer('Name',Name) creates a softmax layer and sets the optional Name property using a name-value pair. For example, softmaxLayer('Name','sm1') creates a softmax layer with the name 'sm1'. Enclose the property name in single quotes.

Properties

Name — Layer name

'' (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. For Layer array input, the trainNetwork, assembleNetwork, layerGraph, and dlnetwork functions automatically assign names to layers with the name ''.

Data Types: char | string

NumInputs — Number of inputs

1 (default)

This property is read-only.

Number of inputs of the layer. This layer accepts a single input only.

Data Types: double

InputNames — Input names

{"in"} (default)

This property is read-only.

Input names of the layer. This layer accepts a single input only.

Data Types: cell

NumOutputs — Number of outputs

1 (default)

This property is read-only.

Number of outputs of the layer. This layer has a single output only.

Data Types: double

OutputNames — Output names

{'out'} (default)

This property is read-only.

Output names of the layer. This layer has a single output only.

Data Types: cell

Examples

Create Softmax Layer

Create a softmax layer with the name 'sm1'.

layer = softmaxLayer('Name','sm1')

layer = 
  SoftmaxLayer with properties:

    Name: 'sm1'

Include a softmax layer in a Layer array.

layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer]

layers = 
  7x1 Layer array with layers:

     1   ''   Image Input             28x28x1 images with 'zerocenter' normalization
     2   ''   2-D Convolution         20 5x5 convolutions with stride [1  1] and padding [0  0  0  0]
     3   ''   ReLU                    ReLU
     4   ''   2-D Max Pooling         2x2 max pooling with stride [2  2] and padding [0  0  0  0]
     5   ''   Fully Connected         10 fully connected layer
     6   ''   Softmax                 softmax
     7   ''   Classification Output   crossentropyex

Algorithms

Softmax Layer

A softmax layer applies a softmax function to the input.

For classification problems, a softmax layer and then a classification layer usually follow the final fully connected layer.

The output unit activation function is the softmax function:

$$y_r\left(x\right)=\frac{\exp \left(a_r\left(x\right)\right)}{{\displaystyle \sum _{j=1}^k\exp \left(a_j\left(x\right)\right)}},$$

where $$0\le y_r\le 1$$ and $${\displaystyle \sum _{j=1}^k{y}_j=1}$$.

The softmax function is the output unit activation function after the last fully connected layer for multi-class classification problems:

$$P\left(c_r|x,\theta \right)=\frac{P\left(x,\theta |c_r\right)P\left(c_r\right)}{{\displaystyle \sum _{j=1}^{k}P\left(x,\theta |c_j\right)P\left(c_j\right)}}=\frac{\exp \left(a_r\left(x,\theta \right)\right)}{{\displaystyle \sum _{j=1}^k\exp \left(a_j\left(x,\theta \right)\right)}},$$

where $$0\le P\left(c_r|x,\theta \right)\le 1$$ and $${\displaystyle \sum _{j=1}^{k}P\left(c_j|x,\theta \right)=1}$$. Moreover, $$a_r=\ln \left(P\left(x,\theta |c_r\right)P\left(c_r\right)\right)$$, $$P\left(x,\theta |c_r\right)$$ is the conditional probability of the sample given class r, and $$P\left(c_r\right)$$ is the class prior probability.

The softmax function is also known as the normalized exponential and can be considered the multi-class generalization of the logistic sigmoid function [1].

Layer Input and Output Formats

Layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters, in which each character describes the corresponding dimension of the data. The formats consists of one or more of these characters:

• "S" — Spatial

• "C" — Channel

• "B" — Batch

• "T" — Time

• "U" — Unspecified

For example, 2-D image data represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, can be described as having the format "SSCB" (spatial, spatial, channel, batch).

You can interact with these dlarray objects in automatic differentiation workflows such as developing a custom layer, using a functionLayer object, or using the forward and predict functions with dlnetwork objects.

This table shows the supported input formats of SoftmaxLayer objects and the corresponding output format. If the output of the layer is passed to a custom layer that does not inherit from the nnet.layer.Formattable class, or a FunctionLayer object with the Formattable property set to 0 (false), then the layer receives an unformatted dlarray object with dimensions ordered corresponding to the formats in this table.

Input Format	Output Format
"CB" (channel, batch)	"CB" (channel, batch)
"SCB" (spatial, channel, batch)	"SCB" (spatial, channel, batch)
"SSCB" (spatial, spatial, channel, batch)	"SSCB" (spatial, spatial, channel, batch)
"SSSCB" (spatial, spatial, spatial, channel, batch)	"SSSCB" (spatial, spatial, spatial, channel, batch)
"CBT" (channel, batch, time)	"CBT" (channel, batch, time)
"SCBT" (spatial, channel, batch, time)	"SCBT" (spatial, channel, batch, time)
"SSCBT" (spatial, spatial, channel, batch, time)	"SSCBT" (spatial, spatial, channel, batch, time)
"SSSCBT" (spatial, spatial, spatial, channel, batch, time)	"SSSCBT" (spatial, spatial, spatial, channel, batch, time)
"CU" (channel, unspecified)	"CU" (channel, unspecified)
"SC" (spatial, channel)	"SC" (spatial, channel)
"SSC" (spatial, spatial, channel)	"SSC" (spatial, spatial, channel)
"SSSC" (spatial, spatial, spatial, channel)	"SSSC" (spatial, spatial, spatial, channel)

In dlnetwork objects, SoftmaxLayer objects also support these input and output format combinations.

Input Format	Output Format
"CT" (channel, time)	"CT" (channel, time)
"SCT" (spatial, channel, time)	"SCT" (spatial, channel, time)
"SSCT" (spatial, spatial, channel, time)	"SSCT" (spatial, spatial, channel, time)
"SSSCT" (spatial, spatial, spatial, channel, time)	"SSSCT" (spatial, spatial, spatial, channel, time)

References

[1] Bishop, C. M. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Version History

Introduced in R2016a