# crossentropy

Cross-entropy loss for classification tasks

## Syntax

## Description

The cross-entropy operation computes the cross-entropy loss between network predictions and target values for single-label and multi-label classification tasks.

The `crossentropy`

function computes the cross-entropy loss between
predictions and targets represented as `dlarray`

data.
Using `dlarray`

objects makes working with high
dimensional data easier by allowing you to label the dimensions. For example, you can label
which dimensions correspond to spatial, time, channel, and batch dimensions using the
`"S"`

, `"T"`

, `"C"`

, and
`"B"`

labels, respectively. For unspecified and other dimensions, use the
`"U"`

label. For `dlarray`

object functions that operate
over particular dimensions, you can specify the dimension labels by formatting the
`dlarray`

object directly, or by using the `DataFormat`

option.

**Note**

To calculate the cross-entropy loss within a `layerGraph`

object
or `Layer`

array for use
with the `trainNetwork`

function, use `classificationLayer`

.

returns the categorical cross-entropy loss between the formatted `loss`

= crossentropy(`Y`

,`targets`

)`dlarray`

object `Y`

containing the predictions and the target values
`targets`

for single-label classification tasks. The output
`loss`

is an unformatted scalar `dlarray`

scalar.

For unformatted input data, use the `'DataFormat'`

option.

also specifies the dimension format `loss`

= crossentropy(___,'DataFormat',FMT)`FMT`

when `Y`

is not
a formatted `dlarray`

.

specifies options using one or more name-value pair arguments in addition to the input
arguments in previous syntaxes. For example,
`loss`

= crossentropy(___,`Name,Value`

)`'TargetCategories','independent'`

computes the cross-entropy loss for a
multi-label classification task.

## Examples

### Cross-Entropy Loss for Single-Label Classification

Create an array of prediction scores for 12 observations over 10 classes.

```
numClasses = 10;
numObservations = 12;
Y = rand(numClasses,numObservations);
dlY = dlarray(Y,'CB');
dlY = softmax(dlY);
```

View the size and format of the prediction scores.

size(dlY)

`ans = `*1×2*
10 12

dims(dlY)

ans = 'CB'

Create an array of targets encoded as one-hot vectors.

```
labels = randi(numClasses,[1 numObservations]);
targets = onehotencode(labels,1,'ClassNames',1:numClasses);
```

View the size of the targets.

size(targets)

`ans = `*1×2*
10 12

Compute the cross-entropy loss between the predictions and the targets.

loss = crossentropy(dlY,targets)

loss = 1x1 dlarray 2.3343

### Cross-Entropy Loss for Multi-Label Classification

Create an array of prediction scores for 12 observations over 10 classes.

```
numClasses = 10;
numObservations = 12;
Y = rand(numClasses,numObservations);
dlY = dlarray(Y,'CB');
```

View the size and format of the prediction scores.

size(dlY)

`ans = `*1×2*
10 12

dims(dlY)

ans = 'CB'

Create a random array of targets encoded as a numeric array of zeros and ones. Each observation can have multiple classes.

targets = rand(numClasses,numObservations) > 0.75; targets = single(targets);

View the size of the targets.

size(targets)

`ans = `*1×2*
10 12

Compute the cross-entropy loss between the predictions and the targets. To specify cross-entropy loss for multi-label classification, set the `'TargetCategories'`

option to `'independent'`

.

loss = crossentropy(dlY,targets,'TargetCategories','independent')

loss = 1x1 single dlarray 9.8853

### Weighted Cross-Entropy Loss

Create an array of prediction scores for 12 observations over 10 classes.

```
numClasses = 10;
numObservations = 12;
Y = rand(numClasses,numObservations);
dlY = dlarray(Y,'CB');
dlY = softmax(dlY);
```

View the size and format of the prediction scores.

size(dlY)

`ans = `*1×2*
10 12

dims(dlY)

ans = 'CB'

Create an array of targets encoded as one-hot vectors.

```
labels = randi(numClasses,[1 numObservations]);
targets = onehotencode(labels,1,'ClassNames',1:numClasses);
```

View the size of the targets.

size(targets)

`ans = `*1×2*
10 12

Compute the weighted cross-entropy loss between the predictions and the targets using a vector class weights. Specify a weights format of `'UC'`

(unspecified, channel) using the `'WeightsFormat'`

option.

weights = rand(1,numClasses); loss = crossentropy(dlY,targets,weights,'WeightsFormat','UC')

loss = 1x1 dlarray 1.1261

## Input Arguments

`Y`

— Predictions

`dlarray`

| numeric array

Predictions, specified as a formatted `dlarray`

, an unformatted
`dlarray`

, or a numeric array. When `Y`

is not a
formatted `dlarray`

, you must specify the dimension format using the
`DataFormat`

option.

If `Y`

is a numeric array, `targets`

must be a
`dlarray`

.

`targets`

— Target classification labels

`dlarray`

| numeric array

Target classification labels, specified as a formatted or unformatted
`dlarray`

or a numeric array.

Specify the targets as an array containing one-hot encoded labels with the same size
and format as `Y`

. For example, if `Y`

is a
`numObservations`

-by-`numClasses`

array, then
`targets(n,i)`

= 1 if observation `n`

belongs to
class `i`

`targets(n,i)`

= 0 otherwise.

If `targets`

is a formatted `dlarray`

, then its format must
be the same as the format of `Y`

, or the same as
`DataFormat`

if `Y`

is
unformatted.

If `targets`

is an unformatted `dlarray`

or a numeric array,
then the function applies the format of `Y`

or the value of
`DataFormat`

to `targets`

.

**Tip**

Formatted `dlarray`

objects automatically permute the dimensions of the
underlying data to have order `"S"`

(spatial),
`"C"`

(channel), `"B"`

(batch),
`"T"`

(time), then `"U"`

(unspecified). To ensure that the dimensions of `Y`

and
`targets`

are consistent, when
`Y`

is a formatted `dlarray`

, also
specify `targets`

as a formatted
`dlarray`

.

`weights`

— Weights

`dlarray`

| numeric array

Weights, specified as a `dlarray`

or a numeric array.

To specify class weights, specify a vector with a `'C'`

(channel)
dimension with size matching the `'C'`

(channel) dimension of the
`X`

. Specify the `'C'`

(channel) dimension of the
class weights by using a formatted `dlarray`

object or by using the
`'WeightsFormat'`

option.

To specify observation weights, specify a vector with a `'B'`

(batch)
dimension with size matching the `'B'`

(batch) dimension of the
`Y`

. Specify the `'B'`

(batch) dimension of the
class weights by using a formatted `dlarray`

object or by using the
`'WeightsFormat'`

option.

To specify weights for each element of the input independently, specify the weights as an
array of the same size as `Y`

. In this case, if
`weights`

is not a formatted `dlarray`

object, then
the function uses the same format as `Y`

. Alternatively, specify the
weights format using the `'WeightsFormat'`

option.

### Name-Value Arguments

Specify optional pairs of arguments as
`Name1=Value1,...,NameN=ValueN`

, where `Name`

is
the argument name and `Value`

is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.

*
Before R2021a, use commas to separate each name and value, and enclose*
`Name`

*in quotes.*

**Example: **`'TargetCategories','independent','DataFormat','CB'`

evaluates
the cross-entropy loss for multi-label classification tasks and specifies the dimension
order of the input data as `'CB'`

`TargetCategories`

— Type of classification task

`'exclusive'`

(default) | `'independent'`

Type of classification task, specified as the comma-separated pair consisting of
`'TargetCategories'`

and one of the following:

`'exclusive'`

— Single-label classification. Each observation in the predictions`Y`

is exclusively assigned to one category. The function computes the loss between the target value for the single category specified by`targets`

and the corresponding prediction in`Y`

, averaged over the number of observations.`'independent'`

— Multi-label classification. Each observation in the predictions`Y`

can be assigned to one or more independent categories. The function computes the sum of the loss between each category specified by`targets`

and the predictions in`Y`

for those categories, averaged over the number of observations. Cross-entropy loss for this type of classification task is also known as binary cross-entropy loss.

`Mask`

— Mask indicating which elements to include for loss computation

`dlarray`

| logical array | numeric array

Mask indicating which elements to include for loss computation, specified as a
`dlarray`

object, a logical array, or a numeric array with the same
size as `Y`

.

The function includes and excludes elements of the input data for loss computation when the corresponding value in the mask is 1 and 0, respectively.

If `Mask`

is a formatted `dlarray`

object, then its
format must match that of `Y`

. If `Mask`

is not a
formatted `dlarray`

object, then the function uses the same format as
`Y`

.

If you specify the `DataFormat`

option, then the function also uses
the specified format for the mask.

The size of each dimension of `Mask`

must match the size of the
corresponding dimension in `Y`

. The default value is a logical array
of ones.

**Tip**

Formatted `dlarray`

objects automatically permute the dimensions of the
underlying data to have this order: `"S"`

(spatial), `"C"`

(channel), `"B"`

(batch), `"T"`

(time), and
`"U"`

(unspecified). For example, `dlarray`

objects
automatically permute the dimensions of data with format `"TSCSBS"`

to have
format `"SSSCBT"`

.

To ensure that the dimensions of `Y`

and the mask are consistent, when
`Y`

is a formatted `dlarray`

, also specify the mask as
a formatted `dlarray`

.

`Reduction`

— Mode for reducing array of loss values

`"sum"`

(default) | `"none"`

Mode for reducing the array of loss values, specified as one of the following:

`NormalizationFactor`

— Divisor for normalizing reduced loss

`"batch-size"`

(default) | `"all-elements"`

| `"mask-included"`

| `"none"`

Divisor for normalizing the reduced loss when `Reduction`

is
`"sum"`

, specified as one of the following:

`"batch-size"`

— Normalize the loss by dividing it by the number of observations in`X`

.`"all-elements"`

— Normalize the loss by dividing it by the number of elements of`X`

.`"mask-included"`

— Normalize the loss by dividing the loss values by the number of included elements specified by the mask for each observation independently. To use this option, you must specify a mask using the`Mask`

option.`"none"`

— Do not normalize the loss.

`DataFormat`

— Dimension order of unformatted data

character vector | string scalar

Dimension order of unformatted input data, specified as a character vector or string
scalar `FMT`

that provides a label for each dimension of the data.

When you specify the format of a `dlarray`

object, each character provides a
label for each dimension of the data and must be one of the following:

`"S"`

— Spatial`"C"`

— Channel`"B"`

— Batch (for example, samples and observations)`"T"`

— Time (for example, time steps of sequences)`"U"`

— Unspecified

You can specify multiple dimensions labeled `"S"`

or
`"U"`

. You can use the labels `"C"`

,
`"B"`

, and `"T"`

at most once.

You must specify `DataFormat`

when the input data is not a
formatted `dlarray`

.

**Data Types: **`char`

| `string`

`WeightsFormat`

— Dimension order of weights

character vector | string scalar

Dimension order of the weights, specified as a character vector or string scalar that provides a label for each dimension of the weights.

When you specify the format of a `dlarray`

object, each character provides a
label for each dimension of the data and must be one of the following:

`"S"`

— Spatial`"C"`

— Channel`"B"`

— Batch (for example, samples and observations)`"T"`

— Time (for example, time steps of sequences)`"U"`

— Unspecified

You can specify multiple dimensions labeled `"S"`

or
`"U"`

. You can use the labels
`"C"`

, `"B"`

, and
`"T"`

at most once.

You must specify `WeightsFormat`

when
`weights`

is a numeric vector and
`Y`

has two or more nonsingleton
dimensions.

If `weights`

is not a vector, or both
`weights`

and
`Y`

are vectors, then default
value of `WeightsFormat`

is the same as the
format of `Y`

.

**Data Types: **`char`

| `string`

## Output Arguments

`loss`

— Cross-entropy loss

`dlarray`

Cross-entropy loss, returned as an unformatted `dlarray`

. The
output `loss`

is an unformatted `dlarray`

with the
same underlying data type as the input `Y`

.

The size of `loss`

depends on the
`'Reduction'`

option.

## Algorithms

### Cross-Entropy Loss

For each element *Y _{j}* of the input, the

`crossentropy`

function computes the corresponding cross-entropy
element-wise loss values using the formula$${\text{loss}}_{j}=-\left({T}_{j}\text{ln}{Y}_{j}+(1-{T}_{j})\text{ln}(1-{Y}_{j})\right),$$

where *T _{j}* is the corresponding
target value to

*Y*.

_{j}To reduce the loss values to a scalar, the function then reduces the element-wise loss using the formula

$$\text{loss}=\frac{1}{N}{\displaystyle \sum _{j}{m}_{j}{w}_{j}{\text{loss}}_{j},}$$

where *N* is the normalization factor,
*m _{j}* is the mask value for element

*j*, and

*w*is the weight value for element

_{j}*j*.

If you do not opt to reduce the loss, then the function applies the mask and the weights to the loss values directly:

$${\text{loss}}_{j}^{*}={m}_{j}{w}_{j}{\text{loss}}_{j}$$

This table shows the loss formulations for different tasks.

Task | Description | Loss |
---|---|---|

Single-label classification | Cross-entropy loss for mutually exclusive classes. This is useful when observations must have a single label only. |
$$\text{loss}=-\frac{1}{N}{\displaystyle \sum _{n=1}^{N}{\displaystyle \sum}_{i=1}^{K}}{T}_{ni}\text{ln}{Y}_{ni},$$ where |

Multi-label classification | Cross-entropy loss for independent classes. This is useful when observations can have multiple labels. |
$$\text{loss}=-\frac{1}{N}{\displaystyle \sum _{n=1}^{N}{\displaystyle \sum _{i=1}^{K}\left({T}_{ni}\mathrm{ln}({Y}_{ni})+(1-{T}_{ni})\mathrm{ln}(1-{Y}_{ni})\right)}},$$ where |

Single-label classification with weighted classes | Cross-entropy loss with class weights. This is useful for datasets with imbalanced classes. |
$$\text{loss}=-\frac{1}{N}{\displaystyle \sum _{n=1}^{N}{\displaystyle \sum}_{i=1}^{K}}{w}_{i}{T}_{ni}\text{ln}{Y}_{ni},$$ where i. |

Sequence-to-sequence classification | Cross-entropy loss with masked time-steps. This is useful for ignoring loss values that correspond to padded data. |
$$\text{loss}=-\frac{1}{N}{\displaystyle \sum _{n=1}^{N}{\displaystyle \sum}_{t=1}^{S}{m}_{nt}{\displaystyle \sum}_{i=1}^{K}}{T}_{nti}\text{ln}{Y}_{nti},$$ where t of observation
n. |

## Extended Capabilities

### GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

Usage notes and limitations:

When at least one of the following input arguments is a

`gpuArray`

or a`dlarray`

with underlying data of type`gpuArray`

, this function runs on the GPU:`Y`

`targets`

`weights`

`'Mask'`

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

## Version History

**Introduced in R2019b**

## Open Example

You have a modified version of this example. Do you want to open this example with your edits?

## MATLAB Command

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.

# Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list:

## How to Get Best Site Performance

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

### Americas

- América Latina (Español)
- Canada (English)
- United States (English)

### Europe

- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)

- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)