huber
Syntax
Description
The Huber operation computes the Huber loss between network predictions and target values for regression tasks. When the 'TransitionPoint'
option is 1, this is also known as smooth L1 loss.
The huber
function calculates the Huber loss using dlarray
data.
Using dlarray
objects makes working with high
dimensional data easier by allowing you to label the dimensions. For example, you can label
which dimensions correspond to spatial, time, channel, and batch dimensions using the
"S"
, "T"
, "C"
, and
"B"
labels, respectively. For unspecified and other dimensions, use the
"U"
label. For dlarray
object functions that operate
over particular dimensions, you can specify the dimension labels by formatting the
dlarray
object directly, or by using the DataFormat
option.
returns the Huber loss between the formatted loss
= huber(Y
,targets
)dlarray
object
Y
containing the predictions and the target values
targets
for regression tasks. The input Y
is a
formatted dlarray
. The output loss
is an unformatted
dlarray
scalar.
For unformatted input data, use the 'DataFormat'
option.
also specifies the dimension format loss
= huber(___,'DataFormat',FMT)FMT
when Y
is not
a formatted dlarray
.
specifies options using one or more name-value pair arguments in addition to the input
arguments in previous syntaxes. For example,
loss
= huber(___,Name,Value
)'NormalizationFactor','all-elements'
specifies to normalize the loss by
dividing the reduced loss by the number of input elements.
Examples
Huber Loss
Create an array of predictions for 12 observations over 10 responses.
numResponses = 10;
numObservations = 12;
Y = rand(numResponses,numObservations);
dlY = dlarray(Y,'CB');
View the size and format of the predictions.
size(dlY)
ans = 1×2
10 12
dims(dlY)
ans = 'CB'
Create an array of random targets.
targets = rand(numResponses,numObservations);
View the size of the targets.
size(targets)
ans = 1×2
10 12
Compute the Huber loss between the predictions and the targets.
loss = huber(dlY,targets)
loss = 1x1 dlarray 0.7374
Masked Huber Loss for Padded Sequences
Create arrays of predictions and targets for 12 sequences of varying lengths over 10 responses.
numResponses = 10; numObservations = 12; maxSequenceLength = 15; sequenceLengths = randi(maxSequenceLength,[1 numObservations]); Y = cell(numObservations,1); targets = cell(numObservations,1); for i = 1:numObservations Y{i} = rand(numResponses,sequenceLengths(i)); targets{i} = rand(numResponses,sequenceLengths(i)); end
View the cell arrays of predictions and targets.
Y
Y=12×1 cell array
{10x13 double}
{10x14 double}
{10x2 double}
{10x14 double}
{10x10 double}
{10x2 double}
{10x5 double}
{10x9 double}
{10x15 double}
{10x15 double}
{10x3 double}
{10x15 double}
targets
targets=12×1 cell array
{10x13 double}
{10x14 double}
{10x2 double}
{10x14 double}
{10x10 double}
{10x2 double}
{10x5 double}
{10x9 double}
{10x15 double}
{10x15 double}
{10x3 double}
{10x15 double}
Pad the prediction and target sequences in the second dimension using the padsequences
function and also return the corresponding mask.
[Y,mask] = padsequences(Y,2); targets = padsequences(targets,2);
Convert the padded sequences to dlarray
with format 'CTB'
(channel, time, batch). Because formatted dlarray
objects automatically sort the dimensions, keep the dimensions of the targets and mask consistent by also converting them to a formatted dlarray
objects with the same formats.
dlY = dlarray(Y,'CTB'); targets = dlarray(targets,'CTB'); mask = dlarray(mask,'CTB');
View the sizes of the prediction scores, targets, and the mask.
size(dlY)
ans = 1×3
10 12 15
size(targets)
ans = 1×3
10 12 15
size(mask)
ans = 1×3
10 12 15
Compute the Huber loss between the predictions and the targets. To prevent the loss values calculated from padding from contributing to the loss, set the 'Mask'
option to the mask returned by the padsequences
function.
loss = huber(dlY,targets,'Mask',mask)
loss = 1x1 dlarray 8.1834
Input Arguments
Y
— Predictions
dlarray
object | numeric array
Predictions, specified as a formatted or unformatted dlarray
object,
or a numeric array. When Y
is not a formatted
dlarray
, you must specify the dimension format using the
DataFormat
argument.
If Y
is a numeric array, targets
must be a
dlarray
object.
targets
— Target responses
dlarray
| numeric array
Target responses, specified as a formatted or unformatted dlarray
or a
numeric array.
The size of each dimension of targets
must match the size of the
corresponding dimension of Y
.
If targets
is a formatted dlarray
, then its format must
be the same as the format of Y
, or the same as
DataFormat
if Y
is
unformatted.
If targets
is an unformatted dlarray
or a numeric array,
then the function applies the format of Y
or the value of
DataFormat
to targets
.
Tip
Formatted dlarray
objects automatically permute the dimensions of the
underlying data to have the order "S"
(spatial), "C"
(channel), "B"
(batch), "T"
(time), then
"U"
(unspecified). To ensure that the dimensions of
Y
and targets
are consistent, when
Y
is a formatted dlarray
, also specify
targets
as a formatted dlarray
.
weights
— Weights
dlarray
| numeric array
Weights, specified as a dlarray
or a numeric array.
To specify response weights, specify a vector with a 'C'
(channel) dimension with size matching the 'C'
(channel) dimension of
the Y
. Specify the 'C'
(channel) dimension of
the response weights by using a formatted dlarray
object or by using
the 'WeightsFormat'
option.
To specify observation weights, specify a vector with a "B"
(batch)
dimension with size matching the "B"
(batch) dimension of
Y
. Specify the "B"
(batch) dimension of the
class weights by using a formatted dlarray
object or by using the
WeightsFormat
argument.
To specify weights for each element of the input independently, specify the weights as an
array of the same size as Y
. In this case, if
weights
is not a formatted dlarray
object, then
the function uses the same format as Y
. Alternatively, specify the
weights format using the WeightsFormat
argument.
Name-Value Arguments
Specify optional pairs of arguments as
Name1=Value1,...,NameN=ValueN
, where Name
is
the argument name and Value
is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.
Before R2021a, use commas to separate each name and value, and enclose
Name
in quotes.
Example: 'NormalizationFactor','all-elements'
specifies to normalize
the loss by dividing the reduced loss by the number of input elements
TransitionPoint
— Point where Huber loss transitions to a linear function
1 (default) | positive scalar
Point where Huber loss transitions from a quadratic function to a linear function,
specified as the comma-separated pair consisting of
'TransitionPoint'
and a positive scalar.
When 'TransitionPoint'
is 1, this is also known as
smooth L1
loss.
Data Types: single
| double
| int8
| int16
| int32
| int64
| uint8
| uint16
| uint32
| uint64
Mask
— Mask indicating which elements to include for loss computation
dlarray
| logical array | numeric array
Mask indicating which elements to include for loss computation, specified as a
dlarray
object, a logical array, or a numeric array with the same
size as Y
.
The function includes and excludes elements of the input data for loss computation when the corresponding value in the mask is 1 and 0, respectively.
If Mask
is a formatted dlarray
object, then its
format must match that of Y
. If Mask
is not a
formatted dlarray
object, then the function uses the same format as
Y
.
If you specify the DataFormat
argument, then the function also
uses the specified format for the mask.
The size of each dimension of Mask
must match the size of the
corresponding dimension in Y
. The default value is a logical array
of ones.
Tip
Formatted dlarray
objects automatically permute the dimensions of the
underlying data to have this order: "S"
(spatial), "C"
(channel), "B"
(batch), "T"
(time), and
"U"
(unspecified). For example, dlarray
objects
automatically permute the dimensions of data with format "TSCSBS"
to have
format "SSSCBT"
.
To ensure that the dimensions of Y
and the mask are consistent, when
Y
is a formatted dlarray
, also specify the mask as
a formatted dlarray
.
Reduction
— Loss value array reduction mode
"sum"
(default) | "none"
Loss value array reduction mode, specified as "sum"
or
"none"
.
If the Reduction
argument is "sum"
, then the function
sums all elements in the array of loss values. In this case, the output
loss
is a scalar.
If the Reduction
argument is "none"
, then the
function does not reduce the array of loss values. In this case, the output
loss
is an unformatted dlarray
object
of the same size as Y
.
NormalizationFactor
— Divisor for normalizing reduced loss
"batch-size"
(default) | "all-elements"
| "mask-included"
| "none"
Divisor for normalizing the reduced loss when Reduction
is
"sum"
, specified as one of the following:
"batch-size"
— Normalize the loss by dividing it by the number of observations inY
."all-elements"
— Normalize the loss by dividing it by the number of elements ofY
."mask-included"
— Normalize the loss by dividing the loss values by the product of the number of observations and the number of included elements specified by the mask for each observation independently. To use this option, you must specify a mask using theMask
option."none"
— Do not normalize the loss.
DataFormat
— Description of data dimensions
character vector | string scalar
Description of the data dimensions, specified as a character vector or string scalar.
A data format is a string of characters, where each character describes the type of the corresponding data dimension.
The characters are:
"S"
— Spatial"C"
— Channel"B"
— Batch"T"
— Time"U"
— Unspecified
For example, consider an array containing a batch of sequences where the first, second,
and third dimensions correspond to channels, observations, and time steps, respectively. You
can specify that this array has the format "CBT"
(channel, batch,
time).
You can specify multiple dimensions labeled "S"
or "U"
.
You can use the labels "C"
, "B"
, and
"T"
once each, at most. The software ignores singleton trailing
"U"
dimensions after the second dimension.
If the input data is not a formatted dlarray
object, then you must
specify the DataFormat
option.
For more information, see Deep Learning Data Formats.
Data Types: char
| string
WeightsFormat
— Description of dimensions of weights
character vector | string scalar
Description of the dimensions of the weights, specified as a character vector or string scalar.
A data format is a string of characters, where each character describes the type of the corresponding data dimension.
The characters are:
"S"
— Spatial"C"
— Channel"B"
— Batch"T"
— Time"U"
— Unspecified
For example, consider an array containing a batch of sequences where the first, second,
and third dimensions correspond to channels, observations, and time steps, respectively. You
can specify that this array has the format "CBT"
(channel, batch,
time).
You can specify multiple dimensions labeled "S"
or "U"
.
You can use the labels "C"
, "B"
, and
"T"
once each, at most. The software ignores singleton trailing
"U"
dimensions after the second dimension.
If weights
is a numeric vector and
Y
has two or more nonsingleton
dimensions, then you must specify the
WeightsFormat
option.
If weights
is not a vector, or
weights
and
Y
are both vectors, then the
default value of WeightsFormat
is the same
as the format of Y
.
For more information, see Deep Learning Data Formats.
Data Types: char
| string
Output Arguments
loss
— Huber loss
dlarray
Huber loss, returned as an unformatted dlarray
. The output
loss
is an unformatted dlarray
with the same
underlying data type as the input Y
.
The size of loss
depends on the Reduction
option.
Algorithms
Huber Loss
For each element Yj of the input, the
huber
function computes the corresponding element-wise loss values
using the formula
where Tj is the corresponding target value to the prediction Yj and is the transition point where the loss transitions from a quadratic function to a linear function.
When the transition point is 1, this is also known as smooth L1 loss.
To reduce the loss values to a scalar, the function then reduces the element-wise loss using the formula
where N is the normalization factor, mj is the mask value for element j, and wj is the weight value for element j.
If you do not opt to reduce the loss, then the function applies the mask and the weights to the loss values directly:
Deep Learning Array Formats
Most deep learning networks and functions operate on different dimensions of the input data in different ways.
For example, an LSTM operation iterates over the time dimension of the input data, and a batch normalization operation normalizes over the batch dimension of the input data.
To provide input data with labeled dimensions or input data with additional layout information, you can use data formats.
A data format is a string of characters, where each character describes the type of the corresponding data dimension.
The characters are:
"S"
— Spatial"C"
— Channel"B"
— Batch"T"
— Time"U"
— Unspecified
For example, consider an array containing a batch of sequences where the first, second,
and third dimensions correspond to channels, observations, and time steps, respectively. You
can specify that this array has the format "CBT"
(channel, batch,
time).
To create formatted input data, create a dlarray
object and specify the format using the second argument.
To provide additional layout information with unformatted data, specify the formats using the DataFormat
and WeightsFormat
arguments.
For more information, see Deep Learning Data Formats.
Extended Capabilities
GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.
The huber
function
supports GPU array input with these usage notes and limitations:
When at least one of the following input arguments is a
gpuArray
or adlarray
with underlying data of typegpuArray
, this function runs on the GPU:Y
targets
weights
'Mask'
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
Version History
Introduced in R2021a
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)