Main Content

trainAutoencoder

Train an autoencoder

Description

autoenc = trainAutoencoder(X) returns an autoencoder, autoenc, trained using the training data in X.

example

autoenc = trainAutoencoder(X,hiddenSize) returns an autoencoder autoenc, with the hidden representation size of hiddenSize.

autoenc = trainAutoencoder(___,Name,Value) returns an autoencoder autoenc, for any of the above input arguments with additional options specified by one or more Name,Value pair arguments.

For example, you can specify the sparsity proportion or the maximum number of training iterations.

example

Examples

collapse all

Load the sample data.

X = abalone_dataset;

X is an 8-by-4177 matrix defining eight attributes for 4177 different abalone shells: sex (M, F, and I (for infant)), length, diameter, height, whole weight, shucked weight, viscera weight, shell weight. For more information on the dataset, type help abalone_dataset in the command line.

Train a sparse autoencoder with default settings.

autoenc = trainAutoencoder(X);

Figure Neural Network Training (01-Feb-2025 09:12:04) contains an object of type uigridlayout.

Reconstruct the abalone shell ring data using the trained autoencoder.

XReconstructed = predict(autoenc,X);

Compute the mean squared reconstruction error.

mseError = mse(X-XReconstructed)
mseError = 
0.0167

Load the sample data.

X = abalone_dataset;

X is an 8-by-4177 matrix defining eight attributes for 4177 different abalone shells: sex (M, F, and I (for infant)), length, diameter, height, whole weight, shucked weight, viscera weight, shell weight. For more information on the dataset, type help abalone_dataset in the command line.

Train a sparse autoencoder with hidden size 4, 400 maximum epochs, and linear transfer function for the decoder.

autoenc = trainAutoencoder(X,4,'MaxEpochs',400,...
'DecoderTransferFunction','purelin');

Figure Neural Network Training (01-Feb-2025 09:10:19) contains an object of type uigridlayout.

Reconstruct the abalone shell ring data using the trained autoencoder.

XReconstructed = predict(autoenc,X);

Compute the mean squared reconstruction error.

mseError = mse(X-XReconstructed)
mseError = 
0.0046

Generate the training data.

rng(0,'twister'); % For reproducibility
n = 1000;
r = linspace(-10,10,n)';
x = 1 + r*5e-2 + sin(r)./r + 0.2*randn(n,1);

Train autoencoder using the training data.

hiddenSize = 25;
autoenc = trainAutoencoder(x',hiddenSize,...
        'EncoderTransferFunction','satlin',...
        'DecoderTransferFunction','purelin',...
        'L2WeightRegularization',0.01,...
        'SparsityRegularization',4,...
        'SparsityProportion',0.10);

Figure Neural Network Training (01-Feb-2025 09:06:36) contains an object of type uigridlayout.

Generate the test data.

n = 1000;
r = sort(-10 + 20*rand(n,1));
xtest = 1 + r*5e-2 + sin(r)./r + 0.4*randn(n,1);

Predict the test data using the trained autoencoder, autoenc.

xReconstructed = predict(autoenc,xtest');

Plot the actual test data and the predictions.

figure;
plot(xtest,'r.');
hold on
plot(xReconstructed,'go');

Figure contains an axes object. The axes object contains 2 objects of type line. One or more of the lines displays its values using only markers

Load the training data.

XTrain = digitTrainCellArrayData;

The training data is a 1-by-5000 cell array, where each cell containing a 28-by-28 matrix representing a synthetic image of a handwritten digit.

Train an autoencoder with a hidden layer containing 25 neurons.

hiddenSize = 25;
autoenc = trainAutoencoder(XTrain,hiddenSize,...
        'L2WeightRegularization',0.004,...
        'SparsityRegularization',4,...
        'SparsityProportion',0.15);

Load the test data.

XTest = digitTestCellArrayData;

The test data is a 1-by-5000 cell array, with each cell containing a 28-by-28 matrix representing a synthetic image of a handwritten digit.

Reconstruct the test image data using the trained autoencoder, autoenc.

xReconstructed = predict(autoenc,XTest);

View the actual test data.

figure;
for i = 1:20
    subplot(4,5,i);
    imshow(XTest{i});
end

View the reconstructed test data.

figure;
for i = 1:20
    subplot(4,5,i);
    imshow(xReconstructed{i});
end

Input Arguments

collapse all

Training data, specified as a matrix of training samples or a cell array of image data. If X is a matrix, then each column contains a single sample. If X is a cell array of image data, then the data in each cell must have the same number of dimensions. The image data can be pixel intensity data for gray images, in which case, each cell contains an m-by-n matrix. Alternatively, the image data can be RGB data, in which case, each cell contains an m-by-n-3 matrix.

Data Types: single | double | cell

Size of hidden representation of the autoencoder, specified as a positive integer value. This number is the number of neurons in the hidden layer.

Data Types: single | double

Name-Value Arguments

collapse all

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: 'EncoderTransferFunction','satlin','L2WeightRegularization',0.05 specifies the transfer function for the encoder as the positive saturating linear transfer function and the L2 weight regularization as 0.05.

Transfer function for the encoder, specified as the comma-separated pair consisting of 'EncoderTransferFunction' and one of the following.

Transfer Function OptionDefinition
'logsig'

Logistic sigmoid function

f(z)=11+ez

'satlin'

Positive saturating linear transfer function

f(z)={0,if z0z,if 0<z<11,if z1

Example: 'EncoderTransferFunction','satlin'

Transfer function for the decoder, specified as the comma-separated pair consisting of 'DecoderTransferFunction' and one of the following.

Transfer Function OptionDefinition
'logsig'

Logistic sigmoid function

f(z)=11+ez

'satlin'

Positive saturating linear transfer function

f(z)={0,if z0z,if 0<z<11,if z1

'purelin'

Linear transfer function

f(z)=z

Example: 'DecoderTransferFunction','purelin'

Maximum number of training epochs or iterations, specified as the comma-separated pair consisting of 'MaxEpochs' and a positive integer value.

Example: 'MaxEpochs',1200

The coefficient for the L2 weight regularizer in the cost function (LossFunction), specified as the comma-separated pair consisting of 'L2WeightRegularization' and a positive scalar value.

Example: 'L2WeightRegularization',0.05

Loss function to use for training, specified as the comma-separated pair consisting of 'LossFunction' and 'msesparse'. It corresponds to the mean squared error function adjusted for training a sparse autoencoder as follows:

E=1Nn=1Nk=1K(xknx^kn)2mean squared error+λ*ΩweightsL2regularization+β*Ωsparsitysparsityregularization,

where λ is the coefficient for the L2 regularization term and β is the coefficient for the sparsity regularization term. You can specify the values of λ and β by using the L2WeightRegularization and SparsityRegularization name-value pair arguments, respectively, while training an autoencoder.

Indicator to show the training window, specified as the comma-separated pair consisting of 'ShowProgressWindow' and either true or false.

Example: 'ShowProgressWindow',false

Desired proportion of training examples a neuron reacts to, specified as the comma-separated pair consisting of 'SparsityProportion' and a positive scalar value. Sparsity proportion is a parameter of the sparsity regularizer. It controls the sparsity of the output from the hidden layer. A low value for SparsityProportion usually leads to each neuron in the hidden layer "specializing" by only giving a high output for a small number of training examples. Hence, a low sparsity proportion encourages higher degree of sparsity. See Sparse Autoencoders.

Example: 'SparsityProportion',0.01 is equivalent to saying that each neuron in the hidden layer should have an average output of 0.1 over the training examples.

Coefficient that controls the impact of the sparsity regularizer in the cost function, specified as the comma-separated pair consisting of 'SparsityRegularization' and a positive scalar value.

Example: 'SparsityRegularization',1.6

The algorithm to use for training the autoencoder, specified as the comma-separated pair consisting of 'TrainingAlgorithm' and 'trainscg'. It stands for scaled conjugate gradient descent [1].

Indicator to rescale the input data, specified as the comma-separated pair consisting of 'ScaleData' and either true or false.

Autoencoders attempt to replicate their input at their output. For it to be possible, the range of the input data must match the range of the transfer function for the decoder. trainAutoencoder automatically scales the training data to this range when training an autoencoder. If the data was scaled while training an autoencoder, the predict, encode, and decode methods also scale the data.

Example: 'ScaleData',false

Indicator to use GPU for training, specified as the comma-separated pair consisting of 'UseGPU' and either true or false.

Example: 'UseGPU',true

Output Arguments

collapse all

Trained autoencoder, returned as an Autoencoder object. For information on the properties and methods of this object, see Autoencoder class page.

More About

collapse all

References

[1] Moller, M. F. “A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning”, Neural Networks, Vol. 6, 1993, pp. 525–533.

[2] Olshausen, B. A. and D. J. Field. “Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1.” Vision Research, Vol.37, 1997, pp.3311–3325.

Version History

Introduced in R2015b