importNetworkFromTensorFlow

Import TensorFlow network as MATLAB network

Since R2023b

Syntax

net = importNetworkFromTensorFlow(modelFolder)

net = importNetworkFromTensorFlow(modelFolder,PackageName=CustomLayersPackage)

Description

net = importNetworkFromTensorFlow(modelFolder) imports a pretrained TensorFlow™ network from the folder modelFolder, which contains the network in the SavedModel format. The network comprises the layers defined in the saved_model file with the .pb extension and the learned weights in the variables subfolder. The function returns the network net as an initialized dlnetwork object.

importNetworkFromTensorFlow requires the Deep Learning Toolbox™ Converter for TensorFlow Models support package. If this support package is not installed, then importNetworkFromTensorFlow provides a download link.

Note

The importNetworkFromTensorFlow function can generate a custom layer when importing a TensorFlow layer. For more information, see Algorithms. The function saves the generated custom layers in the +modelFolder package.

net = importNetworkFromTensorFlow(modelFolder,PackageName=CustomLayersPackage) imports a pretrained network from TensorFlow and saves the generated custom layers and associated functions in the +CustomLayersPackage package.

Examples

Import TensorFlow Network and Classify Image

Import a pretrained TensorFlow network in the SavedModel format and use the imported network to predict class labels.

Specify the model folder.

if ~exist('digitsdlnetwork','dir')
    unzip('digitsdlnetwork.zip')
end
modelFolder = './digitsdlnetwork';

Specify the class names.

classNames = {'0','1','2','3','4','5','6','7','8','9'};

Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

net = 
  dlnetwork with properties:

         Layers: [12x1 nnet.cnn.layer.Layer]
    Connections: [12x2 table]
     Learnables: [6x3 table]
          State: [0x3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

Read the image you want to classify and display the size of the image. The image is a grayscale (one-channel) image with a size of 28-by-28 pixels.

digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
I = imread(fullfile(digitDatasetPath,'5','image4009.png'));
size(I)

ans = 1×2

    28    28

Display the input size of the network. In this case, the image size matches the network input size. If the sizes do not match, you must resize the image using this command: imresize(I,netInputSize(1:2)).

netInputSize = net.Layers(1).InputSize

netInputSize = 1×3

    28    28     1

Convert the image to a dlarray object. Format the images with the dimensions 'SSCB' (spatial, spatial, channel, batch). In this case, the batch size is 1 and you can omit it ('SSC').

I_dlarray = dlarray(single(I),'SSC');

Classify the sample image and find the predicted label.

prob = predict(net,I_dlarray);
[~,label] = max(prob);

Display the image and the classification result.

imshow(I)
title(['Classification result ' classNames{label}]) 

Import TensorFlow Network with Automatically Generated Custom Layers

Import a pretrained TensorFlow network in the SavedModel format. The network contains layers that the software cannot convert to built-in MATLAB layers. The software automatically generates custom layers when you import this network.

This example uses the findCustomLayers helper function. To view the code for this function, see Helper Function.

Specify the model folder.

if ~exist('digitsdlnetworkwithnoise','dir')
    unzip('digitsdlnetworkwithnoise.zip')
end
modelFolder = './digitsdlnetworkwithnoise';

Import a TensorFlow network in the SavedModel format.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

net = 
  dlnetwork with properties:

         Layers: [14x1 nnet.cnn.layer.Layer]
    Connections: [14x2 table]
     Learnables: [6x3 table]
          State: [0x3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

The imported network contains layers the software cannot convert to built-in MATLAB layers, so importNetworkFromTensorFlow automatically generates custom layers in place of these layers. The function saves each generated custom layer to a separate M file in the +digitsdlnetworkwithnoise package in the current folder.

Find the indices of the automatically generated custom layers using the helper function findCustomLayers, and display the custom layers.

ind = findCustomLayers(net.Layers,'+digitsdlnetworkwithnoise');
net.Layers(ind)

ans = 
  3x1 Layer array with layers:

     1   'concatenate_1'      Concatenate     digitsdlnetworkwithnoise.kConcatenate1Layer3826
     2   'gaussian_noise_1'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise1Layer3766
     3   'gaussian_noise_2'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise2Layer3791

Plot the network architecture.

plot(net)
title('Network Architecture')

Helper Function

This section defines the findCustomLayers helper function. findCustomLayers returns the indices of the custom layers that importNetworkFromTensorFlow automatically generates.

function indices = findCustomLayers(layers,PackageName)

s = what(['.' filesep PackageName]);

indices = zeros(1,length(s.m));
for i = 1:length(layers)
    for j = 1:length(s.m)
        if strcmpi(class(layers(i)),[PackageName(2:end) '.' s.m{j}(1:end-2)])
            indices(j) = i;
        end
    end
end

end

Import and Initialize TensorFlow Network

Import a pretrained TensorFlow network in the SavedModel format, which results in an uninitialized network. Then, initialize the network.

Specify the model folder.

if ~exist('imageClassificationNet','dir')
    unzip('imageClassificationNet.zip')
end
modelFolder = './imageClassificationNet';

Import the model using the importNetworkFromTensorFlow function. Because this model does not have a fixed input size, the function imports the model as an uninitialized dlnetwork object without an input layer. The software displays a warning that describes how to initialize the network.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...

Warning: Size of the input to 'input_1' is unknown in TensorFlow.

Finished translation. Assembling network...

Warning: Network is imported as an uninitialized dlnetwork object. Before using the network, add input layer(s):

inputLayer1 = imageInputLayer(<inputSize1>, Normalization="none");
net = addInputLayer(net, inputLayer1, Initialize=true);

For more information on which format(s) to use, see documentation for dlarray.

Import finished.

net = 
  dlnetwork with properties:

         Layers: [7x1 nnet.cnn.layer.Layer]
    Connections: [6x2 table]
     Learnables: [6x3 table]
          State: [2x3 table]
     InputNames: {'conv2d'}
    OutputNames: {'softmax'}
    Initialized: 0

  View summary with summary.

Specify the input size of the imported network and create an image input layer. Then, add the image input layer to the imported network and initialize the network using the addInputLayer function.

inputLayer1 = imageInputLayer([28 28 3], Normalization="none");
net = addInputLayer(net,inputLayer1,Initialize=true)

net = 
  dlnetwork with properties:

         Layers: [8x1 nnet.cnn.layer.Layer]
    Connections: [7x2 table]
     Learnables: [6x3 table]
          State: [2x3 table]
     InputNames: {'imageinput'}
    OutputNames: {'softmax'}
    Initialized: 1

  View summary with summary.

Analyze the imported network by entering deepNetworkDesigner(net) in the Command Window. The network contains the input layer and is ready to use for prediction.

Input Arguments

modelFolder — Name of TensorFlow model folder

character vector | string scalar

Name of the TensorFlow model folder, specified as a character vector or string scalar. modelFolder must be in the current folder, or you must include a full or relative path to the folder. modelFolder must contain the saved_model file with the .pb extension and the variables subfolder. The folder name can also contain the assets and assets.extra subfolders.

• The saved_model file contains the layer graph architecture and training options (for example, optimizer, losses, and metrics).

• The variables subfolder contains the weights learned by the pretrained TensorFlow network. By default, importNetworkFromTensorFlow imports the weights.

• The assets subfolder contains supplementary files (for example, vocabularies), which the layer graph can use. importNetworkFromTensorFlow does not import the files in assets.

• The assets.extra subfolder contains supplementary files, which coexist with the layer graph.

Example: "MobileNet"

Example: "./MobileNet"

CustomLayersPackage — Name of custom layers package

character vector | string scalar

Name of the custom layers package in which importNetworkFromTensorFlow saves custom layers, specified as a character vector or string scalar. importNetworkFromTensorFlow saves the custom layers package in the current folder. If you do not specify CustomLayersPackage, then importNetworkFromTensorFlow saves the custom layers in a package named +modelFolder in the current folder. For more information about packages, see Packages Create Namespaces.

importNetworkFromTensorFlow tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB^® layer. importNetworkFromTensorFlow saves each generated custom layer to a separate MATLAB code file in +CustomLayersPackage. To view or edit a custom layer, open the associated MATLAB code file. For more information about custom layers, see Custom Layers.

The +CustomLayersPackage package can also contain the +ops subpackage. This subpackage contains MATLAB functions corresponding to TensorFlow operators that the automatically generated custom layers use. importNetworkFromTensorFlow saves the associated MATLAB function for each operator in a separate MATLAB code file in the +ops subpackage. The object functions of dlnetwork, such as the predict function, use these operators when it interacts with the custom layers.

Example: PackageName="MobileNet"

Example: PackageName="CustomLayers"

Output Arguments

net — Pretrained TensorFlow network

dlnetwork object

Pretrained TensorFlow network, returned as an initialized dlnetwork object. In some cases, the software indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

More About

TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers

importNetworkFromTensorFlow supports these TensorFlow-Keras layer types for conversion into built-in MATLAB layers, with some limitations.

TensorFlow-Keras Layer	Corresponding Deep Learning Toolbox Layer
Add	additionLayer
Activation, with activation names: elu exponential gelu relu linear selu softmax softplus softsign sigmoid swish tanh	Layers: eluLayer functionLayer with PredictFcn specified as @exp geluLayer reluLayer or clippedReluLayer None eluLayer with Alpha specified as 1.67326324 and scalingLayer (Reinforcement Learning Toolbox) with Scale specified as 1.05070098 softmaxLayer softplusLayer (Reinforcement Learning Toolbox) nnet.keras.layer.SoftsignLayer sigmoidLayer swishLayer tanhLayer
Advanced activations: ELU Softmax ReLU LeakyReLU PReLu*	Layers: eluLayer softmaxLayer reluLayer, clippedReluLayer, or leakyReluLayer leakyReluLayer nnet.keras.layer.PreluLayer
AveragePooling1D	averagePooling1dLayer with PaddingValue specified as 'mean'
AveragePooling2D	averagePooling2dLayer with PaddingValue specified as 'mean'
BatchNormalization	batchNormalizationLayer
Bidirectional(LSTM(__))	bilstmLayer
Conv1D	convolution1dLayer
Conv2D	convolution2dLayer
Conv2DTranspose	transposedConv2dLayer
CuDNNGRU	gruLayer
CuDNNLSTM	lstmLayer
Dense	fullyConnectedLayer
DepthwiseConv2D	groupedConvolution2dLayer
Dropout	dropoutLayer
Embedding	wordEmbeddingLayer (Text Analytics Toolbox)
Flatten	nnet.keras.layer.FlattenCStyleLayer
GlobalAveragePooling1D	globalAveragePooling1dLayer
GlobalAveragePooling2D	globalAveragePooling2dLayer
GlobalMaxPool1D	globalMaxPooling1dLayer
GlobalMaxPool2D	globalMaxPooling2dLayer
GRU	gruLayer
Input	imageInputLayer, sequenceInputLayer, or featureInputLayer
LSTM	lstmLayer
MaxPool1D	maxPooling1dLayer
MaxPool2D	maxPooling2dLayer
Multiply	multiplicationLayer
SeparableConv2D	groupedConvolution2dLayer or convolution2dLayer
TimeDistributed	sequenceFoldingLayer before the wrapped layer, and sequenceUnfoldingLayer after the wrapped layer
UpSampling2D	resize2dLayer (Image Processing Toolbox)
UpSampling3D	resize3dLayer (Image Processing Toolbox)
ZeroPadding1D	nnet.keras.layer.ZeroPadding1DLayer
ZeroPadding2D	nnet.keras.layer.ZeroPadding2DLayer

* For a PReLU layer, importNetworkFromTensorFlow replaces a vector-valued scaling parameter with the average of the vector elements. You can change the parameter back to a vector after import. For an example, see Import Keras PReLU Layer.

Supported TensorFlow Operators

importNetworkFromTensorFlow supports these TensorFlow operators for conversion into MATLAB functions with dlarray support.

TensorFlow Operator	Corresponding MATLAB Function
Abs	abs
Add	tfAdd
AddN	tfAddN
AddV2	tfAdd
All	tfAll
ArgMin	tfArgMin
Assert	assert
AvgPool	tfAvgPool
BatchMatMulV2	tfBatchMatMulV2
BiasAdd	tfBiasAdd
BroadcastTo	tfBroadcastTo
Cast	tfCast
ConcatV2	tfCat
Const	None (translated to weights in custom layer)
Conv2D	tfConv2D
DepthToSpace	depthToSpace (Image Processing Toolbox)
DepthwiseConv2dNative	tfDepthwiseConv2D
Equal	==
Exp	exp
ExpandDims	tfExpandDims
Fill	tfFill
FloorDiv	tfFloorDiv
FloorMod	tfFloorMod
FusedBatchNormV3	tfBatchnorm
GatherV2	tfGather
Greater	gt>
GreaterEqual	ge, >=
Identity	None (translated to value assignment in custom layer)
IdentityN	tfIdentityN
If	if
L2Loss	tfL2Loss
LeakyRelu	leakyrelu
Less	lt, <
LessEqual	le, <=
Log	log
LogicalAnd	and
MatMul	tfMatMul
MaxPool	tfMaxPool
Max	tfMax
Maximum	tfMaximum
Mean	tfMean
Min	tfMin
Minimum	tfMinimum
MirrorPad	tfMirrorPad
Mul	tfMul
Neg	minus, -
NonMaxSuppressionV5	tfNonMaxSuppressionV5
Pack	tfStack
Pad	tfPad
PadV2	tfPad
PartitionedCall	None (translated to function in custom layer methods)
Pow	power, .^
Prod	tfProd
RandomStandardNormal	tfRandomStandardNormal
Range	tfRange
ReadVariableOp	None (translated to value assignment in custom layer)
RealDiv	tfDiv
Relu	relu
Relu6	relu and min
Reshape	tfReshape
ResizeBilinear	dlresize (Image Processing Toolbox)
ResizeNearestNeighbor	dlresize (Image Processing Toolbox)
Round	round
Rsqrt	sqrt
ScatterNd	tfScatterNd
Select	tfSelect
Shape	tfShape
Sigmoid	sigmoid
Size	tfSize
Slice	tfSlice
Softmax	softmax
SpaceToDepth	spaceToDepth (Image Processing Toolbox)
Split	tfSplit
Square	.^2
Sqrt	sqrt
SquaredDifference	tfMul or tfSub
Squeeze	tfSqueeze
StatefulPartitionedCall	None (translated to function in custom layer methods)
StatelessIf	if
StatelessWhile	while
StopGradient	tfStopGradient
StridedSlice	tfStridedSlice or tfSqueeze
Sub	tfSub
Sum	tfSum
Tanh	tanh
TensorListFromTensor	tfTensorListFromTensor
TensorListGetItem	tfTensorListGetItem
TensorListReserve	tfTensorListReserve
TensorListSetItem	tfTensorListSetItem
TensorListStack	tfTensorListStack
Tile	tfTile
Transpose	tfTranspose
Unpack	tfUnpack
Where	tfWhere

For more information about functions that operate on dlarray objects, see List of Functions with dlarray Support.

Generate Code for Imported Network

You can use MATLAB Coder™ or GPU Coder™ software with Deep Learning Toolbox software to generate MEX, standalone CPU, CUDA^® MEX, or standalone CUDA code for an imported network. For more information, see Code Generation.

• Use MATLAB Coder with Deep Learning Toolbox to generate MEX or standalone CPU code that runs on desktop or embedded targets. You can deploy generated standalone code that uses the Intel^® Math Kernel Library for Deep Neural Networks (MKL-DNN) or the ARM^® Compute library. Alternatively, you can generate generic C or C++ code that does not call third-party library functions. For more information, see Deep Learning with MATLAB Coder (MATLAB Coder).

• Use GPU Coder with Deep Learning Toolbox to generate CUDA MEX or standalone CUDA code that runs on desktop or embedded targets. You can deploy generated standalone CUDA code that uses the CUDA deep neural network library (cuDNN), the TensorRT™ high performance inference library, or the ARM Compute library for Mali GPU. For more information, see Deep Learning with GPU Coder (GPU Coder).

importNetworkFromTensorFlow returns the network net as a dlnetwork object. This object supports code generation. For more information about MATLAB Coder and GPU Coder support for Deep Learning Toolbox objects, see Supported Classes (MATLAB Coder) and Supported Classes (GPU Coder), respectively.

You can generate code for any imported network whose layers support code generation. For lists of the layers that support code generation with MATLAB Coder and GPU Coder, see Supported Layers (MATLAB Coder) and Supported Layers (GPU Coder), respectively. For more information about the code generation capabilities and limitations of each built-in MATLAB layer, see the Extended Capabilities section of the layer. For example, see Code Generation and GPU Code Generation of imageInputLayer.

Use Imported Network on GPU

importNetworkFromTensorFlow does not execute on a GPU. However, importNetworkFromTensorFlow imports a pretrained neural network for deep learning as a dlnetwork object, which you can use on a GPU.

• You can make predictions with the imported network on a CPU or GPU by using predict. The predict function executes on the GPU if you store the input data or network parameters on the GPU.

  • If you use minibatchqueue to process and manage the mini-batches of input data, the minibatchqueue object converts the output to a GPU array by default if a GPU is available.

  • Use dlupdate to convert the learnable parameters of a dlnetwork object to GPU arrays.

    net = dlupdate(@gpuArray,net)

• You can train the imported network on a CPU or GPU by using trainnet. To specify training options, including options for the execution environment, use the trainingOptions function. Specify the hardware requirements using the ExecutionEnvironment name-value argument. For more information about how to accelerate training, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.

Using a GPU requires a Parallel Computing Toolbox™ license and a supported GPU device. For information about supported devices, see GPU Computing Requirements (Parallel Computing Toolbox).

Tips

• To use a pretrained network for prediction or transfer learning on new images, you must preprocess your images in the as same way the images that you use to train the imported model. The most common preprocessing steps are resizing images, subtracting image average values, and converting the images from BGR format to RGB format.

  • To resize images, use imresize. For example, imresize(image,[227 227 3]).

  • To convert images from RGB to BGR format, use flip. For example, flip(image,3).

  For more information about preprocessing images for training and prediction, see Preprocess Images for Deep Learning.

• The members of the +CustomLayersPackage package (custom layers and TensorFlow operators) are not accessible if the package parent folder is not on the MATLAB path. For more information, see Packages and the MATLAB Path.

• MATLAB uses one-based indexing, whereas Python^® uses zero-based indexing. In other words, the first element in an array has an index of 1 and 0 in MATLAB and Python, respectively. For more information about MATLAB indexing, see Array Indexing. In MATLAB, to use an array of indices (ind) created in Python, convert the array to ind+1.

• For more tips, see Tips on Importing Models from TensorFlow, PyTorch, and ONNX.

Algorithms

The importNetworkFromTensorFlow function imports a TensorFlow layer into MATLAB by trying these steps in order:

1. The function tries to import the TensorFlow layer as a built-in MATLAB layer. For more information, see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers.

2. The function tries to import the TensorFlow layer as a built-in MATLAB function. For more information, see Supported TensorFlow Operators.

3. The function tries to import the TensorFlow layer as a custom layer. importNetworkFromTensorFlow saves the generated custom layers and the associated functions in the +CustomLayersPackage package. For an example, see Import TensorFlow Network with Automatically Generated Custom Layers.

4. The function imports the TensorFlow layer as a custom layer with a placeholder function. importNetworkFromTensorFlow saves the placeholder function in the +ops subpackage of the +CustomLayersPackage package. You must complete the placeholder function before you can use the network.

In some cases, the software will indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

References

[1] TensorFlow. “TensorFlow.” Accessed July 3, 2023. https://www.tensorflow.org/.

[2] TensorFlow. “Using the SavedModel Format | TensorFlow Core.” Accessed July 3, 2023. https://www.tensorflow.org/guide/saved_model.

Version History

Introduced in R2023b