importNetworkFromTensorFlow

Import TensorFlow network as MATLAB network

Since R2023b

Syntax

net = importNetworkFromTensorFlow(modelFolder)

net = importNetworkFromTensorFlow(modelFolder,Name=Value)

Description

net = importNetworkFromTensorFlow(modelFolder) imports a pretrained TensorFlow™ network from the folder modelFolder, which contains the network in the SavedModel format. The network comprises the layers defined in the saved_model file with the .pb extension and the learned weights in the variables subfolder. The function returns the network net as an initialized dlnetwork object.

importNetworkFromTensorFlow requires the Deep Learning Toolbox™ Converter for TensorFlow Models support package. If this support package is not installed, then importNetworkFromTensorFlow provides a download link.

Note

The importNetworkFromTensorFlow function can generate a custom layer when importing a TensorFlow layer. For more information, see Algorithms. The function saves the generated custom layers in the +modelFolder namespace.

net = importNetworkFromTensorFlow(modelFolder,Name=Value) specifies additional options using one or more name-value arguments. For example, you can specify a namespace in which to save the generated custom layers and associated functions.

Examples

Import TensorFlow Network and Classify Image

Import a pretrained TensorFlow network in the SavedModel format and use the imported network to predict class labels.

Specify the model folder.

if ~exist('digitsdlnetwork','dir')
    unzip('digitsdlnetwork.zip')
end
modelFolder = './digitsdlnetwork';

Specify the class names.

classNames = {'0','1','2','3','4','5','6','7','8','9'};

Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

net = 
  dlnetwork with properties:

         Layers: [12x1 nnet.cnn.layer.Layer]
    Connections: [12x2 table]
     Learnables: [6x3 table]
          State: [0x3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

Read the image you want to classify and display the size of the image. The image is a grayscale (one-channel) image with a size of 28-by-28 pixels.

digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
I = imread(fullfile(digitDatasetPath,'5','image4009.png'));
size(I)

ans = 1×2

    28    28

Display the input size of the network. In this case, the image size matches the network input size. If the sizes do not match, you must resize the image using this command: imresize(I,netInputSize(1:2)).

netInputSize = net.Layers(1).InputSize

netInputSize = 1×3

    28    28     1

Convert the image to a dlarray object. Format the images with the dimensions 'SSCB' (spatial, spatial, channel, batch). In this case, the batch size is 1 and you can omit it ('SSC').

I_dlarray = dlarray(single(I),'SSC');

Classify the sample image and find the predicted label.

prob = predict(net,I_dlarray);
[~,label] = max(prob);

Display the image and the classification result.

imshow(I)
title(['Classification result ' classNames{label}]) 

Import TensorFlow Network with Automatically Generated Custom Layers

Import a pretrained TensorFlow network in the SavedModel format. The network contains layers that the software cannot convert to built-in MATLAB layers. The software automatically generates custom layers when you import this network.

This example uses the findCustomLayers helper function. To view the code for this function, see Helper Function.

Specify the model folder.

if ~exist('digitsdlnetworkwithnoise','dir')
    unzip('digitsdlnetworkwithnoise.zip')
end
modelFolder = './digitsdlnetworkwithnoise';

Import a TensorFlow network in the SavedModel format.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

net = 
  dlnetwork with properties:

         Layers: [14×1 nnet.cnn.layer.Layer]
    Connections: [14×2 table]
     Learnables: [6×3 table]
          State: [0×3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

The imported network contains layers the software cannot convert to built-in MATLAB layers, so importNetworkFromTensorFlow automatically generates custom layers in place of these layers. The function saves each generated custom layer to a separate M file in the +digitsdlnetworkwithnoise namespace in the current folder.

Find the indices of the automatically generated custom layers using the helper function findCustomLayers, and display the custom layers.

ind = findCustomLayers(net.Layers,'+digitsdlnetworkwithnoise');
net.Layers(ind)

ans = 
  3×1 Layer array with layers:

     1   'concatenate_1'      Concatenate     digitsdlnetworkwithnoise.kConcatenate1Layer3826
     2   'gaussian_noise_1'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise1Layer3766
     3   'gaussian_noise_2'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise2Layer3791

Plot the network architecture.

plot(net)
title('Network Architecture')

Helper Function

This section defines the findCustomLayers helper function. findCustomLayers returns the indices of the custom layers that importNetworkFromTensorFlow automatically generates.

function indices = findCustomLayers(layers,Namespace)

s = what(['.' filesep Namespace]);

indices = zeros(1,length(s.m));
for i = 1:length(layers)
    for j = 1:length(s.m)
        if strcmpi(class(layers(i)),[Namespace(2:end) '.' s.m{j}(1:end-2)])
            indices(j) = i;
        end
    end
end

end

Import and Initialize TensorFlow Network

Import a pretrained TensorFlow network in the SavedModel format, which results in an uninitialized network. Then, initialize the network.

Specify the model folder.

if ~exist('imageClassificationNet','dir')
    unzip('imageClassificationNet.zip')
end
modelFolder = './imageClassificationNet';

Import the model using the importNetworkFromTensorFlow function. Because this model does not have a fixed input size, the function imports the model as an uninitialized dlnetwork object without an input layer. The software displays a warning that describes how to initialize the network.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model...
Translating the model, this may take a few minutes...

Warning: Size of the input to "input_1" is unknown in TensorFlow.

Finished translation. Assembling network...

Warning: Network is imported as an uninitialized dlnetwork object. Before using the network, add input layer(s):

inputLayer1 = imageInputLayer(<inputSize1>, Normalization="none");
net = addInputLayer(net, inputLayer1, Initialize=true);

For more information on which format(s) to use, see documentation for dlarray.

Import finished.

net = 
  dlnetwork with properties:

         Layers: [7x1 nnet.cnn.layer.Layer]
    Connections: [6x2 table]
     Learnables: [6x3 table]
          State: [2x3 table]
     InputNames: {'conv2d'}
    OutputNames: {'softmax'}
    Initialized: 0

  View summary with summary.

Specify the input size of the imported network and create an image input layer. Then, add the image input layer to the imported network and initialize the network using the addInputLayer function.

inputLayer1 = imageInputLayer([28 28 3], Normalization="none");
net = addInputLayer(net,inputLayer1,Initialize=true)

net = 
  dlnetwork with properties:

         Layers: [8x1 nnet.cnn.layer.Layer]
    Connections: [7x2 table]
     Learnables: [6x3 table]
          State: [2x3 table]
     InputNames: {'imageinput'}
    OutputNames: {'softmax'}
    Initialized: 1

  View summary with summary.

The network contains the input layer and is ready to use for prediction. You can analyze the network further using the Deep Network Designer app.

deepNetworkDesigner(net)

Import Network with Nested Networks

Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object. Analyze the network using the analyzeNetwork function.

net = importNetworkFromTensorFlow("CustomConvReluBN");

Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

analyzeNetwork(net)

net is a dlnetwork object with seven layers. The second and third layers are networkLayer objects that each contain a nested neural network.

Expand the networkLayer layer using the expandLayers function and analyze the network.

netExpanded = expandLayers(net);
analyzeNetwork(netExpanded)

The app shows that the expanded network netExpanded has four more layers than net.

Import the TensorFlow network again, using custom layers to represent the nested neural network. Analyze the network.

netCustom = importNetworkFromTensorFlow("CustomConvReluBN",PreferredNestingType="customlayer");

Importing the saved model...
Translating the model, this may take a few minutes...

Warning: The namespace "CustomConvReluBN" already exists in the Current Folder, and will be overwritten.

Finished translation. Assembling network...
Import finished.

analyzeNetwork(netCustom)

netCustom contains seven layers. The second and third layers are custom layers that represent nested neural networks.

Input Arguments

modelFolder — Name of TensorFlow model folder

character vector | string scalar

Name of the TensorFlow model folder, specified as a character vector or string scalar. modelFolder must be in the current folder, or you must include a full or relative path to the folder. modelFolder must contain the saved_model file with the .pb extension and the variables subfolder. The folder name can also contain the assets and assets.extra subfolders.

• The saved_model file contains the layer graph architecture and training options (for example, optimizer, losses, and metrics).

• The variables subfolder contains the weights learned by the pretrained TensorFlow network. By default, importNetworkFromTensorFlow imports the weights.

• The assets subfolder contains supplementary files (for example, vocabularies), which the layer graph can use. importNetworkFromTensorFlow does not import the files in assets.

• The assets.extra subfolder contains supplementary files, which coexist with the layer graph.

Example: "MobileNet"

Example: "./MobileNet"

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: importNetworkFromTensorflow(modelFolder,PreferredNestingType="customlayer") imports the network in modelFolder and uses custom layers to represent network composition.

Namespace — Name of custom layers namespace

character vector | string scalar

Name of the custom layers namespace in which importNetworkFromTensorFlow saves custom layers, specified as a character vector or string scalar. importNetworkFromTensorFlow saves the custom layers namespace in the current folder. If you do not specify Namespace, then importNetworkFromTensorFlow saves the custom layers in a namespace named +modelFolder in the current folder. For more information about namespaces, see Create Namespaces.

importNetworkFromTensorFlow tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB^® layer. importNetworkFromTensorFlow saves each generated custom layer to a separate MATLAB code file in +Namespace. To view or edit a custom layer, open the associated MATLAB code file. For more information about custom layers, see Custom Layers.

The +Namespace namespace can also contain the +ops inner namespace. This inner namespace contains MATLAB functions corresponding to TensorFlow operators that the automatically generated custom layers use. importNetworkFromTensorFlow saves the associated MATLAB function for each operator in a separate MATLAB code file in the +ops inner namespace. The object functions of dlnetwork, such as the predict function, use these operators when it interacts with the custom layers.

Example: Namespace="MobileNet"

Example: Namespace="CustomLayers"

PreferredNestingType — Network composition representation

"networklayer" (default) | "customlayer"

Network composition representation, specified as one of the following values:

• "networklayer" — Represents network composition in the imported network using networkLayer layer objects.

• "customlayer" — Represents network composition in the imported network using nested custom layers. For more information about custom layers, see Define Custom Deep Learning Layers.

Example: PreferredNestingType="customlayer"

Data Types: char | string

Output Arguments

net — Pretrained TensorFlow network

dlnetwork object

Pretrained TensorFlow network, returned as an initialized dlnetwork object. In some cases, the software indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

More About

TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers

importNetworkFromTensorFlow supports these TensorFlow-Keras layer types for conversion into built-in MATLAB layers, with some limitations.

TensorFlow-Keras Layer	Corresponding Deep Learning Toolbox Layer
Add	additionLayer
Activation, with activation names: elu exponential gelu relu linear selu softmax softplus softsign sigmoid swish tanh	Layers: eluLayer functionLayer with PredictFcn specified as @exp geluLayer reluLayer or clippedReluLayer None eluLayer with Alpha specified as 1.67326324 and scalingLayer (Reinforcement Learning Toolbox) with Scale specified as 1.05070098 softmaxLayer softplusLayer (Reinforcement Learning Toolbox) nnet.keras.layer.SoftsignLayer sigmoidLayer swishLayer tanhLayer
Advanced activations: ELU Softmax ReLU LeakyReLU PReLu	Layers: eluLayer softmaxLayer reluLayer, clippedReluLayer, or leakyReluLayer leakyReluLayer preluLayer
AveragePooling1D	averagePooling1dLayer with PaddingValue specified as 'mean'
AveragePooling2D	averagePooling2dLayer with PaddingValue specified as 'mean'
AveragePooling3D	averagePooling3dLayer
BatchNormalization	batchNormalizationLayer
Bidirectional(LSTM(__))	bilstmLayer
Conv1D	convolution1dLayer
Conv2D	convolution2dLayer
Conv3D	convolution3dLayer
Conv2DTranspose	transposedConv2dLayer
Conv3DTranspose	transposedConv3dLayer
CuDNNGRU	gruLayer
CuDNNLSTM	lstmLayer
Dense	fullyConnectedLayer
DepthwiseConv2D	groupedConvolution2dLayer
Dropout	dropoutLayer
Embedding	wordEmbeddingLayer (Text Analytics Toolbox)
Flatten	nnet.keras.layer.FlattenCStyleLayer
GlobalAveragePooling1D	globalAveragePooling1dLayer
GlobalAveragePooling2D	globalAveragePooling2dLayer
GlobalAveragePooling3D	globalAveragePooling3dLayer
GlobalMaxPooling1D	globalMaxPooling1dLayer
GlobalMaxPooling2D	globalMaxPooling2dLayer
GlobalMaxPooling3D	globalMaxPooling3dLayer
GRU	gruLayer
Identity	identityLayer
Input	imageInputLayer, sequenceInputLayer, or featureInputLayer
LayerNormalization	layerNormalizationLayer
LSTM	lstmLayer
MaxPooling1D	maxPooling1dLayer
MaxPooling2D	maxPooling2dLayer
MaxPooling3D	maxPooling3dLayer
Multiply	multiplicationLayer
Resizing	resize2dLayer (Image Processing Toolbox)
SeparableConv2D	groupedConvolution2dLayer or convolution2dLayer
SpatialDropout1D	spatialDropoutLayer
SpatialDropout2D	spatialDropoutLayer
SpatialDropout3D	spatialDropoutLayer
UpSampling2D	resize2dLayer (Image Processing Toolbox)
UpSampling3D	resize3dLayer (Image Processing Toolbox)
ZeroPadding1D	nnet.keras.layer.ZeroPadding1DLayer
ZeroPadding2D	nnet.keras.layer.ZeroPadding2DLayer

Supported TensorFlow Operators

importNetworkFromTensorFlow supports these TensorFlow operators for conversion into MATLAB functions with dlarray support.

TensorFlow Operator	Corresponding MATLAB Function
Abs	abs
Add	tfAdd
AddN	tfAddN
AddV2	tfAdd
All	tfAll
ArgMin	tfArgMin
Assert	assert
AvgPool	tfAvgPool
BatchMatMulV2	tfBatchMatMulV2
BiasAdd	tfBiasAdd
BroadcastTo	tfBroadcastTo
Cast	tfCast
ConcatV2	tfCat
Const	None (translated to weights in custom layer)
Conv2D	tfConv2D
DepthToSpace	tfDepthToSpace
DepthwiseConv2dNative	tfDepthwiseConv2D
Equal	eq, ==
Erf	erf
Exp	exp
ExpandDims	tfExpandDims
Fill	tfFill
FloorDiv	tfFloorDiv
FloorMod	tfFloorMod
FusedBatchNormV3	tfBatchnorm
GatherV2	tfGather
Greater	gt, >
GreaterEqual	ge, >=
Identity	None (translated to value assignment in custom layer)
IdentityN	tfIdentityN
If	if
L2Loss	tfL2Loss
LeakyRelu	leakyrelu
Less	lt, <
LessEqual	le, <=
Log	log
LogicalAnd	and, &
MatMul	tfMatMul
MaxPool	tfMaxPool
Max	tfMax
Maximum	tfMaximum
Mean	tfMean
Min	tfMin
Minimum	tfMinimum
MirrorPad	tfMirrorPad
Mul	tfMul
Neg	minus, -
NonMaxSuppressionV5	tfNonMaxSuppressionV5
NoOp	None (operator is ignored)
Pack	tfStack
Pad	tfPad
PadV2	tfPad
PartitionedCall	None (translated to function in custom layer methods)
Pow	power, .^
Prod	tfProd
RandomStandardNormal	tfRandomStandardNormal
Range	tfRange
ReadVariableOp	None (translated to value assignment in custom layer)
RealDiv	tfDiv
Relu	relu
Relu6	relu and min
Reshape	tfReshape
ResizeBilinear	tfResize
ResizeNearestNeighbor	tfResize
ResourceGather	tfGather
Round	round
Rsqrt	sqrt
ScatterNd	tfScatterNd
Select	tfSelect
Shape	tfShape
Sigmoid	sigmoid
Size	tfSize
Slice	tfSlice
Softmax	softmax
SpaceToDepth	tfSpaceToDepth
Split	tfSplit
Square	power, .^2
Sqrt	sqrt
SquaredDifference	tfMul or tfSub
Squeeze	tfSqueeze
StatefulPartitionedCall	None (translated to function in custom layer methods)
StatelessIf	if
StatelessWhile	while
StopGradient	tfStopGradient
StridedSlice	tfStridedSlice or tfSqueeze
Sub	tfSub
Sum	tfSum
Tanh	tanh
TensorListFromTensor	tfTensorListFromTensor
TensorListGetItem	tfTensorListGetItem
TensorListReserve	tfTensorListReserve
TensorListSetItem	tfTensorListSetItem
TensorListStack	tfTensorListStack
Tile	tfTile
TopKV2	tfTopKV2
Transpose	tfTranspose
Unpack	tfUnpack
Where	tfWhere

For more information about functions that operate on dlarray objects, see List of Functions with dlarray Support.

Generate Code for Imported Network

You can use MATLAB Coder™ or GPU Coder™ software with Deep Learning Toolbox software to generate MEX, standalone CPU, CUDA^® MEX, or standalone CUDA code for an imported network. For more information, see Code Generation.

• Use MATLAB Coder with Deep Learning Toolbox to generate MEX or standalone CPU code that runs on desktop or embedded targets. You can deploy generated standalone code that uses the Intel^® Math Kernel Library for Deep Neural Networks (MKL-DNN) or the ARM^® Compute library. Alternatively, you can generate generic C or C++ code that does not call third-party library functions. For more information, see Deep Learning with MATLAB Coder (MATLAB Coder).

• Use GPU Coder with Deep Learning Toolbox to generate CUDA MEX or standalone CUDA code that runs on desktop or embedded targets. You can deploy generated standalone CUDA code that uses the CUDA deep neural network library (cuDNN), the TensorRT™ high performance inference library, or the ARM Compute library for Mali GPU. For more information, see Deep Learning with GPU Coder (GPU Coder).

importNetworkFromTensorFlow returns the network net as a dlnetwork object. This object supports code generation. For more information about MATLAB Coder and GPU Coder support for Deep Learning Toolbox objects, see Supported Classes (MATLAB Coder) and Supported Classes (GPU Coder), respectively.

You can generate code for any imported network whose layers support code generation. For lists of the layers that support code generation with MATLAB Coder and GPU Coder, see Supported Layers (MATLAB Coder) and Supported Layers (GPU Coder), respectively. For more information about the code generation capabilities and limitations of each built-in MATLAB layer, see the Extended Capabilities section of the layer. For example, see Code Generation and GPU Code Generation of imageInputLayer.

Use Imported Network on GPU

importNetworkFromTensorFlow does not execute on a GPU. However, importNetworkFromTensorFlow imports a pretrained neural network for deep learning as a dlnetwork object, which you can use on a GPU.

• You can make predictions with the imported network on a CPU or GPU by using predict. The predict function executes on the GPU if you store the input data or network parameters on the GPU.

  • If you use minibatchqueue to process and manage the mini-batches of input data, the minibatchqueue object converts the output to a GPU array by default if a GPU is available.

  • Use dlupdate to convert the learnable parameters of a dlnetwork object to GPU arrays.

    net = dlupdate(@gpuArray,net)

• You can train the imported network on a CPU or GPU by using trainnet. To specify training options, including options for the execution environment, use the trainingOptions function. Specify the hardware requirements using the ExecutionEnvironment name-value argument. For more information about how to accelerate training, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.

Using a GPU requires a Parallel Computing Toolbox™ license and a supported GPU device. For information about supported devices, see GPU Computing Requirements (Parallel Computing Toolbox).

Tips

• To use a pretrained network for prediction or transfer learning on new images, you must preprocess your images in the same way as the images that you use to train the imported model. The most common preprocessing steps are resizing images, subtracting image average values, and converting the images from BGR format to RGB format.

  • To resize images, use imresize. For example, imresize(image,[227 227 3]).

  • To convert images from RGB to BGR format, use flip. For example, flip(image,3).

  For more information about preprocessing images for training and prediction, see Preprocess Images for Deep Learning.

• The members of the +Namespace namespace (custom layers and TensorFlow operators) are not accessible if the namespace parent folder is not on the MATLAB path. For more information, see Namespaces and the MATLAB Path.

• MATLAB uses one-based indexing, whereas Python^® uses zero-based indexing. In other words, the first element in an array has an index of 1 and 0 in MATLAB and Python, respectively. For more information about MATLAB indexing, see Array Indexing. In MATLAB, to use an array of indices (ind) created in Python, convert the array to ind+1.

• For more tips, see Tips on Importing Models from TensorFlow, PyTorch, and ONNX.

Algorithms

The importNetworkFromTensorFlow function imports a TensorFlow layer into MATLAB by trying these steps in order:

1. The function tries to import the TensorFlow layer as a built-in MATLAB layer. For more information, see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers.

2. The function tries to import the TensorFlow layer as a built-in MATLAB function. For more information, see Supported TensorFlow Operators.

3. The function tries to import the TensorFlow layer as a custom layer. importNetworkFromTensorFlow saves the generated custom layers and the associated functions in the +Namespace namespace. For an example, see Import TensorFlow Network with Automatically Generated Custom Layers.

4. The function imports the TensorFlow layer as a custom layer with a placeholder function. importNetworkFromTensorFlow saves the placeholder function in the +ops inner namespace of the +Namespace namespace. You must complete the placeholder function before you can use the network.

In some cases, the software will indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

Alternative Functionality

App

You can also import models from external platforms using the Deep Network Designer app. The app uses the importNetworkFromTensorFlow function to import the network. On import, the app shows an import report with details about any issues that require attention.

References

[1] TensorFlow. “TensorFlow.” Accessed July 3, 2023. https://www.tensorflow.org/.

[2] TensorFlow. “Using the SavedModel Format | TensorFlow Core.” Accessed July 3, 2023. https://www.tensorflow.org/guide/saved_model.

Version History

Introduced in R2023b

R2024b: Represent network composition using networkLayer

You can import a network that uses networkLayer objects to represent network composition. To specify whether the imported network represents composition using networkLayer or custom layer objects, use the PreferredNestingType name-value argument. For more information, see Deep Learning Network Composition.

R2024b: Import networks with new layers and operators

You can import the following TensorFlow layers into Deep Learning Toolbox layers:

• tf.keras.layers.Identity

• tf.keras.layers.LayerNormalization

• tf.keras.layers.SpatialDropout1D

• tf.keras.layers.SpatialDropout2D

• tf.keras.layers.SpatialDropout3D

You can also import the following operators into custom layers:

• tf.raw_ops.Erf

• tf.raw_ops.ResourceGather

R2024a: Import Multi-Input-Multi-Output (MIMO) TFOpLambda layers

You can now import a TensorFlow network that includes a TensorFlow-Keras TFOpLambda layer with multiple inputs and outputs.

R2024a: Import networks that include TimeDistributed layers

importNetworkFromTensorFlow now supports importing networks that include tf.keras.layers.TimeDistributed layers.

R2024a: Improved network performance for custom layers

importNetworkFromTensorFlow generates networks with improved inference and training performance for custom layers.