pixelClassificationLayer

Create pixel classification layer for semantic segmentation

Description

A pixel classification layer provides a categorical label for each image pixel or voxel.

Creation

Syntax

layer = pixelClassificationLayer

layer = pixelClassificationLayer(Name,Value)

Description

layer = pixelClassificationLayer creates a pixel classification output layer for semantic image segmentation networks. The layer outputs the categorical label for each image pixel or voxel processed by a CNN. The layer automatically ignores undefined pixel labels during training.

layer = pixelClassificationLayer(Name,Value) returns a pixel classification output layer using Name,Value pair arguments to set the optional Classes, ClassWeights, and Name properties by using name-value pairs. You can specify multiple name-value pairs. Enclose each property name in quotes.

For example, pixelClassificationLayer('Name','pixclass') creates a pixel classification layer with the name 'pixclass'.

Properties

Classes — Classes of the output layer

'auto' (default) | categorical vector | string array | cell array of character vectors

Classes of the output layer, specified as a categorical vector, string array, cell array of character vectors, or 'auto'. If Classes is 'auto', then the software automatically sets the classes at training time. If you specify the string array or cell array of character vectors str, then the software sets the classes of the output layer to categorical(str,str). The default value is 'auto'.

Data Types: char | categorical | string | cell

ClassWeights — Class weights

'none' (default) | vector of real scalar

Class weights, specified as 'none' or as a vector of real scalar. The elements of the vector correspond to the classes in Classes. If you specify ClassWeights, then you must specify Classes.

Use class weighting to balance classes when there are underrepresented classes in the training data.

OutputSize — Output size

'auto' (default)

This property is read-only.

The output size of the layer. The value is 'auto' prior to training, and is specified as a numeric value at training time.

LossFunction — Loss function

'crossentropyex' (default)

This property is read-only.

Loss function used for training, specified as 'crossentropyex'.

Name — Layer name

'' (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. To include a layer in a layer graph, you must specify a nonempty unique layer name. If you train a series network with the layer and Name is set to '', then the software automatically assigns a name to the layer at training time.

Data Types: char | string

NumInputs — Number of inputs

1 (default)

Number of inputs of the layer. This layer accepts a single input only.

Data Types: double

InputNames — Input names

{'in'} (default)

Input names of the layer. This layer accepts a single input only.

Data Types: cell

Examples

Use Pixel Classification Layer to Create Semantic Segmentation Network

Predict the categorical label of every pixel in an input image.

layers = [
         imageInputLayer([32 32 3])
         convolution2dLayer(3,16,'Stride',2,'Padding',1)
         reluLayer
         transposedConv2dLayer(3,1,'Stride',2,'Cropping',1)
         softmaxLayer
         pixelClassificationLayer
      ]

layers = 
  6x1 Layer array with layers:

     1   ''   Image Input                  32x32x3 images with 'zerocenter' normalization
     2   ''   Convolution                  16 3x3 convolutions with stride [2  2] and padding [1  1  1  1]
     3   ''   ReLU                         ReLU
     4   ''   Transposed Convolution       1 3x3 transposed convolutions with stride [2  2] and cropping [1  1  1  1]
     5   ''   Softmax                      softmax
     6   ''   Pixel Classification Layer   Cross-entropy loss 

Use Weighting to Balance Classes in Training Data

Balance classes using inverse class frequency weighting when some classes are underrepresented in the training data. First, count class frequencies over the training data using pixelLabelImageDatastore. Then, set the 'ClassWeights' in pixelClassificationLayer to the computed inverse class frequencies.

Set the location of image and pixel label data.

  dataDir = fullfile(toolboxdir('vision'),'visiondata');
  imDir = fullfile(dataDir,'building');
  pxDir = fullfile(dataDir,'buildingPixelLabels');

Create a pixel label image datastore using the ground truth images in imds and the pixel labeled images in pxds.

  imds = imageDatastore(imDir);
  classNames = ["sky" "grass" "building" "sidewalk"];
  pixelLabelID = [1 2 3 4];
  pxds = pixelLabelDatastore(pxDir,classNames,pixelLabelID);     
  pximds = pixelLabelImageDatastore(imds,pxds);

Tabulate class distribution in dataset.

  tbl = countEachLabel(pximds)

tbl=4×3 table
        Name        PixelCount    ImagePixelCount
    ____________    __________    _______________

    {'sky'     }    3.1485e+05       1.536e+06   
    {'grass'   }    1.5979e+05       1.536e+06   
    {'building'}    1.0312e+06       1.536e+06   
    {'sidewalk'}         25313       9.216e+05   

Calculate inverse frequency class weights.

  totalNumberOfPixels = sum(tbl.PixelCount);
  frequency = tbl.PixelCount / totalNumberOfPixels;
  inverseFrequency = 1./frequency

inverseFrequency = 4×1

    4.8632
    9.5827
    1.4848
   60.4900

Set 'ClassWeights' to the inverse class frequencies.

  layer = pixelClassificationLayer(...
      'Classes',tbl.Name,'ClassWeights',inverseFrequency)

layer = 
  PixelClassificationLayer with properties:

            Name: ''
         Classes: [sky    grass    building    sidewalk]
    ClassWeights: [4x1 double]
      OutputSize: 'auto'

   Hyperparameters
    LossFunction: 'crossentropyex'

Extended Capabilities

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

To generate CUDA^® or C++ code by using GPU Coder™, you must first construct and train a deep neural network. Once the network is trained and evaluated, you can configure the code generator to generate code and deploy the convolutional neural network on platforms that use NVIDIA^® or ARM^® GPU processors. For more information, see Deep Learning with GPU Coder (GPU Coder).

For this layer, you can generate code that takes advantage of the NVIDIA CUDA deep neural network library (cuDNN), NVIDIA TensorRT™ high performance inference library, or the ARM Compute Library for Mali GPU.