Is there any example about fcnLayers and its predictions for semantic segmentation

Even though the structure of fcnLayers was provided in the latest released Matlab, the documentation about fcnLayers seems not clear enough and insufficient. The only example I can find for reference is "Semantic Segmentation Using Deep Learning" that used SegNet. Besides, either SegNet or FCN can take arbitrary size input image, and produce the same size labelled output. The structure that provided in Matlab limited the functions with the fixed size input image. It will be great if there is an example of SegNet or FCN that takes PASCAL VOC data just like it mentioned in the reference papers.

Answer 1

[Birju Patel]

In R2017b, we cannot train with arbitrary size images. This is a feature we will consider for a future release. We will also look to add more examples using datasets like PASCAL VOC.

One potential workaround is to use the pixelLabelImageSource with the OutputSizeMode set to 'randcrop' and provide a suitable OutputSize value for training the network. This will ensure that during training the network is trained with objects at their original scale at the cost of losing some of the scene context.

If loss of scene context is not permitted by your application, then you will have to set OutputSize to 'resize'.

See here for more details on the OutputSizeMode and OutputSize properties:

https://www.mathworks.com/help/vision/ref/nnet.cnn.layer.pixellabelimagesource.html

Answer 2

[azza elbagoury]

why the following error appears when training the fully convolution network. this network is built with (fcnLayers)function??

>>lgraph = fcnLayers(imageSize,numClasses,'type','8s'); >> [net, info] = trainNetwork(datasource,lgraph,options); Training on single CPU. Initializing image normalization.

Error using trainNetwork (line 154) Padding exceeds array bounds. Caused by: Error using builtin Padding exceeds array bounds.

[Birju Patel]

This is a bug in fncLayers when training on the CPU. GPU training works.

I've attached a script that shows how to workaround this bug by inserting a custom padding layer between the input layer and the first conv layer.

This should let you train on the CPU. However, training an FCN model will be quite slow on the CPU, so beware.

[Bhavna Rajasekaran]

The problem is that the workaround allows for padding, but there is conflict between inputsize image fclayers (2 -d input) and trainNetwork. Once the layers are connected, the input size is [ 300 300 3] / [224 224 3].

%% code snippet after reconnecting lgraph from workaround..

inputSize = lgraph.Layers(1,1).InputSize;

imds.ReadFcn = @(loc)imresize(imread(loc),inputSize);

pxds.ReadFcn = @(loc1)imresize(imread(loc1),inputSize);

ds = pixelLabelImageDatastore(imds,pxds);

net = trainNetwork(ds,lgraph,options);

%%

%%trainNetwork gives error!!

Error using ReadFcn @(loc)imresize(imread(loc),inputSize) function handle for file

Error using matlab.images.internal.iptcheckmap (line 55)

Function IMRESIZE expected input number 2, MAP, to be a valid colormap. Valid colormaps cannot have values

outside the range [0,1].

Error in imresize>parsePreMethodArgs (line 366)

matlab.images.internal.iptcheckmap(map, mfilename, 'MAP', 2);

Error in imresize>parseInputs (line 260)

[params.A, params.map, params.scale, params.output_size] = ...

Error in imresize (line 146)

params = parseInputs(args{:});

Error in @(loc)imresize(imread(loc),inputSize)

%%

Something else I missed??

Thanks.