Invalid training data. The output size (7) of the last layer does not match the number of classes (10).

Hi,

I am trying to create a multi input-single output CNN. The two inputs have different sizes. This is the layer plot

I created a combined datastore with image input1 and input2 along with the labels. However while training the network I get an error as "Invalid training data. The output size (7) of the last layer does not match the number of classes (10)" . I have verified that there are 7 label categories and fullyconnected layer also has 7 output. I am unbale to determine how output class has become 10.

This is the code.

%Load and explore data
DatasetPath1 = "D:\DeepNetwork\Image_Datastore\4";
imdset1 = imageDatastore(DatasetPath1, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
labelCount1 = countEachLabel(imdset1);

DatasetPath2 = "D:\DeepNetwork\Image_Datastore\5";
imdset2 = imageDatastore(DatasetPath2, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
labelCount2 = countEachLabel(imdset2);
%Reading size of images
img1 = readimage(imdset1,2);
size_img_input1=size(img1)

img2 = readimage(imdset2,2);
size_img_input2=size(img2)

%Specify Training and Validation Sets
numTrainFiles = 60;
[imdsetTrain1,imdsetValidation1] = splitEachLabel(imdset1,numTrainFiles);

numTrainFiles = 60;
[imdsetTrain2,imdsetValidation2] = splitEachLabel(imdset2,numTrainFiles);


%Combine datastores with labels
read_size = 1;

TrainLabels = imdsetTrain1.Labels; % your labels here
% the order of your images and label must be the same
writematrix(TrainLabels,'TrainLabels.txt');
% %C = categorical
TrainlabelStore = tabularTextDatastore('TrainLabels.txt','TextscanFormats','%C',"ReadVariableNames",false);
TrainlabelStore.ReadSize = read_size;
TrainlabelStoreCell = transform(TrainlabelStore,@setcat_and_table_to_cell);

imdsCombinedTrain = combine(imdsetTrain1,imdsetTrain2, TrainlabelStoreCell);

ValidationLabels = imdsetValidation1.Labels; % your labels here
% the order of your images and label must be the same
writematrix(ValidationLabels,'ValidationLabels.txt');
% %C = categorical
ValidationlabelStore = tabularTextDatastore('ValidationLabels.txt','TextscanFormats','%C',"ReadVariableNames",false);
ValidationlabelStore.ReadSize = read_size;
ValidationlabelStoreCell = transform(ValidationlabelStore,@setcat_and_table_to_cell);

imdsCombinedValidation = combine(imdsetValidation1,imdsetValidation2, ValidationlabelStoreCell);

%Defining Layers
lgraph = layerGraph();
tempLayers = [
    imageInputLayer([size_img_input1(1) size_img_input1(2)],"Name","imageinput_1")
    convolution2dLayer([5 5],8,"Name","conv_1","Padding","same")
    batchNormalizationLayer("Name","batchnorm_1")
    reluLayer("Name","relu_1")
    maxPooling2dLayer([2 2],"Name","maxpool_1","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],16,"Name","conv_2","Padding","same")
    batchNormalizationLayer("Name","batchnorm_2")
    reluLayer("Name","relu_2")
    maxPooling2dLayer([2 2],"Name","maxpool_2","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],32,"Name","conv_3","Padding","same")
    batchNormalizationLayer("Name","batchnorm_3")
    reluLayer("Name","relu_3")];
lgraph = addLayers(lgraph,tempLayers);

tempLayers = [
    imageInputLayer([size_img_input2(1) size_img_input2(2)],"Name","imageinput_2")
    convolution2dLayer([5 5],8,"Name","conv_4","Padding","same")
    batchNormalizationLayer("Name","batchnorm_4")
    reluLayer("Name","relu_4")
    maxPooling2dLayer([2 2],"Name","maxpool_4","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],16,"Name","conv_5","Padding","same")
    batchNormalizationLayer("Name","batchnorm_5")
    reluLayer("Name","relu_5")
    maxPooling2dLayer([2 2],"Name","maxpool_5","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],32,"Name","conv_6","Padding","same")
    batchNormalizationLayer("Name","batchnorm_6")
    reluLayer("Name","relu_6")];
lgraph = addLayers(lgraph,tempLayers);

tempLayers = [
    concatenationLayer(2,2,"Name","concat")
    convolution2dLayer([5 5],8,"Name","conv_7","Padding","same")
    batchNormalizationLayer("Name","batchnorm_7")
    reluLayer("Name","relu_7")
    maxPooling2dLayer([2 2],"Name","maxpool_7","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],16,"Name","conv_8","Padding","same")
    batchNormalizationLayer("Name","batchnorm_8")
    reluLayer("Name","relu_8")
    maxPooling2dLayer([2 2],"Name","maxpool_8","Padding","same","Stride",[2 2])
    convolution2dLayer([5 5],32,"Name","conv_9","Padding","same")
    batchNormalizationLayer("Name","batchnorm_9")
    reluLayer("Name","relu_9")
    fullyConnectedLayer(7,"Name","fc")
    softmaxLayer("Name","softmax")
    classificationLayer("Name","classoutput")];
lgraph = addLayers(lgraph,tempLayers);

% clean up helper variable
clear tempLayers;

lgraph = connectLayers(lgraph,"relu_6","concat/in2");
lgraph = connectLayers(lgraph,"relu_3","concat/in1");
plot(lgraph);

%%
options = trainingOptions('sgdm', ...
    'InitialLearnRate',0.01, ...
    'MaxEpochs',10, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsCombinedValidation, ...
    'ValidationFrequency',3, ...
    'Verbose',false, ...
    'Plots','training-progress', 'ExecutionEnvironment','cpu');

net = trainNetwork(imdsCombinedTrain,lgraph,options);

%Below is the function to create combined datastore with labels (refer https://www.mathworks.com/matlabcentral/answers/586949-multi-input-imagedatastore):
function [dataout] = setcat_and_table_to_cell(datain)
validcats = string(0:9); % define valid labels for categorical array
datain.(1) = setcats(datain.(1),validcats);
dataout = table2cell(datain);
end

Answer 1

[Mohammad Sami]

This is because of this part of the setcat_and_table_to_cell function

validcats = string(0:9); % define valid labels for categorical array
datain.(1) = setcats(datain.(1),validcats);

It sets 10 valid categories. You should change the validcats to the appropriate 7 categories for your case.

[Girish Tiwari]

Thanks.

While training the network with this datastore, I came to know that the combined datastore is not shuffleable. Is there any way we can make it shuffleable?

I did some search and got to know that by using TabularTextDatastore, the datastore is not shuffleable because the underlying TabularTextDatastore objects do not have subset methods. (refer https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.isshuffleable.html). Also while training the network parallel option in execution environment does not work, may be because of datastore not being shufflelable.

[Mohammad Sami]

We can potentially make a custom datastore just for the labels.