454 views (last 30 days)

I am trying to classify image datasets using deep learning.

after getting feature vector of each single image I ve got a matrix 18000x24000 which indicates to No. of images x features.

I used:

trainNetwork (X, Y, Layers, Options)

Where X is the train data and Y is the Labels which is 18000x1. But there is an error says Invalid training data. X and Y must have the same number of observations.

I think I should change the train matrix to 4-D but I don't know how, and if it is correct?

Bhartendu
on 8 Apr 2018

Try the following:

If your data ( 18000 data points) is in form of images with dimensions say, 120*200 (equals to 24000), then reshape it as follows:

X_train = reshape(X, [120,200,1,size(X,1)]);

This should create 4-D Matrix X_train of size (120,200,1,18000), then train the network using:

net = trainNetwork(X_train,Y,Layers,Options)

michael scheinfeild
on 9 Jun 2018

hi i have similar issue and cant solve it i also look at the example Sequence Classification Using Deep Learning

i try to classify between different signals based frequency future

%%generete the signal

fs=500;

f=20;

t=[0:1/fs:5-1/fs];

xsig=10*sin(2*pi*f*t);

sampleLen=length(xsig);

NFFT = 2.^nextpow2(sampleLen);

% signal fft

hwin=hamming(length(xsig))';

Y = fft(xsig.*hwin,NFFT);

magnitudeY = abs(Y);

xsampleFFT=20*log10(magnitudeY(1:NFFT/2));

figure,plot(xsampleFFT);title('signal fft')

% noise fft

xnoise= randn(size(xsig));

Yn = fft(xnoise.*hwin,NFFT);

magnitudeYn = abs(Yn);

xnoiseFFT=20*log10(magnitudeYn(1:NFFT/2));

figure,plot(xnoiseFFT);title('noise')

%%make the data

sampleLen = 100;

NFFT = 128;

hwin=hamming(sampleLen)';

nsamples=length(xsig)/sampleLen;

xsampleFFT={};%zeros(nsamples,NFFT/2);

hwin=hamming(sampleLen)';

kj=1;

for(k=1:sampleLen:length(xsig)-sampleLen+1)

cursig=xsig(k:k+sampleLen-1);

Y = fft(cursig.*hwin,NFFT);

magnitudeY = abs(Y); % Magnitude of the FFT

xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));

%figure,plot(xTrain{kj})

yTrain(kj)=categorical(1);

kj=kj+1;

end

disp('data types')

[size(xTrain) size(yTrain)]

[size(xTrain{1})]

class(xTrain)

class(yTrain)

%append

for(k=1:sampleLen:length(xnoise)-sampleLen+1)

cursig=xnoise(k:k+sampleLen-1);

Y = fft(cursig.*hwin,NFFT);

magnitudeY = abs(Y); % Magnitude of the FFT

xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));

%figure,plot(xTrain{kj})

yTrain(kj)=categorical(0);

kj=kj+1;

end

disp('data types')

[size(xTrain) size(yTrain)]

[size(xTrain{1})]

class(xTrain)

class(yTrain)

figure,plot(yTrain)

figure,subplot(2,1,1),plot(xTrain{10})

subplot(2,1,2);plot(xTrain{30})

function [net] = train_lstm(XTrainLoc,YTrainLoc)

%%lstm

inputSize = 1;

numHiddenUnits = 100;

numClasses = 2;

layers = [ ...

sequenceInputLayer(inputSize)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(numClasses)

softmaxLayer

classificationLayer];

maxEpochs = 1;

miniBatchSize = 100;

options = trainingOptions('adam', ...

'ExecutionEnvironment','cpu', ...

'GradientThreshold',1, ...

'MaxEpochs',maxEpochs, ...

'MiniBatchSize',miniBatchSize, ...

'SequenceLength','longest', ...

'Shuffle','never', ...

'Verbose',0, ...

'Plots','training-progress');

% options = trainingOptions('adam', ...

% 'ExecutionEnvironment','cpu', ...

% 'GradientThreshold',1, ...

% 'MaxEpochs',maxEpochs, ...

% 'MiniBatchSize',miniBatchSize, ...

% 'SequenceLength','longest', ...

% 'Shuffle','never', ...

% 'Verbose',0, ...

% 'Plots','training-progress',...

% 'ValidationData',{XValidation,YValidation},...

% 'ValidationPatience',Inf);

%%train

net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);

====== i recive error

[net] = train_lstm(xTrain,yTrain)

*Error using trainNetwork (line 154)

Invalid training data. If all recurrent layers have output

mode 'sequence', then the responses must be a cell array of

categorical sequences, or a categorical sequence.*

_Error in train_lstm (line 42) net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);

Caused by: Error using nnet.internal.cnn.util.NetworkDataValidator/assertOutputModeCorrespondsToDataForClassification (line 380) Invalid training data. If all recurrent layers have output mode 'sequence', then the responses must be a cell array of categorical sequences, or a categorical sequence._

so what is the issue i tried also change y to cell array of category , transpose the internal x, change network in. i think in this fft i have actually one sample each time with nfft feature. this is the same as the Japanese sample but they have 12 features

Sign in to comment.

SARAH LONER
on 29 Nov 2019

sir i also have the same issue cant able to train network for an image .

my work was image segmentation based on unet segment cant able to clear the code

it showing error at

net = trainNetwork(imds,layers,options);

where in imds i have taken image

kindly help to solve this error

SARAH LONER
on 30 Nov 2019

inp=b % my input image is dicom image

DatasetPath=fullfile('C:\Users\Desktop\to');

imds=imageDatastore(DatasetPath, 'IncludeSubfolders', true,...

'LabelSource','foldernames','fileextension',{'.dcm'});

labelDir = fullfile(DatasetPath,'testImages');

I = readimage(imds,1);

I = histeq(I);

imshow(I)

classes = [

"MALIGNANT","BENIGN"

];

labelIDs=[255 0]

inputlayer = imageInputLayer([512 512 1],'Name','inp')

numFilters = 64;

numLayers = 16;

layers = [ ...

imageInputLayer([512 512 1])

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

convolution2dLayer(5,20)

fullyConnectedLayer(4)

softmaxLayer

pixelClassificationLayer

]

pxds = pixelLabelDatastore(labelDir,classes,labelIDs);

options = trainingOptions('sgdm', ...

'InitialLearnRate',0.01, ...

'MaxEpochs',1, ...

'Shuffle','every-epoch', ...

'ValidationFrequency',30, ...

'Verbose',false);

ds = pixelImageDatastore(imds,pxds);

net=trainNetwork(ds,layers,options);

I = read(imds);

C = read(pxds)

C = semanticseg(I, net);

% Overlay pixel label data on the image and display.

B = labeloverlay(I, C);

figure(12)

imshow(B)

i got error at

ds = pixelImageDatastore(imds,pxds);

once i cleared that error means again i got error at

net=trainNetwork(ds,layers,options);

Sign in to comment.

Sign in to answer this question.

Opportunities for recent engineering grads.

Apply Today
## 2 Comments

## Direct link to this comment

https://nl.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_507064

⋮## Direct link to this comment

https://nl.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_507064

## Direct link to this comment

https://nl.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_775125

⋮## Direct link to this comment

https://nl.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_775125

Sign in to comment.