MATLAB Answers

How to split matrix data into training, validation and testing data based on a categorical label?

86 views (last 30 days)
I have a 184 x 1 cell array called aefm_4 and each cell is 1 x 1599. Each row (184) has a corresponding categorical label in a separate variable called f and its either labeled 'crack-related' or 'noise'. I want to split up my data so that I have 70% training data, 10% validation data and 20% testing data. I want about even amounts of noise and crack related signals in each training, validation and testing dataset. How would I do that?
Right now, I have been manually separating them as seen below but I will be running this for more data and it is too time consuming to do this.
dataTrain = aefm_4([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:); %127 hits (70%)
fTrain = f([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:);
dataVal = aefm_4([12:13,26:27,53:54,83:86,92,105:107,163:168],:); %20 hits (10%)
fVal = f([12:13,26:27,53:54,83:86,92,105:107,163:168],:);
dataTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:); %39 hits (20%)
fTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:);

Accepted Answer

Srivardhan Gadila
Srivardhan Gadila on 23 Aug 2020
You can do something like below:
%% Create categorical label data
numLabels = 18
labels = randi([1 2],numLabels,1);
labelsCat = categorical(labels,[1 2],{'crack-related','noise'});
%% find lndices w.r.t each label
crackInd = find(labelsCat==categorical("crack-related"));
noiseInd = find(labelsCat==categorical("noise"));
%% divide into train, val, test for each label
[trainCrack,valCrack,testCrack] = dividerand(numel(crackInd),0.7,0.1,0.2);
[trainNoise,valNoise,testNoise] = dividerand(numel(noiseInd),0.7,0.1,0.2);
%% get final indices
trainInd = [crackInd(trainCrack); noiseInd(trainNoise)];
valInd = [crackInd(valCrack); noiseInd(valNoise)];
testInd = [crackInd(testCrack); noiseInd(testNoise)];
You can optimize the above code based on your requirement like, dividing the indices randomly or not. Refer to Divide Data for Optimal Neural Network Training, dividerand, divideblock, divideint, divideind for more information.
If you want to randomize final indices then you can make use of randperm as follows:
trainInd = trainInd(randperm(numel(trainInd)));

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!