Why i Get low accuracy when i give unseen data to Trained Model?

4 views (last 30 days)
I have combine dataset of signals which have 14 classes. I have split them using
imds = imageDatastore('E:\SNR-Dataset\DATA-11-time\Data-for-training\', 'FileExtensions', '.mat', 'IncludeSubfolders',true, ...
[imdsTrain,imdsValidation, imdsTest] = splitEachLabel(imds,0.7,0.2, 'randomized');
[net2,tr] = trainNetwork(augimdsTrain,lgraph,options);
imdsTest_resize = augmentedImageDatastore([224,224],imdsTest);
[YPred,probs] = classify(net2,imdsTest_resize);
accuracy = mean(YPred == imdsTest.Labels)
Whenever i use imdsTest from splitEachLabel it give me 99% accuracy (Note that the train validation and test are in one folder)
I have unseen data which save in different folder and i use the following code to check the model accuracy on unseen data
imdsTest1 = imageDatastore('E:\SNR-Dataset\DATA-11-time\snr-test-data\Final-Test-data\snr30', 'FileExtensions', '.mat', 'IncludeSubfolders',true, ...
imdsTest_resize1 = augmentedImageDatastore([224,224],imdsTest1);
[YPred,probs] = classify(net2,imdsTest_resize1);
accuracy = mean(YPred == imdsTest1.Labels)
i got the 30% test accuracy
Please Assist why i get low accuracy when testing a model on unssen data which are in saparate folder?

Answers (2)

yanqi liu
yanqi liu on 7 Mar 2022
may be modify layers,add some dropoutLayer
if possible,may be upload data and code to debug

john karli
john karli on 7 Mar 2022
Edited: john karli on 7 Mar 2022
I am using the same link for data generation and training a model. the above code is modified version of the below link. you can generate the data and test it.
Out of 10,000 sample. I have use first 5000 samples per modulation scheme for training. and used last 500 (9,501:10,000) for testing purpose (saved in different folder).


Find more on Deep Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!