Why i Get low accuracy when i give unseen data to Trained Model?

1 view (last 30 days)
I have combine dataset of signals which have 14 classes. I have split them using
imds = imageDatastore('E:\SNR-Dataset\DATA-11-time\Data-for-training\', 'FileExtensions', '.mat', 'IncludeSubfolders',true, ...
[imdsTrain,imdsValidation, imdsTest] = splitEachLabel(imds,0.7,0.2, 'randomized');
[net2,tr] = trainNetwork(augimdsTrain,lgraph,options);
imdsTest_resize = augmentedImageDatastore([224,224],imdsTest);
[YPred,probs] = classify(net2,imdsTest_resize);
accuracy = mean(YPred == imdsTest.Labels)
Whenever i use imdsTest from splitEachLabel it give me 99% accuracy (Note that the train validation and test are in one folder)
I have unseen data which save in different folder and i use the following code to check the model accuracy on unseen data
imdsTest1 = imageDatastore('E:\SNR-Dataset\DATA-11-time\snr-test-data\Final-Test-data\snr30', 'FileExtensions', '.mat', 'IncludeSubfolders',true, ...
imdsTest_resize1 = augmentedImageDatastore([224,224],imdsTest1);
[YPred,probs] = classify(net2,imdsTest_resize1);
accuracy = mean(YPred == imdsTest1.Labels)
i got the 30% test accuracy
Please Assist why i get low accuracy when testing a model on unssen data which are in saparate folder?

Answers (2)

yanqi liu
yanqi liu on 7 Mar 2022
may be modify layers,add some dropoutLayer
if possible,may be upload data and code to debug

john karli
john karli on 7 Mar 2022
Edited: john karli on 7 Mar 2022
I am using the same link for data generation and training a model. the above code is modified version of the below link. you can generate the data and test it.
Out of 10,000 sample. I have use first 5000 samples per modulation scheme for training. and used last 500 (9,501:10,000) for testing purpose (saved in different folder).

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!