Why is my deeper CNN not learning
4 views (last 30 days)
Show older comments
Hello,
I am trying to build a CNN to identify certain parameters in images taken using holography. I first started using the built-in AlexNet and got great results. However, I think AlexNet is a bit overkill for the task and I wanted a simpler network for faster classification on slower computers. When I build a very simple network as such
layers = [ ...
imageInputLayer([150 150 1])
convolution2dLayer(11, 24, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
convolution2dLayer(5, 48, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
convolution2dLayer(3, 48, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
fullyConnectedLayer(100);
fullyConnectedLayer(19)
softmaxLayer
classificationLayer];
% Training options (default)
% Gradient descent with momentum
opts = trainingOptions('sgdm', ...
'Plots', 'training-progress', ...
'Verbose', true, ...
'MaxEpochs', 100, ...
'MiniBatchSize', 32, ...
'Momentum', 0.9, ...
'Shuffle', 'every-epoch', ...
'InitialLearnRate', 0.001, ...
'LearnRateSchedule', 'piecewise', ...
'LearnRateDropFactor', 0.25, ...
'LearnRateDropPeriod', 10, ...
'CheckpointPath', checkpointPath);
The network learns fine and reaches an accuracy of around 85-90 percent fairly quickly. However, as soon as I introduce any extra layers between the max pooling layers it takes a really long time for the network to learn. See below
layers = [ ...
imageInputLayer([150 150 1])
convolution2dLayer(11, 24, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
convolution2dLayer(5, 48, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
convolution2dLayer(3, 48, 'Padding', 'same')
reluLayer
convolution2dLayer(3, 48, 'Padding', 'same')
reluLayer
maxPooling2dLayer(2, 'Stride', 2)
fullyConnectedLayer(100);
fullyConnectedLayer(19)
softmaxLayer
classificationLayer];
What could be the reason for this? It is worth to note that there is not a lot of variation between the images, so changes are subtle between classes. Any tips for building CNNs that can handle subtle changes well?
0 Comments
Answers (1)
Kenta
on 16 Jul 2020
Constructiong a good CNN architechture from scratch is not easy. If a heavy network such as AlexNet is not necesarily required, how about using SqueezeNet? It is very light and can get good accuracy.
0 Comments
See Also
Categories
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!