- Training Progress Window: You can find the final learning rate which was used by the network. Since, the learning rate schedule is constant it would remain 0.01, even at the end of the training.
- Using ‘info’ object: ‘trainNetwork’ method return two objects one is ‘net’ and other is ‘info’ which is a struct and contains an array of learning rates named ‘BaseLearnRate’, you can access the last element of this array to know the final learning rate the model used. In your case you could replace the line
How can I learn the final learning rate of a deep neural network?
7 views (last 30 days)
Show older comments
After completing the training procedure of a deep neural network over a spesific dataset, how can we learn the last, final, optimum learning rate? We specify a certain learning rate at the beginning, it is OK, but it reaches to an optimum value? What is it? How can I learn it?
Could you please refer a sample test case below:
clc;clear;
[XTrain,YTrain] = japaneseVowelsTrainData;
figure
plot(XTrain{1}')
title("Training Observation 1")
numFeatures = size(XTrain{1},1);
legend("Feature " + string(1:numFeatures),'Location','northeastoutside')
inputSize = 12;
numHiddenUnits = 100;
numClasses = 9;
layers = [ sequenceInputLayer(inputSize)
lstmLayer(numHiddenUnits,'OutputMode','last')
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer]
maxEpochs = 70;
miniBatchSize = 27;
options = trainingOptions('sgdm', ...
'ExecutionEnvironment','cpu', ...
'MaxEpochs',maxEpochs, ...
'MiniBatchSize',miniBatchSize, ...
'GradientThreshold',1, ...
'Verbose',false, ...
'Plots','training-progress');
net = trainNetwork(XTrain,YTrain,layers,options);
[XTest,YTest] = japaneseVowelsTestData;
YPred = classify(net,XTest,'MiniBatchSize',miniBatchSize);
acc = sum(YPred == YTest)./numel(YTest)
0 Comments
Answers (1)
Meet
on 23 Aug 2024
Hi Faruk,
From my understanding of your question, you want to know the final learning rate at the end of training the deep neural network.
There are two ways to do so, let me provide both the ways,
net = trainNetwork(XTrain,YTrain,layers,options);
to
[net,info] = trainNetwork(XTrain,YTrain,layers,options);
finalLearningRate = info.BaseLearnRate(end);
I hope these suggestions prove helpful for your task!
You can refer to this article for more detailed information:
0 Comments
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!