How to determine the correct number of Validation Frequency during neural network training
54 views (last 30 days)
Show older comments
Hello
How is the appropriate and optimal value for parameter Validation Frequency and mini batch size calculated?
Assuming we have 2,000 training and 200 validation data and 20 epochs
Is a value of 10 suitable for Validation Frequency and value of 20 for mini batch size?
0 Comments
Answers (1)
Ullah Nadeem
on 20 Oct 2022
Dear Lech!
Mini-batch size as good as large as your system memory can afford. Large mini-batch size results in more number of training samples to be learned by model which is good to generalize the problem and decrease the loss quickly.
VF: Validation Frequency
As far as I know, the selection of validation frequency depends on the following:
(1) The amount of training data
If the training data is large and the VF if small number (let's say after 30 iterations), the model will not learn enough while validating more oftenly results in long training time and maybe stop the training early if the 'ValidationPatience' is not set to 'Inf'.
(2) How fast the model learns the data well
If a problem is easy and the model learns it quickly, the VF supposed to be in range of 1/15th times of the iterations/epoch to 1/7th times of the iterations/epoch to let the network stop early and not gets overfit. But if the problem is complicated and model can't learns it quickly, then it is better to select the VF in range of 1/7th times of the iterations/epoch to 1/3th times of the iterations/epoch to let the model learns enough before validating.
Note: Iterations/epoch can be calculated as total number of training samples/mini-batch size.
0 Comments
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!