How to determine the correct number of Validation Frequency during neural network training

54 views (last 30 days)
Hello
How is the appropriate and optimal value for parameter Validation Frequency and mini batch size calculated?
Assuming we have 2,000 training and 200 validation data and 20 epochs
Is a value of 10 suitable for Validation Frequency and value of 20 for mini batch size?

Answers (1)

Ullah Nadeem
Ullah Nadeem on 20 Oct 2022
Dear Lech!
Mini-batch size as good as large as your system memory can afford. Large mini-batch size results in more number of training samples to be learned by model which is good to generalize the problem and decrease the loss quickly.
VF: Validation Frequency
As far as I know, the selection of validation frequency depends on the following:
(1) The amount of training data
If the training data is large and the VF if small number (let's say after 30 iterations), the model will not learn enough while validating more oftenly results in long training time and maybe stop the training early if the 'ValidationPatience' is not set to 'Inf'.
(2) How fast the model learns the data well
If a problem is easy and the model learns it quickly, the VF supposed to be in range of 1/15th times of the iterations/epoch to 1/7th times of the iterations/epoch to let the network stop early and not gets overfit. But if the problem is complicated and model can't learns it quickly, then it is better to select the VF in range of 1/7th times of the iterations/epoch to 1/3th times of the iterations/epoch to let the model learns enough before validating.
Note: Iterations/epoch can be calculated as total number of training samples/mini-batch size.

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!