Avoiding Overfitting by Averaging Multiple Neural Network

5 views (last 30 days)
Robert Henson
Robert Henson on 29 Jan 2018
Edited: Greg Heath on 25 May 2018
I’m using a narnet neural network to model stock time series data so I can get step-ahead and multi-step predictions. I’m concerned about overfitting in this model and have been reading in Matlab about techniques to avoid overfitting. I would like to try averaging multiple networks to improve generalization and train multiple neural networks and average their outputs (as in the example provided). The example shows this technique on a feedforward net. So, my question is (and please forgive me is I ask stupid questions since I’m still a relatively new user of Matlab and neural networks) can I use the averaging technique on my narnet network? If so, would the same example work? If not and I need to use the feedforward net, how then can I use that output in my model to find step-ahead and multi-step predictions for my stock time series? I would appreciate any input and/or suggestions. Many thanks. Robert Henson

Answers (1)

Greg Heath
Greg Heath on 25 May 2018
Edited: Greg Heath on 25 May 2018
The best way to mitigate overtraining an overfit net is
The conventional I-H-O NN has I Input nodes, H Hidden nodes, O output nodes and Nw unknown weights where
Nw = (I+1)*H+(H+1)*O = (I+O+1)*H +1
With Ntrn training examples the total number of training equations is
Ntrneq = Ntrn*O
To prevent overfitting: No. eq >= No. unknowns:
Ntrneq >= Nw
H <= Hmax <= Hub = (Ntrn*O-1)/(I+O+1)
I have posted zillions of examples in BOTH
comp.soft-sys.matlab and ANSWERS.
Search on "greg" and one or more of the following Ntrneq, Hmax, Hub
Thank you for formally accepting my answer

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!