# Avoiding Overfitting by Averaging Multiple Neural Network

7 views (last 30 days)
Robert Henson on 29 Jan 2018
Edited: Greg Heath on 25 May 2018
I’m using a narnet neural network to model stock time series data so I can get step-ahead and multi-step predictions. I’m concerned about overfitting in this model and have been reading in Matlab about techniques to avoid overfitting. I would like to try averaging multiple networks to improve generalization and train multiple neural networks and average their outputs (as in the example provided). The example shows this technique on a feedforward net. So, my question is (and please forgive me is I ask stupid questions since I’m still a relatively new user of Matlab and neural networks) can I use the averaging technique on my narnet network? If so, would the same example work? If not and I need to use the feedforward net, how then can I use that output in my model to find step-ahead and multi-step predictions for my stock time series? I would appreciate any input and/or suggestions. Many thanks. Robert Henson

Greg Heath on 25 May 2018
Edited: Greg Heath on 25 May 2018
The best way to mitigate overtraining an overfit net is
MINIMIZE THE NUMBER OF HIDDEN NODES SUBJECT TO A MAXIMUM ALLOWED ERROR RATE.
The conventional I-H-O NN has I Input nodes, H Hidden nodes, O output nodes and Nw unknown weights where
Nw = (I+1)*H+(H+1)*O = (I+O+1)*H +1
With Ntrn training examples the total number of training equations is
Ntrneq = Ntrn*O
To prevent overfitting: No. eq >= No. unknowns:
Ntrneq >= Nw
or
H <= Hmax <= Hub = (Ntrn*O-1)/(I+O+1)
I have posted zillions of examples in BOTH
Search on "greg" and one or more of the following Ntrneq, Hmax, Hub
Thank you for formally accepting my answer
Greg