How can I train neural network with a large data

Hi all, I have a large training dataset to use with a neural network (the that is over 55000 entries ). When using the full training data i keep getting the error that I exceeded the maximum array size preference. I was wondring if anyone knows how can I train my neural network on many steps on each step I use a portion of the training data. Thank you in advance.

Answer 1

[Greg Heath]

INSUFFICIENT INFORMATION!!!

 1. REGRESSION or CLASSIFICATION?
 2. DIMENSIONALITY OF INPUT & OUTPUT VECTORS?

If you haven't started yet you should consider removing unnecessary inputs and outputs that contain redundant information i.e., those vectors that can be obtained from a linear combination of the others.

To do this you might consider PCA for regression and PLS for classification.

Then using the reduced dataset:

 Loop over as many times as necessary:
 1. Divide the total data into subsets. 
 2. Design a net from each subset.
 3. Run all of the subsets through all of the nets. 
 4. Add poorly performing vectors to the training subset for that net.

Hope this gives you some ideas.

Thank you for formally accepting my answer

Greg

Answer 2

[Walter Roberson]

See posting #11 in https://www.mathworks.com/matlabcentral/newsreader/view_thread/128799