Clear Filters
Clear Filters

How to retrain existing and trained Neural Network without destroying trained content?

72 views (last 30 days)
Matlab train() function used for training the neural network initializes all weights and other internal parameters of the network at the beginning. I would like to take a trained network and train it further using new set of data without reinitializing and starting from scratch (destroying the trained net basically). How do I do it ? Is there a way to use existing train() function or is there some re-train() function or do I need to build my own custom solution ? The last option seems unlikely, given the amount of the existing Matlab code. Please advise, Thanks, Mirek Wilmer mirek.wilmer@gmail.com

Answers (4)

Greg Heath
Greg Heath on 7 Aug 2018
Edited: Greg Heath on 7 Aug 2018
TRAIN initializes weights ONLY IF ALL weights are zero.
OTHERWISE
TRAIN will update weights with the new data.
Therefore old weights will be modified to fit the new data.
If the new data is unlike the old data, performance on the old data will be degraded.
In order to prevent "FORGETTING", a characteristic subset of the old data SHOULD BE MIXED with the new data for the retraining.
Hope this helps.
Thank you for formally accepting my answer
Greg

Greg Heath
Greg Heath on 3 Aug 2018
In order to preserve the dominant characteristics of the 1st dataset you must include that information while adapting to the new data. Therefore I always represent the 1st data set by a representative subset and add that to the new set.
I have been most successful when using Elliptic or Gaussian basis functions (with limited effective range) in the net.
Hope this helps.
Thank you for formally accepting my answer
Greg

KSSV
KSSV on 23 Apr 2018
The function train gives out a structure variable net with all the information of training the inputs have under gone.
[net,tr] = train(net,.......)
If you don't clear your workspace, you can use the variable net for what you want. It will retain all the previous data.
[net,tr] = train(net,newdata.....)
  4 Comments
Gideon Prior
Gideon Prior on 7 Aug 2018
That isnt the case unfortunately. Calls to 'train' reinitialize the weights, which is odd since there is a method 'init' already included in the toolbox. I dont know why the mathworks folks decided to not include an option to prevent initialization when 'train' is called.
Kinga Wilmer
Kinga Wilmer on 7 Aug 2018
As a mater of fact, if the workspace is not cleared and the definition of net is not changed, then train() function does not destroy the existing weights. I did trace the whole process with a debugger step by step and confirmed it. Therefore you can re-use your trained net for further training. Mind you, the additional data may re-train your net, so if you want to retain the existing trained capacity, you need to keep statistically important sample representation of your previous training data in the new training dataset. Therefore only small incremental retraining steps make sense. Another way is to train number of separate networks and combine them into hierarchical model.

Sign in to comment.


kira
kira on 26 Sep 2018
If you want to train incrementally on your data set, you can also do it all at once by dealing with them as concurrent data https://www.mathworks.com/help/deeplearning/ug/multiple-sequences-with-dynamic-neural-networks.html

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!