MATLAB Answers

0

How to re-train a model optimized by Bayesian optimization on new data?

Asked by Sebastian on 20 Sep 2019 at 9:30
Latest activity Edited by Don Mathis on 20 Sep 2019 at 18:44
Hi,
I optimized a regression ensemble
model = fitrensemble(X,Y,...);
with active HyperparameterOptimizationOptions, so the resulting model object of type RegressionBaggedEnsemble contains a HyperparameterOptimizationResults object. How can I use this easily to re-train the model on a new data set with the best point found by the hyperparameter optimization algorithm? Something like this would be nice:
newModel = fitrensemble(Xnew,YNew,model.HyperparameterOptimizationResults.bestPoint);
My current approach is tedious, because I have to distinguish between the method the optimization algorithm selected (Bag,LSBoost) and set all parameters manually (and I'm even not sure if this is correct):
best = model.HyperparameterOptimizationResults.bestPoint;
ttmp = templateTree('MinLeafSize',best.MinLeafSize,'MaxNumSplits',...
best.MaxNumSplits,'NumVariablesToSample',best.NumVariablesToSample);
if best.Method=='Bag' %#ok<BDSCA>
newModel = fitrensemble(Xnew,YNew,...
'Method','Bag','Learners',ttmp,'NumLearningCycles',best.NumLearningCycles);
else
newModel = fitrensemble(Xnew,YNew,...
'Method','LSBoost','Learners',ttmp,'NumLearningCycles',best.NumLearningCycles,'LearnRate',best.LearnRate);
end
This question is not restricted to fitrensemble, but includes all similar model functions available in Statistics and Machine Learning Toolbox (fitrlinear, fitrgp ...).

  0 Comments

Sign in to comment.

1 Answer

Answer by Don Mathis on 20 Sep 2019 at 18:42
Edited by Don Mathis on 20 Sep 2019 at 18:44
 Accepted Answer

Your current approach (explicitly passing the values that the optimization found) is the right way to do it.
There is a faster/simpler way which uses undocumented functionality, and which therefore may change in the future. So, use at your own risk. You create a template from your model and then fit it:
model = fitrensemble(X,Y,...);
tmp = classreg.learning.FitTemplate.makeFromModelParams(model.ModelParameters);
newModel = fit(tmp,Xnew,Ynew)

  0 Comments

Sign in to comment.