Regularize an ensemble of bagged trees.

Generate sample data.

You can create a bagged classification ensemble of 300 trees from the sample data.

bag = fitrensemble(X,Y,'Method','Bag','NumLearningCycles',300);

`fitrensemble`

uses a default template tree object `templateTree()`

as a weak learner when `'Method'`

is `'Bag'`

. In this example, for reproducibility, specify `'Reproducible',true`

when you create a tree template object, and then use the object as a weak learner.

Regularize the ensemble of bagged regression trees.

Starting lasso minimization for Lambda=0.001. Initial MSE=0.110607.
Lasso minimization completed pass 1 for Lambda=0.001
MSE = 0.0899652
Relative change in MSE = 0.229442
Number of learners with non-zero weights = 12
Lasso minimization completed pass 2 for Lambda=0.001
MSE = 0.064488
Relative change in MSE = 0.39507
Number of learners with non-zero weights = 43
Lasso minimization completed pass 3 for Lambda=0.001
MSE = 0.0608422
Relative change in MSE = 0.0599211
Number of learners with non-zero weights = 64
Lasso minimization completed pass 4 for Lambda=0.001
MSE = 0.060069
Relative change in MSE = 0.0128723
Number of learners with non-zero weights = 82
Lasso minimization completed pass 5 for Lambda=0.001
MSE = 0.0599398
Relative change in MSE = 0.00215497
Number of learners with non-zero weights = 96
Lasso minimization completed pass 6 for Lambda=0.001
MSE = 0.0599369
Relative change in MSE = 4.80374e-05
Number of learners with non-zero weights = 109
Lasso minimization completed pass 7 for Lambda=0.001
MSE = 0.0599364
Relative change in MSE = 9.35973e-06
Number of learners with non-zero weights = 113
Lasso minimization completed pass 8 for Lambda=0.001
MSE = 0.0599364
Relative change in MSE = 1.99253e-08
Number of learners with non-zero weights = 114
Lasso minimization completed pass 9 for Lambda=0.001
MSE = 0.0599364
Relative change in MSE = 5.04823e-08
Number of learners with non-zero weights = 113
Completed lasso minimization for Lambda=0.001.
Resubstitution MSE changed from 0.110607 to 0.0599364.
Number of learners reduced from 300 to 113.
Starting lasso minimization for Lambda=0.1. Initial MSE=0.110607.
Lasso minimization completed pass 1 for Lambda=0.1
MSE = 0.113013
Relative change in MSE = 0.0212927
Number of learners with non-zero weights = 10
Lasso minimization completed pass 2 for Lambda=0.1
MSE = 0.086583
Relative change in MSE = 0.30526
Number of learners with non-zero weights = 27
Lasso minimization completed pass 3 for Lambda=0.1
MSE = 0.080426
Relative change in MSE = 0.0765551
Number of learners with non-zero weights = 42
Lasso minimization completed pass 4 for Lambda=0.1
MSE = 0.0795375
Relative change in MSE = 0.0111715
Number of learners with non-zero weights = 57
Lasso minimization completed pass 5 for Lambda=0.1
MSE = 0.0792383
Relative change in MSE = 0.00377496
Number of learners with non-zero weights = 67
Lasso minimization completed pass 6 for Lambda=0.1
MSE = 0.0786905
Relative change in MSE = 0.00696198
Number of learners with non-zero weights = 75
Lasso minimization completed pass 7 for Lambda=0.1
MSE = 0.0787969
Relative change in MSE = 0.00134974
Number of learners with non-zero weights = 77
Lasso minimization completed pass 8 for Lambda=0.1
MSE = 0.0788049
Relative change in MSE = 0.00010252
Number of learners with non-zero weights = 87
Lasso minimization completed pass 9 for Lambda=0.1
MSE = 0.0788065
Relative change in MSE = 1.98213e-05
Number of learners with non-zero weights = 87
Completed lasso minimization for Lambda=0.1.
Resubstitution MSE changed from 0.110607 to 0.0788065.
Number of learners reduced from 300 to 87.

`regularize`

reports on its progress.

Inspect the resulting regularization structure.

ans = *struct with fields:*
Method: 'Lasso'
TrainedWeights: [300x2 double]
Lambda: [1.0000e-03 0.1000]
ResubstitutionMSE: [0.0599 0.0788]
CombineWeights: @classreg.learning.combiner.WeightedSum

Check how many learners in the regularized ensemble have positive weights. These are the learners included in a shrunken ensemble.

Shrink the ensemble using the weights from `Lambda = 0.1`

.

cmp =
classreg.learning.regr.CompactRegressionEnsemble
ResponseName: 'Y'
CategoricalPredictors: []
ResponseTransform: 'none'
NumTrained: 87
Properties, Methods

The compact ensemble contains `87`

members, less than 1/3 of the original `300`

.