Ridge regularization for lsboost regression

Dear all,

I would like to know if "Ridge regularization, i.e., L2" is applicable to the LSBoost regression model or not. After searching the MATLAB documentation, I found code only for "Lasso (L1) regularization."

For example:

t = templateTree(Reproducible=true);

bag = fitrensemble(X,Y,Method="Bag",NumLearningCycles=300,Learners=t);

bag = regularize(bag,Lambda=[0.001 0.1],Verbose=1);

Best regards,

Answer 1

[Sumukh]

Hi Farzad,

The regression ensembles can only be regularized using lasso regularization with the “regularize” command. The possible workarounds are as follows:

• The “fitrlinear” command can be used in place of the “fitrensemble” command to create regression model with the training data. Unlike the “fitrensemble” command, the “fitrlinear” command has the option to regularize the model using both “Ridge” and “Lasso” regularization. You can refer to the following documentation to know about the “fitrlinear” command along with the regularization option available:

https://www.mathworks.com/help/releases/R2023b/stats/fitrlinear.html#d126e465178

• The objective of regularization is to improve predictive performance and prevent overfitting. This is can indirectly achieved by tuning the ensemble parameters using the input arguments available in the “fitrensemble” command before regularization. The “LearnRate” option available specifically for “LSBoost” type ensembles can be tuned to improve the accuracy of the model. You can refer to the following tips about setting the input arguments for “fitrensemble” method to improve accuracy:

https://www.mathworks.com/help/releases/R2023b/stats/fitrensemble.html#bvcj_t1-14

I hope this answers your query.