Improve generalized linear regression model by adding or removing terms
mdl1 = step(mdl)
mdl1 = step(mdl,Name,Value)
specifies additional options using one or more name-value pair arguments. For example,
you can specify the criterion to use to add or remove terms and the maximum number of
steps to take.
mdl1 = step(
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Criterion'— Criterion to add or remove terms
Criterion to add or remove terms, specified as the comma-separated pair consisting of
'Criterion' and one of the following:
'Deviance' — p-value for F or
chi-squared test of the change in the deviance by adding or removing the term.
F-test is for testing a single model. Chi-squared test is
for comparing two different models.
'sse' — p-value for an F-test of the
change in the sum of squared error by adding or removing the term.
'aic' — Change in the value of Akaike information criterion (AIC).
'bic' — Change in the value of Bayesian information criterion (BIC).
'rsquared' — Increase in the value of R2.
'adjrsquared' — Increase in the value of adjusted R2.
'PEnter'— Threshold for criterion to add term
Threshold for the criterion to add a term, specified as the comma-separated pair
'PEnter' and a scalar value, as described in this
|0.05||If the p-value of F-statistic
or chi-squared statistic is less than |
|0.05||If the SSE of the model is less than |
|0||If the change in the AIC of the model is less than
|0||If the change in the BIC of the model is less than
|0.1||If the increase in the R-squared value of the model is greater than
|0||If the increase in the adjusted R-squared value of the model is
greater than |
For more information, see the
Criterion name-value pair
'PRemove'— Threshold for criterion to remove term
Threshold for the criterion to remove a term, specified as the comma-separated pair
'PRemove' and a scalar value, as described in this
|0.10||If the p-value of F-statistic
or chi-squared statistic is greater than |
|0.10||If the p-value of the F statistic is greater than
|0.01||If the change in the AIC of the model is greater than
|0.01||If the change in the BIC of the model is greater than
|0.05||If the increase in the R-squared value of the model is less than
|-0.05||If the increase in the adjusted R-squared value of the model is less
At each step, the
step function also checks whether a term
is redundant (linearly dependent) with other terms in the current model. When any term
is linearly dependent with other terms in the current model, the
step function removes the redundant term, regardless of
the criterion value.
For more information, see the
Criterion name-value pair
Fit a Poisson regression model using random data and a single predictor, then step in other predictors.
Generate artificial data with 20 predictors, using three of the predictors for the responses.
rng('default') % for reproducibility X = randn(100,20); mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1); y = poissrnd(mu);
Construct a generalized linear model using
X(:,1) as the only predictor.
mdl = fitglm(X,y,... 'y ~ x1','Distribution','poisson')
mdl = Generalized linear regression model: log(y) ~ 1 + x1 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ ______ __________ (Intercept) 1.1278 0.057487 19.618 1.0904e-85 x1 0.061287 0.04848 1.2642 0.20617 100 observations, 98 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 1.59, p-value = 0.208
Add a variable to the model using
mdl1 = step(mdl)
1. Adding x5, Deviance = 134.2976, Chi2Stat = 50.80176, PValue = 1.021821e-12
mdl1 = Generalized linear regression model: log(y) ~ 1 + x1 + x5 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ _______ __________ (Intercept) 1.0418 0.062341 16.712 1.07e-62 x1 0.018803 0.049916 0.37671 0.70639 x5 0.47881 0.067875 7.0542 1.7357e-12 100 observations, 97 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 52.4, p-value = 4.21e-12
Add another variable to the model using
mdl1 = step(mdl1)
2. Adding x15, Deviance = 105.9973, Chi2Stat = 28.30027, PValue = 1.038814e-07
mdl1 = Generalized linear regression model: log(y) ~ 1 + x1 + x5 + x15 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ _______ __________ (Intercept) 1.0459 0.0627 16.681 1.7975e-62 x1 0.026907 0.05003 0.53782 0.5907 x5 0.3983 0.068376 5.8251 5.7073e-09 x15 0.28949 0.053992 5.3618 8.2375e-08 100 observations, 96 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 80.7, p-value = 2.18e-17
Stepwise regression is a systematic method
for adding and removing terms from a linear or generalized linear
model based on their statistical significance in explaining the response
variable. The method begins with an initial model, specified using
and then compares the explanatory power of incrementally larger and
step function uses forward and backward stepwise regression to
determine a final model. At each step, the function searches for terms to add to the model
or remove from the model based on the value of the
The default value of
'Criterion' for a linear regression model is
'sse'. In this case,
LinearModel use the
p-value of an F-statistic to test models with and
without a potential term at each step. If a term is not currently in the model, the null
hypothesis is that the term would have a zero coefficient if added to the model. If there is
sufficient evidence to reject the null hypothesis, the function adds the term to the model.
Conversely, if a term is currently in the model, the null hypothesis is that the term has a
zero coefficient. If there is insufficient evidence to reject the null hypothesis, the
function removes the term from the model.
Stepwise regression takes these steps when
Fit the initial model.
Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.
If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.
At any stage, the function will not add a higher-order term if the model does not also include
all lower-order terms that are subsets of the higher-order term. For example, the function
will not try to add the term
X1:X2^2 unless both
X2^2 are already in the model. Similarly, the function will not
remove lower-order terms that are subsets of higher-order terms that remain in the model.
For example, the function will not try to remove
X1:X2^2 remains in the model.
You can specify other criteria by using the
'Criterion' name-value pair
argument. For example, you can specify the change in the value of the Akaike information
criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion
to add or remove terms.
Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.
step treats a categorical predictor as follows:
A model with a categorical predictor that has L levels
(categories) includes L – 1 indicator variables. The model uses the first category as a
reference level, so it does not include the indicator variable for the reference
level. If the data type of the categorical predictor is
categorical, then you can check the order of categories
categories and reorder the
categories by using
reordercats to customize the
step treats the group of L – 1 indicator variables as a single variable. If you want to treat
the indicator variables as distinct predictor variables, create indicator
variables manually by using
dummyvar. Then use the
indicator variables, except the one corresponding to the reference level of the
categorical variable, when you fit a model. For the categorical predictor
X, if you specify all columns of
dummyvar(X) and an intercept term as predictors, then the
design matrix becomes rank deficient.
Interaction terms between a continuous predictor and a categorical predictor with L levels consist of the element-wise product of the L – 1 indicator variables with the continuous predictor.
Interaction terms between two categorical predictors with L and M levels consist of the (L – 1)*(M – 1) indicator variables to include all possible combinations of the two categorical predictor levels.
You cannot specify higher-order terms for a categorical predictor because the square of an indicator is equal to itself.
step adds or removes
a categorical predictor, the function actually adds or removes the group of indicator variables
in one step. Similarly, if
step adds or removes an interaction term
with a categorical predictor, the function actually adds or removes the group of interaction
terms including the categorical predictor.
'' (empty character vector),
"" (empty string),
<undefined> values in
Y to be missing values.
step does not use observations with missing values in the fit.
ObservationInfo property of a fitted model indicates whether or not
step uses each observation in the fit.
stepwiseglm to select a model from a starting model, continuing until no
single step is beneficial.