Accelerating the pace of engineering and science

# ClassificationTree class

Superclasses: CompactClassificationTree

Binary decision tree for classification

## Description

A decision tree with binary splits for classification. An object of class ClassificationTree can predict responses for new data with the predict method. The object contains the data used for training, so can compute resubstitution predictions.

## Construction

tree = fitctree(X,Y) returns a classification tree based on the input variables (also known as predictors, features, or attributes) X and output (response) Y. tree is a binary tree, where each branching node is split based on the values of a column of X.

tree = fitctree(X,Y,Name,Value) fits a tree with additional options specified by one or more Name,Value pair arguments. If you use one of the following five options, tree is of class ClassificationPartitionedModel: 'CrossVal', 'KFold', 'Holdout', 'Leaveout', or 'CVPartition'. Otherwise, tree is of class ClassificationTree.

### Input Arguments

expand all

 X A matrix of numeric predictor values. Each column of X represents one variable, and each row represents one observation. NaN values in X are taken to be missing values. Observations with all missing values for X are not used in the fit. Observations with some missing values for X are used to find splits on variables for which these observations have valid values. Y A categorical array, cell array of strings, character array, logical vector, or a numeric vector with the same number of rows as X. Each row of y represents the classification of the corresponding row of X. For numeric Y, consider using fitrtree instead of fitctree. NaN values in Y are taken to be missing values. Observations with missing values for Y are not used in the fit.

#### Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

### 'AlgorithmForCategorical' — Algorithm for best categorical predictor split'Exact' | 'PullLeft' | 'PCA' | 'OVAbyClass'

Algorithm to find the best split on a categorical predictor with C categories for data and K ≥ 3 classes, specified as the comma-separated pair consisting of 'AlgorithmForCategorical' and one of the following.

 'Exact' Consider all 2C–1 – 1 combinations. 'PullLeft' Start with all C categories on the right branch. Consider moving each category to the left branch as it achieves the minimum impurity for the K classes among the remaining categories. From this sequence, choose the split that has the lowest impurity. 'PCA' Compute a score for each category using the inner product between the first principal component of a weighted covariance matrix (of the centered class probability matrix) and the vector of class probabilities for that category. Sort the scores in ascending order, and consider all C – 1 splits. 'OVAbyClass' Start with all C categories on the right branch. For each class, order the categories based on their probability for that class. For the first class, consider moving each category to the left branch in order, recording the impurity criterion at each move. Repeat for the remaining classes. From this sequence, choose the split that has the minimum impurity.

fitctree automatically selects the optimal subset of algorithms for each split using the known number of classes and levels of a categorical predictor. For K = 2 classes, fitctree always performs the exact search. Use the 'AlgorithmForCategorical' name-value pair argument to specify a particular algorithm.

Example: 'AlgorithmForCategorical','PCA'

### 'CategoricalPredictors' — Categorical predictors listnumeric or logical vector | cell array of strings | character matrix | 'all'

Categorical predictors list, specified as the comma-separated pair consisting of 'CategoricalPredictors' and one of the following:

• A numeric vector with indices from 1 through p, where p is the number of columns of X.

• A logical vector of length p, where a true entry means that the corresponding column of X is a categorical variable.

• A cell array of strings, where each element in the array is the name of a predictor variable. The names must match entries in PredictorNames values.

• A character matrix, where each row of the matrix is a name of a predictor variable. The names must match entries in PredictorNames values. Pad the names with extra blanks so each row of the character matrix has the same length.

• 'all', meaning all predictors are categorical.

Example: 'CategoricalPredictors','all'

Data Types: single | double | char

### 'ClassNames' — Class namesnumeric vector | categorical vector | logical vector | character array | cell array of strings

Class names, specified as the comma-separated pair consisting of 'ClassNames' and an array representing the class names. Use the same data type as the values that exist in Y.

Use ClassNames to order the classes or to select a subset of classes for training. The default is the class names that exist in Y.

Data Types: single | double | char | logical | cell

### 'Cost' — Cost of misclassificationsquare matrix | structure

Cost of misclassification of a point, specified as the comma-separated pair consisting of 'Cost' and one of the following:

• Square matrix, where Cost(i,j) is the cost of classifying a point into class j if its true class is i.

• Structure S having two fields: S.ClassNames containing the group names as a variable of the same data type as Y, and S.ClassificationCosts containing the cost matrix.

The default is Cost(i,j)=1 if i~=j, and Cost(i,j)=0 if i=j.

Data Types: single | double | struct

### 'CrossVal' — Flag to grow cross-validated decision tree'off' (default) | 'on'

Flag to grow a cross-validated decision tree, specified as the comma-separated pair consisting of 'CrossVal' and 'on' or 'off'.

If 'on', fitctree grows a cross-validated decision tree with 10 folds. You can override this cross-validation setting using one of the 'KFold', 'Holdout', 'Leaveout', or 'CVPartition' name-value pair arguments. Note that you can only use one of these four arguments at a time when creating a cross-validated tree.

Alternatively, cross validate tree later using the crossval method.

Example: 'CrossVal','on'

### 'CVPartition' — Partition for cross-validated treecvpartition object

Partition to use in a cross-validated tree, specified as the comma-separated pair consisting of 'CVPartition' and an object created using cvpartition.

If you use 'CVPartition', you cannot use any of the 'KFold', 'Holdout', or 'Leaveout' name-value pair arguments.

### 'Holdout' — Fraction of data for holdout validation0 (default) | scalar value in the range [0,1]

Fraction of data used for holdout validation, specified as the comma-separated pair consisting of 'Holdout' and a scalar value in the range [0,1]. Holdout validation tests the specified fraction of the data, and uses the rest of the data for training.

If you use 'Holdout', you cannot use any of the 'CVPartition', 'KFold', or 'Leaveout' name-value pair arguments.

Example: 'Holdout',0.1

Data Types: single | double

### 'KFold' — Number of folds10 (default) | positive integer value

Number of folds to use in a cross-validated tree, specified as the comma-separated pair consisting of 'KFold' and a positive integer value.

If you use 'KFold', you cannot use any of the 'CVPartition', 'Holdout', or 'Leaveout' name-value pair arguments.

Example: 'KFold',8

Data Types: single | double

### 'Leaveout' — Leave-one-out cross-validation flag'off' (default) | 'on'

Leave-one-out cross-validation flag, specified as the comma-separated pair consisting of 'Leaveout' and 'on' or 'off'. Specify 'on' to use leave-one-out cross-validation.

If you use 'Leaveout', you cannot use any of the 'CVPartition', 'Holdout', or 'KFold' name-value pair arguments.

Example: 'Leaveout','on'

### 'MaxNumCategories' — Maximum category levels10 (default) | nonnegative scalar value

Maximum category levels, specified as the comma-separated pair consisting of 'MaxNumCategories' and a nonnegative scalar value. fitctree splits a categorical predictor using the exact search algorithm if the predictor has at most MaxNumCategories levels in the split node. Otherwise, fitctree finds the best categorical split using one of the inexact algorithms.

Passing a small value can lead to loss of accuracy and passing a large value can increase computation time and memory overload.

Example: 'MaxNumCategories',8

### 'MergeLeaves' — Leaf merge flag'on' (default) | 'off'

Leaf merge flag, specified as the comma-separated pair consisting of 'MergeLeaves' and 'on' or 'off'.

If MergeLeaves is 'on', then ClassificationTree:

• Merges leaves that originate from the same parent node, and that yields a sum of risk values greater or equal to the risk associated with the parent node

• Estimates the optimal sequence of pruned subtrees, but does not prune the classification tree

Otherwise, ClassificationTree does not merge leaves.

Example: 'MergeLeaves','off'

### 'MinLeafSize' — Minimum number of leaf node observations1 (default) | positive integer value

Minimum number of leaf node observations, specified as the comma-separated pair consisting of 'MinLeafSize' and a positive integer value. Each leaf has at least MinLeafSize observations per tree leaf. If you supply both MinParentSize and MinLeafSize, fitctree uses the setting that gives larger leaves: MinParentSize = max(MinParentSize,2*MinLeafSize).

Example: 'MinLeafSize',3

Data Types: single | double

### 'MinParentSize' — Minimum number of branch node observations10 (default) | positive integer value

Minimum number of branch node observations, specified as the comma-separated pair consisting of 'MinParentSize' and a positive integer value. Each branch node in the tree has at least MinParentSize observations. If you supply both MinParentSize and MinLeafSize, fitctree uses the setting that gives larger leaves: MinParentSize = max(MinParentSize,2*MinLeafSize).

Example: 'MinParentSize',8

Data Types: single | double

### 'NumVariablesToSample' — Number of predictors for split'all' | positive integer value

Number of predictors to select at random for each split, specified as the comma-separated pair consisting of 'NumVariablesToSample' and a positive integer value. You can also specify 'all' to use all available predictors.

Example: 'NumVariablesToSample',3

Data Types: single | double

### 'PredictorNames' — Predictor variable names{'x1','x2',...} (default) | cell array of strings

Predictor variable names, specified as the comma-separated pair consisting of 'PredictorNames' and a cell array of strings containing the names for the predictor variables, in the order in which they appear in X.

### 'Prior' — Prior probabilities'empirical' (default) | 'uniform' | vector of scalar values | structure

Prior probabilities for each class, specified as the comma-separated pair consisting of 'Prior' and one of the following.

• A string:

• 'empirical' determines class probabilities from class frequencies in Y. If you pass observation weights, fitctree uses the weights to compute the class probabilities.

• 'uniform' sets all class probabilities equal.

• A vector (one scalar value for each class)

• A structure S with two fields:

• S.ClassNames containing the class names as a variable of the same type as Y

• S.ClassProbs containing a vector of corresponding probabilities

If you set values for both weights and prior, the weights are renormalized to add up to the value of the prior probability in the respective class.

Example: 'Prior','uniform'

### 'Prune' — Flag to estimate optimal sequence of pruned subtrees'on' (default) | 'off'

Flag to estimate the optimal sequence of pruned subtrees, specified as the comma-separated pair consisting of 'Prune' and 'on' or 'off'.

If Prune is 'on', then ClassificationTree grows the classification tree and estimates the optimal sequence of pruned subtrees, but does not prune the classification tree. Otherwise, ClassificationTree grows the classification tree without estimating the optimal sequence of pruned subtrees.

To prune a trained classification tree, pass the classification tree to prune.

Example: 'Prune','off'

### 'PruneCriterion' — Pruning criterion'error' (default) | 'impurity'

Pruning criterion, specified as the comma-separated pair consisting of 'PruneCriterion' and 'error' or 'impurity'.

Example: 'PruneCriterion','impurity'

### 'ResponseName' — Response variable name'Y' (default) | string

Response variable name, specified as the comma-separated pair consisting of 'ResponseName' and a string representing the name of the response variable Y.

Example: 'ResponseName','Response'

### 'ScoreTransform' — Score transform function'none' | 'symmetric' | 'invlogit' | 'ismax' | function handle | ...

Score transform function, specified as the comma-separated pair consisting of 'ScoreTransform' and a function handle for transforming scores. Your function should accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).

Alternatively, you can specify one of the following strings representing a built-in transformation function.

StringFormula
'doublelogit'1/(1 + e–2x)
'invlogit'log(x / (1–x))
'ismax'Set the score for the class with the largest score to 1, and scores for all other classes to 0.
'logit'1/(1 + ex)
'none'x (no transformation)
'sign'–1 for x < 0
0 for x = 0
1 for x > 0
'symmetric'2x – 1
'symmetriclogit'2/(1 + ex) – 1
'symmetricismax'Set the score for the class with the largest score to 1, and scores for all other classes to -1.

Example: 'ScoreTransform','logit'

### 'SplitCriterion' — Split criterion'gdi' (default) | 'twoing' | 'deviance'

Split criterion, specified as the comma-separated pair consisting of 'SplitCriterion' and 'gdi' (Gini's diversity index), 'twoing' for the twoing rule, or 'deviance' for maximum deviance reduction (also known as cross entropy).

Example: 'SplitCriterion','deviance'

### 'Surrogate' — Surrogate decision splits flag'off' | 'on' | 'all' | positive integer value

Surrogate decision splits flag, specified as the comma-separated pair consisting of 'Surrogate' and 'on', 'off', 'all', or a positive integer value.

• When set to 'on', fitctree finds at most 10 surrogate splits at each branch node.

• When set to 'all', fitctree finds all surrogate splits at each branch node. The 'all' setting can use considerable time and memory.

• When set to a positive integer value, fitctree finds at most the specified number of surrogate splits at each branch node.

Use surrogate splits to improve the accuracy of predictions for data with missing values. The setting also lets you compute measures of predictive association between predictors.

Example: 'Surrogate','on'

### 'Weights' — Observation weightsones(size(x,1),1) (default) | vector of scalar values

Vector of observation weights, specified as the comma-separated pair consisting of 'Weights' and a vector of scalar values. The length of Weights equals the number of rows in X. fitctree normalizes the weights in each class to add up to the value of the prior probability of the class.

Data Types: single | double

## Properties

 CategoricalPredictors List of categorical predictors, a numeric vector with indices from 1 to p, where p is the number of columns of X. CategoricalSplits An n-by-2 cell array, where n is the number of categorical splits in tree. Each row in CategoricalSplits gives left and right values for a categorical split. For each branch node with categorical split j based on a categorical predictor variable z, the left child is chosen if z is in CategoricalSplits(j,1) and the right child is chosen if z is in CategoricalSplits(j,2). The splits are in the same order as nodes of the tree. Find the nodes for these splits by selecting 'categorical' cuts from top to bottom in the CutType property. Children An n-by-2 array containing the numbers of the child nodes for each node in tree, where n is the number of nodes. Leaf nodes have child node 0. ClassCount An n-by-k array of class counts for the nodes in tree, where n is the number of nodes and k is the number of classes. For any node number i, the class counts ClassCount(i,:) are counts of observations (from the data used in fitting the tree) from each class satisfying the conditions for node i. ClassNames List of the elements in Y with duplicates removed. ClassNames can be a categorical array, cell array of strings, character array, logical vector, or a numeric vector. ClassNames has the same data type as the data in the argument Y. ClassProbability An n-by-k array of class probabilities for the nodes in tree, where n is the number of nodes and k is the number of classes. For any node number i, the class probabilities ClassProbability(i,:) are the estimated probabilities for each class for a point satisfying the conditions for node i. Cost Square matrix, where Cost(i,j) is the cost of classifying a point into class j if its true class is i. CutCategories An n-by-2 cell array of the categories used at branches in tree, where n is the number of nodes. For each branch node i based on a categorical predictor variable X, the left child is chosen if X is among the categories listed in CutCategories{i,1}, and the right child is chosen if X is among those listed in CutCategories{i,2}. Both columns of CutCategories are empty for branch nodes based on continuous predictors and for leaf nodes. CutPoint contains the cut points for 'continuous' cuts, and CutCategories contains the set of categories. CutPoint An n-element vector of the values used as cut points in tree, where n is the number of nodes. For each branch node i based on a continuous predictor variable X, the left child is chosen if X=CutPoint(i). CutPoint is NaN for branch nodes based on categorical predictors and for leaf nodes. CutPoint contains the cut points for 'continuous' cuts, and CutCategories contains the set of categories. CutType An n-element cell array indicating the type of cut at each node in tree, where n is the number of nodes. For each node i, CutType{i} is: 'continuous' — If the cut is defined in the form X < v for a variable X and cut point v.'categorical' — If the cut is defined by whether a variable X takes a value in a set of categories.'' — If i is a leaf node. CutPoint contains the cut points for 'continuous' cuts, and CutCategories contains the set of categories. CutPredictor An n-element cell array of the names of the variables used for branching in each node in tree, where n is the number of nodes. These variables are sometimes known as cut variables. For leaf nodes, CutPredictor contains an empty string. CutPoint contains the cut points for 'continuous' cuts, and CutCategories contains the set of categories. IsBranchNode An n-element logical vector that is true for each branch node and false for each leaf node of tree. ModelParameters Parameters used in training tree. NumObservations Number of observations in the training data, a numeric scalar. NumObservations can be less than the number of rows of input data X when there are missing values in X or response Y. NodeClass An n-element cell array with the names of the most probable classes in each node of tree, where n is the number of nodes in the tree. Every element of this array is a string equal to one of the class names in ClassNames. NodeError An n-element vector of the errors of the nodes in tree, where n is the number of nodes. NodeError(i) is the misclassification probability for node i. NodeProbability An n-element vector of the probabilities of the nodes in tree, where n is the number of nodes. The probability of a node is computed as the proportion of observations from the original data that satisfy the conditions for the node. This proportion is adjusted for any prior probabilities assigned to each class. NodeRisk An n-element vector of the risk of the nodes in the tree, where n is the number of nodes. The risk for each node is the measure of impurity (Gini index or deviance) for this node weighted by the node probability. If the tree is grown by twoing, the risk for each node is zero. NodeSize An n-element vector of the sizes of the nodes in tree, where n is the number of nodes. The size of a node is defined as the number of observations from the data used to create the tree that satisfy the conditions for the node. NumNodes The number of nodes in tree. Parent An n-element vector containing the number of the parent node for each node in tree, where n is the number of nodes. The parent of the root node is 0. PredictorNames Cell array of strings containing the predictor names, in the order which they appear in X. Prior Numeric vector of prior probabilities for each class. The order of the elements of Prior corresponds to the elements of ClassNames. PruneAlpha Numeric vector with one element per pruning level. If the pruning level ranges from 0 to M, then PruneAlpha has M + 1 elements sorted in ascending order. PruneAlpha(1) is for pruning level 0 (no pruning), PruneAlpha(2) is for pruning level 1, and so on. PruneList An n-element numeric vector with the pruning levels in each node of tree, where n is the number of nodes. The pruning levels range from 0 (no pruning) to M, where M is the distance between the deepest leaf and the root node. ResponseNames String describing the response variable Y. ScoreTransform Function handle for transforming predicted classification scores, or string representing a built-in transformation function. none means no transformation, or @(x)x. To change the score transformation function to, e.g., function, use dot notation.For available functions (see fitctree), enter`Mdl.ScoreTransform = 'function';`You can set a function handle for an available function, or a function you define yourself by entering`tree.ScoreTransform = @function;` SurrogateCutCategories An n-element cell array of the categories used for surrogate splits in tree, where n is the number of nodes in tree. For each node k, SurrogateCutCategories{k} is a cell array. The length of SurrogateCutCategories{k} is equal to the number of surrogate predictors found at this node. Every element of SurrogateCutCategories{k} is either an empty string for a continuous surrogate predictor, or is a two-element cell array with categories for a categorical surrogate predictor. The first element of this two-element cell array lists categories assigned to the left child by this surrogate split, and the second element of this two-element cell array lists categories assigned to the right child by this surrogate split. The order of the surrogate split variables at each node is matched to the order of variables in SurrogateCutPredictor. The optimal-split variable at this node does not appear. For nonbranch (leaf) nodes, SurrogateCutCategories contains an empty cell. SurrogateCutFlip An n-element cell array of the numeric cut assignments used for surrogate splits in tree, where n is the number of nodes in tree. For each node k, SurrogateCutFlip{k} is a numeric vector. The length of SurrogateCutFlip{k} is equal to the number of surrogate predictors found at this node. Every element of SurrogateCutFlip{k} is either zero for a categorical surrogate predictor, or a numeric cut assignment for a continuous surrogate predictor. The numeric cut assignment can be either –1 or +1. For every surrogate split with a numeric cut C based on a continuous predictor variable Z, the left child is chosen if Z

## Methods

 compact Compact tree crossval Cross-validated decision tree cvloss Classification error by cross validation prune Produce sequence of subtrees by pruning resubEdge Classification edge by resubstitution resubLoss Classification error by resubstitution resubMargin Classification margins by resubstitution resubPredict Predict resubstitution response of tree

### Inherited Methods

 edge Classification edge loss Classification error margin Classification margins predict Predict classification predictorImportance Estimates of predictor importance surrogateAssociation Mean predictive measure of association for surrogate splits in decision tree view View tree

## Definitions

### Impurity and Node Error

ClassificationTree splits nodes based on either impurity or node error.

Impurity means one of several things, depending on your choice of the SplitCriterion name-value pair argument:

• Gini's Diversity Index (gdi) — The Gini index of a node is

$1-\sum _{i}{p}^{2}\left(i\right),$

where the sum is over the classes i at the node, and p(i) is the observed fraction of classes with class i that reach the node. A node with just one class (a pure node) has Gini index 0; otherwise the Gini index is positive. So the Gini index is a measure of node impurity.

• Deviance ('deviance') — With p(i) defined the same as for the Gini index, the deviance of a node is

$-\sum _{i}p\left(i\right)\mathrm{log}p\left(i\right).$

A pure node has deviance 0; otherwise, the deviance is positive.

• Twoing rule ('twoing') — Twoing is not a purity measure of a node, but is a different measure for deciding how to split a node. Let L(i) denote the fraction of members of class i in the left child node after a split, and R(i) denote the fraction of members of class i in the right child node after a split. Choose the split criterion to maximize

$P\left(L\right)P\left(R\right){\left(\sum _{i}|L\left(i\right)-R\left(i\right)|\right)}^{2},$

where P(L) and P(R) are the fractions of observations that split to the left and right respectively. If the expression is large, the split made each child node purer. Similarly, if the expression is small, the split made each child node similar to each other, and hence similar to the parent node, and so the split did not increase node purity.

• Node error — The node error is the fraction of misclassified classes at a node. If j is the class with the largest number of training samples at a node, the node error is

1 – p(j).

## Copy Semantics

Value. To learn how value classes affect copy operations, see Copying Objects in the MATLAB® documentation.

## Examples

expand all

### Construct a Classification Tree

Construct a classification tree for the data in ionosphere.mat.

```load ionosphere
tc = fitctree(X,Y)```
```tc =

ClassificationTree
PredictorNames: {1x34 cell}
ResponseName: 'Y'
ClassNames: {'b'  'g'}
ScoreTransform: 'none'
CategoricalPredictors: []
NumObservations: 351

Properties, Methods```

## References

[1] Breiman, L., J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Boca Raton, FL: CRC Press, 1984.