# loss

Classification error for naive Bayes classifier

## Syntax

``L = loss(Mdl,tbl,ResponseVarName)``
``L = loss(Mdl,tbl,Y)``
``L = loss(Mdl,X,Y)``
``L = loss(___,Name,Value)``

## Description

````L = loss(Mdl,tbl,ResponseVarName)` returns the minimum classification loss (see Classification Loss), a scalar representing how well the trained naive Bayes classifier `Mdl` classifies the predictor data in table `tbl`) as compared to the true class labels in `tbl.ResponseVarName`.`loss` normalizes the class probabilities in `tbl.ResponseVarName` to the prior class probabilities `fitcnb` used for training, stored in the `Prior` property of `Mdl`.```
````L = loss(Mdl,tbl,Y)` returns the minimum classification loss (`L`), a scalar representing how well the trained naive Bayes classifier `Mdl` classifies the predictor data in table `tbl`) as compared to the true class labels in `Y`.`loss` normalizes the class probabilities in `Y` to the prior class probabilities `fitcnb` used for training, stored in the `Prior` property of `Mdl`.```

example

````L = loss(Mdl,X,Y)` returns the minimum classification loss (`L`), a scalar representing how well the trained naive Bayes classifier `Mdl` classifies the predictor data (`X`) as compared to the true class labels (`Y`).`loss` normalizes the class probabilities in `Y` to the prior class probabilities `fitcnb` used for training, stored in the `Prior` property of `Mdl`.```

example

````L = loss(___,Name,Value)` returns the classification loss with additional options specified by one or more `Name,Value` pair arguments, using any of the previous syntaxes.```

## Input Arguments

expand all

Naive Bayes classifier, specified as a `ClassificationNaiveBayes` model or `CompactClassificationNaiveBayes` model returned by `fitcnb` or `compact`, respectively.

Sample data, specified as a table. Each row of `tbl` corresponds to one observation, and each column corresponds to one predictor variable. Optionally, `tbl` can contain additional columns for the response variable and observation weights. `tbl` must contain all the predictors used to train `Mdl`. Multi-column variables and cell arrays other than cell arrays of character vectors are not allowed.

If you trained `Mdl` using sample data contained in a `table`, then the input data for this method must also be in a table.

Data Types: `table`

Response variable name, specified as the name of a variable in `tbl`.

You must specify `ResponseVarName` as a character vector or string scalar. For example, if the response variable `y` is stored as `tbl.y`, then specify it as `'y'`. Otherwise, the software treats all columns of `tbl`, including `y`, as predictors when training the model.

The response variable must be a categorical, character, or string array, logical or numeric vector, or cell array of character vectors. If the response variable is a character array, then each element must correspond to one row of the array.

Data Types: `char` | `string`

Predictor data, specified as a numeric matrix.

Each row of `X` corresponds to one observation (also known as an instance or example), and each column corresponds to one variable (also known as a feature). The variables making up the columns of `X` should be the same as the variables that trained `Mdl`.

The length of `Y` and the number of rows of `X` must be equal.

Data Types: `double` | `single`

Class labels, specified as a categorical, character, or string array, logical or numeric vector, or cell array of character vectors. `Y` must be the same as the data type of `Mdl.ClassNames`. (The software treats string arrays as cell arrays of character vectors.)

The length of `Y` and the number of rows of `tbl` or `X` must be equal.

Data Types: `categorical` | `char` | `string` | `logical` | `single` | `double` | `cell`

### Name-Value Pair Arguments

Specify optional comma-separated pairs of `Name,Value` arguments. `Name` is the argument name and `Value` is the corresponding value. `Name` must appear inside quotes. You can specify several name and value pair arguments in any order as `Name1,Value1,...,NameN,ValueN`.

Loss function, specified as the comma-separated pair consisting of `'LossFun'` and a built-in, loss-function name or function handle.

• The following table lists the available loss functions. Specify one using its corresponding character vector or string scalar.

ValueDescription
`'binodeviance'`Binomial deviance
`'classiferror'`Classification error
`'exponential'`Exponential
`'hinge'`Hinge
`'logit'`Logistic
`'mincost'`Minimal expected misclassification cost (for classification scores that are posterior probabilities)
`'quadratic'`Quadratic

`'mincost'` is appropriate for classification scores that are posterior probabilities. Naive Bayes models return posterior probabilities as classification scores by default (see `predict`).

• Specify your own function using function handle notation.

Suppose that `n` be the number of observations in `X` and `K` be the number of distinct classes (`numel(Mdl.ClassNames)`, `Mdl` is the input model). Your function must have this signature

``lossvalue = lossfun(C,S,W,Cost)``
where:

• The output argument `lossvalue` is a scalar.

• You choose the function name (`lossfun`).

• `C` is an `n`-by-`K` logical matrix with rows indicating which class the corresponding observation belongs. The column order corresponds to the class order in `Mdl.ClassNames`.

Construct `C` by setting `C(p,q) = 1` if observation `p` is in class `q`, for each row. Set all other elements of row `p` to `0`.

• `S` is an `n`-by-`K` numeric matrix of classification scores. The column order corresponds to the class order in `Mdl.ClassNames`. `S` is a matrix of classification scores, similar to the output of `predict`.

• `W` is an `n`-by-1 numeric vector of observation weights. If you pass `W`, the software normalizes them to sum to `1`.

• `Cost` is a K-by-`K` numeric matrix of misclassification costs. For example, ```Cost = ones(K) - eye(K)``` specifies a cost of `0` for correct classification, and `1` for misclassification.

Specify your function using `'LossFun',@lossfun`.

For more details on loss functions, see Classification Loss.

Data Types: `char` | `string` | `function_handle`

Observation weights, specified as the comma-separated pair consisting of `'Weights'` and a numeric vector or the name of a variable in `tbl`. The software weighs the observations in each row of `X` or `tbl` with the corresponding weight in `Weights`.

If you specify `Weights` as a vector, then the size of `Weights` must be equal to the number of rows of `X` or `tbl`.

If you specify `Weights` as the name of a variable in `tbl`, you must do so as a character vector or string scalar. For example, if the weights are stored as `tbl.w`, then specify `Weights` as `'w'`. Otherwise, the software treats all columns of `tbl`, including `tbl.w`, as predictors.

If you do not specify your own loss function, then the software normalizes `Weights` to add up to `1`.

Data Types: `double` | `char` | `string`

## Output Arguments

expand all

Classification loss, returned as a scalar. `L` is a generalization or resubstitution quality measure. Its interpretation depends on the loss function and weighting scheme, but, in general, better classifiers yield smaller loss values.

## Examples

expand all

```load fisheriris X = meas; % Predictors Y = species; % Response rng(1); % For reproducibility```

Train a naive Bayes classifier. Specify a 15% holdout sample for testing. It is good practice to specify the class order. Assume that each predictor is conditionally normally distributed given its label.

```CVMdl = fitcnb(X,Y,'ClassNames',{'setosa','versicolor','virginica'},... 'Holdout',0.15); CMdl = CVMdl.Trained{1}; % Extract the trained, compact classifier testInds = test(CVMdl.Partition); % Extract the test indices XTest = X(testInds,:); YTest = Y(testInds);```

`CVMdl` is a `ClassificationPartitionedModel` classifier. It contains the property `Trained`, which is a 1-by-1 cell array holding a `CompactClassificationNaiveBayes` classifier that the software trained using the training set.

Determine how well the algorithm generalizes by estimating the test sample minimum cost loss.

`L = loss(CMdl,XTest,YTest)`
```L = 0.0476 ```

The test sample average classification cost is approximately 0.05.

You might improve the classification error by specifying better predictor distributions when you train the classifier.

```load fisheriris X = meas; % Predictors Y = species; % Response rng(1); % For reproducibility```

Train a naive Bayes classifier. Specify a 15% holdout sample for testing. It is good practice to specify the class order. Assume that each predictor is conditionally normally distributed given its label.

```CVMdl = fitcnb(X,Y,'ClassNames',{'setosa','versicolor','virginica'},... 'Holdout',0.15); CMdl = CVMdl.Trained{1}; % Extract the trained, compact classifier testInds = test(CVMdl.Partition); % Extract the test indices XTest = X(testInds,:); YTest = Y(testInds);```

`CVMdl` is a `ClassificationPartitionedModel` classifier. It contains the property `Trained`, which is a 1-by-1 cell array holding a `CompactClassificationNaiveBayes` classifier that the software trained using the training set.

Determine how well the algorithm generalizes by estimating the test sample classification error.

`L = loss(CMdl,XTest,YTest,'LossFun','classiferror')`
```L = 0.0476 ```

The classifier misclassified approximately 5% of the test sample observations.

expand all

## References

[1] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, second edition. Springer, New York, 2008.