kfoldfun
Cross-validate function using cross-validated ECOC model
Description
Examples
Estimate Classification Error Using Custom Loss Function
Train a multiclass ECOC classifier, and then cross-validate the model using a custom k-fold loss function.
Load Fisher’s iris data set. Specify the predictor data X
, the response data Y
, and the order of the classes in Y
.
load fisheriris X = meas; Y = categorical(species); classOrder = unique(Y); % Class order rng(1); % For reproducibility
Train and cross-validate an ECOC model using support vector machine (SVM) binary classifiers. Standardize the predictors using an SVM template, and specify the class order.
t = templateSVM('Standardize',1); CVMdl = fitcecoc(X,Y,'CrossVal','on','Learners',t,... 'ClassNames',classOrder);
CVMdl
is a ClassificationPartitionedECOC
model. By default, the software implements 10-fold cross-validation.
Compute the classification error (proportion of misclassified observations) for the validation-fold observations.
L = kfoldLoss(CVMdl)
L = 0.0400
Examine the result when the cost of misclassifying a flower as versicolor
is 10
and the cost of any other error is 1
. Write a function named noversicolor
that assigns a cost of 1
for general misclassification and a cost of 10
for misclassifying a flower as versicolor
.
If you use the live script file for this example, the noversicolor
function is already included at the end of the file. Otherwise, you need to create this function at the end of your .m file or add it as a file on the MATLAB® path.
Compute the mean misclassification error with the noversicolor
cost.
foldLoss = kfoldfun(CVMdl,@noversicolor); mean(foldLoss)
ans = single
0.0667
This code creates the function noversicolor
.
function averageCost = noversicolor(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest) % noversicolor: Example custom cross-validation function that assigns a cost of % 10 for misclassifying versicolor irises and a cost of 1 for misclassifying % the other irises. This example function requires the fisheriris data % set. Ypredict = predict(CMP,Xtest); misclassified = not(strcmp(Ypredict,Ytest)); % Different result classifiedAsVersicolor = strcmp(Ypredict,'versicolor'); % Index of bad decisions cost = sum(misclassified) + ... 9*sum(misclassified & classifiedAsVersicolor); % Total differences averageCost = single(cost/numel(Ytest)); % Average error end
Input Arguments
CVMdl
— Cross-validated ECOC model
ClassificationPartitionedECOC
model
Cross-validated ECOC model, specified as a ClassificationPartitionedECOC
model.
fun
— Cross-validated function
function handle
Cross-validated function, specified as a function handle.
fun
has this syntax:
testvals = fun(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)
CMP
is a compact model stored in one element of theCVMdl.Trained
property.Xtrain
is the training matrix of predictor values.Ytrain
is the training array of response values.Wtrain
is the set of training weights for observations.Xtest
andYtest
are the validation data, with associated weightsWtest
.The returned value
testvals
must have the same size across all folds.
Data Types: function_handle
Output Arguments
vals
— Cross-validation results
numeric matrix
Cross-validation results, returned as a numeric matrix.
vals
corresponds to the arrays of the
testvals
output, concatenated vertically over all the
folds. For example, if testvals
from every fold is a
numeric vector of length n, kfoldfun
returns a KFold
-by-n numeric matrix
with one row per fold.
Extended Capabilities
GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.
Usage notes and limitations:
kfoldfun
supports models trained withgpuArray
inputs iffun
also supportsgpuArray
inputs.
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
Version History
Introduced in R2014b
See Also
kfoldPredict
| kfoldEdge
| kfoldMargin
| kfoldLoss
| crossval
| ClassificationPartitionedECOC
| ClassificationECOC
| fitcecoc
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)