kfoldfun

Cross-validate function for regression

Syntax

vals = kfoldfun(CVMdl,fun)

Description

vals = kfoldfun(CVMdl,fun) cross-validates the function fun by applying fun to the data stored in the cross-validated model CVMdl. You must pass fun as a function handle.

Examples

Estimate Regression Loss Using Custom Loss Function

Train a regression tree model, and then cross-validate it using a custom k-fold loss function.

Load the imports-85 data set. Train a regression tree using a subset of the data.

load imports-85
Mdl = fitrtree(X(:,[4 5]),X(:,16),...
    'PredictorNames',{'Length','Width'},...
    'ResponseName','Price');

Cross-validate the regression tree, and obtain the mean squared error.

CVMdl = crossval(Mdl);
L = kfoldLoss(CVMdl)

L = 1.9167e+07

Examine the error when you use a simple averaging of training responses instead of predictions in the calculation.

f = @(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)...
    mean((Ytest-mean(Ytrain)).^2)

f = function_handle with value:
    @(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)mean((Ytest-mean(Ytrain)).^2)

mean(kfoldfun(CVMdl,f))

ans = 6.3586e+07

Input Arguments

CVMdl — Cross-validated model

RegressionPartitionedModel object | RegressionPartitionedEnsemble object | RegressionPartitionedGAM object | RegressionPartitionedSVM object

Cross-validated model, specified as a RegressionPartitionedModel object, RegressionPartitionedEnsemble object, RegressionPartitionedGAM object, or RegressionPartitionedSVM object.

fun — Cross-validated function

function handle

Cross-validated function, specified as a function handle. fun has the syntax:

testvals = fun(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)

• CMP is a compact model stored in one element of the CVMdl.Trained property.

• Xtrain is the training matrix of predictor values.

• Ytrain is the training array of response values.

• Wtrain are the training weights for observations.

• Xtest and Ytest are the test data, with associated weights Wtest.

• The returned value testvals must have the same size across all folds.

Data Types: function_handle

Output Arguments

vals — Cross-validation results

numeric matrix

Cross-validation results, returned as a numeric matrix. vals contains the arrays of testvals output, concatenated vertically over all folds. For example, if testvals from every fold is a numeric vector of length N, kfoldfun returns a KFold-by-N numeric matrix with one row per fold.

Data Types: double