Cross validation settings for Classification Discriminant models

3 views (last 30 days)
I foumd the 'KFold' in classification models can not be controlled when 'HyperparameterOptimizationOptions' is enabled. Here is a snippet of my codes:
discrMdl = fitcdiscr(X,y,'discrimType','linear',...
'OptimizeHyperparameters', VariableDescriptions,'HyperparameterOptimizationOptions',...
struct('ShowPlots',0,'Verbose',0,'Kfold',5));
After training, typing
discrMdl.crossval.KFold
gives a number of 10, implying 10-fold crosss validations had been perforemd, although 5-fold was defined in the function. I also tried other numbers, but it is alwasy 10-fold. It does not make sense according to the documentation athttps://www.mathworks.com/help/stats/fitcdiscr.html#bt6d86x-1_sep_shared-HyperparameterOptimizationOptions, unless the hyparameter optimization is only done in a 10-fold cross-validation fashion?
If the 'KFold' is put outside the strucured options, an error message will occur:
When optimizing parameters, validation arguments may only appear in the
'HyperparameterOptimizationOptions' argument.
Also, the resulting model is a ClassificationDiscriminant model rather than ClassificationPartitionedModel. Should the model be the latter given the cross validation was enabled in the hyperparameter optimization, according to the documentation at https://www.mathworks.com/help/stats/fitcdiscr.html#bt6d86x-1-Mdl.
Did I interpret something wrong?

Answers (1)

Aditya Patil
Aditya Patil on 14 Jul 2020
When you use the crossval function, it creates a partioned model with KFold set to default value of 10. Currently, it's not possible to read the value of Kfold for ClassificationDiscriminant class. I have brought this issue to the concerned people and it might be considered in any future release.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!