X is an n x k matrix of predictors. Y is n x 1 vector of responses; this is binary. I implement a probit, generate predictions from the probit, and then assign each response a 1 or zero depending on whether it is greater than 1/2.
I am using cross validation to get an estimate of the model's classification error, not to choose the Xs. In particular, I run
fcn = @(Xtr,Ytr,Xte) (predict( fitglm(Xtr,Ytr,'Distribution','binomial','Link','probit'), Xte) > 0.5);
mcr = crossval('mcr',X,Y,'Predfun',fcn,'kfold',5);
When I run this, Matlab always gives me mcr=1. Using subsets of the data, I've estimated the model, made predictions on the left out subset, and then compared those to the true responses (for that left out subset). The model consistently gets about 65-70% of the predictions correct.
Any help is much appreciated.