Trying to create ROC curves for n-class classification problem

8 views (last 30 days)
I have trained two classification models, a Naive Bayes model, and a Decision Tree, now I am looking to create ROC curves for each model but I get a "Positive class is not found in the input data." Error.
I have 11 factors and 6 classes, here is my code for the Naive Bayes ROC curves
%ROC Curves for our final models
% hypernormoutnb >Final NB model
% optctree >final DT model
% NBxtestx = partitioned X values for testing
% NBxtesty = Partitioned Y values for testing
[predictions,score1,cost1] = predict(hypernormoutNB,NBxtestx)
%Then you pass in scores from predictions on test set and generate metrics for each class. Each line will identify each class as the positive label.
[fpr0,tpr0,T0,AUC0,OPTROCPT0] = perfcurve(NBxtesty,score1(:,1:1),0);
[fpr1,tpr1,T1,AUC1,OPTROCPT1] = perfcurve(NBxtesty,score1(:,2:2),1);
[fpr2,tpr2,T2,AUC2,OPTROCPT2] = perfcurve(NBxtesty,score1(:,3:3),2);
[fpr3,tpr3,T3,AUC3,OPTROCPT3] = perfcurve(NBxtesty,score1(:,4:4),3);
[fpr4,tpr4,T4,AUC4,OPTROCPT4] = perfcurve(NBxtesty,score1(:,5:5),4);
[fpr5,tpr5,T5,AUC5,OPTROCPT5] = perfcurve(NBxtesty,score1(:,6:6),5);
title('Naive Bayes ROC Curves')
xlabel('False Positive')
ylabel('True Positive')
hold on
hold off
I get the following error message:
Error using perfcurve>membership (line 705)
Positive class is not found in the input data.
Error in perfcurve (line 449)
[W,subYnames] = membership(cls(sorted),weights(sorted),...
Error in Final_Project (line 642)
[fpr0,tpr0,T0,AUC0,OPTROCPT0] = perfcurve(NBxtesty,score1(:,1:1),0);
Can anyone offer some insight into why I haven't defined a positive class?

Accepted Answer

Pranav Verma
Pranav Verma on 11 Dec 2020
Hi Jeremy,
You have defined the positive class as scalar value in perfcurve (the last parameter in perfcurve). Please check if the parameters you have defined are the same as expected by the perfcurve function.
Documentation for perfcurve.
labels - numeric vector | logical vector | character matrix | string array | cell array of character vectors | categorical array
scores - vector of floating points
posclass - numeric scalar | logical scalar | character vector | string scalar | cell containing a character vector | categorical scalar
Also, please go through the the below question on similar lines:
  1 Comment
Jeremy Brecevic
Jeremy Brecevic on 11 Dec 2020
Ah so my positive value should be my class name? I have 6 classes which are all integers (3,4,5,6,7,8)
so it should be:
[fpr0,tpr0,T0,AUC0,OPTROCPT0] = perfcurve(NBxtesty,score1(:,1:1),3);
[fpr1,tpr1,T1,AUC1,OPTROCPT1] = perfcurve(NBxtesty,score1(:,2:2),4);
[fpr2,tpr2,T2,AUC2,OPTROCPT2] = perfcurve(NBxtesty,score1(:,3:3),5);
[fpr3,tpr3,T3,AUC3,OPTROCPT3] = perfcurve(NBxtesty,score1(:,4:4),6);
[fpr4,tpr4,T4,AUC4,OPTROCPT4] = perfcurve(NBxtesty,score1(:,5:5),7);
[fpr5,tpr5,T5,AUC5,OPTROCPT5] = perfcurve(NBxtesty,score1(:,6:6),8);
Or am I misunderstanding?

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!