How to get the mean of ROC curves using Matlab?

8 views (last 30 days)
I met a problem to plot the mean ROC curve of the 10-fold cross-validation using Matlab.
I run the code
cvPartition = cvpartition(dataSize,'k', 10);
to get 10 fold of training and testing. However, as it randomly choose the number of training and testing. The ROC curve I got from each fold is with different size. In addition, I want to plot the mean ROC of these ten ROC curves I got from the cross-validation. Anyone knows how to do this? I read another post using Python perfectly solve the problem using 1D interpolation. Not sure how to do this in Matlab.
All the FPR and TPR values:
FPR_All =
Columns 1 through 9
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0.2500 0.2000 0 0.1667 0.1667 0.1429 0.3333 0.2000 0
0.5000 0.4000 0.2500 0.3333 0.3333 0.2857 0.6667 0.4000 0.3333
0.7500 0.6000 0.5000 0.5000 0.5000 0.4286 1.0000 0.6000 0.6667
1.0000 0.8000 0.7500 0.6667 0.6667 0.5714 NaN 0.8000 1.0000
NaN 1.0000 1.0000 0.8333 0.8333 0.7143 NaN 1.0000 NaN
NaN NaN NaN 1.0000 1.0000 0.8571 NaN NaN NaN
NaN NaN NaN NaN NaN 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN
Column 10
0
0
0.1429
0.2857
0.4286
0.5714
0.7143
0.8571
1.0000
NaN
TPR_All =
Columns 1 through 9
0 0 0 0 0 0 0 0 0
1.0000 1.0000 0.8333 1.0000 1.0000 1.0000 1.0000 1.0000 0.8571
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 NaN 1.0000 1.0000
NaN 1.0000 1.0000 1.0000 1.0000 1.0000 NaN 1.0000 NaN
NaN NaN NaN 1.0000 1.0000 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN
Column 10
0
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
NaN

Answers (1)

Hiago Brajato
Hiago Brajato on 21 Nov 2019
I don`t know if my question will answer your question, but a way you can an average ROC Curve from 10 ROC curves (10 k-folds) for example is to use the function interp1. Below you can see the code responsible for producing the image as follows.
image.png
The idea is that you have to make the interpolation of each ROC Curve (each fold), and divide for the number of folds (k). It is important to remember that interp1 requires unique points (the points in X axis cannot repeat), so is needed to make a little pertubation in the points to achieve a good mean. The code:
intervals= linspace(0, 1, 100);
%beginning of the loop of folds.....
%ROC Curve and AUC (plotting the ROC Curve for each fold and an average of the folds)
[Xroc, Yroc, ~, AUCroc_per_fold(i)]= perfcurve(label_matrix(idx_test, :), scores(:, 1), 'maligna');
plot(Xroc, Yroc, 'LineWidth', 1.5); legends{i}= sprintf('fold %d (AUC = %.2f)', i, AUCroc_per_fold(i)); hold on;
%for getting an average of the ROC curves from each fold
x_adj= adjust_unique_points(Xroc); %interp1 requires unique points
if i==1 %if is the first fold
mean_curve= (interp1(x_adj, Yroc, intervals))/k;
else
mean_curve= mean_curve+ (interp1(x_adj, Yroc, intervals))/k;
end
%end of the loop of folds....
figure(1); plot(intervals, mean_curve, 'Color', 'Black', 'LineWidth', 3.0);
And below you can see the function for making a little pertubation in X axis. You can choose the alternative that is better for you. Probably you will not achieve a good average using the function unique because a lot of information will be lost.
function x= adjust_unique_points(Xroc)
x= zeros(1, length(Xroc));
aux= 0.0001;
for i=1: length(Xroc)
if i~=1
x(i)= Xroc(i)+aux;
aux= aux+0.0001;
end
end
end

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!