Evaluate a Video Classifier

This example shows how to evaluate a pretrained SlowFast Video Classifier using a collection of videos. To learn more about how to train a video classifier network for your dataset, see Gesture Recognition using Videos and Deep Learning.

Note: This example requires the Computer Vision Toolbox™ Model for SlowFast Video Classification, which can be installed from the Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

Download Pretrained Video Classifier

Download the pretrained SlowFast video classifier. The size of the downloaded zip file is around 245 MB.

downloadFolder = fullfile(tempdir,"gesture");
zipFile = "slowFastPretrained_fourClasses.zip";

if ~isfolder(downloadFolder)
    mkdir(downloadFolder);
end

if ~isfile(fullfile(downloadFolder,zipFile))
    disp("Downloading the pretrained network...");    
    downloadURL = "https://ssd.mathworks.com/supportfiles/vision/data/" + zipFile;    
    zipFile = fullfile(downloadFolder,zipFile);
    websave(zipFile,downloadURL);
    unzip(zipFile,downloadFolder);
    disp('Downloaded.');
end

Load the pretrained SlowFast video classifier.

pretrainedDataFile = fullfile(downloadFolder,"slowFastPretrained_fourClasses.mat");
pretrained = load(pretrainedDataFile);
slowFastClassifier = pretrained.data.slowFast;

Display the class label names of the pretrained video classifier.

classes = slowFastClassifier.Classes

classes = 4×1 categorical
     clapping 
     noAction 
     somethingElse 
     wavingHello 

Load Videos For Evaluation

This examples uses videos stored in folders with names that correspond to their labels. Extract video sequences of specific scene labels and store them in a folder named "videoScenes". To learn how to extract video sequences from a collection of videos and the corresponding ground truth scene label information, see Extract Training Data for Video Classification.

Use the supporting function downloadGroundTruthData, defined at the end of this example, to download the video scenes and the corresponding ground truth scene label information.

groundTruthFolder = fullfile(downloadFolder,"groundTruthFolder");
trainingFolder = fullfile(downloadFolder,"videoScenes");
groundTruthAlreadyExtracted = downloadGroundTruthData(groundTruthFolder,trainingFolder,classes);

Use the supporting function extractVideoScenes, defined at the end of this example, to extract the video scenes from ground truth scene label information. Extract the video scene data only when the ground truth data was not extracted before.

if ~groundTruthAlreadyExtracted    
    extractVideoScenes(groundTruthFolder,trainingFolder);
end

Classify Video Files

Use folders2labels to create label information from folders and to list all the files in the "videoScenes" folder.

[trueLabels, filenames] = folders2labels(trainingFolder,...
    "FileExtensions",".avi", ...
    "IncludeSubfolders",true);

Pre-allocate categorical label values with empty label names.

numFiles = numel(filenames);
emptyLabelNames = repmat("",numFiles,1);
predictedLabels = categorical(emptyLabelNames,string(classes));

Choose 5 randomly selected video sequences to classify the video, to uniformly cover the entirety of a file to find the action class that is predominant in the video.

numSequences = 5;

In order to classify and assign a label to each of the video files in the collection, use classifyVideoFile function.

for ii = 1:numFiles
    filename = filenames(ii);    
    predictedLabels(ii) = classifyVideoFile(slowFastClassifier, filename,...
        "NumSequences", numSequences);
end

Compute Video Accuracy

Compute the average video accuracy for the video classifier.

videoAccuracy = sum(trueLabels == predictedLabels)/numel(trueLabels)

videoAccuracy = 1

Display the confusion matrix.

figure
chart = confusionchart(trueLabels,predictedLabels);

Supporting Functions

downloadGroundTruthData

The downloadGroundTruthData function downloads ground truth video data and its corresponding collection of scene labels.

function groundTruthAlreadyExtracted = downloadGroundTruthData(groundTruthFolder,trainingFolder,classes)

% If the video scenes are already extracted, no need to download
% the dataset and extract video scenes.
groundTruthAlreadyExtracted = false;
if isfolder(trainingFolder)
    classFolders = fullfile(trainingFolder, string(classes));
    groundTruthAlreadyExtracted = true;
    for ii = 1:numel(classFolders)
        if ~isfolder(classFolders(ii))
            groundTruthAlreadyExtracted = false;
            break;
        end
    end
    if groundTruthAlreadyExtracted
        return;
    end
end
if ~isfolder(groundTruthFolder)
    mkdir(groundTruthFolder);
end
downloadURL = "https://ssd.mathworks.com/supportfiles/vision/data/videoClipsAndSceneLabels.zip";
filename = fullfile(groundTruthFolder,"videoClipsAndSceneLabels.zip");
if ~isfile(filename)
    disp("Downloading the video clips and the corresponding scene labels to " + groundTruthFolder);
    websave(filename,downloadURL);    
end

% Unzip the contents to the download folder.
unzip(filename,groundTruthFolder);
end

extractVideoScenes

The extractVideoScenes function extracts training video data from a collection of videos and its corresponding collection of scene labels, by using the functions sceneTimeRanges and writeVideoScenes.

function extractVideoScenes(groundTruthFolder,trainingFolder)
labelDataFiles = dir(fullfile(groundTruthFolder,"*_labelData.mat"));
labelDataFiles = fullfile(groundTruthFolder,{labelDataFiles.name}');
numGtruth = numel(labelDataFiles);

% Load the label data information and create ground truth objects.
gTruth = groundTruth.empty(numGtruth,0);
for ii = 1:numGtruth
    ld = load(labelDataFiles{ii});
    videoFilename = fullfile(groundTruthFolder,ld.videoFilename);
    gds = groundTruthDataSource(videoFilename);
    gTruth(ii) = groundTruth(gds,ld.labelDefs,ld.labelData);
end

% Gather all the scene time ranges and the corresponding scene labels 
% using the sceneTimeRanges function.
[timeRanges, sceneLabels] = sceneTimeRanges(gTruth);

% Specify the subfolder names for each duration as the scene label names. 
foldernames = sceneLabels;

% Delete the folder if it already exists.
if isfolder(trainingFolder)
    rmdir(trainingFolder, 's');
end
% Video files are written to the folders specified by the folderNames input.
writeVideoScenes(gTruth,timeRanges,trainingFolder,foldernames);
end