Automate Ground Truth Labeling for Point Cloud Using Pretrained Deep Learning Model

This example shows how to automate labeling in a point cloud using a pretrained deep learning model in the Lidar Labeler app.

Lidar Labeler App

Good ground truth data is crucial for the development and performance evaluation of automated driving and flight algorithms. However, creating and maintaining a diverse, high-quality, and labeled data set requires significant effort. The Lidar Labeler app provides a framework to automate the labeling process using the AutomationAlgorithm class. You can create a custom algorithm and use it in the app to label your entire data set. You can also edit the automated labels for any challenging scenarios missed by the algorithm.

In this example, you:

• Use a PointSeg semantic segmentation network trained on a highway scene data set collected using an Ouster® OS1 sensor.

• Import a pretrained PointSeg network automation algorithm that you can use in the Lidar Labeler app to automatically label two object categories, car and truck, in the point cloud.

Download Point Cloud Data

Run this code to download the highway scene data set. The data set contains 1617 point clouds stored as pointCloud objects in a PCD format.

url = "https://www.mathworks.com/supportfiles/lidar/data/WPI_LidarData.tar.gz";
outputFolder = fullfile(tempdir,"WPI");
lidarDataTarFile = fullfile(outputFolder,"WPI_LidarData.tar.gz");
 
if ~exist(lidarDataTarFile,"file") 
    mkdir(outputFolder);
    disp("Downloading WPI Lidar driving data (760 MB)...");
    websave(lidarDataTarFile,url);
    untar(lidarDataTarFile,outputFolder); 
end
 
% Check if tar.gz file is downloaded, but not uncompressed.
if ~exist(fullfile(outputFolder,"WPI_LidarData.mat"),"file")
    untar(lidarDataTarFile,outputFolder);
end
lidarData = load(fullfile(outputFolder,"WPI_LidarData.mat"));
 
path = strcat(outputFolder,"\","lidarData")

path = 
"C:\Users\ktripp\AppData\Local\Temp\WPI\lidarData"

mkdir(path);

Warning: Directory already exists.

for i = 1:numel(lidarData.lidarData)
    temp = lidarData.lidarData{i};
    pcwrite(temp,strcat(path,"\",num2str(i,"%05d"),".pcd")); 
end

Note: Depending on your internet connection, the download can take some time. The code suspends MATLAB® execution until the download process is complete. Alternatively, you can download the data set to your local disk using your web browser, and then extract WPI_LidarData. To use the file you downloaded from the web, change the outputFolder variable in the code to the location of the downloaded file.

Download Pretrained Model

Download the pretrained PointSeg network using this command. For more information on how to train a PointSeg network, see the Lidar Point Cloud Semantic Segmentation Using PointSeg Deep Learning Network example.

pretrainedURL = "https://www.mathworks.com/supportfiles/lidar/data/trainedPointSegNet.mat";

Import Pretrained Network into Lidar Labeler

Open the Lidar Labeler app and load the point cloud sequence.

pointCloudDir = fullfile(outputFolder,"lidarData"); 
lidarLabeler(pointCloudDir)

In the ROI Labels pane, click Label. Define two ROI labels with the names Car and Truck, of label type Cuboid or Voxel. Optionally, you can select colors for the labels. Click OK.

On the Label tab of the app toolstrip, in the Automate Labeling section, click Select Algorithm > Automation Using Pre-trained Models.

Click Automate, the app opens the Automate tab with directions to use the algorithm.

Select Settings to open the Pre-trained Model Setting dialog box. Now, click Import Network and browse to the downloaded PointSeg network folder.

Select the imported model by setting Select Network to trainedPointSegNet.

Map the Label Definitions to the Output classes of the network and click OK.

Click Run to generate automated labels for the output classes of the network. After the automation is complete, you can view the generated labels in all point cloud frames. You can also fine-tune the automation results by manually adjusting the detected bounding boxes or adding new ones.

When you are satisfied with the detected vehicle bounding boxes for the entire sequence, click Accept. You can then continue to manually adjust labels or export the labeled ground truth to the MATLAB workspace.

Supported Deep Learning Networks for Automation

• You can automate labeling using any DAGNetwork trained for semantic segmentation and object detection in point clouds.

• For other network architectures, or to perform training, preprocessing, or postprocessing on the network data, use the custom automation algorithm template to automate labeling. For more information, see Create Automation Algorithm for Labeling.