Main Content

Export and Explore Ground Truth Labels for Multiple Signals

Open Script
Step 3 of 3 in Get Started with Ground Truth Labelling

After labeling the signals by following the Label Ground Truth for Multiple Signals procedure, export the labels and explore how they are stored.

Setup

Open the Ground Truth Labeler app session containing the labeled signals. You can open the session from the MATLAB® command line. For example, if you saved the session to a MAT-file named groundTruthLabelingSession, enter this command.

groundTruthLabeler groundTruthLabelingSession.mat

On the app toolstrip, select Export > To Workspace. In the export to workspace window, use the default export variable name, gTruth, and click OK. The app exports a groundTruthMultisignal object, gTruth, to the MATLAB® workspace. This object contains the ground truth labels captured from the app session.

If you did not export a groundTruthMultisignal object to the workspace, load a predefined object from the variable gTruth. The function used to load this object is attached to this example as a supporting file. If you are using your own object, data such as label positions can differ from the data shown in this example.

if (~exist('gTruth','var'))
    gTruth = helperLoadGTruthGetStarted;
end

Display the properties of the groundTruthMultisignal object, gTruth. The object contains information about the signal data sources, label definitions, and ROI and scene labels. This information is stored in separate properties of the object.

gTruth

gTruth = 

  groundTruthMultisignal with properties:

          DataSource: [1x2 vision.labeler.loading.MultiSignalSource]
    LabelDefinitions: [3x7 table]
        ROILabelData: [1x1 vision.labeler.labeldata.ROILabelData]
      SceneLabelData: [1x1 vision.labeler.labeldata.SceneLabelData]

In this example, you examine the contents of each property to learn how the object stores ground truth labels.

Data Sources

The DataSource property contains information about the data sources. This property contains two MultiSignalSource objects: one for the video source and one for the point cloud sequence source. Display the contents of the DataSource property.

gTruth.DataSource

ans = 

  1x2 heterogeneous MultiSignalSource (VideoSource, PointCloudSequenceSource) array with properties:

    SourceName
    SourceParams
    SignalName
    SignalType
    Timestamp
    NumSignals

The information stored in these objects includes the paths to the data sources, the names of the signals that they contain, and the timestamps for those signals. Display the signal names for the data sources.

gTruth.DataSource.SignalName

ans = 

    "video_01_city_c2s_fcw_10s"


ans = 

    "lidarSequence"

Label Definitions

The LabelDefinitions property contains a table of information about the label definitions. Display the label definitions table. Each row contains information about an ROI or scene label definition. The car label definition has two rows: one for when the label is drawn as a rectangle on Image signals and one for when the label is drawn as a cuboid on PointCloud signals.

gTruth.LabelDefinitions

ans =

  3x7 table

       Name        SignalType    LabelType       Group        Description           LabelColor            Hierarchy  
    ___________    __________    _________    ____________    ___________    ________________________    ____________

    {'car'    }    Image         Rectangle    {'Vehicles'}    {0x0 char}     {[0.5862 0.8276 0.3103]}    {1x1 struct}
    {'car'    }    PointCloud    Cuboid       {'Vehicles'}    {0x0 char}     {[0.5862 0.8276 0.3103]}    {1x1 struct}
    {'daytime'}    Time          Scene        {'None'    }    {0x0 char}     {[          0.0588 1 1]}    {0x0 double}

The Hierarchy column stores information about the sublabel and attribute definitions of a parent ROI label. Display the sublabel and attribute information for the car label when it is drawn as a rectangle. This label contains one sublabel, brakeLight, and no attributes.

gTruth.LabelDefinitions.Hierarchy{1}

ans = 

  struct with fields:

     brakeLight: [1x1 struct]
           Type: Rectangle
    Description: ''

Display information about the brakeLight sublabel for the parent car label. The sublabel contains one attribute, isOn. Sublabels cannot have their own sublabels.

gTruth.LabelDefinitions.Hierarchy{1}.brakeLight

ans = 

  struct with fields:

           Type: Rectangle
    Description: ''
     LabelColor: [0.5862 0.8276 0.3103]
           isOn: [1x1 struct]

Display information about the isOn attribute for the brakeLight sublabel. This attribute has no default value, so the DefaultValue field is empty.

gTruth.LabelDefinitions.Hierarchy{1}.brakeLight.isOn

ans = 

  struct with fields:

    DefaultValue: []
     Description: ''

ROI Label Data

The ROILlabelData property contains an ROILabelData object with properties that contain ROI label data for each signal. The names of the properties match the names of the signals. Display the object property names.

gTruth.ROILabelData

ans = 

  ROILabelData with properties:

    video_01_city_c2s_fcw_10s: [204x1 timetable]
                lidarSequence: [34x1 timetable]

Each property contains a timetable of ROI labels drawn at each signal timestamp, with one column per label. View a portion the video and the lidar point cloud sequence timetables. Set a time interval from 8 to 8.5 seconds. This time interval corresponds to the start of the time interval labeled in the Label Ground Truth for Multiple Signals procedure. The video timetable contains more rows than the point cloud sequence timetable because the video contains more label frames.

timeInterval = timerange(seconds(8),seconds(8.5));
videoLabels = gTruth.ROILabelData.video_01_city_c2s_fcw_10s(timeInterval,:)
lidarLabels = gTruth.ROILabelData.lidarSequence(timeInterval,:)

videoLabels =

  10x1 timetable

      Time          car     
    ________    ____________

    8 sec       {1x1 struct}
    8.05 sec    {1x1 struct}
    8.1 sec     {1x1 struct}
    8.15 sec    {1x1 struct}
    8.2 sec     {1x1 struct}
    8.25 sec    {1x1 struct}
    8.3 sec     {1x1 struct}
    8.35 sec    {1x1 struct}
    8.4 sec     {1x1 struct}
    8.45 sec    {1x1 struct}


lidarLabels =

  2x1 timetable

       Time           car     
    __________    ____________

    8.0495 sec    {1x1 struct}
    8.3497 sec    {1x1 struct}

View the rectangle car labels for the first video frame in the time interval. The label data is stored in a structure.

videoLabels.car{1}

ans = 

  struct with fields:

      Position: [296 203 203 144]
    brakeLight: [1x2 struct]

The Position field stores the positions of the car labels. This frame contains only one car label, so in this case, Position contains only one rectangle bounding box. The bounding box position is of the form [x y w h], where:

  • x and y specify the upper-left corner of the rectangle.

  • w specifies the width of the rectangle, which is the length of the rectangle along the x-axis.

  • h specifies the height of the rectangle, which is the length of the rectangle along the y-axis.

The car label also contains two brakeLight sublabels at this frame. View the brakeLight sublabels. The sublabels are stored in a structure array, with one structure per sublabel drawn on the frame.

videoLabels.car{1}.brakeLight

ans = 

  1x2 struct array with fields:

    Position
    isOn

View the bounding box positions for the sublabels.

videoLabels.car{1}.brakeLight.Position

ans =

   304   245    50    46


ans =

   435   243    54    51

View the values for the isOn attribute in each sublabel. For both sublabels, this attribute is set to logical 1 (true).

videoLabels.car{1}.brakeLight.isOn

ans =

  logical

   1


ans =

  logical

   1

Now view the cuboid car labels for the first point cloud sequence frame in the time interval. Point cloud sequences do not support sublabels or attributes. Instead of storing cuboid labels in the Position field of a structure, cuboid bounding box positions are stored in an M-by-9 matrix, where M is the number of cuboid labels. Because this frame contains only one cuboid label, in this case M is 1.

lidarLabels.car{1}

ans = 

  struct with fields:

      Position: [-1.1559 -0.7944 1.2012 12.6196 5.9278 3.0010 0 0 0]
    brakeLight: []

The 1-by-9 bounding box position is of the form [xctr, yctr, zctr, xlen, ylen, zlen, xrot, yrot, zrot], where:

  • xctr, yctr, and zctr specify the center of the cuboid.

  • xlen, ylen, and zlen specify the length of the cuboid along the x-, y-, and z-axis, respectively, before rotation has been applied.

  • xrot, yrot, and zrot specify the rotation angles for the cuboid along the x-, y-, and z-axis, respectively. These angles are clockwise-positive when looking in the forward direction of their corresponding axes.

This figure shows how these values specify the position of a cuboid.

Scene Label Data

The SceneLabelData property contains a SceneLabelData object with properties that contain scene label data across all signals. The names of the properties match the names of the scene labels. Display the object property names.

gTruth.SceneLabelData

ans = 

  SceneLabelData with properties:

    daytime: [0 sec    10.15 sec]

The daytime label applies to the entire time interval, which is approximately 10 seconds.

Use Ground Truth Labels

The labels shown in this example are for illustrative purposes only. For your own labeling, after you export the labels, you can use them as training data for object detectors. To gather label data from the groundTruthMultisignal object for training, use the gatherLabelData function.

To share labeled ground truth data, share the ground truth MAT-file containing the groundTruthMultisignal object, not the MAT-file containing the app session. For more details, see Share and Store Labeled Ground Truth Data.

See Also

| | |

Related Topics

Get Started with Ground Truth Labelling