Main Content

trackAssignmentMetrics

Track establishment, maintenance, and deletion metrics

Description

The trackAssignmentMetrics System object™ compares tracks from a multi-object tracking system against known truth by automatic assignment of tracks to the known truths at each track update. An assignment distance metric determines the maximum distance for which a track can be assigned to the truth object. A divergence distance metric determines when a previously assigned track can be reassigned to a different truth object when the distance exceeds another set threshold.

To generate track assignment metrics:

  1. Create the trackAssignmentMetrics object and set its properties.

  2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Description

assignmentMetrics = trackAssignmentMetrics creates a trackAssignmentMetrics System object, assignmentMetrics, with default property values.

example

assignmentMetrics = trackAssignmentMetrics(Name,Value) sets properties for the trackAssignmentMetrics object using one or more name-value pairs. For example, assignmentMetrics = trackAssignmentMetrics('AssignmentThreshold',5) creates a trackAssignmentMetrics object with an assignment threshold of 5. Enclose property names in quotes.

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

Maximum permitted assignment distance between a newly encountered or divergent track and a truth object, specified as a nonnegative scalar. For distances beyond this value, assignments between the track and the truth cannot take place. Units are in normalized estimation error squared (NEES).

Data Types: single | double

Maximum permitted divergence distance between a track state and the state of an assigned truth object, specified as a nonnegative scalar. For distances beyond this value, tracks are eligible for reassignment to a different truth object. Units are in NEES.

Data Types: single | double

Distance function format specified as 'built-in' or 'custom'.

  • 'built-in' – Enable the MotionModel, AssignmentDistance, and DivergenceDistance properties. These properties are convenient interfaces when tracks are reported by any built-in multi-object tracker (such as trackerGNN), and truths reported by the platformPoses object function of a trackingScenario object.

  • 'custom' – Enable custom properties: AssignmentDistanceFcn, DivergenceDistanceFcn, IsInsideCoverageAreaFcn, TruthIdentifierFcn, and TrackIdentifierFcns. You can use these properties to construct acceptance or divergence distances, coverage areas, and identifiers for arbitrary 'tracks' and 'truths' input arrays.

Maximum time that must elapse before the metric deletes tracks or truths that were not updated, specified as a nonnegative scalar.

When specify 0, the metric does not check the time of the tracks or truths input arguments and deletes a track or a truth if it was not updated at any single time step.

When specified as a positive scalar,

  • The tracks input must have a property or field named UpdateTime. The truths input must have a field named Time.

  • The tracks and truths inputs for each update must be at the same time.

  • The metric deletes tracks or truths that were not updated within the time period specified in the MaxUnreportedPeriod property.

  • The track summary and truth summary outputs form the trackAssignmentMetric object, the trackMetricsTable object function, or the truthMetricsTable function have two additional fields: MeanTimeBetweenReports and MaxTimeBetweenReports.

Example: 0.5

Data Types: single | double

Built-in Distance Function Format

Desired platform motion model, specified as 'constvel', 'constacc', 'constturn', or 'singer'. This property selects the motion model used by the tracks input.

The motion models expect the 'State' field of the tracks to have a column vector containing these values:

  • 'constvel' — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

  • 'constacc' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

  • 'constturn' — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

  • 'singer' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The 'StateCovariance' field of the tracks input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the 'State' field of the tracks input.

Type of assignment distance, specified as 'posnees', 'velnees', 'posabserr', or 'velabserr'. The type specifies the physical quantity used for assignment. When a new track is detected or a track becomes divergent, the track is compared against truth using this quantity. The assignment seeks the closest truth within the threshold defined by the AssignmentThreshold property.

  • 'posnees' – NEES error of track position

  • 'velnees' – NEES error in track velocity

  • 'posabserr' – Absolute error of track position

  • 'velabserr' – Absolute error of track velocity

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'built-in'.

Type of divergence distance, specified as 'posnees', 'velnees', 'posabserr', or 'velabserr'. The type specifies the physical quantity used for assessing divergence. When a track was previously assigned to truth, the distance between them is compared to this quantity on subsequent update steps. Any track whose divergence distance to its truth assignment exceeds the value of DivergenceThreshold is considered divergent and can be reassigned to a new truth.

  • 'posnees' – NEES error of track position

  • 'velnees' – NEES error in track velocity

  • 'posabserr' – Absolute error of track position

  • 'velabserr' – Absolute error of track velocity

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'built-in'.

Custom Distance Function Format

Assignment distance function, specified as a function handle. This function determines the assignment distance between truths and tracks. Whenever a new track is detected or an existing track becomes divergent, the track needs to be compared against all truths at the current step. This function help to find the closest truth relative to the track within the threshold defined by the AssignmentThreshold property.

The function must have the following syntax:

dist = assignmentdistance(onetrack,onetruth)
The function must return a nonnegative assignment distance, dist, typically expressed in units of NEES. onetrack is an element of the tracks array input argument. onetruth is an element of the truths array input argument.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

Divergence distance function, specified as a function handle. This function determines the divergence distance between truths and tracks. If the divergence distance from a track to its truth assignment exceeds the DivergenceThreshold, the track is considered divergent and can be reassigned to a new truth.

The function must have the following syntax:

dist = divergencedistance(onetrack,onetruth)
The function must return a non-negative divergence distance, dist, typically expressed in units of NEES. onetrack is an element of the tracks array input argument. onetruth is an element of the truths array input argument.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

Function to determine the time that a truth object is detectable, specified as a function handle. This function determines the time that a truth object is inside the coverage area of the sensors and is therefore detectable.

The function must have the following syntax:

status = isinsidecoveragearea(truths)
and return a logical array, status. truths is an array of truth objects expected to be passed in on each step. status is a logical array with the same size as the truths input. An entry of status is true when the corresponding truth object specified by truths is within the coverage area of the sensors.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

Track identifier function for the tracks input, specified as a function handle. The track identifiers are unique strings or numeric values.

The function must have the following syntax

trackids = trackidentifier(tracks)
and return a numeric array, trackids. trackids must have the same size as tracks input argument. The default track identification function assumes Tracks is an array of struct or class with a TrackID field or property.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

Truth identifier function for the truths input, specified as a function handle. The truth identifiers are unique strings or numeric values.

The function must have the following syntax

truthids = truthidentifier(truths)
and return a numeric array, truthids. truthids must have the same size as the truths input argument. The default truth identification function assumes truths is an array of struct or class with a PlatformID field or property.

Dependencies

To enable this property, set the DistanceFunctionFormat property to 'custom'.

Data Types: function_handle

Track identifier for invalid assignment, specified as a scalar or string. This value is returned when the track assignment is invalid. The value must be of the same class as returned by the function handle specified in TrackIdentifierFcn.

Example: -1

Data Types: single | double | string

Truth identifier for invalid assignment, specified as a scalar or string. This value is returned when the truth assignment is invalid. The value must be of the same class as returned by the function handle specified in TruthIdentifierFcn.

Example: -1

Data Types: single | double | string

Usage

To compute metrics, call the track assignment metrics with arguments, as if it were a function (described here).

Description

[tracksummary,truthsummary] = assignmentMetrics(tracks,truths) returns structures, tracksummary and truthsummary, containing cumulative metrics across all tracks and truths, obtained from the previous object update.

Input Arguments

expand all

Track information, specified as an array of objects or an array of structures. When you set the MaxUnreportedPeriod property to a nonzero value, the object or structure must have an UpdateTime property or field, respectively.

If the DistanceFunctionFormat property is specified as 'built-in', then you must specify this argument as an array of objectTrack objects or as an array of track structures. Each track structure must contain the State, StateCovariance, and TrackID properties or fields. The track outputs from built-in trackers, such as trackerGNN, are compatible with the tracks input.

Data Types: struct

Truth information, specified as an array of structures. When you set the MaxUnreportedPeriod property as a nonzero value, the structure must have a Time field.

If the DistanceFunctionFormat property is specified as 'built-in', then you must specify this argument as an array of truth structures. Each truth structure must contain the PlatformID, Position, and Velocity fields. You can obtain truth structures from the platformPoses function of the trackingScenario object.

Data Types: struct

Output Arguments

expand all

Cumulative metrics over all tracks, returned as a structure. The metrics are computed over all tracks since the last call to the reset object function. The structure has these fields:

FieldDescription
TotalNumTracks

The total number of unique track identifiers encountered

NumFalseTracks

The number of tracks never assigned to any truth

MaxSwapCount

Maximum number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.

TotalSwapCount

Total number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.

MaxDivergenceCount

Maximum number of divergences. A track is divergent when the result of the DivergenceDistanceFcn is greater than the divergence threshold.

TotalDivergenceCount

Total number of divergences. A track is divergent when the result of the divergence distance function is greater than the divergence threshold.

MaxDivergenceLength

Maximum number of updates during which each track was in a divergent state

TotalDivergenceLength

Total number of updates during which each track was in a divergent state

MaxRedundancyCount

The maximum number of additional tracks assigned to the same truth

TotalRedundancyCount

The total number of additional tracks assigned to the same truth

MaxRedundancyLength

Maximum number of updates during which each track was in a redundant state

TotalRedundancyLength

Total number of updates during which each track was in a redundant state

MaxTimeBetweenReportsThe maximum time interval between adjacent track updates to the metric across all tracks. To enable this field, set the MaxUnreportedPeriod property to a nonzero value. The field is returned as a NaN value if there was only one update to the metric.
MeanTimeBetweenReportsThe average time interval between adjacent track updates to the metric across all tracks. To enable this field, set the MaxUnreportedPeriod property to a non-zero value. The field is returned as a NaN value if there was only one update to the metric.

Data Types: struct

Cumulative assignment metrics over all truths, returned as a structure. The metrics are computed over all truths since the last call to the reset object function. The structure has these fields:

FieldDescription
TotalNumTruths

The total number of unique truth identifiers encountered

NumMissingTruths

The number of truths never established with any track

MaxEstablishmentLength

Maximum number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.

TotalEstablishmentLength

Total number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.

MaxBreakCount

Maximum number of times each truth was unassociated by any track after being established.

TotalBreakCount

Total number of times each truth was unassociated by any track after being established.

MaxBreakLength

Maximum number of updates during which each truth was in a broken state

TotalBreakLength

Total number of updates during which each truth was in a broken state

MaxTimeBetweenReportsThe maximum time interval between adjacent truth updates to the metric across all truths. To enable this field, set the MaxUnreportedPeriod property to a non-zero value. The field is returned as a NaN value if there was only one update to the metric.
MeanTimeBetweenReportsThe average time interval between adjacent truth updates to the metric across all truths. To enable this field, set the MaxUnreportedPeriod property to a non-zero value. The field is returned as a NaN value if there was only one update to the metric.

Data Types: struct

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

currentAssignmentMapping of tracks to truth
trackMetricsTableCompare tracks to truth
truthMetricsTableCompare truth to tracks
evaluateBatch Compute assignment metrics for recorded data
releaseRelease resources and allow changes to System object property values and input characteristics
resetReset internal states of System object
isLockedDetermine if System object is in use
cloneCreate duplicate System object

Examples

collapse all

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

  • Extract the tracks and ground truth at the i th tracker update.

  • Analyze and retrieve the current track-to-truth assignment.

  • Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)
ans=4×16 table
    TrackID    AssignedTruthID    Surviving    TotalLength    UnreportedStatus    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ________________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120             false              false               0                false                 3                   3                false                 0                   0                false                  0               0    
       2             NaN            false         1736             false              false               0                false                 8                  88                false                 0                   0                false                 28               3    
       6               3            true          1138             false              false               0                false                 4                 314                false                 1                  28                false                  0               2    
       8               2            true           662             false              false               0                false                 2                  29                false                 1                 169                false                 28               0    

Show the truth metrics table.

truthMetricsTable(tam)
ans=2×11 table
    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    UnreportedStatus    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ________________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678             false             false           4             168            true                 true                    56         
       3               6                false            2678             false             false           3             645            true                 true                    84         

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)
ans=2×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453 

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)
ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325 

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)
ans=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813 

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)
ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183 

Version History

Introduced in R2018b

expand all