Tracking and Motion Estimation

Optical flow, activity recognition, motion estimation, object re-identification, and tracking

Motion estimation and tracking are key activities in many computer vision applications, including activity recognition, traffic monitoring, automotive safety, and surveillance.

Computer Vision Toolbox™ provides video tracking algorithms, such as continuously adaptive mean shift (CAMShift) and Kanade-Lucas-Tomasi (KLT). You can use these algorithms for tracking a single object, or as building blocks in a more complex tracking system. The toolbox also provides a framework for multiple object tracking that includes a Kalman filter and using the Hungarian algorithm for assigning object detections to tracks.

Motion estimation is the process of determining the movement of blocks between adjacent video frames. This toolbox includes motion estimation algorithms, such as optical flow, block matching, and template matching. These algorithms create motion vectors, which can relate to the whole image, blocks, arbitrary patches, or individual pixels. For block and template matching, the evaluation metrics for finding the best match include mean square error (MSE), mean absolute deviation (MAD), maximum absolute difference (MaxAD), sum of absolute difference (SAD), and sum of squared difference (SSD).

Functions

expand all

Load, Save, and Display Video

`vision.BinaryFileReader`	Read video data from binary files
`vision.BinaryFileWriter`	Write binary video data to files
`vision.DeployableVideoPlayer`	Display video
`vision.VideoPlayer`	Play video or display image
`vision.VideoFileWriter`	Write video frames and audio samples to video file
`VideoReader`	Create object to read video files

Object Tracking

`assignDetectionsToTracks`	Assign detections to tracks for multiobject tracking
`bbox2points`	Convert rectangle to corner points list
`configureKalmanFilter`	Create Kalman filter for object tracking
`vision.KalmanFilter`	Correction of measurement, state, and state estimation error covariance
`vision.HistogramBasedTracker`	Histogram-based object tracking
`vision.PointTracker`	Track points in video using Kanade-Lucas-Tomasi (KLT) algorithm
`vision.BlockMatcher`	Estimate motion between images or video frames
`vision.TemplateMatcher`	Locate template in image

Object Re-Identification

`reidentificationNetwork`	Re-identification deep learning network for re-identifying and tracking objects (Since R2024a)
`extractReidentificationFeatures`	Extract object re-identification (ReID) features from image (Since R2024a)
`trainReidentificationNetwork`	Train re-identification (ReID) deep learning network (Since R2024a)
`evaluateReidentificationNetwork`	Evaluate re-identification network using cumulative matching characteristic (CMC) and mean average precision (mAP) metrics (Since R2024a)
`reidentificationMetrics`	Re-identification (ReID) quality metrics (Since R2024a)

Motion Estimation

`opticalFlow`	Object for storing optical flow matrices
`opticalFlowRAFT`	Estimate optical flow using RAFT deep learning algorithm (Since R2024b)
`opticalFlowFarneback`	Object for estimating optical flow using Farneback method
`opticalFlowHS`	Object for estimating optical flow using Horn-Schunck method
`opticalFlowLK`	Object for estimating optical flow using Lucas-Kanade method
`opticalFlowLKDoG`	Object for estimating optical flow using Lucas-Kanade derivative of Gaussian method
`vision.BlockMatcher`	Estimate motion between images or video frames
`vision.TemplateMatcher`	Locate template in image

Visualization and Display

`insertMarker`	Insert markers in image or video
`insertShape`	Insert shapes in image or video
`insertObjectAnnotation`	Annotate truecolor or grayscale image or video
`insertText`	Insert text in image or video
`imshow`	Display image
`imshowpair`	Compare differences between images

Topics

Multiple Object Tracking
Locate a moving object or multiple objects over time in a video stream.

Featured Examples

New

Automate Labeling of Objects in Video Using RAFT Optical Flow

Use a pretrained RAFT optical flow estimation network to propagate a predefined object mask from one frame to the next in a video sequence.

Since R2024b
Open Live Script

Reidentify People Throughout a Video Sequence Using ReID Network

Track people throughout a video sequence using re-identification with a residual network.

Open Live Script

Automate Ground Truth Labeling for Object Tracking and Re-Identification

Create an automation algorithm to automatically label data for object tracking and for object re-identification.

Open Live Script

Convert Ground Truth Labeling Data for Object Re-Identification

Convert a groundTruth object to the re-identification training data format.

Open Live Script

Convert Ground Truth Labeling Data for Object Tracking

Convert a groundTruth object to the trackCLEARmetrics truth format.

Open Live Script

Multi-Object Tracking with DeepSORT

Integrate appearance features from a re-Identification (Re-ID) deep neural network with a multi-object tracker.

Open Live Script

Tracking Pedestrians from a Moving Car

Track pedestrians using a camera mounted in a moving car.

Open Script

Motion-Based Multiple Object Tracking

Perform automatic detection and motion-based tracking of moving objects in a video from a stationary camera.

Open Script

Visual Tracking of Occluded and Unresolved Objects

Resolve challenging tracking scenarios when objects are occluded or when they are in close proximity to each other.

Open Live Script

Implement Simple Online and Realtime Tracking

Implement the Simple Online and Realtime (SORT) object tracking algorithm using the Computer Vision Toolbox and the Sensor Fusion and Tracking Toolbox™.

Open Live Script

Track a Face in Scene

Track a face using a minimum Eigen features detection function and a point tracker.

Open Script

Face Detection and Tracking Using Live Video Acquisition

Automatically detect and track a face in a live video stream, using the KLT algorithm.

Open Script

Face Detection and Tracking Using CAMShift

Automatically detect and track a face.

Open Script

Face Detection and Tracking Using the KLT Algorithm

Automatically detect and track a face using feature points. The approach in this example keeps track of the face even when the person tilts his or her head, or moves toward or away from the camera.

Open Live Script

Use Kalman Filter for Object Tracking

Use the vision.KalmanFilter object and configureKalmanFilter function to track objects.

Open Script

Import Camera-Based Datasets in MOT Challenge Format for Object Tracking

Read camera image sequences and convert the ground truth and detections to Sensor Fusion and Tracking Toolbox formats using a custom dataset.

Open Live Script

Multi-Object Tracking and Human Pose Estimation

Detect multiple people, track them, and estimate their body poses in a video by using pretrained deep learning networks and a global nearest-neighbor (GNN) assignment tracking approach.

Open Live Script