hrnetObjectKeypointDetector

Create object keypoint detector using HRNet deep learning network

Since R2023b

Description

The hrnetObjectKeypointDetector object creates a high resolution deep learning network (HRNet) object keypoint detector for detecting object keypoints in an image. Using this object, you can:

• Create a pretrained HRNet object keypoint detector by using HRNet deep learning networks trained on the COCO keypoint detection data set.

• Create a custom HRNet object keypoint detector by using a pretrained HRNet deep learning network.

Creation

Syntax

detector = hrnetObjectKeypointDetector

detector = hrnetObjectKeypointDetector(name)

detector = hrnetObjectKeypointDetector(name,keypointClassNames)

Description

Pretrained HRNet Object Keypoint Detector

detector = hrnetObjectKeypointDetector creates a default pretrained HRNet object keypoint detector. To create the default HRNet object keypoint detector, MATLAB uses an HRNet deep learning network, HRNet-W32, trained on the COCO keypoint detection data set. In an HRNet-W32 network, the last three stages of the high-resolution subnetworks have 32 convolved feature maps. For more information about HRNet architecture, see Getting Started with HRNet.

detector = hrnetObjectKeypointDetector(name) creates a pretrained HRNet object keypoint detector by using the specified HRNet deep learning network, which has been trained on the COCO keypoint detection data set.

Custom HRNet Object Keypoint Detector

detector = hrnetObjectKeypointDetector(name,keypointClassNames) creates a pretrained HRNet object detector, and configures it to perform transfer learning using a specified set of keypoint classes. For optimal results, you must train the detector on new training images before performing detection.

detector = hrnetObjectKeypointDetector(name,keypointClassNames,Name=Value) sets the ModelName, InputSize, KeypointConnections and Threshold properties of the object keypoint detector using one or more name-value arguments.

Example: hrnetObjectKeypointDetector(name,keypointClassNames,ModelName="customDetector") sets the name for the object detector to "customDetector".

Note

This functionality requires Deep Learning Toolbox™ and the Computer Vision Toolbox™ Model for Object Keypoint Detection. You can download and install the Computer Vision Toolbox Model for Object Keypoint Detection from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

Input Arguments

name — Name of pretrained HRNet deep learning network

"human-full-body-w32" | "human-full-body-w48"

Name of the pretrained HRNet deep learning network, specified as one of these options:

• "human-full-body-w32" — A pretrained HRNet deep learning network created using HRNet-W32 as the base network, and trained on the COCO keypoint detection data set.

• "human-full-body-w48" — A pretrained HRNet deep learning network created using HRNet-W48 as the base network, and trained on the COCO keypoint detection data set.

In HRNet-W32 and HRNet-W48 networks, the last three stages of the high-resolution subnetworks have 32 and 48 convolved feature maps, respectively.

Data Types: char | string

keypointClassNames — Names of object keypoint classes

vector of strings | cell array of character vectors | categorical vector

Names of object keypoint classes for training the detector, specified as a vector of strings, cell array of character vectors, or categorical vector. This argument sets the KeyPointClasses property of the hrnetObjectKeypointDetector object.

Data Types: cell | string | categorical

Properties

Network — HRNet deep learning network

dlnetwork object

This property is read-only.

HRNet deep learning network to use for object keypoint detection, stored as a dlnetwork (Deep Learning Toolbox) object.

ModelName — Name of object keypoint detector

'human-full-body-w32' (default) | character vector | string scalar

This property is read-only.

Name for the object keypoint detector, specified as a character vector or string scalar. To set this property, specify it at object creation. You can set this property only when you also specify the keypointClassNames input argument.

Example: hrnetObjectKeypointDetector(name,keypointClassNames,ModelName="customDetector") sets the name for the object detector to "customDetector".

InputSize — Image size used for training

row vector of integers

This property is read-only.

Image size used for training the object detector, specified as a row vector of integers of the form [H W C], where H, W, and C are the height, width, and number of channels of the image, respectively. The default value is the network input size. To set this property, specify it at object creation. You can set this property only when also specify the keypointClassNames input argument.

Example: detector = hrnetObjectKeypointDetector(name,keypointClassNames,InputSize=[384 288 3]) specifies the image size of the training images as 384-by-288 pixels across 3 channels.

KeyPointClasses — Names of object keypoint classes

categorical vector

This property is read-only.

Names of object keypoint classes to detect, specified as a categorical vector.

You can set this property only when you specify the keypointClassNames input argument. Otherwise, the object uses the default classes for the specified HRNet deep learning network.

KeypointConnections — Keypoint class indices of keypoint connections

L-by-2 matrix

This property is read-only.

Keypoint class indices of the keypoint connections, specified as an L-by-2 matrix. Each row denotes the keypoint class indices that connect two keypoints, where L is the number of keypoint connections. To set this property, specify it at object creation. You can set this property only when you specify the keypointClassNames input argument.

Example: hrnetObjectKeypointDetector(name,keypointClassNames,KeypointConnections=[2 4; 3 5]) establishes two connections: one between a set of keypoints with keypoint class indices 2 and 4, and the other between a set of keypoints with the keypoint classes indices 3 and 5.

Threshold — Threshold value of valid keypoints

0.2 (default) | scalar in the range [0, 1]

Threshold value of valid keypoints, specified as a scalar in the range [0, 1]. For a threshold value of 0.4, the object considers all keypoints with a confidence score greater than or equal to 0.4 to be valid keypoints. To set this property, specify it at object creation. You can set this property only when you specify the keypointClassNames input argument.

Example: hrnetObjectKeypointDetector(name,keypointClassNames,Threshold=0.4) considers all keypoints with confidence score greater than or equal to 0.4 to be valid keypoints.

Object Functions

detect	Detect object keypoints using HRNet object keypoint detector
visibleKeypoints	Extract visible keypoints from keypoints detected by HRNet object keypoint detector

Examples

Create Pretrained HRNet Object Keypoint Detector

Specify the name of a pretrained HRNet object keypoint detector.

name = "human-full-body-w32";

Create an HRNet object keypoint detector by using the pretrained HRNet deep learning network.

keypointDetector = hrnetObjectKeypointDetector(name);

Display and inspect the properties of the HRNet object keypoint detector.

disp(keypointDetector)

  hrnetObjectKeypointDetector with properties:

              ModelName: 'human-full-body-w32'
                Network: [1x1 dlnetwork]
        KeyPointClasses: [17x1 categorical]
              InputSize: [384 288 3]
    KeypointConnections: [17x2 double]
              Threshold: 0.2000

analyzeNetwork(keypointDetector.Network)

Read a test image into the workspace.

I = imread("visionteam.jpg");

Use a YOLOv4 object detector to detect bounding boxes around humans in the image.

detector = yolov4ObjectDetector("csp-darknet53-coco");
[bboxes,scores,labels] = detect(detector,I,Threshold=0.4);

Select the bounding box detections for the person class.

personIndices = labels == "person";
personBoxes = bboxes(personIndices,:);

Detect the object keypoints of the people in the test image by using the pretrained HRNet object keypoint detector.

[keypoints,keypointScores,valid] = detect(keypointDetector,I,personBoxes);

Insert the detected keypoints into the test image and display the results.

detectedKeypoints = insertObjectKeypoints(I,keypoints,KeypointColor="yellow",KeypointSize=2);
imshow(detectedKeypoints)

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

• The detect function does not support ImageDatastore objects as input for code generation.

• The bboxes argument to the detect function must be a code generation constant (coder.const()) and a M-by-4 matrix. M is the number of objects in the input image.

• Code generation does not support the MiniBatchSize, ExecutionEnvironment, and Acceleration name-value arguments for detect.

• To prepare an hrnetObjectKeypointDetector object for code generation, use loadHRNETObjectKeypointDetector.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

• The detect function does not support ImageDatastore object as input for code generation.

• The bboxes argument to the detect function must be a code generation constant (coder.const()) and a M-by-4 matrix. M is the number of objects in the input image.

• Code generation does not support the MiniBatchSize, ExecutionEnvironment, and Acceleration name-value arguments for detect.

• To prepare an hrnetObjectKeypointDetector object for code generation, use loadHRNETObjectKeypointDetector.

GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

Version History

Introduced in R2023b