Deploy an Edge Detection Application to Target ARM Cores of NVIDIA Jetson

Since R2024a

This example uses:

MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms
MATLAB Coder MATLAB Coder

This example shows how to target the ARM cores of an NVIDIA Jetson board. This example deploys Sobel edge detection application on NVIDIA Jetson board to perform edge detection on images captured by a connected camera in real-time.

Prerequisites

Target Board Requirements

NVIDIA Jetson embedded platform.
V4L2 and SDL (v1.2) libraries.
GStreamer libraries.
The camera module must be connected to the target board.
Environment variables for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.

Additionally, if you cannot connect the target board to a local network, this example requires an Ethernet crossover cable to connect the target board and host PC.

Development Host Requirements

MATLAB Coder for C++ code generation. For a tutorial, see Get Started with MATLAB Coder.

Connect the Host Computer to NVIDIA Jetson

The MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms uses an SSH connection over TCP/IP to execute commands while building and running the generated code on the Jetson platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see the NVIDIA documentation.

Create Jetson Object

To communicate with the NVIDIA hardware, create a Jetson hardware connection object by using the jetson function.

hwobj = jetson('jetson-deviceaddress','username','password');

When connecting to the target board for the first time, you must provide the host name or IP address, user name, and password of the target board. On subsequent connections, you do not need to supply the address, user name, and password. The Jetson hardware connection object reuses these settings from the most recent successful connection to an NVIDIA board.

By default, this example reuses the settings from the most recent successful connection to a NVIDIA Jetson board.

hwobj = jetson;

Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Gathering hardware details is complete.
 Board name              : NVIDIA Jetson Orin Nano Developer Kit
 CUDA Version            : 11.4
 cuDNN Version           : 8.6
 TensorRT Version        : 8.5
 GStreamer Version       : 1.16.3
 V4L2 Version            : 1.18.0-2build1
 SDL Version             : 1.2
 OpenCV Version          : 4.5.4
 Available Webcams       : C270 HD WEBCAM
 Available GPUs          : Orin
 Available Digital Pins  : 7  11  12  13  15  16  18  19  21  22  23  24  26  29  31  32  33  35  36  37  38  40

When you create the Jetson hardware connection object, the support package performs hardware and software checks, installs the MATLAB I/O server on the target board, and gathers information on peripheral devices connected to the target. The MATLAB Command Window displays this information. In case of a connection failure, a diagnostics error message is reported at the MATLAB command line. If the connection fails, the most likely cause is an incorrect IP address or host name.

If you have multiple Jetson hardware, the code generator performs a remote build on the target board for which the most recent Jetson hardware connection object was created. To choose a hardware board for performing a remote build, use the setupCodegenContext() method of the respective Jetson hardware connection object. If you create only one Jetson hardware connection object, you do not need to call this method.

Identify Cameras Connected to Target Board

To find the list of cameras connected to the target, use the getCameraList function. If this function outputs an empty table, reconnect the camera and execute the function again.

camlist = getCameraList(hwobj);

      Camera Name       Video Device     Available Resolutions    Pixel Formats
    ________________    _____________    _____________________    _____________

    "C270 HD WEBCAM"    "/dev/video2"    "(View resolutions)"      "YUYV,MJPG"

Select the first camera from the list of cameras and storing the required camera resolution.

camName = table2array(camlist(1,"Camera Name"));
camResolution = [1280 720];

The `sobelEdgeDetection` Entry-Point Function

The sobelEdgeDetection.m entry-point function implements an algorithm that captures live images from camera connected to the hardware board, applies edge detection algorithm, and displays the result on the monitor connected to the hardware board. The algorithm consists of a 3-by-3 Sobel operator that the function applies to the image in horizontal and vertical directions and then threshold against a constant value.

type sobelEdgeDetection.m

function sobelEdgeDetection(cameraName,resolution) %#codegen
%SOBELEDGEDETECTION() Entry-point function for Sobel edge detection
%   This function is the entry-point function that supports examples in
%   MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE 
%   Platforms that use Sobel algorithms for edge detection.

%   Copyright 2023 The MathWorks, Inc.

hwobj = jetson;
camObj = camera(hwobj,cameraName,resolution);
dispObj = imageDisplay(hwobj);

% Sobel kernel
kern = [1 2 1; 0 0 0; -1 -2 -1];

% Main loop
for k = 1:1000
    % Capture the image from the camera on hardware.
    img = snapshot(camObj);
    
    % Finding horizontal and vertical gradients.
    h = conv2(img(:,:,2),kern,'same');
    v = conv2(img(:,:,2),kern','same');
    
    % Finding magnitude of the gradients.
    e = sqrt(h.*h + v.*v);
    
    % Threshold the edges
    edgeImg = uint8((e > 100) * 240);
    
    % Display image.
    image(dispObj,edgeImg');
end

end

Generate C++ Code for the Jetson Target Using MATLAB Coder

To generate an executable that you can deploy on to a NVIDIA target, create a code generation configuration object

cfg = coder.config('exe');

Specify the language standard to use for code generation, and set the TargetLang property of the code configuration object.

cfg.TargetLang = "C++";

Create a configuration object for the Jetson platform and assign it to the hardware property of the code configuration object.

cfg.Hardware = coder.hardware('NVIDIA Jetson');

Use use the BuildDir property to specify the folder for performing remote build process on the target board. If the specified build folder does not exist on the target board, then the software creates a folder with the given name. If you do not assign a value to cfg.Hardware.BuildDir, the remote build process occurs in the last specified build folder. If there is no stored build folder value, the build process takes place in the home folder.

cfg.Hardware.BuildDir = '~/remoteBuildDir';

Set the GenerateExampleMain property to generate an example C++ main file and compile it. This example does not require modifications to the generated main files.

cfg.GenerateExampleMain = 'GenerateCodeAndCompile';

To enable parallel processing of parfor-loops and utilize all ARM cores on your NVIDIA Jetson board, set the EnableOpenMP property of the code configuration object to true.

cfg.EnableOpenMP = true;

To generate C++ code, use the codegen function and pass the code configuration object and the input specifications for the sobelEdgeDetection.m entry-point function. After the code generation takes place on the host, MATLAB copies the generated files on the target and then builds them.

inputArgs = {coder.Constant(camName),coder.Constant(camResolution)};
codegen('-config ',cfg,'-args',inputArgs,'sobelEdgeDetection','-report');

Code generation successful: View report

Run Sobel Edge Detection on the Target Board

To run the generated executable on the target board, use the runApplication function.

Set the appropriate display environment.

hwobj.setDisplayEnvironment('1.0');

Run the application on the target.

pid = runApplication(hwobj,'sobelEdgeDetection');

### Launching the executable on the target...
Executable launched successfully with process ID 313852.
Displaying the simple runtime log for the executable...

Note: For the complete log, run the following command in the MATLAB command window:
system(hwobj,'cat /home/ubuntu/remoteBuildDir/MATLAB_ws/R2024a/home/npatil/Documents/MATLAB/ExampleManager/npatil.Bdoc24a.j2483507.g3196700/nvidia-ex93774442/sobelEdgeDetection.log')

A window opens on the target hardware display and shows the Sobel edge detection output of the live camera feed.

Stop the Application

To stop the sobelEdgeDetection application on the board, use killApplication function.

hwobj.killApplication('sobelEdgeDetection');