I can't synchronize lidar and camera together for fusion

This my code. When I move camera detects that am moving then 5 seconds later I start moving in the lidar.

% lidarFig = subplot(1, 2, 2); % Create subplot for lidar data
countinueStreaming = true;
while countinueStreaming
    % Record the current timestamp before capturing the image
    
    % Capture the image from the Jetson camera hardware.
    I = snapshot(cam);
    
    % Record the timestamp after capturing the image
    
    % Call to detect method
    [bboxes, scores, labels] = detect(detector, I);
    personIndices = find(labels == 'person');
    personBboxes = bboxes(personIndices, :);
    
    % Convert categorical labels to cell array of character vectors
    labels = cellstr(labels);
    
    % Annotate detections in the camera image.
    img = insertObjectAnnotation(I, 'rectangle', bboxes, labels);
    
    % Display camera image
    imshow(img);
    hold on;
    
    % Read data from Velodyne Lidar
    pc = read(lidar);
    % Remove ground plane
    groundPtsIndex = segmentGroundFromLidarData(pc, 'ElevationAngleDelta', 0, ...
        'InitialElevationAngle', 0);
    nonGroundPts = select(pc, ~groundPtsIndex);
    imPts = projectLidarPointsOnImage(pc, intrinsics, LidarToCam);
    scatter(imPts(:, 1), imPts(:, 2), 10, 'g', 'filled');
    % Display lidar data
    % pcshow(nonGroundPts.Location, 'Parent', lidarFig);
    
    if ~isempty(personIndices)
        for i = 1:numel(personIndices)
            index = personIndices(i);
            personBbox = bboxes(index, :);
            
            % Process lidar data for each detected person
            [lidarBbox, ~, boxUsed] = bboxCameraToLidar(personBbox, nonGroundPts, intrinsics, ...
                camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]);
            [distance, nearestRect, idx] = helperComputeDistance(personBbox, nonGroundPts, lidarBbox, ...
                intrinsics, LidarToCam);
            
            % Print distance information
            fprintf('DistOfPerson %d: %.2f meters\n', i, distance);
            % Update image with bounding boxes
            % im = updateImage(display, im, nearestRect, distance);
            % updateLidarBbox(display, lidarBbox)
        end
    else
        % Process lidar data for other objects
        [lidarBbox, ~, boxUsed] = bboxCameraToLidar(bboxes, nonGroundPts, intrinsics, ...
            camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]);
        [distance, nearestRect, idx] = helperComputeDistance(bboxes, nonGroundPts, lidarBbox, ...
            intrinsics, LidarToCam);
        
        % Print distance information
        fprintf('DistOfObj: %.2f meters\n', distance);
        % Update image with bounding boxes
        % im = updateImage(display, im, nearestRect, distance);
        % updateLidarBbox(display, lidarBbox)
    end    
    hold off;
end

[Image Analyst]

I don't understand the statement. Can you type it into Google translator

https://translate.google.com/?sl=auto&tl=en&op=translate

in your native language and then paste the translation here.

I don't even see a question. There is no sentence that ends with a question mark.

[Ali sameh]

Look my camera and lidar isn't synchronised. There is a lidar delay of 5 seconds from the camera my question is how can I synchronise lidar and camera together?

Answer 1

[Sanjana]

Hi,

To Synchronize Camera Sensor Data and Lidar Sensor Data, you can use the synchronize function of scenariobuilder sensorData objects.

Kindly, refer to the following code to synchronize LidarData with CameraData,

% Step - 1: Create a scenariobuilder.LidarData object
% timestamps - timestamps of lidarData
% pointClouds - Lidar point cloud data
lidarData = scenariobuilder.LidarData(timestamps,pointClouds)

% Step - 2: Create a scenariobuilder.CameraData object
% timestamps - timestamps of cameraData
% frames - frames of images of cameraData
cameraData = scenariobuilder.CameraData(frames,timestamps);

% Step - 3: Synchronize LidarData with CameraData as reference using the synchronize
% function of LidarData
synchronizedLidarData = synchronize(lidarData,cameraData);

Refer to the following documentation for more information,

https://www.mathworks.com/help/driving/ref/scenariobuilder.cameradata.html

https://www.mathworks.com/help/driving/ref/scenariobuilder.lidardata.html

https://www.mathworks.com/help/driving/ref/scenariobuilder.gpsdata.synchronize.html

Hope this helps!