stereoParametersFromOpenCV

Convert stereo camera parameters from OpenCV to MATLAB

Since R2021b

Syntax

stereoParams = stereoParametersFromOpenCV(intrinsicMatrix1,distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2,rotationOfCamera2,translationOfCamera2,imageSize)

stereoParams = stereoParametersFromOpenCV(___,WorldUnits=worldUnits)

Description

stereoParams = stereoParametersFromOpenCV(intrinsicMatrix1,distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2,rotationOfCamera2,translationOfCamera2,imageSize) converts the OpenCV stereo parameters, specified by the input arguments, into a MATLAB^® stereoParameters object stereoParams.

The OpenCV spatial coordinate system specifies the upper-left pixel center at (0,0), whereas the MATLAB spatial coordinate system specifies the pixel center at (1,1). The stereoParametersFromOpenCV function compensates for this difference by adding 1 to both of the x and y-values for the converted principal point.

OpenCV stereo parameters cannot be converted to a MATLAB stereoParameters object when an OpenCV fisheye model is used. In this case, you can recalibrate the stereo camera using the Stereo Camera Calibrator app.

stereoParams = stereoParametersFromOpenCV(___,WorldUnits=worldUnits) specifies a string worldUnits, that describes the units of the world points in addition to the input arguments from the previous syntax. Specify worldUnits as a character vector or string scalar. For example, stereoParametersFromOpenCV( ___ ,WorldUnits="mm") describes all parameters in terms of the world units "mm".

Examples

Rectify Stereo Images in MATLAB Using Stereo Parameters from OpenCV

Load OpenCV stereo parameters from a MAT file into the workspace.

load openCVStereoParameters.mat

Convert the loaded stereo parameters from OpenCV to MATLAB format.

stereoParams = stereoParametersFromOpenCV(intrinsicMatrix1, ...    
    distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2, ...
    rotationOfCamera2,translationOfCamera2,imageSize);

Load the images to rectify.

imageDir = fullfile(toolboxdir('vision'),'visiondata','calibration','stereo');
I1 = imread(fullfile(imageDir,'left','left01.png'));
I2 = imread(fullfile(imageDir,'right','right01.png'));

Display the unrectified images.

imshow(stereoAnaglyph(I1,I2));

Rectify the images using the full output view.

[J1,J2] = rectifyStereoImages(I1,I2,stereoParams,OutputView='full');

Display the resulting rectified images.

imshow(stereoAnaglyph(J1,J2));

Import Stereo Camera Parameters from ROS

The ROS camera calibration package estimates stereo camera parameters using the OpenCV camera calibration tools [1]. After calibrating a stereo camera in ROS, you can export its camera parameters to an INI file using the camera calibration parser. To use the calibrated stereo camera with Computer Vision Toolbox™ functions, such as rectifyStereoImages, you must read the camera parameters from the INI file and convert them into a stereoParameters object using stereoParametersFromOpenCV.

Note: The stereoParametersFromOpenCV function supports importing stereo camera parameters for only those pinhole camera models that use the ROS plumb-bob distortion model.

Read Stereo Camera Parameters from ROS INI File

Read the stereo camera parameters stored in stereoParams.ini using the helper function helperReadINI.

stereoParamsINI = helperReadINI("stereoParams.ini");

Compute Baseline Parameters of Stereo Camera

The baseline parameters of a stereo camera describe the relative translation and rotation of the two cameras in the stereo camera pair. The relative rotation and translation of camera 2 with respect to camera 1 is required to create the stereoParameters object using stereoParametersFromOpenCV. You can compute these from the rectification and projection matrices read from the ROS INI file [2].

Extract the two camera parameters from the stereoParams structure.

cameraParams1 = stereoParamsINI.narrow_stereo_left;
cameraParams2 = stereoParamsINI.narrow_stereo_right;

Extract the translation of camera 2 relative to camera 1 from the last column of the projection matrix.

translationOfCamera2 = cameraParams2.projection(:,end);

The rotation of camera 2 relative to camera 1, ${\mathit{R}}_{21}$, is derived from the rectification matrices of the stereo pair ${\mathit{R}}_1$ and ${\mathit{R}}_2$. The rectification matrices are the rotation matrices that align the camera coordinate system to the ideal stereo image plane such that epipolar lines in both stereo images are parallel. Compute the rotation of camera 2 relative to camera 1 as ${\mathit{R}}_{21}$= ${\mathit{R}}_2$*${\mathit{R}}_1^{\mathit{T}}$.

rotationOfCamera2 = cameraParams2.rectification*cameraParams1.rectification';

Create stereoParameters Object Using stereoParametersFromOpenCV

Extract the intrinsic matrices and distortion coefficients of the two cameras from the stereoParams structure.

intrinsicMatrix1 = cameraParams1.camera_matrix;
intrinsicMatrix2 = cameraParams2.camera_matrix;

distortionCoefficients1 = cameraParams1.distortion;
distortionCoefficients2 = cameraParams2.distortion;

Obtain the image size from the image field of the stereoParams structure.

imageSize = [stereoParamsINI.image.height stereoParamsINI.image.width];

Use stereoParametersFromOpenCV to create a stereoParameters object from the ROS stereo camera parameters.

stereoParametersObj = stereoParametersFromOpenCV(intrinsicMatrix1, ...
    distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2, ...
    rotationOfCamera2,translationOfCamera2,imageSize);

Rectify Pair of Stereo Images

Use the imported stereo parameters with rectifyStereoImages to rectify an image pair captured using the calibrated stereo camera.

% Load the image pair.
imageDir = fullfile(toolboxdir("vision"),"visiondata","calibration","stereo");
leftImages = imageDatastore(fullfile(imageDir,"left"));
rightImages = imageDatastore(fullfile(imageDir,"right"));
I1 = readimage(leftImages,1);
I2 = readimage(rightImages,1);

% Rectify the image pair.
[J1,J2] = rectifyStereoImages(I1,I2,stereoParametersObj,OutputView="full");

% Display the results.
figure
J = stereoAnaglyph(J1,J2);
imshow(J)

Supporting Functions

helperReadINI

The helperReadINI function reads the camera parameters from its input INI file that has been exported from ROS.

function cameraParams = helperReadINI(filename)
% helperReadINI reads a ROS INI file, filename, and returns a structure with
% these fields: image, <camera_name1>, <camera_name2>. image is a
% structure describing the height and width of the image captured by the
% cameras of the stereo pair. The fields <camera_name1> and <camera_name2>
% are structures named after the camera names present in the INI file, and they contain
% these fields: camera_matrix, distortion, rectification_matrix,
% and projection_matrix. These fields are stored in the INI file with their
% values placed in a new line followed by their name.

    f = fopen(filename,"r");
    sectionName = '';
    
    while ~feof(f)
        % Read line from file.
        line = fgetl(f);

        % Trim leading and trailing whitespaces.
        line = strtrim(line);
        
        if isempty(line) || line(1)=='#'
            % Skip empty line and comments.
            continue
        elseif line(1) == '[' && line(end) == ']'
            % Identify section names and continue reading.
            sectionName = line(2:end-1);
            sectionName = strrep(sectionName,'/','_');
            continue
        end

        % Replace blankspaces with underscores to create valid MATLAB variable
        % name.
        name = line;
        name(name == ' ') = '_';
        
        % Read the value data in upcoming lines.
        value = [];
        while ~feof(f)
            line = fgetl(f);
            line = strtrim(line);

            if isempty(line)
                % A empty line indicates end of value data.
                break
            elseif line(1)=='#'
                % Skip comment lines.
                continue
            end
            line = str2num(line); %#ok
            value = [value; line]; %#ok
        end
    
        % Store post-processed value.
        if isempty(sectionName)
            cameraParams.(name) = value;
        else
            cameraParams.(sectionName).(name) = value;
        end
    end
    
    fclose(f);
end

References

[1] http://wiki.ros.org/camera_calibration

[2] http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/CameraInfo.html

Input Arguments

intrinsicMatrix1 — Intrinsics matrix of camera 1

3-by-3 matrix

Intrinsics matrix of camera 1 from OpenCV, specified as a 3-by-3 matrix of the form:

$$\left[\begin{array}{ccc}fx& 0& cx\\ 0& fy& cy\\ 0& 0& 1\end{array}\right]$$

where fx and fy are the focal lengths in the x and y-directions, and cx,cy) is the principal point in specified in OpenCV.

intrinsicMatrix2 — Intrinsics matrix of camera 2

3-by-3 matrix

Intrinsics matrix of camera 2 from OpenCV, specified as a 3-by-3 matrix of the form:

$$\left[\begin{array}{ccc}fx& 0& cx\\ 0& fy& cy\\ 0& 0& 1\end{array}\right]$$

where fx and fy are the focal lengths in the x and y-directions, and cx,cy is the principal point in specified in OpenCV.

distortionCoefficients1 — Camera 1 radial and tangential distortion coefficients

5-element vector | 8-element vector

Camera 1 radial and tangential distortion coefficients, returned as a 5- or 8-element vector.

• 5-element vector — Of the form [k1 k2 p1 p2 k3]

• 8-element vector — Of the form [k1 k2 p1 p2 k3 k4 k5 k6].

  The values of k1, k2, …, k6 describe the radial distortion and p1 and p2 describe the tangential distortion, specified in OpenCV.

distortionCoefficients2 — Camera 2 radial and tangential distortion coefficients

5-element vector | 8-element vector

Camera 2 radial and tangential distortion coefficients, returned as a 5- or 8-element vector.

• 5-element vector — Of the form [k1 k2 p1 p2 k3]

• 8-element vector — Of the form [k1 k2 p1 p2 k3 k4 k5 k6].

  The values of k1, k2, …, k6 describe the radial distortion and p1 and p2 describe the tangential distortion, specified in OpenCV.

rotationOfCamera2 — Rotation of camera 2 relative to camera 1

3-by-3 matrix

Rotation of camera 2 relative to camera 1 from OpenCV, specified as a 3-by-3 matrix.

translationOfCamera2 — Translation of camera 2 relative to camera 1

three-element vector

Translation of camera 2 relative to camera 1 from OpenCV, specified as a three-element vector.

imageSize — Image size

two-element vector

Image size, specified as a two-element vector in the form [mrows,ncols].

Output Arguments

stereoParams — Stereo parameters

stereoParameters object

Stereo parameters, returned as a stereoParameters object. The object contains OpenCV intrinsics for a pinhole camera model. In cases where the OpenCV pinhole model uses more than eight distortion coefficients, which cannot be converted to a cameraIntrinsics object, you can recalibrate your camera using the Stereo Camera Calibrator app as an alternative solution.

Version History

Introduced in R2021b

R2024b: Import OpenCV pinhole camera model with six radial distortion coefficients

The stereoParametersFromOpenCV has been updated to support OpenCV pinhole camera model with 6 radial distortion coefficients.