Camera calibration is the process of computing the extrinsic and intrinsic parameters of a camera. Once you calibrate a camera, you can use the image information to recover 3-D information from 2-D images. You can also undistort images taken with a fisheye camera.
Fisheye cameras are used in odometry and to solve the simultaneous localization and mapping (SLAM) problems visually. Other applications include, surveillance systems, GoPro, virtual reality (VR) to capture 360 degree field of view (fov), and stitching algorithms. These cameras use a complex series of lenses to enlarge the camera's field of view, enabling it to capture wide panoramic or hemispherical images. However, the lenses achieve this extremely wide angle view by distorting the lines of perspective in the images
Because of the extreme distortion a fisheye lens produces, the pinhole model cannot model a fisheye camera.
The Computer Vision Toolbox™ calibration algorithm uses the fisheye camera model proposed by Scaramuzza. You can use this model with cameras up to a field of view (FOV) of 150 degrees. The model uses an omnidirectional camera model. The process treats the imaging system as a compact system. In order to relate a 3-D world point on to a 2-D image, you must obtain the camera extrinsic and intrinsic parameters. World points are transformed to camera coordinates using the extrinsics parameters. The camera coordinates are mapped into the image plane using the intrinsics parameters.
The extrinsic parameters consist of a rotation, R, and a translation, t. The origin of the camera's coordinate system is at its optical center and its x- and y-axis define the image plane.
The transformation from world points to camera points is:
For the fisheye camera model, the intrinsic parameters include the polynomial mapping coefficients of the projection function. The alignment coefficients are related to sensor alignment and the transformation from the sensor plane to a pixel location in the camera image plane.
The following equation maps an image point into its corresponding 3-D vector.
are the ideal image projections of the real-world points.
represents a scalar factor.
are polynomial coefficents described by the Scaramuzza model, where .
is a function of (u,v) and depends only on the distance of a point from the image center: .
The intrinsic parameters also account for stretching and distortion. The stretch matrix compensates for the sensor-to-lens misalignment, and the distortion vector adjusts the (0,0) location of the image plane.
The following equation relates the real distorted coordinates (u'',v'') to the ideal distorted coordinates (u,v).
To remove lens distortion from a fisheye image, you can detect a checkerboard
calibration pattern and then calibrate the camera. You can find the checkerboard points
generateCheckerboardPoints functions. The
estimateFisheyeParameters function uses the detected points and returns
fisheyeParameters object that contains the intrinsic and extrinsic
parameters of a fisheye camera. You can use the
fisheyeCalibrationErrors to check the accuracy of the calibration.
Remove lens distortion from a fisheye image by detecting a checkboard calibration pattern and calibrating the camera. Then, display the results.
Gather a set of checkerboard calibration images.
images = imageDatastore(fullfile(toolboxdir('vision'),'visiondata', ... 'calibration','gopro'));
Detect the calibration pattern from the images.
[imagePoints,boardSize] = detectCheckerboardPoints(images.Files);
Generate world coordinates for the corners of the checkerboard squares.
squareSize = 29; % millimeters worldPoints = generateCheckerboardPoints(boardSize,squareSize);
Estimate the fisheye camera calibration parameters based on the image and world points. Use the first image to get the image size.
I = readimage(images,1); imageSize = [size(I,1) size(I,2)]; params = estimateFisheyeParameters(imagePoints,worldPoints,imageSize);
Remove lens distortion from the first image
I and display the results.
J1 = undistortFisheyeImage(I,params.Intrinsics); figure imshowpair(I,J1,'montage') title('Original Image (left) vs. Corrected Image (right)')
J2 = undistortFisheyeImage(I,params.Intrinsics,'OutputView','full'); figure imshow(J2) title('Full Output View')
 Scaramuzza, D., A. Martinelli, and R. Siegwart. "A Toolbox for Easy Calibrating Omnidirectional Cameras." Proceedings to IEEE International Conference on Intelligent Robots and Systems, (IROS). Beijing, China, October 7–15, 2006.