Ambiguous Disparity Map and Inadequate 3D Scene Reconstruction
34 views (last 30 days)
Show older comments
YILDIRAY YILMAZ on 3 May 2016
Commented: Pamela Erin Rosario on 22 Feb 2020
I'm trying to measure distance from camera to object using stereo images. I used 40 image pairs for calibration with matlab stereo calibration app. At the result of the calibration, overall mean error was 1.49 pixels. I applied all of the technics which are in the Matlab's depth estimation from stereo video tutorial.
My actual distance from camera to object is 2.97 meters. However, I found 4,32 meters as a result of my program's calculation. I guess there is something wrong in my disparity map and 3d scene reconstruction. Because my disparity map is really ambiguous and point cloud is so inadequate. I would like to get any suggestion regarding this topic. Also i tried to apply some technics such as median and wiener filter to remove noise from images which were written at this topic: http://uk.mathworks.com/matlabcentral/answers/153348-tips-and-tricks-about-3d-scene-reconstruction
These are my outputs:
This is my code:
Read and Rectify Images
imageLeft = imread('D:\stereo\imgPairLeft1.png');
imageRight = imread('D:\stereo\imgPairRight1.png');
imageLeft = undistortImage(imageLeft, stereoParams.CameraParameters1, ...
imageRight = undistortImage(imageRight, stereoParams.CameraParameters2, ...
[imageLeftRect, imageRightRect] = ...
rectifyStereoImages(imageLeft, imageRight, stereoParams);
imageLeftGray = rgb2gray(imageLeftRect);
imageRightGray = rgb2gray(imageRightRect);
imageLeftGrayHisteq = histeq(imageLeftGray);
imageRightGrayHisteq = histeq(imageRightGray);
disparityRange = [0 80];
disparityMap = disparity(imageLeftGrayHisteq, imageRightGrayHisteq, ...
'DisparityRange', disparityRange, 'BlockSize', 15);
Reconstruct the 3-D scence
points3D = reconstructScene(disparityMap, stereoParams);
% Convert to meters and create a pointCloud object
points3D = points3D ./ 1000;
ptCloud = pointCloud(points3D, 'Color', imageLeftRect);
binaryImage = imageLeftGrayHisteq > 0 & imageLeftGrayHisteq < 60;
binaryImage = imfill(binaryImage, 'holes');
% Assign each blob different color
labeledImage = bwlabel(binaryImage, 8);
coloredLabels = label2rgb(labeledImage, 'hsv', 'k', 'shuffle');
blobMeasurements = regionprops(labeledImage, imageLeftGrayHisteq, 'all');
numberOfBlobs = size(blobMeasurements, 1);
% Sort the rows by Area.
[~,index] = sortrows([blobMeasurements.Area].');
blobMeasurements = blobMeasurements(index);
newIndex = sortrows(index);
% After sorting, last index is the largest blob
tvUnitBoundingBox = blobMeasurements(newIndex(end)).BoundingBox;
Determine the distance of tv unit to the camera. Find the centroids of tv unit.
centroid = [round(tvUnitBoundingBox(:,1) + tvUnitBoundingBox(:,3) / 2) ...
round(tvUnitBoundingBox(:,2) + tvUnitBoundingBox(:,4) / 2)];
% Find 3-D world coordinates of the centroids.
centroidsIdx = sub2ind(size(disparityMap), centroid(:,2), centroid(:,1));
X = points3D(:, :, 1);
Y = points3D(:, :, 2);
Z = points3D(:, :, 3);
centroids3D = [X(centroidsIdx)'; Y(centroidsIdx)'; Z(centroidsIdx)'];
% Find the distances from the camera in meters
distanceFromTvUnitToCam = sqrt(sum(centroids3D .^ 2));
% Display the tv unit and its distance.
label = sprintf('%02f meters', distanceFromTvUnitToCam);
imshow(insertObjectAnnotation(imageLeftRect, 'rectangle', tvUnitBoundingBox, label));
Pamela Erin Rosario on 22 Feb 2020
Hi! May i ask if the BoundingBox is only for TV units? or can i use it for other objects to be detected as well?
Dima Lisin on 3 May 2016
Edited: Dima Lisin on 3 May 2016
There are several problems with your calibration:
- The checkerboard is not really flat. You have to glue it to a hard flat surface, like a flat sheet of plastic or glass, which does not bend easily.
- There seems to be a long delay between taking the two images of each stereo pair. If you look at the calibration images, you can see that your face had the time to change expression between the two shots. That means that the checkerboard also moves noticeably in the time between taking of the two images, because a person cannot hold anything perfectly still for any length of time. To fix this, you either have to get a "real" stereo camera with hardware triggering, which takes the two images nearly simultaneously, or you need to lean your checkerboard on something, so that it does not move between the shots.
- You need to have a greater variation of 3D orientations of the checkerboard. In most of your calibration images you are rotating the checkerboard in-plane, but it is always more or less parallel to the image plane. In other words, most of your checkerboards are almost parallel to each other, which does not give you much information for the calibration. You need to slant the checkerboard forward and backward, and left and right.
- Finally, when you calibrate you have to look at the reprojection errors graph, and exclude the images with really high reprojection errors. Ideally, you want your average reprojection error over all the images to be less than .5 of a pixel.
More Answers (1)
Vaddeti Nikitha on 3 Feb 2020
Can I rectify car wheel images of a stereo camera with only cameraparams of the checkerboard images ???
Find more on Stereo Vision in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!