How to rotate objects of a bounding box

Hi everybody,

I have an image with three cards that I've processed to the point where I'm creating bounding boxes for every object detected. My question is, how and what can I use to rotate the objects determined by the bounding boxes so that they are in an upright position, mainly being the outline of the entire card? Here is the code I have so far that is creating the bounding boxes, and the image I have. The image isn't the best, but just so you guys have an idea of what I'm trying do.

 %e
imshow(J)
%stats = regionprops(J)
stats = regionprops(J,'BoundingBox','Area');
AreaOb = regionprops(J,'Area')
PerOB = regionprops(J,'Perimeter')
[B,L] = bwboundaries(J, 'noholes');
figure; imshow(J); hold on;
for k = 1:length(B),
     boundary = B{k};
     plot(boundary(:,2),boundary(:,1),'g','LineWidth',2);
end

[Image Analyst]

Let's say you have an irregularly shaped blob, say one that looks like an asterisk or a splat. What determines "upright"? Do you want the major axis, as determined by the property "Orientation" returned from regionprops(), to be vertical? Or do you have some other definition? And why do you want it upright? Why can't it be analyzed as it is? A picture would be helpful.

[Ryan]

My goal is to create an playing card detection and recognition system. So if the cards are rotated in the image, I wanted to rotate them so they are completely upright, or vertical. Once I do this, I figured I could extract the upper left corner of the card, and compare that to a template image that will be in the same upright position. Atleast, this was the only way I could think of that would work for this. I thought I did include an image, but let me try again if it didn't come up. You should see an image, BoundingBoxImage.png attached

[Ryan]

Here is another image of it, so you have an idea. Just, this one isn't showing the bounding boxes for the outline of the cards

[Image Analyst]

I see only one image.

[Ryan]

They're the same image, I just tried to include one if the other wasn't showing.

Answer 1

[Image Analyst]

Since you seem to have a complete rectangular outline, fill the cards with imfill.

binaryImage = imfill(edgeImage, 'holes');

Then get the orientation with regionprops(). The orientation for an "upright" card of a certain aspect ratio should be some known angle (not 0 or 90 though since the major axis may not align with the card edges). So then, knowing that, you can figure out the angle to rotate the cropped card subimage.

Put regionprops in a for loop where you're using ismember() to extract out each card one at a time, then use imcrop to crop it out. Untested code follows:

[labeledImage, numBlobs] = bwlabel(binaryImage);
for k = 1 : numBlobs
  thisBlob = ismember(labeledImage, k);
  measurements = regionprops(thisBlob, 'Orientation', 'BoundingBox');
  croppedImage = imcrop(rgbImage, measurements.BoundingBox);
  % Compute angle from measurements.Orientation
  angle = .............
  % Rotate image
  uprightImage = imrotate(croppedImage, angle);
end

[Ryan]

Thank you, that will help out me out a lot. The only thing is, if I use the imfill() function, won't that completely get rid of the suit and rank information in the corners of the cards? Is there any way to 'unfill' the card after the rotation?

[Image Analyst]

No. You're filling the edge detection image so we can determine the angle. Then, note how we're cropping the original RGB image, not the binary image, so you're getting the original color image of just one card.

Answer 2

[Ryan]

Image Analyst, is there any way I might be able to send you what I have on my m-file so you could check everything out. I've got all the preprocessing done, but I'm really having a hard time understanding and implementing this part with my code, even with the help you gave me. I know it's asking a lot, but if you could add any additional code to help me solve this, or just add some more notes/advice on how I can do it directly to my code I would appreciate it very much. This is frustrating me more than anything, and I feel like I keep going backwards with everything I try.

[Walter Roberson]

Attach the .m file. Image Analyst does not accept email about Questions.

[Ryan]

Thanks Walter, that would really help me a lot. I've attached two .m files. PlayingCards.m is the .m file where I'm acquiring my live video feed, and performing all my preprocessing. I eventually want to work everything I'm trying to do into this file. The second file, Testing.m, is where I've just been trying to accomplish creating the boundary for the playing card image to be used to rotate, scale, and eventually crop the ROI to compare against a template. The image and .mat file I've included is for this file. Any help you could give me with all this would be greatly appreciated. I really like to get good at all this, but I'm having difficulty teaching everything to myself, and trying to find out what and what not to use.

[Ryan]

Here's the image I've been trying to work with in the Testing.m File as well.

[Ryan]

Did you think you could help at all with any of these files Walter? I'm still having trouble trying to solve this all.

[Image Analyst]

I don't think Walter has the Image Acquisition Toolbox. I do, but I don't know when or if I'll ever get enough time to spend on completing your project for you. It looks like it will take more than 5 minutes, which is usually about all I'll spend on consulting free for someone. Perhaps if you ask smaller, more targeted questions that can be quickly answered.