How to render an accurate image of a 3D model

I want to create an accurate image of a 3D model as if it were seen by a camera. I am using functions such as "plot3" but the rendered points in the image are a little displaced from where I would predict. The simplest example is to try to create an image of a single point, as seen by a camera using a simple pinhole camera model.

After running the code below, it renders an image of a point located at (x,y) = (416,248). However, I predict that the point should appear at (x,y) = ( 450.5,256). Any ideas?

% Create a window to hold an image of size 512x512 pixels
figure('Position',[1 1 512 512]); 

% Define the XYZ location of a point with respect to camera
P = [1; 0; 10];   

% This creates the rendered image of the point
plot3(P(1),P(2),P(3),'.');

fov = 15;   % Define field of view of the camera in degrees
camva(15);  % Set the camera field of view 

campos([0 0 0]);  % Put the camera at the origin
camtarget([0 0 1]);  % The camera looks along the +Z axis
camproj('perspective');

axis off
axis vis3d
daspect([1 1 1])

F = getframe(gcf);      % Grab the rendered frame
I=rgb2gray(F.cdata);    % This is the rendered image

figure, imshow(I,[]), impixelinfo;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Calculate where I expect the point to appear

% Calculate effective focal length using pinhole camera model.
focal = 256/tand(fov/2);

% Use perspective projection equations to predict image point (x,y)
x = focal * P(1)/P(3) + 256
y = focal * P(2)/P(3) + 256

Answer 1

[William Hoff]

Hi - I finally found the answer to my own question (actually, my student found it). The code I originally had was almost right, but there are two important things you must do:

1. After creating the plot, call the function "set(gca, 'Units', 'pixels', 'Position', [1 1 512 512]);". Here, (512,512) was the size of my image.
2. When you predict the location of your 3D point P, remember that the center of an image with an even number of row and columns lies in a fractional position. For an image of size 512x512, the center is at (256.5,256.5). So you should predict the point location using: x = focal * P(1)/P(3) + 256.5; y = focal * P(2)/P(3) + 256.5;

The following is the modified code, that correctly renders a 3D point using "plot3", and it lies in the expected location in the image:

    % Create a window to hold an image of size 512x512 pixels
    figure('Position',[100 100 512 512]);

    % Define the XYZ location of a point with respect to camera
    X = 2*(rand-0.5);   % Generate random location in X, -1..+1
    Y = 2*(rand-0.5);   % Generate random location in Y, -1..+1
    P = [X; Y; 10];

    % This creates the rendered image of the point
    plot3(P(1),P(2),P(3),'.');

    fov = 15;   % Define field of view of the camera in degrees
    camva(15);  % Set the camera field of view

    campos([0 0 0]);  % Put the camera at the origin
    camtarget([0 0 1]);  % The camera looks along the +Z axis
    camproj('perspective');

    axis image
    axis off;
    set(gca, 'Units', 'pixels', 'Position', [1 1 512 512]);
    set(gcf, 'Color', [1 1 1]); % Sets figure background (optional)

    F = getframe(gcf);      % Grab the rendered frame
    I=rgb2gray(F.cdata);    % This is the rendered image

    figure, imshow(I,[]), impixelinfo;

    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    % Detect the centroid of the point in the rendered image
    region = regionprops(I<255);
    disp('Point detected at: '), disp(region.Centroid);

    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    % Calculate where I expect the point to appear

    % Calculate effective focal length using pinhole camera model.
    focal = 256/tand(fov/2);

    % Use perspective projection equations to predict image point (x,y)
    disp('Point should be at: ');
    x = focal * P(1)/P(3) + 256.5
    y = focal * P(2)/P(3) + 256.5

[sadrazamintasagi]

Helle Mr. Hoff,

I am trying to make something like you did in your code. I runned your code but the predicted coordinates are still different than the detected centroid in rendered image. Also, the rendered image (I) has resolution of 640x640 instead 512x512. I think this resolution difference causes the misplacement of pixels. Do you have any solution for this situation.

Have a nice day.

Answer 2

[John Barber]

A few thoughts:

1) I concur with Walter's point about getting the actual figure position. You should also be familiar with a number of axes properties - see the MATLAB documentation for 'Axes Properties' for details.

2) For absolute pixel accuracy, be aware that the drawn position of a particular plot object is quantized into integer pixels. I have noticed occasional offsets of 1 pixel between a drawn point and a tick mark at the exact same location. This behavior may be different for different settings of the figure's 'Renderer' property.

3) If you really want to get a peek behind the curtain, the following undocumented axes properties will be of interest:

• XForm
• x_RenderTransform
• x_NormRenderTransform
• x_ViewTransform
• x_ViewPortTransform
• x_RenderScale
• x_RenderOffset
• x_ProjectionTransform

The code in the following MATLAB functions will help with understanding the above properties:

• viewmtx
• legendcolorbarlayout (see the 'topixels' subfunction)

[Yair Altman]

Re XForm, see the following: http://www.mathworks.com/matlabcentral/newsreader/view_thread/253563

Answer 3

[Walter Roberson]

When you use

figure('Position',[1 1 512 512]);

the figure might not actually be placed at the location you expect, and might not be the size you expect. MATLAB (or Java, or something) displaces the figure in order to allow operating system margins such as the Windows Task Bar, and might end up making it a bit smaller.

You should get() the figure position to see what you really ended up with; likewise you should check the actual image size that you obtained through getframe() as MATLAB has been known to grab slightly different sizes (for reasons that are as yet unclear.)