ROI in video using the first frame

Hi,

I would like to specify the box on the first frame of a video and then have all subsequent frames use that same box location.

I have a for loop here but unsure where should add this part in. Thank you!

for frame = 1 : numberOfFrames

% Extract the frame from the movie structure.

thisFrame = read(videoObject, frame);

thisFrame = thisFrame(557:702,70:216, :, :);

% Display it

hImage = subplot(1, 2, 1);

image(thisFrame);

axis image;

ongoing = frame/framerate;

total = numberOfFrames/framerate;

caption = sprintf('Time %.3f of %.3f sec.', ongoing, total);

title(caption, 'FontSize', 20);

drawnow; % Force it to refresh the window.

[Kevin Holly]

Are you trying to crop a video to a rectangular region of interest (ROI)? Is the ROI defined by the user or based on the image?

[Shu-An Hsieh]

I am trying to crop a circle on my video. The ROI will be based on the user.

Answer 1

[Kevin Holly]

Read video and have user define Circle ROI:

v = VideoReader('xylophone.mp4');
frame = readFrame(v);
imshow(frame)
c = drawcircle;

After Circle ROI is define, you could have a pushbutton callback that creates a mask and plays video:

 mask = createMask(c);

while hasFrame(v)
frame = readFrame(v);
imshow(uint8(mask).*frame)
end

[Shu-An Hsieh]

Hi, thank you for your suggestions. I did successfully get the circle mask, however, it is not inplemented into my video. Can you provide a bit more instruction on how I should preceed this? Thank you.

[Kevin Holly]

v = VideoReader('xylophone.mp4');
frame = readFrame(v);
size(frame)

ans = 1×3

240 320 3

Here the frame is a 240x320x3 matrix that represents the image in one frame of the video.

imshow(frame)

Everytime readFrame is called, the next frame is read.

figure
frame = readFrame(v);
frame = readFrame(v);
frame = readFrame(v);
frame = readFrame(v);
imshow(frame)

Loading mask

load('mask.mat')
size(mask)

ans = 1×2

240 320

After getting the mask, I multiplied the binary matrix to the color image. In this case, the frame was an 8-bit signed integer array. So, I used unint8() to convert the logical array (mask) to an 8-bit signed integer array.

imshow(uint8(mask).*frame)

[Shu-An Hsieh]

Thank you for your detailed explination! I am wondering if I can get a video out of these output frames.

[Kevin Holly]

mask = createMask(c);

v2 = VideoWriter('newfile.avi'); % Create new video file
open(v2) % open it
while hasFrame(v)
    frame = readFrame(v);
A=uint8(mask).*frame;
imshow(A)
writeVideo(v2,A) % Write frame to video file
end
close(v2) % close it

[Shu-An Hsieh]

frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);

ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
while hasFrame(videoObject)
    frame = readFrame(videoObject);
ROI_frames=uint8(mask).*frame;
imshow(ROI_frames)
writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)

for frame = 1 : numberOfFrames
	% Extract the frame from the movie structure.
    thisFrame = read(ROI_movie, ROI_frames);

I tried to incorporate this into my code, it gave me an error on the last line I showed here. I am wondering if you can help me with this. Thank you so much for your time.

[Shu-An Hsieh]

Also, the new video that creates only darken the other places except from the ROI. I am wondering if there is anyway that I can get just the ROI region for the video. In other words, the video will be a circle.

[Walter Roberson]

No, there are no video file formats that I know of that can encode non-rectangular areas.

There are some video file formats that support Alpha data, which would allow you to say that everything outside of the mask should be transparent. However MATLAB does not support those. (I do not recall at the moment whether MATLAB supports transparent animated GIF)

What would be possible is to crop down to the inside of the bounding box of the circle.

[Kevin Holly]

frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);
ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
while hasFrame(videoObject)
    frame = readFrame(videoObject);
ROI_frames=uint8(mask).*frame;
imshow(ROI_frames)
writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)

What are you trying to do below? And what error are you getting?

for frame = 1 : numberOfFrames
	% Extract the frame from the movie structure.
    thisFrame = read(ROI_movie, ROI_frames);
    

'm about to attend a meeting, so it may be a while before I can fully answer you, but you can use regionprops to get the bounding box around your circular ROI. You can use this to crop your image. I may have time to look at this again in a few hours.

rp=regionprops(mask)
rp.BoundingBox % You can use these coordinates to crop your image.

[Shu-An Hsieh]

Thank you Walter! I think cropping down the frame would work for me. Could you provide more information on this? Thank you!

[Shu-An Hsieh]

Thank you Kevin! I really appreciate your help! I am also thinking if it is possible (or easier) just to chose the ROI and I can analyze the data in the ROI. I saw the video labeling tool has a function that I can chose ROI and I can still see the whole frame.

[Walter Roberson]

mask = createMask(c);
hprof = any(mask, 1);
vprof = any(mask, 2);
firstrow = find(vprof, 1);
lastrow = find(vprof, 1,'last');
firstcol = find(hprof, 1);
lastcol =find(hprof, 1, 'last');
mask = mask(firstrow:lastrow, firstcol:lastcol);

Now as you get a frame extract firstrow:lastrow, firstcol:lastcol, :)

and multiply by the mask and write that.

[Kevin Holly]

filename = 'xylophone.mp4';

videoObject = VideoReader(filename);
% Determine how many frames there are.
numberOfFrames = videoObject.NumFrames;
% Determine the frame rate.
framerate = videoObject.FrameRate;
% Determine the whole video length.
sec_total = videoObject.Duration;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;
numberOfFramesWritten = 0;
% Prepare a figure to show the images in the upper half of the screen.
figure
% 	screenSize = get(0, 'ScreenSize');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);
rp = regionprops(mask);
rp.BoundingBox = round(rp.BoundingBox);

ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
while hasFrame(videoObject)
    frame = readFrame(videoObject);
    ROI_frames=uint8(mask).*frame;
    imshow(ROI_frames)
    writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)


videoObject = VideoReader('newfile.avi');
numberOfFrames = videoObject.NumFrames; % Added this because new file has different number of frames
% Loop through the movie, writing all frames out.
% Each frame will be in a separate file with unique name.
meanGrayLevels = zeros(numberOfFrames, 1);
meanRedLevels = zeros(numberOfFrames, 1);
meanGreenLevels = zeros(numberOfFrames, 1);
meanBlueLevels = zeros(numberOfFrames, 1);
for frame = 1 : numberOfFrames
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, frame);

    % Crop to ROI
    thisFrame = thisFrame(rp.BoundingBox(2):rp.BoundingBox(4)+rp.BoundingBox(2),rp.BoundingBox(1):rp.BoundingBox(3)+rp.BoundingBox(1), :, :);

    % Display it557:702,70:216
    hImage = subplot(1, 2, 1);
    image(thisFrame);
    axis image;
    ongoing = frame/framerate;
    total = numberOfFrames/framerate;
    caption = sprintf('Time %.3f of %.3f sec.', ongoing, total);
    title(caption, 'FontSize', 20);
    drawnow; % Force it to refresh the window.

    % Calculate the mean gray level.
    %grayImage = rgb2gray(thisFrame);
    %meanGrayLevels(frame) = mean(grayImage(:));

    % Calculate the mean R, G, and B levels.
    meanRedLevels(frame) = mean(mean(thisFrame(:, :, 1)));
    meanGreenLevels(frame) = mean(mean(thisFrame(:, :, 2)));
    meanBlueLevels(frame) = mean(mean(thisFrame(:, :, 3)));

    % Plot the mean gray levels.
    hPlot = subplot(12, 2,[2 6]);
    hold off;
    %plot(meanGrayLevels, 'k-', 'LineWidth', 3);
    %hold on;
    plot(meanRedLevels, 'r-', 'LineWidth', 2);
    hold on
    plot(meanGreenLevels, 'g-', 'LineWidth', 2);
    plot(meanBlueLevels, 'b-', 'LineWidth', 2);
    grid on;
    %set x labels
    xlabel('Time (sec)');
    % Put title back because plot() erases the existing title.
    title('RGB chromatogram', 'FontSize', 20);
    if frame == 1
        xlabel('Frame Number');
        ylabel('RGB chromatogram')
        % Get size data later for preallocation if we read
        % the movie back in from disk.
        [rows, columns, numberOfColorChannels] = size(thisFrame);
    end
end
% Alert user that we're done.		finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFrames, filename);
disp(finishedMessage); % Write to command window.
uiwait(msgbox(finishedMessage)); % Also pop up a message box.

[Shu-An Hsieh]

Thank you! Kevin! This is very close to what I originally wanted. I am looking at the code and wondering if the RGB is calculated based on the ROI or the square that showed in the later on region.

[Shu-An Hsieh]

Also, why would the frame -1 after the chosing the ROI? Appreciated your help!

[Kevin Holly]

I fixed it, so that it takes the mean of only the ROI.

filename = 'xylophone.mp4';

videoObject = VideoReader(filename);
% Determine how many frames there are.
numberOfFrames = videoObject.NumFrames;
% Determine the frame rate.
framerate = videoObject.FrameRate;
% Determine the whole video length.
sec_total = videoObject.Duration;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;
numberOfFramesWritten = 0;
% Prepare a figure to show the images in the upper half of the screen.
figure
% 	screenSize = get(0, 'ScreenSize');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);
rp = regionprops(mask);
rp.BoundingBox = round(rp.BoundingBox);

ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
while hasFrame(videoObject)
    frame = readFrame(videoObject);
    ROI_frames=uint8(mask).*frame;
    imshow(ROI_frames)
    writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)


videoObject = VideoReader('newfile.avi');
numberOfFrames = videoObject.NumFrames; % Added this because new file has different number of frames
% Loop through the movie, writing all frames out.
% Each frame will be in a separate file with unique name.
meanGrayLevels = zeros(numberOfFrames, 1);
meanRedLevels = zeros(numberOfFrames, 1);
meanGreenLevels = zeros(numberOfFrames, 1);
meanBlueLevels = zeros(numberOfFrames, 1);

for frame = 1 : numberOfFrames
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, frame);

    % Crop to ROI
    thisFrame = thisFrame(rp.BoundingBox(2):rp.BoundingBox(4)+rp.BoundingBox(2),rp.BoundingBox(1):rp.BoundingBox(3)+rp.BoundingBox(1), :, :);

    % Display it557:702,70:216
    hImage = subplot(1, 2, 1);
    image(thisFrame);
    axis image;
    ongoing = frame/framerate;
    total = numberOfFrames/framerate;
    caption = sprintf('Time %.3f of %.3f sec.', ongoing, total);
    title(caption, 'FontSize', 20);
    drawnow; % Force it to refresh the window.

    % Calculate the mean gray level.
    %grayImage = rgb2gray(thisFrame);
    %meanGrayLevels(frame) = mean(grayImage(:));

    % Total Mask Pixels
    Total_Pixels = sum(sum(mask));

    % Calculate the mean R, G, and B levels.
    meanRedLevels(frame) = sum(sum(thisFrame(:, :, 1)))/Total_Pixels;
    meanGreenLevels(frame) = sum(sum(thisFrame(:, :, 2)))/Total_Pixels;
    meanBlueLevels(frame) = sum(sum(thisFrame(:, :, 3)))/Total_Pixels;

    % Plot the mean gray levels.
    hPlot = subplot(12, 2,[2 6]);
    hold off;
    %plot(meanGrayLevels, 'k-', 'LineWidth', 3);
    %hold on;
    plot(meanRedLevels, 'r-', 'LineWidth', 2);
    hold on
    plot(meanGreenLevels, 'g-', 'LineWidth', 2);
    plot(meanBlueLevels, 'b-', 'LineWidth', 2);
    grid on;
    %set x labels
    xlabel('Time (sec)');
    % Put title back because plot() erases the existing title.
    title('RGB chromatogram', 'FontSize', 20);
    if frame == 1
        xlabel('Frame Number');
        ylabel('RGB chromatogram')
        % Get size data later for preallocation if we read
        % the movie back in from disk.
        [rows, columns, numberOfColorChannels] = size(thisFrame);
    end
end
% Alert user that we're done.		finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFrames, filename);
disp(finishedMessage); % Write to command window.
uiwait(msgbox(finishedMessage)); % Also pop up a message box.

[Kevin Holly]

First frame wasn't being saved. Changed code to fix that as well.

filename = 'xylophone.mp4';

videoObject = VideoReader(filename);
% Determine how many frames there are.
numberOfFrames = videoObject.NumFrames;
% Determine the frame rate.
framerate = videoObject.FrameRate;
% Determine the whole video length.
sec_total = videoObject.Duration;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;
numberOfFramesWritten = 0;
% Prepare a figure to show the images in the upper half of the screen.
figure
% 	screenSize = get(0, 'ScreenSize');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);
rp = regionprops(mask);
rp.BoundingBox = round(rp.BoundingBox);

ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
ROI_frames=uint8(mask).*frame; % Save first frame
imshow(ROI_frames)
writeVideo(ROI_movie, ROI_frames)

while hasFrame(videoObject)
    frame = readFrame(videoObject);
    ROI_frames=uint8(mask).*frame;
    imshow(ROI_frames)
    writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)


videoObject = VideoReader('newfile.avi');
numberOfFrames = videoObject.NumFrames; % Added this because new file has different number of frames
% Loop through the movie, writing all frames out.
% Each frame will be in a separate file with unique name.
meanGrayLevels = zeros(numberOfFrames, 1);
meanRedLevels = zeros(numberOfFrames, 1);
meanGreenLevels = zeros(numberOfFrames, 1);
meanBlueLevels = zeros(numberOfFrames, 1);

for frame = 1 : numberOfFrames
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, frame);

    % Crop to ROI
    thisFrame = thisFrame(rp.BoundingBox(2):rp.BoundingBox(4)+rp.BoundingBox(2),rp.BoundingBox(1):rp.BoundingBox(3)+rp.BoundingBox(1), :, :);

    % Display it557:702,70:216
    hImage = subplot(1, 2, 1);
    image(thisFrame);
    axis image;
    ongoing = frame/framerate;
    total = numberOfFrames/framerate;
    caption = sprintf('Time %.3f of %.3f sec.', ongoing, total);
    title(caption, 'FontSize', 20);
    drawnow; % Force it to refresh the window.

    % Calculate the mean gray level.
    %grayImage = rgb2gray(thisFrame);
    %meanGrayLevels(frame) = mean(grayImage(:));

    % Total Mask Pixels
    Total_Pixels = sum(sum(mask));

    % Calculate the mean R, G, and B levels.
    meanRedLevels(frame) = sum(sum(thisFrame(:, :, 1)))/Total_Pixels;
    meanGreenLevels(frame) = sum(sum(thisFrame(:, :, 2)))/Total_Pixels;
    meanBlueLevels(frame) = sum(sum(thisFrame(:, :, 3)))/Total_Pixels;

    % Plot the mean gray levels.
    hPlot = subplot(12, 2,[2 6]);
    hold off;
    %plot(meanGrayLevels, 'k-', 'LineWidth', 3);
    %hold on;
    plot(meanRedLevels, 'r-', 'LineWidth', 2);
    hold on
    plot(meanGreenLevels, 'g-', 'LineWidth', 2);
    plot(meanBlueLevels, 'b-', 'LineWidth', 2);
    grid on;
    %set x labels
    xlabel('Time (sec)');
    % Put title back because plot() erases the existing title.
    title('RGB chromatogram', 'FontSize', 20);
    if frame == 1
        xlabel('Frame Number');
        ylabel('RGB chromatogram')
        % Get size data later for preallocation if we read
        % the movie back in from disk.
        [rows, columns, numberOfColorChannels] = size(thisFrame);
    end
end
% Alert user that we're done.		finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFrames, filename);
disp(finishedMessage); % Write to command window.
uiwait(msgbox(finishedMessage)); % Also pop up a message box.

[Shu-An Hsieh]

Thank you so much Kevin! I have a few more issues with this code. I am wondering if you can guide me on changing the x axis scale on the right image showed. It is showing as frame but not second right now. I saw some functions to use with xtickslabels however you will need to change the labels manually one by one instead of adjusted with the video.

In addition, I am trying to get the area of the RGB value being plotted. I am wondering if that is something we can do with this code. Thank you so much!

[Kevin Holly]

For the x axis, you can add Time as your indepent variable. You can define the time based on your framerate as such:

Time = (1:length(meanRedLevels))/framerate;
plot(Time,meanRedLevels, 'r-', 'LineWidth', 2);

When you say area, are you talking about area under the curve for each plotted line or are you talking about the pixel area of the ROI? For the ROI pixel area, you can use regionprops. For your code, you can view it with

rp.Area

For the area under the curve, you could use the trapz function.

[Shu-An Hsieh]

Thank you! I am thinking about the area under the curve.

[Kevin Holly]

filename = 'xylophone.mp4';

videoObject = VideoReader(filename);
% Determine how many frames there are.
numberOfFrames = videoObject.NumFrames;
% Determine the frame rate.
framerate = videoObject.FrameRate;
% Determine the whole video length.
sec_total = videoObject.Duration;
vidHeight = videoObject.Height;
vidWidth = videoObject.Width;
numberOfFramesWritten = 0;
% Prepare a figure to show the images in the upper half of the screen.
figure
% 	screenSize = get(0, 'ScreenSize');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
frame = readFrame(videoObject);
imshow(frame)
c = drawcircle;
mask = createMask(c);
rp = regionprops(mask);
rp.BoundingBox = round(rp.BoundingBox);

ROI_movie = VideoWriter('newfile.avi');
open(ROI_movie)
ROI_frames=uint8(mask).*frame; % Save first frame
imshow(ROI_frames)
writeVideo(ROI_movie, ROI_frames)

while hasFrame(videoObject)
    frame = readFrame(videoObject);
    ROI_frames=uint8(mask).*frame;
    imshow(ROI_frames)
    writeVideo(ROI_movie, ROI_frames)
end
close(ROI_movie)


videoObject = VideoReader('newfile.avi');
numberOfFrames = videoObject.NumFrames; % Added this because new file has different number of frames
% Loop through the movie, writing all frames out.
% Each frame will be in a separate file with unique name.
meanGrayLevels = zeros(numberOfFrames, 1);
meanRedLevels = zeros(numberOfFrames, 1);
meanGreenLevels = zeros(numberOfFrames, 1);
meanBlueLevels = zeros(numberOfFrames, 1);

for frame = 1 : numberOfFrames
    % Extract the frame from the movie structure.
    thisFrame = read(videoObject, frame);

    % Crop to ROI
    thisFrame = thisFrame(rp.BoundingBox(2):rp.BoundingBox(4)+rp.BoundingBox(2),rp.BoundingBox(1):rp.BoundingBox(3)+rp.BoundingBox(1), :, :);

    % Display it557:702,70:216
    hImage = subplot(1, 2, 1);
    image(thisFrame);
    axis image;
    ongoing = frame/framerate;
    total = numberOfFrames/framerate;
    caption = sprintf('Time %.3f of %.3f sec.', ongoing, total);
    title(caption, 'FontSize', 20);
    drawnow; % Force it to refresh the window.

    % Calculate the mean gray level.
    %grayImage = rgb2gray(thisFrame);
    %meanGrayLevels(frame) = mean(grayImage(:));

    % Total Mask Pixels
    Total_Pixels = sum(sum(mask));

    % Calculate the mean R, G, and B levels.
    meanRedLevels(frame) = sum(sum(thisFrame(:, :, 1)))/Total_Pixels;
    meanGreenLevels(frame) = sum(sum(thisFrame(:, :, 2)))/Total_Pixels;
    meanBlueLevels(frame) = sum(sum(thisFrame(:, :, 3)))/Total_Pixels;

    % Plot the mean gray levels.
    hPlot = subplot(12, 2,[2 6]);
    hold off;
    %plot(meanGrayLevels, 'k-', 'LineWidth', 3);
    %hold on;
    Time = (1:length(meanRedLevels))/framerate;
    plot(Time,meanRedLevels, 'r-', 'LineWidth', 2);
    hold on
    plot(Time,meanGreenLevels, 'g-', 'LineWidth', 2);
    plot(Time,meanBlueLevels, 'b-', 'LineWidth', 2);
    grid on;
    %set x labels
    xlabel('Time (sec)');
    % Put title back because plot() erases the existing title.
    title('RGB chromatogram', 'FontSize', 20);
    if frame == 1
        xlabel('Frame Number');
        ylabel('RGB chromatogram')
        % Get size data later for preallocation if we read
        % the movie back in from disk.
        [rows, columns, numberOfColorChannels] = size(thisFrame);
    end
end

% Area under curve (AUC)
RedAUC = trapz(Time,meanRedLevels);
GreenAUC = trapz(Time,meanGreenLevels);
BlueAUC = trapz(Time,meanBlueLevels);

% Alert user that we're done.		finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFrames, filename);
disp(finishedMessage); % Write to command window.
uiwait(msgbox(finishedMessage)); % Also pop up a message box.

[Shu-An Hsieh]

Thank you Kevin! I am wondering if it is possible the get several ranges and substact a base number out of it. Also, I am wondering if I can further save an additional figure of the right figure when the code ends.

[Shu-An Hsieh]

Actually, I am thinking to the get the area between two points, so like you drag a line between two points and the area above and between those two is the area I wanted. Thank you!