Is it possible to use the vision toolbox to read specific frames for cutting multiple videos?

Dear community

I've been struggling with this problems for a few days and it's driving me crazy, so I thought to see if you could help.

I want to cut video files (with audio). I need to cut 75 files. I have variables containing the frames and times of start and end of the segments. I use VideoReader to cut the video, and the vision toolbox to attach the audio. The 'read' command comes in really handy. For some video's it works perfectly, for some it doesn't. Using the vision.VideoFileReader function I can't use the 'read command anymore and my code fails. Can anybody help me?

Here is my current code

%% parameters

N = 3; % Number of participants to cut (not used for the moment, but when it works I can make it 75)
directory    = 'D:\Location_Of_Videos\';
prefix.vid   = 'Name_Video_';


%% cut all videos

for k = 1%:N
    load(['' markerDataFolder 'FrameMarker_' num2str(k) '.mat' '']) % Folder with markers of frames
    input = [directory prefix.vid num2str(k) '.mp4']; % Original video
    v = VideoReader(input);
    vidFrameRate = v.FrameRate;

    newVideo = vision.VideoFileWriter(['' 'cutVideo' num2str(k) '.avi' ''],'FileFormat', 'AVI', 'AudioInputPort', true, 'FrameRate', vidFrameRate); % Cut video
    newVideo.VideoCompressor = 'MJPEG Compressor'; % Compress video. For uncompressed copypaste: 'None (uncompressed)' instead of 'MJPEG Compressor'
    newVideo.AudioCompressor = 'None (uncompressed)'; % Audio compressing, not needed
    [audio,fs]=audioread(input); % Original audio
    op=floor(fs/vidFrameRate); % Makes audio frames comparable to video frames
    count=1;
 

    for i = Frame.segmentStart:Frame.segmentEnd % these variables are the frames of the video to cut (loaded in at the beginning of the loop)
        videoFrame = read(v, round(i));
        audios = audio( (i-1)*op + 1 : i*op , : );
        newVideo(videoFrame,audios)
        count=count+1;
    end
    
    
    clear v
    release(newVideo);
end

The error I get is

Error using VideoReader/read>readFramesInIndex (line 280)
Frame index must be a numeric value greater than zero and less than or equal to
the number of frames in the file.

Error in VideoReader/read (line 144)
    videoFrames = readFramesInIndex(obj, varargin{1});

Error in cutVideosJobfit2 (line 72)
        videoFrame = read(v, round(i));

I've tried

load(['' markerDataFolder 'FrameMarker_' num2str(1) '.mat' '']);
input = [directory prefix.vid num2str(1) '.mp4']; % Original video
v = vision.VideoFileReader(input); % Original video
cutVideo = vision.VideoFileWriter('cutVideoTest.avi','FileFormat', 'AVI', 'AudioInputPort', true, 'FrameRate', v.info.VideoFrameRate);
cutVideo.VideoCompressor = 'MJPEG Compressor';
[audio,fs]=audioread(input);
count=1;
op=floor(fs/v.info.VideoFrameRate);
disp 'alle variabele geladen'
for i=Frame.segmentStart:Frame.segmentEnd
    videoFrame= step(v);
    audios=audio( (count-1)*op + 1 : count*op , : );
    step(cutVideo, videoFrame,audios);
 
    % disp (i)
 
    count=count+1;
end

release(v);
release(cutVideo);

disp 'cutVideoTest released'

But the video starts from the beginning instead from given frame

Many thanks for looking at this question!

[Walter Roberson]

Remember that with variable frame rate files that the Framerate property is an approximation and that you cannot use it to convert times to frame numbers.

[Piet Jonker]

Thanks for the answer Walter! So would it be more wise to use time in seconds instead of certain frames? If so, how can I implement them into the forloop I use when MATLAB goes over the videofile frame by frame? I can't think of a way and wonder if it is even possible.

[Walter Roberson]

With variable frame rate files, the length of audio associated with each frame might not be exactly the same: it might be just enough for the duration of that particular frame. This is a problem for VideoFileWriter, which requires that each frame of audio be exactly the same length (including the final one.) As far as I have been able to figure out so far, this means that it is not always possible to use VideoFileWriter to write out exactly the same thing that you might read with the readers.

Never count on the FrameRate to position by time. The FrameRate is an average, not necessarily correct for any given frame. It is common for FrameRate to be rounded to 2 decimals in computation, which is an error that can add up quickly. It is common for the NTSC frame rate of approximately 29.97 Hz to be represented as 30 Hz.

Thus, to position you should really examine the timecode after reading each frame.

[Piet Jonker]

Thanks Walter. Might be useful to know, that I worked around the matlab function by using FFmpeg. I installed it and used MATLAB to give commands to the windows command prompt. It is easily a factor 10 to 20 faster, because it doesn't need to load in every frame first. I installed FFmpeg and made this code:

markerDataFolderFFmpeg  = 'directory' % set directory
directoryInput_string   = '"D:\\directoryIn\\Video_%s.mp4"'; % creates a string which can be used by sprintf function in loop below
directoryOutput_string  = '"D:\\directoryOut\\Video_%s.mp4"'; % creates a string which can be used by sprintf function in loop below
FFmpegCommand2          = ['ffmpeg -ss %s -i ' directoryInput_string ' -to %s -c copy -copyts ' directoryOutput_string ' & ']; % creates a string which can be used by sprintf function in loop below and eventually is executed by command prompt
p = 1:75; % numbers videos to be cut
remove = [19 54 72]; % removes unusable videos

for r = remove
    p(p==r) = [];
end
nSubs = length(p);


allFFmpegCommandStrings = []; % creates empty variable to put strings in
for z = p 
    load(['' markerDataFolderFFmpeg 'TimeMarker_' num2str(z) '.mat' '']) % loads in timestamps for video z

    Start1_str = datestr(seconds(startTime),'HH:MM:SS.FFF'); % turns timestamp of video start into string
    End1_str = datestr(seconds(endTime),'HH:MM:SS.FFF'); % turns video duration timestamp into string
   
    FFmpegCommandString = sprintf(FFmpegCommand2,jobfitStart1_str,num2str(z),jobfitEnd1_str,num2str(z)); % creates one big string which contains a input command for the command prompt
    
    allFFmpegCommandStrings = [allFFmpegCommandStrings FFmpegCommandString]; 
   

end

dlmwrite('runFFmpeg.bat',allFFmpegCommandStrings,'delimiter','');
runFFmpeg = 'runFFmpeg.bat';


%% uncomment segment to cut videos

% tic
% system(runFFmpeg) 
% toc

And it worked like a charm!