From Multimedia File

Read video frames and audio samples from compressed multimedia file

Library

Sources

`visionsources`

Description

The From Multimedia File block reads audio samples, video frames, or both from a multimedia file. The block imports data from the file into a Simulink® model.

Note

This block supports code generation for the host computer that has file I/O available. You cannot use this block with Simulink Desktop Real-Time™ software because that product does not support file I/O.

The generated code for this block relies on prebuilt library files. You can run this code outside the MATLAB® environment, or redeploy it, but be sure to account for these extra library files when doing so. The `packNGo` function creates a single zip file containing all of the pieces required to run or rebuild this code. See `packNGo` (Simulink Coder) for more information.

To run an executable file that was generated from a model containing this block, you may need to add precompiled shared library files to your system path. See Simulink Coder, Simulink Shared Library Dependencies, and Accelerating Simulink Models for details.

This block allows you to read WMA/WMV streams to disk or across a network connection. Similarly, the To Multimedia File block allows you to write WMA/WMV streams to disk or across a network connection. If you want to play an MP3/MP4 file in Simulink, but you do not have the codecs, you can re-encode the file as WMA/WMV, which are supported by the Computer Vision Toolbox™.

PlatformSupported File Name Extensions
All PlatformsAVI (`.avi`)
Windows® Image:
`.jpg`,`.bmp`
Video:
MPEG (`.mpeg`)
MPEG-2 (`.mp2`)
MPEG-1`.mpg`

MPEG-4, including H.264 encoded video (`.mp4`, `.m4v`)
Motion JPEG 2000 (`.mj2`)
Windows Media Video (`.wmv`,`.asf`, `.asx`, .asx)
and any format supported by Microsoft DirectShow® 9.0 or higher.
Audio:
WAVE (`.wav`)
Windows Media Audio File (`.wma`)
Audio Interchange File Format (`.aif`, `.aiff`)
Compressed Audio Interchange File Format(`.aifc`),
MP3 (`.mp3`)
Sun Audio (`.au`)
Apple (`.snd`)
Macintosh Video:
`.avi`
Motion JPEG 2000 (`.mj2`)
MPEG-4, including H.264 encoded video (`.mp4`, `.m4v`)
Apple QuickTime Movie (.mov)
and any format supported by QuickTime as listed on http://support.apple.com/kb/HT3775.
Audio:
Uncompressed `.avi`
Linux® Motion JPEG 2000 (`.mj2`)
Any format supported by your installed plug-ins for GStreamer 0.1 or higher, as listed on http://gstreamer.freedesktop.org/documentation/plugins.html, including Ogg Theora (.ogg).

Ports

The output ports of the From Multimedia File block change according to the content of the multimedia file. If the file contains only video frames, the Image, intensity I, or R,G,B ports appear on the block. If the file contains only audio samples, the Audio port appears on the block. If the file contains both audio and video, you can select the data to emit. The following table describes available ports.

PortDescription
Image

M-by-N-by-P color video signal where P is the number of color planes.

I

M-by-N matrix of intensity values.

R, G, B

Matrix that represents one plane of the RGB video stream. Outputs from the R, G, or B ports must have same dimensions.

Audio

Vector of audio data.

Y, Cb, Cr

Matrix that represents one frame of the YCbCr video stream. The Y, Cb, Cr ports produce the following outputs:

 Y: M x N Cb: M x$\frac{N}{2}$ Cr: M x$\frac{N}{2}$

Sample Rates

The sample rate that the block uses depends on the audio and video sample rate. While the FMMF block operates at a single rate in Simulink, the underlying audio and video streams can produce different rates. In some cases, when the block outputs both audio and video, makes a small adjustment to the video rate.

Sample Time Calculations Used for Video and Audio Files

 Sample time = $\frac{ceil\left(AudioSampleRate}{FPS}\right)}{AudioSampleRate}$. When audio sample time, $\frac{AudioSampleRate}{FPS}$ is noninteger, the equation cannot reduce to $\frac{1}{FPS}$. In this case, to prevent synchronization problems, the block drops the corresponding video frame when the audio stream leads the video stream by more than $\frac{1}{FPS}$. In summary, the block outputs one video frame at each Simulink time step. To calculate the number of audio samples to output at each time step, the block divides the audio sample rate by the video frame rate (fps). If the audio sample rate does not divide evenly by the number of video frames per second, the block rounds the number of audio samples up to the nearest whole number. If necessary, the block periodically drops a video frame to maintain synchronization for large files.

Parameters

File name

Specify the name of the multimedia file from which to read. The block determines the type of file (audio and video, audio only, or video only) and provides the associated parameters.

If the location of the file does not appear on your MATLAB path, use the button to specify the full path. Otherwise, if the location of this file appears on your MATLAB path, enter only the file name. On Windows platforms, this parameter supports URLs that point to MMS (Microsoft Media Server) streams.

Inherit sample time from file

Select the Inherit sample time from file check box if you want the block sample time to be the same as the multimedia file. If you clear this check box, enter the block sample time in the Desired sample time parameter field. The file that the From Multimedia File block references, determines the block default sample time. You can also set the sample time for this block manually. If you do not know the intended sample rate of the video, let the block inherit the sample rate from the multimedia file.

Desired sample time

Specify the block sample time. This parameter becomes available if you clear the Inherit sample time from file check box.

Number of times to play file

Enter a positive integer or `inf` to represent the number of times to play the file.

Output end-of-file indicator

Use this check box to determine whether the output is the last video frame or audio sample in the multimedia file. When you select this check box, a Boolean output port labeled EOF appears on the block. The output from the EOF port defaults to 1 when the last video frame or audio sample is output from the block. Otherwise, the output from the EOF port defaults to 0.

Multimedia outputs

Specify `Video and audio`, ```Video only```, or `Audio only` output file type. This parameter becomes available only when a video signal has both audio and video.

Samples per audio channel

Specify number of samples per audio channel. This parameter becomes available for files containing audio.

Output color format

Specify whether you want the block to output `RGB`, `Intensity`, or `YCbCr 4:2:2` video frames. This parameter becomes available only for a signal that contains video. If you select `RGB`, use the Image signal parameter to specify how to output a color signal.

Image signal

Specify how to output a color video signal. If you select ```One multidimensional signal```, the block outputs an M-by-N-by-P color video signal, where P is the number of color planes, at one port. If you select `Separate color signals`, additional ports appear on the block. Each port outputs one M-by-N plane of an RGB video stream. This parameter becomes available only if you set the Image color space parameter to `RGB` and the signal contains video.

Audio output sampling mode

Select Sample based or Frame based output. This parameter appears when you specify a file containing audio for the File name parameter.

Audio output data type

Set the data type of the audio samples output at the Audio port. This parameter becomes available only if the multimedia file contains audio. You can choose `double`, `single`, `int16`, or `uint8` types.

Video output data type

Set the data type of the video frames output at the R, G, B, or Image ports. This parameter becomes available only if the multimedia file contains video. You can choose `double`, `single`, `int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, or `Inherit from file` types.

Supported Data Types

For source blocks to display video data properly, double- and single-precision floating-point pixel values must be between `0` and `1`. For other data types, the pixel values must be between the minimum and maximum values supported by their data type.

PortSupported Data TypesSupports Complex Values?

Image

• Double-precision floating point

• Single-precision floating point

• 8-, 16-, and 32-bit signed integers

• 8-, 16-, and 32-bit unsigned integers

No

R, G, B

Same as the Image port

No

Audio

• Double-precision floating point

• Single-precision floating point

• 16-bit signed integers

• 8-bit unsigned integers

No

Y, Cb,Cr

Same as the Image port

No