audioFeatureExtractor
Streamline audio feature extraction
Description
audioFeatureExtractor encapsulates multiple audio feature
extractors into a streamlined and modular implementation.
Creation
Description
aFE = audioFeatureExtractor()
aFE = audioFeatureExtractor(Name=Value)aFE using one or more name-value
arguments.
Properties
Object Functions
extract | Extract audio features |
setExtractorParameters | Set nondefault parameter values for individual feature extractors |
info | Output mapping and individual feature extractor parameters |
generateMATLABFunction | Create MATLAB function compatible with C/C++ code generation |
plotFeatures | Plot extracted audio features |
Examples
Read in an audio signal.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");Create an audioFeatureExtractor object that extracts the MFCC, delta MFCC, delta-delta MFCC, pitch, spectral centroid, zero-crossing rate, and short-time energy of the signal. Use a 30 ms analysis window with 20 ms overlap.
aFE = audioFeatureExtractor( ... SampleRate=fs, ... Window=hamming(round(0.03*fs),"periodic"), ... OverlapLength=round(0.02*fs), ... mfcc=true, ... mfccDelta=true, ... mfccDeltaDelta=true, ... pitch=true, ... spectralCentroid=true, ... zerocrossrate=true, ... shortTimeEnergy=true);
Call extract to extract the audio features from the audio signal.
features = extract(aFE,audioIn);
Use info to determine which column of the feature extraction matrix corresponds to the requested pitch extraction.
idx = info(aFE)
idx = struct with fields:
mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13]
mfccDelta: [14 15 16 17 18 19 20 21 22 23 24 25 26]
mfccDeltaDelta: [27 28 29 30 31 32 33 34 35 36 37 38 39]
spectralCentroid: 40
pitch: 41
zerocrossrate: 42
shortTimeEnergy: 43
Plot the detected pitch over time.
t = linspace(0,size(audioIn,1)/fs,size(features,1)); plot(t,features(:,idx.pitch)) title("Pitch") xlabel("Time (s)") ylabel("Frequency (Hz)")
Plot the zero-crossing rate over time.
plot(t,features(:,idx.zerocrossrate)) title("Zero-Crossing Rate") xlabel("Time (s)")
Plot the short-time energy over time.
plot(t,features(:,idx.shortTimeEnergy)) title("Short-Time Energy") xlabel("Time (s)")
Create an audio datastore that points to audio samples included with Audio Toolbox™.
folder = fullfile(matlabroot,"toolbox","audio","samples"); ads = audioDatastore(folder);
Create an audioFeatureExtractor object to extract the mel spectrum, Bark spectrum, ERB spectrum, and linear spectrum from each audio file. Use the default analysis window and overlap length for the spectrum extraction.
aFE = audioFeatureExtractor(SampleRate=44.1e3, ... melSpectrum=true, ... barkSpectrum=true, ... erbSpectrum=true, ... linearSpectrum=true);
Call extract to extract the features from each audio file in the datastore. Specify SampleRateMismatchRule as "resample" to resample the audio files in the datastore if they do not match 44.1 kHz, the sample rate of the audioFeatureExtractor object. If you have Parallel Computing Toolbox™, specify UseParallel as true to read the files and extract the features in parallel.
specs = extract(aFE,ads,SampleRateMismatchRule="resample",UseParallel=true);Starting parallel pool (parpool) using the 'Processes' profile ... 17-Dec-2024 09:28:59: Job Queued. Waiting for parallel pool job with ID 3 to start ... Connected to parallel pool with 4 workers.
The specs variable is a numFiles-by-1 cell array, where numFiles is the number of files in the datastore. Each element of the cell array is a numHops-by-numFeatures-by-numChannels array, where the number of hops and number of channels depends on the length and number of channels of the audio file, and the number of features is the requested number of features from the audio data.
numFiles = numel(specs)
numFiles = 39
[numHops1,numFeaturesFile1,numChanelsFile1] = size(specs{1})numHops1 = 1053
numFeaturesFile1 = 620
numChanelsFile1 = 1
[numHops2,numFeaturesFile2,numChanelsFile2] = size(specs{2})numHops2 = 1724
numFeaturesFile2 = 620
numChanelsFile2 = 4
Use plotFeatures to visualize audio features extracted with an audioFeatureExtractor object.
Read in an audio signal from a file.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");Create an audioFeatureExtractor object that extracts the gammatone cepstral coefficients (GTCCs) and the delta of the GTCCs. Set the SampleRate property to the sample rate of the audio signal, and use the default values for the other properties.
afe = audioFeatureExtractor(SampleRate=fs,gtcc=true,gtccDelta=true);
Plot the features extracted from the audio signal.
plotFeatures(afe,audioIn)
Algorithms
The audioFeatureExtractor creates a feature extraction pipeline based on
your selected features. To reduce computations, audioFeatureExtractor reuses
intermediary representations and outputs some intermediate representations as features.
For example, to create an object that extracts the centroid of the Bark spectrum, the flux
of the Bark spectrum, the pitch, the harmonic ratio, and the delta-delta of the MFCC, specify
the audioFeatureExtractor as
follows.
aFE = audioFeatureExtractor( ... SpectralDescriptorInput="barkSpectrum", ... spectralCentroid=true, ... spectralFlux=true, ... pitch=true, ... harmonicRatio=true, ... mfccDeltaDelta=true)
aFE =
audioFeatureExtractor with properties:
Properties
Window: [1024×1 double]
OverlapLength: 512
SampleRate: 44100
FFTLength: []
SpectralDescriptorInput: 'barkSpectrum'
Enabled Features
mfccDeltaDelta, spectralCentroid, spectralFlux, pitch, harmonicRatio
Disabled Features
linearSpectrum, melSpectrum, barkSpectrum, erbSpectrum, mfcc, mfccDelta
gtcc, gtccDelta, gtccDeltaDelta, spectralCrest, spectralDecrease, spectralEntropy
spectralFlatness, spectralKurtosis, spectralRolloffPoint, spectralSkewness, spectralSlope, spectralSpread
To extract a feature, set the corresponding property to true.
For example, obj.mfcc = true, adds mfcc to the list of enabled features.
Note
Because audioFeatureExtractor reuses intermediary representations, the
features output from audioFeatureExtractor might not correspond with the
default configuration of features output by corresponding individual feature
extractors.
