Signal Labeler

Label signals for analysis or machine and deep learning applications

Description

Signal Labeler is an interactive tool that enables you to label signals for analysis or for use in machine learning and deep learning applications. Using Signal Labeler, you can

  • Label signals with attributes, regions, and points of interest

  • Use logical, categorical, numerical, or string-valued labels

  • Automatically label signal peaks or apply custom labeling functions

  • Add, edit, and delete labels or sublabels

  • Display selected subsets of signals and labels

Signal Labeler saves data to Signal Analyzer as labeledSignalSet objects. You can use labeledSignalSet objects to train a network, classifier, or analyze data and report statistics.

Open the Signal Labeler

In the Signal Analyzer app, on the Analyzer tab, click Label.

Examples

expand all

Recordings of whale songs contain trills and moans. Trills sound like series of clicks. Moans are low-frequency cries similar to the sound made by a ship's horn. You want to look at each signal and label it to identify the whale type, the trill regions, and the moan regions. For each trill region, you also want to label a few selected signal peaks.

Load Unlabeled Data

Start by loading a data set that includes two recordings of whale songs. The signals are called whale1 and whale2 and are sampled at 4 kHz. whale1 consists of a trill followed by three moans. whale2 consists of two moans, a trill, and another moan.

load labelwhalesignals

% To hear, type soundsc(whale1,Fs), pause(22), soundsc(whale2,Fs)

Bring the signals into Signal Labeler:

  1. Start Signal Analyzer and drag the signals to the Signal table.

  2. Select the signals and add time information. On the Analyzer tab, click Time Values, select Sample Rate and Start Time, and enter the sample rate, Fs.

  3. With the signals selected, click Label on the Analyzer tab.

Add Signal Label Definitions

Define labels to attach to the signals. Labels can be of three types:

  • Attribute labels define signal characteristics.

  • Region-of-interest (ROI) labels define signal characteristics over regions of interest.

  • Point labels define signal characteristics over points of interest.

Each label can have one of four data types:

  • Logical labels can be either true or false.

  • Categorical labels can belong to any one of a set of categories that you specify.

  • Numeric labels can have any numeric value.

  • String labels can have any value represented by a string.

Any label can have any number of sublabels. Sublabels themselves cannot have sublabels.

For the whale song signals:

  1. Define a categorical attribute label to store whale types. Call it WhaleType. The possible categories are blue whale, humpback whale, and white whale.

  2. Define a logical region-of-interest (ROI) label that is true for moan regions. Call it MoanRegions.

  3. Define a logical ROI label that is true for trill regions. Call it TrillRegions.

  4. Define a numeric point label to capture trill peaks. Call it TrillPeaks. Set this label as a sublabel of the TrillRegions label.

To define each label, click Add Definition on the Label tab. To define the sublabel, select the TrillRegions label in the Label Definitions browser, click Add Definition ▼, and select Add sublabel definition.

Enter the following values in the fields in the dialog box that appears for each signal label or sublabel definition. Leave the Default field empty in each case.

LabelNameLabelTypeLabelDescriptionDataTypeCategoriesWhaleTypeAttributeWhaletypecategorical{bluehumpbackwhiteMoanRegionsROIRegionswheremoansoccurlogical---TrillRegionsROIRegionswheretrillsoccurlogical---TrillPeaksPointTrillpeaksnumeric---

You can export the signal definitions you created to a MAT-file by clicking Export. A dialog box appears that prompts you for a file name. At any point you can import signal definitions stored in a MAT-file by clicking Import.

Label Signal Attributes

The songs in the data are from two blue whales. Set the WhaleType values for both signals:

  1. Select WhaleType on the Label Definitions browser.

  2. Click Label ▼ and select Label Signals.

  3. In the dialog box that appears, verify that both whale1 and whale2 are selected and that the Value field is set to blue. (If you do not specify a default value in a categorical signal label definition, Signal Labeler sets the label to the first category specified in the definition.)

  4. Click OK.

Plot the whale1 signal by selecting the check box next to its name. Signal attributes appear both in the Labeled Signal Set browser and under the time plot.

Label Signal Regions

Visualize the whale songs and label the trill and moan regions.

  • Trill regions have distinct bursts of sound punctuated by silence. whale1 has a trill centered at about 2 seconds.

  • Moan regions are sustained low-frequency wails. whale1 has moans centered at about 7 seconds, 12 seconds, and 17 seconds.

Label the signals one at a time:

  1. On the Plot column of the Labeled Signal Set browser, check the box next to the signal name to plot the signal.

  2. To label a moan, on the Label Definitions browser, select the MoanRegions label definition.

  3. Click Label ▼ and select Label Plotted. A shaded region appears, framed by an animated dashed line. (The animated frame indicates the region is active.) Move and resize the active region until it encloses a moan region. For better label placement, you can go to the Display tab and choose a zoom action or activate the panner.

  4. Click the check mark next to the Label button, press Enter, or double-click to label the ROI. The region changes to a gradient of the signal color. If you do not specify a default value in a logical label definition, Signal Labeler sets the label to true.

  5. Repeat the procedure for the other two moans.

  6. To label a trill, on the Label Definitions browser, select the TrillRegions label definition. Label the trill region using steps 3 and 4.

  7. Before labeling the second whale song signal, remove the first whale song signal from the plot by clearing the check box next to its name in the Labeled Signal Set browser. If you have the two signals plotted when you label a region or point, Signal Labeler associates the label with both signals.

The label viewer axes show the locations and widths of the regions of interest. They also show the value assigned to each region.

Label Signal Points

Trill regions have distinct peaks that correspond to bursts of sound. Label three peaks in each trill region. Because trill peaks are sublabels, each one must be associated with a particular TrillRegions label.

Label the signals one at a time:

  1. On the Plot column of the Labeled Signal Set browser, check the box next to the signal name to plot the signal. Also check the box corresponding to the trill region whose peaks you want to label.

  2. On the Label Definitions browser, select TrillPeaks.

  3. On the toolstrip, under Value, enter 1, corresponding to the first peak.

  4. On the Labeled Signal Set browser, select the trill region. The trill region becomes active and is framed by an animated dashed line.

  5. Click Label ▼ and select Label Plotted. The trill region is framed by a solid line, and an animated dashed (active) line appears for the point being labeled.

  6. Move the active line until it crosses the signal at a peak of your choice. For better label placement, you can go to the Display tab and choose a zoom action or activate the panner.

  7. Click the check mark next to the Label button, press Enter, or double-click to label the peak. The dashed line changes to a solid line of the same color as the signal.

  8. Repeat for two more peaks, entering 2 and 3 to identify them.

  9. Before labeling trill peaks for the second whale song signal, remove the first whale song signal from the plot by clearing the check box next to its name in the Labeled Signal Set browser.

The label viewer axes show the locations of the points of interest and the value assigned to each point.

Plot the two signals to see a summary of their labels in the Label Viewer. Expand the labeled signal set hierarchy in the Labeled Signal Set browser to see details for all the labels. (To expand the hierarchy, right-click any signal in the browser and select Expand All.) For each signal, plot the first moan region and the third trill peak that you labeled.

Edit Signal Label Values

At any point, you can edit any signal label using the Labeled Signal Set browser. To edit an attribute label, select it, right-click, and select Edit. For example, if you discover that the second whale is actually a white whale, you can select the WhaleType attribute for whale2, right-click, select Edit, and, on the dialog box that appears, select white from the drop-down menu.

If you want to edit the value of an ROI label or a point label, you can modify the value in the dialog box. To modify the location of an ROI label or a point label, you can modify the location fields in the dialog box. Alternatively, you can:

  1. Plot the label by checking the box next to its name.

  2. Select the label.

  3. Move the region or point in the time plot when it becomes active.

Export Labeled Signal Set

Export labeled signals by saving your labeling and exporting the new labeledSignalSet object. Click the Save Labels button on the toolstrip. In the dialog box that appears, give the name whalesongs to the labeled signal set. Clicking the OK button gets you back to Signal Analyzer. See Signal Labeler Import and Export Behavior for more information on how Signal Labeler exports labeled signal sets.

On the Signal table, select whalesongs and right-click to export it to a file called Whale_Songs.mat.

Load into the MATLAB® workspace the MAT-file you created in the Label Signal Attributes, Regions of Interest, and Points example. Verify that the labeled signal set contains the definitions that you added using Signal Labeler.

load Whale_Songs

labelDefinitionsSummary(whalesongs)
ans=3×9 table
      LabelName        LabelType     LabelDataType     Categories     ValidationFunction    DefaultValue             Sublabels             Tag            Description         
    ______________    ___________    _____________    ____________    __________________    ____________    ___________________________    ___    ____________________________

    "WhaleType"       "attribute"    "categorical"    {3x1 string}       {["N/A"   ]}       {0x0 double}    {0x0 double               }    ""     "Whale type"                
    "MoanRegions"     "roi"          "logical"        {["N/A"   ]}       {0x0 double}       {0x0 double}    {0x0 double               }    ""     "Regions where moans occur" 
    "TrillRegions"    "roi"          "logical"        {["N/A"   ]}       {0x0 double}       {0x0 double}    {1x1 signalLabelDefinition}    ""     "Regions where trills occur"

Verify that TrillPeaks is a sublabel of TrillRegions.

labelDefinitionsHierarchy(whalesongs)
ans = 
    'WhaleType
       Sublabels: []
     MoanRegions
       Sublabels: []
     TrillRegions
       Sublabels: TrillPeaks
     '

Retrieve the second member of the set. Retrieve the names of the timetable variables.

song = getSignal(whalesongs,2);

summary(song)
RowTimes:

    Time: 76579x1 duration
        Values:
            Min           0 sec       
            Median        9.5722 sec  
            Max           19.144 sec  
            TimeStep      0.00025 sec 

Variables:

    whale2: 76579x1 double

        Values:

            Min       -0.37326
            Median           0
            Max        0.37914

Plot the signal.

t = song.Time;
sng = song.whale2;

plot(t,sng)

Visualize Labeled Regions

Display and identify the regions of interest that you labeled. For more details, see the code for the labelIntervals function at the end of the example.

mvals = getLabelValues(whalesongs,2,'MoanRegions');
tvals = getLabelValues(whalesongs,2,'TrillRegions');

cmap = lines;

hold on

tmoan = mvals.ROILimits;
for kj = 1:size(tmoan,1)
    tv = find(seconds(t)>tmoan(kj,1) & seconds(t)<tmoan(kj,2));
    plot(t(tv),sng(tv),'Color',cmap(2,:))
end

ttrill = tvals.ROILimits;
for kj = 1:size(ttrill,1)
    tv = find(seconds(t)>ttrill(kj,1) & seconds(t)<ttrill(kj,2));
    plot(t(tv),sng(tv),'Color',cmap(3,:))
end

labelIntervals(mvals,tvals,cmap(4,:))

hold off

Visualize Labeled Points

Display and identify the trill peaks that you labeled.

pk = getLabelValues(whalesongs,2,{'TrillRegions','TrillPeaks'});

locs  = zeros(size(pk,1),1);
for kj = 1:length(locs)
    locs(kj) = find(seconds(t) == pk.Location(kj));
end

hold on
plot(t(locs),sng(locs)+0.01,'v','MarkerSize',8,'Color',[0.929,0.694,0.125])
text(t(locs)+seconds(0.2),sng(locs)+0.05,int2str(cell2mat(pk.Value)), ...
    'HorizontalAlignment','center')
hold off

This helper function displays and identifies regions of interest.

function labelIntervals(mvals,tvals,clr)
    [X,Y] = meshgrid(seconds([mvals.ROILimits;tvals.ROILimits]),ylim);
    plot(X,Y,':k')
    topts = {'HorizontalAlignment','center','FontWeight','bold', ...
        'FontSize',12,'Color',clr};
    text((X(1,1:4)+X(1,5:end))/2,Y(2,5:end)-0.1, ...
        ["moan" "moan" "moan" "trill"],topts{:})
end

This example shows how to use custom automated labeling functions in Signal Labeler to label QRS complexes and R peaks of electrocardiogram (ECG) signals. One custom function uses a previously trained recurrent deep learning network to identify and locate the QRS complexes. Another custom function uses a simple peak finder to locate the R peaks. In the example, the network labels the QRS complexes of two signals that are completely independent of the network training and testing process.

The QRS complex, which consists of three deflections in the ECG waveform, reflects the depolarization of the right and left ventricles of the heart. The QRS is also the highest-amplitude segment of the human heartbeat. Study of the QRS complex can help assess the overall health of a person's heart and the presence of abnormalities [1]. In particular, by locating R peaks within the QRS complexes and looking at the time intervals between consecutive peaks, a diagnostician can compute the heart-rate variability of a patient and detect cardiac arrhythmia.

The deep learning network in this example was introduced in Waveform Segmentation Using Deep Learning, where it was trained using ECG signals from the publicly available QT Database [2] [3]. The data consists of roughly 15 minutes of ECG recordings from a total of 105 patients, sampled at 250 Hz. To obtain each recording, the examiners placed two electrodes on different locations on a patient's chest, which resulted in a two-channel signal. The database provides signal region labels generated by an automated expert system [1]. The added labels make it possible to use the data to train a deep network. See Waveform Segmentation Using Deep Learning for more details.

Load, Resample, and Import Data into Signal Labeler

The signals labeled in this example are from the MIT-BIH Arrhythmia Database [4]. Each signal in the database was sampled at 360 Hz and was annotated by two cardiologists, allowing for verification of the results.

Load two of the MIT database signals, corresponding to records 200 and 203. Resample the signals to 250 Hz, the sample rate of the QT Database data.

load mit200
y200 = resample(ecgsig,25,36);

load mit203
y203 = resample(ecgsig,25,36);

Start Signal Analyzer and drag the signals to the Signal table. Select the signals. Add time information: on the Analyzer tab, click Time Values, select Sample Rate and Start Time, and specify a sample rate of 250 Hz. On the Analyzer tab, click Label. The signals appear in the Labeled Signal Set browser.

Define Labels

Define labels to attach to the signals.

  1. Define a categorical region-of-interest (ROI) label for the QRS complexes. Click Add Definition on the Label tab. Specify the Label Name as QRSregions, select a LabelType of ROI, enter the Data Type as categorical, and add two Categories, QRS and n/a, each on its own line.

  2. Define a numerical point label for the R peaks and set it as a sublabel of QRSregions. Click QRSregions in the Label Definitions browser to select it. Click Add Definition ▼ and select Add sublabel definition. Specify the Label Name as Rpeaks, select a LabelType of Point, and enter the Data Type as numeric.

Create Custom Autolabeling Functions

Create two custom functions, one to locate and label the QRS complexes and another to locate and label the R peak within each QRS complex. (Code for the findQRS, computeFSST, p2qrs, and findRpeaks functions appears later in the example.) To create each function, in the Analyzer tab, click Automate Value ▼ and select Add Custom Function. Signal Labeler shows a dialog box asking for the name, description, and label type of the function to add.

  1. For the function that locates the QRS complexes, enter findQRS in the Name field and select ROI as the Label Type. You can leave the Description field empty or you can enter your own description.

  2. For the function that locates the R peaks, enter findRpeaks in the Name field and select Point as the Label Type. You can leave the Description field empty or you can enter your own description.

If you already have written the functions, and the functions are in the current folder or in the MATLAB® path, Signal Labeler adds the functions to the gallery. If you have not written the functions, Signal Labeler opens blank templates in the Editor for you to type or paste the code. Save the files. The functions appear in the gallery.

Label QRS Complexes and R Peaks

Find and label the QRS complexes of the input signals.

  1. In the Labeled Signal Set browser, select the check box next to y200.

  2. Select QRSregions in the Label Definitions browser.

  3. On the Automate Value gallery, select findQRS.

  4. Click Auto-Label and click OK in the dialog box that appears.

Signal Labeler locates and labels the QRS complexes for all signals but displays only those of the signal whose check box you selected. The QRS complexes appear as shaded regions in the plot and in the label viewer axes. Activate the panner by clicking Panner on the Display tab and zoom in on a region of the labeled signal.

Find and label the R peaks corresponding to the QRS complexes.

  1. Select Rpeaks in the Label Definitions browser.

  2. Go back to the Label tab. On the Automate Value gallery, select findRpeaks.

  3. Click Auto-Label and click OK in the dialog box that appears.

The labels and their numeric values appear in the plot and in the label viewer axes.

Export Labeled Signals and Compute Heart-Rate Variability

Export the labeled signals to compare the heart-rate variability for each patient. On the Label tab, click Save Labels. In the dialog box that appears, give the name heartrates to the labeled signal set. Click OK to return to Signal Analyzer. In the Signal table, select heartrates and right-click to export it to a file called HeartRates.mat.

Load the labeled signal set. For each signal in the set, compute the heart-rate variability as the standard deviation of the time differences between consecutive heartbeats. Plot a histogram of differences and display the heart-rate variability.

load HeartRates

nms = getMemberNames(heartrates);

for k = 1:heartrates.NumMembers
    
    v = getLabelValues(heartrates,k,{'QRSregions','Rpeaks'});
    
    hr = diff(cellfun(@(x)x.Location,v));
    
    subplot(2,1,k)
    histogram(hr,0.5:.025:1.5)
    legend(['hrv = ' num2str(std(hr))])
    ylabel(nms(k))
    ylim([0 6])

end

findQRS Function: Find QRS Complexes

The findQRS function finds and labels the QRS complexes of the input signals.

The function uses two auxiliary functions, computeFSST and p2qrs. (Code for both auxiliary functions appears later in the example.) You can either store the functions in separate files in the same directory or nest them inside findQRS by inserting them before the final end statement.

Between calls to computeFSST and p2qrs, findQRS uses the classify function and the trained deep network net to identify the QRS regions. Before calling classify, findQRS converts the data into the format expected by net, as explained in Waveform Segmentation Using Deep Learning:

  • Each signal must be sampled at 250 Hz and partitioned into a stack of 2-by-N cell arrays, where each row corresponds to a channel and N is a multiple of 5000. The actual partitioning and stacking is done in the computeFSST function.

  • Each of the resampled MIT signals has 6945 samples, a number that is not a multiple of 5000. To keep all the data in each signal, pad the signal with random numbers. Later in the process, the p2qrs function labels the random numbers as not belonging to QRS complexes and discards them.

function [labelVals,labelLocs] = findQRS(x,t,parentLabelVal,parentLabelLoc,varargin)
% This is a template for creating a custom function for automated labeling
%
%  x is a matrix where each column contains data corresponding to a
%  channel. If the channels have different lengths, then x is a cell array
%  of column vectors.
%
%  t is a matrix where each column contains time corresponding to a
%  channel. If the channels have different lengths, then t is a cell array
%  of column vectors.
%
%  parentLabelVal is the parent label value associated with the output
%  sublabel or empty when output is not a sublabel.
%  parentLabelLoc contains an empty vector when the parent label is an
%  attribute, a vector of ROI limits when parent label is an ROI or a point
%  location when parent label is a point.
%
%  labelVals must be a column vector with numeric, logical or string output
%  values.
%  labelLocs must be an empty vector when output labels are attributes, a
%  two column matrix of ROI limits when output labels are ROIs, or a column
%  vector of point locations when output labels are points.

labelVals = [];
labelLocs = [];

Fs = 250;

load('trainedQTSegmentationNetwork','net')

for kj = 1:size(x,2)

    sig = x(:,kj);
    
    % Create 10000-sample signal expected by the deep network
    
    sig = [sig;randn(10000-length(sig),1)/100]';
    
    % Resize input and compute synchrosqueezed Fourier transforms

    mitFSST = computeFSST(sig,Fs);
    
    % Use trained network to predict which points belong to QRS regions
    
    netPreds = classify(net,mitFSST,'MiniBatchSize',50);
    
    % Convert stack of cell arrays into a single vector
    
    Location = [1:length(netPreds{1}) length(netPreds{1})+(1:length(netPreds{2}))]';
    Value = [netPreds{1} netPreds{2}]';
    
    % Label QRS complexes as regions of interest and discard non-QRS data
    
    [Locs,Vals] = p2qrs(table(Location,Value));
    
    labelVals = [labelVals;Vals];
    labelLocs = [labelLocs;Locs/Fs];

end

% Insert computeFSST and p2qrs here if you want to nest them inside
% queryQRS instead of including them as separate functions in the folder.

end

computeFSST Function: Resize Input and Compute Synchrosqueezed Fourier Transforms

This function reshapes the input data into the form expected by net and then uses the fsst function to compute the Fourier synchrosqueezed transform (FSST) of the input. In Waveform Segmentation Using Deep Learning, the network performs best when given as input a time-frequency map of each training or testing signal. The FSST results in a set of features particularly useful for recurrent networks because the transform has the same time resolution as the original input. The function:

  • Specifies a Kaiser window of length 128 to provide adequate frequency resolution.

  • Extracts data over the frequency range from 0.5 Hz to 40 Hz.

  • Subtracts the mean of each signal and divides by the standard deviation.

  • Treat the real and imaginary parts of the FSST as separate features.

function signalsFsst = computeFSST(xd,Fs)

targetLength = 5000;
signalsOut = {};

for sig_idx = 1:size(xd,1)

    current_sig = xd(sig_idx,:)';

    % Compute the number of targetLength-sample chunks in the signal
    numSigs = floor(length(current_sig)/targetLength);

    % Truncate to a multiple of targetLength
    current_sig = current_sig(1:numSigs*targetLength);

    % Create a matrix with as many columns as targetLength signals
    xM = reshape(current_sig,targetLength,numSigs);

    % Vertically concatenate into cell arrays
    signalsOut = [signalsOut; mat2cell(xM.',ones(numSigs,1))];

end

signalsFsst = cell(size(signalsOut));

for idx = 1:length(signalsOut)

   [s,f] = fsst(signalsOut{idx},Fs,kaiser(128));

   % Extract data over the frequency range from 0.5 Hz to 40 Hz
   f_indices = (f > 0.5) & (f < 40);
   signalsFsst{idx}= [real(s(f_indices,:)); imag(s(f_indices,:))];

   signalsFsst{idx} = (signalsFsst{idx}-mean(signalsFsst{idx},2)) ...
       ./std(signalsFsst{idx},[],2);

end

end

p2qrs Function: Label QRS Complexes as Regions of Interest

The deep network outputs a categorical array that labels every point of the input signal as belonging a P region, a QRS complex, a T region, or to none of those. This function converts those point labels to QRS region-of-interest labels.

  • To perform the conversion, the function assigns integer numerical values to the categories and uses the findchangepts function to find the points where the numerical array changes value.

  • Each of those changepoints is the left endpoint of a categorical region, and the point that precedes it in the array is the right endpoint of the preceding region.

  • The algorithm adds 1e-6 to the right endpoints to prevent one-sample regions from having zero duration.

  • The df parameter selects as regions of interest only those QRS complexes whose duration is greater than df samples.

function [locs,vals] = p2qrs(k)

fc = 1e-6;
df = 20;

ctgs = categories(k.Value);
levs = 1:length(ctgs);
for jk = levs
   cat2num(k.Value == ctgs{jk}) = levs(jk);
end
chpt = findchangepts(cat2num,'MaxNumChanges',length(cat2num));
locs = [[1;chpt'] [chpt'-1;length(cat2num)]+fc];

vals = categorical(cat2num(locs(:,1))',levs,ctgs);
locs = locs+round(k.Location(1))-1;

qrs = find(vals=='QRS' & diff(locs,[],2)>df);

vals = categorical(string(vals(qrs)),["QRS" "n/a"]);

locs = locs(qrs,:);

end

findRpeaks Function: Find R Peaks

This function locates the most prominent peak of the QRS regions of interest found by findQRS. The function applies the MATLAB® islocalmax function to the absolute value of the signal in the intervals located by findQRS.

function [labelVals,labelLocs] = findRpeaks(x,t,parentLabelVal,parentLabelLoc,varargin)

Fs = 250;

if isempty(t)
    t = (0:length(x)-1)'/Fs;
end

labelVals = zeros(size(parentLabelLoc,1),1);
labelLocs = zeros(size(parentLabelLoc,1),1);

for kj = 1:size(parentLabelLoc,1)
    tvals = t>=parentLabelLoc(kj,1) & t<=parentLabelLoc(kj,2);
    ti = t(tvals);
    xi = x(tvals);
    lc = islocalmax(abs(xi),'MaxNumExtrema',1);
    labelVals(kj) = xi(lc);
    labelLocs(kj) = ti(lc);
end

end

References

[1] Laguna, Pablo, Raimon Jané, and Pere Caminal. "Automatic detection of wave boundaries in multilead ECG signals: Validation with the CSE database." Computers and Biomedical Research. Vol. 27, No. 1, 1994, pp. 45–60.

[2] Goldberger, Ary L., Luis A. N. Amaral, Leon Glass, Jeffery M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. "PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals." Circulation. Vol. 101, No. 23, 2000, pp. e215–e220. [Circulation Electronic Pages: http://circ.ahajournals.org/content/101/23/e215.full].

[3] Laguna, Pablo, Roger G. Mark, Ary L. Goldberger, and George B. Moody. "A Database for Evaluation of Algorithms for Measurement of QT and Other Waveform Intervals in the ECG." Computers in Cardiology. Vol. 24, 1997, pp. 673–676.

[4] Moody, George B., and Roger G. Mark. "The impact of the MIT-BIH Arrhythmia Database." IEEE Engineering in Medicine and Biology Magazine. Vol. 20, No. 3, May–June 2001, pp. 45–50.

This example shows how to label spoken words in Signal Labeler. The example uses the IBM® Watson Speech to Text API and Audio Toolbox™ software. See Speech-to-Text Transcription (Audio Toolbox) for instructions about:

  1. Downloading the Audio Toolbox speech2text extended functionality, available from MATLAB® Central.

  2. Setting up the IBM Watson Speech API, offered through IBM Cloud Services. You must create an IBM Cloud account, a Speech to Text service instance, and go to the service dashboard and copy your credentials – API Key and URL values. See the Getting Started Tutorial in the IBM documentation for more details.

Load Speech Data

Load an audio data file containing the sentence "Oak is strong, and also gives shade" spoken by a male voice. The signal is sampled at 44,100 Hz.

[y,fs] = audioread('oak.m4a');

% To hear, type soundsc(y,fs)
  1. Start Signal Analyzer and drag the signal to the Signal table. Select the signal.

  2. Add time information: on the Analyzer tab, click Time Values, select Sample Rate and Start Time, and specify fs as the sample rate.

  3. On the Analyzer tab, click Label. The signal appears in the Labeled Signal Set browser.

Define Label

Define a label to attach to the signal. Click Add Definition on the Label tab. Specify the Label Name as Words, select a Label Type of ROI, and enter the Data Type as string.

Create Custom Autolabeling Function

Create a custom function to label the words spoken in the audio file. (Code for the stt function appears later in the example.)

  1. Go to the directory where you have stored the speech2text P-code files and the JSON file that stores your IBM Coud credentials.

  2. To create the function, in the Analyzer tab, click Automate Value ▼ and select Add Custom Function. Signal Labeler shows a dialog box asking you to enter the name, description, and label type of the function to add. Enter stt in the Name field and select ROI as the Label Type. You can leave the Description field empty or you can enter your own description.

  3. Copy the function code and paste it in the empty template that appears. Save the file. The function appears in the gallery.

Locate and Identify Spoken Words

Locate and identify the words spoken in the input signal.

  1. In the Labeled Signal Set browser, select the check box next to y.

  2. Select Words in the Label Definitions browser.

  3. On the Automated Value gallery, select stt.

  4. Click Auto-Label and click OK in the dialog box that appears.

Signal Labeler locates and labels the spoken words.

Export Labeled Signal

Export the labeled signal. On the Label tab, click Save Labels. In the dialog box that appears, give the name transcribedAudio to the labeled signal set. Clicking OK returns you to Signal Analyzer. On the Signal table, select transcribedAudio and right-click to export it to a file called Transcription.mat.

Load the labeled signal set. The set has only one member. Get the names of the labels, and use the name to obtain and display the transcribed words.

load Transcription

ln = getLabelNames(transcribedAudio);

v = getLabelValues(transcribedAudio,1,ln)
v=7×2 table
     ROILimits       Value  
    ____________    ________

    0.09    0.56    "oak"   
    0.59    0.97    "is"    
       1    1.78    "strong"
    1.94    2.19    "and"   
    2.22    2.67    "also"  
    2.67    3.22    "gives" 
    3.25    3.91    "shade" 

Rearrange the words so that the sentence reads "Oak gives shade, and also is strong." Plot the signal using a different color for each word.

k = v([1 6:7 4:5 2:3],:);

s = getSignal(transcribedAudio,1);

sent = [];
sgs = NaN(height(s),height(k));
lgd = [];

for kj = 1:height(k)
    lm = length(sent);
    word = s.y(timerange(seconds(k.ROILimits(kj,1)),seconds(k.ROILimits(kj,2))));
    sent = [sent;word];
    sgs(lm+(1:length(word)),kj) = word;
    lgd = [lgd;(length(sent)-length(word)/2)/fs];
end

sgs(length(sent)+1:end,:) = [];

% To hear, type soundsc(sent,fs)

plot((0:length(sgs)-1)/fs,sgs)
text(lgd,-0.7*ones(size(lgd)),k.Value,'HorizontalAlignment',"center")
axis tight

stt Function: Locate and Identify Spoken Words

This function uses the IBM Watson Speech API and the Audio Toolbox speech2text extended functionality to extract spoken words from an audio file.

function [labelVals,labelLocs] = stt(x,t,parentLabelVal,parentLabelLoc,varargin)

aspeechObjectIBM = speechClient('IBM','timestamps',true,'model','en-US_NarrowbandModel');

fs = 1/(t(2)-t(1));

tixt = speech2text(aspeechObjectIBM,x,fs);

numLabels = numel(tixt.TimeStamps{:});
labelVals = strings(numLabels,1);
labelLocs = zeros(numLabels,2);

for idx =1:numLabels
    labelVals(idx) = tixt.TimeStamps{:}{idx}{1};
    labelLocs(idx,1) = tixt.TimeStamps{:}{idx}{2};
    labelLocs(idx,2) = tixt.TimeStamps{:}{idx}{3};
end

end

Tips

  • When saving labels, Signal Labeler converts all signals with time information to timetables. This conversion results in a deeper hierarchy of nested channels in the saved labeledSignalSet. See Signal Labeler Import and Export Behavior for more details.

  • Signal Labeler averages channel colors when it renders labels corresponding to multichannel signals. For best results, customize the line color so that it is the same for all channels in a given signal. Perform the customization in Signal Analyzer before entering Signal Labeler.

    Compare the label color for diffr, a signal whose channels have different colors, to the label color for equal, a signal whose channels all have the same color. Any labels for equal are rendered in the shade of blue that all the channels share. Labels for diffr are rendered in a shade of brown that does not match any of the channel colors.

  • When you reduce the width of the Signal Labeler window, the attribute table under the time plot shows a horizontal scrollbar. In Apple macOS systems, the scrollbar is hidden by default. If you have a trackpad and scroll horizontally, the scrollbar appears. If you use a mouse and want the scrollbar to be visible always, you can modify the system behavior in the System Preferences.

Introduced in R2019a