Signal Labeler

Label signals for analysis or machine and deep learning applications

Description

Signal Labeler is an interactive tool that enables you to label signals for analysis or for use in machine learning and deep learning applications. Using Signal Labeler, you can

• Label signals with attributes, regions, and points of interest

• Use logical, categorical, numerical, or string-valued labels

• Automatically label signal peaks or apply custom labeling functions

• Add, edit, and delete labels or sublabels

• Display selected subsets of signals and labels

Signal Labeler saves data to Signal Analyzer as `labeledSignalSet` objects. You can use `labeledSignalSet` objects to train a network, classifier, or analyze data and report statistics.

Open the Signal Labeler

In the Signal Analyzer app, on the Analyzer tab, click .

Examples

expand all

Recordings of whale songs contain trills and moans. Trills sound like series of clicks. Moans are low-frequency cries similar to the sound made by a ship's horn. You want to look at each signal and label it to identify the whale type, the trill regions, and the moan regions. For each trill region, you also want to label a few selected signal peaks.

Start by loading a data set that includes two recordings of whale songs. The signals are called `whale1` and `whale2` and are sampled at 4 kHz. `whale1` consists of a trill followed by three moans. `whale2` consists of two moans, a trill, and another moan.

```load labelwhalesignals % To hear, type soundsc(whale1,Fs), pause(22), soundsc(whale2,Fs)```

Bring the signals into Signal Labeler:

1. Start Signal Analyzer and drag the signals to the Signal table.

2. Select the signals and add time information. On the Analyzer tab, click Time Values, select `Sample Rate and Start Time`, and enter the sample rate, `Fs`.

3. With the signals selected, click Label on the Analyzer tab.

Define labels to attach to the signals. Labels can be of three types:

• Attribute labels define signal characteristics.

• Region-of-interest (ROI) labels define signal characteristics over regions of interest.

• Point labels define signal characteristics over points of interest.

Each label can have one of four data types:

• Logical labels can be either true or false.

• Categorical labels can belong to any one of a set of categories that you specify.

• Numeric labels can have any numeric value.

• String labels can have any value represented by a string.

Any label can have any number of sublabels. Sublabels themselves cannot have sublabels.

For the whale song signals:

1. Define a categorical attribute label to store whale types. Call it `WhaleType`. The possible categories are blue whale, humpback whale, and white whale.

2. Define a logical region-of-interest (ROI) label that is true for moan regions. Call it `MoanRegions`.

3. Define a logical ROI label that is true for trill regions. Call it `TrillRegions`.

4. Define a numeric point label to capture trill peaks. Call it `TrillPeaks`. Set this label as a sublabel of the `TrillRegions` label.

To define each label, click Add Definition on the Label tab. To define the sublabel, select the `TrillRegions` label in the Label Definitions browser, click Add Definition ▼, and select `Add sublabel definition`.

Enter the following values in the fields in the dialog box that appears for each signal label or sublabel definition. Leave the Default field empty in each case.

`$\phantom{\rule{3.6em}{0ex}}\begin{array}{c}{\phantom{\rule{0.2777777777777778em}{0ex}}}_{}\\ \begin{array}{ccccc}{Label\phantom{\rule{0.2777777777777778em}{0ex}}Name}_{}& Label\phantom{\rule{0.2777777777777778em}{0ex}}Type& Label\phantom{\rule{0.2777777777777778em}{0ex}}Description& Data\phantom{\rule{0.2777777777777778em}{0ex}}Type& Categories\\ WhaleType& Attribute& Whale\phantom{\rule{0.2777777777777778em}{0ex}}type& categorical& \left\{\begin{array}{c}blue\\ humpback\\ white\end{array}\\ MoanRegions& ROI& Regions\phantom{\rule{0.2777777777777778em}{0ex}}where\phantom{\rule{0.2777777777777778em}{0ex}}moans\phantom{\rule{0.2777777777777778em}{0ex}}occur& logical& ---\\ TrillRegions& ROI& Regions\phantom{\rule{0.2777777777777778em}{0ex}}where\phantom{\rule{0.2777777777777778em}{0ex}}trills\phantom{\rule{0.2777777777777778em}{0ex}}occur& logical& ---\\ TrillPeaks& Point& Trill\phantom{\rule{0.2777777777777778em}{0ex}}peaks& numeric& ---\end{array}\\ {\phantom{\rule{0.2777777777777778em}{0ex}}}_{}\\ \phantom{\rule{0.2777777777777778em}{0ex}}\end{array}$`

You can export the signal definitions you created to a MAT-file by clicking Export. A dialog box appears that prompts you for a file name. At any point you can import signal definitions stored in a MAT-file by clicking Import.

Label Signal Attributes

The songs in the data are from two blue whales. Set the `WhaleType` values for both signals:

1. Select `WhaleType` on the Label Definitions browser.

2. Click Label ▼ and select `Label Signals`.

3. In the dialog box that appears, verify that both `whale1` and `whale2` are selected and that the Value field is set to `blue`. (If you do not specify a default value in a categorical signal label definition, Signal Labeler sets the label to the first category specified in the definition.)

4. Click OK.

Plot the `whale1` signal by selecting the check box next to its name. Signal attributes appear both in the Labeled Signal Set browser and under the time plot.

Label Signal Regions

Visualize the whale songs and label the trill and moan regions.

• Trill regions have distinct bursts of sound punctuated by silence. `whale1` has a trill centered at about 2 seconds.

• Moan regions are sustained low-frequency wails. `whale1` has moans centered at about 7 seconds, 12 seconds, and 17 seconds.

Label the signals one at a time:

1. On the Plot column of the Labeled Signal Set browser, check the box next to the signal name to plot the signal.

2. To label a moan, on the Label Definitions browser, select the `MoanRegions` label definition.

3. Click Label ▼ and select `Label Plotted`. A shaded region appears, framed by an animated dashed line. (The animated frame indicates the region is active.) Move and resize the active region until it encloses a moan region. For better label placement, you can go to the Display tab and choose a zoom action or activate the panner.

4. Click the check mark next to the Label button, press Enter, or double-click to label the ROI. The region changes to a gradient of the signal color. If you do not specify a default value in a logical label definition, Signal Labeler sets the label to `true`.

5. Repeat the procedure for the other two moans.

6. To label a trill, on the Label Definitions browser, select the `TrillRegions` label definition. Label the trill region using steps 3 and 4.

7. Before labeling the second whale song signal, remove the first whale song signal from the plot by clearing the check box next to its name in the Labeled Signal Set browser. If you have the two signals plotted when you label a region or point, Signal Labeler associates the label with both signals.

The label viewer axes show the locations and widths of the regions of interest. They also show the value assigned to each region.

Label Signal Points

Trill regions have distinct peaks that correspond to bursts of sound. Label three peaks in each trill region. Because trill peaks are sublabels, each one must be associated with a particular `TrillRegions` label.

Label the signals one at a time:

1. On the Plot column of the Labeled Signal Set browser, check the box next to the signal name to plot the signal. Also check the box corresponding to the trill region whose peaks you want to label.

2. On the Label Definitions browser, select `TrillPeaks`.

3. On the toolstrip, under Value, enter `1`, corresponding to the first peak.

4. On the Labeled Signal Set browser, select the trill region. The trill region becomes active and is framed by an animated dashed line.

5. Click Label ▼ and select `Label Plotted`. The trill region is framed by a solid line, and an animated dashed (active) line appears for the point being labeled.

6. Move the active line until it crosses the signal at a peak of your choice. For better label placement, you can go to the Display tab and choose a zoom action or activate the panner.

7. Click the check mark next to the Label button, press Enter, or double-click to label the peak. The dashed line changes to a solid line of the same color as the signal.

8. Repeat for two more peaks, entering `2` and `3` to identify them.

9. Before labeling trill peaks for the second whale song signal, remove the first whale song signal from the plot by clearing the check box next to its name in the Labeled Signal Set browser.

The label viewer axes show the locations of the points of interest and the value assigned to each point.

Plot the two signals to see a summary of their labels in the Label Viewer. Expand the labeled signal set hierarchy in the Labeled Signal Set browser to see details for all the labels. (To expand the hierarchy, right-click any signal in the browser and select `Expand All`.) For each signal, plot the first moan region and the third trill peak that you labeled.

Edit Signal Label Values

At any point, you can edit any signal label using the Labeled Signal Set browser. To edit an attribute label, select it, right-click, and select Edit. For example, if you discover that the second whale is actually a white whale, you can select the `WhaleType` attribute for `whale2`, right-click, select Edit, and, on the dialog box that appears, select `white` from the drop-down menu.

If you want to edit the value of an ROI label or a point label, you can modify the value in the dialog box. To modify the location of an ROI label or a point label, you can modify the location fields in the dialog box. Alternatively, you can:

1. Plot the label by checking the box next to its name.

2. Select the label.

3. Move the region or point in the time plot when it becomes active.

Export Labeled Signal Set

Export labeled signals by saving your labeling and exporting the new `labeledSignalSet` object. Click the Save Labels button on the toolstrip. In the dialog box that appears, give the name `whalesongs` to the labeled signal set. Clicking the OK button gets you back to Signal Analyzer. See Signal Labeler Import and Export Behavior for more information on how Signal Labeler exports labeled signal sets.

On the Signal table, select `whalesongs` and right-click to export it to a file called `Whale_Songs.mat`.

Load into the MATLAB® workspace the MAT-file you created in the Label Signal Attributes, Regions of Interest, and Points example. Verify that the labeled signal set contains the definitions that you added using Signal Labeler.

```load Whale_Songs labelDefinitionsSummary(whalesongs)```
```ans=3×9 table LabelName LabelType LabelDataType Categories ValidationFunction DefaultValue Sublabels Tag Description ______________ ___________ _____________ ____________ __________________ ____________ ___________________________ ___ ____________________________ "WhaleType" "attribute" "categorical" {3x1 string} {["N/A" ]} {0x0 double} {0x0 double } "" "Whale type" "MoanRegions" "roi" "logical" {["N/A" ]} {0x0 double} {0x0 double} {0x0 double } "" "Regions where moans occur" "TrillRegions" "roi" "logical" {["N/A" ]} {0x0 double} {0x0 double} {1x1 signalLabelDefinition} "" "Regions where trills occur" ```

Verify that `TrillPeaks` is a sublabel of `TrillRegions`.

`labelDefinitionsHierarchy(whalesongs)`
```ans = 'WhaleType Sublabels: [] MoanRegions Sublabels: [] TrillRegions Sublabels: TrillPeaks ' ```

Retrieve the second member of the set. Retrieve the names of the timetable variables.

```song = getSignal(whalesongs,2); summary(song)```
```RowTimes: Time: 76579x1 duration Values: Min 0 sec Median 9.5722 sec Max 19.144 sec TimeStep 0.00025 sec Variables: whale2: 76579x1 double Values: Min -0.37326 Median 0 Max 0.37914 ```

Plot the signal.

```t = song.Time; sng = song.whale2; plot(t,sng)```

Visualize Labeled Regions

Display and identify the regions of interest that you labeled. For more details, see the code for the `labelIntervals` function at the end of the example.

```mvals = getLabelValues(whalesongs,2,'MoanRegions'); tvals = getLabelValues(whalesongs,2,'TrillRegions'); cmap = lines; hold on tmoan = mvals.ROILimits; for kj = 1:size(tmoan,1) tv = find(seconds(t)>tmoan(kj,1) & seconds(t)<tmoan(kj,2)); plot(t(tv),sng(tv),'Color',cmap(2,:)) end ttrill = tvals.ROILimits; for kj = 1:size(ttrill,1) tv = find(seconds(t)>ttrill(kj,1) & seconds(t)<ttrill(kj,2)); plot(t(tv),sng(tv),'Color',cmap(3,:)) end labelIntervals(mvals,tvals,cmap(4,:)) hold off```

Visualize Labeled Points

Display and identify the trill peaks that you labeled.

```pk = getLabelValues(whalesongs,2,{'TrillRegions','TrillPeaks'}); locs = zeros(size(pk,1),1); for kj = 1:length(locs) locs(kj) = find(seconds(t) == pk.Location(kj)); end hold on plot(t(locs),sng(locs)+0.01,'v','MarkerSize',8,'Color',[0.929,0.694,0.125]) text(t(locs)+seconds(0.2),sng(locs)+0.05,int2str(cell2mat(pk.Value)), ... 'HorizontalAlignment','center') hold off```

This helper function displays and identifies regions of interest.

```function labelIntervals(mvals,tvals,clr) [X,Y] = meshgrid(seconds([mvals.ROILimits;tvals.ROILimits]),ylim); plot(X,Y,':k') topts = {'HorizontalAlignment','center','FontWeight','bold', ... 'FontSize',12,'Color',clr}; text((X(1,1:4)+X(1,5:end))/2,Y(2,5:end)-0.1, ... ["moan" "moan" "moan" "trill"],topts{:}) end```

This example shows how to use custom automated labeling functions in Signal Labeler to label QRS complexes and R peaks of electrocardiogram (ECG) signals. One custom function uses a previously trained recurrent deep learning network to identify and locate the QRS complexes. Another custom function uses a simple peak finder to locate the R peaks. In the example, the network labels the QRS complexes of two signals that are completely independent of the network training and testing process.

The QRS complex, which consists of three deflections in the ECG waveform, reflects the depolarization of the right and left ventricles of the heart. The QRS is also the highest-amplitude segment of the human heartbeat. Study of the QRS complex can help assess the overall health of a person's heart and the presence of abnormalities [1]. In particular, by locating R peaks within the QRS complexes and looking at the time intervals between consecutive peaks, a diagnostician can compute the heart-rate variability of a patient and detect cardiac arrhythmia.

The deep learning network in this example was introduced in Waveform Segmentation Using Deep Learning, where it was trained using ECG signals from the publicly available QT Database [2] [3]. The data consists of roughly 15 minutes of ECG recordings from a total of 105 patients, sampled at 250 Hz. To obtain each recording, the examiners placed two electrodes on different locations on a patient's chest, which resulted in a two-channel signal. The database provides signal region labels generated by an automated expert system [1]. The added labels make it possible to use the data to train a deep network. See Waveform Segmentation Using Deep Learning for more details.

Load, Resample, and Import Data into Signal Labeler

The signals labeled in this example are from the MIT-BIH Arrhythmia Database [4]. Each signal in the database was sampled at 360 Hz and was annotated by two cardiologists, allowing for verification of the results.

Load two of the MIT database signals, corresponding to records 200 and 203. Resample the signals to 250 Hz, the sample rate of the QT Database data.

```load mit200 y200 = resample(ecgsig,25,36); load mit203 y203 = resample(ecgsig,25,36);```

Start Signal Analyzer and drag the signals to the Signal table. Select the signals. Add time information: on the Analyzer tab, click Time Values, select `Sample Rate and Start Time`, and specify a sample rate of 250 Hz. On the Analyzer tab, click Label. The signals appear in the Labeled Signal Set browser.

Define Labels

Define labels to attach to the signals.

1. Define a categorical region-of-interest (ROI) label for the QRS complexes. Click Add Definition on the Label tab. Specify the Label Name as `QRSregions`, select a LabelType of `ROI`, enter the Data Type as `categorical`, and add two Categories, `QRS` and `n/a`, each on its own line.

2. Define a numerical point label for the R peaks and set it as a sublabel of `QRSregions`. Click QRSregions in the Label Definitions browser to select it. Click Add Definition ▼ and select `Add sublabel definition`. Specify the Label Name as `Rpeaks`, select a LabelType of `Point`, and enter the Data Type as `numeric`.

Create Custom Autolabeling Functions

Create two custom functions, one to locate and label the QRS complexes and another to locate and label the R peak within each QRS complex. (Code for the `findQRS`, `computeFSST`, `p2qrs`, and `findRpeaks` functions appears later in the example.) To create each function, in the Analyzer tab, click Automate Value ▼ and select Add Custom Function. Signal Labeler shows a dialog box asking for the name, description, and label type of the function to add.

1. For the function that locates the QRS complexes, enter `findQRS` in the Name field and select `ROI` as the Label Type. You can leave the Description field empty or you can enter your own description.

2. For the function that locates the R peaks, enter `findRpeaks` in the Name field and select `Point` as the Label Type. You can leave the Description field empty or you can enter your own description.

If you already have written the functions, and the functions are in the current folder or in the MATLAB® path, Signal Labeler adds the functions to the gallery. If you have not written the functions, Signal Labeler opens blank templates in the Editor for you to type or paste the code. Save the files. The functions appear in the gallery.

Label QRS Complexes and R Peaks

Find and label the QRS complexes of the input signals.

1. In the Labeled Signal Set browser, select the check box next to `y200`.

2. Select `QRSregions` in the Label Definitions browser.

3. On the Automate Value gallery, select `findQRS`.

4. Click Auto-Label and click OK in the dialog box that appears.

Signal Labeler locates and labels the QRS complexes for all signals but displays only those of the signal whose check box you selected. The QRS complexes appear as shaded regions in the plot and in the label viewer axes. Activate the panner by clicking Panner on the Display tab and zoom in on a region of the labeled signal.

Find and label the R peaks corresponding to the QRS complexes.

1. Select `Rpeaks` in the Label Definitions browser.

2. Go back to the Label tab. On the Automate Value gallery, select `findRpeaks`.

3. Click Auto-Label and click OK in the dialog box that appears.

The labels and their numeric values appear in the plot and in the label viewer axes.

Export Labeled Signals and Compute Heart-Rate Variability

Export the labeled signals to compare the heart-rate variability for each patient. On the Label tab, click Save Labels. In the dialog box that appears, give the name `heartrates` to the labeled signal set. Click OK to return to Signal Analyzer. In the Signal table, select `heartrates` and right-click to export it to a file called `HeartRates.mat`.

Load the labeled signal set. For each signal in the set, compute the heart-rate variability as the standard deviation of the time differences between consecutive heartbeats. Plot a histogram of differences and display the heart-rate variability.

```load HeartRates nms = getMemberNames(heartrates); for k = 1:heartrates.NumMembers v = getLabelValues(heartrates,k,{'QRSregions','Rpeaks'}); hr = diff(cellfun(@(x)x.Location,v)); subplot(2,1,k) histogram(hr,0.5:.025:1.5) legend(['hrv = ' num2str(std(hr))]) ylabel(nms(k)) ylim([0 6]) end```

`findQRS` Function: Find QRS Complexes

The `findQRS` function finds and labels the QRS complexes of the input signals.

The function uses two auxiliary functions, `computeFSST` and `p2qrs`. (Code for both auxiliary functions appears later in the example.) You can either store the functions in separate files in the same directory or nest them inside `findQRS` by inserting them before the final `end` statement.

Between calls to `computeFSST` and `p2qrs`, `findQRS` uses the `classify` function and the trained deep network `net` to identify the QRS regions. Before calling `classify`, `findQRS` converts the data into the format expected by `net`, as explained in Waveform Segmentation Using Deep Learning:

• Each signal must be sampled at 250 Hz and partitioned into a stack of 2-by-N cell arrays, where each row corresponds to a channel and N is a multiple of 5000. The actual partitioning and stacking is done in the `computeFSST` function.

• Each of the resampled MIT signals has 6945 samples, a number that is not a multiple of 5000. To keep all the data in each signal, pad the signal with random numbers. Later in the process, the `p2qrs` function labels the random numbers as not belonging to QRS complexes and discards them.

```function [labelVals,labelLocs] = findQRS(x,t,parentLabelVal,parentLabelLoc,varargin) % This is a template for creating a custom function for automated labeling % % x is a matrix where each column contains data corresponding to a % channel. If the channels have different lengths, then x is a cell array % of column vectors. % % t is a matrix where each column contains time corresponding to a % channel. If the channels have different lengths, then t is a cell array % of column vectors. % % parentLabelVal is the parent label value associated with the output % sublabel or empty when output is not a sublabel. % parentLabelLoc contains an empty vector when the parent label is an % attribute, a vector of ROI limits when parent label is an ROI or a point % location when parent label is a point. % % labelVals must be a column vector with numeric, logical or string output % values. % labelLocs must be an empty vector when output labels are attributes, a % two column matrix of ROI limits when output labels are ROIs, or a column % vector of point locations when output labels are points. labelVals = []; labelLocs = []; Fs = 250; load('trainedQTSegmentationNetwork','net') for kj = 1:size(x,2) sig = x(:,kj); % Create 10000-sample signal expected by the deep network sig = [sig;randn(10000-length(sig),1)/100]'; % Resize input and compute synchrosqueezed Fourier transforms mitFSST = computeFSST(sig,Fs); % Use trained network to predict which points belong to QRS regions netPreds = classify(net,mitFSST,'MiniBatchSize',50); % Convert stack of cell arrays into a single vector Location = [1:length(netPreds{1}) length(netPreds{1})+(1:length(netPreds{2}))]'; Value = [netPreds{1} netPreds{2}]'; % Label QRS complexes as regions of interest and discard non-QRS data [Locs,Vals] = p2qrs(table(Location,Value)); labelVals = [labelVals;Vals]; labelLocs = [labelLocs;Locs/Fs]; end % Insert computeFSST and p2qrs here if you want to nest them inside % queryQRS instead of including them as separate functions in the folder. end```

`computeFSST` Function: Resize Input and Compute Synchrosqueezed Fourier Transforms

This function reshapes the input data into the form expected by `net` and then uses the `fsst` function to compute the Fourier synchrosqueezed transform (FSST) of the input. In Waveform Segmentation Using Deep Learning, the network performs best when given as input a time-frequency map of each training or testing signal. The FSST results in a set of features particularly useful for recurrent networks because the transform has the same time resolution as the original input. The function:

• Specifies a Kaiser window of length 128 to provide adequate frequency resolution.

• Extracts data over the frequency range from 0.5 Hz to 40 Hz.

• Subtracts the mean of each signal and divides by the standard deviation.

• Treat the real and imaginary parts of the FSST as separate features.

```function signalsFsst = computeFSST(xd,Fs) targetLength = 5000; signalsOut = {}; for sig_idx = 1:size(xd,1) current_sig = xd(sig_idx,:)'; % Compute the number of targetLength-sample chunks in the signal numSigs = floor(length(current_sig)/targetLength); % Truncate to a multiple of targetLength current_sig = current_sig(1:numSigs*targetLength); % Create a matrix with as many columns as targetLength signals xM = reshape(current_sig,targetLength,numSigs); % Vertically concatenate into cell arrays signalsOut = [signalsOut; mat2cell(xM.',ones(numSigs,1))]; end signalsFsst = cell(size(signalsOut)); for idx = 1:length(signalsOut) [s,f] = fsst(signalsOut{idx},Fs,kaiser(128)); % Extract data over the frequency range from 0.5 Hz to 40 Hz f_indices = (f > 0.5) & (f < 40); signalsFsst{idx}= [real(s(f_indices,:)); imag(s(f_indices,:))]; signalsFsst{idx} = (signalsFsst{idx}-mean(signalsFsst{idx},2)) ... ./std(signalsFsst{idx},[],2); end end```

`p2qrs` Function: Label QRS Complexes as Regions of Interest

The deep network outputs a categorical array that labels every point of the input signal as belonging a P region, a QRS complex, a T region, or to none of those. This function converts those point labels to QRS region-of-interest labels.

• To perform the conversion, the function assigns integer numerical values to the categories and uses the `findchangepts` function to find the points where the numerical array changes value.

• Each of those changepoints is the left endpoint of a categorical region, and the point that precedes it in the array is the right endpoint of the preceding region.

• The algorithm adds `1e-6` to the right endpoints to prevent one-sample regions from having zero duration.

• The `df` parameter selects as regions of interest only those QRS complexes whose duration is greater than `df` samples.

```function [locs,vals] = p2qrs(k) fc = 1e-6; df = 20; ctgs = categories(k.Value); levs = 1:length(ctgs); for jk = levs cat2num(k.Value == ctgs{jk}) = levs(jk); end chpt = findchangepts(cat2num,'MaxNumChanges',length(cat2num)); locs = [[1;chpt'] [chpt'-1;length(cat2num)]+fc]; vals = categorical(cat2num(locs(:,1))',levs,ctgs); locs = locs+round(k.Location(1))-1; qrs = find(vals=='QRS' & diff(locs,[],2)>df); vals = categorical(string(vals(qrs)),["QRS" "n/a"]); locs = locs(qrs,:); end```

`findRpeaks` Function: Find R Peaks

This function locates the most prominent peak of the QRS regions of interest found by `findQRS`. The function applies the MATLAB® `islocalmax` function to the absolute value of the signal in the intervals located by `findQRS`.

```function [labelVals,labelLocs] = findRpeaks(x,t,parentLabelVal,parentLabelLoc,varargin) Fs = 250; if isempty(t) t = (0:length(x)-1)'/Fs; end labelVals = zeros(size(parentLabelLoc,1),1); labelLocs = zeros(size(parentLabelLoc,1),1); for kj = 1:size(parentLabelLoc,1) tvals = t>=parentLabelLoc(kj,1) & t<=parentLabelLoc(kj,2); ti = t(tvals); xi = x(tvals); lc = islocalmax(abs(xi),'MaxNumExtrema',1); labelVals(kj) = xi(lc); labelLocs(kj) = ti(lc); end end```

References

[1] Laguna, Pablo, Raimon Jané, and Pere Caminal. "Automatic detection of wave boundaries in multilead ECG signals: Validation with the CSE database." Computers and Biomedical Research. Vol. 27, No. 1, 1994, pp. 45–60.

[2] Goldberger, Ary L., Luis A. N. Amaral, Leon Glass, Jeffery M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. "PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals." Circulation. Vol. 101, No. 23, 2000, pp. e215–e220. [Circulation Electronic Pages: http://circ.ahajournals.org/content/101/23/e215.full].

[3] Laguna, Pablo, Roger G. Mark, Ary L. Goldberger, and George B. Moody. "A Database for Evaluation of Algorithms for Measurement of QT and Other Waveform Intervals in the ECG." Computers in Cardiology. Vol. 24, 1997, pp. 673–676.

[4] Moody, George B., and Roger G. Mark. "The impact of the MIT-BIH Arrhythmia Database." IEEE Engineering in Medicine and Biology Magazine. Vol. 20, No. 3, May–June 2001, pp. 45–50.

This example shows how to label spoken words in Signal Labeler. The example uses the IBM® Watson Speech to Text API and Audio Toolbox™ software. See Speech-to-Text Transcription (Audio Toolbox) for instructions about:

1. Downloading the Audio Toolbox `speech2text` extended functionality, available from MATLAB® Central.

2. Setting up the IBM Watson Speech API, offered through IBM Cloud Services. You must create an IBM Cloud account, a Speech to Text service instance, and go to the service dashboard and copy your credentials – API Key and URL values. See the Getting Started Tutorial in the IBM documentation for more details.

Load an audio data file containing the sentence "Oak is strong, and also gives shade" spoken by a male voice. The signal is sampled at 44,100 Hz.

```[y,fs] = audioread('oak.m4a'); % To hear, type soundsc(y,fs)```
1. Start Signal Analyzer and drag the signal to the Signal table. Select the signal.

2. Add time information: on the Analyzer tab, click Time Values, select `Sample Rate and Start Time`, and specify `fs` as the sample rate.

3. On the Analyzer tab, click Label. The signal appears in the Labeled Signal Set browser.

Define Label

Define a label to attach to the signal. Click Add Definition on the Label tab. Specify the Label Name as `Words`, select a Label Type of `ROI`, and enter the Data Type as `string`.

Create Custom Autolabeling Function

Create a custom function to label the words spoken in the audio file. (Code for the `stt` function appears later in the example.)

1. Go to the directory where you have stored the `speech2text` P-code files and the JSON file that stores your IBM Coud credentials.

2. To create the function, in the Analyzer tab, click Automate Value ▼ and select Add Custom Function. Signal Labeler shows a dialog box asking you to enter the name, description, and label type of the function to add. Enter `stt` in the Name field and select `ROI` as the Label Type. You can leave the Description field empty or you can enter your own description.

3. Copy the function code and paste it in the empty template that appears. Save the file. The function appears in the gallery.

Locate and Identify Spoken Words

Locate and identify the words spoken in the input signal.

1. In the Labeled Signal Set browser, select the check box next to `y`.

2. Select `Words` in the Label Definitions browser.

3. On the Automated Value gallery, select `stt`.

4. Click Auto-Label and click OK in the dialog box that appears.

Signal Labeler locates and labels the spoken words.

Export Labeled Signal

Export the labeled signal. On the Label tab, click Save Labels. In the dialog box that appears, give the name `transcribedAudio` to the labeled signal set. Clicking OK returns you to Signal Analyzer. On the Signal table, select `transcribedAudio` and right-click to export it to a file called `Transcription.mat`.

Load the labeled signal set. The set has only one member. Get the names of the labels, and use the name to obtain and display the transcribed words.

```load Transcription ln = getLabelNames(transcribedAudio); v = getLabelValues(transcribedAudio,1,ln)```
```v=7×2 table ROILimits Value ____________ ________ 0.09 0.56 "oak" 0.59 0.97 "is" 1 1.78 "strong" 1.94 2.19 "and" 2.22 2.67 "also" 2.67 3.22 "gives" 3.25 3.91 "shade" ```

Rearrange the words so that the sentence reads "Oak gives shade, and also is strong." Plot the signal using a different color for each word.

```k = v([1 6:7 4:5 2:3],:); s = getSignal(transcribedAudio,1); sent = []; sgs = NaN(height(s),height(k)); lgd = []; for kj = 1:height(k) lm = length(sent); word = s.y(timerange(seconds(k.ROILimits(kj,1)),seconds(k.ROILimits(kj,2)))); sent = [sent;word]; sgs(lm+(1:length(word)),kj) = word; lgd = [lgd;(length(sent)-length(word)/2)/fs]; end sgs(length(sent)+1:end,:) = []; % To hear, type soundsc(sent,fs) plot((0:length(sgs)-1)/fs,sgs) text(lgd,-0.7*ones(size(lgd)),k.Value,'HorizontalAlignment',"center") axis tight```

`stt` Function: Locate and Identify Spoken Words

This function uses the IBM Watson Speech API and the Audio Toolbox `speech2text` extended functionality to extract spoken words from an audio file.

```function [labelVals,labelLocs] = stt(x,t,parentLabelVal,parentLabelLoc,varargin) aspeechObjectIBM = speechClient('IBM','timestamps',true,'model','en-US_NarrowbandModel'); fs = 1/(t(2)-t(1)); tixt = speech2text(aspeechObjectIBM,x,fs); numLabels = numel(tixt.TimeStamps{:}); labelVals = strings(numLabels,1); labelLocs = zeros(numLabels,2); for idx =1:numLabels labelVals(idx) = tixt.TimeStamps{:}{idx}{1}; labelLocs(idx,1) = tixt.TimeStamps{:}{idx}{2}; labelLocs(idx,2) = tixt.TimeStamps{:}{idx}{3}; end end```

Tips

• When saving labels, Signal Labeler converts all signals with time information to timetables. This conversion results in a deeper hierarchy of nested channels in the saved `labeledSignalSet`. See Signal Labeler Import and Export Behavior for more details.

• Signal Labeler averages channel colors when it renders labels corresponding to multichannel signals. For best results, customize the line color so that it is the same for all channels in a given signal. Perform the customization in Signal Analyzer before entering Signal Labeler.

Compare the label color for `diffr`, a signal whose channels have different colors, to the label color for `equal`, a signal whose channels all have the same color. Any labels for `equal` are rendered in the shade of blue that all the channels share. Labels for `diffr` are rendered in a shade of brown that does not match any of the channel colors.

• When you reduce the width of the Signal Labeler window, the attribute table under the time plot shows a horizontal scrollbar. In Apple macOS systems, the scrollbar is hidden by default. If you have a trackpad and scroll horizontally, the scrollbar appears. If you use a mouse and want the scrollbar to be visible always, you can modify the system behavior in the System Preferences.