Remove Outliers from Wind Speed Data

This example shows how to read data from a public channel, modify the data, and display select elements. In the example, you modify one of the code templates provided by the MATLAB Analysis and MATLAB Visualizations apps. The example uses data from ThingSpeak channel 12397, which collects weather data from an Arduino® based weather station in Natick, MA.

Create a MATLAB Analysis Script from Template Code

To detect and remove outliers in wind speed data from the Natick weather station, you can write a MATLAB® script using a code template.

Go to the Apps tab in ThingSpeak and select MATLAB Analysis. Click New, select Remove outliers from wind speed data, and click Create.

Analyze Your Data

The MATLAB Code field is prepopulated with code to detect and remove outliers from wind speed data over the past six hours.

1) Set the variables for communicating with ThingSpeak. readChannelID is the channel ID for the public channel that collects data from the weather station. windSpeedFieldID is the field in the channel that contains wind speed values. Assign a value to readAPIkey only if you are reading data from a private channel. The weather station is public, so for this example, do not set readAPIkey.

readChannelID = 12397;
windSpeedFieldID = 2;
readAPIKey = '';

2) Read wind speed values and timestamps from the past six hours using the thingSpeakRead function.

[windSpeed,timeStamp] = thingSpeakRead(readChannelID,'fields',windSpeedFieldID,'NumMinutes',360,'ReadKey',readAPIKey);

3) Check for outliers in the wind speed data using the MATLAB isoutlier function. With the default settings, this function calculates whether a value is more than three scaled median absolute deviations away from the median of the input data set. You can adjust the input arguments to customize your outlier results. Identify the indices of outlier data points and the indices of clean data points. Use these indices to select data points and timestamps corresponding to outlier data and clean data.

outlierDataIndex = isoutlier(windSpeed);
cleanDataIndex = ~outlierDataIndex;

outlierData = windSpeed(outlierDataIndex);
cleanData = windSpeed(cleanDataIndex);

outlierTimeStamps = timeStamp(outlierDataIndex);
cleanTimeStamps = timeStamp(cleanDataIndex);

4) Create timetables with outlier data and clean data, and display the result of the outlier data points.

outlierDataTable = timetable(outlierTimeStamps,outlierData);
cleanDataTable = timetable(cleanTimeStamps,cleanData);

display(outlierDataTable,'Outlier data');

  3×1 timetable

     outlierTimeStamps      outlierData
    ____________________    ___________

    03-Jun-2019 10:20:54         17    
    03-Jun-2019 13:26:14       16.6    
    03-Jun-2019 13:39:33       16.8    

Execute your code by clicking Save and Run. The Output field displays your results.

Write Data to a Channel

1) Store your clean data results by writing them to a private channel. To create a ThingSpeak channel, go to the Channels tab and select My Channels. Click New Channel. Select the corresponding check box, and enter these channel setting values:

• Name — Cleaned Wind Speed Measurements

• Field 1 — Wind speed (mph)

Click Save Channel.

2) In the MATLAB Code field, set the variables for writing to your private channel. Replace the given values for writeChannelID and writeAPIKey with your values. You can find the channel ID and API key in the Channel Info panel on the right side of the page.

% Replace with the ID of the channel to write data to.
writeChannelID = 17504;
% Enter the write API key between the ''.
writeAPIKey = '23ZLGOBBU9TWHG2H';

3) Write the clean wind speed readings with their respective timestamps to your channel.

thingSpeakWrite(writeChannelID,cleanData,'timestamp',cleanTimeStamps,'Writekey',writeAPIKey);

4) Execute your code by clicking Save and Run. The chart in your ThingSpeak channel is populated with time series data for wind speed without the calculated outliers. You can access your channel by clicking the channel link in the Channel Info panel on the right side of the page.

To download your data in CSV format, click the Data Export button or the Data Import / Export tab. To clear all saved data in your channel, click the Channel Settings tab.

Calculate Moving Mean

You can add code to the template to further analyze the wind speed data. Besides removing outliers, another method of smoothing out a dataset is to calculate the moving mean. In this approach, the mean of a group of local data points is calculated over a sliding window throughout the entire data set. Use the MATLAB movmean function with a sliding window of five minutes to smooth your wind speed data. This section is not included in the code template. You can include it in your code after calling thingSpeakRead.

smoothData = movmean(windSpeed,minutes(5),'SamplePoints',timeStamp);

To save your new data, write it to your ThingSpeak channel. Comment out the existing thingSpeakWrite function and save the new smoothed time series data to your channel.

thingSpeakWrite(writeChannelID,smoothData,'timestamp',timeStamp,'Writekey',writeAPIKey);

Toupdate your results, click Save and Run again.