Is it possible to detect and replace abnormal/wrong numbers in an array?

Rikke (view profile)

on 20 Mar 2019
Latest activity Commented on by Chris Turnes

on 21 Mar 2019

Kevin Phung (view profile)

I have a dataset which is an array containing values, and which are plotted in the figure below. A section of the array for the figure is: A = [.... 25.9, 25.9, 26.2, 27, 28, 29, 29.3, 29.6, 3, 30.4, 30.5, 30.4, 30.3, 30.3, ....]; Here is number 3 an abnormality in the array. Is it possible to detect these abnormal numbers and replace them with the average value of the number before and after? So for the number 3 in A I want it to be (29.6+30.4)/2. Kevin Phung (view profile)

on 20 Mar 2019

A = [25.9, 25.9, 26.2, 27, 28, 29, 29.3, 29.6, 3, 30.4, 30.5, 30.4, 30.3, 30.3]
abnorm =mean(A) - 3*std(A); % you can set however many standard deviations to constitute 'abnormal'
n = find(A<abnorm); %locations of abnormals
for i = 1:numel(n)
if or(n(i) == 1, n(i)==numel(A)) %if first or last index, replace with NaN;
A(n(i)) = nan;
else
A(n(i)) = (A(n(i) - 1) + A(n(i) + 1)) / 2;
end
end
A(isnan(A)) = []; %remove the endpoints that are NaN
let me know if this works for you

Rikke

Rikke (view profile)

on 20 Mar 2019
I got the solution of correcting the velocity values by using almost all the code you sent, just adjusted the first code lines. I got the 'isoutlier' from the link Walter sent written below this answer. Thanks for your help!
TF = isoutlier(A,'movmedian',5);
abnorm = 0;
n = find(TF>abnorm); %locations of abnormals
for i = 1:numel(n)
if or(n(i) == 1, n(i)==numel(A)) %if first or last index, replace with NaN;
A(n(i)) = nan;
else
A(n(i)) = (A(n(i) - 1) + A(n(i) + 1)) / 2;
end
end
A(isnan(A)) = []; %remove the endpoints that are NaN
Kevin Phung

Kevin Phung (view profile)

on 20 Mar 2019
happy to help!
edit: if number of lines of code is important to you at all, you can actually condense the first 3 lines to one
n = find(isoutlier(A,'movmedian',5)); %locations of abnormals
for i = 1:numel(n)
if or(n(i) == 1, n(i)==numel(A)) %if first or last index, replace with NaN;
A(n(i)) = nan;
else
A(n(i)) = (A(n(i) - 1) + A(n(i) + 1)) / 2;
end
end
A(isnan(A)) = []; %remove the endpoints that are NaN
Rikke

on 20 Mar 2019
Aha, perfect!

on 20 Mar 2019

Rikke

on 20 Mar 2019
Thanks!
Chris Turnes

Chris Turnes (view profile)

on 21 Mar 2019
You may also be interested in the filloutliers function, which lets you not only identify outliers but replace them as well.