You are now following this question
- You will see updates in your followed content feed.
- You may receive emails, depending on your communication preferences.
I tried Hampel filtering to discard the bad data, but it was not fruitful. How to avoid/filter
2 views (last 30 days)
Show older comments
Dear all,
I have attached the data plot. I tried Hampel filtering to discard the bad data, but it was not fruitful. How to avoid/filter
this bad data ?
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/968800/image.jpeg)
10 Comments
Rik
on 20 Apr 2022
The function KSSV suggested should do the trick in this case. Feel free to post a follow up comment if you have questions.
Ismita
on 20 Apr 2022
when I am running the following code in Matlab online it shows the plot, but it is not working while running in installed Matlab. Could you please suggest me the solution? Thanks again.
"Code:
x = 1:10;
A = [60 59 49 49 58 200 61 57 48 58];
[TF,L,U,C] = isoutlier(A);
plot(x,A,x(TF),A(TF),'x',x,L*ones(1,10),x,U*ones(1,10),x,C*ones(1,10))
legend('Original Data','Outlier','Lower Threshold','Upper Threshold','Center Value')
"
Mathieu NOE
on 21 Apr 2022
if you have an older release the second code example is for you
A = repmat([60 59 49 49 58 200 61 57 48 58],[1 10]);
nA = numel(A);
x = 1:nA;
% %% original code
% [TF,L,U,C] = isoutlier(A);
% figure(1),plot(x,A,x(TF),A(TF),'dr',x,L*ones(1,nA),x,U*ones(1,nA),x,C*ones(1,nA))
% legend('Original Data','Outlier','Lower Threshold','Upper Threshold','Center Value')
%% alternative method
upper_threshold = 5; % above mean smoothed curve
lower_threshold = 25; % below mean smoothed curve
% apply smoothing (here twice)
N = 15;
B = myslidingavg(A, N); % or use smooth or smoothdata depending of what you have
B = myslidingavg(B, N); % or use smooth or smoothdata depending of what you have
TF = (A>B+upper_threshold) | (A<B-lower_threshold);
figure(2),plot(x,A,x,B,x,B-lower_threshold,x,B+upper_threshold,x(TF),A(TF),'dr')
legend('Original Data','Smoothed Data','Lower Threshold','Upper Threshold','Outlier')
function out = myslidingavg(in, N)
% OUTPUT_ARRAY = MYSLIDINGAVG(INPUT_ARRAY, N)
%
% The function 'slidingavg' implements a one-dimensional filtering, applying a sliding window to a sequence. Such filtering replaces the center value in
% the window with the average value of all the points within the window. When the sliding window is exceeding the lower or upper boundaries of the input
% vector INPUT_ARRAY, the average is computed among the available points. Indicating with nx the length of the the input sequence, we note that for values
% of N larger or equal to 2*(nx - 1), each value of the output data array are identical and equal to mean(in).
%
% * The input argument INPUT_ARRAY is the numerical data array to be processed.
% * The input argument N is the number of neighboring data points to average over for each point of IN.
%
% * The output argument OUTPUT_ARRAY is the output data array.
if (isempty(in)) | (N<=0) % If the input array is empty or N is non-positive,
disp(sprintf('SlidingAvg: (Error) empty input data or N null.')); % an error is reported to the standard output and the
return; % execution of the routine is stopped.
end % if
if (N==1) % If the number of neighbouring points over which the sliding
out = in; % average will be performed is '1', then no average actually occur and
return; % OUTPUT_ARRAY will be the copy of INPUT_ARRAY and the execution of the routine
end % if % is stopped.
nx = length(in); % The length of the input data structure is acquired to later evaluate the 'mean' over the appropriate boundaries.
if (N>=(2*(nx-1))) % If the number of neighbouring points over which the sliding
out = mean(in)*ones(size(in)); % average will be performed is large enough, then the average actually covers all the points
return; % of INPUT_ARRAY, for each index of OUTPUT_ARRAY and some CPU time can be gained by such an approach.
end % if % The execution of the routine is stopped.
out = zeros(size(in)); % In all the other situations, the initialization of the output data structure is performed.
if rem(N,2)~=1 % When N is even, then we proceed in taking the half of it:
m = N/2; % m = N / 2.
else % Otherwise (N >= 3, N odd), N-1 is even ( N-1 >= 2) and we proceed taking the half of it:
m = (N-1)/2; % m = (N-1) / 2.
end % if
for i=1:nx, % For each element (i-th) contained in the input numerical array, a check must be performed:
dist2start = i-1; % index distance from current index to start index (1)
dist2end = nx-i; % index distance from current index to end index (nx)
if dist2start<m | dist2end<m % if we are close to start / end of data, reduce the mean calculation on centered data vector reduced to available samples
dd = min(dist2start,dist2end); % min of the two distance (start or end)
else
dd = m;
end % if
out(i) = mean(in(i-dd:i+dd)); % mean of centered data , reduced to available samples at both ends of the data vector
end % for i
end
Answers (0)
See Also
Categories
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!An Error Occurred
Unable to complete the action because of changes made to the page. Reload the page to see its updated state.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)