Can I write a script that keeps rows that are within 5% and 10 cells above and below a specific cell?

The idea is to have a script that takes stable data eleminating the peaks and valleys.

I want to have that cell compared with the 10 before and after itself and if all this 21 values are within 5% then I can keep that data. That script should also be able to check all the rows of an excel spreadsheet.

Any suggestions and ideas are much appreciated! I am a newby trying to do advanced stuff :)

Answer 1

[Rik]

If you have a cell array you should convert it to a numeric type (e.g. a double).

You can find outliers by using the movmean function:

k=21;
outlier=abs((data./movmean(data, k))-1)>=0.05;

Now you have a logical array with the outliers. Do you want to remove them or replace them?

[Diego Dranuta]

i want to remove those rows that don't meet the criteria

[Rik]

Then data(outlier)=[]; should work for you.

[Diego Dranuta]

Thank you very much, For what I have seen this command compares the array to the 10th before and 10th after values. Is it possible to create any kind of loop where instead of comparing to 2 values it compares it to all 20th values? and if the given array meets the requirement of falling on a range of 5% along those 20 values then is useful information

[Rik]

movmean returns the moving window average, so it already considers all values. Please describe what exactly should happen with a segment of 21 values. Then we can help you do that, likely with something more efficient than a loop.

[Diego Dranuta]

I am trying to desinguish between transient and steady state data from an engine. Each of those "bins" of information need to be trated diferentyl. My idea was to first, clean up the data getting rid of the points that are way off the curve. I have done that using

Time = 1:length(PercentLoad);

window = 30;

meanPercentLoad = movmean(PercentLoad,window);

Now I need to distinguish the transient and the steady state. My definition would be that if a point in the Percent Load vs time graph is within 5% diference of 10 seconds before and 10 seconds after, it is steady. If not it is transient.

But as I said I am very new on matlab and coding so im figuer it out step by step.

Any help is much appreciated !!

[Rik]

I meant a numerical example. For a list of 21 values (10 before, 10 after, and the value itself), what do you want to happen exactly? And within 5% is not necessarily clear. For a 21 element example vector, please write some code that shows why you think my code only looks at 2 values, and what should happen instead.

[Diego Dranuta]

This excel sheet is representative of what I am trying to do. I need to separate those transient regions from those "steady" regions to analyse individually.

My idea to do so is to have a code that separates them by using this rule. If x has a max 5% difference with x-1,x-2,x-3,x-4,x-5,x-6,x-7,x-8,x-9,x-10 and x+1,x+2,x+3,x+4,x+5,x+6,x+7,x+8,x+9,x+10 then it is steady else is transient.

for expample is x-9 is 6% away from x then x is transient

Hope this gives you a better insight of what I am trying to do.

Thank you very much for your kind help!

[Rik]

So for each value within the window x/x_window must be between 0.95 and 1.05? If that is the case you could use something like the code below.

%your data and setting
data=[55;56;54;55;55;55;59;54;55;52;55;56;56;57;50;60;71;75;115;60;50;40;...
    23;20;25;30;40;50;55;56;55;44;55;55;55;55;56;54;57;54;54;56;58;54;...
    58;56;55;55;55;54;55;55;56;55;55;55;55;55;55;54;54;56;54;54;54;56;...
    55;54;58;58;57;58;56;54;55;55;55;55;55;56;55;55;55;54;55;55;55;56;...
    55;55;55;54;55;55;55;56;55;55;55;54;55;55;55;56;55;55;55;54;55;55;...
    55;55;57;55;55;55;57;55;55;55;56;56;57;56;57;56;58;57;56;55;56;56;...
    56;57;57;57;100;90;15;20;4;5;56;47;58;57;21;44;33;66;88;77;44;55;...
    66;23;55;54;56;57;58;98;54;2;42;52;5;44;44;56;87;45;15;35;8;55;65;...
    45;78;77;66;55;44;54;56;54;58;59;54;56;54;56;54;56;54;54;56;54;56;54];
k=21;
%%
%example data and setting
data=[100 100 100 106 100 98 100];
k=3;
%%
clc
L=false(size(data));
for n=( -(k-1)/2 ):( (k-1)/2 )
    if n==0,continue,end
    current=circshift(data,n);%shift the vector one position over
    ratio=data./current;
    p_diff=abs(ratio-1)*100;
    L=L | p_diff>5;%store all positions where the ratio is off
end
data(L)=[]

With the example data it is easy to see that a lot of values are getting removed if you do it like this. The reason is that an invalid value 'taints' the surrounding values. In this case the 106 value causes the 100 on either to be removed as well.