# Abrupt changes in data; 'ischange', means, and 'stairs'

22 views (last 30 days)
noMathWiz on 10 Jul 2020
Edited: Matt J on 10 Jul 2020 at 22:04
I have multiple, large data sets that I need to acertain the means along certain linear regions. The regions are dileneated by relatively abrupt changes, as shown in the figure below that I manually marked up.
Ultimately I want a 7 element array of the means of each of these regions (aka the y-intercepts as shown). Thoughts? Other types of curve fittings that could help?
A sample data set is attached.
The below code not working for me, and I've tried a variety of max number of changes and threshold levels.
[TF,S1] = ischange(a, 'linear','MaxNumChanges',12);
plot(a, '*')
hold on
stairs(S1)

Matt J on 10 Jul 2020 at 20:48
Edited: Matt J on 10 Jul 2020 at 21:01
Here is a method using the Image Processing Toolbox (treating the signal as a 1D image, in other words).
w=5000;
b=movmedian(a,w);
cmax=movmax(b,[w,1]);
cmin=movmin(b,[1,w]);
q=cmax-cmin;
lmap=bwlabel(q<0.01);
result = regionprops('table',lmap,b,'MeanIntensity')
plot(a)
hold on
for i=1:size(result,1)
plot(xlim,result{i,1}*[1,1],'--')
end
hold off

Matt J on 10 Jul 2020
Edited: Matt J on 10 Jul 2020
Yes, you can use splitapply(@mean,data,G)
with G identifying the regions you wwant grouped together,

Show 1 older comment
Matt J on 10 Jul 2020
So, here is an example where I find the means of the first 4 and last 3 elements of the 1x7 vector data,
>> data=[10,20,30,40,50,60,70], G=[1 1 1 1,2 2 2]
data =
10 20 30 40 50 60 70
G =
1 1 1 1 2 2 2
>> splitapply(@mean,data,G)
ans =
25 60
As you can see, the G vector labels the first 4 elements of data with '1' and the final 3 elements with '2'.
but in my case I have indices that should not be included (the sloped rigions between average 1 and 2, between 2 and 3, etc.
To handle the unincluded regions, either remove them from 'data', or assign them a group label of their own. In the latter case, you can just disregard whatever mean value is calculated for the unwanted regions.
noMathWiz on 10 Jul 2020 at 19:52
Thanks for your help, but I think this is too much manual work (i.e. it requires me to manually find where the changes in data are, delete the data, then create G vectors of the same length).
Matt J on 10 Jul 2020 at 21:53
If you were to construct G manually, it would indeed be a lot of work, but the philosophy behind splitapply is that you would find some automated way to construct the group labels. Below is a method that makes use of group1s from the File Exchange.
w=5000;
b=movmedian(a,w);
cmax=movmax(b,[w,1]);
cmin=movmin(b,[1,w]);
q=cmax-cmin;
lmap=group1s(q<0.01);
result = splitapply(@mean, b, findgroups(lmap))
result(1)=[];
plot(a)
hold on
for i=1:size(result,1)
plot(xlim,result(i)*[1,1],'--')
end
hold off