Removing row from a matrix if value in row < previous value
13 views (last 30 days)
Show older comments
I have data that when sorted based on column 1, column 6 should be in ascending order, so n should >n-1. However the program that outputs this data creates incorrect values for column 6 that are much lower than they should be for a few data points, then they return to normal. I want to remove these values (or make n = n-1 if n<n-1)
At the moment I'm doing this with an if command in excel after sorting the file, but with 100's of files this is incredibly tedious.
I've tried the below, choosing a threshold value for the "incorrect" data to be below, however the threshold changes between files so this is no use.
datas = sortrows(data,-1) %sorts data by descending distance data
rowremove = datas(:,6)<=athres %removes row if area data is <= threshold - corrects for "bad" area data
datas(rowremove,:) = []
Is this possible in matlab?
2 Comments
Answers (1)
dpb
on 16 Nov 2016
Jan's point is valid; if the bad data are excessively corrupted removing the offending rows may only create a new set of offending values. If, otoh, the overall slope is large enough and the erroneous values not too far of, then perhaps
ix=[true; diff(x(:,6))>0];
x=x(ix,:);
may work. If the above causes the issue that you then have a new set of offenders, then you'll likely have to use the above to
- locate the first of each offending section
- search from that point to the next
- replace/remove those sections before processing next
A sample dataset (relatively short) but showing typical result would be helpful, probably. (Only need the two columns; the additional are immaterial to the problem of selection/retention/disposal).
See Also
Categories
Find more on Descriptive Statistics in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!