Efficient Moving Quantile Function
22 views (last 30 days)
Matthew Casiano on 11 Jan 2020
I am looking for thoughts on a recipe to perform a moving quantile in Matlab. Essentially I am trying to obtain the 98% value, for example, of a window that is instantaneously tracked through a large data set. I have tried looping in the quantile function, but this is extremely slow. I have had reasonable success using the sort function each time the window is moved, and then selecting the value based on the index that is 98% of the window size - see script below. But it is still several orders of magnitude slower than the built-in movmedian function. I don't see how a moving quantile should be much different then a moving median, just that the sorted value selected is not at 50%.
My thoughts are that after the initial window is sorted, an insertion sort algorithm is most appropriate since only a single value comes into the window each time the window moves 1 point. But really I am guessing at what is inside the movmedian function. I am not a programmer guy, so any advice here or examples on how to code an efficient moving quantile function is appreciated.
The example below is relatively quick with the given sampling rate, however in my example the sampling rate is 100,000 sps and takes much longer.
% Moving Quantile
fs=1000; % sampling rate (sps)
timeData=0:1/fs:40; % time (sec)
Data=rand(1,length(timeData)); % data (EU)
window=6300; % number of points used in 1 pt sliding window (pts)
dt_wnd=(window-1)/fs; % window time span (sec)
time_wnd=timeData(1)+dt_wnd/2:1/fs:timeData(end)-dt_wnd/2; % time vector, sliding window center points (sec)
Ind=floor(0.98*window); % index at 98% quantile
for j=1:length(time_wnd) % loop through all windows moving by 1 pt
SortedData=sort(Data(j:j+window-1)); % sorts data in the jth window
Val(j)=SortedData(Ind); % value at index
axis([0 40 0.9 1.0])
Eike Petersen on 11 Dec 2020
My guess is that your guess is pretty good. :-)
I just hand-implemented a version of this with a naive insertion sort algorithm, and it's roughly 50-80x slower than medfilt1. I'm not sure you can get much quicker without switching to C / a mex-file.
You can find it here; it works for N-D arrays, with NaNs, and with different padding strategies: https://de.mathworks.com/matlabcentral/fileexchange/84200-movquant