Calculating per year slope using daily time series data

Dear all,

I have a daily time series data and I want to calculate per year slope from this. While calculating the slope, I wanted to select only those time-points which were at least 1-month apart compared to first timepoint. I tried to use "retime" function that would give me possible time-points, but the problem is Var1 data are duplicated in TT2 and placed in new time-points, but I want to choose the next available time-point from TT1.mat. For example, if my 30th day from 1st time point (-10 days) is 20th day and there is no data for 20th day in TT1.mat, I will choose next available timepoint from TT1.mat i.e., 26th day. So my questions are:

1. How can I choose actual time-points from TT1.mat that are at 1, 2, 3, 4, 5......month distant from first timepoint?
2. How can I select 12-month timepoint (from first timepoint) as the cut-off timepoint?
3. Is converting "days" to "years" and using them as X variable the correct way to calculate slope per year? Or, do I have to do divide it by the difference of last time point and first time point?

dt = days(30);
TT2 = retime(TT1,'regular','next','TimeStep',dt); %please see TT1.mat in attachment
T = timetable2table(TT2); %converting to table
[C,IA,IC] = unique(T.Var1,'first'); %take first instance for duplicate data
B = sortrows(T(IA,:)); %sort ascending
B=B(~any(ismissing(B),2),:); %remove missing or NaN values
daynumbers = floor(datenum(B.Time)); %convert to day numbers

non_zero_X=[daynumbers/365]; %converting daynumbers to yearnumber
non_zero_Y=[B.Var1]; %eGFR values

M=polyfit(non_zero_X,non_zero_Y,1);
y1 = polyval(M,non_zero_X);

cc(m,1)=M(1); %raw slope
cc(m,2)=M(2); %raw intercept

[Peter Perkins]

There are a couple things not clear in your question that you are going to have to explain:

Not at all clear what you are trying to achieve with retime.

Does annual slope mean "slope for each year", or does it mean "slope for the one year period following each time point"? It's especially unclear given that your sample data runs for only 84 days, has duration timestamps that are not even regular, and has repeated time points.

What does "actual time-points that are at least 1-month apart compared to first timepoint" mean.

To use polyval, you'll need to convert your timestamps into numeric values, so you are on the right track there. I recommend not using datenum, for one thing, such large numbers don't help with accuracy. I would say use the day function on your datetimes to get day of year, but you don't seem to have datetimes. And using hard-coded 365 is a bug about 25% of the time.

[Anwar S]

Hi Peter,

Thanks so much for responding. I have updated the question. But I think the following will clarify more:

I want to choose time-points from TT1.mat (updated), which are at least 1-month apart. To choose every 1-month time-point, I used retime so that I can create a matrix of possible time-points. Now, all possible time-points would not have data in TT1.mat, so I would choose the next available time-point from TT1.mat. For example, here my first time point is -10 days (10 days back from 0th day) and compared to that my 30th day is supposed to be at 20 days. But I have actual data at 18 days and 26 days. I want the algorithm to pick the next available time-point, not the nearest one, i.e., 26th day and its corresponding value. Consequently, the next time-point should be the 60th day from first time-point and supposed to be (-10+60) 50th day. But in actual I do not have data at exact 50th day in TT1.mat, I have it for 60 day. So I would choose 60th day and its corrsponding value and so on. So I want my final B matrix to look like this,

-10days 40.9563

26 days 43.9932

60 days 54.1012

.........and so on. That way I keep 1, 2, 3, 4-month distance from first time-point even though chosen time-points are not exactly 1-month apart.

From B matrix, I want to calculate the slope (which will have the unit "per day") and convert it in such a way so that the unit of the slope becomes "per year" (hence I divided the days by 365). I hope this clarifies your questions. Thanks!

Answer 1

[Mohith Kulkarni]

Here is the workaround to get your desired B matrix. This code runs a loop through the TT1 to find the next time stamp of the 30 day interval.

dur = -10:30:1660;
rows = zeros(length(dur),1);
currDurIdx = 1;
for idx = 1:size(TT1,1)
   if TT1.Time(idx) >= days(dur(currDurIdx))
       rows(currDurIdx) = idx;
       currDurIdx = currDurIdx +1;     
   end 
   if currDurIdx > length(dur)
       break
   end
end
B = timetable(TT1.Time(rows),TT1{rows,1})