how to calculate the standard deviation for each month in monthly time series data of 46 years?

bushra raza (view profile)

on 14 Jan 2019
Latest activity Commented on by bushra raza

on 20 Jan 2019

Stephan (view profile)

Hi,
i have a time series of monthly means for 46 years. its a univariate time series. i have shifted it to a timetable of size 557x1. since dat is from Jan 1971 - May 2017. it is attached here with.
i need to calculate the standard deviation for each month, using all the yeas in the data. my idea is to reshape the timetable so as to show years in rows and months in columns. so i tried to reshape this timetable to 46X12 sized timetable by using the following code, after wards, the resultant timetable 'A' woulld have 12 columns for each year. thus for each column of month , standard devviation may then be calculated.
but i am getting an error. please correct me, if i am wrong? or any better solution is also appreciated.
monthlymeans_ObsData=retime(ObservedData,'monthly',@nanmean);
x = monthlymeans_ObsData.timmendorf_time;
A = reshape(monthlymeans_ObsData,46,12);
%% i get the following error
% Error using tabular/reshape (line 155)
% Undefined function 'reshape' for input arguments of type 'timetable'.
mean_y = nanmean(A);
A_std = nanstd(A);
plot(x, mean_y, 'b', x, mean_y + A_std, 'r',x, mean_y - A_std, 'g');

Stephan (view profile)

on 16 Jan 2019
Edited by Stephan

Stephan (view profile)

on 17 Jan 2019

Hi,
the following code reshapes your table the way you want it:
monthlymeans_ObsData.Properties.RowTimes.Format = 'MMM';
% reshape the data as needed
waterlevel = nan(564,1);
waterlevel(1:557) = monthlymeans_ObsData.timmendorf_waterlevel;
waterlevel = reshape(waterlevel,12,47);
% New table
T = splitvars(table(waterlevel));
% Variable Names for new table - columns
years = cell(1,47);
for y = 1:47
years{y} = sprintf('Year_%d',1970+y);
end
T.Properties.VariableNames = years;
% Variable Names for new table - rows
months = cell(1,12);
month_names = string(monthlymeans_ObsData.Properties.RowTimes(1:12));
for m = 1:12
months{m} = sprintf('%s',month_names(m));
end
T.Properties.RowNames = months;
% clean up
clear waterlevel y years months month_names m ans
With this result you can access all data for the months May and June over all years that way:
T_may_june = T({'May', 'Jun'},:);
which results in a new table. Calculate mean and standard deviation for all years for may and june and append those values as columns:
T_may_june.mean = mean(T_may_june{:,:},2,'omitnan');
T_may_june.std = std(T_may_june{:,:},1,2,'omitnan');
If you like an array more you can use:
may_june = T_may_june{:,:};
Best regards
Stephan

bushra raza

bushra raza (view profile)

on 16 Jan 2019
Hi,
Thanx a lot Mr. Stephan for your reply. logically it sounds great as i wanted to solve.
i am using Matlab R2017b.
i tried it, i got this message "Undefined function or variable 'splitvars'."
i searched splitvar, and save this splitvar function in my workspace. and then again execute your advised code, its giving me this message
"Error using splitvar
Too many output arguments. "
any idea Sir?
thanx once again for your time and guidance.
Best Regards,
BR
Stephan

Stephan (view profile)

on 16 Jan 2019
Always a problem, if there is no Matlab Release provided... Im sure there is a smart solution. splitvars was introduced in R2018a. My solution is, that i will attach a .mat file containing the results as soon as i have access to my computer in a few hours...
EDIT:
.mat-file is attached at my answer now. Also note the useful answer from Akira Agata.

Akira Agata (view profile)

on 17 Jan 2019

If your goal is to calculate the standard deviation for each month, you don't need to reshape the data to 46X12.
The following is one possible solution for R2017b.
[group,month] = findgroups(monthlymeans_ObsData.timmendorf_time.Month);
stdVal = splitapply(@std,monthlymeans_ObsData.timmendorf_waterlevel,group);
tblResult = table(month,stdVal);
FYI: If you can update to R2018a or later, you can utilize groupsummary function to do this task.
monthlymeans_ObsData.month = monthlymeans_ObsData.timmendorf_time.Month;
tblResult = groupsummary(monthlymeans_ObsData,'month','std');

bushra raza

on 20 Jan 2019