Calculate the maximum and minimum values
Show older comments
Hello I have a big timetable I want to calculate the maximum and minimum values
Any ideas how to proceed?
1 Comment
othman warde
on 14 Feb 2023
Edited: Walter Roberson
on 14 Feb 2023
To determine the maximum and minimum values of every month and the corresponding dates on which these values occurred, you can use the resample function in MATLAB. Here's an example of how you can use it:.
% Convert the timetable to a timetable with monthly intervals
monthlyData = retime(x, 'monthly', 'mean');
% Extract the maximum and minimum values of each month
MaxTemp = resample(x.Var, 'monthly', 'max');
MinTemp = resample(x.Var, 'monthly', 'min');
% Find the dates corresponding to the maximum and minimum values
MaxTempDates = resample(x.date_time, 'monthly', @(x) x(find(x.Var == max(x.Var), 1))); MinTempDates = resample(x.date_time, 'monthly', @(x) x(find(x.Var == min(x.Var), 1)));
In the code above, retime is used to convert the original timetable to a new timetable with monthly intervals. Then, resample is used to extract the maximum and minimum values of each month. Finally, resample is used again to find the dates corresponding to the maximum and minimum values. The @(x) x(find(x.Var == max(x.Var), 1)) and @(x) x(find(x.Var == min(x.Var), 1)) functions are used to find the first occurrence of the maximum and minimum values in each month, respectively. The resulting MaxTempDates and MinTempDates variables will be timetables with the date of the maximum and minimum values for each month.
Accepted Answer
Star Strider
on 6 Jan 2023
0 votes
That dealt with the hour that the daily maximum occurred. It would be straightforward to change that code to do what you want to do here.
24 Comments
Ancalagon8
on 6 Jan 2023
It took a bit longer than I anticipated to adapt my previous code to this problem.
Try this —
DT = datetime('01-Dec-2022 00:00:00') + days(0:365).';
Temperature = sin(2*pi*(0:numel(DT)-1)/24-0.5).'*12+5 + randn(size(DT))*1.5;
TT1 = timetable(DT, Temperature)
figure
plot(TT1.DT, TT1.Temperature)
grid
MonthlyMax = retime(TT1, 'monthly', 'max')
DailyMax = retime(TT1, 'daily', 'max')
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthlySTD = retime(TT1, 'monthly', @std);
T2 = table(MonthlyMax.DT, MonthlyMax.Temperature, MonthlyMin.Temperature, MonthlySTD.Temperature, 'VariableNames',{'DT','Max','Min','StDv'})
Thresholdv = [16; 17; 18; 19; 20]; % Column Vector
for k = 1:numel(Thresholdv)
DaysMeetingThreshold(k,:) = nnz(T2.Max >= Thresholdv(k)); % Days Equal To Or Higher Than Thresholdv (°C)
end
ThresholdTable = table(Thresholdv, DaysMeetingThreshold)
[yd,md,dd] = ymd(DailyMax.DT); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyMax)
[ym,mm,dm] = ymd(MonthlyMax.DT(k));
Lv = ismember([yd,md], [ym,mm], 'rows');
idx = DailyMax.Temperature(Lv) == MonthlyMax.Temperature(k);
Mv = find(Lv);
Dv = find(idx);
HM = DailyMax(Mv(Dv),:);
MonthDayMax(k,:) = timetable2table(HM);
end
MonthDayMax % Table Of Day In Each Month At Which Maximum Temperature Occurred
.
There were duplicate maxima, and that caused the problem.
Changing this assignment to only select the first index:
HM = DailyMax(Mv(Dv(1)),:);
solved the problem.
I initially forgot the corresponding minima, so I added a separate loop for that. It is simply a duplicate of the first loop with appropriate changes.
Bothe the maxima and minima are now calculated.
% DT = datetime('01-Dec-2022 00:00:00') + days(0:365).';
% Temperature = sin(2*pi*(0:numel(DT)-1)/24-0.5).'*12+5 + randn(size(DT))*1.5;
% TT1 = timetable(DT, Temperature)
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'))
TT1 = LD.TT1
figure
plot(TT1.date_time, TT1.Temperature)
grid
% return
MonthlyMax = retime(TT1, 'monthly', 'max')
DailyMax = retime(TT1, 'daily', 'max')
DailyMin = retime(TT1, 'daily', 'min')
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthlySTD = retime(TT1, 'monthly', @std);
T2 = table(MonthlyMax.date_time, MonthlyMax.Temperature, MonthlyMin.Temperature, MonthlySTD.Temperature, 'VariableNames',{'date_time','Max','Min','StDv'})
Thresholdv = [16; 17; 18; 19; 20]; % Column Vector
for k = 1:numel(Thresholdv)
DaysMeetingThreshold(k,:) = nnz(T2.Max >= Thresholdv(k)); % Days Equal To Or Higher Than Thresholdv (°C)
end
ThresholdTable = table(Thresholdv, DaysMeetingThreshold)
[ydx,mdx,ddx] = ymd(DailyMax.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyMax)
[ym,mm,dm] = ymd(MonthlyMax.date_time(k));
Lv = ismember([ydx,mdx], [ym,mm], 'rows');
idx = DailyMax.Temperature(Lv) == MonthlyMax.Temperature(k);
Mv = find(Lv);
Dv = find(idx);
HM = DailyMax(Mv(Dv(1)),:);
MonthDayMax(k,:) = timetable2table(HM);
end
[ydn,mdn,ddn] = ymd(DailyMin.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyMin)
[ym,mm,dm] = ymd(MonthlyMin.date_time(k));
Lv = ismember([ydn,mdn], [ym,mm], 'rows');
idx = DailyMin.Temperature(Lv) == MonthlyMin.Temperature(k);
Mv = find(Lv);
Dv = find(idx);
HM = DailyMin(Mv(Dv(1)),:);
MonthDayMin(k,:) = timetable2table(HM);
end
MonthDayMax % Table Of Day In Each Month At Which Maximum Temperature Occurred
MonthDayMin % Table Of Day In Each Month At Which Minimum Temperature Occurred
Are those the desired results?
.
I thought you only wanted the daily max and min.
This tweak gives the day and hour for the max and min each month —
% DT = datetime('01-Dec-2022 00:00:00') + days(0:365).';
% Temperature = sin(2*pi*(0:numel(DT)-1)/24-0.5).'*12+5 + randn(size(DT))*1.5;
% TT1 = timetable(DT, Temperature)
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'))
TT1 = LD.TT1
figure
plot(TT1.date_time, TT1.Temperature)
grid
% return
MonthlyMax = retime(TT1, 'monthly', 'max')
MinutelyMax = retime(TT1, 'minutely', 'max')
MinutelyMin = retime(TT1, 'minutely', 'min')
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthlySTD = retime(TT1, 'monthly', @std);
T2 = table(MonthlyMax.date_time, MonthlyMax.Temperature, MonthlyMin.Temperature, MonthlySTD.Temperature, 'VariableNames',{'date_time','Max','Min','StDv'})
Thresholdv = [16; 17; 18; 19; 20]; % Column Vector
for k = 1:numel(Thresholdv)
DaysMeetingThreshold(k,:) = nnz(T2.Max >= Thresholdv(k)); % Days Equal To Or Higher Than Thresholdv (°C)
end
ThresholdTable = table(Thresholdv, DaysMeetingThreshold)
[yhx,mhx,dhx] = ymd(MinutelyMax.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyMax)
[ym,mm,dm] = ymd(MonthlyMax.date_time(k));
Lv = ismember([yhx,mhx], [ym,mm], 'rows');
idx = MinutelyMax.Temperature(Lv) == MonthlyMax.Temperature(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyMax(Mv(Dv(1)),:);
MonthDayMax(k,:) = timetable2table(HM);
end
[yhn,mhn,dhn] = ymd(MinutelyMin.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyMin)
[ym,mm,dm] = ymd(MonthlyMin.date_time(k));
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyMin.Temperature(Lv) == MonthlyMin.Temperature(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyMin(Mv(Dv(1)),:);
MonthDayMin(k,:) = timetable2table(HM);
end
MonthDayMax % Table Of Day In Each Month At Which Maximum Temperature Occurred
MonthDayMin % Table Of Day In Each Month At Which Minimum Temperature Occurred
I kept the previous code intact, and copied everything to here to tweak it.
EDIT — (7 Jan 2023 at 17:20)
Changed ‘Hourly’ to ‘Minutely’ to get accuracy to the minute.
.
Star Strider
on 7 Jan 2023
Edited: Star Strider
on 7 Jan 2023
As always, my pleasure!
Yes! Improved.
.
Thank you!
That depends on how ‘3 max values and the 3 min values’ are defined. For the entire year, use maxk and mink. The second output will be the indices, so to get the entire row, do something like this —
T = array2table(randn(10,2))
[Col1,idx] = maxk(T{:,1},3);
Tx3 = T(idx,:)
With respect to tweaking my code to do something similar, that could be possible, however not without a significant amount of recoding. The problem would be how to define the output table or timetable. That’s not immmediately obvious to me. The retime function would not work for this, however maxk and mink likely could.
retime(TT, 'minutely', @(x)maxk(x,3));
However that wouldn’t work in practice. The reason is that according to the documentation in the method section Aggregation Methods with respect to funciton handle methods, ‘Use the function specified by the function handle (for example, @std to calculate the standard deviation for the values in each time bin). func must return an output argument that is a scalar or a row vector, and must accept empty inputs.’ (emphasis added). So returning the 3 maxima would likely throw an error.
One way around that could be to first back up the original timetable, then run the code on a copy of it to first find the maximum for each month, save that, delete it from the timetable, run the code again, save that, delete the maxima, and repeat as long as necessary to return the desired numbers of maxima (or minima). The result would be as many output tables or timetables as required to get the required number of maxima (or minima).
.
Star Strider
on 8 Jan 2023
Thank you!
As always, my pleasure!
That was a shile ago and I’ve forgotten what we were doing with it.
Try something like this —
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
TT1 = LD.TT1
figure
plot(TT1.date_time, TT1.Temperature)
grid
Thresholdv = (9.8:0.1:10.4).'; % Column Vector
for k1 = 1:12
Lv = month(TT1.date_time) == k1; % Select Month
Month = retime(TT1(Lv,:),'daily','mean'); % Daily Mean Temperature
MMM{k1,:} = month(TT1(Lv,:).date_time(1),'shortname'); % For Row Names
for k2 = 1:numel(Thresholdv)-1 % Histogram Loop
Tempv = Month.Temperature;
DaysMeetingThreshold(k1,k2) = nnz((Tempv >= Thresholdv(k2)) & (Tempv < Thresholdv(k2+1))); % Days Equal To Or Higher Than Thresholdv (°C)
end
end
DaysThresholdTable = array2table(DaysMeetingThreshold, 'VariableNames',compose('%.2f',Thresholdv(1:end-1)), 'RowNames',[MMM{:}])
This calculates a histogram of sorts without calling any histogram functions. I changed the code a bit to fit the actual data.
.
Ancalagon8
on 26 Jan 2023
Star Strider
on 26 Jan 2023
As always, my pleasure!
Star Strider
on 27 Jan 2023
Perhaps —
MonthlyMean = retime(TT1,'monthly','mean')
That would seem to work (unless I am not understanding something).
Star Strider
on 27 Jan 2023
I thought you wanted the mean.
For that, probably:
MonthlyMax = retime(TT1, 'monthly', 'max')
MonthlyMin = retime(TT1, 'monthly', 'min')
Then do the monthly range calculation from those, similar to the ‘DailyRange’ calculation.
Star Strider
on 27 Jan 2023
I am a bit lost. The monthly range should be 12, not 365.
One way to do that would be to comkpare the month value in the ‘date_time’ variable with the months in a loop. Equate them, and then save those data to a different variable. Since I have no idea what you are doing at this point with the ‘MonthlyRange’ array, that is as precise as I can be. I hesitate to code with information I have no access to. There may be examples of that approach in code I have already written here.
.
Star Strider
on 1 Feb 2023
it seems that we did versions of that already. Isn’t it possible to just adapt those to the current problem? It would likely be necessary to do the timing separately for the max, min, range, and any others, each in different timetable arrays, similarly to the approach in my previous Comment.
They would of necessity each have to be different timetable arrays, because I seriously doubt that the same times would be applicable to the various max, min, range, and others. The way timetable arrays work, they could not be combined into one timetable, although it might be possible to put the verious dates, times, max, min, range, and other datetime-variable arrays in the same table, providing that they all had the same (e.g. 12) row lengths. That would have to be coded separately and they would first have to be copied to table arrays after all the timetable arrays were calcualted.
I’m even more impressed with the table and timetable array types than I was previously!
In order to make it easier to get the exact times of the various maxima, minima, and the rest, I created a function ‘MonthlyToMinutely’ for that. It takes the 'monthly' and 'minutely' aggretated timetable arrays as arguments, and then returns a timetable with the exact times during the year that those values occurred in each month. It also checks to be certain that the timetable arguments have the same variables. It only works with timetable arrays with one variable, however that is what we are currently dealing with.
Here is an example of it with the 'min' timetable arrays —
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
TT1 = LD.TT1
MinutelyMin = retime(TT1, 'minutely', 'min');
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthDayMin = MonthlyToMinutely(MonthlyMin,MinutelyMin)
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
I did not test it with the others, however it should work with them.
.
Star Strider
on 2 Feb 2023
Either save the function to a file called ‘MonthlyToMinutely.m’ (that is likely the best approach, and save it to a subdirectory on your MATLAB search path), or put it at the end of the script you are using it with, similar to what I did here. Saving it to a file is likely the easiest solution.
The code you would save to the ‘MonthlyToMinutely.m’ file is:
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
.
They are not the same when I run it —
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
TT1 = LD.TT1
MinutelyMax = retime(TT1, 'minutely', 'max');
MonthlyMax = retime(TT1, 'monthly', 'max');
MonthDayMax = MonthlyToMinutely(MonthlyMax,MinutelyMax)
MinutelyMin = retime(TT1, 'minutely', 'min');
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthDayMin = MonthlyToMinutely(MonthlyMin,MinutelyMin)
MinutelyMax = retime(TT1, 'minutely', 'max');
MonthlyMax = retime(TT1, 'monthly', 'max');
MonthDayMax = MonthlyToMinutely(MonthlyMax,MinutelyMax)
MinutelyMin = retime(TT1, 'minutely', 'min');
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthDayMin = MonthlyToMinutely(MonthlyMin,MinutelyMin)
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
The function appears to work correctly.
.
To calculate the ranges, it would likely be better to calculate something other than the the minutely ranges and then the maximum and minimum ranges from that. A range will not work for the ‘minutely’ values, since there is only one temperature recorded each minute.
There is simply no possible way to calculate a range from single values, since there is no variation in them.
A fix for that is to aggregate over small intervals, and for this I chose:
'Timestep',minutes(2)
in ‘MinutelyMin’ and ‘MinutelyMax’ and from those I calculated the ranges. Other intervals are of course possible. The ‘Monthly’ range values were calcualted from the ‘Minutely’ values, since it is higholy unlikely that a monthly range is going to be in any way equivalent to a range over 2 minutes (or whatever aggregation interval is used). The reason of course is that the ranges are unique values, and do not exist in the original data. They first must be calculated.
With those changes, it works for the ranges —
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
TT1 = LD.TT1
MinutelyMin = retime(TT1, 'regular', 'min', 'Timestep',minutes(2))
MinutelyMax = retime(TT1, 'regular', 'max', 'Timestep',minutes(2))
MinutelyRange = table(MinutelyMax.Temperature - MinutelyMin.Temperature, 'VariableNames',{'Temperature'});
MinutelyRange = addvars(MinutelyRange, MinutelyMin.date_time, 'Before','Temperature');
MinutelyRange.Properties.VariableNames = {'date_time','Temperature'};
MinutelyRange = table2timetable(MinutelyRange)
MonthlyRangeMin = retime(MinutelyRange, 'monthly', 'min')
MonthlyRangeMax = retime(MinutelyRange, 'monthly', 'max')
MonthlyRange = table(MonthlyRangeMax.Temperature - MonthlyRangeMin.Temperature, 'VariableNames',{'Temperature'});
MonthlyRange = addvars(MonthlyRange, MonthlyRangeMin.date_time, 'Before','Temperature');
MonthlyRange.Properties.VariableNames = {'date_time','Temperature'};
MonthlyRange = table2timetable(MonthlyRange);
MonthDayRange = MonthlyToMinutely(MonthlyRange,MinutelyRange)
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
This is the only possible way to calculate the ranges in order to conform to the other calculations.
.
Star Strider
on 3 Feb 2023
My pleasure!
The ‘MonthlyRangeMin’ and ‘MonthlyRangeMax’ default to midnight on the first of the month for the monthly values. That is simply how retime works. Similarly, the ‘Minutely’ values would default to zero seconds of the appropriate minute if seconds were reported.
The ‘MonthlyDayRange’ (or whatever you choose to call it, if different from my code) reports the dates and times —
MonthDayRange = 12×2 table
date_time Temperature
__________________ ___________
25-Jan-19 04:20:00 0.00549
03-Feb-19 21:26:00 0.01117
22-Mar-19 09:36:00 0.03645
19-Apr-19 16:10:00 0.08332
25-May-19 06:56:00 0.00787
29-Jun-19 18:16:00 0.06977
18-Jul-19 13:52:00 0.00531
22-Aug-19 08:36:00 0.00385
14-Sep-19 11:48:00 0.04743
04-Oct-19 04:10:00 0.00933
My ‘MonthlyToMinutely’ function turns out to be reasonably robust, so experiment with it with different arguments to get the result you want.
We used it earlier to get the hourly maxima and minima. The range values must be calculated from the 2-minute ranges (or whatever interval you choose) because the ranges are not an inherent part of the original data, that report single values only. The ranges must be calculated from them, although the accumulation interval for the ranges can be anything you want. I chose 2 minutes here because it is the shortest interval these data permit.
.
To the best of my knowledge, the aggregation time of the values for the range was not previously defined. For the daily range, this is straightforward.
Try this —
LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
TT1 = LD.TT1
% MinutelyMin = retime(TT1, 'regular', 'min', 'Timestep',minutes(2))
%
% MinutelyMax = retime(TT1, 'regular', 'max', 'Timestep',minutes(2))
%
% MinutelyRange = table(MinutelyMax.Temperature - MinutelyMin.Temperature, 'VariableNames',{'Temperature'});
% MinutelyRange = addvars(MinutelyRange, MinutelyMin.date_time, 'Before','Temperature');
% MinutelyRange.Properties.VariableNames = {'date_time','Temperature'};
% MinutelyRange = table2timetable(MinutelyRange)
DailyMin = retime(TT1, 'daily', 'min')
DailyMax = retime(TT1, 'daily', 'max')
DailyRange = table2timetable(table(DailyMin.date_time, DailyMax.Temperature - DailyMin.Temperature, 'VariableNames',{'date_time','Temperature'}))
MonthlyRangeMin = retime(DailyRange, 'monthly', 'min')
MonthlyRangeMax = retime(DailyRange, 'monthly', 'max')
% MonthlyRange = table(MonthlyRangeMax.Temperature - MonthlyRangeMin.Temperature, 'VariableNames',{'Temperature'});
% MonthlyRange = addvars(MonthlyRange, MonthlyRangeMin.date_time, 'Before','Temperature');
% MonthlyRange.Properties.VariableNames = {'date_time','Temperature'};
% MonthlyRange = table2timetable(MonthlyRange)
MonthDayRangeMax = MonthlyToMinutely(MonthlyRangeMax,DailyRange) % Output: Day In Month With Maximum Range
MonthDayRangeMin = MonthlyToMinutely(MonthlyRangeMin,DailyRange) % Output: Day In Month With Mionimum Range
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT) % FUNCTION: 'MonthlyToMinutely'
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
The times are all at midnight (00:00:00) because the ranges are now daily. The retime function defaults to midnight for daily aggregation.
And again, ‘MonthlyToMinuitely’ works here and gives the correct result!
.
Star Strider
on 3 Feb 2023
For a daily range (over 24 hours), the hour and minute are not defined. You can calculate a daily maximum and daily minimum and get those times (I believe we already did that a while ago), however they will by definition not be the same times, so there will not be any one specific time for the range.
Probably the best you can do is to store the times for the maximum and minimum for a specific day and the times they occurred, and save them each in single rows for each day (and the range, if desired, as a separate variable) in a separate table (not a timetable, since it requires a single time vector). We already did those calculations, and I have examples of how to create that sort of table in the code I already posted, although not in that exact format.
Because you wanted the hour and minute, I initially defined the range over consecutive 2-minute intervals.
I am slightly lost. I have no idea where we are with this data set.
My guess is that we’re doing this —
LD = load(websave('T','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1289405/T.mat'));
TT1 = LD.T
% return
% LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
% TT1 = LD.TT1
MinutelyMax = retime(TT1, 'minutely', 'max');
MonthlyMax = retime(TT1, 'monthly', 'max');
MonthDayMax = MonthlyToMinutely(MonthlyMax,MinutelyMax)
MinutelyMin = retime(TT1, 'minutely', 'min');
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthDayMin = MonthlyToMinutely(MonthlyMin,MinutelyMin)
MinutelyMax = retime(TT1, 'minutely', 'max');
MonthlyMax = retime(TT1, 'monthly', 'max');
MonthDayMax = MonthlyToMinutely(MonthlyMax,MinutelyMax)
MinutelyMin = retime(TT1, 'minutely', 'min');
MonthlyMin = retime(TT1, 'monthly', 'min');
MonthDayMin = MonthlyToMinutely(MonthlyMin,MinutelyMin)
function ExactTime = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
I have no idea what the units are here, however it seems to work when I run it.
There may not be unique values in each daily data set for the maxima and minima. My code defaults to whatever retime does in those situations. In my ‘MonthlyToMinutely’ function, the ‘idx’ vector is a logical vector, and could return multiple results. I solve that problem by returning the first result (the ‘HM’ assignment in the loop) rather than all of them, because returning all of them would disrupt the table result. It allows for only one result per row. Returning all of them causes problems with the table format.
.
It seems to me that would simply be this —
LD = load(websave('T','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1289405/T.mat'));
TT1 = LD.T
% return
% LD = load(websave('dataset','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1255052/dataset.mat'));
% TT1 = LD.TT1
MinutelyMax = retime(TT1, 'minutely', 'max');
DailyMax = retime(TT1, 'daily', 'max');
DayMinuteMax = MonthlyToMinutely(DailyMax,MinutelyMax);
DayMinuteMax
MinutelyMin = retime(TT1, 'minutely', 'min');
DailyMin = retime(TT1, 'daily', 'min');
DayMinuteMin = MonthlyToMinutely(DailyMin,MinutelyMin);
DayMinuteMin
% G = findgroups(day(AllMin.date_time),month(AllMin.date_time),hour(AllMin.date_time),minute(AllMin.date_time),AllMin{:,1});
% MinutelyMax = retime(TT1, 'minutely', 'max');
% DailyMax = retime(TT1, 'daily', 'max');
% DayMinujteMax = MonthlyToMinutely(DailyMax,MinutelyMax)
%
% MinutelyMin = retime(TT1, 'minutely', 'min');
% DailyMin = retime(TT1, 'daily', 'min');
% DayMinuteMin = MonthlyToMinutely(DailyMin,MinutelyMin)
function [ExactTime,AllMatches] = MonthlyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
Lv = ismember([yhn,mhn], [ym,mm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
end
As I mentioned, my ‘MonthlyToMinutely’ function returns the first instance of the match that it finds, if there are more than one. This preserves the integrity of the ‘ExactTime’ output table.
I changed it to return multiple times for the same values as the second output, if the variable values are not unique. That works for the minimum values, however it’s returning anomalous results for the maximum values. I’ll work on it offlline and include it when I understand what the problem is.
EDIT — Forgot to run the code before I submitted it.
.
Only one max value per day appears in the ‘ExactTime’ result, although there can be more than one detected. My ‘MonthlyToMinutely’ function returns the first one it finds.
The problem was the resolution in the two arrays. For the time being, I’m using two different functions. I’ll work on the function to add an option (extra arguments) to make it more flexible. It will likely take some time to get that working properly. In the interim, the differences are explained by the code in the different functions, specifically in the ‘Lv’ and ‘idx’ assignments.
Try this —
LD = load(websave('T','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1289405/T.mat'));
TT1 = LD.T
MinutelyMax = retime(TT1, 'minutely', 'max');
DailyMax = retime(TT1, 'daily', 'max');
[DayMinuteMax,AllMax] = DailyToMinutely(DailyMax,MinutelyMax);
DayMinuteMax
AllMax = cat(1,AllMax{:})
MinutelyMin = retime(TT1, 'minutely', 'min');
DailyMin = retime(TT1, 'daily', 'min');
[DayMinuteMin,AllMin] = DailyToMinutely(DailyMin,MinutelyMin);
DayMinuteMin
AllMin = cat(1,AllMin{:})
function [ExactTime,AllMatches] = DailyToMinutely(MonthlyTT,MinutelyTT)
MinutelyVN = MinutelyTT.Properties.VariableNames;
MonthlyVN = MonthlyTT.Properties.VariableNames;
if ~strcmp(MinutelyVN{:}, MonthlyVN{:})
ExactTime = timetable();
fprintf('Variable names %s and %s in the argument timetables are not the same.',MonthlyVN{:},MinutelyVN{:})
return
end
[yhn,mhn,dhn] = ymd(MinutelyTT.date_time); % Year, Month, Day Of Daily Maxima
for k = 1:numel(MonthlyTT)
[ym,mm,dm] = ymd(MonthlyTT.date_time(k)); % Year, Month, Day Of Monthily Maxima
% YrMoDa = [ym mm dm]
Lv = ismember([yhn,mhn,dhn], [ym,mm,dm], 'rows');
idx = MinutelyTT{:,1}(Lv) == MonthlyTT{:,1}(k);
% LvSize = nnz(Lv)
Mv = find(Lv);
Dv = find(idx);
HM = MinutelyTT(Mv(Dv(1)),:);
AllHM{k,:} = MinutelyTT(Mv(Dv),:);
MonthDayTT(k,:) = timetable2table(HM);
end
ExactTime = MonthDayTT;
AllMatches = AllHM;
end
.
More Answers (0)
Categories
Find more on Tables in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
