How to use datetime with leap second strings?

90 views (last 30 days)
James Tursa
James Tursa on 16 Nov 2024 at 4:23
Moved: James Tursa on 23 Nov 2024 at 1:43
Suppose we have the following date time string:
ds1 = "30-Jun-1972 23:59:59.5";
You can convert that to a datetime easily:
datetime(ds1)
ans = datetime
30-Jun-1972 23:59:59
But now suppose you have a string that is in the middle of a leap second:
ds2 = "30-Jun-1972 23:59:60.5";
MATLAB won't convert this:
try
datetime(ds2)
catch
disp('Error')
end
Error
So apparently the seconds value >= 60 is causing this. But what if I tell it to use leap seconds? Still doesn't work:
try
datetime(ds2,'TimeZone','UTCLeapSeconds')
catch
disp('Error')
end
Error
This is disappointing, and quite frankly inconsistent with other ways of creating a datetime within a leap second. E.g., this works fine:
datetime(1972,6,30,23,59,60.5,'TimeZone','UTCLeapSeconds')
ans = datetime
1972-06-30T23:59:60.500Z
So, is there a way to coax datetime( ) into reading a date time string in the middle of a leap second? And at the same time retain full precision of the string? I would like to avoid manually parsing this myself if possible, but currently see no other way. E.g., the precision loss issue with string input illustrated:
format longg
ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
d3 = datetime(ds3+" "+dt3)
d3 = datetime
01-Jan-2024 23:59:01
d3.Second
ans =
1.234567891
d3.Second == 1.2345678912345 % LOSES PRECISION!
ans = logical
0
d4 = datetime(ds3) + duration(dt3)
d4 = datetime
01-Jan-2024 23:59:01
d4.Second
ans =
1.23456789124012
d4.Second == 1.2345678912345 % LOSES PRECISION!
ans = logical
0
You can see the loss of precision datetime has when reading strings in general. The duration class has a similar issue. So even if datetime could handle seconds >= 60, there would be a precision loss issue. I would like to avoid that.
At the moment, the only solution I see is to manually parse out the seconds and then piece things together. E.g.,
dt3a = "23:59:00";
dt3b = "00:00:01.2345678912345";
d5 = datetime(ds3) + duration(dt3a) + duration(dt3b)
d5 = datetime
01-Jan-2024 23:59:01
d5.Second
ans =
1.2345678912345
d5.Second == 1.2345678912345
ans = logical
1
Even this approach has problems with duration in the middle of a leap second because it can't handle string seconds >=60 for some reason:
seconds(60.5) % this is OK
ans = duration
60.5 sec
duration("00:00:60.5") % but not this!
Error using duration (line 126)
Could not recognize the format of '00:00:60.5'.
Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.
So to handle the leap second issue I would be forced to read the seconds manually instead of using duration. (Why???)
I am hoping for a simpler way.

Answers (2)

Shivam Gothi
Shivam Gothi on 17 Nov 2024 at 8:58
As per my understanding, you are trying to save leap-seconds in "datetime" object, in such a way that it does not looses precision. You want to pass the date-time string as an input argument to the "datetime" function.
I also faced similiar issue, but tried to resolve it by using the following work-around.
The documentation of "datetime" function (https://www.mathworks.com/help/matlab/ref/datetime.html#d126e365305) say that:
While passing character string as an input argument to the "datetime" function, we can specify the precision upto 9 decimal places by specifing an additional input argument "infmt", as demonstrated below:
DT_string= "2024-01-01T23:59:01.2345678912345Z"; %Same date and time as used in the above question
t = datetime(DT_string,'InputFormat','uuuu-MM-dd''T''HH:mm:ss.SSSSSSSSS''Z')
t = datetime
01-Jan-2024 23:59:01
format longg;
t.Second
ans =
1.234567891
This says that the maximum precision available is "9". Therefore, "1.2345678912345" is getting truncated to "1.234567891" in your case. Even if we do not explicitely specify the "inputFormat", it will automatically truncate to the maximum 9 digits after decimal.
Therefore, if you want to retain the precision, the below given command works perfectly.
t=datetime(1972,1,1,23,59,01.2345678912345,'TimeZone','UTCLeapSeconds') % same date and time entered manually
t = datetime
1972-01-01T23:59:01.234Z
t.Second %This is not truncated
ans =
1.2345678912345
In this case, instead of passing the date-time string, you have to manually enter the the values of year, months, days and time. This is not favourable as mentioned by you.
I am suggesting one of the possible work-around which I implemented in order to pass date-time string and store it in a "datetime" object which supports leap seconds as well as retains precision.
I have made a user defined function "DateTime" which takes the date-time string and returns a "datetime" object. It is demonstrated below:
format longg
ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
stringg=(ds3+" "+dt3);
t=DateTime(stringg)
t = datetime
2024-01-01T23:59:01.234Z
t.Second %this does not losses precision
ans =
1.2345678912345
function out=DateTime(DT_string)
arguments
DT_string string
end
C = textscan(DT_string,'%s %s'); %gives (1 x 2) cell array consisting of date and time
date_string=string(C{1});
time_string=string(C{2});
Date=textscan(date_string,'%f %s %f','delimiter','-'); %save the day, month, year in cell array
Time=textscan(time_string,'%f %f %f','delimiter',':'); %Save hour, minutes and seconds in celll array
DT_object=datetime(date_string);
DT_object.Hour = Time{1}; %extract the hour, minutes and seconds and save them in variable
DT_object.Minute = Time{2};
DT_object.Second = Time{3};
DT_object.TimeZone='UTCLeapSeconds';
out=DT_object;
end
I hope you find this information useful !!

Peter Perkins
Peter Perkins on 21 Nov 2024 at 21:25
Parsing timestamps with leap seconds
At the moment, con9verting from text to the UTCLeapSeconds "time zone" requires a specifc (ISO) format. Your try/catch was hiding the err msg that isn't exactly spot on but probably would have helped figure out what was happening:
ds2 = "30-Jun-1972 23:59:60.5"
ds2 = "30-Jun-1972 23:59:60.5"
try, datetime(ds2), catch, disp(lasterr); end
Error using datetime (line 257) Could not recognize the date/time format of '30-Jun-1972 23:59:60.5'. You can specify a format using the 'InputFormat' parameter. If the date/time text contains day, month, or time zone names in a language foreign to the 'en_US' locale, those might not be recognized. You can specify a different locale using the 'Locale' parameter.
ds3 = "1972-06-30T23:59:60.5Z"
ds3 = "1972-06-30T23:59:60.5Z"
datetime(ds3,TimeZone="UTCLeapSeconds")
ans = datetime
1972-06-30T23:59:60.500Z
There is work to relax that restriction, but for now that's what you have to do. I'd be curious to hear where you are getting timestamps that honor leap seconds but are in something other than an ISO format.
Parsing duration timestamps
try, duration("00:00:60.5"), catch, disp(lasterr); end
Error using duration (line 126) Could not recognize the format of '00:00:60.5'. Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.
This does not work because duration is not a time of day. It is an elapsed time. The concept of leap seconds doesn't apply here. A duration can certainly hold 60.5 seconds, but to parse a "timer format" string for 60.5 seonds, it'd have to be
duration("00:01:00.5",Format="hh:mm:ss.SSS")
ans = duration
00:01:00.500
Not sure but I suspect you are trying to use duration to parse the time of day portion of a datetime timestamp. That isn't gonna work for a leap second. Not sure what you are using this for, but I bet there's another way to do what you need.
Precision issues
I think the doc is pretty clear that only 9 fractional second digits are honored when converting from text, and only 9 can be displayed. So
ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
d3 = datetime(ds3+" "+dt3,Format="dd-MMM-uuuu HH:mm:ss.SSSSSSSSS")
d3 = datetime
01-Jan-2024 23:59:01.234567891
is gonna throw away the last "2345". Perhaps it should warn, I will make a note about that. I'd be curious to hear the use case for text timestamps with precisions that are that high. I would have thought that sub-picoseconds would more typically be measured using elapsed time, not absolute time.
So I think you are going to need to parse the seconds field separately, but as you say, duration won't work because of leap seconds. So I recommend splitting the timestamps at the ms as two strings, using datetime to convert the leading portion, stick a decimal in front of the second piece and use double to convert the other string to numeric, and add that to the datetime:
str1 = "21-Nov-2024 16:15:14.123";
str2 = "4567890123";
msFrac = double("." + str2);
dt = datetime(str1,Format="dd-MMM-uuuu HH:mm:ss.SSS") + milliseconds(msFrac)
dt = datetime
21-Nov-2024 16:15:14.123
sprintf("%.13f",dt.Second)
ans = "14.1234567890123"
Internally, datetime has the precision to handle sub-nanoseconds, but I don't think internal precision is the problem you are flagging.
  1 Comment
James Tursa
James Tursa on 23 Nov 2024 at 0:19
Moved: James Tursa on 23 Nov 2024 at 1:43
@Peter Perkins Thanks for the reply. My comments ...
"... I'd be curious to hear where you are getting timestamps that honor leap seconds but are in something other than an ISO format ..."
I'm creating my own class to do time scale conversions and I am trying to make my code robust for various user inputs. Forcing the user to reformat their input into strict ISO format is too overbearing IMO when the intent of the user is obvious.
"... Not sure but I suspect you are trying to use duration to parse the time of day portion of a datetime timestamp ..."
Yes, that is one thing my class does. As well as simply allowing the user to enter strings that get parsed the same way as individual entries to create durations or my own custom duration objects that maintain input precision. E.g., I see no compelling reason why these two don't do the same thing:
try, duration("00:00:60.5"), catch, disp(lasterr); end
Error using duration (line 126) Could not recognize the format of '00:00:60.5'. Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.
duration(00,00,60.5)
ans = duration
00:01:00
My code will make it easy on the user and do the same thing in both cases.
As a side issue, I also see no compelling reason why these two do different things:
duration("1:2:3:4")
ans = duration
26:03:04
duration(1,2,3,4)
ans = duration
01:02:03
But to maintain consistency with MATLAB, I suppose my code will follow this inconsistency.
"... I think the doc is pretty clear that only 9 fractional second digits are honored when converting from text, and only 9 can be displayed ..."
Yes, it is. I was hoping to learn something to avoid it, but it looks like I will have to manually parse the string to get the data loaded into the datetime accurately. Again, I don't have a specific use case to cite to you. I am creating a class and I want certain accurate behavior from the class without presuming what the user is using the class for.

Sign in to comment.

Categories

Find more on Data Type Conversion in Help Center and File Exchange

Products


Release

R2024b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!