Error when converting Matlab time (double) to datetime (raw data captured from RealTerm)

16 views (last 30 days)
I am reading raw data from an instrument via RealTerm (3.0.1.44) which gives me the option to dump it in a txt file while also appending a Matlab time stamp. My main interest is to keep millisecond precision as I must get the actual sampling rate given by the instrument
The first 10 captured Matlab time rows are the following:
1. 738545.739572000
2. 738545.739575000
3. 738545.739578000
4. 738545.739581000
5. 738545.739583000
6. 738545.739586000
7. 738545.739589000
8. 738545.739591000
9. 738545.739594000
10. 738545.739597000
As you can see, they are sequential double-precision numbers, however, when converting them to their corresponding datetime values I get a mismatch on the sixth value:
1. '05:01:59.021000000'
2. '05:01:59.280000000'
3. '05:01:59.539000000'
4. '05:01:59.798000000'
5. '05:01:59.971000000'
6. '05:01:00.230000000'
7. '05:01:00.490000000'
8. '05:01:00.662000000'
9. '05:01:00.922000000'
10. '05:01:01.181000000'
Why the clock time jumps from '05:01:59.971000000' to '05:01:00.230000000' if the double type times are sequential?
These values are obtained applying the following line inside a for-loop to dataCell, a cell that contains the imported values in columns where the first corresponds to the times and pivot is just an index:
currentTime = datetime(datestr(dataCell{pivot,1}, 'hh:mm:ss.FFF'),'InputFormat','HH:mm:ss.SSSSSS');
currentTime.Format = 'hh:mm:ss.SSSSSSSSS'
Is this the wrong way to convert Matlab times to hours with millisecond precision?
It should be noted that this also happens in other data acquisitions where the numbers are sequential too.
I intend to use the first value as the start time then compute the time deltas in milliseconds using datevec then datenum, however, I clearly get negative deltas:
1. 0
2. 0.00431666666666667
3. 0.00863333333333333
4. 0.0129500000000000
5. 0.0158333333333333
6. -0.979850000000000
7. -0.975516666666667
8. -0.972650000000000
9. -0.968316666666667
10. -0.964000000000000
Thanks for yor help

Accepted Answer

Stephen23
Stephen23 on 26 Jan 2022
Edited: Stephen23 on 26 Jan 2022
"Why the clock time jumps from '05:01:59.971000000' to '05:01:00.230000000' if the double type times are sequential?"
Because your data are less sequential than you think they are. Let have a look:
N = [738545.739572000; 738545.739575000; 738545.739578000; 738545.739581000; 738545.739583000; 738545.739586000; 738545.739589000; 738545.739591000; 738545.739594000; 738545.739597000];
format long G
diff(N) % not that regular
ans = 9×1
1.0e+00 * 3.00002284348011e-06 3.00002284348011e-06 2.99990642815828e-06 2.00001522898674e-06 3.00002284348011e-06 3.00002284348011e-06 2.00001522898674e-06 3.00002284348011e-06 2.99990642815828e-06
"Is this the wrong way to convert Matlab times to hours with millisecond precision?"
Yes. Not only is this the wrong way to convert such values, it is also the wrong way to store such values: a serial date number is limited by the precision of the DOUBLE class, which means that it cannot store more accuracy than
P = eps(N(1)) % days
P =
1.16415321826935e-10
P*24*60*60 % seconds
ans =
1.00582838058472e-05
So what you are attempting is already beyond anything that can be stored in a DOUBLE serial date number: they cannot store micro-second precision for the dates that you are using. As the documentation clearly states, avoid the old, deprecated, low-precision functions DATESTR, DATENUM, and DATEVEC. Use DATETIME and DURATION objects instead.
And finally to answer your question, this is the best way (but of course it will not magically make information appear that is not stored due to the precision limit of the DOUBLE class):
T = datetime(N,'ConvertFrom','datenum','Format','HH:mm:ss.SSSSSSSSS')
T = 10×1 datetime array
17:44:59.020799316 17:44:59.280000000 17:44:59.539203125 17:44:59.798395019 17:44:59.971196533 17:45:00.230398437 17:45:00.489600341 17:45:00.662401611 17:45:00.921603759 17:45:01.180795654
D = diff(T);
D.Format = 'hh:mm:ss.SSSSSS'
D = 9×1 duration array
00:00:00.259200 00:00:00.259203 00:00:00.259191 00:00:00.172801 00:00:00.259201 00:00:00.259201 00:00:00.172801 00:00:00.259202 00:00:00.259191
  1 Comment
Cris LaPierre
Cris LaPierre on 26 Jan 2022
Note that datetimes have a date associated with them, even if it is not displayed.
N = [738545.739572000; 738545.739575000; 738545.739578000; 738545.739581000; 738545.739583000; 738545.739586000; 738545.739589000; 738545.739591000; 738545.739594000; 738545.739597000];
T = datetime(N,'ConvertFrom','datenum','Format','hh:mm:ss.SSSSSSSSS')
T = 10×1 datetime array
05:44:59.020799316 05:44:59.280000000 05:44:59.539203125 05:44:59.798395019 05:44:59.971196533 05:45:00.230398437 05:45:00.489600341 05:45:00.662401611 05:45:00.921603759 05:45:01.180795654
T.Format = 'default'
T = 10×1 datetime array
24-Jan-2022 17:44:59 24-Jan-2022 17:44:59 24-Jan-2022 17:44:59 24-Jan-2022 17:44:59 24-Jan-2022 17:44:59 24-Jan-2022 17:45:00 24-Jan-2022 17:45:00 24-Jan-2022 17:45:00 24-Jan-2022 17:45:00 24-Jan-2022 17:45:01
Also, I would suggest using HH instead of hh to not confuse am with pm (5pm = 17hr).
T.Format = 'HH:mm:ss.SSSSSSSSS'
T = 10×1 datetime array
17:44:59.020799316 17:44:59.280000000 17:44:59.539203125 17:44:59.798395019 17:44:59.971196533 17:45:00.230398437 17:45:00.489600341 17:45:00.662401611 17:45:00.921603759 17:45:01.180795654

Sign in to comment.

More Answers (1)

Daniel Melendrez
Daniel Melendrez on 26 Jan 2022
Your answers are awesome, guys!
This clarifies a lot.
Let me test your recommended solutions and I will get back asap with the final answer.
Have a great day
  1 Comment
Daniel Melendrez
Daniel Melendrez on 30 Jan 2022
To mark this question as answered, the trick was on this line, as recommended by Stephen:
T = datetime(N,'ConvertFrom','datenum','Format','hh:mm:ss.SSSSSSSSS')
and just to clarify: I was aware the the precision limit from DOUBLE type values was limiting the resolution from my values, however, the data was imported using the Import Data tool from ML. I couldn't find any options for saving them using uint_64.
Now I know that DATETIME and DURATION are the way to go.
Thank you once agian!

Sign in to comment.

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!