How to parse text data

Hi

I have data in the below format. I need the mechanism to parse the data from below format with expected output.

Input data format:
07/16 12:55:22.012 INFO | test_runner_utils:0812| Began logging to /tmp/test_that_results_hatch_deL3lZ
07/16 12:55:27.477 INFO | test_runner_utils:0259| autoserv| Processing control file

Expected Output format:
Define level of message extraction based on the  marker sign ==> |
-Step 1: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|
-Step 2: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>| extract full text in a variable, option to  grab variable if associated with value
-Step 3: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|
-Step 4: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|extract full text in a variable, option to  grab variable if associated with value
-Step 5: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<string>|
-Step 6: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<string>|extract full text in a variable, option to  grab variable if associated with value

Input data format:
07/16 12:55:27.620 DEBUG|             utils:0287| [stdout] CHROMEOS_RELEASE_BOARD=hatch
07/16 13:28:58.330 INFO |     mode_switcher:0673| -[FAFT]-[ start wait_for_client ]---
Expected Output format:
-Step 1: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<[string]>
-Step 2: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<[string]> extract full text in a variable, option to  grab variable if associated with value
-Step 3: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<[string]>
-Step 4: Extract Timestamp in mm/dd HH:MM:sec.millisec <string>|<string>:%1.3f|<[string]> [string] extract full text in a variable, option to  grab variable if associated with value

Input data format:
2019-07-16 12:55:30 > string
2019-07-16 12:55:30 powerbtn: released
Expected Output format:
Note  the  marker >
-Step 1: Extract Timestamp in YYYY:MM:DD HH:mm:sec > < string> 
-Step 2: Extract Timestamp in YYYY:MM:DD HH:mm:sec  < full string> 

Input data format
2019-07-16 12:55:31 > [12074.734997 HC 0x121 err 1]
Expected Output format
-Step 1: Extract Timestamp in YYYY:MM:DD HH:mm:sec > [< %1.3f string extract full text in a variable, option to  grab variable if associated with value>]

Thanks a lot

[Bob Thompson]

Do you know about regexp? I think that can get you started here.

[Life is Wonderful]

Thanks! Yes- I am aware of regexp .

Below is my code for

07/16 12:55:22.012 INFO | test_runner_utils:0812| Began logging to /tmp/test_that_results_hatch_deL3lZ
Datacontent = regexp(filecontent, '^(?<Time_MDY>[^ ]+) (?<Time_HMSsss>[^ ]+) (?<first>[^|]\w+)|\s+(?<last>[^|\r\n]+)|(?<last>[^|\r\n]+),\s+(?<first>[^|]|\w+)', 'names', 'lineanchors');  %parse all lines according to common format

I need next steps

• Convert Datacontent into cell's - like timestamp , message data-1,message data-2
• Put cell in proper format
• Create Matlab variables
• Display Matlab variable for good analysis

[Guillaume]

I've not looked at this question in details. Does the file format differ much from the one in your previous question?

If not, it should be fairly trivial to adapt the parser I wrote, which would be a lot less effort than starting again from scratch.

[Life is Wonderful]

Guillaume

Previous one timestamp was in UTC format and this one is dd/mm format.

Can you please support me?

Thanks

[Life is Wonderful]

Any feedback ?

Answer 1

[Guillaume]

Are you still on very old version (please fill the release field next to the question)?. If on a modern version, the file can easily be read with:

VariableNames = {'Date', 'Level', 'delim1', 'PID', 'delim2', 'Message'};
VariableWidths = [19, 5, 1, 23, 2, 5000];
VariableTypes = {'datetime', 'char', 'char', 'char', 'char', 'char'};
opts = fixedWidthImportOptions('VariableNames', VariableNames, 'VariableWidths', VariableWidths, 'VariableTypes', VariableTypes, 'SelectedVariableNames', [1, 2, 4, 6]);
opts = setvaropts(opts, 'Date', 'InputFormat', 'MM/yy hh:mm:ss.SSS');
content = readtable('test_that.txt', opts);

results in:

If on a version fo matlab that doesn't have tables, use textscan with fixed width fields:

fid = fopen('test_that.txt', 'rt');
content = textscan(fid, '%18c%*c%5c%*c%23c%*2c%s', 'Delimiter', '', 'Whitespace', '');
fclose(fid);
content = [cellstr(content{1}), cellstr(content{2}), cellstr(content{3}), content{4}]

[Life is Wonderful]

Thanks !

I have moved to latest version. Release field is grayed out for me {I am unable to edit the release field}.

>> ver
-----------------------------------------------------------------------------------------------------
MATLAB Version: 9.6.0.1072779 (R2019a)
-----------------------------------------------------------------------------------------------------

Instead of readtable I use "readcell".

[Guillaume]

You need to select a product before you can select the release (I know it's silly, we've complained enough to mathworks).

Then use the fixedWidthImportOptions version, which is more reliable than the textscan version. Matlab does all the hard work for you.

[Life is Wonderful]

Couple of more things.

Since both readcell and textscan provide the data in the form cell. I would like to know

How to convert the cell data into Matlab variable ? Like

content.Date
content.Level
content.delim1
content.PID
content.delim2
content.Message

Then how I want to use above Matlab variables so that plotting is controlled by GUI .

[Guillaume]

Use readtable as I've shown, not readcell.

A table is better than a structure, you can access the data by table variable exactly as you want:

content.Date

and by rows:

content(1:10, :)

and if you convert the table into a timetable, easily select a time period:

content = table2timetable(content);
content = rmmissing(content);  %to remove the bad rows
subcontent = content(timerange('1-Jul-2018 11:30', '1-Jul-2018 12:00'), :)

[Life is Wonderful]

readtable is giving below error

Error using matlab.io.ImportOptions/readtable (line 492)
An error occurred while trying to determine whether "readData" is a function name.

[Guillaume]

I'm not sure what you've done. The code I've shown above doesn't throw that error. Whatever modification you've done is most likely the culprit.

[Life is Wonderful]

Below is the code that I ran in the command window.

K>> VariableNames   = {'Date', 'Level', 'delim1', 'PID', 'delim2', 'Message'};
    VariableWidths  = [19, 5, 1, 23, 2, 5000];
    VariableTypes   = {'datetime', 'char', 'char', 'char', 'char', 'char'};
    opts            = fixedWidthImportOptions('VariableNames', VariableNames, 'VariableWidths', VariableWidths, 'VariableTypes', VariableTypes, 'SelectedVariableNames', [1, 2, 4, 6]);
    opts            = setvaropts(opts, 'Date', 'InputFormat', 'MM/yy hh:mm:ss.SSS');
    content         = readtable('test_that.txt', opts);
Error using matlab.io.ImportOptions/readtable (line 492)
An error occurred while trying to determine whether "readData" is a function name.

[Life is Wonderful]

How do i know what I am doing wrong ?

opts = 

  FixedWidthImportOptions with properties:

   Format Properties:
                  Whitespace: '\b\t '
                  LineEnding: {'\n'  '\r'  '\r\n'}
                CommentStyle: {}
               EmptyLineRule: 'skip'
                    Encoding: 'system'

   Replacement Properties:
                 MissingRule: 'fill'
             ImportErrorRule: 'fill'
            ExtraColumnsRule: 'addvars'
            PartialFieldRule: 'keep'

   Variable Import Properties: Set types by name using setvartype
               VariableNames: {'Date', 'Level', 'delim1' ... and 3 more}
               VariableTypes: {'datetime', 'char', 'char' ... and 3 more}
              VariableWidths: [19 5 1 23 2 5000]
       SelectedVariableNames: {'Date', 'Level', 'PID' ... and 1 more}
             VariableOptions: Show all 6 VariableOptions 
	Access VariableOptions sub-properties using setvaropts/getvaropts

   Location Properties:
                   DataLines: [1 Inf]
           VariableNamesLine: 0
              RowNamesColumn: 0
           VariableUnitsLine: 0
    VariableDescriptionsLine: 0 

[Guillaume]

what does

which readData -all

return?

[Life is Wonderful]

>> VariableNames   = {'Date', 'Level', 'delim1', 'PID', 'delim2', 'Message'};
    VariableWidths  = [19, 5, 1, 23, 2, 5000];
    VariableTypes   = {'datetime', 'char', 'char', 'char', 'char', 'char'};
    opts            = fixedWidthImportOptions('VariableNames', VariableNames, 'VariableWidths', VariableWidths, 'VariableTypes', VariableTypes, 'SelectedVariableNames', [1, 2, 4, 6]);
    
%     opts = detectImportOptions('test_that.txt')
    opts            = setvaropts(opts, 'Date', 'InputFormat', 'MM/yy hh:mm:ss.SSS');
    content         = readtable('test_that.txt', opts);
Error using matlab.io.ImportOptions/readtable (line 492)
An error occurred while trying to determine whether "readData" is a function name.
 
>> which readData -all
Bundle#361 start failed: Loading C:\Program
Files\MATLAB\R2019a\bin\win64\builtins\sl_graphical_classes\mwsl_graphical_classes_builtinimpl.dllfailed with
error: The specified procedure could not be found.

What is missing

[Guillaume]

"Bundle#361 start failed"

Wow, that's some very weird error that I've never come across. There's something very weird going on with your install. If you have active support you should contact Mathworks. which shouldn't error, it's a built-in function.

[Life is Wonderful]

I will re-install the software and check again the solution.

[Guillaume]

sriram shastry comment posted as an answer moved here:

Thanks .I did the software re-install. I can see the data without any errors.

What is not clear for me now is how to plot the data . I can plot content.Data but rest I can't .Date is not in string format but message,PID and message are string cell.

Please suggest plotting mechanism so that data can be looked more efficiently.

>> content.Date(1:10,1)

ans = 

  10×1 datetime array

   07/18 11:27:02.968
   07/18 11:27:02.968
   07/18 11:27:02.968
   07/18 11:27:02.968
   07/18 11:27:02.968
   07/18 11:27:02.969
   07/18 11:27:02.969
   07/18 11:27:02.970
   07/18 11:27:02.972
   07/18 11:27:02.972

>> content.Level(1:10,1)

ans =

  10×1 cell array

    {'DEBUG'}
    {'DEBUG'}
    {'DEBUG'}
    {'INFO' }
    {'INFO' }
    {'DEBUG'}
    {'DEBUG'}
    {'DEBUG'}
    {'DEBUG'}
    {'DEBUG'}

>> content.PID(1:10,1)

ans =

  10×1 cell array

    {'autoserv:0729' }
    {'autoserv:0730' }
    {'autoserv:0731' }
    {'autoserv:0739' }
    {'pidfile:0016'  }
    {'host_info:0263'}
    {'host_info:0267'}
    {'base_job:0357' }
    {'base_job:0357' }
    {'base_job:0357' }

>> content.Message(1:10,1)

ans =

  10×1 cell array

    {'autoserv is running in drone lab_chrome-debug.'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                }
    {'autoserv command was: /build/hatch/usr/local/build/autotest/server/autoserv -p -r /tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion -m 10.223.131.106 --no_console_prefix -u autotest_system -l ad_hoc_build/ad_hoc_suite/firmware_UpdateKernelSubkeyVersion -s --no_use_packaging /tmp/tmphhqbvd --args  servo_host=localhost servo_port=9999'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 }
    {'autoserv parsed options: Namespace(args=' servo_host=localhost servo_port=9999', cleanup=False, client=False, cloud_trace_context='', cloud_trace_context_enabled='False', collect_crashinfo=False, control_filename=None, control_name=None, disable_sysinfo=False, execution_tag='', group_name='', host_attributes={}, host_info_subdir='host_info_store', install_in_tmpdir=False, isolate='', job_labels=None, lab='', label='ad_hoc_build/ad_hoc_suite/firmware_UpdateKernelSubkeyVersion', local_only_host_info=False, machines='10.223.131.106', machines_file=None, no_console_prefix=True, no_logging=False, no_tee=False, no_use_packaging=True, parent_job_id=None, parse_job='', pidfile_label='autoserv', provision=False, repair=False, require_ssp=False, reset=False, results='/tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion', server=True, skip_crash_collection=False, ssh_options='', ssh_pass='', ssh_port=22, ssh_user='root', ssh_verbosity=0, test_source_build='', use_existing_results=False, user='autotest_system', verbose=False, verify=False, verify_job_repo_url=False, write_pidfile=True)'}
    {'Results placed in /tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            }
    {'Logged pid 23629 to /tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion/.autoserv_execute'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        }
    {'Committing HostInfo to store InMemoryHostInfoStore[HostInfo[Labels: [], Attributes: {}]]'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      }
    {'HostInfo updated to: HostInfo[Labels: [], Attributes: {}]'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     }
    {'Persistent state global_properties.tag now set to '''                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          }
    {'Persistent state global_properties.fast now set to False'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      }
    {'Persistent state global_properties.max_result_size_KB now set to 40000'                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        }

later comment moved here as well

Any Feedback

• on plotting the data.
• Convert timestamp into CPUtime

Thanks

[Guillaume]

I have no idea what you mean by plotting. How do you plot text? What do you want to plot against what?

I also don't know what you mean by converting to cpu time. cpu time usually mean the amount of time that a process/program spend on calculation. It's a duration, so has nothing to do with timestamps which are points in time.

[Life is Wonderful]

I have no idea what you mean by plotting. How do you plot text? What do you want to plot against what?

I need following plot's

plot(content.Date,content.Level);
plot(content.Date,content.PID);
plot(content.Date,content.Message);

I have text Analytics Toolbox support and analysis of Text Data Using Deep Learning ( CNN,RNN,HAN)

I also don't know what you mean by converting to cpu time. cpu time usually mean the amount of time that a process/program spend on calculation. It's a duration, so has nothing to do with timestamps which are points in time.

By cpu time, starting time of log set to 0 as (example Logfile Timestamp:07/01 12:00:06.355 reference as timestamp 0seconds un-till end of Logfile Timestamp

[Guillaume]

Level is either 'DEBUG', 'INFO', 'ERROR', etc. Again, I've no idea what you mean by plotting these. I don't even have a clue what sort of plot you're looking for. A line plot?, A bar plot? A pie chart? A scatter plot? None of these make sense for text.

If you want to convert the date into duration from the start of the log (calling that cputime would be very misleading), then it's trivial:

content.Duration = content.Date - Content.Date(1);

[Life is Wonderful]

Level is either 'DEBUG', 'INFO', 'ERROR', etc. Again, I've no idea what you mean by plotting these. I don't even have a clue what sort of plot you're looking for. A line plot?, A bar plot? A pie chart? A scatter plot? None of these make sense for text

I am fine with Line plot but all three I want

subplot(3,1,1);plot(content.Date)
set(gca,'xticklabel',content.Level); hold on; grid on;
subplot(3,1,2);plot(content.Date)
set(gca,'xticklabel',content.PID); hold on; grid on;
subplot(3,1,3);plot(content.Date)
set(gca,'xticklabel',content.Message); hold on; grid on;

But my plotting is not good at all.

I need your suggestion that make sense in analysis of text information over the time.

[Guillaume]

I am fine with Line plot

A line plot of what? What does 'DEBUG' look like on a line plot? I'm sorry I really don't understand what you want to plot. For me, plotting text makes no sense at all. You plot numbers, not text.

[Life is Wonderful]

OK. My requirement is plot event i.e. text print over time. For debug the log information - Instead of doing manual search in Notepad++ editor , I want to analyze information based on the event over time. So I need plot of text over time.

[Life is Wonderful]

I just saw there is a way to display text over data points "Text and Annotations in MATLAB "

• if data point's is time i.e content.Date then content.Messag will be text

[Guillaume]

Yes, you can put text on a figure. How do you determine the position of said text. You need at least an x (time?) and a y (????).

Maybe you should draw what you want (on paper, in excel, whatever) because I still don't understand.

On the other, we've completely deviated from your original question which I believe has been answered, so maybe you should start a new question dedicated to your plotting. Others may be more likely to contribute that way.

[Life is Wonderful]

Yes, you can put text on a figure. How do you determine the position of said text. You need at least an x (time?) and a y (????).

Sure.

x time = content.Date
y time = content.Level

x time = content.Date
y time = content.PID

x time = content.Date
y time = content.Message % May a trimmed one for view purpose

Thank a ton . Highly useful tip that too in time help.

[Life is Wonderful]

Probably you can suggest on time DATAPOINTS plot text information which is text annotations

Answer 2

[Bob Thompson]

I need next steps

◾Convert Datacontent into cell's - like timestamp , message data-1,message data-2

◾Put cell in proper format

◾Create Matlab variables

◾Display Matlab variable for good analysis

1) regexp automatically outputs all results in a cell, each containing a string.

2) You can convert strings to date time formats using datetime. To do this 'quickly' I suggest using a loop through your regexp results, or by using cellfun (which is really still a loop).

3) What exactly do you mean by this? I personally do not know of a way to dynamically create variables within Matlab, and I think you would be better served to keep the information in a cell array, or to make a table out of it. It is certainly possible to create new variables in a table from a captured string from regexp.

4) Displaying Matlab variables is simply a matter of not suppressing them, or if specifically wanting to display them then you can use fprintf with no target so it defaults to the command window.

[Life is Wonderful]

My code looks like

fileData                   = regexp(filecontent, '^(?<Time_MDY>[^ ]+) (?<Time_HMSsss>[^ ]+) (?<first>[^|]\w+)|\s+(?<last>[^|\r\n]+)|(?<last>[^|\r\n]+),\s+(?<first>[^|]|\w+)', 'names', 'lineanchors');
fileData                   = cellfun(@(t) vertcat(t{:}), fileData, 'UniformOutput', false);        

But I am getting error

Error using cellfun
Input #2 expected to be a cell array, was struct instead.

Please suggest better approach to resolve the issue.

can you please give me a small implementation snippet ? Thanks

[Life is Wonderful]

For datetime code like below

dates = num2cell(datetime(Datacontent, 'InputFormat', 'MM/dd HH:mm:ss.S'));   %decode date

I see following error

Error using datetime (line 658)
Input data must be a numeric array, a string array, a cell array containing character vectors, or a char matrix.

[Bob Thompson]

You're getting the structure class error because 'names' outputs the results as a structure, rather than a cell, as I was expecting. Personally, I prefer 'tokens' or 'match' as my output flag for regexp.

Cellfun will not work with any input that is not a cell, hence the error.

I would suggest something like the following:

fileData = regexp(filecontent, '^(?<Time_MDY>[^ ]+) (?<Time_HMSsss>[^ ]+) (?<first>[^|]\w+)|\s+(?<last>[^|\r\n]+)|(?<last>[^|\r\n]+),\s+(?<first>[^|]|\w+)', 'tokens', 'lineanchors');
dates = datetime([fileData{1}{1},' ',fileData{1}{2}], 'InputFormat', 'MM/dd HH:mm:ss.SSS');

I did use a singular line to test this, so if you have multiple rows of inputs and outputs from regexp then you may need to investigate using a loop.

[Bob Thompson]

Are you only looking to capture the timestamp? It seems like the issue is more in the initial regexp processing than in the date time conversion.

If you are only looking to capture the timestamp I would suggest doing a regexp call like this:

filedata = regexp(filecontent'(\d\d.\d\d\s\d\d.\d\d.\d\d.\d\d\d)\D+\d\d\d\d\D+\n','tokens');
dates = datetime([filedata{:}], 'InputFormat', 'MM/dd HH:mm:ss.SSS');

If you are looking to capture more than the timestamps then please explain more. I know you outline some more in your OP, but I'm not entirely sure what you're referring to.

[Life is Wonderful]

I am looking not only for timestamp but associated data.I wrote in the begining how my requirement/algorithm looks like.

My text source file contains data as below

07/18 11:27:02.968 DEBUG|          autoserv:0729| autoserv is running in drone lab_chrome-debug.
07/18 11:27:02.968 DEBUG|          autoserv:0730| autoserv command was: /build/hatch/usr/local/build/autotest/server/autoserv -p -r /tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion -m 10.223.131.106 --no_console_prefix -u autotest_system -l ad_hoc_build/ad_hoc_suite/firmware_UpdateKernelSubkeyVersion -s --no_use_packaging /tmp/tmphhqbvd --args  servo_host=localhost servo_port=9999
07/18 11:27:02.968 INFO |           pidfile:0016| Logged pid 23629 to /tmp/test_that_results_hatch_iGVg61/results-1-firmware_UpdateKernelSubkeyVersion/.autoserv_execute
07/18 11:27:02.969 DEBUG|         host_info:0263| Committing HostInfo to store InMemoryHostInfoStore[HostInfo[Labels: [], Attributes: {}]]
07/18 11:27:02.969 DEBUG|         host_info:0267| HostInfo updated to: HostInfo[Labels: [], Attributes: {}]
07/18 11:27:02.970 DEBUG|          base_job:0357| Persistent state global_properties.tag now set to ''
07/18 11:27:02.972 DEBUG|          base_job:0357| Persistent state global_properties.fast now set to False
07/18 11:28:16.561 DEBUG|             servo:0666| Setting power_state to 'rec'
07/18 11:28:23.419 WARNI|              test:0606| The test failed with the following exception
Traceback (most recent call last):
  File "/build/hatch/usr/local/build/autotest/client/common_lib/test.py", line 567, in _exec
    _cherry_pick_call(self.initialize, *args, **dargs)
  File "/build/hatch/usr/local/build/autotest/client/common_lib/test.py", line 715, in _cherry_pick_call
    return func(*p_args, **p_dargs)
  File "/build/hatch/usr/local/build/autotest/server/site_tests/firmware_UpdateKernelSubkeyVersion/firmware_UpdateKernelSubkeyVersion.py", line 60, in initialize
    self.switcher.setup_mode('dev' if dev_mode else 'normal')
  File "/build/hatch/usr/local/build/autotest/server/cros/faft/utils/mode_switcher.py", line 427, in setup_mode
    self.reboot_to_mode(mode)
  File "/build/hatch/usr/local/build/autotest/server/cros/faft/utils/mode_switcher.py", line 474, in reboot_to_mode
    self._enable_dev_mode_and_reboot()
  File "/build/hatch/usr/local/build/autotest/server/cros/faft/utils/mode_switcher.py", line 717, in _enable_dev_mode_and_reboot
    self._enable_rec_mode_and_reboot(usb_state='host')
  File "/build/hatch/usr/local/build/autotest/server/cros/faft/utils/mode_switcher.py", line 590, in _enable_rec_mode_and_reboot
    psc.power_on(psc.REC_ON)
  File "/build/hatch/usr/local/build/autotest/server/cros/servo/servo.py", line 134, in power_on
    self._servo.set_nocheck('power_state', rec_mode)
  File "/build/hatch/usr/local/build/autotest/server/cros/servo/servo.py", line 672, in set_nocheck
    raise error.TestFail(err_msg)
TestFail: Setting 'power_state' to 'rec' :: Timeout waiting for response.
07/18 11:28:23.420 DEBUG|              test:0611| Running cleanup for test.
07/18 11:44:07.043 DEBUG|          ssh_host:0310| Running (ssh) 'true' from '_install|wait_up|is_up|ssh_ping|run|run_very_slowly'

I have to parse the data as following

Example

fileData.timestamp                                   = 07/18 11:28:23.420
fileData.timestamp.Msglib                            = DEBUG
fileData.timestamp.MsgSublib                         = test
fileData.timestamp.MsgSublib.idx                     = 0611
fileData.timestamp.MsgSublib.FullContenet            = Running cleanup for test.

If error is seen, then skip those line from the input text file and continue parsing the information

Answer 3

[Life is Wonderful]

Adding the input file