Read text file with fixed width columns and mixed data types

11 views (last 30 days)
I want to read the data columns similar to the attached file "wds00524-6930.txt". The format of the file is explained fully in the "datarequest_key.txt". Briefly, the data begins on line 12 and each line has 25 data columns making a width of 121 per data line. For example, the first column of data has a format of f10.5 and is from width 008 to 017. Some columns have data types 'double' others 'string'. I have tried all sort of ways to read this but have always resorted to importing the text file into excel and selecting what I want and saving it as a csv or xlsx file . This works well on an individual file basis, but I want to be able to read hundreds-thousands of text files in the same format.

Answers (1)

Rod Letchford
Rod Letchford on 20 Feb 2019
KSSV, the data always begins on line 12 (Notepad++ counting) with the first date, in this case being '1834.84'. The last line - in this particular case - is line 62.
  1 Comment
Rod Letchford
Rod Letchford on 21 Feb 2019
Apart from not finding the last line correctly, this python code I wrote does what I want Matlab to do:
import pandas as pd
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
filename = 'wds00524-6930.txt';
colspecs = [(7,17),(18,19),(19,26),(27,33),(34,35),(35,44),(45,46),(46,53),\
(54,55),(55,61),(61,62),(62,67),(68,69),(69,75),(75,76),(76,81),\
(82,90),(90,91),(92,97),(97,98),(99,101),(102,110),(111,113),\
(114,116),(116,117)]
names = ['date','tflag','theta','terr','rflag','rho','reflag','rerr','mflag1',\
'mag1','m1eflag','m1err','mflag2','mag2','m2eflag','m2err','filter',\
'fflag','tel','teflag','nn','ref','tech','codes','d.e. note']
WDS = pd.read_fwf(filename,header=10,colspecs=colspecs,names=names)

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!