MATLAB Answers

Import from data from messy text file

3 views (last 30 days)
Zekeftw
Zekeftw on 28 Jul 2016
Edited: dpb on 31 Jul 2016
Hi, I have looked through the questions and answers that are already available for this topic but wasn't able to apply those to my situation.
For the attached file, I need to extract the data for each "Specimen." The data that i need are the three columns corresponding to the 500+ rows for each specimen. If possible, I would like to save it in to different arrays. One for each specimen.

  0 Comments

Sign in to comment.

Accepted Answer

dpb
dpb on 28 Jul 2016
Edited: dpb on 28 Jul 2016
Pretty straightforward, just use the features in the file to find the wanted sections...it's quite regular in its construction so don't even have to do very much looking for what want--should get you started:
fid=fopen('Sample_DATA.TXT','r'); % open file
N=cell2mat(textscan(fid,'Number of specimens: %d %*[^\n]','headerlines',11));
for i=1:N % loop over number specimens in file
l=fgetl(fid); % initialize string to find start of section
while isempty(strfind(l,'Specimen:')),l=fgetl(fid);end % find the Specimen: n start
while isempty(strfind(l,'_')), l=fgetl(fid);end % then line break before data
d(i)=textscan(fid,'','delimiter',',','collectoutput',1); % the nth specimen data set
end
fid=fclose(fid);
Result will be a cell array of N nx3 data arrays. Above at command line here leaves:
> whos d
Name Size Bytes Class Attributes
d 1x3 39564 cell
>> d
d =
[532x3 double] [561x3 double] [548x3 double]
>>
Looks about right...
If the sections of auxiliary inputs, etc., are always a fixed length could use line counts in 'headerlines' parameter and avoid the while loops looking for section start, but the above will work for different sizes of those elements as long as the sections remain the same. For files of this size, that won't be that much overhead to process on a record-by-record basis anyway...

  2 Comments

Zekeftw
Zekeftw on 31 Jul 2016
Thanks dpb. This worked great. However, d is a cell array and I'm not familiar with those. How would I be able to access specific collunms in each d?
For example, I need to divide the second and third column of each d by a scalar. Then plot the second vs third column of each d.
dpb
dpb on 31 Jul 2016
Under help for cell arrays in Data Types section is "Access the contents of cells by indexing with curly braces, {}. For more information, see <Access Data in a Cell Array>. Also note cellfun to do operations on cell arrays w/o explicitly dereferencing them which would serve for step 1 above although plot will need need to have the arrays and while cellfun will work given plot as the function, it doesn't have enough granularity to let you set hold on in between or create a new axes unless you write a complementary function to use that can handle the details so "use the curlies, Luke!"

Sign in to comment.

More Answers (1)

Shameer Parmar
Shameer Parmar on 28 Jul 2016
Here is that can help you..
Data = textread('SAMPLE_DATA.txt', '%s', 'delimiter', '');
count=1;
newData = {};
for i=1:length(Data)
if ~isempty(strfind(Data{i},','))
newData{count,1} = Data{i};
count=count+1;
end
end
count1=1;
count2=0;
Specimen = {};
for i=1:length(newData)
linedata = newData{i};
if ~isempty(strfind(linedata(1:2),'1,'))
count2=count2+1;
count1=1;
Specimen{count1,count2} = linedata;
else
count1=count1+1;
Specimen{count1,count2} = linedata;
end
end
to check the output
type
Specimen(:,2);

  0 Comments

Sign in to comment.

Sign in to answer this question.