[Beginer] How can I extract column from several .txt and add them to another .txt ?

9 views (last 30 days)
Ilan Boulet
Ilan Boulet on 23 Jun 2022
Edited: dpb on 24 Jun 2022
Hello,
I have several .txt datas but I am only interested by one column in each, so I would like to take this column from each and create a .txt file which contains each columns (the number of columns of this new .txt is the number of files treated).
The name of each file is a number (from 1 to the number of file into the folder).
I had the following idea: to do a loop that create an empty .txt, and add the column specified column form each file scanned until it covered all files, then save it. There are probably more igenious ways to do that.
I am struggling to create a loop that takes into account the order of files.
If you have any advices, I thank you a lot.
Greetings

Accepted Answer

dpb
dpb on 23 Jun 2022
Edited: dpb on 24 Jun 2022
If you do that (create the names with
fname=sprintf('BaseName%03d.txt',i);
that is), then it's a piece o' cake because dir() returns the list in sorted order...
ColumnToKeep=8; % what it says, use variables
d=dir(fullfile('yourWorkingDirectory','BaseName*.txt')); % use appropriate wild card name
fnameNew=fullfile('yourWorkingDirectory','OutputNameOfChoice.txt');
nFiles=numel(d); % how many are there?
for i=1:nFiles
data=readmatrix(fullfile(d(i).folder,d(i).name)); % read the file in succession
if i==1, OutData=zeros(size(data,1),nFiles);end % preallocate the output data array
OutData(:,i)=data(:,ColumnToKeep); % store ith file Nth column
end
writematrix(OutData,fnameNew) % and write it out when done
See doc for the various functions used for all the details on each. In particular, writematrix is comma-delimited by default, you can change as desired.
Above assumes the data are all numeric and no header; there are other options to deal with that or if mix of data types, then read/writecell are probably more apropos. Attaching an example data file would let write specific code.
  7 Comments
dpb
dpb on 24 Jun 2022
OutData(:,ColumnToKeep)=data; % store ith file Nth column
There's an error in the above -- I put the keeping column in the wrong place...it's the data in the read in array to keep and write to the next column in the output array...
OutData(:,i)=data(:,ColumnToKeep);
is what was intended, of course.

Sign in to comment.

More Answers (3)

dpb
dpb on 23 Jun 2022
See the MATLAB FAQ @FAQ#How_can_I_process_a_sequence_of_files? for a fairly lengthy exposition of several alternatives. I'm partial to the dir solution outlined there, but if you didn't name your files with leading zeros, then they won't sort in numeric order -- in which case the FEX submittal natsort will be helpful.
There's also a section in the MATLAB doc itself that's a shortened version under the title "Import or Export a Sequence of Files"
As common as is, it isn't all that easy to find the example and it isn't excessively long, for sure.
Neither of the above have been updated to reflect the new Strings class which allows one to build the numeric file name with the overloaded plus "+" operator in lieu of the explicit format string building route --
>> numfiles=4;
>> for k = 1:numfiles
myfilename = "file" + k +".txt"
end
myfilename =
"file1.txt"
myfilename =
"file2.txt"
myfilename =
"file3.txt"
myfilename =
"file4.txt"
>>
which is handy if you didn't use the leading zeros -- if did, then still need the '%02d' formatting string.
  1 Comment
Ilan Boulet
Ilan Boulet on 23 Jun 2022
Thanks a lot for your answer dpb.
I actually can name my files "file00i" directly when they are exporte from my experiment.
So if I understood well, since my files are ordered with the 00i at the end, I can create an already sorted list?
It would be amazing because what I want to do is a for-loop that extracts the eighth column of my files and put them into an empty table,but by keeping the good order of the files! Still struggling on that lol.
Thanks again

Sign in to comment.


dpb
dpb on 24 Jun 2022
Edited: dpb on 24 Jun 2022
OK, the other may be of some use to others; given the revelation of old version of ML in use here I'll just post a different Answer entirely...same ideas hold, just use old
ColumnToKeep=8; % what it says, use variables
d=dir(fullfile('yourWorkingDirectory','BaseName*.txt')); % use appropriate wild card name
fnameNew=fullfile('yourWorkingDirectory','OutputNameOfChoice.txt');
nFiles=numel(d); % how many are there?
for i=1:nFiles
data=dlmread(fullfile(d(i).folder,d(i).name),' ',1,ColumnToKeep-1); % space-delimited, skip first row, read one column
if i==1, OutData=zeros(size(data,1),nFiles);end % preallocate the output data array
OutData(:,i)=data;
end
dlmwrite(fnameNew,OutData,' ','precision','%.6E') % and write it out when done
The above only reads the specified column; I note that in the given files column 8 is all zero which isn't that interesting.
Also NB: the row and column offsets in dlmread are zero-based--hence the "-1" in from the ColumntToKeep and the "1" for the row offset.
I'll also note you complained before that "function OutData" wasn't found -- it is NOT a function but the output array you're creating -- it is preallocated in the if..end construction the first pass thru the loop -- it's inside the loop so have the first file read from which to be able to determine the needed size from the length of the first file. This does presume all files are the same length as must be to go into a single double array -- otherwise would have to use a cell array instead that gets a lot more painful to write with older release.

Ilan Boulet
Ilan Boulet on 24 Jun 2022
I bought the 2022 version and your previous code works perfectly !!
I'm so grateful, thanks a lot !!
  1 Comment
dpb
dpb on 24 Jun 2022
No problem, teaching is part (most?) of the fun of participation here...
NB: The "ColumnToKeep" value can be a vector of columns, not just a single value...although the column addressing in the assignment to OutData would have to be updated to handle more than the one column it does at present.

Sign in to comment.

Products


Release

R2007b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!