Reading data from multiple files in ascending order.

5 views (last 30 days)
I have many folders and in each I have potentially 480 txt files (“potentially” because in some folders some files are missing) which names changing in ascending order from:
aaaa001a00.txt
aaaa001a30.txt
aaaa001b00.txt
aaaa001b30.txt
aaaa001x00.txt
aaaa001x30.txt
aaaa002a00.txt
aaaa010x30.txt
(bold number change from 1 to 10, bold italic letter changes from “a” to “x” (24 letters) and number after this letter is 00 or 30).
In each file there are four columns with data, as below:
0.5031 0.5709 0.1937 0.7852
0.5074 0.5719 0.1878 0.7873
0.5127 0.5727 0.1823 0.7900
0.5170 0.5729 0.1787 0.7921
0.5211 0.5733 0.1730 0.7938
But the number of line can be different in each file…
In each folder I would like to read all 480 “potentially” files in ascending order (as listed).
If file exist I would like to read data from the last line… (for example: 0.5211 0.5733 0.1730 0.7938)
If file don’t exist I would like “to read”: NaN NaN NaN NaN.
How it could be done in matlab?

Accepted Answer

Mathieu NOE
Mathieu NOE on 9 Apr 2021
hello
this is my suggestion , based on natsortfiles (from FEX : Natural-Order Filename Sort - File Exchange - MATLAB Central (mathworks.com), also attached here ) that allows you to load the files according to natural name sorting
then each last line of the data is stored in a cell ,and the final array is then saved as excel file
now I don't understand how you could do the "NaN" line for a file that does not exist and therefore cannot be identified in the folder with dir ??
my code :
fileDir = cd;
outfile = 'OUT.xlsx';
fileNames = dir(fullfile(cd,'*.txt')); % get list of files in directory
fileNames_sorted = natsortfiles({fileNames.name}); % sort file names into order
M= length (fileNames_sorted);
for f = 1:M
raw = importdata( fullfile(fileDir, fileNames_sorted{f}));
[m,n] = size(raw);
last_line{f} = raw(m,:);
end
% write all last lines into a matrix and store it in excel file
writecell(last_line',fullfile(cd,outfile));
  2 Comments
Stephen23
Stephen23 on 9 Apr 2021
Edited: Stephen23 on 9 Apr 2021
The leading zeros on the integers means that NATSORTFILES is not required:
C = {'aaaa001a30.txt';'aaaa001b00.txt';'aaaa001b30.txt';'aaaa001a00.txt';'aaaa001x00.txt';'aaaa010x30.txt';'aaaa001x30.txt';'aaaa002a00.txt'};
C = C(randperm(numel(C)))
C = 8×1 cell array
{'aaaa002a00.txt'} {'aaaa001b30.txt'} {'aaaa001a30.txt'} {'aaaa001a00.txt'} {'aaaa010x30.txt'} {'aaaa001x00.txt'} {'aaaa001b00.txt'} {'aaaa001x30.txt'}
D = sort(C) % no need for NATSORTFILES
D = 8×1 cell array
{'aaaa001a00.txt'} {'aaaa001a30.txt'} {'aaaa001b00.txt'} {'aaaa001b30.txt'} {'aaaa001x00.txt'} {'aaaa001x30.txt'} {'aaaa002a00.txt'} {'aaaa010x30.txt'}
The challenge is to detect the missing files.
UWM
UWM on 12 Apr 2021
Thank you very much for your suggestions - they were very helpful.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!