How to load large number of files and perform looping through each file?

I have 153 files stored in a directory with extension of .txt. I want to load each and every file in a script and perform a mathematical calculation using for loop.

[Rik]

What have you tried so far? Have you searched for examples? How did they not work for you?

Have a read here and here. It will greatly improve your chances of getting an answer. (And help you find your own answers)

[John D'Errico]

Use a loop. WTP? Start with a list of file names.

[Cedric]

In addition to what Rik mentions, define better the structure of the content of these files, and/or provide a slice of a file if possible (even better, attach one of these files), so we can see exactly how the data is structured/formatted.

[Stephen23]

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

https://www.mathworks.com/matlabcentral/answers/57446-faq-how-can-i-process-a-sequence-of-files

[Chris Martin]

The files name are like this

 GSM-2_2002244-2002273_0030_GRGS_0080_03v3.txt
 GSM-2_2002274-2002304_0031_GRGS_0080_03v3.txt

like this there are 153 files are there and I want to load them one by another and the file content is like below

 FIRST GSM-2_2002274-2002304_0031_GRGS_0080_03v3.txt SHM     CNES / GRGS  20161025
 EARTH 0.3986004415E+15 0.6378136460E+07
 SHM      80   80 1.00 fully normalized exclusive permanent tide
 GRCOF2    0    0 0.100000000000E+01 0.000000000000E+00 0.0000E+00 0.0000E+00 19500101.0000 20500101.0000 nnnn
 GRCOF2    1    0 -.115491468132E-09 0.000000000000E+00 1.1643E-15 0.0000E+00 19500101.0000 20500101.0000 ynyn

I am trying it but have no clue how to proceed thats why I posted this, I have little knowledge on computer programming Regards...

Answer 1

[Image Analyst]

This is a FAQ, perhaps the most F of the FAQs that we see. So see the FAQ document on your question:

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

[Rik]

It always amazes me how people find this forum and decide to post their question, without even going to the trouble of a single Google search.

[Jan]

@Rik: Note that some users consider it even as "arrogant", if it is mentioned, that an internet search engine solves many problems faster than the forum. Obviously forum users have the right not to search by their own, but to let others do this for them. See https://www.mathworks.com/matlabcentral/answers/127327-vector-0-to-2pi-with-1000-point-equal-spacing#comment_486300

[Rik]

I hope I don't need to clarify I don't mean to come across as arrogant, I'm genuinely surprised how you can find this forum, but not have the idea to Google the problem. Of course everyone has their own style of looking for a solution to a problem, but it seems so obvious to me that I just don't understand it.

Of course, for some problems it can be tricky to find the right keywords. I have spent many hours in frustrating searches to simple things for which I couldn't find the right keywords to search for. In my experience LaTeX is notorious for calling things just slightly different from what I would call them (similar to how new Matlab users sometimes confuse matrix and table).

[Image Analyst]

Several times in the past when I have given a link to the FAQ, or to Vision Bibliography to find an algorithm, instead of providing a turnkey solution with full source code, people have accused me of being rude, and basically say to shut up if all I do is to give them a link. I ignore them.

In the past some would refer them to "Let me Google that for you" http://lmgtfy.com/?q=MATLAB+FAQ

Kind of funny, though maybe a bit less so since they toned it down (removed the snarky "There, was that so hard?" comment).

Answer 2

[Cedric]

You can use the output of DIR to produce a struct array of file names/folders and iterate through files:

 D = dir( '*.txt' ) ;
 for fileId = 1 : numel( D )
    % Do something with D(fileId).name.
 end

and then there are many functions (TEXTSCAN, IMPORTDATA, READTABLE, etc) for reading/parsing the content of text files when the structure seems to be a regular table/array with 3 rows of header.

If you need data from the header, you can use e.g. pattern matching through REGEXP.

Train on a single file, and when you understand how that work try to integrate it in the loop. Here I created a file named based on your example, and I filed it with the slice of content that you provided.

 >> content = fileread( 'GSM-2_2002244-2002273_0030_GRGS_0080_03v3.txt' ) ;
 >> earthParams = str2double( regexp( content, 'EARTH (\S+) (\S+)', 'tokens', 'once' )) ;

with that you get:

 >> earthParams(1)
 ans =
   3.9860e+14
 >> earthParams(2)
 ans =
   6.3781e+06

 >> data = textscan( content, '%s%d%d%f%f%f%f%f%f%s', 'HeaderLines', 3, 'CollectOutput', true ) ;

and I let you look at what data contains.