How can I process line by line while also using iteration to find a group in the data?

I have a very large data file (.txt) and that's why I get 'Out of memory' error. I want to limit the memory by reading/ process the data line by line, but I am having dificulties with this.

My data looks something like this

1 0 0.0283709 0.00291316 0.000373098 -0.00332569 e- CB 1
1 0 0.0351616 0.00312706 0.000380735 -0.00359444 e- CB 1
1 0 0 0.00337517 0.000336677 -0.00385419 e- CB 2
1 0 0 0.00343182 0.000328684 -0.00391487 e- CB 2

I would like to select all the data that has the same number in row 9 and process the data and plot it and move on to the next number. So here in this case the first and second row will be read and processed and afterwards the iteration goes to 2 and the third and fourth column will be processed.

I looked up several things but it only reads one line of the data and processes it and not a group of the data.

If anyone has an idea how to solve this... thanks!

[Guillaume]

the same number in row 9 Do you mean column 9?

the same number as what?

What does plotting a number mean?

What does will be read and processed mean?

[Gabriella Tany]

Sorry for being so vague and yes I meant column 9.

So column 2 repressents a Energy value and 3, 4, 5 the x,y,z position. I want to add the values of Energies when they are at the same coordinates. column 9 just repressents the group of data that they belong to and it goes on to how many runs I have my data, so run 1 has the number 1 in column 9 and run 2, the number 2.

What I would like to do is gather all the coordinates and Energy from run 1 and process it (adding it and plot it as a scatterplot) and then move on to run 2 etcetc.

Hope this is more clear

[Guillaume]

Can you attach a demo file? Does readtable really run out of memory parsing the file?

Note that data is a 1x9 cell array, so data(:, n) is the same as data(1, n) and the same as data(n). In addition, your cell2mat are a waste of time. The proper way to extract the content of a cell array is with {}. So all your cell2mat(data(:, n)) should be replaced by data{n}

[KSSV]

How long is your file? What's is it's size?

[Gabriella Tany]

I can run the data up untill ~3GB but I also have data sets which are 15GB...

[Guillaume]

The best solution is to indeed use tall arrays as Stephen says. More details for the desired plots are required for us to tell you how to do it.

Are all the groups plotted on the same plot?

[Gabriella Tany]

Already found the bug in my code, it's working now without the tall array :) Didn't want to give more details since it is unpublished data