How do I import multiple text files and automatically add them into a single cell array? Maybe 3D?

Hey everyone, I'm trying to streamline some mass spec data I want to prepocess before uploading to a statistical software package. I exported my data as ASCII to just get the x and y coordinates and where each spectra is saved in seperate file. I have saved all these files I want to process into one folder.

Below is the script I used so far in order to import the data to matlab and it's not the greatest.

%This code extracts the data from the folder
%written for ".txt" files  below
files=dir('*.txt');
names_cell={files.name}

%Extract the Y axis for each spectra in out all of them into a single array
for i=1:length(names_cell)
     Y(:,i)=dlmread(names_cell{i},' ',0,1);
end

%Extract the X axis from the first data file (assuming they are all te same
for i=1
    MZ=dlmread(names_cell{1},' ',0,0)
    MZ(:,2)=[]
end

This has avenue hasn't been fruitful since many Matlab Answers recommends to avoid dynamic variable naming and down the line, this method has made things more confusing when writing script. My biggest concern is the script I created above above assumes that the x axis (m/z) is uniform for each data file (I might input data from different instruments soon). Also maintaining the filename would be great so I know what spectra goes with what sample.

Is there way to generate a cell array that contains the filename, the x axis, and the y axis? Would making a 3D array help becuase that has gotten me out of pickles before, but not sure how to do it with importing data.

Thanks for any help!! Going through the school of hard knocks to learn Matlab.

Answer 1

[Bob Thompson]

In looking over your code I don't actually see any dynamic naming occuring. Dynamic naming is manually naming variables with different numbers (a1, a2, a3, a4, etc) and should be replaced with indexed type stuff (a(1),a(2),a(3),etc).

Based on the data you have, I think it might be most efficient to leave the names in the original structure array generated by dir(), and to add your data directly to that.

files = dir('*.txt');
for i = 1:length(files)
    files(i).data = dlmread(files(i).name,' ',0,1);
    files(i).x = files(i).data(:,1); % If you really want x and y separate
    files(i).y = files(i).data(:,2); 
end

I suggest leaving it in the structure array because you are able to get your 'third' dimension from indexing the different entries in the structure, and the file names are already contained here. If I were to use a cell array I would likely do a similar arrangement, with rows representing different files, and columns for the file name, data, and other important pieces of information.

files = dir('*.txt');
for i = 1:length(files)
    stuff{i,1} = files(i).name;
    stuff{i,2} = dlmread(files(i).name,' ',0,1);
end

The only time I would use a 3D array is if I were putting all of the data into a single array, without the names.

files = dir('*.txt');
for i = 1:length(files)
    data(:,:,i) = dlmread(files(i).name,' ',0,1);
end

The benefit of using this last method is that if your assumption that all x ranges are the same is correct (or at least that the same number of values exist) then the data is much easier to access from an indexing perspective, but you do have to reference the files list to know the names of the files associated with each page.

[Stephen Zambrzycki]

Thank you so much for the help!! For the first script you mentioned, I kept getting a "Index exceeds matrix dimensions" for the 5th line of that script.

files = dir('*.txt');
for i = 1:length(files)
    files(i).data = dlmread(files(i).name,' ',0,1);
    files(i).x = files(i).data(:,1); % If you really want x and y separate
    files(i).y = files(i).data(:,2); %%ERROR: Index exceeds matrix dimensions%%
end

I tried the cell array method and that worked perfectly! Only modicication to the script I needed to do was for the 4th line of code where I changed a 1 to a 0

files = dir('*.txt');
for i = 1:length(files)
    stuff{i,1} = files(i).name;
    stuff{i,2} = dlmread(files(i).name,' ',0,0);
end

This allowed me to incorporate the x axis (mz) to each array with the y axis (intensity) data. Now everything is super organized.

Thanks you so much for the help and I also learned more about constructing loops!!

[Bob Thompson]

Mmm, in thinking it over a bit I suspect the error for the first method was that the dlmread output your data in to a cell by default, so there wouldn't be a second column of data to call (all data would be in a single cell). Whether it's worth investigating further or not is up to you, but I'm glad you found at least one solution that works for you.