How can I analyse multiple variables with similar names after importing them in a loop?

I have a dataset of 246 text files and I can easily load them in MATLAB as 246 variables with exactly the same names as the text files.

files = dir('.../controls/*.txt');
for k = 1:length(files)
load(fullfile('/controls/', files(k).name), '-ascii');
end

The problem is that now I have to modify the variables and work on specific columns. But, unfortunately, I cannot do this in a loop as I do not have access to the names of the variables after loading them.

For example, I want to modify the second column:

L_C9_16(:,2) = L_C9_16(:,2) * 20.32;

And this has to be done to all the variables that have been loaded.

Is there anyway I can simply work on all the variables that are now loaded in a loop? I am not sure if it helps, but the names of the variables are quite similar as they all start with L_C and then it changes according to each subject.

Many thanks in advance :)

Answer 1

[Matt J]

You should always call load() with an output argument. In this case, it allows you to put the file contents into something indexable, like a cell array.

   for k = 1:length(files)
      S{k} = load(fullfile('/controls/', files(k).name), '-ascii');
   end

and now you can do column-wise manipulations as follows,

   S{k}(:,2) = S{k}(:,2) * 20.32;

[Amir Dehsarvi]

Hello Matt, Thank you so much for your reply. This works perfectly fine and gives me a variable (here "S"), which stores all my data in each cell. Is there any way we can have them separately with the names of the text files though? Also, I think they should be sorted according to the files being imported (I guess it is not sorted by file name, which makes things a bit confusing and complicated).

[Matt J]

Is there any way we can have them separately with the names of the text files though?

You can read at length here about why you shouldn't do this,

http://www.mathworks.com/matlabcentral/answers/143-how-do-i-make-a-series-of-variables-a1-a2-a3-a10

Basically, if you do, you lose the ability to index the variables, to loop over them, and to process them as a group.

However, a possible compromise -- one which would allow you to index by name -- would be to make S a struct and use dynamic fieldnames,

     for k = 1:length(files)
      kthName=  files(k).name;
      S.(kthName) = load(fullfile('/controls/', kthName), '-ascii');
   end

[Amir Dehsarvi]

Hello again Matt, and thank you so much for your help. I changed the code to have the names as you recommended and I get the following error:

Invalid field name: 'L_C11_3.txt'.

As you can see, the names should not be problematic as they all start with a letter and they look like a normal string...

[Steven Lord]

The extension (or more specifically, the period separating it from the first part of the file name) is what makes that an invalid field name. You can use FILEPARTS to remove the extension.

If there's a chance your filenames contain characters other than letters, numbers, and underscores you may want to pass them through matlab.lang.makeValidName, matlab.lang.makeUniqueStrings, or GENVARNAME before trying to use them as field names.

Answer 2

[Stephen23]

https://www.mathworks.com/help/matlab/matlab_prog/string-evaluation.html

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[Amir Dehsarvi]

Brilliant. Thank you very much for detailed explanations. I am definitely going to use this method from now on and I read more or less all the information in the links.

On another note, now I have modified my code and am trying to work on vectors inside the structure (arrays), but when I have an if statement inside the for loop, it doesn't do what I am expecting it to. This is to remove the unwanted (pen-up) data.

Plus, I need to extract the corners of a drawing task data. i.e., we have asked subjects to draw some patterns for us like cubes and now I have to detect the corners of each drawing in order to compare it to the original pattern that we have (although the pattern is so big and I have problems matching them with the drawings) and analyse the tremor when drawing. What I was doing was plotting the graphs for each vector (I have x and y coordinates) to get the whole drawing and then I wanted to compare it to the original template using imshowpair (do we have any other ways of doing it that seems more reasonable to you?). But findpeak does not work either as I just need the corners that act as local maximums (I used the threshold to get only 4 maximums for cubes, etc.). I understand the code for this part is incomplete, I just stopped as it was not doing what I wanted.

I have the simple code:

files = dir('.../controls/*.txt');
for k = 1:length(files)
  data{k} = load(fullfile('.../controls/', files(k).name), '-ascii');
      if data{k}(:,6) < 0.1
          data{k}(:,6) = 0;
      end
  data{k}(:,2) = data{k}(:,2) * 20.32;
  data{k}(:,3) = data{k}(:,3) * 15.24;
   % Rescaling the drawings to match the 
   % dimensions of the active area of
   % the tablet, which were 20.32 x 15.24 cm
  plot(data{1,k}(:,2));
  [pksx{k},locsx{k}] = findpeaks(data{1,k}(:,2),'MinPeakHeight',11);
  plot(data{1,k},data{1,k}(:,2),data{1,k}(locsx{k}),pksx{k},'or')
  plot(data{1,k}(:,3));
  [pksy{k},locsy{k}] = findpeaks(data{1,k}(:,3),'MinPeakHeight',11);
  plot(data{1,k},data{1,k}(:,3),data{1,k}(locsy{k}),pksy{k},'og')
end