Getting the average and then the standard deviation of multiple .mat files

Hi all,

Slightly over my head here and need some help! I am trying to get the average of several .mat files, each being 8x22 workspaces of data (see the screenshots for more information). My code is below but despite my efforts, I have no idea how to get the average of each file as well as their standard deveation. As you can see, I have loaded them all through the following process but have gotten stuck with the averaging part for now. Could anyone help?

myFolder = '/Users/ben/Desktop/Stats part/Data'; % Define your working folder
if ~isdir(myFolder)
  errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
  uiwait(warndlg(errorMessage));
  return;
end
filePattern = fullfile(myFolder, '*.mat');
matFiles = dir(filePattern);
for k = 1:length(matFiles)
  baseFileName = matFiles(k).name;
  fullFileName = fullfile(myFolder, baseFileName);
  fprintf(1, 'Now reading %s\n', fullFileName);
  matData(k) = load(fullFileName);
end
% the loaded data is in the workspace called matData.
 
 Z = rand(1,16,16);
 Z_m = mean(A(1:20,:,:));

Thanks in advance!!

[Stephen23]

"see the screenshots for more information"

You forgot to attach any files.

"each being 8x22 workspaces of data"

Do all of the .mat files contain the same variables? Which of these variables do you want to work with?

[Ben Timm]

Ahh! That was a rookie error, my bad!

All of the .mat files have different numbers in them but aside from that, they are of the same format and do not change at all from what is seen in the screenshots.

Thanks again!

Answer 1

[Vinai Datta Thatiparthi]

Hey Ben,

From your description, I understood that you're trying to evaluate the mean and standard deviation values for each of the 16 matrix values that you have inside your struct matData. So, your output results would be 16 values for mean & 16 more for standard deviation.

If that's the case,

% Number of elements within the structure matData
len = numel(fieldnames(matData));

% Array to store average of every matrix
meanArr = zeros(1, len);

% Array to store standard deviation of every matrix
stdArr = zeros(1, len);

% Iterate through every element (matrix) within matData to calculate mean & std
for idx = 1:len
    data = matData(idx).average_final_results;
    
    % Store into array
    meanArr(idx) = mean(data,'all');
    stdArr(idx) = std(data, 0, 'all');
end

Documentation links for mean & std.

Note: Since you've not mentioned what exactly you're calculating std for, I've gone ahead and assumed a simple case and calculated for the entire matrix.

Hope this helps!

[Ben Timm]

Hi Vinai,

Thank you so much for getting back to this! I would have been completely stumped without it..

Just to make sense of what you've done here. The two Arr variables in the workspace (as seen in the screenshot), are the final results, particularly the standard deviation? What are the idx, k, len and data variables?

Essentially my aim was to get a model that understood the average and standard deviation, then my following aim for this was to see if each individual group was normally distributed. What you have done so far is incredible thank you! But could you tell me how/what/where I go from there to get the following step done?

Thanks in advance!

[Vinai Datta Thatiparthi]

Hey Ben,

Glad I could help.

Further,

• Yes, the meanArr & stdArr arrays were meant to hold the mean & std values of each of the 16 matrices of the matData file.
• The variable len was meant to hold the value of the number of matrices within the struct array (matData)
• idx is the variable used to iterate through all the matrices within the struct array (matData)
• For every iteration, the matrix values are stored in a temporary variable, data

Finally, the function normplot may be of help to you. This function creates a normal probability plot comparing the distribution of the data to the normal distribution. This may help you evaluate if your data is normally distributed or not.

Hope this helps!

[Ben Timm]

Definitely! You're an angel, thank you!

I will look into the normplot function now, would it be possible to get that distribution from the array you created?

Kind regards,

Ben

[Vinai Datta Thatiparthi]

Hey Ben,

I guess using normplot on the variable data is what you're looking for. Since you wanted to check if each individual group was normally distributed, maybe you could use normplot iteratively within the for loop itself.

Hope this helps!

[Ben Timm]

Hi Vinai,

Thank you so much for the help! It is incredibly appreciated.

Kind regards,

Ben