How to save numerous variables from different files in a csv in a for loop

Im reading multiple files, extracting variables from each file, plotting them, then saving the variables in the a csv. So far I have figured out how to open each file in a for loop and how to plot them the way I want, I have also figured out how to save a single file in a csv file with the proper headers. What I would like to do is plot the next file in a new row, when I put the the save portion in a for loop it just saves over the data from the previous file with the new file. How do I add the next file as a new row? Code is below:

FinalData = [sprintf("File: %s", F) EndTime d_sum1 e_sum AvgSpeed1 maxSpeed minSpeed ... 
                    AvgVolt1 maxVolt minVolt AvgCur1 maxCur minCur AvgWatt1 maxWatt minWatt ...
                    AvgRoll1 maxRoll minRoll AvgPitch1 maxPitch minPitch];
T = array2table(FinalData);
T.Properties.VariableNames(1:22) = {'File Name', 'Mission Time (sec)', 'Distance Traveled', 'Mission Distance', 'Average Speed', ...
    'Maximum Speed', 'Minimum Speed', 'Average Voltage', 'Maximum Voltage', 'Minimum Voltage', 'Avgerage Current', ...
    'Maximum Current', 'Minimum Current', 'Average Wattage', 'Maximum Wattage', 'Minimum Wattage', 'Average Roll', ...
    'Maximum Roll', 'Minimum Roll', 'Average Pitch', 'Maximum Pitch', 'Minimum Pitch'};

writetable(T, 'FinalUSVData.csv')

This is what im using to save the data I want from a single file to be save on one row with the proper headers. When the next file is read, I want those variables saved on the next row. Thanks!

Answer 1

[Voss]

writetable(T, 'FinalUSVData.csv', 'WriteMode', 'append')

https://www.mathworks.com/help/matlab/ref/writetable.html#btyc9ne-1_sep_mw_612fbe52-8722-4cdc-a3ac-4c1f131f1272

[Walter Roberson]

Note that writemode append does not leave any blank rows in the file -- the new data will be written immediately after the previous data, with no header of any kind between the parts.

Answer 2

[Arjun]

Hi @Bradley,

It looks like you want to add multiple rows to a CSV file within a loop and currently your data is being overwritten.

To efficiently save data from multiple files into a CSV, accumulate the data in a single table or array (allData) during the loop over each file. Append the data from each file to this structure as you process them. After processing all files, write the entire accumulated dataset to the CSV in one operation. This minimizes file I/O operations, prevents data overwriting, and ensures all rows from every file are included in the final output, maintaining data integrity and consistency.

Kindly refer to the code below for better understanding:

% Initialize a table for storing all data
allData = [];
% Loop through each file
for i = 1:numFiles
    % Your code to read and process each file
    % ...

    % Create a table with the data from the current file
    FinalData = [sprintf("File: %s", F) EndTime d_sum1 e_sum AvgSpeed1 maxSpeed minSpeed ... 
                        AvgVolt1 maxVolt minVolt AvgCur1 maxCur minCur AvgWatt1 maxWatt minWatt ...
                        AvgRoll1 maxRoll minRoll AvgPitch1 maxPitch minPitch];
    T = array2table(FinalData);
    T.Properties.VariableNames(1:22) = {'File Name', 'Mission Time (sec)', 'Distance Traveled', 'Mission Distance', 'Average Speed', ...
        'Maximum Speed', 'Minimum Speed', 'Average Voltage', 'Maximum Voltage', 'Minimum Voltage', 'Avgerage Current', ...
        'Maximum Current', 'Minimum Current', 'Average Wattage', 'Maximum Wattage', 'Minimum Wattage', 'Average Roll', ...
        'Maximum Roll', 'Minimum Roll', 'Average Pitch', 'Maximum Pitch', 'Minimum Pitch'};
    
    % Append the new data to allData
    allData = [allData; T];
end

% Write the accumulated data to a CSV file
writetable(allData, 'FinalUSVData.csv');

You can also refer to the documentation of 'writetable' for in-depth understanding of different modes of this function: https://www.mathworks.com/help/matlab/ref/writetable.html

I hope this will help!

[Bradley]

Excellent, this worked great, thanks again!

[Arjun]

Welcome! Consider accepting the solution if it helped!