My exported EDF file is up to 100 times bigger than my imported EDF file

9 views (last 30 days)
Hi everyone,
I would like to import a set of EDF files, remove some artifacts and export again as EDF files (artifact free signal). The code in general works fine, but while the size of imported EDF files is 10-40 MB, the size of exported one is 600-800 MB. In the following code I excluded the lines related to artifact removal:
%% Preparation
clc
clear all
close all
SR=1000; % Sampling Rate in Hertz
start_path = fullfile('EEG Analysis Code');
topLevelFolder = uigetdir(start_path);
allSubFolders = genpath(topLevelFolder);
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames);
set(0,'DefaultFigureVisible','on') %Change 'off' to 'on' if you want plotted figures to show, also refer to the last line of code (313)
% This block of code is where data collecting and processing begins ---------------------------------------
for k = 1 : numberOfFolders
thisFolder = listOfFolderNames{k}; % This specifies the current folder for searching
fprintf('Processing folder %s', thisFolder);
filePattern = sprintf('%s/*.edf', thisFolder); % This obtains the names of EDF files in the current folder
baseFileNames = dir(filePattern); % This saves the name of EDF files in the current folder
numberOfEDFFiles = length(baseFileNames); % This counts the number of EDF files present
Summary_Names = {zeros(numberOfEDFFiles,1)}; % Saves the mouse number for export to excel
Summary_Data = zeros(numberOfEDFFiles,3); % Saves the mouse spikes/seizures for export to excel
if numberOfEDFFiles >= 1 %This loop scans through all the EDF files and extracts data
for f = 1:numberOfEDFFiles
fullFileName = fullfile(thisFolder, baseFileNames(f).name);
fprintf('\nProcessing EDF file %s \n \n', fullFileName);
Mouse_Number = erase(baseFileNames(f).name, '.edf'); % This saves the mouse number (e.g. S20, S21, etc.)
Data = edfread(fullFileName,'TimeOutputType','datetime'); % This is where EDF data is brought into matlab
Data.Properties.DimensionNames = ["DateTime" "Variables"]; % Converts EDF data to dd/mm/yyyy - hh/mm/ss format
info = edfinfo(fullFileName); % Saves parameters for the EDF file
FilenameStr=info.Filename;
PatientStr = info.Patient;
RecordingStr = info.Recording;
StartDateStr = info.StartDate;
StartTimeStr = info.StartTime;
Record = 1; % EEG Record
Lead = 1; % EEG Lead/Electrode
Data.Properties.VariableNames(1)=cellstr('EEG'); % in the case of wrong labeling of the signal
Voltage_V = vertcat(Data.EEG{:});
Time_total = info.NumDataRecords*info.DataRecordDuration; % Total of data points in the EEG trace
Time_seg = Time_total/length(Voltage_V); % Converts frequency to time in seconds
% Date and Time Conversion --------------------------------------------------------------------
Start_Date = Data.DateTime(1,1); % Start date and time for EEG trace
End_Date = Data.DateTime(end,1); % End date and time for EEG trace
DateTime_Vector = linspace(Start_Date,End_Date,length(Voltage_V)); % Creates array of date/time variables
Timeframe = round(hours(End_Date-Start_Date),1); % Rounds length of EEG trace to nearest 0.1 hours
Timeframe_String = num2str(Timeframe); % Converts length of EEG trace to a string variable
end
end
end
%% edfheader
hdr = edfheader("EDF+");
hdr.Patient = PatientStr;
hdr.Recording = RecordingStr;
hdr.StartDate = StartDateStr;
hdr.StartTime = StartTimeStr;
hdr.Reserved = "EDF+C";
hdr.NumSignals = 1;
hdr.NumDataRecords = length(Voltage_V)/SR;
hdr.SignalLabels="EEG";
hdr.TransducerTypes="";
hdr.PhysicalDimensions="mV";
hdr.PhysicalMin = min(Voltage_V);
hdr.PhysicalMax = max(Voltage_V);
hdr.DigitalMin = [-32256];
hdr.DigitalMax = [32256];
hdr.DataRecordDuration = [seconds(1)];
%%
Fname=strrep(FilenameStr, '.edf', '_revised.edf');
edfwrite(Fname,hdr,Voltage_V,'InputSampleType',"physical");
What could be the reason for this issue?
Thanks in advance for your suggestions.
All the best,
Sadegh
Update 10/11/2023:
P.S. Below, I am sharing an EDF file that I am importing with edfread. After the export with edfwrite, it is more than 600 MB:
https://drive.google.com/file/d/1FD8MmqYhyvVuEi_Wjj-9cnf1d9usJlUW/view?usp=sharing

Accepted Answer

Alex Wu
Alex Wu on 17 Nov 2023
Hi Sadegh,
This appears to be a deficiency in the software. When 'Reserved' in header is set to something other than an empty string "", edfwrite function will add an annotation as a placeholder for each data record, which is unnecessary. But even blank annotation placeholders are huge, especially in this case where you have 18000 data records which means 18000 redundant placeholders.
This issue will be addressed in an upcoming release. Thanks for reporting it.
As for the current workaround, if you want to reduce the saved file size, you can set hdr.Reserved to "" instead of "EDF+C".
  1 Comment
Sadegh Rahimi
Sadegh Rahimi on 17 Nov 2023
Hi Alex,
Thanks for your informative reply. I will set hdr.Reserved to "", hoping to see a significant reduction in the size of exported files.

Sign in to comment.

More Answers (2)

Sulaymon Eshkabilov
Sulaymon Eshkabilov on 8 Nov 2023
Without seeing your data files and just quick looking at your code, this can be concluded. You are augmenting data from numberOfEDFFiles of files that means you are exporting the augmented data into a single file that answers your question of much larger exported file size.
  1 Comment
Sadegh Rahimi
Sadegh Rahimi on 8 Nov 2023
Edited: Sadegh Rahimi on 10 Nov 2023
Thanks for your comment, but it is not the case. At the moment, I only import one EDF file (I put one EDF file in size of 10-40 MB there) and when I export it into EDF, it is always 600-800 MB.

Sign in to comment.


Peter Perkins
Peter Perkins on 10 Nov 2023
I don't know much about EDF, and there's no fil attached, but my guess is that the culprit is
Voltage_V = vertcat(Data.EEG{:});
Post the file, or give details what's in it.
  1 Comment
Sadegh Rahimi
Sadegh Rahimi on 10 Nov 2023
Thanks for your comment. Below, I am sharing an EDF file that I am importing with edfread. After the export with edfwrite, it is more than 600 MB:
https://drive.google.com/file/d/1FD8MmqYhyvVuEi_Wjj-9cnf1d9usJlUW/view?usp=sharing

Sign in to comment.

Products


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!