My exported EDF file is up to 100 times bigger than my imported EDF file
14 views (last 30 days)
Show older comments
Hi everyone,
I would like to import a set of EDF files, remove some artifacts and export again as EDF files (artifact free signal). The code in general works fine, but while the size of imported EDF files is 10-40 MB, the size of exported one is 600-800 MB. In the following code I excluded the lines related to artifact removal:
%% Preparation
clc
clear all
close all
SR=1000; % Sampling Rate in Hertz
start_path = fullfile('EEG Analysis Code');
topLevelFolder = uigetdir(start_path);
allSubFolders = genpath(topLevelFolder);
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames);
set(0,'DefaultFigureVisible','on') %Change 'off' to 'on' if you want plotted figures to show, also refer to the last line of code (313)
% This block of code is where data collecting and processing begins ---------------------------------------
for k = 1 : numberOfFolders
thisFolder = listOfFolderNames{k}; % This specifies the current folder for searching
fprintf('Processing folder %s', thisFolder);
filePattern = sprintf('%s/*.edf', thisFolder); % This obtains the names of EDF files in the current folder
baseFileNames = dir(filePattern); % This saves the name of EDF files in the current folder
numberOfEDFFiles = length(baseFileNames); % This counts the number of EDF files present
Summary_Names = {zeros(numberOfEDFFiles,1)}; % Saves the mouse number for export to excel
Summary_Data = zeros(numberOfEDFFiles,3); % Saves the mouse spikes/seizures for export to excel
if numberOfEDFFiles >= 1 %This loop scans through all the EDF files and extracts data
for f = 1:numberOfEDFFiles
fullFileName = fullfile(thisFolder, baseFileNames(f).name);
fprintf('\nProcessing EDF file %s \n \n', fullFileName);
Mouse_Number = erase(baseFileNames(f).name, '.edf'); % This saves the mouse number (e.g. S20, S21, etc.)
Data = edfread(fullFileName,'TimeOutputType','datetime'); % This is where EDF data is brought into matlab
Data.Properties.DimensionNames = ["DateTime" "Variables"]; % Converts EDF data to dd/mm/yyyy - hh/mm/ss format
info = edfinfo(fullFileName); % Saves parameters for the EDF file
FilenameStr=info.Filename;
PatientStr = info.Patient;
RecordingStr = info.Recording;
StartDateStr = info.StartDate;
StartTimeStr = info.StartTime;
Record = 1; % EEG Record
Lead = 1; % EEG Lead/Electrode
Data.Properties.VariableNames(1)=cellstr('EEG'); % in the case of wrong labeling of the signal
Voltage_V = vertcat(Data.EEG{:});
Time_total = info.NumDataRecords*info.DataRecordDuration; % Total of data points in the EEG trace
Time_seg = Time_total/length(Voltage_V); % Converts frequency to time in seconds
% Date and Time Conversion --------------------------------------------------------------------
Start_Date = Data.DateTime(1,1); % Start date and time for EEG trace
End_Date = Data.DateTime(end,1); % End date and time for EEG trace
DateTime_Vector = linspace(Start_Date,End_Date,length(Voltage_V)); % Creates array of date/time variables
Timeframe = round(hours(End_Date-Start_Date),1); % Rounds length of EEG trace to nearest 0.1 hours
Timeframe_String = num2str(Timeframe); % Converts length of EEG trace to a string variable
end
end
end
%% edfheader
hdr = edfheader("EDF+");
hdr.Patient = PatientStr;
hdr.Recording = RecordingStr;
hdr.StartDate = StartDateStr;
hdr.StartTime = StartTimeStr;
hdr.Reserved = "EDF+C";
hdr.NumSignals = 1;
hdr.NumDataRecords = length(Voltage_V)/SR;
hdr.SignalLabels="EEG";
hdr.TransducerTypes="";
hdr.PhysicalDimensions="mV";
hdr.PhysicalMin = min(Voltage_V);
hdr.PhysicalMax = max(Voltage_V);
hdr.DigitalMin = [-32256];
hdr.DigitalMax = [32256];
hdr.DataRecordDuration = [seconds(1)];
%%
Fname=strrep(FilenameStr, '.edf', '_revised.edf');
edfwrite(Fname,hdr,Voltage_V,'InputSampleType',"physical");
What could be the reason for this issue?
Thanks in advance for your suggestions.
All the best,
Sadegh
Update 10/11/2023:
P.S. Below, I am sharing an EDF file that I am importing with edfread. After the export with edfwrite, it is more than 600 MB:
https://drive.google.com/file/d/1FD8MmqYhyvVuEi_Wjj-9cnf1d9usJlUW/view?usp=sharing
0 Comments
Accepted Answer
Alex Wu
on 17 Nov 2023
Hi Sadegh,
This appears to be a deficiency in the software. When 'Reserved' in header is set to something other than an empty string "", edfwrite function will add an annotation as a placeholder for each data record, which is unnecessary. But even blank annotation placeholders are huge, especially in this case where you have 18000 data records which means 18000 redundant placeholders.
This issue will be addressed in an upcoming release. Thanks for reporting it.
As for the current workaround, if you want to reduce the saved file size, you can set hdr.Reserved to "" instead of "EDF+C".
More Answers (2)
Sulaymon Eshkabilov
on 8 Nov 2023
Without seeing your data files and just quick looking at your code, this can be concluded. You are augmenting data from numberOfEDFFiles of files that means you are exporting the augmented data into a single file that answers your question of much larger exported file size.
Peter Perkins
on 10 Nov 2023
I don't know much about EDF, and there's no fil attached, but my guess is that the culprit is
Voltage_V = vertcat(Data.EEG{:});
Post the file, or give details what's in it.
See Also
Categories
Find more on AI for Signals in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!