Matlab freeze when fprintf to file
11 views (last 30 days)
Show older comments
Hi
I have a large amount of data, many years. It is given in daily files.
For reasons irrelevant for the question I have to combine these files to yearly data files ~ 3Gb.
The daily files are of .cdf format and contain more variables than needed and also lack some that has to be derived.
The following shows an example that makes my computer freeze for around 5 min. But that data datafile is complete with seconds.
fid = fopen('test.m', 'w');
data = randi(5, 40000, 10);
for i=1:2 %5
fprintf(fid, '%d %d %d %d %d %d %d %d %d %d\n', data);
end
fprintf(fid, '%d %d %d %d %d %d %d %d %d %d\n', zeros(1, 10)); %Newly added line
fclose(fid)
Does anyone know why it freezes for so long when the files is written so fast?
Using Matlab R2018a on a remote desktop with Ubuntu OS
4 Comments
Stephen23
on 3 Aug 2018
Edited: Stephen23
on 3 Aug 2018
@Michael Madelaire: this sounds similar to when many files are opened but not closed. When there are too many open file then MATLAB can become slow or unresponsive. Run fclose('all') and check your code carefully that it really closes all files properly.
How many files do you have open at once?
Accepted Answer
OCDER
on 3 Aug 2018
Instead of 'test.m', write to 'test.txt' or something else. The fprintf is fast, but having a "test.m" file in the current directory seems to freeze Matlab.
fid = fopen('test.txt', 'w'); %Don't write to .m files, unless it's a matlab file type
data = randi(5, 40000, 10);
for i=1:2 %5
fprintf(fid, '%d %d %d %d %d %d %d %d %d %d\n', data);
end
fprintf(fid, '%d %d %d %d %d %d %d %d %d %d\n', zeros(1, 10)); %Newly added line
fclose(fid)
My Matlab also freezes when there is a "test.m" file in the current directory that Matlab cannot determine if it's a script or function - it just says it's a "Matlab Code File". I'm guessing this step for determining what file type this "test.m" file is causes Matlab to freeze. I think this should go into the bug report for 2017 versions - not sure if 2018 version fixed this.
6 Comments
Stephen23
on 4 Aug 2018
Edited: Stephen23
on 4 Aug 2018
"But I am getting a bit tired of this issue and am just going to use textscan."
Or you could just follow the instructions given in the error message.
Or use dlmread.
Both would be much simpler than using textscan.
Or use save and load (these are designed to be used together).
OCDER
on 4 Aug 2018
do you want to save/load matlab variables?
data = randi(5, 40000, 10)
save('test.mat', 'data', '-v7.3', '-nocompression') %works for > 2GB files
S = load('test.mat', 'data')
S.data = %your 'data'.
More Answers (0)
See Also
Categories
Find more on Large Files and Big Data in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!