delete(filename) in parfor ?? does it execute right away? is it delayed ??
3 views (last 30 days)
Show older comments
I am trying to use delete(filename) inside the iterations of the parfor, and it looks like the file is still there even when I see that the script has actually executed passed it by following the print outs. I do not see any no error or warning message about Transparence violation.
I am using some sizeable .bmp files as temprary store and have to delete them once they are consumed to avoid runing out of disk space, and would like the delete to execute as soon as possible.
code looks like this:
parfor idx=1:n
imfilename=strcat('xxxx_',num2str(idx),'.bmp')
imread(imfilename)
.... some processing code
delete(imfilename)
... more processing code
fprintf('idx=%d executing here...\n',idx)
end
and I see the printouts "idx=?? executing here.." on the screen, but the file is still there in the folder and disk space is not freeing up??
-Thanks,
-Michael
0 Comments
Accepted Answer
Walter Roberson
on 27 Jun 2021
My tests say they are deleted immediately, at least on MacOS.
td = tempdir();
parfor idx = 1 : 20
t = getCurrentTask();
if isempty(t)
error('Bah! This is not a real parfor!');
end
tid = t.ID;
outfile = fullfile(td, sprintf('junk_%d_%d.junk', tid, idx));
[fid, msg] = fopen(outfile, 'w');
if fid < 0
error('failed to open file "%s" because "%s"', outfile, msg);
end
fwrite(fid, rand(1,1e6));
fclose(fid);
dinfo = dir( fullfile(td, '*.junk') );
filenames = strjoin({dinfo.name}, ', ');
fprintf('task %d iter %d before delete files are: %s\n', tid, idx, filenames);
delete(outfile);
dinfo = dir( fullfile(td, '*.junk') );
filenames = strjoin({dinfo.name}, ', ');
fprintf('task %d iter %d after delete files are: %s\n', tid, idx, filenames);
end
dinfo = dir( fullfile(td, '*.junk') );
filenames = strjoin({dinfo.name}, ', ');
if isempty(filenames)
fprintf('after parfor, no remaining files\n');
else
fprintf('after parfor files are: %s\n', filenames);
end
task 1 iter 1 before delete files are: junk_1_1.junk, junk_2_2.junk, junk_3_3.junk, junk_4_4.junk
task 3 iter 3 before delete files are: junk_1_1.junk, junk_2_2.junk, junk_3_3.junk, junk_4_4.junk
task 3 iter 3 after delete files are: junk_1_1.junk, junk_2_2.junk, junk_4_4.junk
task 1 iter 1 after delete files are: junk_2_2.junk, junk_4_4.junk
task 1 iter 6 before delete files are: junk_1_6.junk, junk_2_8.junk, junk_4_12.junk
task 1 iter 6 after delete files are: junk_2_8.junk, junk_3_9.junk, junk_4_12.junk
task 1 iter 5 before delete files are: junk_1_5.junk, junk_2_7.junk, junk_4_11.junk
task 1 iter 5 after delete files are: junk_2_7.junk, junk_4_11.junk
task 1 iter 14 before delete files are: junk_1_14.junk, junk_2_16.junk, junk_4_15.junk
task 1 iter 14 after delete files are: junk_4_15.junk
task 1 iter 18 before delete files are: junk_1_18.junk, junk_2_19.junk, junk_4_20.junk
task 1 iter 18 after delete files are: junk_2_19.junk, junk_4_20.junk
task 2 iter 2 before delete files are: junk_2_2.junk, junk_4_4.junk
task 2 iter 2 after delete files are:
task 2 iter 8 before delete files are: junk_1_5.junk, junk_2_8.junk, junk_3_9.junk, junk_4_12.junk
task 2 iter 8 after delete files are: junk_1_5.junk, junk_3_9.junk, junk_4_12.junk
task 2 iter 7 before delete files are: junk_2_7.junk, junk_4_11.junk
task 2 iter 7 after delete files are: junk_3_13.junk, junk_4_11.junk
task 2 iter 16 before delete files are: junk_1_14.junk, junk_2_16.junk, junk_4_15.junk
task 2 iter 16 after delete files are: junk_4_15.junk
task 2 iter 19 before delete files are: junk_1_18.junk, junk_2_19.junk, junk_4_20.junk
task 2 iter 19 after delete files are: junk_4_20.junk
task 3 iter 10 before delete files are: junk_1_6.junk, junk_2_8.junk, junk_3_10.junk, junk_4_12.junk
task 3 iter 10 after delete files are: junk_1_6.junk, junk_2_8.junk, junk_4_12.junk
task 3 iter 9 before delete files are: junk_1_5.junk, junk_2_7.junk, junk_3_9.junk, junk_4_11.junk
task 3 iter 9 after delete files are: junk_1_5.junk, junk_2_7.junk, junk_4_11.junk
task 3 iter 13 before delete files are: junk_1_14.junk, junk_2_16.junk, junk_3_13.junk, junk_4_15.junk
task 3 iter 13 after delete files are: junk_1_14.junk, junk_2_16.junk, junk_4_15.junk
task 3 iter 17 before delete files are: junk_1_18.junk, junk_2_19.junk, junk_3_17.junk
task 3 iter 17 after delete files are: junk_1_18.junk, junk_2_19.junk, junk_4_20.junk
task 4 iter 4 before delete files are: junk_2_2.junk, junk_4_4.junk
task 4 iter 4 after delete files are:
task 4 iter 12 before delete files are: junk_1_5.junk, junk_3_9.junk, junk_4_12.junk
task 4 iter 12 after delete files are: junk_1_5.junk, junk_3_9.junk
task 4 iter 11 before delete files are: junk_1_14.junk, junk_2_16.junk, junk_3_13.junk, junk_4_11.junk
task 4 iter 11 after delete files are: junk_1_14.junk, junk_2_16.junk, junk_3_13.junk
task 4 iter 15 before delete files are: junk_1_18.junk, junk_2_19.junk, junk_3_17.junk, junk_4_15.junk
task 4 iter 15 after delete files are: junk_1_18.junk, junk_2_19.junk, junk_3_17.junk
task 4 iter 20 before delete files are: junk_4_20.junk
task 4 iter 20 after delete files are:
after parfor, no remaining files
Notice that in each case, the "after delete" does not have any files junk_* starting with the number that is the same as the task number. For example,
task 3 iter 9 after delete files are: junk_1_5.junk, junk_2_7.junk, junk_4_11.junk
The junk_1 junk_2 junk_4 exist because those are being created simultaneously in other workers, but junk_3 has been deleted immediately in worker 3.
More Answers (0)
See Also
Categories
Find more on File Operations in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!