import over 1000 files
2 views (last 30 days)
Show older comments
i want to change many files including files in subfolders.
but if file outnumber 1001, my code stops.
i'd like to import over 1000 files to "files" struct.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/699532/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/699612/image.png)
2 Comments
Accepted Answer
Paul Kaufmann
on 4 Aug 2021
Don't know what the problem is with exceeding N=1000. Works for me:
path = "data/"
files = [ dir(fullfile(path,'**','*.txt')); dir(fullfile(path,'**','*.txt')) ]; % do it twice, to exceed 1000
N = numel(files)
Gives this result for my datafolder:
ans = 1890
As I assume you want to rename files or something, you can now just iteratively work on them.
However the following will get you into trouble:
>> numel(files.name)
Warning: Number of elements exceeds maximum flint 2^53-1.
The result may be inaccurate.
ans =
Inf
0 Comments
More Answers (3)
Chunru
on 31 Jul 2021
You may hit the limit os the number of files that can be opened simutaneously. While it possible to increase the maximum number of opened files in OS, you may want to try open file one by one and then close the file once it is not used.
0 Comments
Image Analyst
on 31 Jul 2021
Try code in the FAQ: https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F
It will work for more than 1000 files, if you have more than that.
% Specify the folder where the files live.
myFolder = 'G:\';
% Check to make sure that folder actually exists. Warn user if it doesn't.
if ~isfolder(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder);
uiwait(warndlg(errorMessage));
myFolder = uigetdir(); % Ask for a new one.
if myFolder == 0
% User clicked Cancel
return;
end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.png'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
numFiles = length(theFiles)
Result = zeros(numFiles, 1);
for k = 1 : numFiles
baseFileName = theFiles(k).name;
fullFileName = fullfile(theFiles(k).folder, baseFileName);
fprintf('Now reading %s, file #%d of %d.\n', fullFileName, k, numFiles);
% Now do whatever you want with this file name,
% such as reading it in as an image array with imread()
imageArray = imread(fullFileName);
imshow(imageArray); % Display image.
drawnow; % Force display to update immediately.
Result(k) = mean(imageArray(:)); % Whatever you want.
end
14 Comments
michael adams
on 14 Dec 2023
I can confirm this is an issue and it is not consistent. If I query many directories (all with more than 1000 files), sometimes 'dir' returns the correct structure of all files, but sometimes it just stops at 1000.
I do not run into this issue on local drives. I see this issue on network drives only.
1 Comment
Rik
on 15 Dec 2023
How did you confirm this is an issue with Matlab and not with the network architecture and/or the OSes involved?
See Also
Categories
Find more on Environment and Settings in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!