You are now following this question
- You will see updates in your followed content feed.
- You may receive emails, depending on your communication preferences.
Read all files inside more than one directory
2 views (last 30 days)
Show older comments
Hello, I have a data directory that has 2 more subdirectories which contain binary data in them. the binary data are chunks of 100Sa/s binary files of data taken over a period of time. I want to read all the data inside both directories and manipulate them. in addition, All data in each directory should construct a plot. so far this is what i have and I keep getting errors when i am trying to open the files.
Files=dir('*.*');
Arrayfile=[]
for k=1:length(Files)
FileNames=Files(k).name
if Files(cnt).isdir, continue
Arrayfile=[Arrayfile;FileNames]
fid=fopen(filedir(cnt).name, 'rb', 'ieee-be')%file open inside
timestamp = fread(fid,4,'uint32');
dt = fread(fid,1,'float64');
cols=fread(fid,1, 'uint32');%get the column size
rows=fread(fid,1,'uint32');%get the row size
data=fread(fid,[rows,cols],'double')% get the full data r x c
end
end
Accepted Answer
Image Analyst
on 11 Sep 2014
See my attached demo that recurses into all subdirectories and lists files.
18 Comments
Mini Me
on 11 Sep 2014
with my code above I can get it to display the names of files in the directory. It just that the two subdirectories . and .. comes up also. therefor making it hard for me to read. plus I'm having a hardtime with fopen since I get the file names like this: filename=file(k). name %% assuming k is a loop count integer then fid= fopen(filename,'rb') give me invalide fid error
dpb
on 11 Sep 2014
Edited: dpb
on 11 Sep 2014
As told you before, use an appropriate wildcard...don't use '*.*' to return files. Use something w/ the appropriate extension to match the data and then you won't have directories in the file names.
And, as IA shows in his function, build the array of directories to traverse first and separately from that dir call for each subdirectory in turn.
You're making this harder than it need be by not following the hints given...
Or, of course, if you insist on mixing the two, simply test that if the directory is one of the two don't want, skip it...
for i=1:length(d)
if d(i).isdir
if all(d(i).name=='.'),continue,end % skip the two references
d(i).name % do the traverse in the subdirectory here
end
end
Mini Me
on 11 Sep 2014
@ dpb there seem to be a misunderstanding. One file inside the directory does not have all the information. think of it as big audio file that is chopped up in chunks. so Id need to read all the files and combine them all together in the right order to get the final harmony. the same application works for binary files full of data info.
dpb
on 11 Sep 2014
So what's the problem? As noted, put the dir call for the files wanted in the subdirectory where I noted and simply displayed the subdirectory name as a placeholder above. What you do with the files at the point is up to you.
What's the extension of the files in question and how are they identified to know which is the proper order?
Image Analyst
on 11 Sep 2014
How are you going to be sure you stitch all the files together in the correct order?
Mini Me
on 11 Sep 2014
Edited: Mini Me
on 11 Sep 2014
Here's the dir setup:
directory
subdirectory 1
file 1
file 2
file n
subdirectory 2
file 1
file 2
file n
subdirectory n
@ Image Analyst my goal is to read all the files inside directory 1 inside a loop and dumped in an array and append each time. Then have flexibility to plot the column and rows inside that array. Then do the same for the second directory. At the end i want to be able to play( add or subtract) with the final Array data of directory 1 and 2.
dpb
on 11 Sep 2014
That's exactly the structure above...as said, just do a dir on the files in each directory at the point outlined above and do your thing.
However, are these stream files after, perhaps, the first one? If so, instead of reading and appending, I'd use the OS facilities and simply copy/append to one file and then read that one.
Also, again I ask--what is the file extension and how are they named precisely in order to know the sequencing?
Image Analyst
on 11 Sep 2014
Edited: Image Analyst
on 11 Sep 2014
As dpb says, just make minor modifications, like use '*.wav' instead of '*.png' and you should be almost ready to go. I assume you already know how to stitch your audio files together. Not sure why you couldn't do it, so I did it for you. It's extraordinarily well commented so I'm pretty sure you should be able to understand it. Here, use this (also attached below code as a m-file test3.m):
% Start with a folder and get a list of all subfolders.
% Finds and prints names of all wav files in
% that folder and all of its subfolders.
clc; % Clear the command window.
workspace; % Make sure the workspace panel is showing.
format longg;
format compact;
% Define a starting folder.
startingFolder = cd;
% Ask user to confirm or change.
topLevelFolder = uigetdir(startingFolder);
if topLevelFolder == 0
return;
end
% Get list of all subfolders.
allSubFolders = genpath(topLevelFolder);
% Parse into a cell array.
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames)
% Process all wav files in all those folders.
for k = 1 : numberOfFolders
% Get this folder and print it out.
thisFolder = listOfFolderNames{k};
fprintf('Processing folder %s\n', thisFolder);
% Get wav files in this one folder.
filePattern = sprintf('%s/*.wav', thisFolder);
baseFileNames = dir(filePattern);
numberOfFiles = length(baseFileNames);
% Now we have a list of all files in this one folder.
if numberOfFiles >= 1
% Go through all those files in this folder.
for f = 1 : numberOfFiles
fullFileName = fullfile(thisFolder, baseFileNames(f).name);
fprintf(' Processing wav file %s\n', fullFileName);
% Now do your stitching together or whatever you want to do.
end
else
fprintf(' Folder %s has no wav files in it.\n', thisFolder);
end
end
Mini Me
on 13 Sep 2014
@ Image Analyst I have one more question After I do the stitching, my plan is to add all the data inside each folder into an array. Meaning, for the first folder: each files contain data and each time the loop iterate those data are fed into the array and append until the loop ends. Now for the second folder, I want to create a new array to put those data into when the loop iterates the second time reading the files inside the 2nd folder; and the same for all the remaining folders you have any idea to go about that? thank you in advance
Image Analyst
on 13 Sep 2014
in the inner loop over f, read in the file, add the data to some accumulation variable.
thisData = MyReadingFunction(fullFileName); % You write this
if k == 1
allData = thisData;
else
allData = allData + thisData;
end
Mini Me
on 13 Sep 2014
I understand what you did there. I already have my data read and appended into the array for the the 1st folder. the problem is when it's goinig through the 2nd one, the new data are added to the the old data. I want each folder of data to have their own array. i.e each time the k loop iterates, it opens the folder, then the f loop read the 1st file and add the data into the array, read the second file and append it in the array and so on until it's done. then back in the k loop a second time for the 2nd folder, the f loop does the same thing but only this time create a different array to put the data in until it'd done reading all the file in the second folder. and so on
Image Analyst
on 13 Sep 2014
Make all data a cell array or 3D array, such as
thisData = MyReadingFunction(fullFileName); % You write this
if f == 1
% Create a new cell, the k'th one, for each new loop over f
allData{k} = thisData;
else
allData{k} = allData{k} + thisData;
end
Mini Me
on 13 Sep 2014
Thanks alot @ Image Analyst. That does the job for me to have control on the data set for each folder. but unfortunately, at each iteration it doesnt append the new data. It replaces it because I take the szie of allData{k} and it remains the same throughout all iteration. it doesnt increase.
Image Analyst
on 13 Sep 2014
Edited: Image Analyst
on 13 Sep 2014
I don't understand what you said. At each iteration of f, the data gets added to the existing data. So when all iterations over f are done, you have the sum of all of them and that sum is in allData{k}. allData has a new cell appended at each iteration of k so the size of allData does not stay the same, it grows by one cell for each folder until in the end it has the same number of cells as there are folders.
What is your definition of "add"? Do you mean the "plus" operation as usual , or do perhaps you mean the less common "append" or "stitch" where the array on each iteration grows larger because you're stitching/appending on new elements? If you do mean append, you need to know which dimension you want to append new data onto.
Mini Me
on 13 Sep 2014
no by add i mean append...meaning if the first file has 10 samples making the cell size 10 x 1 ; when the next file is read and added into the cell, the additional 10 samples will be added in that cell making the cell size to be 20 x 1. the 1st sample of the second file will be the 11 sample in that cell
Image Analyst
on 13 Sep 2014
use either comma or semicolon depending if you want to append horizontally or vertically.
allData{k} = [allData{k}, thisFData]; % Stitch side by side
allData{k} = [allData{k}; thisFData]; % Stitch vertically
Mini Me
on 16 Sep 2014
Thanks @ Image Analyst so I decide to take the mean of allData{k}. Now the mean will be one value for each row. meaning a 1 x 8 matrix. I want to take each column of that matrix and feed inside a new array each time the loop read a folder of data files (k loop). meaning I want the average of all the files in folder one. take the 1st column and feed in an array, take the second and feed it in an other array. and so on. move on to the 2nd folder, it takes the 1st column and feed in the same array that contain the first column of the previous folder. then the same for the second folder and so on. If can help me it'd be much appreciated as I am stuck on this for a while now
More Answers (0)
See Also
Categories
Find more on Convert Image Type in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!An Error Occurred
Unable to complete the action because of changes made to the page. Reload the page to see its updated state.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)