dir-command: import only files that have datenum between two dates

Hi everyone,

I am using the dir-command to get file information of many files in a directory (>800 000 files).

However, I only need the files with datenum between 01.01.2018 and 01.01.2019, which are significantly less files.

I was trying to work with cellfun, as shown in the MATLAB documentation of the dir command (https://de.mathworks.com/help/matlab/ref/dir.html):

MyFolderInfo = dir; 
MyFolderInfo = MyFolderInfo(~cellfun('isempty', {MyFolderInfo.date})); 

This is my current status:

date_2018 = datenum('2018/01/01 00:00:00','yyyy/mm/dd HH:MM:SS');
date_2019 = datenum('2019/01/01 00:00:00','yyyy/mm/dd HH:MM:SS');
listdir = dir(some_path);
listdir = listdir(cellfun(@(s) {s.datenum}<date_2019 && {s.datenum}>date_2018));

This gives the error:

    Error using cellfun
    Not enough input arguments.

So, obviously I'm not good with cellfun in general, but this is an example where I wanted to learn a bit about this tool.

Can anyone help me with this task?

Firstly, it's about getting rid of the not needed entries in the struct-variable. If this can be done with cellfun, that would be nice.

Thanks a lot in advance!

JE

Answer 1

[Steven Lord]

You don't need cellfun or serial date numbers. I'd use datetime instead. For instance, here's how to list the .m files in the directory whose dates are on or after January 1st, 2019. First, get the list of all the files.

D = dir('*.m');

Next make a new vector containing the dates converted to datetime. You could do this using a for loop, or you could take advantage of the fact that dir returns an array of structs and use arrayfun to iterate through the elements of that array.

thedates = arrayfun(@(x) datetime(x.date), D);

Use logical indexing into the struct returned by dir.

createdIn2019 = D(thedates >= datetime('01-Jan-2019'))

To include multiple conditions use &.

createdIn2018 = D(thedates >= datetime('01-Jan-2018') & thedates < datetime('01-Jan-2019'))

There are things you can do avoid having to hard-code the year in those datetime arrays used in the conditions, like using the year function on thedates.

createdIn2019_take2 = D(year(thedates) == 2019)
isequal(createdIn2019, createdIn2019_take2)

Or you could use dateshift to identify files created since the beginning of this year. [In this case, hard-coding the year number in the variable name may not be the best idea, since if you run this code next year it will select files from 2020, but for purposes of this example I think it's okay.]

startOfCurrentYear = dateshift(datetime('today'), 'start', 'year')
createdIn2019_take3 = D(thedates >= startOfCurrentYear)
isequal(createdIn2019, createdIn2019_take3)

[Jonathan Eberle]

Thanks a lot Steven, that helped me to get it done in a nice and easy way!

Answer 2

[Bob Thompson]

I don't know that cellfun is going to work on the dir results because they are an array of structures. You should be able to solve this using indexing alone.

Something like:

listdir = [listdir.date >= date2018 & listdir.date<= date2019];