Time difference taken between two methods: changing folder manually and adding path to Matlab search directory

I recently observed this effect. I have the data to be loaded in another directory. So I usually use "addpath" command. But for a change I tried to do it manually and I observed a huge time difference.

Example:

Addpath method ran x4 times

    tic
    addpath ('C:\MATLAB\2D\data')
    load x.dat;load y.dat;load z.dat;
    toc
Elapsed time is 0.156776 seconds.
Elapsed time is 0.147188 seconds.
Elapsed time is 0.158762 seconds.
Elapsed time is 0.172729 seconds.

Manually changing directory method ran x4 times

    tic
    current_folder=pwd;
    cd ('C:\MATLAB\2D\data')
    load x.dat;load y.dat;load z.dat;
    cd(current_folder);
    toc
Elapsed time is 0.074755 seconds.
Elapsed time is 0.076723 seconds.
Elapsed time is 0.074736 seconds.
Elapsed time is 0.078758 seconds.

Definitely manually changing folder method is twice as fast as addpath method.. Can anyone try to explain why this is happening?

[Adam]

Why add to your path or change directory at all? Just use the full path to your data files when you load them and you don't need the overhead of doing either.

Adding to your path is not likely to be especially fast, especially if your path is long. The path is for code files though.

[Addy]

Yeah but I just like to have the code look simple. As you said, my folder path is like very long that matlab has to go across 11 folders to fetch the file. So it will code will look like an essay if i have to fetch many files :p

[Stephen23]

Both of those are the wrong approach: the Search Path is about where MATLAB looks for code, and it should not be used to include every directory that might contain some data. I have never seen any well-written code that would require doing those actions repeatedly: any additions to the search path would typically be performed once by a large toolbox, or manually by the user (who should know where their code is saved). However neither of those actions should occur for reading data because using absolute/relative file paths is more efficient and easier to debug (changing directories is much slower than simply using the full file path when reading data).

"Yeah but I just like to have the code look simple"

I don't see much difference in "simplicity":

addpath('C:\MATLAB\2D\data')
S = load('somefile.mat');

Versus much more efficient full path:

D = 'C:\MATLAB\2D\data';
S = load(fullfile(D,'somefile.mat'));

Of course you should always use function syntax and always return an output argument from load!

[Rik]

I fully agree with Adam an Stephen.

Adding to that: it doesn't surprise me that cd takes much less time than addpath, because the latter is more complicated in how it works for your task. load will look first in your current folder, and then use the path to look in different folders. So it could just be the case that it's not the addpath versus cd that is causing the time difference, but just the time load spends looking for that file.

[Adam]

Even if the code is longer to do it the right way that is what functions were invented for (well, one of many reasons!) - in your main code you can just have a one line function call to load your data which is very simple and neat. What that function does is hidden away in the function and needn't be seen if it does what it should.

[Addy]

I haven't come across "fullfile" function before. Thanks for letting me know. And I modified the code as

tic
D = 'C:\MATLAB\2D\data';
cellfun(@load,fullfile(D,{'x.dat';'y.dat';'z.dat'}));
toc
Elapsed time is 0.065269 seconds.
Elapsed time is 0.065727 seconds.
Elapsed time is 0.066649 seconds.
Elapsed time is 0.063870 seconds.

So, fullfile gave me the fastest time.

Regarding the output argument.. I avoid using it because, when I use an output argument and when I load "dat" file it is okay. But when I load a "mat" file with several variables, it loads as a struct. But for now It is easy for me as normal variables than using structures for this application.

[Stephen23]

"But when I load a "mat" file with several variables, it loads as a struct."

Loading into a structure is actually better: it avoids overwriting existing variables without warning, and avoids some other subtle bugs. Read more here:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236127

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_274740

[Addy]

Thank you Stephen.

[Saunok Chakrabarty]

Hello, I'm sort of new to Matlab. Could you tell me when I should change directories (using cd), when I should use addpath, fullfile, etc? What's the utility of each?

"The Search Path is about where MATLAB looks for code, and it should not be used to include every directory that might contain some data. I have never seen any well-written code that would require doing those actions repeatedly: any additions to the search path would typically be performed once by a large toolbox, or manually by the user (who should know where their code is saved)." -- could you explain what this means? Should I store my data and functions/scripts in separate directories?

[Stephen23]

@Saunok Chakrabarty: most important to understand: the Search Path is just a list of folders where MATLAB code is saved:

https://www.mathworks.com/help/matlab/matlab_env/what-is-the-matlab-search-path.html

Note: where code is saved. Not data.

When to use:

• CD: at the command line, when you want to change what directory you are currently working in.
• ADDPATH: when you have folders of MATLAB code (e.g. apps or downloaded toolboxes) that you want to use.
• FULLFILE: whenever you write code that needs to access (i.e. read/write) data files.

PersonalIy I rarely use CD/ADDPATH, not because I don't change the current directory or the Search Path, but simply because the MATLAB IDE has buttons or drop-down menus or right-click context options that perform both of these (and much more). So I prefer using the IDE, rather than command line commands. But you can use whatever suits you best.

"Should I store my data and functions/scripts in separate directories?"

Yes.

In general keep your code folders neat and focussed, i.e. with only code files and anything closely related.

Data files can be saved anywhere appropriate, and then you just need to use FULLFILE to access them. This is more efficient and makes debugging easier.

[Walter Roberson]

Unfortunately, MATLAB does not have a data search path.

I would point out that it is not productive for users to have to know where Mathworks has moved camerman.tif or rhinos.avi this release: it is useful for them to be able to just read from the appropriate file without having to search for it.