Executing .m files function in R2015b is not working properly

Hi everyone!

I have a main funtion which calls ExecuteMFiles() at certain point. I also have an inputdocPath to a folder with many different files ( .dat, .m, etc.), these files have an important information in their name so it can't be changed but unfortunatelly they are written starting with numers and dots (eg "32.521_parameter.m") which I know it's one of the very first rules to not follow so what I've done is make a copy, execute it and delete it.

That's done for each .m file so I can run it without an error in MATLAB R2016b but when I use R2015b version, someting wrong happends. If I run my main code in R2015b I get an error in relation with non-seen parameters which should been defined while running ExecuteMFiles() but if I use debugging from copyfile() line and then I go step by step, all the .m file run well so I'm confused!

Could anyone help me with this? I let you here the ExecuteMFiles()'s first part (the second part is just the assigment of names for each parameter in the workspace) which I guess it's failing for something.

function ctl = ExecuteMFiles( inputdocPath )

cd (inputdocPath);
found = dir(fullfile(inputdocPath, '*.m') );

for iF = 1:length(found) %run all *.m file with parameters
    copyfile(found(iF).name,'copy.m');
    eval('copy');
    delete('copy.m');
end

Thank you very much!

[Stephen23]

You forgot to tell us some of the most important information, whether those M-files are scripts or functions. In either case you should avoid the counter-productive eval and the slow-and-pointless copying-and-deleting of files:

• scripts -> call run.
• functions -> use str2func to generate a function handle and call that.

You should avoid cd as much as possible.

[Lluis Ballber Mayans]

Oh right! They are scripts but the way they are named make them impossible to call using run...

Try to run a file which name is "32.521_parameter.m" and you will see my first problem which I solved it with the bad-slow-and-pointless copying-and-deleting of files.

Note: I forgot to tell you that the parameters are from simulink, I don't know if it can be usefull!

[Rik]

Why did someone make the mistake of putting data in the filename, and why are you not allowed to move that information to a better place? Jumping through hoops has already bitten you in the butt, you should expect that to happen again.

Also, if you use run instead of eval in your code, what happens?

[Guillaume]

"someting wrong happends"

More details needed. If you get an error message, give us the full text of the error message. Otherwise, tell us what you expected to happen and what happened instead.

Considering that names starting with numbers has never been valid script, function or variable names in matlab, why was this ever used? At the very least, going forward you could ensure that the numbers are after the text part.

As has been said, there's never any reason to cd into anything. Just use full paths. Once you've renamed your files, you also don't need eval. run will work just as well.

[Lluis Ballber Mayans]

@Rik

Maybe because it's a critical information for some reason and they way it's exported is not the best for matlab but it's used for many things. I tried using run and it stills doesnt work as expected... Thanks anyway!

@Guillaume

I don't get any error just some .m doesn't run or the parameters didn't appear in the workspace in the R2015b version but in R2016b it runs perfectly! I really don't get the point of your criticism if you don't try to help me, the input files are what they are, as I said, I can't change them, why do you complain then? Btw I've just removed cd and used run, nothing has change...

[Rik]

Is there any pattern in which file is not run?

Also, you should really consider convincing the person who made this mess to make them valid script names by starting with a letter and replacing decimals with an underscore.

[Joel Handy]

Sounds like maybe one of two things may be happening. One, there is an issue with one of the scripts being run. Two, you have a race condition where matlab is trying to execute commands faster than your file system can respond.

Copyfile outputs a success flag. I recomend checking this flag to verify all of your copy commands are executing successfully. You could also try putting "pause(1)" between your copyfile, run, and delete calls and see if that clears up the issue.

If it is a race condition, you might consider creating a subdirectory, and copying all the files there with a fixed name, running all of the scripts, then deleting the directory. That would alleviate the need for any arbitrary pauses.

[Stephen23]

Race conditions, system response time, pause, copyfile, eval, delete, etc. etc.

Keep in mind that all of these are just fragile-behaviors/slow-hacks due to badly named files.

In case it is not clear enough by now: using standard valid MATLAB names would mean that you would not need any hacks at all.

[Guillaume]

@Lluis,

We are trying to help. We are also trying to understand fully your situation. At the moment, we don't have enough information to solve your problem. I'm also not criticising, I'm trying to understand why you're trying to run m files that are not and have never been valid matlab files. It's also not clear why you can't just permanently rename all the files (inside or outside matlab) to make them valid matlab files. We're also interested in helping people develop good code, so expect to see suggestions to not create the mess in the first place. "I'm creating matlab files that are not compatible with matlab" is a strange thing to do.

It probably would help if you attached an example file (one that does not appear to be executed).

One thing to consider with your current code is that you rename each file to the same name copy.m. What if instead you generate a unique name for each file using e.g tempname? If that solve the problem, then what you're seeing may be due to caching (OS or matlab).

[Lluis Ballber Mayans]

@ Guillaume

First of all, sorry for what I said to you before, I felt attacked and I was triggered with this, I have been fighting it for 2 long days and I continue stacked, I hope you understand it.

@Dear all

As an intern I could not change anything but for sure that I told them that the use of numers at the beggining and dots in a name's file was a big big mistake, but their old program export them like that so I have to deal with it...

One of my first trials was using pause because I guessed the same but I tried with 4 seconds (which is crazy) and still didnt worked as it should. I have designed a silly example which I hope it clarifies what's my problem (I should do it before, I'm sorry...).

Main

% Main code

indocPath='...\inputDoc';

y=ExecuteMfiles(indocPath);

length(struct2cell(y))==4

Note: Don't forget to put your own path!

ExecuteMFiles

function y = ExecuteMfiles( inputdocPath )

found = dir(fullfile(inputdocPath, '*.m') );

for iF = 1:length(found) %run all *.m file with parameters
    copyfile(fullfile(inputdocPath,found(iF).name),'copy.m');
    run('copy');
    delete('copy.m');
end

% collect variables starting with P_ and V_
prmfound=[whos('^P_*','-regexp'); whos('^V_*','-regexp')];
for ip = 1:length(prmfound)
    y.(prmfound(ip).name) = eval([prmfound(ip).name '.Value']);
end

23.156_fparameter

P_1 = Simulink.Parameter;
P_1.Description = '  Parameter 1';
P_1.Value = 0.01;
P_1.DocUnits = 's';

P_2 = Simulink.Parameter;
P_2.Description = 'Parameter 2 ';
P_2.Value = 4;
P_2.DocUnits = 's';

23.156_sparameter

P_3 = Simulink.Parameter;
P_3.Description = ' Parameter 3';
P_3.Value = 80;
P_3.DocUnits = 'rpm';

P_4 = Simulink.Parameter;
P_4.Description = ' Parameter 4';
P_4.Value = 955;
P_4.DocUnits = 'rpm';

I also forward you the folder with the structure I have to follow so you just download it and run it.

As you can see, we have 4 params but if you run the main code just get 2 (the first ones) but now put your debugging mark on the line which says " copyfile(fullfile(inputdocPath,found(iF).name),'copy.m') " then run it, go step by step until we go out of the first loop and then click continue. We have 4 params now, the code is ok! :O

Thanks again for your time.

[Guillaume]

I'm not sure many here have 2015b installed anymore so it's going to be difficult to diagnose the issue. Other than the fact that the whole thing completely violates all programming best practices I can't see much wrong with it, with the exception of the regular expression in whos. Maybe the behaviour of whos changed between 2015b and 2016a.

Note that your regular expression matches any variable starting with P (or V) followed by 0 or more _. If you meant to match P (or V) followed by one _ and any character the regexp is ^P_.*. You could combine both whos into one with:

prmfound = whos('^[PV]_.*', '-regexp');  %you actually don't need the .*, so '^[PV]_' would work as well

However, if all the scripts you want to run follow the same format (variable creation followed by field assignments) I would recommend a completely different approach to importing them. I would actually parse the files which would avoid having to rename them, avoid eval, and avoid the risk of a file accidentally stomping on existing variables.

Answer 1

[Stephen23]

Ugh, invalid filenames and anti-pattern dynamic variable names... it seems that your task was explicitly designed to torture interns with! It is a lesson in how NOT to design code and data. Most likely you should actually parse the files to get the data, rather than try to run them, and so avoid this entire ugly, fragile mess.

As for your original question, try adding

clear('copy')

before run (experiment to find the best location), to clear the script from memory:

https://www.mathworks.com/matlabcentral/answers/29096-stale-files-used-instead-of-updated-files#answer_164463

I also recommend using a name that is less likely to be used elsewhere, e.g. mycopy.

[Rik]

Or even better than mycopy: the tempname function (which was new to me, thanks Guillaume).

[Lluis Ballber Mayans]

Oh great! It worked, thank you very much!

I will try to use all the tricks you all said to me but for the original question Stephen's answer worked so I have accepted his answer.

As a new member here, should I modify something about my post? In order to be more clear I mean, or add more information about what you told me?

And if one day I'm someone in the company I will make them change that file's name for sure hahaha

[Stephen23]

"should I modify something about my post?"

No, you can leave the question just as it is. It is standard practice on this forum to use comments to add information and clarifications, which you have already done as part of our discussions about your topic.

Go and enjoy your intern-torture!

[Guillaume]

I will try to use all the tricks you all said to me

I would strongly recommend writing your own parser for the files instead of executing them. If the files are as simple as the one you've shown it could be a very simple parser. As I said, that would completely remove the need for renaming the files and would also remove the risk of stomping over existing variables. As it is any file that has any of:

y = ...
ip = ...   %an ip address for example!
prmfound = ...

will break your code when it's eval'ed