Can I force Matlab to wait 'til it finishes one command to run another?

I have a script that loads a SimBiology model using the simbiology() command. Then I try to get a handle for the model

simbiology('mymodel.sbproj');    % There are reasons why I use simbiology and not sbioloadproject
sbr = sbioroot;
mdls = sbr.models;
nmdls = numel(mdls);
mc = mdls(nmdls);

The problem is when I first try to get mdls. I get an empty array. Apparently, the model is still loading when the sbr=sbioroot and mdls = sbr.models commands run.

In windows command line files, you can use

START /WAIT myprogram.exe

to force myprogram.exe to finish running before going on the next line. Is there a similar artifice in Matab/Simbiology? Something like

finish('simbiology('mymodel.sbproj')'  ;

% or 

whilerunning
    simbiology('mymodel.sbproj');
end

I keep thinking that there's an easy way that I've forgotten...

The above is a general case. Of course, if simbiology() were re-written to return the model handle, and when the handle was requested, would not allow the script to proceed until the model was loaded, it would be great. Logical, too, because generally when you're loading a model you want to do something with that model before proceeding.

mc = simbiology('mymodel.sbproj');   % This solves the specific problem!

% or 

mc = simbiology('mymodel.sbproj','WaitToComplete');   % or this

[Guillaume]

With regards to your question: Can I force Matlab to wait 'til it finishes one command to run another?

This is what matlab and most other languages, do already. A statement will not be executed until the previous one has completed. e.g. the next line of code will not be executed until simbiology returns.

Now your problem is that simbiology itself launches another task in the background and returns more or less immediately. Unfortunately, I don't know anything about simbiology so can't help with that. Typically something that launches a task in the background will have a callback that can be executed when the task is complete.

So the problem, and thus the solution, is not with matlab but with simbiology.

[Walter Roberson]

sbioloadproject()

[Jim Bosley]

Guillaume,

Thanks - nicely explained.

Its interesting. I tried to use a pause command.

simbiology('mymodel.sbproj');
pause(20);
sbr = sbioroot; %.......

That should allow the background process to work but if I recall correctly, it didn't work. I'll check your proposal, as well rechecking the pause command.

Thanks,

Jim

[Walter Roberson]

simbiology() itself is the command to start up the simbiology interface and return control to MATLAB. It makes no promises that any project will have finished loading before it returns to MATLAB. For use in a function or script you would sbioloadproject() and sbiosimulate() and related commands.

[Jim Bosley]

Walter,

I understand your point. I use Simbiology to open the model for a reason. Simbiology opens ALL model elements including datasets and graphical location information, and it opens the dashboard as well.

I often use a "double breasted" approach to model dev. Open the model with simbiology, run a script, possibly modify the model (in dashboard), write new scripts, etc. So the dashboard and liveplots are very useful.

I have large models that are graphically well-developed (at least, I put a heck of a lot of effort into making them graphically attractive, understandable, and logically laid out). If I open the model with sbioloadproject, none of the positional notation is opened. Then, if I save the project with sbiosaveproject tasks and datasets are lost, and the resulting sbproj file has no positional notation. That is, my carefully crafted graphical representation is reduced to a bowl of spaghetti.

The cycle

sbioloadproject('myproj.sbproj')
sbiosaveproject('myproj.sbprog')

loses model information.

So sbioloadproject doesn't work for me and I must use Simbiology() as stated in my OP.

Thanks,

Jim

[Walter Roberson]

simbiology('mymodel.sbproj');
sbioloadproject('mymodel.sbproj');

The simbiology() command might return before the interface is completely open, but the sbioloadproject() after it should not return until the project is completely loaded.

[Jim Bosley]

Walter,

I guess that's the point of the question. sbioloadproject is inadequate, as it doesn't retain positional notation (or tasks or data).

As a consultant for the past 20 years, I've found that the quicket, surest way to have no influence in pharma, and be shown the door, is to show biologists and physicians Matlab code. To a systems biology superman, a biologist exposed to code is kryptonite. If, on the other hand, you show client experts a diagram that makes physiological sense (and your model diagram SHOULD make physiological sense, or you've done it wrong!), then there's a possibility that they're with you.

Running sbioloadproject and then modifying the model (or even if you do nothing) and then running sbiosaveproject actually LOSES information.

There is no downside (save the problem that is the issue of this thread) to using SimBiology() to open the project. And there are many upsides.

And its easy to run scripts using the simbiology loaded model (see code above).

For this reason, I'd REALLY like it (hint, hint, guys) if sbioloadproject would load positional stuff (and tasks) and save positional stuff (and tasks). Even if the user doesn't have access to those elements. Then a sbioloadproject/sbiosaveproject cycle doesn't lose data. And you could open a model in SimBiology desktop, and modify things, and then save it with sbiosaveproject. But these commands don't work this way, so for now its simbiology().

Thx,

Jim

[Walter Roberson]

If you

simbiology('mymodel.sbproj');

then that would start to load the position information, but might not completely finish before it returns. Would the

sbioloadproject('mymodel.sbproj');

discard the position information ?

If so, then you will probably need to open a support request.

[Jim Bosley]

Walter,

I think that Pax and the team know about this - we've had several email discussions. On the other hand, they're putting a lot of cool stuff into new versions - this may not be high priority.

But there is a disconnect between the dashboard, and using the model loaded in memory by sbioloadproject. My workflow is

1) Create model structure graphically. Discuss with biologists and MDs (esp. client biologists and MDs). Requires good graphical model.

2) Develop. Open model via simbiology(). Run and fit using tasks (initially) and scripts.

One key element is the ability to plot state_subplots. This is a trellis plot of all states in a matrix. If you set sampling to once per day (or hour, or whatever, depending upon your model) this gives you an amazingly sensitive test for stability. Great for when your model has complexities or periodicities that preclude a steady-state solution.

Another key element in this workflow is live plots, allowing for plotting data from datasets. I can mod a parameter, kick off a liveplot, and what as the plot develops. If its a long simulation (simulating a 100 state model for a year), I can see trends develop early and can terminate the simulation and modify the model. With scripts I have to wait til the sim ends and the plotting script runs to see this.

Because I open the model with simbiology() when I save the positional stuff, and all datasets and tasks are stored as well.

3) Develop scripts for cases and virtual persons/patients (VPs). Inludes validation plots showing whole-body and subsystem agreement with data. Also key is to show that dynamics (time frame of up- and down-regulation) agree with data. Automate VP creation and vetting if you'd like (see Youngan Cheng et als. et al. QSP Toolbox Paper in the AAPS journal).

4) Develop scripts for use-cases (simulations that answer client questions or that predict stuff the client wants to know).

I'd love to be able to write scripts that exploit the dashboard live plots, or that could use (or load) datasets to the model. Or that could create simulation tasks in the model. Right now, can't do that with scripting. Scripting this would allow a validatable way to create sim tasks and data in the sbproj structure for use in the dashboard. But right now, I don't think you can do this.

[Jim Bosley]

Ok. Guillame, as much as I liked your proposed solution, it didn't work. The first instance of mdls = sbr.Models does not return an empy object [], it returns a 0x1 Model object structure. So the while/end loop terminates before the model has loaded.

I suspect that I may have an insoluble problem here. Because I think that while the script is running, the Simbiology desktop stops running. At least the simbiology('myproj.sbproj') comman doesn't complete. So if I run this the code below

pstate = pause('on');       % Capture pause state and turn pause on
sbr = sbioroot;             % Get sbioroot handle
simbiology('mymodel.sbproj')     % Tell SimBiology to open file
mdls = sbr.Models;          % Get original model state (numel should == 0)
i = 0;                      % Safety counter
while numel(mdls)==0
    pause(1);               % Don't chew up CPU with a tight, do-nothing loop
    sbr=sbioroot;           % Doesn't work with this line, or without this line.
    mdls = sbr.Models;      % To force mdls to be current
    i = i + 1;    
    nmdls=numel(mdls);      % Get the number of models    
    disp(['Time is ' num2str(i) '   Number is: ' num2str(nmdls)]);
    if i>120
        break;
    end
end
mdls = sbr.Models;      
pause(pstate);          % back to original state
clear pstate;

I get a output with the value of i increasing to 120, and the value of nmlds constant at 0. If I then see what it returned in mdls by typing mdls, I get:

mdls = 
  0×1 Model array with properties:

And the model doesn't complete loading til the script ends. So after the script ends if I wait a few seconds and let the model load, I can type mdls=sbr.models and get:

mdls=sbr.Models 

   SimBiology Model - REGN TG 
   Model Components:
     Compartments:      7
     Events:            0
     Parameters:        300
     Reactions:         84
     Rules:             143
     Species:           46

So its pretty clear that the model doesn't load until the script finishes.

I see no sensible reason for not letting the model load while the script runs. This seems like a bug to me.

Answer 1

[Arthur Goldsipe]

Hi Jim,

The previous responses are correct: The simbiology command returns before the desktop completely loads the project, because the desktop is operating on different threads. So, welcome to the world of multi-threaded / parallel / asynchronous / <insert buzzword> complexity! Believe me, it makes life harder for SimBiology's developers, too. But hopefully you're mostly seeing the benefits of us living in that complexity and hiding it from you as much as possible.

Right now, there's one-line command that does what you want, but I'll suggest a workaround below. But thinking longer term, we hope to eliminate the need to open a project in the desktop just to preserve the diagram. If you can think of other reasons why it's important to know when a project is fully loaded, let us know so we can address those workflows, too.

Anyway, if all you care about is whether the model is accessible from the MATLAB command prompt (and not whether the desktop has completely finished loading the project), I came up with a solution (pasted below) that uses timer objects to periodically check whether the models have been loaded.

Here's the function. It takes the name of a project, the name of a variable to assign models to in the base workspace, and an optional callback that you can use to execute tasks that take the models as input arguments. So here's an example usage that simulates the Lotka demo project as soon as it's loaded (assigning the result to the base workspace variable sd): simbiologyNotify('lotka.sbproj', 'models', @(m) assignin('base', 'sd', sbiosimulate(m)))

function simbiologyNotify(projectName, varName, callback)
if ~exist('callback', 'var')
    callback = @(models) [];
end
root = sbioroot;
oldModels = root.Models;
t = timer('ExecutionMode', 'fixedSpacing');
t.TimerFcn = @(t,e) timerFcn(t, root, oldModels, varName, callback);
simbiology(projectName);
start(t);
end

function timerFcn(t, root, oldModels, varName, callback)
models = setdiff(root.Models, oldModels);
if isempty(models)
    return
end
beep
fprintf('Models have been loaded. Assigning to base workspace variable "%s".\n', varName);
assignin('base', varName, models);
stop(t);
delete(t);
callback(models);
end

Note that you can't directly call this function from a script or function. If you do, you'll block the MATLAB interpreter, and that will prevent the desktop from loading the project. So you have to start thinking about asynchronous tasks the way we developers do: as a series of callback functions that get called as soon as their preconditions are met.

And one last note: If you really need to know whether the desktop has fully loaded the project into the GUI, I'll see if there's a way to get that information.

-Arthur

[Jim Bosley]

Arthur,

Appreciate your note. I'm trying to do something really basic, for which there's an obvious workaround. So my bone-headed persistence is pretty idiotic. But I do learn some cool stuff this way. I'll have to study your answer for a bit. Glad that its doable.

My work flow is a combination scripts and dashboard. I need the graphical representation for my clients and colleagues (and for me, too) but scripting has all of the great things needed too, like reproducibility, ability to publish successful scripts and such. And I like live plots as mentioned. So with the model open in the dashboard, and the ability to change stuff on the fly and run scripts that use the current model, I'm happy.

In scripts that run or fit or optimize models I'd like them to open up and check if there's a model in memory. If so, use the most recent model. If not, load a model:

So I can use this:

sbr = sbioroot;

if numel(sbr.models)==0
    simbiology('mymodel.sbproj');
    wait for model to load...
end
mc = sbr.models(end);

For now, the workaround is just to manually load the model.

From the stuff I've been writing, using

simbiology('mymodel.sbproj');
other_stuff();

launches the simbiology stuff, and releases the thread to run other_stuff, allocating all resources to Other_Stuff until Other_stuff completes. So at no time in the script will the model be loaded using this approach: the model loads when the script ends.

Jim

[Arthur Goldsipe]

Hi Jim,

In case it helps you understand my example better, here's how you could adapt your "other_stuff" sample code to work with my function:

simbiologyNotify('mymodel.sbproj', 'model', @(model) other_stuff);

So the basic idea is that you put everything that needs to run after the model loads into another function, and that function is only called after the model loads. That avoids the problem you describe (which also occurs if you try to use "pause" to wait).

[Jim Bosley]

Arthur,

Great stuff!

I recall having to something similar on a DEC VAXStation when in grad school. I was optimizing a batch distillation process (my PhD involved running a moonshine still) and had a commercial distillation simulator. I wrote an optimization routie using sequential quadratic programming. The metric function created an input file for the batch simulator, started the simulation, and waited for "semaphore" (a flag variable) to indicate simulation was complete. Then the program read the simulator output file and computed a metric which it passed back up to the simulator. Worked pretty well, IIRC.

I also recall having my DEC tape backup (TDK50) jam in mid-restore. The tape had my entire suite of research on it and I was restoring because other tapes had failed. I had to disassemble the tape unit, disassemble the tape cartridge, free up the jam and manually rewind the tape. It was terrifying, but ultimately successful. Good times. Sheesh.

Thx again, Jim

[Walter Roberson]

When I sysadmin'd, I was a believer in The Rule Of Three for backups.

The first tape will be scanned by the automatic tape library, noticed that it has "expired", and promptly have its label overwritten because the backups have been sitting waiting for a writable tape to continue writing with.

You then cancel the backups and reconfirm that nothing is waiting to write on the tape drives. Put in the second backup tape, type in appropriate commands... and swear because you accidentally used a tape backup command instead of a tape restore command, so you just scribbled on the tape you were about to restore from.

You unmount that, and this time before putting in the third backup, you make #$@ sure that you enable the write-protect, and proceed to try to restore. Not everything will restore properly. You will start to whimper.

At this point you dig around and dig around, and after panicing a while, find that a three-year old backup got made and overlooked, and you retrieve the rest of the backup from that 4th tape.

Why call it the Rule Of Three? Because next time you remember the write-protect feature on the second tape, and so manage to get through it all in only three tapes.

[Jim Bosley]

NOW he tells me... :)

Seriously, this is good advice, Walter. And a story I can relate to.

These days I do an ISO image of my systems disk periodically, I have a service (Crashplan) that runs in background and stores new versions of files pretty much in real time, and I have a NAS to back up to manually. The NAS runs in UNIX so hopefully a Windows Virus won't cripple it.

I actually had an excellent example to look up to in Grad School. John Eaton (of Octave fame!) dragged our whole department into a more modern software age by setting up all our VAXstations in Unix. He then had a bunch of SW libraries set up, and ran the system very professionally and effectively. A very capable man. I recall John had a great system of backups, varying the level of the backup, so that there was an optimal mix of efficiency and security. And no one was ever in danger of losing more than a day's work.

My VAXStation ran some real-time code that required I be running VMS. So I had to do all the backups and clearly I'm not as good a sysadmin as John was.

[Jim Bosley]

Arthur,

I think I'm just going to open the model with simbiology() manually anyway, but I did want to understand your code. I went through it and prettyprinted it and commented it to ensure I understood it. But I don't. Three questions:

1) Initially, if the callback function handle doesn't exist, you create it as a null function (at least what I'm calling it). So

callback = @(models) [];

defines a function that is passed a handle pointing to any new function, and that returns a null ( [ ] ) . Can you confirm my understanding?

2) The assignin statement is documented for Simulink but not for simbiology. If I understand it corrrectly, we use the assignin command for our model to ensure that results for the model are written to the base workspace. Yes?

3) Not sure what the callback(models) command (last line in timerFcn function) does. All it should do is return a null ([ ]), correct?

Thanks,

Jim

function simbiologyNotify(projectName, varName, callback)
%
% simbiologyNotify opens a project using simbiology()
% (From A. Goldsipe, Mathworks)
%
    if ~exist('callback', 'var')	  % If the variable callback doesn't exists then ...
		callback = @(models) [];  % --> set callback to a function handle returning null ([])
    end
    root = sbioroot;		% get the sbioroot object before opening the projectName
    oldModels = root.Models;	% oldModels contains a list of preexisting models
    t = timer('ExecutionMode', 'fixedSpacing');   % Creates a timer that runs and restarts a
     					          % after a fixed time space
    t.TimerFcn = @(t,e) timerFcn(t, root, oldModels, varName, callback);  % defines a handle to the
						  	    % function that timer runs periodically
    simbiology(projectName);	% This calls simbiology to open the project and returns
			        % note that it returns BEFORE the project is loaded
    start(t);			% this starts the timer, t, which will run the timerFcn every 
				% time period. By default, this period is 1 second.
end

function timerFcn(t, root, oldModels, varName, callback)
% timerFcn is the function called periodically by the timer above, to check if the new model is loaded
	models = setdiff(root.Models, oldModels);	% Compares current model list with original
	if isempty(models)	% If the current model list is the same as before, then ...
		return		% --> do nothing and return
	end
	beep			% Code below only executes when the root object has the new model in it
	fprintf('Models have been loaded. Assigning to base workspace variable "%s".\n', varName);
	assignin('base', varName, models);	% Sets varName to the Simbiology.Model object models
	stop(t);		% If we are here, the model is in sbioroot and we can clean up
	delete(t);		% Get rid of the timer t
	callback(models);	% Not sure about this...
end

[Arthur Goldsipe]

Hi Jim,

Regarding #1: I made the "callback" input optional. If you don't specify one, then I create a "dummy" one that does nothing. If all you want is to assign the model to a variable in the base workspace, then you don't need a callback. But if you want to do something else (programmatically) as soon after the model is loaded (like simulate it), then you need to specify a callback function to do whatever task you want.

Regarding #2: Here is the reference page for assignin. It's a core MATLAB function, not a SimBiology function. And as you are understanding it correctly. We are using it to create a variable in the base workspace.

Regarding #3: You're right. This is related to #1. If you don't specify a callback function of your own, then this one is used, and it doesn't do anything useful. I could have written this in a much less confusing way, but I had my "developer" hat on and wasn't thinking about writing it in a way that's easiest for users to understand.

[Jim Bosley]

Arthur,

I get the logic now, if you specify callback. You specify a model with projectname, and what you want to do with the model with callback.

But why can't the pointer to the model ("models") be an output of TimerFcn, and of SimBiologyNotify? Then when you pass the models handle back to the calling routine, couldn't you just say

If simbiologyNotify returned a model handle mc (or even if it didn't, and you used sbr=sbioroot;mc=sbr.Models(end)) wouldn't you be able to run the model?

[Arthur Goldsipe]

The reason that SimBiologyNotify doesn't just return the model as an output is because the model may not be loaded yet when the function exits. I don't know how to write the function in a way that waits for the the model to load before it exits. All the things I can think of to try (like pause) are things you tried that can end up preventing/delaying the loading of the model.

And the reason that TimerFcn doesn't just return the model as an output is that it has nowhere to return that output to. TimerFcn is called by the timer object, and that object doesn't provide any way for a TimerFcn to return results to the user. So I use assignin as a way of getting that information back to you in the base workspace.

I guess the big picture here is that I needed to think differently about how to write code that works with the SimBiology graphical interface. Instead of writing precedural code (do A, then B, then C), I need to write "event-driven code." When an event happens (in this case, when a model gets loaded), it calls a function that I specify. That function does whatever task I want to do next. But I leave the details of when to execute this code to someone else (in this case, to SimBiologyNotify and its Timer). Whenever that task gets executed (let's assume it's a simulation), then we presumably want to store the results somewhere we can access them later. That could be saving them to a file, assigning them to a variable in the base workspace, plotting them, or passing the results on to the next step of your analysis. But we can't just return those results as an output in an event-driven world, unless the machinery that drives the events specifically supports it.

[Jim Bosley]

Aha. So its a "There are more things in heaven and earth [and Matlab], Horatio, Than are dreamt of in your philosophy" type of thing. Where I'm Horatio. In any case, I know that its a difficult problem: not something I was stupidly missing an easy answer to. Cuz there isn't one.

I did test your procedure and it works nicely, and so one can open models using simbiology via script, and the callback feature does wait until the model is loaded. At least it waits till sbioroot.Models changes - I assume that once a new model object is put into the sbioroot.Models vector, this indicates that the whole model is loaded. But for a simple test (my callback function is essentionally models.Dose, to show that I have doses loaded).

Your replies have helped me learn - thanks.