Storing a Finite Difference History

Is there a good way to store finite difference history. I initially tried getframe() but that required that the frame actually pop up, which prohibited me from using the computer while the simulation was running. I next turned to making a 3 dimensional matrix but this requires a ton of memory, even if I store the values as singles rather than doubles. It's been pretty tedious to recalculate how much memory I'll be using every time I change a parameter on the simulation. Last time I ran it, it ran 8 hours before erroring out with a memory error. I don't really even understand why because I pre declared my history variable: basically

Hist = zeros(251,251,400);

which took fine, but then errored hours into the program when saving a time frame e.g.:

Hist(:,:,I) = NewMap;

where I is still less than 400;

So I've kind of got a problem and a half here. I have a memory problem that I'd like to find a new way around altogether, but I'd also like to understand why I'm getting this memory error to begin with.

Thanks, Jason

[Sean de Wolski]

Could you show a little bit more of the code please.

(251*251*400)elements * 4 bytes/element (single)

should not be cause memory errors on most modern systems.

[Jason Smoker]

The system is a couple years old (I'm a grad student). It's a stock Dell Optiplex GX620 running at 3.0GHz with 2GB ram. I'm not sure what to post for you. The entirety of the code including subfunctions is about 1k lines long. I guess I could post just the loop part which is about 100 lines (less without comments).

I can tell you this, though, which I thought was odd. If I run

[code]

clear all

Hist = zeros(251,251,400);

[\code]

it saves ok. but if I run

[code]

clear all

Hist = zeros(251,251,400);

Hist = zeros(251,251,400);

[\code]

the second save to Hist causes a memory error.

[Image Analyst]

Why are you "saving" the zeros array to the exact same variable called "Hist" again? That's bad practice, by the way, to use a built-in function name as the name of your variable even if MATLAB is case sensitive.

[Image Analyst]

Try clearvars instead of clear all.

[Jason Smoker]

It was just an example. I could have used "ones" or "magic". Talking to a CS friend I found later that this error comes from the fact that the old matrix isn't deleted until the new one is put together. Therefore maxing out the memory.

Answer 1

[Dr. Seis]

It may be more time consuming, but if you could code something up to simply write out the results to a text file (or binary file) then you at least have a copy of what was done and all would not be lost. You could even set it up to pick up where it left off if for some reason the run bombs (and you still think the prior results in the history are valid).

[Jason Smoker]

I've thought of that, but there's several issues with that. I don't know how to just add one piece of an matrix in unless save it as a brand new variable name each time.

e.g.

if a = 4 x 4 matrix and I save it

[code] save savefile.mat a [/code]

I can't save the next one as an increment of the previous

[code] save savefile.mat a(:,:,2) [/code]

I'd have to save it as a new variable resulting in a file with 400+ variables. I guess that's a possible method, but it seems very "inelegant" in programming kind of way. There must be a smoother method.

I've thought of implementing the

[code]

try

catch me

end

[/code]

but again, this just a bandaid and doesn't really solve the problem

[Dr. Seis]

I don't understand why you need 400+ variables. In order to compute the current state you should only need the last state. So you should have only 2 matrices (i.e., "a" and "a_old"). "a_old" would be the initial state before you begin, then "a" would be computed based on "a_old". For the next loop, "a" would become "a_old" and the process would repeat... except after each loop you write out "a".

[Jason Smoker]

I am only using the last one to compute the new one. The 400+ variables is a "log". I want to know what state the field is along the way; how the waves bounce off structures, etc. The actual iterations number many times the 400, but I'm saving a snap shot every 1000 or so.

[Dr. Seis]

There are other ways to write out a matrix to a txt file in a way that you can append the next, say, 4x4 matrix to the the bottom of the file. For example, each line you print out to the file could be the 16 elements of your matrix. Afterwards, you can visualize that data by pulling out a sub-set of rows and reconstruct to the original 4x4 matrix.

If you want to use something like "getframe" but without a figure/frame popping up, then check out "im2frame" in the help navigation window. If you can figure out how to make an indexed image, then you can basically save each frame without having those pesky figure/frames pop up.

[Jason Smoker]

I may try that 4x4 to 1x16 idea and append after so many iterations. that may be my best bet. I haven't figured out how to preallocate for a frame matrix and would guess it would take more memory anyways. I'll definitely log that function away in my memory for later though. Thanks.