How do I stop the code and get the results?

I am trying to run the Matlab code for Arnoldi method. The code dose not stop because the matrix is too large,. Is there a way to get the output?

[V,H] = Arnoldi();

function [V,H]=Arnoldi(A,v1,m)
%  Load the data
 load('soc-Epinions1.mat');
 A = Problem.A;
 n=size(A,1);
 m=9000;
V=zeros(n,m);
H=zeros(m,m);
v1=rand(n,1);
V(:,1)=v1/norm(v1);
for j=1:m % the Anoldi loop 
    z=A*V(:,j);
    for i=1:j
        H(i,j)=V(:,i)'*z;
        z=z-H(i,j)*V(:,i);
    end
H(j+1,j)=norm(z);
if H(j+1,j)==0, break, end
V(:,j+1)=z/H(j+1,j);
end
H=H(1:m,1:m);
V=V(1:n,1:m);
f=diag(V*(expm(H)-eye(m))*V')
end

[Walter Roberson]

Your calculation does

9000 * (9000+1)/2 

ans = 40504500

iterations. That takes time.

Which matrix is too large?

[Omar B.]

The size of the matrix A is 75888 * 75888.

The number of iteration m= 9000.

[Walter Roberson]

You have a double-nested loop. Your i goes 1 to j, where j is going 1:9000. That requires 1+2+3+...9000 iterations = 40504500

On my system, it takes a notable amount of time to execute, but at the end

Name Size Bytes Class Attributes

H 9000x9000 648000000 double

V 75888x9000 5463936000 double

[Walter Roberson]

The size of the matrix A is 75888 * 75888.

Yes, but it is sparse.

Your H and V matrices would not be sparse.

You could potentially use techniques such as "tall arrays" to break up the calculation into smaller steps, but if your H + V matrices are too large for your system (such as if you only have about 8 gigabytes), then saving memory during calculations is not the problem: you would still need the roughly 6 gigabytes of memory to hold H and V no matter how you did the calculation. If your system does not have the memory available, then the problem is too large for your system.

[Omar B.]

I do not know if the authers used a technique to get the results. In their paper, they just used Arnoldi process with different values of iteration (m) and obtained the results when m=9000 by compute the diagonal of Vf(H)V' . Anyways, thank you for your help.

[Walter Roberson]

Perhaps they just ran the code on a larger system. The code ran on my system. Not quickly, but it ran.

Answer 1

[Steven Lord]

If you're out of time and need to get the results that have been computed, under certain circumstances (the main ones being the MATLAB Editor is open and you're using a sufficiently new release, plus the code has to give MATLAB a chance to recognize and process events) press the Pause button on the Editor tab of the toolstrip. save or assignin (into the base workspace) the results that you want to access and then either quit debugging (if you want to terminate the execution of the function) or continue (if you want to let it continue.)

Note that if you do this and quit debugging there's no way to "let MATLAB restart from where it was" unless your function was specifically designed to accept variables representing the state of the function's execution.

[Walter Roberson]

If you are executing and you have the editor open to a function that is called by the code, you can use the editor interface to turn on a breakpoint in the function. The next time the line is reached, MATLAB will stop at the debugger and you can then edit other functions or add a "dbstop if error" or whatever is appropriate, or use dbup to get to the level where you have copies of the variables you want to save()

If you are executing and you have the editor open at all, you might be able to use the menu to open the function that is executing, so that you can inject a breakpoint into it. In my experience this does not always work: sometimes it holds off on opening the file until the next time it can process events.

[Omar B.]

I pressed the Pause button on the Editor as you said and take m=2000 , but I got the following error

Error using  * 
Requested 75888x75888 (42.9GB) array exceeds maximum array size preference. Creation of
arrays greater than this limit may take a long time and cause MATLAB to become unresponsive.

[Walter Roberson]

f=diag(V*(expm(H)-eye(m))*V')

You do not return f from the function. You display the results of the diag, but that will be 75888 elements long and will just scroll off the display. So not a lot of point in calculating f there.

I suggest that you comment out that line, and run your code and save the V and H that result. And then I suggest that you write a new function to implement the f calculation.

V is 75888 by 9000.

H is 9000 by 9000. expm(H) is 9000 by 9000. eye(m) is eye(9000) is 9000 by 9000. So expm(H)-eye(m) is 9000 by 9000.

V*the above is then 75888 by 9000 * 9000 by 9000, giving 75888 by 9000.

You now have 75888 by 9000 * transpose(75888 by 9000) which is 75888 by 9000 * 9000 by 75888. That is 75888 by 75888 result. That is the place that your MATLAB runs out of memory.

But you have the diag() around that. You only need 75888 of the results, not 75888 by 75888 of the results.

If you have a new enough MATLAB I wonder if MATLAB would be smart enough to only calculate the diagonal elements? You did not happen to mention the release you are using.

So... implement a function to calculate f from V and H. Have it get as far as calculating V*(expm(H) - eye(m)). Then have it only calculate the diagonal elements of the *V', either by some clever rearranging (maybe) or by looping (maybe not as fast as possible but reliable and leaves room for finding some clever solution later if required.)

[Omar B.]

Thank you so much for your help. I tried what you suggested, but I got the same error.

What do you mean by rearranging or by looping?

load('soc-Epinions1.mat');
 A = Problem.A;
 n=size(A,1);
 v1=randn(n,1);
 m=2000;
[V,H]=Arnoldi4(A,v1,m);
f=V*(expm(H)-eye(m,m));
b=diag(f*V')

%............................
Error using  * 
Requested 75888x75888 (42.9GB) array exceeds maximum array size preference. Creation of
arrays greater than this limit may take a long time and cause MATLAB to become unresponsive.

[Walter Roberson]

syms f [5 3]
syms V [5 3]

b1 = diag(f*V')

b1 = 

b2 = sum(f.*conj(V),2)

b2 = 

b1 - b2

ans = 

So... you can use

load('soc-Epinions1.mat');
 A = Problem.A;
 n = size(A,1);
 v1 = randn(n,1);
 m = 2000;
[V, H] = Arnoldi4(A, v1, m);
f = V*(expm(H)-eye(m));
b = sum(f .* conj(V),2);

[Omar B.]

Thank you very much. I got the same results as in the paper. Thank you again.