help with maintaining order of values when removing NANs and converting matrix to column vector

Hi all, I have looked at a few questions/answers here for how to convert a matrix to column vector and how to remove NAN values from an array. What I would like to do is take some outputs of a finite element model I am running and plot values against each other in a scatter plot. The outputs are in the form of 201x201 double matrices. Using the following code I can acheive what I was hoping to do. My question, when I convert the matrix to column vector and remove the NAN values, are the values I am plotting against eachother located in the same place in the original matrix? The NAN values are in same place in each matrix. Is there a way to maintain the order of values when converting a matrix to column and then removing values? Is this the default way the action is performed? I have attached two matrices for consideration or example.

%start with stress
load("cora-01_micro_stress_griddata.mat");
%load 2nd invariant of stress plot
B=flipud(micro_stress{1,1,10});
%convert to dif stress 
%dif=2*sqrt(2)*B;
%convert matrix to column vector
D=reshape(B,[],1);
%remove NAN values
D = D(~isnan(D)); 
%now strain rate
load("cora-01_micro_strain_griddata.mat");
%2nd invariant of strain rate
A=flipud(micro_strain{1,1,9});
%convert matrix to column vector
C=reshape(A,[],1);
%remove NAN values
C = C(~isnan(C)); 
%scatter stress over strain rate
figure(1);
scatter(log10(C),D);
set(gca,'XDir','reverse');
title('2nd Invariant of Stress over Strain Rate')
xlabel('Log10(2nd invariant of strain rate)');
ylabel('2nd Invariant of Stress MPa');

[Stephen23]

Why do you need to remove the NaN values?

Answer 1

[dpb]

Well, let's find out...

load A
load B
C=[A(:) B(:)];                      % combine the two as column vectors
ixBoth=nnz(all(isnan(C),2));        % count how many rows are both NaN
ixEither=nnz(any(isnan(C),2));       % count how many rows have at least one NaN
assert(ixBoth==ixEither,'NaN Elements Don''t match')

No error, so they are the same.

Now, you don't need to remove the NaN elements only to plot; MATLAB plotting routines silently ignore NaN on their own...

scatter(log10(A),B)

●

[dpb]

"...Is there a way to maintain the order of values when converting a matrix to column and then removing

values? "

MATLAB always orders by storage order unless told differently so reshape(x,[],1) or the more common idiom of x(:) will reliably and always order the elements in sequence so the two arrays are guaranteed to be associated element-by-element.

It's easy to illustrate with a small example that can observe...

M=reshape(1:9,3,3)

M = 3×3

1 4 7 2 5 8 3 6 9

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

M(:)

ans = 9×1

1 2 3 4 5 6 7 8 9

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Or, see what happens if transpose first...

reshape(M.',[],1) % M.'(:) is invalid syntax, must use reshape here on the transposed array

ans = 9×1

1 4 7 2 5 8 3 6 9

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

You observe is still by the first dimension going down by row, then column, just that the array is now transposed so the sequential numbers go across columns instead of down.

One could also prove the point of the NaN value locations being in the same place by actually comparing the two logical arrays of

all(isnan(A)==isnan(B))  % compare the original array locations in 2D

or

all(isnan(A(:))==isnan(B(:)))  % compare the original array locations in vector form

However, all bets are off if there are not the same number of NaN in the two arrays or if they are not in the same locations--then treating the two independently will be guaranteed to get them out of sequence with each other immediately after the first location that doesn't match.

If that were possible, then the way to approach it would be to either find the locations and OR them or do as above and combine the two and use the "either" vector to remove those rows.

But, as noted, if it is only for plotting (and as @Stephen23 asked) then there is no need to actually remove them at all; they'll automagically be ignored as the above illustrates. Only if you are doing calculations that use the NaN values that would then get propagated into the result (like averages or other summary statistics, for example) would you need to exclude them. Most of the standard statistical functions have optional input parameters to allow them to also ignore NaN so again there may not be a need to actually remove them for calculations as well. If any calculations are just on point-by-point basis, then those locations could/would return NaN, but the rest would still be ok, so that wouldn't require removing them, either.

So, while it may seem as a necessry step initiallly, it may not actually be necessary to do what is needed...

[dpb]

OBTW, while it doesn't support it directly, you might like to use a semilog axis...

load A, load B
scatter(A,B)
hAx=gca;
hAx.XScale='log';

to show the actual values on the log scale instead of the log itself on linear.

figure
semilogx(A,B,'o')
xlim tight

will produce the same effect.