MATLAB Answers

Locks
0

Find first non NaN element in a column of a matrix

Asked by Locks
on 17 Sep 2015
Latest activity Commented on by Sebastian Castro on 17 Sep 2015
Hi,
I have a realtively large Matrix with time series data. each column stands for one series. given a variable that defines which column I am interested in, I want to extract all non NaN element into a new Array.
Example
lets say myMatrix is 3000X100 and I am looking at column 15. In this column the first 15 elements are NaN and then 300 elements contain data and the rest of the column is NaN again.
I want to extract now exactly those 300 elements in a new Array, let's call it myVec and in Addition make sure it is sorted backwards (last element in myMatrix becomes first element in myVec).
any ideas?
thy

  0 Comments

Sign in to comment.

3 Answers

Answer by Sebastian Castro on 17 Sep 2015
 Accepted Answer

Sure, there's lots of functionality that can help you. Here is an example.
First, I made up some "fake" data for myself. I have a 100x100 matrix where, in the 15th column, I set elements 1 through 15 and 75 through 100 to NaN. You can try it with your data if you'd like:
x = rand(100);
x([1:15,75:100], 15) = NaN;
Next you can pick out the "valid" indices of this column by logical indexing using the "isnan" function. Note that you want the values that are NOT NaN, hence the "~".
validIndices = ~isnan(x(:,15))
You can use these to index into the 15th column of your data. Just to verify, you can display the size of the vector.
myVec = x(validIndices,15)
size(myVec)
Finally, if you want to flip the matrix, you can use the "flipud" (flip up-down) function:
myVec = flipud(myVec)
If you're one of those people who prefers borderline illegible one-liners to show off their MATLAB prowess, you could always do this:
myVec = flipud(x(~isnan(x(:,15)),15));
- Sebastian

  3 Comments

"If you're one of those people who prefers borderline illegible one-liners to show off their MATLAB prowess"
Actually, if I was reviewing your code and I saw this expression split over several lines I would ask what would be the purpose of splitting it in to and creating a variable part way through. Is the variable going to be reused later in the code? Is it just a temporary? Why then is there no comment stating its purpose? etc.
It's nothing about showing off, it's about writing code that cleary express your intent.
thanks guys, exactly what I was looking for, works perfectly!
No problem!
@ Guillaume: I was joking with that comment, and I typed it even before I saw your response! Maybe it came out weird because of that, so my apologies if it seemed offensive.
Indeed, it's nice to eliminate temporary variables especially for large matrices that eat up a lot of memory. I just wanted to explain things step-by-step first so it's easier to understand, and then provide the efficient way at the end :)

Sign in to comment.


Answer by Guillaume
on 17 Sep 2015

This will get you all the non-nan elements of column col in a reverse order:
coldata = flipud(myMatrix(~isnan(myMatrix(:, col)), col))
This will do it for all the columns at once:
nonanmat = arrayfun(@(col) flipud(myMatrix(~isnan(myMatrix(:, col)), col)), ...
1:size(myMatrix, 2), ...
'UniformOutput', false);

  0 Comments

Sign in to comment.


Answer by Tim Jackman on 17 Sep 2015

Making an example matrix:
>> MyMatrix = zeros(3000,100);
>> MyMatrix(1:15,15) = NaN;
>> MyMatrix(16:315,15) = rand(300,1);
>> MyMatrix(316:end,15) = NaN;
Now pulling out only the 15th column:
>> ExtractVec = MyMatrix(:,15);
Using logical indexing, I can extract all values from this array that are not NaN:
>> myVec = ExtractVec(isnan(ExtractVec) == 0);
They will be sorted by their original index in the column. Use flipud to reverse the order:
>> myVec = flipud(myVec);
Hopefully this will help.

  1 Comment

Traditionally,
isnan(x) == 0
is written
~isnan(x)
I don't know how good matlab's JIT is at optimising the former, but theoretically it involves a conversion from double to logical before a logical comparison while the latter is just a logical inversion.

Sign in to comment.