Sparse for arrays with lots of NaN.
15 views (last 30 days)
Show older comments
Hi,
I am using very large arrays containing a lot fo NaNs. If they were zeros, I could use 'sparse' to save a lot of memory, but since they are NaNs this does not work. Does a sparse datatype exist for arrays with a lot of NaNs?
0 Comments
Accepted Answer
Matt J
on 4 Sep 2013
No, but why do you insist on using NaNs instead of zeros?
4 Comments
Matt J
on 4 Dec 2017
Edited: Matt J
on 4 Dec 2017
How often does sensor failure occur? Is it really often enough to benefit from a sparse representation? That would mean that at least half of your records are invalid.
But in any case, you could, for example, add +1 to all the valid wind speed values so that they are encoded into the interval [1,101]. Then reserve 0 for missing data. Since you clearly don't plan to do linear algebra with this matrix, I don't think this should interfere with anything.
Walter Roberson
on 4 Dec 2017
Edited: Walter Roberson
on 4 Dec 2017
Construct a sparse logical array of locations that are nan. Zero those locations out in the other array and sparse that (assuming that less than about 1/3 of the entries remain as non-zero, to make it worth while to construct the sparse array.)
If more than about 2/3 of your locations are nan, then it might instead make sense to create a logical array of locations which were originally 0 and not nan, and zero out the nan values and sparse the result of that; then the entries in the logical array would say where the "true" zeros are, and the rest of the zeros in the second array would be assumed to be nan.
More Answers (0)
See Also
Categories
Find more on Sparse Matrices in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!