Lagged/Cross Correlations with missing values

I need to compute the cross correlation (up to 7 days lag) for each column in my matrix. However, many of the cells contain missing data. The data is satellite data and the missing days mean it was too cloudy. Therefore, the cells with NaN cannot be turned into 0's, but instead must be included in the cross correlation because they are relevant information. However, the xcorr function does not include the 'pairwise' feature like the corrcoef function that omits rows with missing values. So more specifically, how do I calculation the cross correlation (xcorr) with a 7 day lag while omitting rows where an NaN is present. Here is an example of a small segment of data and the equation I am using so far.

Equation: [Cor, lag] = xcorr(A, B, 7, 'coeff');

A = [.2, .33, .4, .34, .56, NaN, .7, .9, .1, NaN]

B = [NaN, .1, .2, .3, NaN, NaN, .4, .5, .55, .34]

Answer 1

[dpb]

isOK=isfinite(A) & isfinite(B);   % both rows finite (neither NaN)
[r,lag]=xcorr(A(isOK),B(isOK),'coeff');

[Tyler Smith]

Matches closely with what I calculated by hand. Thanks for your help!

[dpb]

If you'll look at the code you'll see that's what the 'rows','pairwise' option does in corrcoef.

Why not available in xcorr I dunno; seems like reasonable enhancement.

[Simon de Szoeke]

This doesn't assign the products to the right lags. The lags of the data input to xcorr are changed by truncating missing values with x(isOK). Here's a demonstration for the autocovariance:

t = 1:200;
x = sin(2*pi*t/20);
[a1,lag] = xcov(x,30); % the lag autocovariance
isOK = ~mod(t,2);   % find the effect of every other datum being missing
a2 = xcov(x(isOK),30);
plot(lag,a1, lag,a2); xlabel('lag'); ylabel('xcov'); legend('a1','a2')

In this example, this method of ignoring the missing values doubles the frequency of the data input to xcov.

[JOSE OCHOA-DE-LA-TORRE]

Totally correct , I agree 100% with this warning/advice.

The use of such isOK messes up the delays or spacing of samples, the only possible lag that might be possibel rigt is lag=0.

Answer 2

[Reza Sameni]

An alternative method that does not change the data length or the time-series lags is to replace the NaNs with random variables to avoid them influencing the true cross-correlations:

nanA = find(isnan(A));
nanB = find(isnan(B));
A(nanA) = randn(1, length(nanA));
B(nanB) = randn(1, length(nanB));

[Simon de Szoeke]

This doesn’t change the lags, but introducing random (theoretically uncorrelated) data does influence the covariances .