How to streamline this code to average certain rows?
1 view (last 30 days)
Show older comments
I have a three row column data: lon lat and Z. What I want to do is to round longitude and latitude to 2 digits. For example, 65.3796 will be rounded 65.38. Then the program will sort the data based on longitude and latitude, and average any rows with identical longitudes and latitudes. For example,
The below matrix:
65.39 23.56 10.5
66.70 25.36 6.7
66.70 25.36 7.8
Will become:
65.39 23.56 10.5
66.70 25.36 7.25
Here is my code. The problem is that I have several dozens of millions of data. It will take forever to complete. Anyone could help me create a more streamlined code, so that the program will be much faster?
I really appreciate your help!
clear all
close all
clc
load data/data_3row % D: lon lat Z
lon = D(:,1);
lat = D(:,2);
for i=1:length(lon)
lon(i,1) = round(lon(i,1)*100)/100;
lat(i,1) = round(lat(i,1)*100)/100;
end
D2 = [lon, lat, D(:,3)];
D3 = sortrows(D2, [1:2]);
D3 = [D3; 0 0 0];
[m n] = size(D3);
Ind1 = D3(1,1);
Ind2 = D3(1,2);
row = D3(1,:);
D4 = [];
n4 = 0;
for i=2:m
if D3(i,1)==Ind1 & D3(i,2)==Ind2
row = [row; D3(i,:)];
else
n4 = n4+1;
D4(n4,:) = [row(1,1), row(1,2), nanmean(row(:,3))];
Ind1 = D3(i,1);
Ind2 = D3(i,2);
row = D3(i,:);
end
end
save('data/data_3row_avg.mat', 'D4');
0 Comments
Accepted Answer
Walter Roberson
on 20 Sep 2018
Replace
for i=1:length(lon)
lon(i,1) = round(lon(i,1)*100)/100;
lat(i,1) = round(lat(i,1)*100)/100;
end
with
lon = round(lon, 2);
lat = round(lat, 2);
After that,
vals = D(:,3);
[ulon, ~, ulonidx] = unique(lon);
[ulat, ~, ulatidx] = unique(lat);
meanarray = accumarray([ulonidx, ulatidx], vals, [], @mean, nan);
valididx = find( ~isnan(meanarray) );
[lonidx, latidx] = ind2sub( size(meanarray), valididx );
means = meanarray(valididx);
D4 = [ulon(lonidx), ulat(latidx), means];
11 Comments
Walter Roberson
on 21 Sep 2018
Try
means = splitapply(@(M)mean(M,1), Z, G);
The difference against @mean is that in the case where a single row happened to be a group of its own, mean() would be applied to the row vector and that would return a scalar, whereas in the case where multiple rows were in a group, mean() would be applied to the 2D array that that would return a row vector with one column per column of input. Using the mean(M,1) forces it to always take the column-wise mean, even if there is only one row.
More Answers (0)
See Also
Categories
Find more on Map Display in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!