How to find differences between 2 large tables that have a mix of string and numbers?
27 views (last 30 days)
Show older comments
Tanika Bawa
on 17 Jul 2023
Commented: Peter Perkins
on 18 Jul 2023
Hello! I have 2 very large tables that have the same number of rows and columns (20441x30), and I would like to know exactly which cells are different. I have tried 'Compare' in the Home tab, but it only tells me in the difference summary 'added', not which values/variables were added). I have also tried setdiff and ismember (error below), but it has a problem with one of my variables (maybe because it has NaNs too in the table).
Error using cell/union>cellunionR2012a (line 242)
Input A of class double and input B of class cell must be cell arrays of character vectors, unless one is a character
vector.
Do you know how I can compare tables that have a mix of string and numbers?
Thank you!
1 Comment
Dyuman Joshi
on 17 Jul 2023
"(maybe because it has NaNs too in the table)"
FYI - MATLAB treats NaNs (as well as NaTs, <undefined> and <missing>) values as not equal to each other, nor equal to themselves as well.
If you want to compare NaNs (and others) and output true, use isequaln
Accepted Answer
Image Analyst
on 17 Jul 2023
Have you tried iterating over every element of both tables using isequal to determine whether the locations are the same? Something like (untested)
[rows, columns] = size(table1);
matches = false(rows, columns); % Initialize to no matches (no equality).
for col = 1 : columns
for row = 1 : rows
% See if this location is equal.
% Set equality map to true if they're equal.
if isequal(table1{row, col}, table2{row, col})
matches(row, col) = true;
end
end
end
If you have any more questions, then attach your data and code to read it in with the paperclip icon after you read this:
More Answers (1)
Peter Perkins
on 17 Jul 2023
Edited: Peter Perkins
on 17 Jul 2023
I thnk looping over every row in the tables will not be very performant. I would think this will do better:
matches = false(size(table1));
for var = 1 : width(table1)
matches(:,var) = (table1.(var)==table2.(var)) ...
|| (ismissing(table1.(var)) && ismissing(table2.(var)))
end
That assumes the variables support ismissing; you might need a try/catch in general.
3 Comments
See Also
Categories
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!