Clear Filters
Clear Filters

Comparing a Row with doubles and strings to a large table for deleting

2 views (last 30 days)
I need to filter a large table of data for very specific characteristics, the table contains strings and doubles, I managed to filter it to the point where I now have a separate table with the rows i need to remove (rows_rejected) copied over. However I can't get the ismember or contains functions to give me a logical array I can use to filter out the options I want deleted.
I know it's not a floating point issue since I did not perform any math on the tables and the doubles provided never go beyond 2 decimals and are predefined.
Ignore the loops in %
Litho_Raw = innerjoin(Litho_Raw,Metadata);
%Check for uniqueness up to the rock code to get rid of complete duplicates
[u,I,J] = unique(Litho_Raw(:,1:5), 'rows','last');
DupRows = setdiff(1:size(Litho_Raw,1), I)';
dupRowValues = Litho_Raw(DupRows,:);
Litho_Processed = Litho_Raw(I,:);
%use for loops to filter and verify for depth duplicates on each well
i = height(WellIDs);
Litho_Processed.rockCode = str2double(Litho_Processed.rockCode);
Litho_Processed.freeText = char(Litho_Processed.freeText);
Litho_Processed.metadataLithology = char(Litho_Processed.metadataLithology);
Litho_Processed.metadataStratigraphy = char(Litho_Processed.metadataStratigraphy);
Litho_Processed.metadataChronology = char(Litho_Processed.metadataChronology);
%Litho_Processed.Well_ID = num2table(Litho_Processed.Well_ID);
%for 1:1:i
Well = WellIDs.Well_ID(151);
Rows_ID = Litho_Processed.Well_ID == Well;
Well_Measurements = Litho_Processed(Rows_ID,:);
j = height(Well_Measurements);
%for 1:1:j
depthf = Well_Measurements(1,2);
depthf = table2array(depthf);
Rows_depth = Well_Measurements.fromDepth == depthf;
repeated_base = Well_Measurements(Rows_depth,:);
for k=1:height(repeated_base)
val(k)= repeated_base{k,6};
chosen = max(val);
toremove = ~ismember(repeated_base.rockCode,chosen);
rows_rejected = repeated_base(toremove,:);
%Litho_Processed = table2cell(Litho_Processed);
%rows_rejected = table2array(rows_rejected);
%for iLitho = 1:numel(Litho_Processed)
% tmpLitho = Litho_Processed{iLitho};
%tmpLitho(contains(tmpLitho,rows_rejected)) = [];
%Litho_Processed{iLitho} = tmpLitho;
tokeep = ~ismember(Litho_Processed,rows_rejected);
%Litho_Processed_substracted = Litho_Processed(tokeep,:);
the table has the variables
{'Well_ID'} {'fromDepth'} {'toDepth'} {'primaryRock'} {'secondaryRock'} {'rockCode'}
Columns 7 through 10
{'freeText'} {'additionalComme…'} {'coments'} {'stratigraphyCode'}
Columns 11 through 13
{'metadataLithology'} {'metadataStratig…'} {'metadataChronol…'}
which are various doubles, strings and cells

Answers (1)

Pranav Verma
Pranav Verma on 18 Feb 2021
Hi Albie,
From your question I have the understanding that you have two tables where your other table contains the rows you want to delete from the original table. You can use the setdiff function in MATLAB to filter out the rows. You can use C = setdiff(A,B,setOrder) where C will contain entries present in A but are not present in B and setorder will preserve the order of A.


Find more on Numeric Types in Help Center and File Exchange




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!