Determine whether a specific array element is already included in the large array

Dear all,

I want to check whether the specific array is already included in the target large array (for example, larger than > size(1e7, 5))

for example, if there is a target array 'arr' which is n*5 array which is quite large

arr = [0 0 0 0 0]
      [0 0 0 0 1]
      [0 0 0 1 0]
      [0 0 1 1 0]
      [0 1 0 0 1]
      ...
      [19980 0 0 0 1]
      [19980 0 1 0 1]
      ... 
      [30000 1 0 0 1]

I would like to check whether a new array element [19980 0 1 0 1] is in the target array.

Now I am iterating the rows of the arrays to check whether a new array element 'arr_temp' is equal to some line of target array,

      arr_temp = [k l m n e]; % our new array element 'arr_temp' with some specific value
      tf = 0;
      for m = 1:u % u is the row size of target array 'a'
         tf = tf + isequal(arr(m,:), arr_temp); % check if arr_temp is in any [arr]
      end     
      if (tf == 0) % if arr_temp is not in the [arr]
         arr(u+1,:) = arr_temp;
         u = u + 1; % increment u
      end

But it seems that this takes too long time (especially, if the size of target array is large,

Is there any other way which takes short time while checking whether a new array element 'arr_temp' is equal to some line of target array?

Thank you very much for your time.

Sincerely, Lee.

Answer 1

[James Tursa]

A brute force way:

tf = any(all(arr == arr_temp,2));

Or, for older versions of MATLAB:

tf = any(all(bsxfun(eq,arr,arr_temp),2));

But, did you want to keep arr in some type of sorted order?

Answer 2

[Walter Roberson]

One way would be to insert each row as keys into a containers.Map object, after which querying for specific contents takes constant time.

The example you gave hints that the rows might be sorted. If so then ismember() of the first column against the first column: it will detect that the column is sorted and will then do a binary search. You probably cannot do better without calling one of the mex membership functions from the File Exchange. Just watch out that the index it gives will not necessarily be of the first match: you would need to scan back slightly and forward slightly to figure out which rows match for a more detailed comparison.

If you are scanning a number of times with different candidates then there are other approaches including using unique() or findgroups() to break the rows up into groups according to their first element; once that is done you have a much smaller subarray to search.