"The result should show that A has not been found in B...
You do not explain why you think that. I am guessing that you misunderstand the meaning of the third input.
Lets read the ISMEMBERTOL documentation, which describes the third input tol as: "Two values, u and v, are within tolerance if abs(u-v) <= tol*max(abs([A(:);B(:)]))". Now lets calculate this tolerance for your arrays:
A = [11538.0882247944, 21569.4133365453];
B = [11537.6120343925, 21611.5704794796];
T = tol*max(abs([A(:);B(:)]))
So we know that any values that are within 216.1157... of each other will be considered to be matching. Lets check how different your array values are:
So the values you are comparing differ by much less than 216.1157..., and so ISMEMBERTOL correctly identifies them as matching. So far ISMEMBERTOL is working exactly as documented. You have not explained anywhere why you think is it not.
Regarding the third input tol:
- you provided a tol value of 10e-3, which is simpler written as 1e-2. Did you really mean 1e-3 ?
- perhaps you expect tol to be the tolerance used in the comparison. That is not the default behavior, but can be achieved very easily by reading the documentation and specifying DataScale = 1.