How can I determine if my data is part of one population or another?

pop1=randn(1000,1);
>> pop2=randn(9000,1)-1;
>> pop=[pop1;pop2];
>> k=randsample(pop,500);

How do I go from here to put the elements of k that are originally found in pop1 into their own matrix?

Answer 1

[John BG]

Hi Mr Anzulis

This is John BG <mailto:[e-mail] [e-mail]>

since not yet sure the sampling is needed, let's start determining common elements

1.

Simplified version of your input data

    pop1=randi([-10 10],5,1)
     =
        -6
        -5
         2
        -1
        -3

    pop2=randi([-10 10],10,1)-1
     =
         6
         1
         0
         8
        -5
         4
         4
        -4
         0
       -10

2.

Merged data

    pop=[pop1;pop2]
     =
        -6
        -5
         2
        -1
        -3
         6
         1
         0
         8
        -5
         4
         4
        -4
         0
       -10

3.

what elements of pop1 are in pop, a their locations

[pop1_in_pop locs_in_pop locs_in_pop1]=intersect(pop,pop1)
pop1_in_pop =
    -6
    -5
    -3
    -1
     2
locs_in_pop =
     1
     2
     5
     4
     3
locs_in_pop1 =
     1
     2
     5
     4
     3

checking indices match

    pop(locs_in_pop)
    =
        -6
        -5
        -3
        -1
         2
    pop1(locs_in_pop1)
     =
        -6
        -5
        -3
        -1
         2

Same for pop and pop2

4.

[pop2_in_pop locs_in_pop locs_in_pop2]=intersect(pop,pop2)
pop2_in_pop =
   -10
    -5
    -4
     0
     1
     4
     6
     8
locs_in_pop =
    15
     2
    13
     8
     7
    11
     6
     9
locs_in_pop2 =
    10
     5
     8
     3
     2
     6
     1
     4

Now you know exactly

• what elements of pop1 are in pop
• what elements of pop2 are in pop
• what elements of pop are in pop2
• what elements of pop are in pop2

And you also know all their respective indices.

5.

to randomly sample, you can use command randsample

length_sample=3
sample1=randsample([1:1:length(pop)],length_sample)

sample1 =
     8    12     1

if you find this answer useful would you please be so kind to consider marking my answer as Accepted Answer?

To any other reader, if you find this answer useful please consider clicking on the thumbs-up vote link

thanks in advance

John BG

<mailto:[e-mail] [e-mail]>

Answer 2

[Jan]

pop1 = randn(1000,1);
pop2 = randn(9000,1)-1;
pop  = [pop1;pop2];
k    = randperm(pop, 500);  % or:  randsample(numel(pop), 500))

pop1_k = pop(k(k <= numel(pop1)));
pop2_k = pop(k(k > numel(pop1)));

You obtain the indices of the random samples instead of the values, and then you can check easily, if they belong to the first part of [pop1; pop2] or not.

Perhaps you mean another problem: All you have is the vector of random samples. Then it is hard to predict, if e.g. a 1.0 belong to the randn or the randn-1 group. Both is possible. The identification of the 2 clusters without any further knowledge is hard, I do not have enough experiences for this. Do you know, that both are normal distributions and have a distance of 1.0?

Not that there is a chance to select only values from pop2. Then you cant recognize, if there is a 2nd cluster.

Answer 3

[Image Analyst]

Simply call randsample() on pop1, not pop. Why did you call randsample() on pop when you knew in advance that you wanted elements strictly from pop1, and you still have the pop1 variable to use in calling randsample()???

[Robyn Anzulis]

I want to take a random sample from all of the data, but I want to find out which of the individual elements in k were originally from pop1 and pop2. However, I'm not sure that this is a calculation that I need at this point.

[Image Analyst]

Try ismember().

Answer 4

[Walter Roberson]

If you have two non-overlapping distributions, then you can examine the compare the individual values to the known ranges of the two distributions to determine which populations they are from.

If you have two overlapping distributions, you can never be sure which distribution any given element is from. However, if the two distributions are not identical, you can calculate zscores to figure out which distribution the item has the higher probability of belonging to. If the zscores are "close" then this would not be at all powerful.