You need to define what you want. For example, do you want just the largest blob with bwareafilt(bw, 1)? Or blobs larger than a certain area with bwareaopen(bw, someArea)? Do you want to fill holes with imfill(bw, 'holes')? Also you should make sure your initial segmentation gives the pothole(s) as white, not black, since binary operations are done on true/white/1 valued pixels, not false/black/0 valued objects.