Remove image halo / unwanted light reflection

I am working to segment the different parts of the candle flame seen below.

The candle is burning in a container, which leads to a halo/blurry region around the actual flame.

I woud like to get rid of it / detect the flame region excluding this bit.

I have used a combination of thresholding and edge detection but I can't seem to get anything to work. I have looked at erosion but found nothing on getting rid of this specific feature.