Integrating Tracking and Recognition

Hi all,

I'm planning to integrate a face tracking algorithm with face recognition algorithm for improving the accuracy of the overall system. Right now I'm confused about the integration part. Any help with this will be greatly appreciated.

Thanks, Saketh

Answer 1

[Dima Lisin]

Hi Sai,

There a few ways to do this, depending on whether tracking is more robust than recognition or the other way around. If your tracking algorithm is really good, but your recognition makes mistakes, then you classify every face in a track and use some voting scheme in case you get different labels for individual views.

Conversely, if you trust the classifier more than the tracker, then if the identity of the person suddenly changes, then you can assume that the tracker made a mistake.

Ideally you would want to have some confidence measure for the tracker and the classifier, so that you know which one to trust at any given time.

[sai saketh nandagiri]

Thanks for the reply Dima. In my case, I want to integrate TLD algorithm with a poor face recognition software and show there is improved performance of the system. But, I'm not sure how exactly to integrate them. I have written code in OpenCV for both tracking and recognition. Can you please guide me on how to integrate them.

Answer 2

[Image Analyst]

I don't see how they're related. How could a perfect recognition algorithm (if you had it) improve the detection and tracking of a face? First you have to detect a face. Then you recognize it as a specific person in your database or as an unknown person. Then you can track it if the face occurred in prior frames. You don't even need to recognize the person to track the person's face - you can track an unknown person without even using recognition at all.

[sai saketh nandagiri]

Thanks for the reply. And the answer to your question, say a face recognition system detects a face in 15 frames and the tracking algorithm detects the same face in 18 frames. We can say that the face appears in 18 frames if the confidence of the tracker is high. So overall we improved the accuracy of the system.

Any suggestions related to integration are welcome.

[Image Analyst]

Doesn't make sense to me, but whatever.... Let's say you have a face recognition system that both detects faces in a frame AND recognizes it as a face it has seen before. And let's say you have a tracking algorithm that tracks objects, regardless of what they are. If the tracking algorithm that does no face analysis at all finds and tracks an object, then that should not help. Maybe it tracked something other than a face. Or maybe it tracked a face for 15 frames and some other object for 3 frames. I just don't see how that makes the face detection and recognition system better. I guess you could ignore or override the face recognition algorithm's decision and just go with the tracking system's decision if they both match up for the first 15 frames.