UC San Diego

Given an image containing more than one individual, face recognition systems so far have assumed statistical independence between each detected face when making a recognition decision. Contrary to this, for face recognition in unconstrained and natural settings, we show that there is potential for an increase in recognition accuracy by identifying people in groups. We propose models based on conditional and joint probabilities for handling recognition of pairs of individuals. These models are subsequently evaluated on two datasets - one from a television show and another, a personal photo album. In addition to using various state-of-the-art attribute based features, we design new descriptors of our own that can capture naturally occurring color and height correlations in group images. We report recognition accuracy achieved by our relative models and compare this to existing models that assume statistical independence. We examine issues related to data scarcity when building relative models and propose techniques to combine group recognition decisions with statistical independence decisions to overcome these issues. Although improvements in accuracy over baseline techniques are modest for our implementation, we show that there is indeed potential in relative face recognition by using color and height based descriptors in conjunction with our relative models