UC San Diego

At first sight of a new person, people quickly form impressions on how attractive, trustworthy, and warm the person looks. Despite the dubious accuracy of these first impressions, people rely on them to navigate social interactions regarding interpersonal relationships, political voting decisions, and financial choices. As we progress into a digital era with frequent social media usage and cultural exchanges, it is crucial to understand the universal principles and cultural idiosyncrasies of impression formation from face images.

In this thesis, we examine social impressions of faces through the lens of computational modeling, psychological experiments, and cultural comparisons. First, we develop a model that automatically predicts human social impression judgments using neural networks. Building on the predictive model, second, we build a generative model that can change faces holistically to augment or decrease specific characteristics, such as attractiveness and aggressiveness of a face. Third, we examine how specific physical attributes, such as hair color, affect impressions of faces, using a GAN model, and psychological experiments. Finally, we conduct a large-scale cross-cultural study, using 18 traits related to approachability, youthful-attractiveness, and competence evaluation, with Caucasian and Asian participants rating Caucasian and Asian faces. We investigate the mediating factors behind impression formation and estimate how high-level facial features such as age, eyeglasses, and smiles are judged similarly and differently by people from two cultures.

Overall, our work provides a computational framework to predict and modify faces, which is practically useful in real-life scenarios regarding optimal self-image presentation. Our model lays the foundation for the quantitative study of first impressions. Our psychological and cross-cultural studies reveal the universality and culture-specific judgments mediated by high-level features that affect first impressions. These findings motivate future research in social psychology to understand the deeper cultural roots behind the differences in first impression formation and be aware of the potential bias toward certain social groups.