California Policy Lab

A subset of undergraduate applicants to the University of California, Berkeley were invited to submit letters ofrecommendation as part of their applications. I use scraped text of the submitted letters, natural languageprocessing tools, and a within-subject experimental design wherein applications were read in parallel with andwithout their letters to understand the role that this qualitative information plays in admissions. I show thatletters written on behalf of underrepresented applicants were modestly distinctive. I also construct an index ofletter strength, measuring the predicted impact of the letter on the student’s application score. I show thatunderrepresented applicants tend to get weaker letters, but that readers pay less attention to letter strength forunderrepresented students. Overall, the inclusion of letters modestly improved application outcomes for theaverage underrepresented student.

This work has been supported, in part, by the University of California Multicampus Research Programs and Initiatives grants MRP-19-600774 and M21PR3278.