Antibodies to Zika virus (ZIKV) can be protective. To examine the antibody response in individuals who develop high titers of anti-ZIKV antibodies, we screened cohorts in Brazil and Mexico for ZIKV envelope domain III (ZEDIII) binding and neutralization. We find that serologic reactivity to dengue 1 virus (DENV1) EDIII before ZIKV exposure is associated with increased ZIKV neutralizing titers after exposure. Antibody cloning shows that donors with high ZIKV neutralizing antibody titers have expanded clones of memory B cells that express the same immunoglobulin VH3-23/VK1-5 genes. These recurring antibodies cross-react with DENV1, but not other flaviviruses, neutralize both DENV1 and ZIKV, and protect mice against ZIKV challenge. Structural analyses reveal the mechanism of recognition of the ZEDIII lateral ridge by VH3-23/VK1-5 antibodies. Serologic testing shows that antibodies to this region correlate with serum neutralizing activity to ZIKV. Thus, high neutralizing responses to ZIKV are associated with pre-existing reactivity to DENV1 in humans.