The interactions between antibodies, SARS-CoV-2 and immune cells contribute to the pathogenesis of COVID-19 and protective immunity. To understand the differences between antibody responses in mild versus severe cases of COVID-19, we analyzed the B cell responses in patients 1.5 months post SARS-CoV-2 infection. Severe, and not mild, infection correlated with high titers of IgG against Spike receptor binding domain (RBD) that were capable of ACE2:RBD inhibition. B cell receptor (BCR) sequencing revealed that VH3-53 was enriched during severe infection. Of the 22 antibodies cloned from two severe donors, six exhibited potent neutralization against authentic SARS-CoV-2, and inhibited syncytia formation. Using peptide libraries, competition ELISA and mutagenesis of RBD, we mapped the epitopes of the neutralizing antibodies (nAbs) to three different sites on the Spike. Finally, we used combinations of nAbs targeting different immune-sites to efficiently block SARS-CoV-2 infection. Analysis of 49 healthy BCR repertoires revealed that the nAbs germline VHJH precursors comprise up to 2.7% of all VHJHs. We demonstrate that severe COVID-19 is associated with unique BCR signatures and multi-clonal neutralizing responses that are relatively frequent in the population. Moreover, our data support the use of combination antibody therapy to prevent and treat COVID-19.