The East Asian summer monsoon (EASM) supplies vital rainfall for over one billion people. El Niño-Southern Oscillation (ENSO) markedly affects the EASM, but its impacts are more robust following El Niño than La Niña. Here, we show that this asymmetry arises from the asymmetry in ENSO evolution: though most El Niño events last for one year, La Niña events often persist for 2-3 years. In the summers between consecutive La Niña events, the concurrent La Niña opposes the delayed effect of the preceding winter La Niña on the EASM, causing a reduction in the magnitude and coherence of climate anomalies. Results from a large ensemble climate model experiment corroborate and strengthen the observational analysis with an order of magnitude increase in sample size. The apparent asymmetry in the impacts of the ENSO on the EASM can be reduced by considering the concurrent ENSO, in addition to the ENSO state in the preceding winter. This has important implications for seasonal climate forecasts.