The persistence effect contribution of legacy nutrients is often cited as a reason for little or no improvement in water quality following extensive implementation of watershed nutrient mitigation actions, yet there is limited knowledge concerning factors influencing this response, often called the “persistence effect.” Here, we adopted detrended fluctuation analysis and Spearman analysis methods to assess the influence of land use on the watershed phosphorus (P) persistence effect, using monthly water quality records during 2010–2016 in 13 catchments within a drinking water reservoir watershed in eastern China. Detrended fluctuation analysis was used to calculate the Hurst exponent α to assess watershed legacy P characteristics (α ≈ 0.5, α > 0.5, and α < 0.5 indicate white noise, persistence, and anti-persistence, respectively). Results showed weak to strong P persistence (0.60–0.81) in the time series of riverine P in the 13 catchments. The Hurst exponent α had negative relationships with agricultural land (R = −.47, p =.11) and developed land (R = −.67, p =.01) and a positive relationship with forest land cover (R =.48, p =.10). The persistence effect of riverine P was mainly determined by retention ability (biogeochemical legacy) and migration efficiency (hydrological legacy). A catchment with strong retention capacity (e.g., biomass uptake/storage and soil PO4 sorption) and low migration efficiency results in a stronger persistence effect for riverine P. In practice, source control is more effective in catchments with weak persistence, whereas sink control (e.g., riparian buffers and wetlands) is preferred in catchments with strong persistence effects.