The top-of-atmosphere (TOA) radiation field from a coupled atmosphere-ocean system (CAOS) includes contributions from the atmosphere, surface, and water body. Atmospheric correction of ocean color imagery is to retrieve water-leaving radiance from the TOA measurement, from which ocean bio-optical properties can be obtained. Knowledge of the absolute and relative magnitudes of water-leaving signal in the TOA radiation field is important for designing new atmospheric correction algorithms and developing retrieval algorithms for new ocean biogeochemical parameters. In this paper we present a systematic sensitivity study of water-leaving contribution to the TOA radiation field, from 340 nm to 865 nm, with polarization included. Ocean water inherent optical properties are derived from bio-optical models for two kinds of waters, one dominated by phytoplankton (PDW) and the other by non-algae particles (NDW). In addition to elastic scattering, Raman scattering and fluorescence from dissolved organic matter in ocean waters are included. Our sensitivity study shows that the polarized reflectance is minimized for both CAOS and ocean signals in the backscattering half plane, which leads to numerical instability when calculating water leaving relative contribution, the ratio between polarized water leaving and CAOS signals. If the backscattering plane is excluded, the water-leaving polarized signal contributes less than 9% to the TOA polarized reflectance for PDW in the whole spectra. For NDW, the polarized water leaving contribution can be as much as 20% in the wavelength range from 470 to 670 nm. For wavelengths shorter than 452 nm or longer than 865 nm, the water leaving contribution to the TOA polarized reflectance is in general smaller than 5% for NDW. For the TOA total reflectance, the water-leaving contribution has maximum values ranging from 7% to 16% at variable wavelengths from 400 nm to 550 nm from PDW. The water leaving contribution to the TOA total reflectance can be as large as 35% for NDW, which is in general peaked at 550 nm. Both the total and polarized reflectances from water-leaving contributions approach zero in the ultraviolet and near infrared bands. These facts can be used as constraints or guidelines when estimating the water leaving contribution to the TOA reflectance for new atmospheric correction algorithms for ocean color imagery.