In optical coherence tomography (OCT), mapping the polarization state of the reflected light provides additional information about tissue structure and prevents polarization induced image artifacts. As OCT is increasingly used with subjects in vivo, demands on the imaging system and data acquisition rates increase. We present a fiber-based, rapid scanning, polarization-sensitive OCT system capable of acquiring image data at the rate of 20 K pixels/s. To achieve high scan rates, a rapid-scanning optical delay (RSOD) line generates a reference arm group delay while a waveguide-based phase modulator generates a suitable, stable carrier frequency for the detection electronics. Group delays up to 4.5 mm are obtained at frequencies approaching 1 kHz. The group delay and carrier frequency are independently controllable, which has the advantage that either the lateral or the axial scan direction may be chosen as the fast axis. Tomographic images corresponding to the intensity and polarization components of the Stokes vector describing the backscattered light are obtained by analyzing the interference signals from two orthogonally polarized channels in the detection arm for each of four polarization states incident on the sample. Polarization images of human skin taken in vivo are shown.