Cross-modal priming experiments have shown that surface variations in speech are perceptually tolerated as long as they occur in phonologically viable contexts. For example, [klim] {cleam) gains access to the mental representation of clean when in the context of [klimpoks] (cleam parks), since the change is a natural one, reflecting the phonological process of place assimilation. This implies that speech perception involves processes of phonological inference, which recover the underlying form of speech. Here we investigate the locus of these inference processes, using the phoneme monitoring task. A set of stimulus sentences were created containing deviations that were either phonologically viable (as in cleain parks above) or unviable. In Experiment 1, subjects monitored for the segment underlying the surface change (in the above example, /n/) and in Experiment 2 the following segment (/p/) was the taiget. In addition, the lexical status of the carrier word was manipulated (e.g., clean vs threan), contrasting lexical and non-lexical theories of phonological inference. Both experiments showed stiong effects of phonological viability for real words, with weaker effects for the non-word stimuli. These results suggest that phonological inference can occur non-lexically, but that it interacts strongly with the process of lexical access.