In this chapter, we examine the nature of the long-term memory representation of the pronunciations of words. A fundamental question concerns how abstract these representations are vis à vis the physical manifestation of words, both as gestures and as physical percepts. We consider this question and related issues within the traditions of linguistic cognition and generative phonology. We first explore the general nature of abstraction, and then review the arguments in generative phonology for positing that the units of speech stored in long-term memory (so called ‘underlying forms’) abstract away from many phonetic details. Motivations for concepts such as phonemes and distinctive phonological features are given. We then visit the open question regarding how abstract underlying forms may be allowed to be. We conclude by highlighting the contributions that evidence from neuroscience and sign language linguistics brings to these issues of phonological abstraction in the mental lexicon.