Motor simulation has emerged as a mechanism for both predictive action perception and language comprehension. By deriving a motor command for an observed stimulus, and engaging in covert imitation, individuals can predictively represent the outcome of unfolding action as a Forward Model. In the context of these proposals, language is described as a highly systematized form of action that relies on the same simulation mechanisms. Some evidence also points to motor stimulation as a supplementary mechanism only under noisy or high-demand circumstances. Evidence of simulation can be derived from error patterns that defer to attributes of the predicting individual, as Egocentric Bias, or through differential responses to Symmetricity, one- vs. two-handed stimuli. Additionally, the sign language literature provides evidence that signers generate predictive representations based on information prior to the onset of a target sign. It is currently unclear, however, what features of transitions during the fluid sign stream make such predictions possible.

Experiment 1 examines the role of (a) motor simulation during action prediction, (b) linguistic status (i.e., pseudosigns vs. grooming gesture) in predictive representations and (c) language experience (i.e., signers vs. nonsigners) in generating predictions. As Egocentric Bias was only observed for non-linguistic stimuli, and only for nonsigners, Experiments 1 does not support strong motor simulation proposals and instead highlights the role of stimulus familiarity. The Experiment 2 focuses on movement and handshape as possible informative transitional information. While movement facilitated predictions regardless of language background and linguistic status of the stimulus, only sign language users relied on transitional handshape information and only for linguistic stimuli. Experiments 3 and 4 examine predictive processing through the lens of motor memory and motor imagery to further investigate the hypotheses that sign language users (a) only exhibit improved performance in linguistic contexts, and (b) are not sufficiently taxed by the present tasks to engage motor simulation during predictive processing. Only participants without sign language experience (a) showed evidence of using motor simulation, and (b) recruited memory and imagery abilities in generating predictive representations. While predicting the future can be difficult, sign language experience seems to shape how some predictions are made.

Skilled hearing readers activate phonological, orthographic, and semantic representations in order to recognize and comprehend words. However, deaf readers may achieve reading comprehension by different means. More specifically, enhanced visual attention, reduced access to phonology, and bimodal bilingualism may influence how deaf readers represent and process words. In this dissertation, I introduce the topic of deaf readers and review literature describing the unique sensory and linguistic experiences that shape how they read. I then present data from a set of behavioral, eye tracking, and electrophysiological studies to compare deaf and hearing readers matched on reading skill and identify alternative approaches to achieving reading success. In Chapter 2, I present a pair of eye tracking studies that suggest deaf readers are more efficient at processing words in sentences and more sensitive to letter transpositions compared to hearing readers. In Chapter 3, I present ERP data from a masked lexical decision experiment showing similarities in how deaf and hearing readers process words and nonwords at sublexical and lexical levels. In Chapter 4, I use ERP evidence to demonstrate that deaf readers co-activate English word representations when recognizing ASL signs. Overall, deaf readers are more sensitive to the visual-orthographic structure of words, which makes them more reliant on orthographic representations during word recognition and more efficient at processing visual word forms in sentence reading.

Inferring another’s perspective is critical for successful communication and social interaction. Speakers must infer another’s visual perspective in order to reference objects appropriately (e.g., objects on your right may be on my left). When spatial relations are expressed via the visuospatial modality as with sign languages (e.g., American Sign Language, ASL), signers and addressees each have different, conflicting visual perspectives of the signer’s articulators. For example, in face-to-face interactions, a sign produced on the left side is perceived by the addressee on the right side. Therefore, in order to avoid miscommunication, one interlocutor must adopt the other’s visual perspective. Little is known about the cognitive mechanisms underlying the perspective-alignment process that is critical for correctly conveying and comprehending spatial descriptions in a signed language. This dissertation examines the relative contributions from inhibitory control, social skills, and mental rotation abilities on the ability to adopt another’s visual-spatial perspective during communication or when performing nonlinguistic perspective-taking tasks.

Chapter 1 provides an overview of the literature describing a) the relationships between inhibitory control, social skills, and visual-spatial perspective-taking (VSPT) abilities and b) how experience with a sign language may affect these relationships. Chapter 2 presents a study examining the relative cognitive burdens for producers and comprehenders of manual gestures from non-egocentric perspectives. Comprehending, but not producing, gestures from non-egocentric perspectives was related to inhibitory control abilities. Chapter 3 examines whether deaf ASL-signing adults approached a nonlinguistic VSPT task socially (like hearing English-speaking adults) or whether they utilized nonsocial, perceptual strategies similar to deaf children. Results suggest that deaf signing adults used a nonsocial approach to the VSPT task, possibly due to effects of language modality and/or sociocultural experiences. Chapter 4 examines whether nonlinguistic perspective-taking or mental rotation abilities are related to signers’ comprehension of perspective-dependent structures within ASL (e.g., locative classifier constructions). Correlational analyses indicated that linguistic perspective-taking was related to participants’ overall ASL abilities, nonlinguistic VSPT abilities, and (to a lesser extent) mental rotation abilities. The overall findings are discussed in terms of ramifications for educational and clinical practices as well as for understanding the relationship between spatial cognition and language.

Iconicity refers to the structured mapping between a lexical form and the conceptual representation of what it means. In this dissertation, I present data from a set of three experiments using Event-Related Potentials (ERPs) that investigate the potential role of iconicity in sign production. In Study One, I present the findings from a picture-naming paradigm, which found the iconic signs were retrieved more quickly than non-iconic signs, particularly when there was a strong visual overlap between picture and sign. The production of iconic signs was additionally associated with increased N400 amplitude, indicating additional semantic processing, though this was reduced for the trials with the greatest visual overlap. In Study Two, I compared the effects of iconicity in a picture-naming task to that in an English-to-ASL translation task to explore whether the faciliatory effect of iconicity was general or task-specific. I found reduced response latencies and N400 amplitudes for iconic signs only in the picture-naming task, with no differences in the translation task, indicating that the effect of iconicity was task-specific. In Study Three, I extended the investigation of iconic signs in picture-naming paradigms by exploring the effects on two distinct manners of iconic mapping: perceptual vs. motoric iconicity. As in Study Two, reduced N400 amplitudes were observed compared to non-iconic signs, but there were no differences in response latencies. Through the use of Laplacian transformations, I compared the distribution of effects for perceptually-iconic signs, which map onto how the referent is perceived, and motorically-iconic signs, which map onto how the referent is handled. I found increased activation at frontal and central electrode sites for perceptually-iconic signs, and increased activation at parietal sites for motorically-iconic signs. These findings suggest that perceptually-iconic mappings engage the ventral stream, while motorically-iconic mappings engage the dorsal stream. Overall, this dissertation finds evidence that iconicity facilitates sign retrieval when features encoded in the sign form map to visual features of the picture stimulus, and that the type of iconicity impacts the neural regions involved in the mapping between sign form and picture stimulus.