This dissertation presents the results of three projects at the intersection of computational semantics and cognitive semantics. The theme underlying all three projects is that closer coordination between cognitive semantics and computational semantics can lead to more accurate computational semantic tools, and can also provide tools that can help address questions in cognitive semantics.

Facial expressions signal emotions and influence social interactions. One mechanism hypothesized to support the recognition of facial expressions is sensorimotor simulation—the observer simulates the observed expression internally and this affords a first-person, experiential understanding of how the target feels. Given enough sensorimotor simulation, this internal activity can be expressed externally in facial mimicry. Numerous studies have found that interfering with mimicry interferes with emotion recognition, particularly when decoding subtle expressions. This implies that mimicry reflects a form of computation that facilitates recognition when needed.

The Embodied Computation model of mimicry hypothesizes that simulation helps recognition by compensating visual mechanisms when visual emotion information is sparse— the more challenging an expression is to decode, the more simulation is involved in decoding it. It makes the unintuitive prediction that more mimicry will occur when emotion evidence is less available, but only if decoding the emotion is necessary. The Motor-Matching model of mimicry hypothesizes that mimicry is based on an automatic action-perception link (Chartrand & Bargh, 1999; Hess & Fischer, 2013). It makes the prediction that mimicry will reflect the emotion evidence: the more evidence, the more mimicry. If recognition is required, this will only increase attention and amplify the overall mimicry. The Emotional Mimicry in Context model hypothesizes that mimicry is not necessarily based on the amount of emotion evidence that is seen but whether or not the signal is interpreted to promote affiliation (Hess & Fischer, 2014). It predicts that the affiliative meaning of the observed expression determines whether or not mimicry occurs.

These hypotheses were tested in three experiments measuring EMG elicited by emotional faces in various challenging conditions. Results are argued to support a novel proposal that combines the insights of the embodied computation and emotional mimicry in context models.

In recent years, converging evidence has suggested that prediction plays a role in language comprehension, as it appears to do in information processing in a range of cognitive domains. Much of the evidence for this comes from the N400, a neural index of the processing of meaningful stimuli which has been argued to index the extent to which a word was predicted before it was encountered. The main aim of this thesis is to investigate the extent to which this prediction can be explained as arising from the statistics of the linguistic inputs we receive over the course of our lives, in line with predictive processing in other cognitive domains. To do this, I turn to language models—computational systems that can calculate the probability of a word given its context based on the statistics of language—and investigate how well their predictions correlate with the N400. The results show that probabilities calculated using language models are highly correlated with N400 amplitude, in many cases better than human-derived metrics such as cloze probability and plausibility, previously the best predictors of the N400. I also show that language model probabilities are able to qualitatively model a wide range of effects, showing significant differences based on the same experimental manipulations that lead to significant differences in N400 amplitude. In addition, the results show that language models that are better able to predict the next word in a sequence are better able to model N400 amplitude in both of these ways, showing both a closer fit to the data and more of the qualitative effects. Taken together, these results show a high degree of correlation between the N400 and predictions based on the statistics of language, consistent with the idea that the predictions indexed by the N400 are at least partly based on language statistics.

Human social intelligence is one of the defining features of our species, however, its origins and mechanisms are not well understood. The advent of Large Language Models (LLMs)—which learn to produce text on the basis of statistical patterns in the distribution of words—both threaten the uniqueness of human social intelligence and promise opportunities to better understand it. In this dissertation, I evaluate the extent to which distributional information learned by LLMs allows them to approximate human behavior on tasks that appear to require social intelligence. First, I compare human and LLM responses in experiments designed to measure theory of mind—the ability to represent and reason about the mental states of other agents. LLMs achieve parity with humans on some tasks (demonstrating that language statistics can in principle underpin mentalistic reasoning) but lag behind in others, suggesting that humans may rely on additional mechanisms. Second, I evaluate LLMs using the Turing test, which measures a machine's ability to imitate humans in a multi-turn social interaction. One model achieves a 50% pass rate, meaning participants are at chance in distinguishing it from a human. Collectively, the results suggest that LLMs simulate many aspects of our social intelligence, but by mechanisms that are potentially quite different from the ones that underpin human social cognition.

During language production, speakers modify various characteristics of their speech to match their linguistic partner’s speech. Although this linguistic alignment occurs for a wide range of dimensions, including speech rate, phonology, word choice, and gesture, the reasons why speakers align at all is poorly understood. Specifically, do speakers align to aid their listener’s comprehension, or to make their own language production easier? If alignment occurs for the listener’s benefit, then the speaker’s language processing system should track linguistic statistics produced by individual partners, allowing them to match their speech to the production of their current partner. Alternatively, if it occurs for the speaker’s benefit, alignment might be partner-independent, with speakers modulating their speech based solely on the overall statistics of their linguistic input.

This dissertation investigates the underlying cause of alignment by assessing whether linguistic alignment in the domain of syntax is partner-specific. In five experiments, speakers interacted with two partners who each systematically produced distinct syntactic structures. For example, one partner always produced prepositional dative sentences (e.g., “The student handed her dissertation to her committee.”) and the other partner always produced double object dative sentences (e.g., “The student handed her committee her dissertation.”). Thus, speakers received equal overall exposure to multiple syntactic structures, but each particular structure was spoken by only one of the partners.

Speakers did not show partner-specific alignment. However, they did show evidence of partner-independent syntactic learning: When the overall distribution of syntactic structures was biased – for example, when both partners produced only prepositional dative sentences – speakers did align to that bias. Additionally, preliminary evidence suggests that speakers aligned to match a specific partner’s production if – but only if – that partner has reduced proficiency (here, a non-native speaker).

Thus, at least in the syntactic domain, speakers modulate their speech to match the overall statistics of the linguistic context, but not to match the preferences of their current listener. These results suggest that linguistic alignment occurs for the speaker’s own benefit, because the linguistic features they recently heard are more accessible for their own production. However, alignment may be partner-specific when it is plausibly necessary for a listener’s successful comprehension, and thus, perhaps, alignment can be tuned for the listener’s benefit under extenuating linguistic circumstances.