Large Language Models have taken the cognitive science world by storm. It is perhaps timely now to take stock of the various research paradigms that have been used to make scientific inferences about "cognition" in these models or about human cognition. We review several emerging research paradigms---GPT-ology, LLMs-as-computational-models, and "silicon sampling"---and review recent papers that have used LLMs under these paradigms. In doing so, we discuss their claims as well as challenges to scientific inference under these various paradigms. We highlight several outstanding issues about LLMs that have to be addressed to push our science forward: closed-source vs open-sourced models; (the lack of visibility of) training data; and reproducibility in LLM research, including forming conventions on new task "hyperparameters" like instructions and prompts.

People have the fascinating ability to infer causality by observing other humans’ actions. We modelled this process using a Bayesian rational agent model and showed how people can reason about another agent’s beliefs and, by extension, infer the world’s causal structure. We compared the model’s predictions against humans’ causal judgements on a novel inference task. Participants (N = 171) were shown a dynamic scene depicting either a human agent, robot agent, or both agents acting on two objects sequentially before observing an outcome. After observing the human (vs the less intentional robot) agent, people were more likely to infer that both objects (in sequential order) caused the outcome. When two agents of different intentionality were shown, people favored the object that the intentional agent interacted with as the cause of the outcome. Our model captured these inference patterns well and revealed insights into reasoning about semi-intentional agents and multi-agent contexts.

In Ancient Greek philosophy, emotion is considered the opposite of cognition: cognition is rational while emotion is irrational; cognition is cold while emotion is hot. Such thinking has influenced the tradition in the field of cognitive science, where emotion is often described as what contaminates or irrationalizes our judgments and decision-making. Thus, while the ways in which humans express, recognize, and experience emotion have been studied extensively in other subfields of psychology (e.g., affective science), rigorous attempts to understand the role of emotion in thinking, learning, and reasoning have been surprisingly sparse in contemporary cognitive science. As a quick example, a keyword search in the Cognitive Science Proceedings for "Affect" or "Emotion" shows that only 2% of published proceedings over the past three years (55 out of 2454 between 2018 and 2020) have titles including one of these key words.

People act upon their desires, but often, also act in adherenceto implicit social norms. How do people infer these unstatedsocial norms from others’ behavior, especially in novel so-cial contexts? We propose that laypeople have intuitive the-ories of social norms as behavioral constraints shared acrossdifferent agents in the same social context. We formalize in-ference of norms using a Bayesian Theory of Mind approach,and show that this computational approach provides excellentpredictions of how people infer norms in two scenarios. Ourresults suggest that people separate the influence of norms andindividual desires on others’ actions, and have implications formodelling generalizations of hidden causes of behavior.

Navigating the social world relies upon the human capacity for mentalizing, or attributing intentions to social agents.Unfortunately, currently available commercial robots still lack such awareness of human intentionality. Building uponrecently-proposed Bayesian models of Theory of Mind (ToM), we propose a ToM model that can handle intention inferencein dynamic, fast-changing environments like a hospital, where staff have to attend to objectives and emergencies as theyarise. Our model infers and maintains a distribution over possible intentions, and uses the posterior predictives to forecastfuture trajectories, which is essential for robot motion planning. We show that our model performs excellently at inferringthe intentions and trajectories of human players controlling a nurse agent in a simulated environment. This work lays thefoundation for robots that can co-work with humans in dynamic, social environments with high-stake goals.

Recent advances in Artificial Intelligence have produced agents that can beat human world champions at games like Go,Starcraft, and Dota2. However, most of these models do not seem to play in a human-like manner: People infer others’intentions from their behaviour, and use these inferences in scheming and strategizing. Here, using a Bayesian Theory ofMind (ToM) approach, we investigated how much an explicit representation of others’ intentions improves performancein a cooperative game. We compared the performance of humans playing with optimal-planning agents with and withoutToM, in a cooperative game where players have to flexibly cooperate to achieve joint goals. We find that teams with ToMagents significantly outperform non-ToM agents when collaborating with all types of partners: non-ToM, ToM, as well ashuman players, and that the benefit of ToM increases the more ToM agents there are. These findings have implications fordesigning better cooperative agents.

Reasoning about others’ emotions is a crucial component in so-cial cognition. Here, we tested the ability of preschool childrento reason about an agent’s emotions following an unexpectedoutcome. Importantly, we controlled for the actual payoff ofthe outcome, while varying the prior expectation of the agents.Five-year-olds, but not four-year-olds, were able to correctlyjudge an agent’s emotions following an unexpected outcome(Experiment 1). When explicitly provided with the agent’s ex-pectations, 4-year-olds were then also able to correctly judgethe agent’s feelings (Experiment 2). Our results suggest thatthe ability to reason about emotions given outcomes and priorexpectations develops by 4 years of age, while the ability tospontaneously infer such prior expectations develops soon af-ter. We discuss our results in light of the developmental litera-ture on emotion understanding and counterfactual reasoning.

Humans use rich intuitive theories to explain other people’sbehavior. Previous work in lay psychology of behavior havetended to treat emotion as causing primarily unintentional be-havior (e.g., being sad causes one to cry), neglecting how peo-ple incorporate emotions into explanations of rational, inten-tional actions. Here, we provide preliminary explorations intointegrating emotions into a theory of folk psychology. Specif-ically, we show that in the lay theory, people are willing toendorse emotions as causes of intentional actions. Moreover,people readily attribute beliefs and desires as explanations foremotional expressions. This work provides a first step in elabo-rating people’s rich understanding of emotions as an importantcomponent of intuitive social cognition.

It is commonly believed that expressions visually signal rich diagnostic information to human observers. We studied how observers interpret the dynamic expressions that people spontaneously produced during a real-life high-stakes televised game. We find that human observers are remarkably poor at recovering what events elicited others' facial and bodily expressions. Beyond simple inaccuracy, people's causal reasoning exhibits systematic model-based patterns of errors. We show that latent emotion representations can explain people's reasoning about the unseen causes of expressions. A hierarchical Bayesian model simulates which events people infer to be the cause of others' expressions by comparing the emotions inferred from the expressions against the emotions people were predicted to experience in various situations. This causal model provides a close, parameter-free fit to human causal judgments, suggesting that humans interpret expressions in the context of emotion predictions generated by a causally-structured mental model of other minds.