I formalize the problem of care in the mathematical language of sequential decision-making. Drawing upon insights from developmental psychology, robotics, and computational cognitive modeling, I conceptualize care as a dynamic interplay between the caregiver ('one-caring') and the care recipient ('cared-for'). Caring actions maximize the utility of the cared-for at a future point when they are required to act autonomously. Since this quantity cannot be directly optimized, the focus is on enabling increasing levels of autonomy through environmental shaping, risk reduction, and safe exploration. I distinguish caregiving from helping and teaching by care's focus on exploration and autonomy that increase capacity over time. In the context of elderly care, the emphasis shifts towards preserving rather than enhancing capacity. Finally, I consider the role of caregiving in the development of moral values and the possibility of artificially intelligent agents that might someday care for us.

Social rewards shape human behavior. During development, a caregiver guides a learner's behavior towards culturally aligned goals and values. How do these behaviors persist and generalize when the caregiver is no longer present, and the learner must continue autonomously? Here, we propose a model of value internalization where social feedback trains an internal social reward (ISR) model that generates internal rewards when social rewards are unavailable. Through empirical simulations, we show that an ISR model prevents agents from unlearning socialized behaviors and enables generalization in out-of-distribution tasks. Incomplete internalization, akin to "reward hacking" on the ISR, is observed when the model is undertrained. Finally, we show that our model internalizes prosocial behavior in a multi-agent environment. Our work provides a framework for understanding how humans acquire and generalize values and offers insights for aligning AI with human values.

Even simple, ambiguous signals can have a rich interpretation when viewed in the context of an interaction in a shared environment. We create a model called Paternalistic Communication by combining an existing model of overloaded language -- Rational Speech Acts (RSA) -- with a full agent model of Theory of Mind (ToM). This integration allows signals to be processed in conjunction with common ground in a principled manner dependent on task-dependent action utilities. This modeling perspective treats communication as a way to coordinate diverging perspectives in a cooperative setting. Under Paternalistic Communication, a speaker decides what to say by predicting their partner’s reaction based on the information in common ground and then evaluates those reactions using their own mind which may contain additional information. We demonstrate the flexibility and performance of Paternalistic Communication in a case study with ambiguous signaling through a set of simulations.

Humans navigate daily decision-making by flexibly choosing appropriate approximations of what ought to be done. Which mental algorithms do people use, and when? We use behavioural experiments and modelling to investigate three computational principles known to influence decision making: noisy utility maximization, discounting, and the probability warping principle of Prospect Theory. While these principles have been shown to separately influence human behaviour in simple laboratory tasks, such as bandits and gambles, we evaluate their combined use in the context of a naturalistic spatial search that required sequential decision-making. We found that while aggregate human behaviour can be reasonably well explained by an optimal planner with noisy utility maximization, individual-level behaviour exhibits consistent irregularities, that deviate from expected utility theory. We show that model-based prediction of individual-level behaviours in our experiment is significantly improved by combining the three computational principles, and benefits particularly strongly from probability warping. Furthermore, our results suggest that probability warping may be a common factor of human decision making, that generalizes beyond the gambles explored in Prospect Theory, to natural human behaviours such as spatial search and navigation.

We present a generative model of how observers think about the emotions experienced by players in a socially-chargedgame: a public, high-stakes, one-shot Prisoner’s Dilemma. The model extends inverse planning frameworks to captureobservers’ judgments about players’ reactions to hypothetical events. Observers attribute different beliefs and values toplayers based on what decisions the players make. We model how observers’ noisy inferences of players’ mental contentsbias emotion predictions. Incorporation of non-monetary features into forward planning enables us to model emotions thatreflect complex social concerns (e.g. Embarrassment depends on how much players think others will infer that they tried totake advantage of their opponents). In addition to matching the intensities of twenty attributed emotions, the model reflectshow observers’ emotion judgments covary within single stimuli, indicating that the model captures important aspects ofthe generative process underlying humans’ emotion attributions in this game.

Cumulative culture ultimately depends on the fidelity of learning between successive generations. When humans learnfrom others in addition to observing inputs and outputs we often observe the process which led to that output. Forinstance, when preparing a meal we don’t just observe a pile of vegetables and then a ratatouille. Instead, we observe acausal process by which those ingredients are transformed. Here we use programs to represent a cultural process and showthat the observation of an execution trace speeds up program induction even when learning from only a single example.This mechanism could account for (1) the high fidelity of social learning which leads to cumulative culture in humans(2) unify the role of emulation and imitation in social learning and (3) account for aspects of moral learning such asritualization.

We propose a formal model of humans’ intuitive theories of others’ emotions. From a single choice in a social interaction(e.g. the choice to cooperate in a Prisoner’s Dilemma game), human observers can infer a player’s complex values, such asprosocial preferences and reputational concerns. When the player then experiences a new situation (the game’s outcome),observers infer the player’s reaction to the event based on the mental state likely to have produced the player’s action. Herewe capture this process by inverting a richly structured generative model of social gameplay, including social equity andreputational dimensions, and translate players’ subjective motivations, expectations, and prediction errors into forwardpredictions of the emotional experiences of the players. Our model infers players’ values and expectations, generatespatterns of play that match observers’ intuitions, and supports formally generated emotion predictions with substantiallyextended breadth and nuance.

The actions of a rational agent reveal information about its mental states. These inferred mental states, particularly the agent’s intentions, play an important role in the evaluation of moral permissibility. While previous computational models have shown that beliefs and desires can be inferred from behavior under the assumption of rational action they have critically lacked a third mental state, intentions. In this work, we develop a novel formalism for intentions and show how they can be inferred as counterfactual contrasts over influence diagrams. This model is used to quantitatively explain judgments about intention and moral permissibility in classic and novel trolley problems.

Successful and repeated cooperation requires fairly sharingthe spoils of joint endeavors. Fair distribution is often doneaccording to preferences for equitable outcomes even thoughstrictly equitable outcomes can lead to inefficient waste. In ad-dition to preferences about the outcome itself, decision makersare also sensitive to the attributions others might make aboutthem as a result of their choice. We develop a novel mathemat-ical model where decision makers turn their capacity to inferlatent desires and beliefs from the behavior of others (theory-of-mind) towards themselves, anticipating the judgments oth-ers will make about them. Using this model we can construct apreference to be seen as impartial and integrate it with prefer-ences for equitable and efficient outcomes. We test this modelin two studies where the anticipated attribution of impartialityis ambiguous: when one agent is more deserving than the otherand when unbiased procedures for distribution are made avail-able. This model explains both participants’ judgments aboutthe partiality of others and their hypothetical decisions. Ourmodel argues that people avoid inequity not only because theyfind it inherently undesirable, they also want to avoid beingjudged as partial.