AI systems that dynamically navigate the human world will sometimes need to predict and produce human-like moral judgments. This task requires integrating complex information about human moral cognition (what decision would humans make in this situation?), normative ethics (what is the right decision for an AI to make?), and artificial intelligence engineering (how can we implement this functionality in AI systems?). A range of solutions have begun to emerge within the cognitive science community to satisfy these three categories of demands. However, most solutions tend to satisfy some demands, while falling short on others. This symposium highlights four competing solutions for building AI with a human-like moral sense, with the goal of highlighting the strengths and weaknesses of each approach and how each might complement the others in development and deployment going forward.

Rules undoubtedly guide our moral lives. Simple moral rules prohibit lying, cheating, and stealing, for instance. But the moral mind is more flexible than a theory based only on rule-adherence can account for. In this paper, we look at one particular kind of flexibility: the ability to figure out when it is okay to break a moral rule. We elicit judgments of the moral acceptability of breaking a simple rule: it's wrong to cut in line. We created a video game environment in which agents attempt to gather water -- and sometimes must stand in line behind others to do so. Subjects watch clips of the game being played and make judgments of the moral acceptability of cutting in line across a wide range of spatio-temporally varied and dynamic scenarios. Our data suggests that subjects make judgments by using a generative understanding of the underlying function of the rule about waiting in line. We further show that our data cannot be accounted for by either 1) simple rule adherence or 2) utility maximization.

There has been exciting recent progress in computational modeling of moral cognition. Work in this area tends to describe the cognitive mechanisms of human moral judgment using symbolic models, which are interpretable and written in terms of representations that carry meaning. However, these models fall short of capturing the full human capacity to make moral judgments in that they fail to process natural language inputs but instead rely on formal problem specifications. The inability to interface with natural language also limits the usefulness of symbolic models. Meanwhile, there have been steady advances in conversational AI systems built using large language models (LLMs) that interface with natural language. However, these systems fall short as models of human reasoning, particularly in the domain of morality. In this paper we explore the possibility of building neuro-symbolic models of human moral cognition that use the strengths of LLMs to interface with natural language (specifically, to extract morally relevant features from it) and the strengths of symbolic approaches to reason over representations. Our goal is to construct a model of human moral cognition that interfaces with natural language, predicts human moral judgment with high accuracy, and does so in a way that is transparent and interpretable.

We are constantly faced with the question of how to aggregate preferences, views, perspectives and values. This is a problem for groups attempting to accommodate individuals with differing needs and interests, as will be our focus. The problem of ``value aggregation'' therefore crops up in myriads of places across the social sciences---in rational decision theory, social choice models, and proposals for systems of democratic voting, for instance. These sub-disciplines have formalized proposals for how to deal with value aggregation, though, remarkably, no research has yet directly compared people's intuitions of two of the most obvious candidates for aggregation--taking the sum of all the values (the classic ``Utilitarian'' approach) and the product (a less well-known ``contractualist'' approach). In this paper, we systematically explore the proposals suggested by each algorithm, focusing on aggregating preferences across groups. We find that both humans and performant LLMs prefer a contractualist approach.

Recent years have witnessed a burst of progress on building formal models of moral decision-making. In psychology, neuroscience and philosophy, the goal has been to “reverse-engineer” the principles of human morality. Meanwhile, in AI ethics, the goal has been to engineer systems that can make moral decisions, in some ways inspired by how humans do this. We aim to showcase the state of the art in both fields and to show how they can be hybridized into a computational cognitive science of morality.

How do we know when it's OK to break moral rules? We propose that — alongside well-studied outcome-based measures of welfare and harm — people sometimes use universalization, asking "What if everyone felt at liberty to ignore the rule?'' We develop a virtual environment where agents stand in line to gather water. Subjects judge agents who get out of line to try to get water more quickly. If subjects use universalization, they would need to imagine all agents getting out of line and going straight for the water in each environment. To test this prediction, we model an action's universalizability by simulating what would happen if every agent tried to follow a path directly to the water, then evaluating the effects. We also investigate the role of several outcome-based measures, including welfare aggregation and harm-based measures. We find that universalizability plays a important role in rule-breaking judgments alongside outcome-based concerns.

There is wide agreement that the mind has different mechanisms it can use to make moral judgments. But how does it decide which one to use when? Recent theoretical work has suggested that people select mechanisms of moral judgment in a way that is resource-rational --- that is, by rationally trading off effort against utility. For instance, people may follow general rules in low-stakes situations, but engage more computationally intensive mechanisms such as consequentialist or contractualist reasoning when the stakes are high. Here, we evaluate whether humans and large language models (LLMs) exhibit resource-rational moral reasoning in two moral dilemmas by manipulating the stakes of each scenario. As predicted, we found that the higher the stakes, the more people employed a more effortful mechanism over following a general rule. However, there was mixed evidence for similar resource-rational moral reasoning in the LLMs. Our results provide evidence that people's moral judgments reflect resource-rational cognitive constraints, and they highlight the opportunities for developing AI systems better aligned with human moral values.

The actions of a rational agent reveal information about its mental states. These inferred mental states, particularly the agent’s intentions, play an important role in the evaluation of moral permissibility. While previous computational models have shown that beliefs and desires can be inferred from behavior under the assumption of rational action they have critically lacked a third mental state, intentions. In this work, we develop a novel formalism for intentions and show how they can be inferred as counterfactual contrasts over influence diagrams. This model is used to quantitatively explain judgments about intention and moral permissibility in classic and novel trolley problems.

Reducing inequality, mitigating climate change, and responding to public health crises are large-scale goals that require the cooperation and coordination of many individuals. These goals cannot be achieved by one individual alone, and contributing is not always beneficial to each individual. And yet, individuals must contribute in order to make a difference. How do we hold individuals and groups responsible for collective action?

We describe a cognitive mechanism of moral judgment, universalization, that has received little attention up to now. Underuniversalization, an action’s moral permissibility is determined by calculating what the outcome would be if all people whoare similarly situated to the actor also acted in that way. This mechanism is particularly well-suited to capture our moraljudgments of free-rider cases, where one person doing the action increases utility but many people doing it decreasesutility. Universalization fits into an agreement-based (contractualist) theory of moral cognition, and explains properties ofour moral judgments that an outcome-based or rule-based approach cannot. We show patterns of universalization reasoningin young children as well as adults.