Artificial Intelligence (AI) can be harnessed to create sophisticated social and moral scoring systems – enabling people and organisations to form judgements of others at scale. While this capability has many useful applications – e.g., matching romantic partners who are aligned in their moral principles, it also raises many ethical questions. For example, there is widespread concern about the use of social credit systems in the political domain. In this project, we approach this topic from a psychological perspective. With experimental evidence, we show that the acceptability of moral scoring by AI depends on its perceived accuracy, and that perceived accuracy is compromised by people's tendency to see themselves as morally peculiar, and thus less characterizable by AI. That is, we suggest that people overestimate the peculiarity of their moral profile, believe that AI will neglect this peculiarity, and resist for this reason the introduction of moral scoring by AI.

Autonomous vehicles (AVs) should reduce traffic accidents, but they will sometimes have to choose between two evils, such as running over pedestrians or sacrificing themselves and their passenger to save the pedestrians. Defining the algorithms that will help AVs make these moral decisions is a formidable challenge. We found that participants in six Amazon Mechanical Turk studies approved of utilitarian AVs (that is, AVs that sacrifice their passengers for the greater good) and would like others to buy them, but they would themselves prefer to ride in AVs that protect their passengers at all costs. The study participants disapprove of enforcing utilitarian regulations for AVs and would be less willing to buy such an AV. Accordingly, regulating for utilitarian algorithms may paradoxically increase casualties by postponing the adoption of a safer technology.

Recent years have witnessed a burst of progress on building formal models of moral decision-making. In psychology, neuroscience and philosophy, the goal has been to “reverse-engineer” the principles of human morality. Meanwhile, in AI ethics, the goal has been to engineer systems that can make moral decisions, in some ways inspired by how humans do this. We aim to showcase the state of the art in both fields and to show how they can be hybridized into a computational cognitive science of morality.

Centralized sanctioning institutions have been shown to emerge naturally through social learning, displace all other forms of punishment and lead to stable cooperation. However, this result provokes a number of questions. If centralized sanctioning is so successful, then why do many highly authoritarian states suffer from low levels of cooperation? Why do states with high levels of public good provision tend to rely more on citizen-driven peer punishment? Here, we consider how corruption influences the evolution of cooperation and punishment. Our model shows that the effectiveness of centralized punishment in promoting cooperation breaks down when some actors in the model are allowed to bribe centralized authorities. Counterintuitively, a weaker centralized authority is actually more effective because it allows peer punishment to restore cooperation in the presence of corruption. Our results provide an evolutionary rationale for why public goods provision rarely flourishes in polities that rely only on strong centralized institutions. Instead, cooperation requires both decentralized and centralized enforcement. These results help to explain why citizen participation is a fundamental necessity for policing the commons.

Discrete choice models (e.g. logistic regression) are popular models in the economics literature that describe choices between twoor more discrete alternatives. These models have been successfully used to model value-based decisions, e.g. decisions in moraldilemmas, although temporal components of a decision, such as reaction times and changes of mind are not included. In cognitivesciences, another class of decision models, namely sequential-sampling models, has gained popularity in modelling choice accuracy,reaction time and decision uncertainty (e.g. confidence judgments). Here, we model decisions in moral dilemmas using a variant ofa hierarchical drift-diffusion model, factor drift diffusion, that combines the value-based approach with that of evidence accumulationmechanism by sequential-sampling. Specifically, we model the evidence accumulation process as resulting from a subjective weightingof abstract moral dimensions (factors). We train our model on a data set of 6500 moral decisions by 500 respondents on a popularweb platform (MoralMachine.mit.edu) and separately infer different sources of uncertainty in moral decisions. We show that the modelsuccessfully predicts reaction times and choices in moral dilemmas, while also leading to unexpected results