UC Berkeley

People often report confidence in two stages: first, they select an answer; second, they report how confident they are that that answer is correct. But research suggests an alternative: one- stage elicitations ask for a belief distribution over the entire set of answers. This dissertation compares these two elicitation methods. In so doing, I shed light on to the psychology of confidence and provide guidance for survey design.I find across seven studies that one-stage elicitation formats lead to higher average peak confidence than two-stage elicitation formats and that difficulty moderates this relationship, such that the effect is stronger when the question posed is difficult. Familiarity with elicitation format does not attenuate the effect. The effect remains stable across different stimuli, and when participant decisions are incentive compatible. In examining why this effect occurs, I find that it is driven by two-stage responses that violate rationality. Across three of the four pre-registered studies, I find that the two-stage responses that indicate less than 50% confidence in the selected answer are driving the main result. When these responses are filtered out, the difference in peak confidence between elicitation formats disappears, as does the moderating effect of question difficulty. Our confidence in our ability to select the right option has enormous consequences. When we are sure, we can commit without hesitation. When we are unsure, then it might be wise to hedge our bets, to delay, or to gather more information. By gaining insights into how people form and communicate confidence in their judgments and identifying effective confidence elicitation techniques, decision-makers can make more informed and effective decisions, leading to better outcomes for organizations and individuals alike.