Skip to main content
eScholarship
Open Access Publications from the University of California

Crowdsourcing Multiverse Analyses to Examine the Robustness of Research Findings

Creative Commons 'BY' version 4.0 license
Abstract

Researchers typically have a fair amount of freedom when it comes to data processing and analysis selection. In many instances, there isn't one correct way to, for example, deal with outliers, which gives rise to a multitude of reasonable analysis pathways, each with its own outcome. Computational advances provide researchers with a unique opportunity to view the impact of such researcher degrees of freedom on the results from a study. Multiverse analyses involve the computational analysis of all these potential pathways, which can demonstrate the robustness of a particular phenomenon, or the lack thereof. However, even though multiverse analyses are less susceptible to biases compared to the typical single-pathway approach, it is still possible to selectively add or omit pathways. To address this, we propose a more principled approach to conducting multiverse analyses, and illustrate how it can be applied using the Semantic Priming Across Many Languages project.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View