This paper describes a randomized experiment conducted in an undergraduate introductory statistics course that investigated the impact of clickers on students. Specifically, the effects of three features of clicker use on engagement and learning were explored. These features included: 1) the number of questions asked during a class period, 2) the way those questions were incorporated into the material, and 3) the grading or monitoring of clicker use. Several hierarchical linear models of both engagement and learning outcomes were fit. Based on these analyses, there was little evidence that clicker use increased students' engagement. There was some evidence, however, that clicker use improved students' learning. Increases in learning seemed to take place when the clicker questions were well incorporated into the material, particularly if the number of questions asked was low.