For the last century, the quality of large-scale assessment in the United States has been undermined by narrow educational theory and hindered by limitations in technology. As a result, poor assessment practices have encouraged low-level instructional practices that disparately affect students from the most disadvantaged communities and schools. In this historical and theoretical review, we examine the misalignment between educational theory and large-scale assessment practices that rely upon technology, using writing assessment as a case in point. Drawing upon sociocultural theory and critical software studies as conceptual frameworks, we find that today’s software-powered technologies, although capable of taking progressive educational ideals to scale, have not been used for these purposes. Our proposed solution is to shift from using technologies to assess predetermined samples of evidence of learning to using technologies to facilitate complex and negotiated models of assessment. This solution would require policy shifts that honor the needs of various stakeholders in the assessment process. We offer a power-sharing concept called negotiated control that engages policymakers, educators, researchers, and community members inthe assessment process.