UC Riverside

The purpose of this project is to understand how the organization of a student's solution to a problem relates to the correctness of that work. Understanding this relationship will enable software to provide early warnings and targeted feedback to students who are struggling in a course. In this study, students in an undergraduate statics course completed their work, including homework, quizzes, and exams, using Livescribe^{TM Smartpens. These devices record the handwritten solutions as time-stamped pen strokes, enabling the examination of not only the final ink on the page, but also the order in which it was written. This unique database of student work was used to examine how the history of the solution construction process correlates with the correctness of the work. Solution histories were characterized by a number of quantitative features describing the temporal and spatial organization of the work. For example, some features describe the order in which various problem-solving activities, such as the construction of free body diagrams and equilibrium equations, are performed and others describe the amount of time spent on each activity. The spatial organization of the work is characterized by the extent to which a student revisits earlier parts of a solution to revise his/her work. Cross-validated regression models were constructed using the relaxed lasso method to determine the correlation between these features and student performance. On average, the models explained 43% of the variance in performance. This is a surprising result in that the features do not actually consider the semantic content of the writing. The relaxed lasso method also identified which features were most predictive of problem correctness, thus giving insights into which student behaviors are indicative of high or low performance. For example, revising work long after it was written indicates low performance. While our work has focused on engineering statics, we expect that these techniques will generalize to other domains for which problem solutions include both diagrams and equations.}