The CATESOL Journal

This replication study investigates how the level of context in vocabulary assessment affects the scores on tests of American idioms. Using Uçkun’s methodology of 3 tests with 3 levels of context, 85 participants varying in level from high-beginner to advanced took an online test consisting of 30 questions, 10 questions for each level of context. The tests were matching, sentence-level gap filling, and rational-delete cloze. The participants were nonnative speakers of English living in the US. The scores were analyzed for mean differences and in regard to differences in native language and English proficiency level. No significant differences were seen for native language, but there were significant differences for context level and proficiency level. As an exploratory part of the study, 17 of the participants were asked to perform a think-aloud protocol task while taking the test. Their responses were recorded and analyzed descriptively for insight into test-taking strategies.