Oral reading fluency (ORF) as an indicator of reading achievement was examined for a sample of third and fifth grade English learners (ELs). The impact of English language proficiency on the relationship between ORF and reading comprehension, and the prevalence of word callers, or students who are fluent readers, but do not comprehend at a proportionate level, was examined using a series of regression and predictive accuracy analyses. Additionally, teacher judgments of participants' reading achievement were explored with a focus on the accuracy of teachers' word caller nominations in their classrooms. Results showed that word callers emerged, though infrequently and that there were inaccuracies associated with teachers' judgments of reading skills.