UC Merced

The remote Objective Structured Clinical Examination (OSCE) is a cornerstone of medical education, enabling structured and objective assessment of clinical skills, communication, and patient-centered care. However, its widespread adoption has introduced challenges related to cost-effectiveness and efficient use of rater resources. Traditional double scoring (DS) ensures reliability but is labor-intensive and costly, especially in large-scale assessments. To address these challenges, this study introduces Targeted Double Scoring (TDS), a novel methodology that selectively applies DS to specific score ranges, particularly those near the pass/fail threshold. The study was conducted using data from a pilot remote OSCE administered to 550 clinical medicine undergraduates in China. The OSCE consisted of three stations: Clinical Reasoning (CR), Physical Examination (PE), and Fundamental Skills (FS). Each station was scored remotely by two raters, with a cut-off score of 60 out of 100. The TDS methodology was modeled based on the OSCEs DS design and fitted with scoring data. A decision-theoretic approach identified optimal Critical Score Ranges (CSRs) for targeted double scoring, balancing reliability and cost-effectiveness. The findings show that TDS significantly reduces rater workload and costs while maintaining high reliability and fairness. For instance, TDS achieved up to 70% cost savings compared to traditional DS under certain configurations. The study also highlights the flexibility of TDS, which can be tailored to different OSCE designs and scoring rubrics. These results have broad implications for medical education, especially in resource-constrained settings where optimizing assessment efficiency is critical. This study provides a practical solution to the cost-related challenges of remote OSCEs and offers a framework for adopting TDS in assessments. By focusing raters on critical score ranges, TDS maintains rigorous and fair evaluations without overburdening faculty or exceeding budgets. Future research should explore TDS scalability and its integration with emerging technologies like artificial intelligence to enhance efficiency and reliability.