Investigating the Potential Effect of the Proctoring Setting on Examinee Performance During COVID-19

For the latest research update, please click here.

NBME psychometricians analyzed data to investigate the potential effect of the proctoring setting on examinee performance during the COVID-19 pandemic. Principal psychometrician, Carol Morrison, PhD, compared results from examinees who took Clinical Science Subject Exams during the period when web-conferencing with remote proctoring was the only option available (April 7 to May 21, 2020) to performance of examinees who tested under typical in-person proctoring in the preceding six-week period, as well as examinees who tested between April 7 and May 21, 2019.

Results from Medicine, Surgery, Obstetrics and Gynecology, Pediatrics, Psychiatry, Clinical Neurology, Family Medicine, and Emergency Medicine Subject Exams were analyzed with a focus on examinees from LCME-accredited schools testing for the first time for end-of-course or end-of-clerkship with standard timing.

No consistent score differences were associated with proctoring setting. Groups of examinees who tested remotely had slightly higher scores for some exams and slightly lower scores for others. Additionally, effect sizes for proctoring setting were generally very small, indicating no meaningful difference in performance of the remote proctoring group and the comparison groups from six weeks prior and same time period last year.

To support the needs of medical schools, we will continue to analyze data to learn more about the potential effects of remote administration and proctoring on scores and testing experiences. We will share information with the medical education community as findings are available.