Whether online exam proctoring will impact test scores is a question that is often raised by test takers, testing organizations, and other stakeholders in the online assessment process. With the sudden surge to virtual learning and working environments due to the pandemic, online proctoring has followed suit and become a common mode of testing, even for high stakes settings such as college admissions, professional certification, and occupational licensing.
The test taker experience is also important to monitor on factors related to the testing environment and system. And of course, the security of the online proctored test should be monitored to identify any breaches that might undermine the validity and fairness of the test, such as cheating, compromise, or collusion.
At PSI, we have been conducting test security analytics for many years to monitor and assure the measurement quality of the high stakes online proctored exams that we administer, in keeping with professional testing standards. Over the years we have developed analytic methods, including a proprietary measure to detect forms of potential cheating.
A study into the effect of online proctoring on online test scores
In 2017, we published the first comparative study of high stakes credentialing exams administered and proctored in a test center vs online proctored exams completed at a remote kiosk. The results were encouraging – strong measurement quality and reliability of scores, comparability in average test scores between modes, favorable test taker experiences, and test scores unrelated to testing experience.
As organizations have quickly adopted online exams, we have additional opportunities to study high stakes credentialing exam programs that have used test centers with onsite proctors and online delivery of the same exams to test takers at home with remote proctors – so-called mixed mode programs. This has provided a unique opportunity to conduct studies examining measurement quality and potential impact associated with online proctoring.
For example, we recently studied six different occupational licensing and professional certification exams that had used onsite proctoring in test centers and remote online proctoring of tests taken at home during the first half of 2020.
The results of the study showed that:
- When we compare online test results with scores in a test center with onsite proctoring, candidates scored the same on average.
- Both online test scores and scores obtained under onsite proctoring conditions were psychometrically sound, with high levels of score precision.
- Candidates rated online exam proctoring favorably, and ratings of testing conditions were unrelated to both proctoring mode and exam performance
- The proctored online exams were secure and did not result in any higher levels of cheating or compromised content than onsite proctoring.
The Future of Online Proctoring
As we look to the future of test delivery, it seems clear that online exams are here to stay. Certainly, it will continue to evolve as new methods and technologies advance, especially with the advent of AI enabled online proctoring and new approaches to secure test design and delivery evolve.
In the meantime, we plan to continue our research to help inform practice and thought leadership in the testing industry. Research will continue to explore the key measurement considerations across test delivery and proctoring modes for different examination programs, remote proctoring systems, and protocols. The data from the large-scale switch to online testing brought about by COVID-19 will enable adding to this body of evidence.
As for now, the current findings lend encouraging support to the use of well-developed and well-administered online proctored exams.
References:
1. AERA, APA, NCME (2014). Standards for Educational and Psychological Testing.
2. Hurtz, G., & Weiner, J. (2019). Analysis of Test-Taker Profiles Across a Suite of Statistical Indices for Detecting the Presence and Impact of Cheating. Journal of Applied Testing technology.
i. Weiner, J., & Hurtz, G. (2017). A comparative study of online remote proctored versus onsite proctored high-stakes exams. Journal of Applied Testing Technology.