Out of absolute necessity, many educational institutions rapidly implemented remote testing with online proctoring during 2020 and 2021. And the move was well-intentioned – to provide continuity for students and avoid gaps in their learning journey during an already difficult time. There is a great deal to celebrate in the hard work and achievement of moving so many tests online while maintaining the security, fairness, and integrity of tests and qualifications.
However, it’s also important to recognize that this transition came with some challenges. For example, there was student backlash against relying solely on Artificial Intelligence (AI) to monitor for potential dishonesty during online proctoring. There were understandable questions and concerns from students about their data and personal privacy, often in response to a lack of understanding and communication about online proctoring. There have even been inquiries from top levels of government globally about the validity and equity of AI-based proctoring.
While we have some well-deserved down time over the holidays, it’s a great opportunity to review the user experience and communication around moving exams online. That’s why we’ve put together a checklist to help you consider what went well and what maybe didn’t go so well with your testing program in 2021. Use this time to think through how you can apply these lessons to be in the best possible shape for 2022.
1. Get student feedback
If you used online proctoring this past semester, try to find out the students’ point of view and gather direct feedback on their experience and feelings. If you already included a survey of student views about their online proctoring experience at the end of their test, take the time to collate and properly review the results. If not, a quick survey would give you some good insights.
Be prepared – all the responses you receive are unlikely to be positive. It’s only to be expected that the introduction of any change or new technology will come with a level of discomfort and uncertainty. Take the good with the bad and remember, any feedback helps you make better decisions for the future around how and when to utilize online proctoring services. The next step is to develop an action plan to address the areas for improvement, issues, or queries highlighted by the survey.
Read Are You Asking All the Right Questions Before Implementing Online Proctoring? for further information
2. Review your test data
All data is valuable information that you should use to your advantage. After compiling and reviewing direct student feedback, look at the data you have related to test scores and results. Did the results from the most recent testing period raise any concerns about the overall integrity of a test? Have scores changed over time and were recent marks in line with your expectations? Was there any self-reporting of misconduct? Did your proctoring service flag an unexpectedly large number of students for major violations? This level of review and questioning will help you identify any issues and address them as soon as possible.
An issue might be related to student misconduct, but it could also be related to the online proctoring service you use. For example, is your service automated with AI generated flags? Are human proctors always involved? If it is AI dependent, does the suspicion level cause concern?
It’s important to verify flags so that you don’t unfairly accuse students of cheating due to an unnecessary flag. But if you do need to spend extra time watching videos because of the high rate of flagging, it’s worth considering whether an AI dependent service is as efficient and accurate as it initially seemed. Again, if a review of your data does raise a concern, the next step is to investigate and find out exactly why.
PSI Re-Affirms Commitment to Human-Focused, Technology Assisted Proctoring
3. Evaluate vendor service levels
Take the time to reflect on the past year and the service you received from your online proctoring vendor. How did you find the level of professional support you received? Were you able to get support quickly when needed? And when you did get a response, was it effective and timely in resolving your query or problem? Maybe take a moment to do some research on how other institutions and students are rating their experience with the vendor you use. A quick online search or scan through user reviews can shed some light on the issue.
If you don’t currently log, track, and continuously evaluate the level of service your vendor delivers, it’s worth looking at accurate ways to monitor service levels. This isn’t only useful to ensure that you receive a high level of service from your vendor, it’s also invaluable information to access and use in the future as a record of frequently asked questions. This ensures you are learning from experience and improving all the time.
4. Review technical support needs
One of the areas to examine in student surveys and feedback is whether your students feel they would benefit from additional technical support. This is a vital element of a successful online proctoring program, as some of the most common issues experienced by test takers are related to technical issues such as unstable internet connectivity, lack of bandwidth, and out of date operating systems or unsupported devices. Improved communication combined with excellent technical support will minimize the impact of these issues.
How you log and track student support issues is just as important as how you monitor vendor service levels. And a critical element of this is how your vendor performed when students are at their most stressed times – attempting high-stakes exams. When experiences can be shared on social media and in the public domain almost immediately, how well supported your students feel with technical issues when taking important online tests will have a huge impact on the reputation of your institution.
Ready for 2022 and beyond
The move to online testing gives us access to a wealth of data. Student feedback, test scores, vendor service levels, peer reviews, and technical support delivery – it’s all at our fingertips. By taking the time now to review information on past performance, identify any concerns, and start to develop a plan of action, you’ll continually improve how you implement online proctoring and start 2022 ahead of the curve.
