Human Proctors Help Overcome the Fear Factor with AI-Enabled Online Proctoring


Human Proctors Help Overcome the Fear Factor with AI-Enabled Online Proctoring

Douglas M. Winneg


Long before Coronavirus, discussions about automation and AI were divided between visions of a catastrophic apocalypse and a future utopia where humans live a life of leisure. The reality is that artificial intelligence has great potential to transform our lives in a very positive way, but only when security, safety, and psychological concerns are all given appropriate consideration.

This is equally true when it comes to the topic of AI enabled proctoring, a discussion which has become increasingly relevant in a world which has seen online exams become a key part of academic and professional life.

The question is, how do we use AI to our benefit in remote proctoring while mitigating the risks? And how do we balance AI proctoring with human intervention, protecting the security and safety of both education providers and students while also considering understandable uncertainties about new ways of working? Remote proctoring software is already being developed and updated with these questions in mind – but could a fully AI-proctored test really be the norm in the future of online learning? We took a closer look at the rise of AI, and the role it can play in proctoring and remote examinations.

Information and Transparency

As human beings, we are wary of what we don’t understand. It’s an evolutionary attribute that has kept us alive for thousands of years, but one that has the potential to hold us back from progress and experiencing new things. To combat this, information and transparency are all vital when introducing anything new, including technology, for the first time.

In recent months, online proctoring that relies entirely on AI has been causing significant anxiety for students who are being asked to take important assessments online rather than in a traditional exam hall. Part of this anxiety comes from the speed of deploying AI based proctoring and online assessments, which has resulted in important communication about how AI is being used being missed or forgotten.

Knowledge is power, and whenever online proctoring uses automated decision-making – whether that’s AI, machine learning, or algorithms – facts and reasons for the approach should be communicated to students.

Communication is critical. At PSI, we always clearly communicate that while we use AI in both our live and record-and-review proctoring solutions, this is only ever in combination with a human proctor. This enables us to utilize the latest technology to enhance security and efficiency for educational establishments while providing the added familiarity of a human being, which reassures students and staff.

Communicating How AI is Being Used in Online Proctored Tests

In addition to providing notice when AI is being used in proctoring, it is even more critical to communicate how the outcomes of any automated decision will be used. With PSI, this might be to flag concerns during the ID authentication process to raise an alert that malpractice may have taken place during a recorded test or to recommend that a live proctored assessment be stopped or paused due to suspicious behavior.

During the check-in process for example, facial recognition algorithms are able to very quickly authenticate the identity of a test taker, comparing a selfie with a selfie from a prior exam or an ID. This minimizes the amount of students’ personal data that needs to be stored and creates an efficient process for the student. While facial recognition software provides a meaningful benefit to the student and the proctoring provider, it is known to occasionally fail to properly match a person’s image. At PSI, when AI flags a potential failed ID authentication, the images and data are provided to a human proctor who is then able to investigate in more detail.

Proctoring during the exam has also made more efficient use of AI. Review of the webcam recording of a student, their desktop, and surroundings by a human proctor can pick up any anomalies in the testing environment. But when combined with AI using pattern recognition, object recognition, and eye movement detection, this increases the likelihood that an unauthorized person entering the room or a student using a mobile phone or any other unauthorized materials will be detected.

Throughout the process, the decision about whether a test should proceed or if the results should be invalidated is never fully left to automated proctoring. Any suspicious activity will be flagged to a human proctor for further investigation. This approach also means that an AI proctoring system can learn the behaviours to look out for and become more effective over time.

Algorithms and AI

With stories in the media about biased AI leading to poor recruitment, legal, or healthcare decisions, there is a growing public awareness around the need for appropriate controls when developing AI algorithms. In many cases, these algorithms are only replicating the often-unconscious biases of the programmers that built them, which can create biases and issues not found in in-person, live proctoring. Where any decision is made as a result of automation – such as to flag potential malpractice in an exam – the algorithm used should always be subject to thorough and ongoing evaluation for fairness and quality.

We take significant steps to minimize the risk of errors and prevent bias and discrimination when developing AI and machine learning algorithms. With PSI’s latest proctoring platform, every time a proctor flags a violation, that data is captured and used to enhance and train our AI and proctoring systems. Similarly, every time AI flags particular activity for review by our proctor, that proctor serves as a check and trainer for existing AI. Important decisions about whether to stop an assessment or invalidate the results of an exam are never taken by AI alone.

Finding a Balance

While it’s important to stay at the forefront of technological developments, both the process and outcome of any exam, test, or assessment can significantly impact a student – and online proctors can play a key role in the experience. The potential to enhance security and safety should always be weighed against the test taker experience, as well as the psychological impact of introducing something new at what is already a stressful time.

In time, AI will become smart enough to make judgments on the severity of its own findings and take independent and appropriate action. However, this level of AI intervention and adoption of AI enabled proctoring software should only be implemented when we are not only sure of the technology but also that user confidence is high enough to accept an AI proctor as a fair, reliable and trustworthy option in online tests.