A lot has happened in the assessment industry in the past year alone, and we continue to operate in a fast-changing environment. In this blog, our Chief Assessment Officer Isabelle Gonthier offers her perspective on the emerging and growing testing trends for 2024.
AI in test delivery
The use of Artificial Intelligence (AI) came to the fore during the pandemic, when it was used in fully automated proctoring to facilitate a rapid move to secure remote testing. The backlash was understandably strong, and it’s worth restating that PSI has never used fully automated proctoring. We stand by our commitment not to do so, and all decisions are monitored and / or reviewed by trained proctors – a human is always involved.
However, it is our responsibility to explore the potential benefits of AI and other technologies in test delivery for the benefit of our clients. And we fully intend to leverage the technology to start thinking differently and challenge ourselves. While continuing to ensure that any use of AI is always backed by human intervention, with fairness and bias mitigation considered on an ongoing basis.
Any implementation of AI would consider and factor in these key elements:
- Monitoring and analyzing algorithms.
- Ensuring AI models are trained on diverse data.
- Performing regular audits.
- Maintaining transparency about how we use AI in test delivery.
When applied in this way, AI in test delivery is a trend we are fully on board with. And we are continuously reviewing how to implement these new and emerging technologies. For example in identity authentication and verification, spotting anomalies in the background during remote proctored exams, and in post test data analytics.
AI in test development
In just the last year, the use of generative AI tools such as ChatGPT has widened the debate around the use of AI from test delivery to test development. As with test delivery, there are potential threats, but there are also huge opportunities.
For example, AI offers us a tool to look beyond traditional multiple-choice types of questions and explore diverse item types, such as simulations and interactive items. It also helps us leverage the skills and involvement of our valuable Subject Matter Experts (SMEs) in a different way. Introducing efficiencies that not only increase the size and security, but also the quality and size of our item banks.
Again, human involvement is essential. Your SMEs should always have oversight of any AI generated content. As well as quality assurance, this includes regular checks to ensure AI generated items are aligned with your test content specification and don’t contain unintended bias. Full transparency with test takers about how you are using AI in your test development processes is also important.
Read more about how to navigate AI in test development.
Diversity, Equity & Inclusion (DE&I) in testing
The awareness of the importance of DE&I in testing will continue to grow in 2024, and the tools and technologies we have to ensure inclusivity will improve alongside this. There is a shared commitment across the testing industry to make our tests and exams accessible to diverse test takers, including neurodivergent candidates and those with disabilities.
Striving for DE&I in testing is a win-win. Our tests are more accessible for test takers, and our clients can reach a wider pool of test takers to grow and diversify their programs. Testing organizations are seeking to offer tests in alternative formats to accommodate test takers and ensure they have an equal opportunity to demonstrate their skills. Alternative formats such as braille and audio versions are examples of these accommodations.
It would be remiss of me to omit the fact that our test delivery platform is compatible with assistive technologies, such as screen readers and voice recognition. Also, the ability to customize accessibility features to the individual needs of the test taker is improving, whether that’s font size, background color, or contrast.
Test security and data forensics
Strictly speaking, test security isn’t really a trend, as it’s always on the agenda. However, the technologies we have available to improve test security are changing all the time. For example, near to real-time data forensics is something we are developing further and we know our clients are interested in.
The use of data and analysis shortly after a testing session helps us to quickly identify and understand test violations. These might show up in data irregularities or anomalies that are consistent with cheating. We can then work with our clients to respond accordingly, in line with their testing program’s policies.
Near real-time data forensics has multiple benefits. From picking up technical issues and resource allocation in the short term, to supporting continuous improvement in the medium and longer term. Watch this space for updates later in the year.
Test taker voice
The final trend I see coming to the fore in 2024 is the growing importance of the test taker voice. We have more choice than ever before in almost every area of life, and testing is no different. For testing programs to survive and thrive into the future, the test taker experience must be exceptional. And that’s why we’ve recruited a Vice President of Test Taker Experience at PSI.
Our whole team is working together to find ways to improve test taker engagement from start to finish. We’re looking at best practice in other industries and engaging with test takers through focus groups and surveys to capture their views and perspectives. All supported by data and analytics to drive our decisions about how we change and improve.
Embracing opportunity
Over the coming year we will embrace the opportunities that come with new technologies and methods, while ensuring we take a well informed and measured approach to change. We don’t know what the next 12 months will bring, but it’s certainly an exciting time to be working in the assessment industry.
The PSI team will be attending and presenting at the Association of Test Publishers (ATP) Conference in March 2024. I’m looking forward to being involved in sessions on AI, web crawling and data forensics, and assessment essentials.
Check out the full program of PSI sessions on the ATP Conference 2024.
