The testing industry is changing rapidly and it can be challenging to keep up to date with the latest developments and trends. The ETS Future of Assessments Report is a valuable resource for testing organizations, exploring the state of assessments today and what the future might hold for our industry.
Following our previous blog, a summary of the ETS Human Progress Report, this blog summarizes the ETS Future of Assessments Report. Read on to find out more about the current challenges we face, as well as where the future opportunities and prospects lie.
Continuous lifelong learning
There are various incentives for an individual to take a test, other than school admissions or employment screening, which are typically compulsory. The Future of Assessment Report considers the findings of an ETS survey of 17,000 people across 17 countries, and it reveals continuous learning as the top reason for testing:
- Continuous skills improvement (39%)
- Identify current skill levels and strengths (35%)
- Personalize your learning journey (34%)
Another headline finding shared in the report is that a significant majority (78%) of people believe proof of specific skills will be more important than a university degree in the future. In addition, 81% of respondents believe micro-credentials (short term, focused certifications) will become a valuable way to showcase skills.
It seems continuous lifelong learning is becoming the norm. And the ETS survey indicates that people also see the benefits of learning assessments, not just for better job opportunities and career advancement (85%) but also boosting self-esteem (84%) and career satisfaction (84%).
This recognition and requirement for lifelong learning is significant for testing organizations. As the Report highlights, individuals are actively seeking learning opportunities and employers will continue to invest in the ongoing professional development of their people. Positivity around the value of assessments was particularly true for younger respondents (Gen Z and Millennials), so the opportunities this presents for credentialing will likely continue.
So, what are the key areas for testing organizations to be aware of and act on? The Report offers useful insights for testing organizations on four topics: analysis and identification of future skills, innovative approaches to measuring those skills, innovating in testing operations, and giving test takers feedback.
1. Analysis and identification of key skills
To provide useful information to test takers, it is necessary to identify the most important skills needed for attaining their education and career goals. The Report states: “We believe that the future of assessments will largely be driven by the skills needed to be a productive, informed citizen, to maintain health and well-being, and to be a contributing member of the community and society.” In summary, educators and employers will want to know about the skills of students, applicants, and incumbents. So they are then able to make good decisions in admissions, hiring, promotion, and student and workforce development. This has been the role for assessments and will continue to be. But the most highly sought after skills today are not the traditional curricular skills that have been the focus of assessment attention in the past. They are the hard-to-measure skills – which presents an assessment challenge. The National Association of Colleges and Employers (NACE) has been conducting annual surveys of US employers for over a decade. They found the percentage of employers who screen candidates by grade-point average dropped from 73% to 37% in the past four years, perhaps indicating the increased attention on what are sometimes referred to as durable or soft skills.
This hypothesis is supported by a World Economic Forum (WEF) survey in 2023, where employers were asked about the future. Analytic thinking and creative thinking were the top skills, but curiosity and lifelong learning, plus technology literacy, increased substantially to join resilience, flexibility, and agility as the predicted top five skill clusters five years into the future.
2. Advancing methods for assessing hard-to-measure skills
The skills in demand are hard-to-measure, and measurement of these skills has often been poor quality. Assessments have been almost exclusively self-report, with the psychometric qualities of these assessments — reliability, validity, and fairness — rarely mentioned. This presents a clear opportunity for the future of assessments. The need for soft skills, combined with the challenges in measuring them, is significant for the testing industry. Especially when the LinkedIn Talent Solutions (2019) Global Talent Trends report found 91% of talent managers believe soft skills are important to the future of recruiting and 92% believe soft skills matter as much or more than hard skills. However, 57% struggle to assess soft skills accurately. This situation presents a challenge and an opportunity for assessment. The challenge is that the simple self-assessment ratings we rely on today are not sufficient for the task of providing useful information about the skills of the future. The opportunity is that new innovative assessment methods can be developed, so we can assess hard-to-measure constructs with the same level of sophistication as we measure mathematics, reading and science today.
The Report suggests self-reporting and reporting by others will remain in use but is associated with well-documented biases. To supplement or replace these measures, there is a need to develop engaging, personalized, and contextualized performance tasks. These could include games, simulations, and interactive and collaborative tasks, which do not depend on subjective ratings.
3. AI and technology-enabled advance
The Report believes the future of assessments will continue and perhaps accelerate the transition of test development from an art to a science, using advances in technology, operations research methods, and both predictive and generative artificial intelligence (AI) methods. It explores current cutting-edge advances across the assessment lifecycle to provide a basis for where we might see significant breakthroughs. In summary:
Anytime, anywhere, with security
Test centers still exist and they are still popular. For some people in some circumstances, it is more convenient and less costly to take a test in person and that might be true for a long time.
In many cases, employers, higher education, and associations still require in-person testing for a variety of reasons. This can include test security, adding more elements to assessments, the test taker experience, and fairness of access to digital connectivity and high bandwidth. The future is likely to be a mix of increased remote testing with online proctoring and even potentially mobile convenience, with the continued availability of test centers.
Security is more challenging with remote or mobile testing when the testing purpose is high stakes. However, research shows the convenience of taking a test on a mobile device without time restrictions led to no differences in scores for anytime-anywhere testing compared to test center locations, but significant advantages for mobile testing in engagement.
New test and item formats
Some tests are still paper based, but this is increasingly rare. And while computer-based testing did not initially add much functionality to the paper-and-pencil format, other than allowing many more test forms and enabling adaptiveness, new capabilities have been introduced. For example, videos, simulations, and interactivity.
These trends will continue, and increasingly more engaging and immersive formats will be possible. New technologies, such as mixed reality headsets, expand the possibilities. Both for how the inputs for testing (instructions, item stimuli, item prompts) and the responses to test items (gestures, grasping, whole-body movements) can be realized, allowing the testing of new constructs in new ways.
LLMs in item development
Item development has traditionally relied on human experts and has been an expensive and time-consuming process. Now, a successful combination of careful item modelling and capable large language models (LLMs) appears a highly promising approach towards implementing automation into item development.
Of course, draft items need human oversight to review for their accuracy, appropriateness, and fairness. Then items must be calibrated to estimate their difficulty and discrimination, and assembled into a test form before they reach test takers. So it is critical that we innovate the overall item development process alongside initial generation capability.
LLMs also hold the potential to accomplish personalization and contextualization, alongside economies in test content generation to a degree not possible before. And there is positivity around this potential. A high percentage (78%) of respondents in the ETS survey agreed, “AI has the potential to enhance learning assessments by tailoring them specifically to each individual learner’s needs.”
However, to properly implement such an approach, the potential consequences of AI need to be considered carefully. The survey also revealed 71% of respondents are worried, “AI has the potential to negatively impact learning assessments due to unintentional biases and programming flaws within the system.”
We must proceed with care when it comes to AI and build a robust mechanism to monitor and control output. Ensuring the benefits can be realized without reproducing pre-existing biases.
Using AI to detect LLM cheating
The use of ChatGPT and other LLMs on various kinds of high-stakes tests presents new challenges in detecting cheating. The Report explores a variety of prevention measures, such as:
- The use of additional cameras.
- Test redesign to include items less susceptible to LLM assistance, such as critical thinking and performance-based tasks.
- Measures designed to detect ChatGPT contributions to the response, particularly essay responses.
AI scoring methods
The Report covers several scoring topics likely to emerge as important to the future of assessments. These are scoring automatically generated items, scoring essays using AI methods, and scoring new novel item types and test-free assessments.
Deep learning models and LLMs are beginning to be used in essay scoring and in the evaluation of other hard-to-score tasks. They can increase accuracy and provide better explanations to test takers about the strengths and weaknesses of their assessment outcomes.
4. Giving test takers personalized feedback
While ETS survey respondents see the benefits of assessment, the Report also highlights the importance of providing value back to the test taker and stakeholders. A concept described as a Return on Test (ROT) that justifies the expense.
Historically, tests have often failed to provide actionable feedback that might help determine next steps towards education or career goals. But 87% of ETS survey respondents agree, “…learning assessments should provide ongoing feedback, not just a one-time snapshot of performance.” In addition, 68% of respondents said they would be motivated to acquire new skills and knowledge, “if you were able to take skills assessments and receive guidance as a pathway for career growth.”
In some contexts, test scores are returned with norms, benchmarks, and descriptive information that help test takers interpret their scores. But more could be done to provide useful feedback that might help test takers advance towards their goals. Future digital assessment design should consider methods for providing personalized feedback that caters to each learner’s strengths and weaknesses.
The Report authors see a future of assessments that will largely be concerned with providing useful information, especially to test takers, to change the cost-benefit ratio of testing. Tests will shift from assessing what an individual knows now to what they can do with the information provided by the test.
One of the main challenges in testing lies in time. Individuals do not want to sit through long tests and test owners do not want to administer or pay for them. The future lies in tests that can be made more efficient through traditional psychometric approaches and making the experience more useful for the test taker in benefits returned.
Formative assessments, intelligent tutoring systems, and other forms of instruction mixed with assessment take advantage of this principle. This makes the test taker experience more compelling or enjoyable, so test takers willingly allow themselves to spend more time being tested. Multimedia tests, game-based and gamified assessments also have a role to play.
Future of Assessment Report conclusions
The Report concludes that advances in technology, particularly AI, will have profound effects on all aspects of assessment. A core set of soft skills, durable skills, and complex skills are likely to become increasingly important in the future. And alongside this there will be a system in place to assess and recognize skill development.
There is good reason for optimism about the future of testing given attitudes towards assessments are largely positive. And it is positive news for credentialing that a majority of ETS survey respondents around the world believe:
- Non-degree credentials will become a valuable way to showcase skills.
- Proof of specific skills will become more important than a university degree.
Our industry has a vital role to play in the future of human progress. But the Report authors believe this is contingent on the feedback and insights about themselves that test takers gain from an assessment, “Providing personalized, useful, actionable feedback to test takers is an important and achievable goal for future assessment.”