This is Part I in our blog series on Quality Assurance in Testing
Robust quality assurance will ensure sound test outcomes and prevent any detrimental impact on your test takers, so that every test you deliver continues to be:
- Valid and measures what it is intended to measure.
- Reliable and uses consistent outcomes to award your certification.
- Fair and unbiased, with all test takers treated equally.
A good-quality test allows your test takers to demonstrate their competence in your field and go on to pursue their chosen career or specialty. That’s why quality assurance is critical through the entirety of the assessment lifecycle.
Quality assurance in test content development
A rigorous test development process is essential for high quality test content, with regular reviews to uphold standards.
Quality in item writing and review
Subject Matter Experts (SMEs) should be trained to adhere to specific guidelines that will help them to develop quality test items in line with psychometric best practices. As a further quality check, items should be reviewed by different SMEs to ensure they match the requirements.
Quality in test publishing
Quality assurance in test publishing requires items to be imported and checked for render, display and score accuracy to ensure your test will look and perform as intended.
For example, is the cut score being applied properly? Does the score report contain the correct information? Are the items being scrambled as required?
Ongoing quality checks and test reviews
Quality assurance doesn’t stop when your test is published. Continuous review and maintenance is required including with test taker feedback and statistics.
Regular reviews
Recurring and regular reviews help to ensure your tests reflect current practice and remain valid.
Standardized test center experience
To ensure tests are reliable and fair, you want to maintain the very highest standards in the test centers, including a consistent environment with considerations such as the latest technology and workstations, digital video recordings, secure storage for belongings, observational windows, and quiet and distraction-free testing areas.
The physical space must also be compliant with any accessibility regulations, for example the Disability Act in the US or the Disability Discrimination Act in the UK.
The staff your test takers encounter at a test center can have just as much impact on the quality of their experience as the physical environment.
High quality remote proctors
Proctors are essential for a secure remote testing experience. They are there to safeguard the integrity of your tests. But test day is stressful. You need high quality human proctors who are competent in balancing test security needs with the needs and comfort of your test takers.
Ongoing training and development
Online proctors should be trained to be empathetic and proportionate in how they handle potential misconduct. This involves new proctor onboarding with training in security, privacy and confidentiality.
Proctor training should be followed by periods of job shadowing and supervision by an experienced proctor. As well as regular refresher training to keep skills up to date. For example, PSI research shows most errors occur when a proctor has been in post for less than 30 days, or at a complacency point around 12-16 weeks after initial training. We use focused training modules to address this.
Equally, we know change management requires repeated and consistent reminders. To establish new habits, changes to a testing program generate reminders for proctors every week for three weeks, then at the 30-, 60- and 90-day mark.
All these measures put guard rails in place to ensure every test taker has a high-quality testing experience. Issues raised can then be addressed with additional staff training, reviews of video recordings, or additional site visits by your test center provider. You want to maintain the very highest standards in the tests you deliver, for the benefit of your testing program and your test takers.
Download our new guide Quality Assurance in Testing to learn more.
