Challenging the WEIRD Context in Assessment with Steve Dept
In this episode of Tried and Tested, host Isabelle Gonthier, PhD, ICE-CCP speaks with Steve Dept, Founding Partner and Strategic Partnerships Director at cApStAn, about the intersection of artificial intelligence, culture, and global assessment. Returning to the podcast three years after his first appearance, Steve draws on decades of experience in multilingual assessment, translation, and cultural adaptation to explore why developing fair and valid assessments across languages and cultures requires much more than accurate translation.
As AI becomes increasingly integrated into assessment development and adaptation, Steve explains why cultural context, human expertise, and a critical approach to technology are essential to maintaining assessment quality. The conversation explores the influence of Western perspectives on AI models, the concept of WEIRD (Western, Educated, Industrialized, Rich, and Democratic) contexts, the challenges of maintaining comparability across cultures, and why translation and adaptation must be considered from the earliest stages of test design.
What you’ll learn:
- Why accurate translation alone does not guarantee fair or comparable assessments across cultures
- How cultural context can influence the way test takers understand and respond to assessment questions
- What WEIRD (Western, Educated, Industrialized, Rich, and Democratic) means, and why this framework matters when evaluating cultural assumptions in AI and assessment
- Why AI-generated content requires expert review to maintain validity, reliability, and fairness
- How differences in language, educational experiences, and cultural expectations can affect assessment outcomes
- Why efficiency should not be the only consideration when integrating AI into assessment development
- How early linguistic and cultural reviews can identify potential issues before assessments are translated
- Why translation, adaptation, and multidisciplinary expertise should be embedded throughout the test development process
Who should listen:
- Assessment and credentialing leaders developing or expanding international testing programs
- Psychometricians and test developers focused on validity, fairness, and cross-cultural comparability
- Organizations translating or adapting assessments for multilingual audiences
- Language assessment and localization professionals working with high-stakes testing
- Education and certification leaders exploring AI-supported assessment development
- Researchers and measurement professionals studying cultural bias and assessment equity
- Teams balancing AI-driven efficiency with the need for human expertise and quality assurance
Steve Dept
Founding Partner and Strategic Partnerships Director at cApStAn
Steve Dept is a founding partner of cApStAn, a niche language solutions integrator for multilingual assessments and surveys: He grew up in South Africa, Belgium and Germany and received his education in English, Dutch, French and German. He studied Classical philology and comparative linguistics but is essentially an autodidact and a field practitioner. He is known for ensuring cross-language comparability in major International Large-Scale Assessments (ILSAs) such as OECD/PISA, OECD/PIAAC, IEA/PIRLS, IEA/TIMSS, CONFEMEN/PASEC and UNICEF/SEA-PLM. Steve has coordinated linguistic quality assurance in over 90 multilingual surveys and he acts as a translation/ adaptation consultant for data collection instruments in both the public and the private sector, with a special focus on hybrid workflows involving AI and human experts. Over the years, Steve’s attention has shifted towards optimising the master versions by embedding translation and adaptation in the test development process. His translatability assessment methodology has become an integral part of the adaptation design of all major ILSAs and over forty multilingual surveys. Steve is a frequent speaker at international conferences.