PSI is pleased to announce the appointment of Lisa O’Leary, in a newly formed role as Vice President of Assessment Services, to lead PSI’s Content Science division under Chief Assessment Officer, Isabelle Gonthier.
Can you share with us your career journey in the testing industry – how did you get into testing? What initially drew you to the credentialing world?
My career journey in the testing industry began when I started a master’s program in higher education, where I developed a deep interest in statistics. This interest transferred my focus to educational measurement, culminating in a PhD at Boston College in Educational Research, Measurement and Evaluation. I initially worked on student outcomes at MIT, which provided a solid foundation in educational assessment.
I transitioned to a psychometrician role at Alpine Testing; this was the turning point that introduced me to the world of testing through educational assessment and I haven’t looked back. Over the past 20 years, I have focused extensively on assessment and measurement, continuously evolving my expertise in this domain. My career path has been driven by a passion for data, educational psychology, and the impact of assessment on learning and professional development.
Tell us a bit more about your new role at PSI.
In my new role at PSI as Vice President of Assessment Services, I will be leading the Content Science team which includes an experienced team of over 100 test developers, psychometricians, analysts, and test publishers/specialists across all markets, including certification, licensure, federal, English language assessment, admissions, and more. It’s an honor to be leading a team of celebrated industry experts who are deeply involved in the credentialing world. PSI is well known for its leading thought leadership which can be seen with several industry chair positions and active participation in committees for organizations such as ATP, ICE, and CNAR.
The Content Science team at PSI boasts a strength of leadership and depth of talent unrivaled in the industry which will allow me to further effect change on the amazing work they are doing. I am particularly excited by PSI’s unique Data Forensics offering and helping to evolve this further in real-time analysis.
Together with the rest of PSI’s leadership team, we will continue to innovate and enhance our offerings to meet the evolving needs of our clients and their test takers.
What drives you in your work – what is your ‘why’?
My primary motivation is the opportunity to bridge gaps and contribute to assessment solution design. I am passionate about merging the science of measurement with product and technology, particularly in the commercialization of credentialing. My goal is to creatively solve pain points by thinking outside the box, ensuring that our services meet the diverse needs of our clients and programs. I find immense satisfaction in bringing together various elements of science, technology, and practical application to create robust credentialing solutions.
From your perspective, what are the biggest challenges currently facing the credentialing industry, and how can PSI innovate to address these challenges?
The credentialing industry faces several challenges, including the need to stay relevant in a rapidly changing learning landscape. One significant shift is towards individual learning journeys instead of a one-size-fits-all approach to education and professional advancement. The concept of the lifelong learner is not new, but with further advances in technology encouraging shifting career trajectories, testing organizations like PSI, need to keep pace with this and become even more adaptable. This is supported by ETS’s latest research, the Human Progress Report, shows that 88% of global respondents feel continuous learning is essential to succeed in today’s society; this is exacerbated by pessimism about the current state of education.
Additionally, the influx of new technologies necessitates a rethinking of expectations for professional job roles, with a greater emphasis on durable skills such as collaboration, creativity, and communication. To remain effective, we must develop new ways to measure these critical skills, especially as the traditional baseline knowledge components are increasingly being automated by technology. PSI is uniquely positioned to address these challenges due to our solid foundation of proprietary technology and the innovative potential brought by our merger with ETS. By leveraging cutting-edge research and AI technology, we can integrate educational assessment with professional testing to stay ahead.
Equally, what are the biggest opportunities?
One of the biggest opportunities is to modernize the credentialing space across the ecosystem. This includes seamless transitions in the test taker experience through to on-demand technical reporting and leveraging technology to improve systems, automate processes, and enhance our overall output. Technology can provide us with more comprehensive viewpoints that can be embedded by humans, resulting in higher fidelity of the assessments we produce. This reduces error and bias, benefiting all stakeholders, including test takers, clients, and our internal teams.
We also have the opportunity to rethink the full assessment lifecycle, starting with role delineation and job analysis. By harvesting information from various sources, such as job sites and industry conference programs, we can create more relevant and effective assessments that reduce the sole reliance on subject matter experts (SMEs). We can further automate both item generation and review to streamline SME involvement to enhance and refine technology-enabled output, rather than serve as the primary source for content generation. We can think creatively about forms assembly and standard setting to efficiently and reliably release content appropriately aligned with the intended uses and interpretations of our assessments.
As we look forward to the next five years, what are your key goals for PSI’s Content Science division?
My key goals for the Content Science division include:
- Thinking outside the box to creatively deliver on client needs.
- Ensuring the most secure assessments are delivered on time without compromising the test taker experience. PSI is in a unique position in the market to leverage the innovation capability of ETS, coupled with PSI’s extensive knowledge and data of the credentialing market, to deliver the very best in assessment design and delivery. The focus should always be to reduce timelines while maintaining high rigor with the support of technological assistance.
- Using the right tools at the right time to enhance our assessment capabilities, the chance to further integrate AI capability into our solutions is a huge opportunity for PSI and one that is advancing every day.
By focusing on these goals, we can continue to provide high-quality, innovative assessment solutions that meet the evolving needs of our clients and the industry.
Finally, how do you envision fostering a culture of innovation within the PSI team?
To foster a culture of innovation within PSI, I plan to drive new ways of critical thinking across Content Science. This involves encouraging creative problem-solving, staying open to new ideas, and continuously seeking ways to improve our processes and products. By nurturing an environment that values innovation, we can ensure that PSI remains at the forefront of the credentialing industry. My approach includes fostering collaboration, promoting continuous learning, and empowering team members to take risks and explore new ideas.
