We need to talk about the importance of diverse Subject Matter Experts (SMEs). Why? Because diverse perspectives lead to more equitable tests, exams, and assessments. And testing changes lives.
Diversity, equity and inclusion is important throughout the assessment lifecycle. And ensuring all individuals and groups of test takers experience a fair and equitable test is a fundamental principle for testing. Involving a representative group of SMEs in your test development process not only supports this principle, but it also has the potential to affect change. Positively impacting your test takers, testing program, industry, and society.
So what are the reasons for involving diverse SMEs in test content development? What can you do to recruit diverse SMEs? And when you’ve recruited them, how do you keep them engaged? PSI supports multiple clients with SME recruitment and management. This blog shares our experience in this important area of test development.
Who are we looking for?
Everything depends on the quality and diversity of your SME committee. If you have the right SMEs on your committee, you’ll be having the right conversations. The first step is to involve SMEs who are representative of your current target population – and wherever possible the population they serve. However, this is never a case of job done. Testing organizations should always look to further diversify their SMEs.
Best practice, and something we support clients with, is to create a table of the target demographics you are seeking in your SMEs. That way you can ensure you are not only involving diverse SMEs, but you can also demonstrate you are meeting relevant targets. For many organizations, an important step towards recruiting diverse SMEs is to focus on this target demographics table. Again, ensuring it includes all the different groups in your target population – while being aware of who they serve.
Once you’ve established your target demographics, next you need to recruit them.
Where can we find diverse SMEs?
Where you find diverse SMEs is often less important than how you attract them. So ensure your selection criteria are transparent, and clearly define what you are looking for. Make it clear to your potential SMEs why their involvement is important. Show them how they will add value and where their unique perspective will come into play.
Many individuals won’t consider themselves to be ‘experts’ compared to peers who’ve worked in the industry for decades. But a variety of perspectives is essential for any SME committee. Explain that people who are newer to the profession will also add value. Particularly if they have a different perspective to add, for example around cultural sensitivities.
Promote the benefits of being in a room with a group of people who are as passionate about their profession as they are. Talk about the networking and professional development opportunities that come with the role. We’ve seen confidence levels grow massively and lifelong friendships formed because of SME meetings. Being an SME is a very rewarding experience.
Now you have a clear value proposition for your prospective SMEs, look far and wide. This will be different for different sectors, but consider anywhere that encourages and represents diversity:
- Professional organizations.
- Relevant social media groups.
- Organizations specific to your industry.
Your existing SMEs can be another effective route for recruiting new talent. They understand what’s involved and the value it offers, and as a result can be your best ambassadors. Let them know the demographic you are looking for and your current SMEs might be able to get others involved that wouldn’t have otherwise considered it.
What’s important when onboarding diverse SMEs?
SME training is vital. Both for new SMEs and to keep more experienced SMEs up to date with changing processes and ways of working.
Training sessions are also a good time to establish an atmosphere of open communication. You need to get the balance right. It’s important for trainers and committee facilitators to create an environment of mutual respect where people listen to opposing views – and feel comfortable pushing back.
For a diverse group of SMEs to be effective, everyone, regardless of seniority of where they sit in the industry hierarchy, should be given the space to express their opinions. You need different perspectives and a group of SMEs who are going to listen to them. Without this psychological safety, you might be missing out on important perspectives and experiences.
Mentoring programs, where new SMEs are paired with more experienced SMEs, is one way to increase comfort levels. Also ensure your SMEs have a point of contact to go to if they have any questions or concerns they don’t want to address in front of the whole group. Establishing these mechanisms from the outset will give you the best outcomes.
How do we retain diverse SMEs?
Recruiting diverse SMEs won’t be an effective strategy if they don’t stay for very long. Your testing program and test takers won’t experience the benefits and all the time and effort you invested in recruitment will be wasted. Engagement and retention is just as important as recruitment. Making sure meetings are a safe space will help with this.
Anyone working in testing knows that diverse populations have diverse needs. So meeting the needs of a diverse group of SMEs requires flexibility. This might be around the timing of meetings, at certain times of day to accommodate those with childcare responsibilities for instance.
You can also explore the best format for meetings, with online meetings for example. That said, we do find that SMEs get more out of in-person meetings. They offer more opportunities for professional development, as well as the potential for valuable side conversations and connections with industry peers.
Another important tool to retain SMEs is to go back to your why on a regular basis. Demonstrate to your SMEs the impact they are having. Show them the data, how items are performing, where change has happened and improvements have been made. Seeing how they have affected change will motivate SMEs to continue the good work.
Keeping your SMEs in mind through the whole assessment lifecycle will ensure they are engaged. From test content development, through to your results and data.
Read our guide on now to develop the right test content for your certification program.
How long should we retain our SMEs?
This is a tricky question that we are often asked, and again it’s all about balance. Yes, you need to take steps to keep your SMEs engaged and retain them. But you also need a regular cycle of SMEs, to bring new perspectives and creativity, and reflect industry changes. An established group of SMEs that have known and worked with each other for many years can be intimidating for new SMEs.
Having three to four committees that your SMEs move around is one way to keep things fresh. You can also provide different opportunities to keep individuals engaged beyond participating as an SME. Being on your board or part of the governance structure, for example. You can always bring them back in the future when they’ve had an opportunity to get involved elsewhere. Having a pipeline and roadmap of these different opportunities is a useful way to keep your SME group fresh and your willing volunteers engaged.
Read our blog on tips to manage SMEs in test development.
How do we maintain good practice?
It’s easy to get comfortable when you already have a productive group of engaged SMEs. If it’s working why would you change it? However, even if things are going well with your SMEs, it’s worth regularly checking-in that your group is still representative of your target population. Be honest with yourself and take action if you do need to find new SMEs.
Sharing our experiences and successes is another way we can all get better. Industry events are a great place to share what you’ve done, as well as stay in touch with evolving best practice around SME involvement from testing industry colleagues.
Don’t keep the secret sauce to yourself – and try out other people’s recipes! That’s how we’ll bring about change in our own organizations and have a ripple effect across the whole industry.
