Recently there has been a lot of press about how recruiting technology is biased.
Listen carefully. I am about to tell you the worst kept secret in psychology. Are you ready? Here it is: everything is biased. I mean it. Everything.
Whatever the activity, be it an interview, a test, something else, there’s going to be a group that does better and a group that does worse. In some cases it might relate to things that we are explicitly aware of, like gender or ethnicity or socio-demographics. In some cases you won’t know what it is until you do the analysis. But it’s there, waiting to be discovered. Nothing is bias-free.
Bias is like our past haunting us. It is rife in us humans, because we can’t ignore the patterns that have been imprinted on us based on our life experience to date. But it is also rife in non-human factors — like tests — because science demands that they are inevitably designed around what the job has been in the past, or around data from the past, and so they tilt in favor of those who have been successful in the past.
Before we go any further I am going to change my terminology. I think that the word ‘bias’ has very serious connotations and implies something deliberate or destructive. But the fact that performance differences between groups are pervasive in everything tells me that we should not use the word ‘bias’ for everything. I am therefore going to switch to using my preferred technical term: subgroup differences.
So let’s divide this problem up. Recruiting is a process that consists of different activities. You cannot eliminate the subgroup differences from the activities, even reducing them is hard, but you can try to reduce or eliminate subgroup differences in the overall process, so you can maintain representation from the beginning to the end of that process. Here are some of the things you can think about.
1. Analyze everything
You cannot manage representation in your recruiting process if you don’t have good stats on how your subgroups are performing, both through the process and in specific activities. Examples of some of the analysis you should regularly conduct are:
- Funnel analysis — what proportion of each group of interest is progressing to each successive stage
- Ratings/score analysis — what are the differences in the average ratings or scores achieved by the different groups of interest? Are they statistically significant? What does academic research tell us we should expect?
- Influence analysis — how do individual activities or competencies correlate with individual decisions or process outcomes. Do activities or competencies that favor certain subgroups have a larger than usual weight in decision making?
2. Identify ‘hot spots’ in the process
Using your funnel analysis, identify a particular stage or stages in the recruiting process where the biggest difference in subgroup progression is evident. Focus in on the hot spots. Using your ratings and influence analysis, identify whether there are specific activities that have the largest subgroup differences and which would explain the progression differences.
If no such obvious activity exists, then the ‘hot spot’ could be the decision making mechanism. How are decisions made? Who is making them? What structures are in place to determine how the information is used and to make sure that the discussion is fair?
3. Remove the worst offenders
If extreme drivers of subgroup differences are discovered, consider removing these completely from your process. The offenders could be specific activities or individuals.
In the case of activities, subgroup differences that hit as high as one standard deviation between the top and bottom performing groups represent extreme offenders, particularly if they are the only activity in that stage of the process, or if a lot of weight is attached to them. Consider how these can be replaced with less serious offenders.
In the case of people, certain interviewers or assessors can have approaches which, although usually not deliberately intended, do severely negatively impact certain groups. Ratings analysis can highlight who these people may be, and observation of them in action can reveal the behaviors themselves. I strongly encourage organizations to introduce interviewer observation as a way of identifying and controlling for negative behaviors in the selection process. Offending interviewers can receive training, or repeated offenders can be asked not to participate further. Its tough, but sometimes necessary.
4. Introduce a structure and logic to decisions informed by the analytics
The steps above should allow you to adjust the logic of your process to address some of the clear dynamics that are impacting subgroup progression. Some ways of doing this include:
- Changing the steps of the process to remove or add activities
- Articulating a strict decision logic at each stage of the process, informed by the known impact of each activity on subgroup performance. For example, if your screening process consists of two tests, create a matrix decision logic based on the results of both tests which is optimized for the smallest subgroup differences.
- Explicitly identifying where groups of candidates can be considered ‘equally qualified’ based on their performance throughout the process — in this case you can preference under-represented groups in your final decisions if job offers are limited.
5. Democratize data and analytics among all participants in the process
Don’t underestimate the potential of data and analysis to change individual behaviors. Constantly repeat and share the analyses from Step 1 with the key groups and individuals who participate in your selection process. This has two potentially powerful consequences:
- It encourages a more data-driven mindset among all participants and decision makers, a critical factor in ensuring structure and fairness
- It allows individuals to self-correct their behaviors based on real data showing their impact. If you can provide data to interviewers, for example, and combine this with feedback from observation by an expert. this can be a powerful double-whammy.
Improving fairness and representation in recruiting is an important mission, but also a difficult one that requires hard work and an analytical, data-driven approach. Gossip and talk about bias in specific methods and technology only serve to distract professionals from the job at hand. ‘Bias’ is an inevitable and unavoidable part of life which we have to manage. Let’s get on with managing it.