The Ultimate Guide to Fair Interviews

✅ Checklist of actions:

1. Devise a rating scale
2. Prepare relevant interview questions
3. Use sequential interviews, not panels
4. Run structured interviews
5. Compare candidates and decide who to move forward with

1) Devise a scale to rate candidates consistently

Before you kick off your interviews, you need to know what exactly you’re looking for. If you use vague criteria or leave candidate evaluation down to intuition, then stereotypes will fill in your judgements.

Consistent rating scales on the other hand tell interviewers exactly how to assess candidates. Defining qualities for success gives interviewers a crystal-clear picture of what ‘good’ looks like. It’s a tried and true way to judge candidates objectively.

1a) Define what you’re looking for

Each role has unique qualities and capabilities that are crucial for success. Whenever you open a new vacancy, define five to six qualities that an ideal candidate should demonstrate.

These skills or qualities must be measurable and relevant to the job description. Many recruiters fall into the trap of searching for vague, generic qualities in their hires — things like ‘effective communication’ and ‘ability to multitask’. Sure, they sound good on paper, but are these decisive factors for all roles?

If you’re looking to hire an electrician, you wouldn’t ask about their public speaking skills, would you? So does your graphic designer really need to have native-level written English? Drill down on exactly what qualities candidates need to exhibit to do well on the job. Here are some examples:

❌ Eager to excel.

✅ Can clearly articulate their passion for the role, relates it to personal experience.

❌ Team player.

✅ Demonstrates enthusiasm for working in a team and has experience resolving complex disagreements.

1b) Create your rating scale

Now you’re clear on the qualities that make a successful candidate, it’s time to add some numbers to the equation. For each quality, define a rating scale with clear, objective criteria for each score.

Identify what a poor(⭐️), OK (⭐️⭐️⭐️) and great (⭐️⭐️⭐️⭐️⭐️) response looks like. This will keep interviewer scores consistent no matter who’s asking the questions. See an example from our Chief of Staff interview rating scale below.

1c) Create a scoring sheet

Gather the interview scoring materials in one central document that’s easy to access. Make sure to leave room for comments. Interviewers should take notes throughout the interview so that they can justify their scores down the line.

Once you’ve created these rating scales, chances are you will be able to reuse them frequently. Especially for skills that are important across roles, such as problem-solving and collaboration.

Our Fair HQ recommendations:

• Define 5-7 key qualities that are essential for success in the role.
• Make sure these qualities are specific and measurable.
• Create your rating scale and identify what a great, OK and poor answer looks like.
• Create a scoring sheet for interviewers

2) Prepare your interview questions

Now that you’ve defined your list of qualities for success in the role, you need to come up with a set of questions that explore these qualities. This is a crucial step in the roadmap to fair interviews. To get a full understanding of candidates’ potential, the magic lies in carefully preparing questions that focus on essential skills and values.

Always prepare interview questions in advance. This will keep interviewers from straying off-topic. It’s much easier to compare candidates when they’ve answered the same set of questions.

We recommend starting with a skills-based interview before evaluating value fit. Starting with a skills assessment means you’ll deal with candidates who have the technical know-how and relevant experience from the get-go.

2a) Create your questions

Aim for a list of five to seven questions that invite longer ‘story’ answers. Keep things open so that you can explore answers in detail. Make sure your questions cover the qualities and capabilities you identified as critical for success.

Avoid hiring gimmicks that explore candidates’ personalities — this can introduce bias. Instead, focus on skill. We’ve dedicated a post on why it’s important to hire for culture add and value fit, not culture fit. If you’re seeking to master the art of interview questions, it’s definitely recommended reading.

2b) Focus on past evidence, not future potential

Asking candidates to provide solid evidence of times that they’ve demonstrated relevant skills will give you a better idea of their abilities. Past performance is a much better measure of candidates’ potential than responses to hypothetical scenarios.

Our Fair HQ recommendations:

• Start with a skills-based interview.
• Create 5-7 interview questions that explore relevant skills.
• Ask open questions that invite story-like answers.
• Ask candidates to describe past experience rather than future potential.

3)Ditch panels, use sequential interviews instead

If you’re tuned in to the D&I conversation, you may have heard before that conducting panel interviews can improve diversity in the pipeline. It seems common sense — candidates will face a fairer assessment if multiple perspectives are present on the interview panel.

Panel interviews are also popular because they’re a huge time-saver. Interviewers can align their assessments in real-time and hear multiple perspectives on a candidate.

In fact, panel interviews amplify bias. When you have multiple people judging candidates at the same time, it’s all too easy for interviewers to fall prey to ‘groupthink’. Rather than forming their own opinions, their assessments can be swayed by their peers.

There’s a much better solution — sequential one-to-one interviews.

3a) Schedule your one-to-one interviews

Instead of going for long-form panel interviews, schedule a series of short interviews for each candidate.

Decide on who will interview the candidates. Include colleagues they’ll work with closely, the manager they’ll report to and those who have a good understanding of the responsibilities of the role.

3b) Plan collaboratively

Great interviews require some setup. Don’t just wing it! Here’s are the key things to consider:

• Give each interviewer around 20-30 minutes with the candidate to ask their questions.
• Define a unique purpose for each interview, whether that be a skills-based screening, an in-depth look into their work sample or a value-fit interview.
• Plan collaboratively so that there’s no overlap between interviews.

3c) Assign an individual to have oversight

Interviewers should not be able to see each other’s scores until all assessments have been completed. This is absolutely key. Otherwise, people may be swayed by their peers’ scores before they’ve had a chance to formulate their own judgements.

Assign one person to collect interviewer scores after each sequential interview. They’ll store scores separately from the rest of the team.

Our Fair HQ recommendations:

• Schedule your 1-to-1 interviews
• Plan collaboratively so that each interview has a unique purpose
• Nominate an individual to have oversight of the process and collect interviewer scores

4) How to run structured interviews

The tips above help you assess applicants consistently. This is an excellent way to limit bias, but there’s another simple trick that’s cost-free and effective. Ask candidates the same questions in the exact same order.

You might think that having a “casual chat” with a candidate gives you a really great sense of what they’re like as a person, if they’d fit in with your team, and if they’re right for the job. This style of interviewing actually leaves way too much down to intuition. You’ll make judgements based on whether or not you “like” the candidate, which is heavily influenced by personal biases.

Keep things consistent and fair by running structured interviews.

4a) Ask the same questions in the same order

Asking candidates the same questions guarantees that everyone faces consistent treatment and has an equal opportunity to prove themselves.

It might feel a little awkward at first. Naturally, you’ll want to explore different aspects of candidates’ unique answers. But make sure to never stray too far from the prepared list of questions.

Even if you feel you just have to comment on their gorgeous coat or the fact you both have a passion for baking, hold yourself to the plan. It takes some getting used to, but structured interviews are key to fair hiring.

4b) Rate candidate responses immediately

The sooner you rate candidates the better — if you wait, you risk leaving your memory to fill in the gaps. This means that you might just remember the most intense or confident answers.

Instead, rate candidate responses as soon as possible so that you judge based on fresh information.

4c) Take notes throughout the interview

Taking notes will help you explain your scores to the rest of the team after the interviews are completed.

Our Fair HQ recommendations:

• Ask all candidates the same questions in the same order
• Rate candidate responses as soon as possible, ideally immediately after the interview
• Take notes throughout the interview so that you can justify your scores later down the line

5) Make a collective decision on who should move forward

At this stage, all interviewers will have formed their own opinion, based on consistent and objective criteria, free from the perils of groupthink. Now’s the time to reach a decision on who to move forward with.

You can go about it in a few different ways:

Aggregate candidate scores

Before meeting to discuss candidates, aggregate their scores. Add up all scores each manager gave for each candidate across every stage of the interview process. You’ll be able to see – at a glance – which candidates performed best overall.

Or, meet to discuss and combine ratings

The alternative route to go down is simply meeting to discuss ratings. Compare candidates against each other instead of discussing them one by one.

For example, first compare how candidates performed in their technical interview, then in their culture-add interview. Repeat the process for every stage. This approach will help you focus on each person’s performance.

If you find that the scores differ wildly between managers, investigate why. Did the candidate display a red flag that not everyone picked up on? Or was the criteria not applied consistently? Ask managers to justify their scores and come to a collective decision on who to move forward with.

Our Fair HQ recommendations:

There are two ways to go about making a final decision:

• Aggregate the scores of each candidate and set a minimum threshold to advance.
• Discuss ratings together, justify scores and decide collectively on who to move forward with.

And there you have it — our complete roadmap with 5 pitstops on how to create a fair interview process. When you break it down, most of these actions are cost-free and easy to embed. It’s well worth the work. With a clearer picture of candidates’ potential, you’ll avoid costly hiring mistakes and welcome more diversity to your hiring pipeline.

After all, small shifts add up to massive transformations.

Backing it up

Correll, S. (2017) ‘Reducing Gender Biases In Modern Workplaces: A Small Wins Approach to Organizational Change’ Gender & Society 13:6

Bohnet, I. (2016) ‘How to Take the Bias Out of Interviews’ Harvard Business Review

Purkiss, S. L. S., Perrewé, P. L., Gillespie, T. L., Mayes, B. T., & Ferris, G. R. (2006). Implicit sources of bias in employment interview judgments and decisions. Organizational Behavior and Human Decision Processes, 101(2), 152-167.

Bohnet, I. (2016) ‘What Works’ Harvard University Press, Ch.6: ‘Orchestrating Smarter Evaluation Procedures’

Tulshyan, R. (2019) ‘How to Reduce Personal Bias When Hiring‘ Harvard Business Review