Talent Management Column

Re-evaluating Assessment

Considering their obvious importance, you'd think employers would have perfected the candidate-screening and performance-evaluation processes by now. But thanks to a range of factors, including some that are quite subtle, bias continues to persist and cloud the judgment of assessors.

Tuesday, March 25, 2014
Write To The Editor Reprints

Many of the most important decisions in business and management involve making judgments about individual employees. From assessing job candidates to handling performance appraisals, managers are judging people all the time.

Hiring has been on the hot seat recently because there are still so many candidates for each job opening, and whether those candidates are assessed properly has obvious and big implications for fairness as well as whether we find good employees. Something similar has been going on with performance appraisals as the pressure intensifies to hold people accountable by giving more detailed and more accurate feedback. That requires making more accurate judgments.

Given the importance of evaluations, we would think that the process would be getting more sophisticated. Yet the opposite appears to be true, as expertise continues to be cut from the human resource function and more responsibility is pushed onto line managers, who often have little training or time to make accurate judgments about workers.

Without systematic practices or training, biases drive such judgments, and the range of biases that we know shape the outcomes are long and wide -- such as the tendency to pick candidates who look like us. More recently, we've learned that even the timing of the process introduces biases: You don't want to give managers complicated tasks -- such as assessing an unusual candidate - at the end of a long day, because the evidence suggests they are often out of energy by then and may make bad decisions.

Another bias was recently identified by my colleague Uri Simonsohn and his co-author Francesca Gino in the journal Psychological Science. They looked at MBA admissions decisions (not at Wharton), which involve assessing a lot of candidates in a short period of time. But the conclusions should be similar for hiring, where many applicants have to be screened at once, and for performance appraisals, where wider spans of control mean the same manager has many appraisals to perform.

What these researchers found is that the judgments that people doing the assessments make are heavily influenced by the assessments they have given to other candidates that same day. (This is different from comparison biases: i.e., we know you don't want to have your application follow a really strong one.)  Even though the evaluators will see thousands of applications, they act as though the set they see today will be the only ones they see.

If they want to admit one in five applicants, for example, and they process them in batches of five at a time, they will be very reluctant to admit a second person out of that pool of five once they have admitted the first one. On the other hand, if they have admitted none of the first four candidates, and you are the fifth and final applicant out of the batch of five, you are more likely to be admitted because they don't want to turn down all five out of that batch, even though they still have several thousand yet to assess.

Consider what happens when employers are screening job applicants to get down to a pool to interview. Let's say your recruiters are visiting 10 college campuses to interview students for summer internships and the goal is to come up with a short list of 10 students who will receive offers. It is quite likely the recruiters will feel compelled to pick one candidate from each school they visit, even if the average quality of the candidates differs considerably across the schools.

Newsletter Sign-Up:

HR Technology
Talent Management
HR Leadership
Inside HR Tech
Special Offers

Email Address

Privacy Policy

An even-more-common result is what happens when supervisors are only allowed to give one subordinate out of five an "excellent" performance appraisal score, and the supervisor decides to conduct five appraisals each day. Rather than comparing all the subordinates against each other, which is what we think should happen, supervisors will tend to compare them against the four others they see that day.

What should we do about this bias?

If you are the applicant or the subordinate, you do want your case handled on the day when the weakest other individuals are assessed. But you probably want to avoid being the last case heard on any day or in any batch because the outcome will be disproportionately shaped by what has happened previously that day, rather than just on the merits of your case.

If you are the employer, try to avoid having evaluators "batch" candidates, but if they have to, maybe they should be required to compare the winners to see if, in fact, they are all similarly good. And, of course, try to make sure the evaluations are driven by clear and objective standards that reduce the scope for judgment, which is often just another word for bias.

Peter Cappelli is the George W. Taylor Professor of Management and director of the Center for Human Resources at The Wharton School. His forthcoming book is Why Good People Can't Get Jobs: The Skills Gap and What Companies Can Do About It.


Copyright 2017© LRP Publications