Reviews: The Continuing Conundrum
A seven-year examination of performance appraisals at one large organization suggests employers may want to rethink some well-entrenched assumptions.
By Peter Cappelli
We are in the season for performance appraisals, and the fact that many companies are experimenting with changing their appraisal systems has made this the hottest topic in HR.
Virtually every U.S. employer uses performance appraisals, including the government and the military. Surveys suggest around 95 percent of U.S. companies use them, and they are now common around the world. Psychologists have regularly studied the factors that cause appraisal scores to be biased over and over, but the most basic questions about how well these performance reviews work have rarely been asked. Instead, we have preconceived notions that don't match up to reality.
In a recent National Bureau of Economic Research working paper titled What Do Performance Appraisals Do? my colleague Martin Conyon and I tried to answer the basic question of what appraisals actually do and how they work. To do that, we looked at the performance appraisals for management employees of a large U.S. company over a seven-year period, including their associated employment outcomes. Among other things, we found that appraisal scores did, in fact, vary quite a bit across individuals. It wasn't the case that everyone got a good rating and no one got bad ratings. The shape of the distribution looked surprisingly normal, with slightly more "poor" scores than "excellent" scores.
We also found that the appraisals did matter. Employees who got very low scores did tend to get fired, those with pretty low scores did tend to quit, those with the best scores got promoted and so forth. That might seem obvious, but the prior research looking at this simple question -- now decades old -- didn't find that. Seniority tended to beat out performance scores.
What most interested us was a debate about the purpose of appraisals and the merit-pay increases associated with them. Is their purpose to "settle up" last year's performance -- you get rewarded based on how you did last year, as a contract might work? Or is it more about improving performance, treating employment as a continuing relationship? You might be inclined to say "both," but in practice, the outcomes have to lean in one direction or another.
We found more evidence that was consistent with the notion that supervisors treated the appraisal process more like a continuing relationship than the settling up of an annual contract. In particular, the merit-pay increases they gave rewarded employees for improvements in performance, not just for high levels of performance. Merit-pay increases were not determined by a simple relationship with performance, i.e., a 20-percent-better score gets you 20-percent-higher pay. The best performers were rewarded disproportionately more while the lowest were held back disproportionately. This is in contrast to a common assumption, that supervisors tend to clump merit-pay increases in the middle.
While these results intrigued us, the question that most interested everyone else was whether good performers this year would be the good performers next year. I wrote last month about the tendency associated with the "fundamental attribution error" and the belief that performance is really about the person and not the situation -- for example, good performers are always good and bad performers are always bad.
One of the things we looked at was how well we could predict the score for an employee based on knowing his or her score for the previous year. The answer: Not well. We could explain only one-third of the scores next year knowing the scores this year.
Changing managers didn't appear to have any consistent effect on scores, either. That's contrary to the view that supervisors get cozy with subordinates and give them higher scores over time.
While we only looked at one company for this research, nothing about it caused us to think that its experience with appraisals would be unique. Surprisingly, almost no research has ever explored this question before. If you have your own data, why not take a look to see what you find? (Just remember that the correlation between scores isn't the useful measure. You have to "square" the correlation to get the amount of variance explained -- the R2 statistic.)
We had no strong prior assumptions about what we would find on this question of the persistence of scores over time. What has surprised me is how many people do, and that virtually all of them seem to think that good players are always good. This is a neat and tidy view, because it suggests that employees are in control of their performance, that management doesn't do much and that financial incentives are about all that's needed to know to manage people. But just remember: The field of human resources is about the management of people.
If that view is right, then we don't need to be here.
Peter Cappelli is the George W. Taylor Professor of Management and director of the Center for Human Resources at The Wharton School of the University of Pennsylvania in Philadelphia. His latest book is "Will College Pay Off? A Guide to the Most Important Financial Decision You'll Ever Make."