Does Big Data Matter?

The Centers for Disease Control and Prevention look for what people search for on Google to spot outbreaks of diseases, says one expert, while people are using Twitter to accurately predict the opening box-office revenue for movies. Take a look at how Big Data is also changing the role of HR in employment screening, recruiting and building great companies.

Monday, July 23, 2012
Write To The Editor Reprints

Those who wonder about how big a deal Big Data is should take note of Target Corp. To better understand how to market new products to their customers, analysts at the retailer studied years and years of customers' purchase data. They were looking for patterns that would enable them to predict the future.

What they found was a way to spot a pregnancy in the first trimester, based on changes in a woman's buying behavior. A key indicator is the switch to unscented lotion. 

It is amazing what can be found when you look into the data. The Centers for Disease Control and Prevention look for what people search for on Google to spot outbreaks of diseases. People are using Twitter to accurately predict the opening box -office revenue for movies. Political campaigns do house-by-house studies to determine where to target their messaging. Companies are pouring tens of billions of dollars into Big Data to find patterns they can use to predict the future.


The reason for this is clear: the amount of money that can be made or saved through better data targeting is immense. Small changes on the margin, sales or productivity increases of just a few percent, are worth tremendous amounts of money. And often, they're as easy to find as looking at what kind of lotion a woman purchases.


Random Processes, Random Outcomes

One area where the use of data analytics could have a tremendous impact is in recruiting. United States employers collectively spend about $124 billion a year on recruiting, according to Bersin & Associates, and almost $6 trillion on payroll, according to the U.S. Bureau of Labor Statistics. With that level of spending, small improvements in outcomes can easily be worth billions or tens of billions of dollars. 

However, the processes used to select workers have been shown again and again to be highly questionable at best. A recent series of news articles about employers asking for Facebook logins to screen employees highlighted how random and even invasive recruiting processes can be. But, the truth is that many of the most-common screening techniques simply don't hold up under any kind of scrutiny. A review of these screening techniques show just how wrong conventional wisdom can be:

* Job Hoppers and the Unemployed: A common screen for job applicants is work history. Recruiters tend to reject applications from job hoppers, as well as people who are unemployed. The theory is that these are clear signs that the person has something wrong with them.  
When researchers at Evolv looked at the data, they found that there is no predictive value in looking at how many jobs a person has recently held. Candidates with five jobs in five years were no more of an attrition risk than candidates with only one. Candidates who had been unemployed were no more or less likely to quit or be terminated. Screening out job hoppers and the unemployed serves no purpose.

Criminal Background Checks: 92 percent of SHRM member companies use criminal background checks as part of their standard hiring process. In general, a criminal record makes an applicant ineligible for the job. In some cases, candidates are denied employment for having ever been arrested, even absent a conviction. In other cases, a single conviction for a minor crime leads to decades of ineligibility for any kind of work. In general, a felony conviction -- even an old one -- makes it extremely hard to find work.  
Numerous studies have found that criminal convictions, especially old ones, aren't predictive of any future bad behavior. One study, titled Predicting the Counterproductive Employee in a Child-to-Adult Prospective Study, found that crimes committed before a person entered the workforce had no predictive value for any "counterproductive workplace behaviors." Another study found that people with records who stay arrest-free for four to five years are only as likely as the average person to be arrested again. A third study found that, for people arrested when they're 18, their risk of re-arrest drops to that of the normal population by around age 25.  
Background checks can be useful if they uncover a recent, serious offense. Unfortunately, many employers use background checks as a screen to remove any candidate with a criminal record, regardless of the nature of the offense, or the likelihood that the person will become a problem employee.

Job Interviews: The job interview is the most common element of the recruiting process. While employers may not conduct background checks or credit checks, it's hard to imagine a situation where a person could be hired without an interview. 

Research has consistently shown, however, that most interviewers aren't skilled enough to really assess a candidate's capabilities. One study found that interviews are substantially less predictive of candidate quality than simply looking at their resume or checking their references. Another study found that the even untrained observers can predict the outcome of most job interviews after watching the first 15 seconds.

People who study recruiting know how broken the process is. There are dozens of examples of screening heuristics and tools that don't deliver as promised. Educational background and academic achievement predict nothing. Tests for writing software don't find great engineers. Recent news articles claimed some employers were asking for candidate Facebook login information as a way to screen applicants. As obviously critical as it is for companies to really the best people they can, the screening processes used by most employers are terrible.


When the Chips are Down: High Stakes Recruiting

As important as better hiring is for employers, there are situations where who gets the job is even more critical. In some cases, the right hire is worth tens of millions of dollars, or even people's lives. In these cases, hiring processes tend to be far more precise.


The NFL Combine: In football, a small number of key positions can make or break a team. The right quarterback, runningback or receiver makes the difference between the playoffs and a short season. Because of this, top draft picks are offered contracts worth tens of millions of dollars. 

To select players, the National Football League created something called the NFL Combine. This is a job interview on steroids, literally. The top college players spend three days being measured and tested on speed, strength, agility, personality and intelligence. All these attributes are rolled up into a score that, theoretically, would predict performance of the athlete as a professional. 

The Combine has been run every year since 1985, and is a key part of the hiring process for professional football players. In 2008, a study looked at the relationship between combine scores and actual performance on the field. The results: "Using correlation analysis, we find no consistent statistical relationship between combine tests and professional football performance, with the notable exception of sprint tests for running backs."

Officer Selection in the Israeli Defense Force: To determine which soldiers would perform well as officers, the Israeli Defense Force created a series of tests and screening tools. One of the key tests involved a physically challenging task performed in a group. A team of psychologists would observe the men to look for traits found in good officers. 

The psychologists found strong internal consistency with their predictions. They all agreed on what they saw. However, in comparing predictions with the results, one researcher wrote, "despite our certainty about the potential of individual candidates, our forecasts were largely useless." Interestingly, the IDF continued to use the test even after it was shown to have no value, just as many screening methods live on even after they're shown to be useless.

The truth is that finding meaningful screening techniques can be extremely difficult, even in situations where screening is critical. With this as the case, human resource professionals and recruiters may simply accept that the standard practices are good enough, and that no better options are available.


Using Data to Predict Performance Can Work

Despite examples to the contract, it is possible to design a process that actually does predict outcomes. 

Author Michael Lewis' book, Moneyball, shows how the Oakland Athletics used statistics to field a substantially better baseball team. While many teams were looking at a standard set of statistics, the A's developed a more data-driven approach. They looked at different, more telling statistics than the other teams. And they were merciless in following the data. In fact, models developed by an economics major from Yale were used to overturn the recommendations of experienced scouts.

Newsletter Sign-Up:

HR Technology
Talent Management
HR Leadership
Inside HR Tech
Special Offers

Email Address

Privacy Policy

The results of this approach were that a team with one of the lowest payrolls in baseball started going to the playoffs consistently. After decades of going to the playoffs once every five years, the A's went to the playoffs in five out of six years. The A's payroll at this time was only $39 million, while top teams like the Yankee's were paying their players $114 million. For the A's, better hiring processes trumped money when helping to build a winning workforce. 

Data can be used to predict performance in more mundane pursuits than professional baseball. Our own experience at Evolv is that there are generally clear signs telling which candidates will be among the top performers, and which will be among the bottom.

Sometimes, these signs are obvious. For example, one company we engaged with had a typing test as part of their application for a job that involved intensive typing. It's not a surprise that people who scored low on the typing test would not perform well on the job, and have a relatively short tenure. This turned out to be the case.

The surprise is that, despite the value of the typing test in predicting performance, it wasn't being used to screen candidates. 

Across employers, we've found that using the right approach to measuring quality of hire, and using data to guide decisions, can have a tremendous impact on hiring outcomes. In the best cases, the entire workforce starts to show the performance characteristics that were previously considered attributes of the very best performers. 

The economic benefits of this are tremendous. With different employers, we've seen productivity increases of 15 percent, average revenue per sales associate increase by 5 percent per hour, employee engagement scores increase by 8 percent and employee attrition drop by 30 percent. 

Making the Change to Data Driven Workforce Selection

Across industries, the quality of the workforce is becoming increasingly critical as a differentiator and competitive advantage. A key objective of all human resource professionals must be to deliver to their employers the best workforce for the money. This is one of the key areas where HR can deliver true strategic value. Yet, for most recruiting organizations, "quality of hire" isn't measured or tracked against a target. 

There are three ways that HR leaders can help their companies make the needed changes towards a data driven approach to workforce development. 

The first step is to start gathering data on how hiring programs and new employees are performing. Specifically, you need to measure the actual outcomes of the people you hire. Are they staying on the job? How productive are they? What percent of new hires are meeting expectations for performance? What are the performance characteristics of great hires versus bad hires?

The second step is to use the performance data to develop a quantitative score for quality of hire, especially around high-volume positions. Having a score for quality of hire creates the core measurement against which all other hiring activities can be judged. Common metrics like cost per hire and time to hire are only actually useful if you know if you hired the right person. Hiring the wrong person quickly and cheaply has no value to the organization. Additionally, with a quality score, incentives can be set that align recruiters with organizational goals. 

Finally, the recruiting process has to be tracked and managed to make sure that the screening tools and process are delivering against their objectives. The screening needs to be proven to be predictive. The recruiting organization needs to be held accountable for hiring top performers. 

Big Data is changing the world, and it is changing the world of human resources, as well. New technologies and rapidly advancing analytics are changing the nature of the contribution that human resources can make to the organization. With quantitative metrics and a focus on the strategic impact of a more productive workforce, the recruiting function will become of the key drivers in organizational success.

Dan Enthoven is Evolv's chief marketing officer. Evolv is a provider of SaaS-based workforce intelligence which utilizes big data predictive analytics and machine learning to optimize the performance of global hourly workforces. For more information, visit


Copyright 2017© LRP Publications