Eye on Assessments
Assessments must adapt to the changing global marketplace by becoming more relevant and more closely linked to development.
By Jac Fitz-enz
Some managers claim assessments are little better than their intuitive feelings. One told me assessments are better than doing nothing, but not much better. I disagree on several counts.
First, running an ill-conceived, invalid assessment is worse than doing nothing, since it will certainly cause us to jump to the wrong conclusions, spawning dangerous actions.
On the other hand, a well-designed, validated assessment delivers insights that can lead to low-risk, high-return human capital development investments.
The caveat is that reviewers must focus on the data and not skew it to confirm their predispositions. Finally, when assessment and development are linked, we can truly optimize the return on investment.
Clarifying the Field
Assessments come in basically two types: cognitive and behavioral. Figure 1, supplied by Questionmark, a Norwalk, Conn.-based assessment-technology provider, shows the different focal points.
Eric Shepherd, CEO of Questionmark, points out the importance of using assessments to measure cognitive skills such as memory recall, high-level thinking skills and motor-mechanical skills. This is the world of facts and objectivity. Testing employees' competencies and their abilities to carry out operational assignments -- recalling information and using it to successfully complete tasks -- is fundamental for organizational success.
Shepherd says behavioral assessments typically focus on competencies related to managing yourself, managing others, managing priorities, managing for results and managing change. These are important aspects of life at work, to be sure, but more difficult to measure. Cognitive skills are less affected by changes in the work environment than are behavioral traits. It is easier to teach people how to operate or repair something than it is to help them understand the complexity and consequences of interpersonal behavior.
Behavioral assessments aim to uncover interpersonal abilities, the subjective realm. What is sometimes underplayed in business is that behavior is driven by value systems: attitudes, beliefs and values. While some people have great skill in communicating and leading, their motives may be destructive. Dictators and some business executives fall into this category. They can move people, but in the wrong direction. As Peter Koestenbaum points out in his book, Leadership, the Inner Side of Greatness, a leader's values are extremely important. They are part of one's character, and character differentiates a despot from a saint.
At a less-dramatic level, positive values or ethical treatment of people are the hallmarks of great business leaders. It is pointless to ask people directly if they are honest. However, assessment instruments can be, and some are, set up to detect indications of a person's values. Given the competitive nature of business and recalling the highly publicized cases of executive corruption, ethical tendencies should not be ignored.
In 2011, Seattle-based consultancy Institute for Corporate Productivity conducted a survey aimed at finding out which assessment tools respondents felt were effective and determining which assessments were best correlated to market performance.
It found that, in organizations of more than 1,000 employees, 13 of 25 assessment tools had a negative correlation with market performance; six had a positive, but not significant, correlation with market performance; and four showed an extremely high correlation to market performance, but were not utilized by many of the respondents. Results of the study are available through i4cp's white paper, High-Performance Executive Assessment.
The recommendation by i4cp's senior vice president of research, Jay Jamroq, is to use assessment tools to provide relatively objective reads on a person's potential, performance and organizational fit. But, he cautions that a poor or improperly used tool can do more harm than good.
Accounting for Change
Obviously, the marketplace is experiencing rapid and continuous change. Global economics, competition, technology, customers and labor, among others, are all fluctuating on almost a weekly basis, it seems. I talked with leaders of several major vendors including DDI, Hogan Assessments, Kenexa, Lominger, PDI Ninth House, Questionmark and SHL. My concern was how assessments could maintain relevance in this rapidly changing context. I wondered how an assessment company could adjust to these changes when it has a huge investment in its previously validated instruments.
Eugene Burke, chief science and analytics officer at SHL, a London-based talent-measurement provider, acknowledges that relevance is a critical factor in the credibility of assessment. That might mean how the assessment is framed and delivered -- such as, through more interactive videos or avatar-based assessments to provide a better candidate experience and fit to the client's brand. But, ultimately, relevance refers to what the problem is that needs to be solved, starting with the desired resolution and working backward to find the best assessment solution. At SHL, this means identifying the critical behaviors that will predict success for the client and then mapping the assessments to those behaviors. The client can then make more informed decisions about whether the assessment solution is a configuration of existing assessments or a more customized build.
Talking in the language of behaviors and outcomes, which managers understand, is the key to addressing the relevance of the data or people intelligence found in an assessment. It is how the results of assessments should be reported to clients and incorporated in an organization's analytics around its people capabilities.
Ryan Ross, partner and vice president of global alliances at Tulsa, Okla.-based Hogan Assessment Systems, says his company's research division annually evaluates the performance of its client solutions through criterion-related validation and various outcome studies. By continually looking at outcomes and its predictive accuracy for clients, Hogan keeps up with the rapidly changing context. This is essential to maintaining predictive power and reliability, rather than resting on past performance.
Karis Ahlberg, product marketing manager at Wayne, Pa.-based Kenexa, suggests that assessments are becoming more all-encompassing by using a combination of personality, experience, cognitive and situational judgment questions. In addition, she says, companies are realizing the importance of assessing culture for changes and personal fit.
Experts at the other vendor companies also talked about continually upgrading based on current studies. Still, there is room for caution in this arena. The essential question, when considering the application of someone's assessment instrumentation is, How was this designed, developed and validated? Validation is more than usage. The survey items must be shown to be relevant to the behaviors and circumstances in question. Vendors such as those represented in this piece invest heavily in maintaining validity and relevance. Some others may not be as dedicated.
A common mistake on the supply side is when an unqualified consultant attempts to show knowledge of assessments by offering instruments that may be well known but not related to the client's specific needs. A similar problem occurs when a consultant attempts to expand the application of an instrument beyond what it was designed to provide. Many managers inside and outside HR do not have the experience or knowledge to evaluate such proposals.
i4cp's study of assessment tools showed that 77 percent of the 610 surveyed organizations use internally constructed 360s. The obvious question is, How do they know that the resulting data supports the requirement? Another 68 percent use the Myers-Briggs Type Indicator. How is this limited instrument used as part of a development plan? These types of applications sometimes create internal cults that can divide organizations. I recall one instance when this erupted in a famous case wherein the insiders and the outsiders drew battle lines that caused major operating problems and ended the provider's existence altogether.
Assessment is a sensitive field. Behavioral assessing is expensive, time-consuming and not easy to manage because it deals with personality variables. Perhaps most telling is that it is becoming part of the emerging application of predictive analytics for business.
Bloomberg Businessweek Research Services surveyed 930 businesses across the globe on the current state of business analytics. Key findings were that:
* Spreadsheets are still the No. 1 tool used in business analytics,
* Analytics tend to be used within departments and not integrated across the organization,
* Intuition based on experience is still the driving factor in decision-making,
* Data is the No. 1 challenge with access, accuracy and consistency problems,
* There is a lack of proper analytic talent in knowing how to apply the results, and
* Culture plays a crucial role in the effective use of business analytics.
The explosion of software tools, tablets and mobile devices is driving changes in assessment methodologies. In lieu of paper and pencil, or even computer-based questionnaires, tools such as gaming and avatars are particularly appealing to younger generations. The pace of business means that people may not have much time or patience to deal with traditional hour-long assessment tools.
I mentioned at the top of this piece that assessment and development need to be linked. In my view, the people who provide those two services should be at the table throughout the discussion about the investment or initiative. This discussion should start with defining its business purpose. From there, they can fashion a program that ties the services together and connects them to the business goal. A common trait across businesses is for functions to be internally focused and siloed. This is totally unacceptable.
Most importantly, assessment reports have to grow beyond benchmarks and return-on-investment figures. Burke states that the report data should tell clients what their options are and why any option is relevant to their needs. Eventually, assessments must evolve from backroom periodic reviews to simpler analytic tools that management can dial up and apply on a regular basis to assist in low-risk, intelligent decision making.
Jac Fitz-enz is the founder and CEO of Human Capital Source, based in San Jose, Calif. Send questions and comments about this feature to firstname.lastname@example.org.