Predictive Analytics and Quality of Hire

Used correctly, predictive analytics will bolster your strategy

for attracting talent, streamline your recruitment process,

identify your workforce needs and improve Quality of Hire.

PREDICTIVE ANALYTICS AND

QUALITY OF HIRE

© 2016, Future of Talent Institute

Kevin Wheeler

February 2017

Predictive

Analytics and

Quality of Hire

2


3

Prediction is very difficult,

especially if it’s about the

future.Niels Bohr


Technology that learns from data to predict the future behavior of individuals.

In order to drive better decisions

What are Predictive Analytics?

4


Types of Data

5

Source: Adapted from Michael Corcoran

Information Builders

STRUCTUREDUNSTRUCTURED

Sentiment

Analysis

Predictive

Analysis

Text/Word Analysis

Data Analysis

Search

Ad Hoc Analysis

Performance Metrics

Dashboards

BREADTH OF DATA

DE

PT

H O

F A

NA

LY

SIS Personality Analysis


Descriptive & Predictive Analytics Compared

6

Descriptive Analytics Predictive Analytics

Purpose

Understand the Past

Observe Trends

Discuss

Gain Insights

Make Decisions

Take Action

Timeframe Past and Current Future

Metrics Type Lagging Leading

Data Used Raw/Tabulated Information

Data Type Structured Structured and Unstructured

Benefits

Understanding

Efficiency

Information & Insights

Effectiveness


The Promise

7

MORE EFFECTIVE

COMMUNICATION

FASTER

PROCESSES

GREATER

CANDIDATE

UNDERSTANDING

HIGHER

QUALITY

HIRES

INCREASED

INTERACTION

ENAGING

CANDIDATE

SERVICE

PEOPLE

ANALYTICS


“When I use a word,” Humpty Dumpty said, in rather a

scornful tone, “it means just what I choose it to mean—

neither more nor less.” “The question is,” said Alice,

“whether you can make words mean so many different

things.” “The question is,” said Humpty Dumpty, “which is

to be master—that's all.”

The Illusive Definition: Quality of Hire

8

LEWIS CARROLL (Charles L. Dodgson), Through the Looking-

Glass, chapter 6, p. 205 (1934). First published in 1872.


What is a “Quality” Hire?

-Someone who does what is expected consistently over time.

-Someone who exceeds the performance of an “average” performer.

time

perf

orm

ance

L

H

L H

Average performer

High Quality Hires

Low Quality Hires


Solid Predictive Analytics Based on. . .

10

Avoiding Assumptions

Predictions based on proxies

Not questioning

The future = the past

Transparency Privacy

No Black boxes

Never using data without candidates knowledge

Well thought out guidelines

Basics

A clear problem statement

Predicting what has impact

Data Structure

Clean data

Large enough sample size

A model neither too simple nor too complex

Biases

Awareness of unintentional/Inherent bias

Not favoring one set of data over another


But Ah, the Rub!

11

Only 14% of organizations have data to

prove the positive business impact of

their assessment strategy. -Aberdeen 2014

Quality cannot be measured or predicted until there are clear, objective and accepted definitions.

What is expected? At what level or speed?

What is average performance?

What would above average look like?

How can performance be consistently and objectively measured?

How is the information on performance collected?

Do you have the right data? Do you have enough data?

Do you have enough relevant data?

Is there a large enough sample?


Can You PREDICT Who Will be Best?

12

“Talent assessments, backed by science, add a

level of objectivity to the hiring process that goes

beyond “gut feel.”

-Scott A. Scanlon

http://ow.ly/cUUU3095esy


Yes, with the right data &

evidence that can be shown

to cause a correlation in

performance.

13


Hard to show any cause/correlation for these:

Resume quality

GPA

Schools attended

Former employers

Experience

Common Ways Used to Measure QoH

14


Boolean Searching of the Resume Text

Behavioral Interviewing

Concept Mapping of the Resume Text

Ability Testing

Personality Trait Testing

Culture/Fit Testing

The Search for Magical Solution


Boolean Searching of the Resume Text

Can find resumes with specific skill/experience claims but powerless to assess performance and risk.

Behavioral Interviewer Training

Studies prove BIs can accurately predict job performance but best practices in the lab don’t get followed in the field.

Concept Mapping of the Resume Text

Burning Glass and Engenium are examples. Uses artificial intelligence concepts to map resumes. Nice concept but has not demonstrated great success in practice.

Do These Methods Work?

http://www.burning-glass.com/

http://www.engeniumsearch.com/


Ability Tests

Predict job performance on all jobs but can cause adverse impact on protected classes; Require a confirmation scale.

Personality Trait Tests

Correlate moderately with performance for incumbents, but poorly for candidates.

Culture fit/Job Fit Tests

May or may nor correlate with performance

Do These Work?

17


-Team-based work

-Facile communications

-Interpersonal relations

-Judgement-oriented

-Improvisational

-Reliant on expertise across

multiple functions

-Dependent on fluid use of

flexible teams

-Routine work

-Formal rules, procedures, &

training

-Low discretion workers or

automation

-Judgment-oriented work

-High skill/knowledge level

-Reliant on individual

expertise & experience

-Dependent on star

performers

COMPLEXITY OF WORK

AM

OU

NT

OF

CO

LL

AB

OR

AT

ION

RE

QU

IRE

D

Routine Interpretation &

Judgment

Ind

ivid

ua

lTe

am

Transactional Experts

InnovatorsCollaborators &

Connectors

Modified from: Thomas H. Davenport, 2005

18

Different Types of Workers – Different

Quality of Hire Measures


Knowledge, Skills and Abilities (KSAs)

Role-Related Knowledge (based on experience)

Judgement (quality of decisions, success)

Past Performance (both measured and opinions)

Integrity (past activities and traits)

Fit (traits)

Roles can be Deconstructed into a Mix of

Core Components

19


Structured

Past performance

Role-Based Knowledge

KSAs

Unstructured

Risk / Background check

What Predicts Success for Transactional

Roles

20

Representative importance of factors


Structured

Role-Based Knowledge

Past Performance

Unstructured

Judgement

Citations/patents/awards

What Predicts Success for Expert or

Technical Roles?

21



Structured

Past Performance

Size of network

Volume of social media activity

Number of articles written

Role-Related Knowledge

Unstructured

Judgement

Fit

Teamwork

What Predicts Success for

Collaborator/Communicator Roles?

22



Structured

Role-Related Knowledge

Number of patents/awards

Number of ideas incorporated into products and services

Contributions that led to increased profits/sales/new customers/better customer satisfaction

Unstructured

Cognitive ability

Mentions in press

Citations

Activities/events

Personality/fit

What Predicts Success for Innovator

Roles?

23



Job Knowledge Tests

May be predictive for Transactional and Expert roles

Virtual Job Tryouts

Effective & predictive for Transactional, Expert, and Collaborator roles

Skills Testing

May be predictive for Transactional & Expert roles

Competency Profiling

May be predictive for Expert roles

Other Common Measures of QoH

24


So, Can Hire Quality be Measured

and Predicted?

YES! But. . .

Different roles require different measures of quality

Methods of determining quality are specific to roles

Correlation must equal causation

Unstructured data can be valuable but must be used along with structured data.

You must collect and analyze objective data on performance and not rely on proxies, assumptions or opinions.

Predictions are probabilities – not certainties.


By analyzing who is currently an excellent performer through data from unstructured feedback and performance reviews.

By discovering attributes, competencies and skills that lead to that performance from analyzing social networks.

By analyzing potential employees for similar characteristics via their web profiles and social media.

Emerging Ways of Predicting Candidate

Quality

26


Matching algorithms- Many providers

Accuracy? Relevance? Privacy? Discrimination?

Passive Personality Assessment- IBM Watson & others

Privacy? Relevance? Accuracy?

Chatbots- Olivia, Mya, Job Pal, Karen

Assumptions? Discrimination?

Online Assessments/games- Many providers

Privacy? Relevance? Correlation=Causation?

A Few Emerging Tools that May Help

27

But a big question: How transparent are the vendors

about their algorithms and assumptions?


Have a privacy disclosure policy that is shared with candidates and is on your career site.

Have clear definitions of what you are looking for and how you will know when you find it.

Let candidates know how you measure quality of hire.

Let candidates know why you have rejected them, especially if based on social media information.

Train recruiters in what is acceptable, reliable, and accurate.

Always use human judgement along with any AI or other assessment or prediction.

Good Predictive Analytics Practices

28


. . .use more than one test or predictive tool.

. . .when choosing a tool, know what is in the algorithm. Know what is being looked for, tested, analyzed, and how each factor is weighted.

. . .make sure proxies are valid and really predictive.

. . .not adopt tools hastily and without careful thought and knowledge.

. . .always question our own assumptions and beliefs.

You should. . .

29


30

THANKS

YOUR THOUGHTS & QUESTIONS

Follow me on Twitter: @kwheeler

Email: [email protected]

Recruiting & HR

Predictive Analytics and Quality of Hire