Getting Started With Evidence-Based HR

Getting started with Evidence-Based Management

Dublin, April 28th, 2016

Evidence-Based HR: is it ‘a thing’?

25 min EBMgt: What is it and why you you need it?

20 min A practical example

Agenda

Evidence based management:What is it?

Evidence-based managementCentral Premise:

Decisions should be based on the ‘best available evidence‘.

Evidence?

information, facts or data supporting (or contradicting) a claim, assumption or hypothesis

Evidence?

outcome of scientific research, organizational facts & figures, benchmarking, best practices,

personal experience

All managers and leaders base their decisions on ‘evidence’

But…many managers pay little or no attention to

the quality of the evidence they base their decisions on

and use too few sources of evidence

Trust me, 20 years of management experience

Sources of evidence

problem solution

Practitionersprofessional expertise

Organization internal data

Stakeholdersvalues and concerns

Scientific literature empirical studies

AskAcquire

AppraiseAggregate

ApplyAssess

Evidence based practice:Where does it come from?

McMaster University Medical School, Canada

Medicine: Founding fathers

David Sackett Gordon Guyatt

How it all started

1. Ask: translate a practical issue into an answerable question

2. Acquire: systematically search for and retrieve the evidence

3. Appraise: critically judge the trustworthiness of the evidence

4. Apply: incorporate the evidence into the decision-making process

5. Assess: evaluate the outcome of the decision taken

5 steps of EBmed

Evidence-Based Practice

1991Medicine

1998Education

2000Social care, public policy

Nursing, Criminal justice,

Policing, Architecture, Conservation

2010Management



Evidence-based … whatever =

the use of evidence from multiple sources to increase the likelihood of a

favourable outcome

Focus on the decision making processThink in terms of probability

In general: managers don’t like EBMgt

Undermines formal authority

They feel it constrains freedom to make managerial decisions

Speed valued and rewarded more than accuracy

Feel they cannot use their own experience and judgment (not true)

Managers not necessarily rewarded for doing what works (organizations rarely evaluate)

THEY LOVE FADS & QUICK FIXES

Why don’t managers like EBMgt?

32

Evidence-Based Decision-Making Why do we need it?

Advice: lie babies down to sleep on their belly(unanimous support through to the 1990s)

Example: medicine

Nr of cot deaths (Holland)

Collateralized Debt Obligations > AAA

p = 0.12 (about 1 chance in 850) default in 5 years

Example: finance

Forecasted Actual

Forecasted and actual 5-year default rates for AAA-rated CDO tranches

0.12%

28%

Scared straight

Example: policy / prevention

Example: HR management

1. Incompetent people benefit more from feedback than highly competent people.

2. Task conflict improves work group performance while relational conflict harms it.

3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals.

Likely or unlikely?

ALL NOT LIKELY !

How evidence-based are HR managers?

959 (US) + 626 (Dutch) HR professionals 35 statements, based on an extensive body of

evidence true / false / uncertain

HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)

Outcome: not better than random chance

Relying on only 1 source: bad idea!

problem solution





AskAcquire

AppraiseAggregate

ApplyAssess

Discuss with your neighbor (1 min)

Over a 5 year period,

why is an orthopedic surgeon's experience, as a rule, more trustworthy than an change manager’s experience?

0

Developing expertise

1. A sufficiently regular, predictable environment

2. Numerous opportunities to practice

3. Receive accurate (objective) feedback

The management domain is not highly favorable to expertise!

Learning from feedback is hard!

Bounded rationality

How your brain works

System 1 Fast Intuitive, associative heuristics & biases emotional

System 2

Lazy Slow Deliberate Rational

dominant

System 1: short cuts

System 1 or system 2?

10 seconds

System 1 or system 2?

A bat and a ball cost $1.10 in total.

The bat costs $1 more than the ball

How much does the ball cost?

0

System 1: necessary to survive

95%

Pattern recognition Overconfidence bias Halo effect False consensus effect Group think Self serving attribution bias Sunk cost fallacy Cognitive dissonance reduction

System 1: prone to cognitive errors

Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Availability bias

Pattern recognition Overconfidence bias Halo effect False consensus effect Group think Self serving attribution bias Sunk cost fallacy Cognitive dissonance reduction

System 1: prone to cognitive errors

Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Availability bias

When it is your job to make decisions,

you need to know how your brain works

when making decisions!

“I’ve been studying judgment for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over-

confident. I fall for every one of the biases.”





AskAcquire

AppraiseAggregate

ApplyAssess

Four sources of evidence (not only 1)

The performance of knowledge workers

A Practical Example

550 beds

3300 employees

210 medical specialists

225,000 admissions

Top Clinical & Teaching hospital

Organization

2015: 7.2 2016: 6.3

How can we increase job satisfaction and employee engagement?

Dear HR department,

Evidence-based approach, step 1: ASK

problem solution





AskAcquire

AppraiseAggregate

ApplyAssess

What is the problem?

Why is this a problem: what are its organizational consequences?

How big: what is its impact on the organization when nothing is done?

Why does this problem exist, what is the assumed major cause?

What is the assumed causal mechanism? How does the cause lead to the problem and its consequences?

Step 1: What is the problem?

problem & underlying cause

hidden assumptions

causal mechanism

Some terminology

A happy & engaged employee is a productive employee

Fundamental assumption

What is the evidence for this assumption?

Where multiple sources consulted?

How trustworthy is the evidence?

Step 2: What is the evidence?

Let’s have a look

Professional experience and

judgment

Organizational data, facts and figures

Stakeholders’ values and concerns

Scientific research

outcomes

AskAcquire

AppraiseApply

Assess

problem solution

… The effect of job attitudes (e.g. satisfaction, commitment)

on performance was weak (β = .06) ….

… The relationship was almost eliminated after controlling for

personality traits …. and self esteem.

GREAT! NOW WHAT?

Outcome

Evidence-based managers, please

Evidence-based managers, please

Step 1: ASK

Translate a practical issue into an answerable question

Population? Knowledge workers!

Whether nurses, lawyers, engineers, managers, or staff members, nowadays most workers in organizations are highly dependent on information and communication technology and are involved in

work that involves a high level of cognitive activity.

Question

“Which of the factors that are related to the

performance of knowledge workers are most

widely studied and what is known of their

effect?”

Step 2: ACQUIRE

Search for the best available scientific evidence

ABI, BSP, PsycINFO

Scholarly journals, peer reviewed

1980 – 2013

English

performance, productivity, knowledge work*

ACQUIRE

step 3: APPRAISE & AGGREGATE

Effect size?

Largest effect

1. Social cohesion .5 / .7

2. Perceived supervisory support .5

3. Information sharing / TM.5

4. Vision / goal clarity.5

5. Trust.3 / .6

step 3b: CROSS VALIDATE

Step 4: APPLY

Three examples

social cohesion supervisory support

information sharing

Social cohesion

Social cohesion

… a shared liking or team attraction that includes bonds of friendship, caring,

closeness, and enjoyment of each other’s company.

Social cohesion

Measuring social cohesion

Perceived supervisory support

…how employees feel the supervisor helps them in times of need, praises

them for a job well done or recognizes them for extra effort.



Measuring perc. sup. support

Information sharing

Information sharing?

…refers to how teams pool and access their knowledge and expertise – which positively

affects decision making and team processes. This has led to the idea of a team ‘Transactive

Memory System’ (TMS), which can be thought of as a collective memory in a collective mind - enabling a team to think and act together

Information sharing

Measuring information sharing

Outcome

The departments with the lowest performance scored under average on most factors

Reactions

Who knew?

Evidence-based … whatever =

the use of evidence from multiple sources to increase the likelihood of a

favourable outcome

Focus on the decision making processThink in terms of probability

Multiple sources of evidence

problem solution





AskAcquire

AppraiseAggregate

ApplyAssess

Postgraduate Course

www.cebma.org

Postgraduate Course

> 80 Fellows

CEBMa: what we do

Promote (seminars, papers, blogs, tweets)

Educate (universities & business schools)

Train & coach (companies > projects)

Support / REAs (companies)

Support / 2nd opinion (BS detector)