Behavioural event interviewing science or religion

21/07/2010 1PERSONNEL PSYCHOLOGYNZ LTD

Behavioural Event Interviewing:Science or Religion

Keith McGregor

Personnel Psychology NZ Ltdwww.ppnz.com

NZPsS Annual Conference

Rotorua - July 2010



Structured Interviews

1. Base questions on a person spec from a job analysis

2. Ask exactly the same questions of each candidate

3. Limit prompting, follow-up questioning and elaboration

4. Use better types of questions, namely;

- future situational questions (What would you do if ...?)

- past experience questions (Tell us about a time ..?)

- work history questions (What experience with ...?)

- job knowledge, competency questions (How do you?)

Campion, M., A, D. Palmer, K, et al. (1997). "A Review of Structure in the Selection Interview." Personnel Psychology 50: 655-702.



5. Use longer interviews or more questions

6. Withhold access to ancillary information

(application forms, resumes, test scores, etc.)

7. Don’t allow questions from candidates until

after the interview

8. Rate each answer using multiple scales

9. Use detailed anchored rating scales

10. Take detailed notes



11. Use multiple interviewers on panels

12. Use the same interviewers across all

candidates

13. Don’t discuss candidates or answers between

interviews

14. Provide extensive interview training

15. Use statistical methods to analyse the scores


Hypothetical Situational Questions

• Identify required competencies

• Obtain examples of critical incidents

• Create a scoring template

• Score the answers

• Appoint the person with the best score

• “What would you do if you discovered a co-worker was planning to leak information?”


Behavioural Event Questions

• Identify required competencies

• Write questions focusing on past behaviours

• Situation – Behaviour – Outcome - Motivation

• Create a scoring schedule

• Score the answers

• Appoint the person with the best score

• “Can you tell us about a time when….”


Question Sent To Candidate

Customer Service Orientation:

Tell us about a time when you recognised that

more was required than the customer (internal

or external) asked for and what you then went

on to do.

Tell us of a situation when you have had to say

‘no’ to a customer or client, what did you do?

(Taken from a 1996 questionnaire sent to a candidate prior to interview.)


Inconsistent Application70% based their questions on the person specification

14% used ad-lib questions.

43% described using a set format

70% included at least some generic questions, such as

describing strengths and weaknesses, that were not specifically job related.

28% had complete freedom in the questioning process

22% who stayed with the set questions. The balance

used a semi-structured approach.

52% indicated they used some form of rating.

Taylor, P., Y. Keelty, et al. (2002). "Evolving Personnel Selection Practices in New Zealand Organisations and Recruitment Firms." New Zealand Journal of Psychology 31(1): 8-18.


Reasons for Variation• Interviewers may be unaware of the research

• needs for power or autonomy may be frustrated by the

structured approach

• Interview may become a boring, monotonous exercise

• Applicants may be alienated by the formality of the process or may find the process frustrating

• Process may be at odds with the values and beliefs of the organisation

• There may be financial or time constraints

• Applicants perceive both the organisation and interview more positively when unstructured or semi-structured.

van der Zee, K. I., A. B. Bakker, et al. (2002). "Why are structured interviews so rarely used in personnel selection?" Journal of Applied Psychology. 87(1): 176-184.


CoachingFormal coaching and preparation for public service workers scheduled to attend a structured interview

- Improvement in interview performance- More favourable impression with interviewers

- Correlated with performance on the job

A de facto IQ test - candidates demonstrating ability to learn and apply the taught skills

Maurer, T. J., J. M. Solamon, et al. (2001). "Interviewee Coaching, Preparation Strategies, and Response Strategies in Relation to Performance in Situational Employment Interviews: An extension of

Maurer, Solamon, and Troxtel (1998)." Journal of Applied Psychology86(4): 709-717.

Maurer, T. J. and J. M. Solamon (2006). "The Science And Practice Of A Structured Employment Interview Coaching Program." Personnel Psychology 59(2): 433-456.


Impression Management

Assertive tactics - ingratiation, self-promotion

Defensive tactics - excuses, justifications, apologies

Applicants search environment for cues and tailor their self

presentation to the situation, e.g. cues inherent in the type

of question being asked

Experience based questions can evoke self promotion tactics such as claiming responsibility for successes or

enhancing responsibilities

Situational questions may lead to ingratiation by supplying answers likely to appeal to the interviewer Ellis, A. P. J., B. J. West, et al. (2002). "The Use of Impression Management Tactics in Structured Interviews: A Function of Question

Type?" Journal of Applied Psychology 87(6): 1200-1208.


The ‘Russia House’ Dilemma

“Okay? We show Bluebird the shopping list and he turns out to be their asset not ours. Have I considered that

possibility? Ned, day and night I have considered little else. If Bluebird is, if the girl is, if Barley is, if all or any of

the players is less than strictly kosher, the shopping list will shine a very bright light up the anal orifice of the

United States of America.

…It will show the Sovs what we don’t know and how we don’t know it …

Cleverly analysed, the shopping list can show them the gaps in our intelligence gathering machinery”

John Le Carre, ‘The Russia House’


Weapon of Choice

“A clever psychopath can present such a well-

rounded picture of a perfect job candidate that

even seasoned interviewers" Hare, R., D (1993). Without Conscience: The Disturbing World of the Psychopaths Among Us. New York, The Guilford Press.

“Questions are a short course for the subject on

how to lie to us”

“We cannot trust information we have to solicit”Avinoam Sapir (1988) Laboratory for Scientific Interrogation. Personal Communication.


Johari Window

UNKNOWN

BLIND

SPOT

FACADE

PUBLIC

ARENA

OTHERS

SELF

Things othersKNOW

about us

Things others DO NOT KNOW

about us

Things we KNOW

about ourselves

Things we DO NOT KNOW

about ourselves


Interview Implications

Will discuss

openly

Will not discuss

because they

are unaware

e.g. gender

bias

Will not discuss

because they do

not want you to

know e.g.

procrastination

Will not discuss

because they

are unaware

e.g. self

sabotage


Values and AttitudesUnstructured interviewing was more effective than structured interviewing in identifying counterproductive

work behaviours.

Allowing freedom to explore aspects of the interviewee’s

experience and attitudes produced more valid assessments of personality traits and produced more

information with fewer personality related questions than

did interviews which were highly structured.

The degree of structure rather than the type of questions

was the differentiating factor.

Blackman, M. C. and D. C. Funder (2002). "Effective Interview Practices for Accurately Assessing Counterproductive Traits." International Journal of Selection and Assessment 10(1&2): 109-116.


What is Being Measured?

Conventionally structured interviews (focusing on

credentials and achievements, descriptions of experience

and self evaluative information such as likes, dislikes, strengths and weaknesses) were assessing social skills

and temperament.

Behaviour structured interviews, (focusing on technical

knowledge and behaviour description) were assessing general mental ability and work knowledge.

Moscoso, S. (2000). "Selection Interview: A Review of Validity Evidence,

Adverse Impact and Applicant Reactions." International Journal of Selection and Assessment 8(4): 237-247.


Counterintuitive Hypothesis

Is increased validity of structured interview due to higher

reliability?

How many unstructured interviews equate to validity of one

structured interview?

“Perhaps the best interpretation of the mixed results in this study as that they indicate that there is evidence that is

countertheoretical to the prevailing beliefs about the structured interview and that additional research should be

conducted to understand this issue more fully.” (Bugger!)

Schmidt, F.L., & Zimmerman, R.D. (2004). A Counterintuitive Hypothesis About Employment Interview Validity and Some Supporting Evidence. Journal of Applied Psychology, 89(3), 553-561


Six Pieces of Paper

RELIABILITY Police officers need to be supportive, dependable and to make themselves available to other police officers. Please describe a situation where you had to keep going without quitting. What was the situation and the demands put upon you? What exactly did you do? What did you do to keep yourself going? How did you support others? What was the result?

“These initial results show that the six-page competency-based application form designed for police officer recruitment obtains quite similar results to other more expensive, labour-intensive selection processes.”

Lamsdale, C., R. Wood, et al. (1999). "An Alternative to an Assessment Centre on Six Pieces of Paper?" International Journal of Selection and Assessment 7(3): 170-176.


Structured Interview Problems

• Job analysis seldom evidence based and

seldom mention environmental factors

• Narrow scope of inquiry – may miss key

behaviours

• Trying to quantify a subjective judgement

• Unnatural and artificial

• Everyone is different

• Stressful (unethical?)


Does the Past Predict The Future?

• Theory states that past behaviour is the best

predictor of future behaviour

• But - we seldom have access to past behaviour

• We only have access to current behaviour

• Body language is unreliable

• Only verbal behaviour is reliable


Described

versus

DemonstratedBehaviour

• Described behaviour - what they talk about

• Demonstrated behaviour - how they talk about

it

• Demonstrated behaviour is the most reliable

indicator


What Is Demonstrated?

David, can you please give us an example of a situation

where you had to deal with an angry customer, explain what you did and what outcome resulted.

That’s easy. When I was working as a salesperson at Gifts-For-All a customer came in with this ridiculous complaint. I

mean it is amazing what some people expect. Anyway, I calmed her down and apologised for what had happened. I

told her it was a manufacturing error, it wasn’t really but that

always sounds good. I replaced her purchase and gave her a discount voucher and she was really happy. I thought it

was a good outcome and my boss said I had shown excellent customer service skills.


Structure is Essential

• Unstructured interviewing is NOT the solution

• Elicits irrelevant information

• Relies on ‘gut feel’

• Decisions difficult to justify – “his eyes were too

close together”

• The question is not ‘structured’ versus

‘unstructured’ but what kind of structure


Purpose of an Assessment

1. To enable us to predict how a candidate is most likely to behave on the job

2. To use our prediction to guide the selection decision


Objective of an Interview

The primary objective of the interview is to obtain

sufficient valid, unbiased behavioural data to

enable an accurate prediction of how the person

will behave in the workplace.


“She Interviewed Well….”

• We are not interested in how they ‘perform’ in the interview

• We are not looking for anything - we are looking to see what is there

• Must leave a legally robust ‘audit trail’ showing: – what information was obtained, – how it was analysed and – how it related to potential on-the-job

behaviour


Natural Skills

As a generalisation, most people are natural

interviewers

How would your partner / relative go as a central

city bus driver?

What questions would you ask at a party to see if

someone would fit into your work environment?


We Predict By Matching…

How the person

normally behaves

With the known

requirements of the role


The Key Elements

JobKnowledge

SkillsAttitudes

Candidate

Knowledge

Skills

Attitudes

Predictedon-the-job

performance


KSA’s(Bloom’s Taxonomy)

Knowledge (Cognitive) – the what

Skills (Psychomotor) – the how

Attitude (Affective) – the will

For every inch training technology gains, selection technology loses a mile - don’t waste time assessing things you can train


‘Natural Selection’

Humans have a natural

tendency to select OUT

rather than select IN

We really want the candidate

to be good but when all the

lights go red it’s all over


‘Incompetency’ Profile

The norm is to use a competency profile - what

we looking for. These are often generic and not

based on empirical evidence – people have

usually ‘discussed’ them into existence

Ideally, the competency profile should include an

‘Incompetency’ profile – behaviours shown by

people who have failed in the role.


The ‘Pure’ Interview

We do not need to know what the job is in order to do the interview.

A good interview will produce two main pieces of information:

– what motivates and de-motivates the candidate

– what competencies they have

Armed with this information it should then be possible to predict how the candidate will behave in any given job


The Theoretical Basis

Ivan Petrovich Pavlov Burrhus Frederic Skinner

Classical/Respondent Operant/Instrumental


Reinforcing and Aversive Stimuli

We have two poles

Things we move

TOWARDS

Things we move

AWAY FROM

DETAIL?

PEOPLE?

CONFLICT?

STATUS?


Assessing Personality

We are unable to read other people’s minds.

All we can do is observe their behaviour.

Personality can therefore be viewed as the:

• Long term (stable)

• Probability (predictable)

• of Specific (definable)

• Behaviour (observable)


Assessing Personality

There are no absolutes in human behaviour. A

person may need the company of others most of

the time but still need time alone

To assess ‘personality’ we need to be able to

assign probabilities to the behaviours

We need to recognise that the interview is an

unnatural environment – the candidate has no

competition, they are being evaluated


Sampling Behaviour

• Sampling is a cornerstone of science

• To assess personality we need an unbiased sample of behaviours

• The sampling process is systematic not selective

• Secondary school, for example, is simply a starting point, it has no special significance

• Anything can be included – sports, part time jobs, research projects, trips


Interview Structure

POSITIVES NEGATIVES

NEGATIVES

NEGATIVES

POSITIVES

POSITIVES

SKILLS

SKILLS

SKILLS


Getting Started

Start with very broad, open questions:

‘Just to get the ball rolling, tell us a bit about

yourself.’

- Do they seek clarification?

- What do they talk about?

- How long do they spend on each topic?

- How organised or orderly are they?

- Are there any ‘weirdies’?

- How long are their replies - long / medium / short?


Interview Structure

Work through entire academic and work history:

• ‘Going back to the dim dark past, what were some of the ‘highs’ and ‘lows’ of your last couple of years at school?’

• ‘After school you worked at Grange Ltd. What was that like?’

• ‘What prompted you to move on?’

• ‘While you were with Pastrycooks International, what aspects gave you the most job satisfaction?’


Drawing Out The Values

What was it

like working

there?

‘Brilliant’ ...

‘Great’ ...

‘Good’ ...

‘Okay’ ...

‘Interesting’ ...

‘Awful’

‘Ummm’


Probe the Adjectives

Everyone speaks a different language

Never assume you know how they interpret words like ‘challenging’ or ‘awful’

‘When you say ‘awful’, what sort of things are you thinking of?’


Interpretable Information

• ‘Interpretable’ information reveals values and

attitudes

• The facts of the situation are irrelevant

• Consider the responses of two people who

attended the same Outward Bound course:

1. “Terrible, disorganised, lousy food”

2. “Amazing, I learned so much about myself”


Laddering

Laddering Up (Why? Why? Why?) What was that job like? “Great, I worked in a team”

As opposed to? “Working on my own”Which do you prefer? “Being part of a team”

Because? “I know I can get help if I need it”As opposed to? “Having to work things out for myself”

You prefer help because? “I struggle to make decisions”

Laddering Down (How? How? How?)You mentioned helping others. How do you do that?“I always seem to know when people need help”

Which you achieve by?“I guess I am always looking around to see what is going

on”


Fly on the Wall

Can you get a fly-on-the-wall picture of the candidate in the workplace?

How did you find that role?

Oh, great

Great in what way?

I was the youngest rep in the history of the company

And what did you love getting your teeth into?

Winning customers of the other reps, that was a real buzz


Fly (cringing) on the Wall

What about the frustrations, if something was going to get up Dave’s nose what would it be?

Some of the other reps

Go on?

Oh, they used to play these stupid tricks, in the end I just refused to talk to them

Anything else apart from that?

Some of the customers could be difficult, I think basically they were just plain thick


Never Sell Before Deciding

• Candidates are coached to try to get the interviewer talking to gain cues about how to respond.

• Give away nothing until the end of the interview.

• If they want to know more about the job, ask them to give you their best guess.

• “Just so we don’t double up, how about you tell us what you think the key points are and we can fill in the gaps.”


Notetaking

Write as close to word-for-word as possible

• Ensures accurate analysis

• Wards off subsequent challenges

• EEO fair

• Allows the candidate to relax

• Gives interviewer time to review

• Prevents interview ‘white-out’


Assessing Skills and Abilities

• Drill down into the detail

• Use ‘closed’ or directive questions

• Get very specific information

• Ask ‘dumb’ questions

• If inexperienced solicit ‘best guess’


The CV contains the Questions

Read carefully through the C.V. and note any

activities which are close to those you require.

Use those as the basis for skill and ability

questions.

“I see here you set up a chart of accounts. What

on earth is a chart of accounts, how do you set

those up?”


‘Unconscious’ Non-Answers

Sam, could you please elaborate on your role and your job with TAP Inc., specifically the organisational climate re-engineering project.

My role was as an internal consultant employed by TAP and the role extended straight across the organisation. I had responsibility for the non-government accounts. I worked with the training department itself.

The team was directly responsible for facilitating organisational change. We were involved at the invitation of other departments. The intention was that any unit undergoing change would call us in to help. In some cases the bosses said we had to be involved.

We tried to be involved as early as possible and worked in a facilitation role. The idea was for us to be involved as soon as people recognised they needed help. We identified the problem, the real problem, and suggested solutions and in some cases we were involved in the implementation.


What did Sam actually do?

Could you give us examples - what were you involved in the whole time? And what were some of the smaller activities? How was consistency achieved across the organisation?

Our aim was to achieve consistency across the organisation by restructure and climate change. It was reorganised into customergroups. We lost about 200 people across the whole company.

For the climate change we analysed the existing situation. We prepared an analysis plan, we read up on similar programmes, we looked at the language, we held focus groups and we carried out surveys. We used a number of different methods to gather the information we needed.

There were 6 of us working on the project. I was the team leader for the data management. We developed a picture of the existing climate and from here we looked at where we wanted to be.


‘Unconscious’ Non-Answers

Sam, could you please elaborate on your role and your job with TAP Inc., specifically the organisational climate re-engineering project.

My role was as an internal consultant employed by TAP and the roleextended straight across the organisation. I had responsibility for the non-government accounts. I worked with the training department itself.

The team was directly responsible for facilitating organisational change. We were involved at the invitation of other departments. The intention was that any unit undergoing change would call us in to help. In some cases the bosses said we had to be involved.

We tried to be involved as early as possible and worked in a facilitation role. The idea was for us to be involved as soon as people recognised they needed help. We identified the problem, the real problem, and suggested solutions and in some cases we were involved in the implementation.


What did Sam actually do?

Could you give us examples - what were you involved in the whole time? And what were some of the smaller activities? How was consistency achieved across the organisation?

Our aim was to achieve consistency across the organisation by restructure and climate change. It was reorganised into customergroups. We lost about 200 people across the whole company.

For the climate change we analysed the existing situation. Weprepared an analysis plan, we read up on similar programmes, welooked at the language, we held focus groups and we carried out surveys. We used a number of different methods to gather the information we needed.

There were 6 of us working on the project. I was the team leader for the data management. We developed a picture of the existing climate and from here we looked at where we wanted to be.


Analysing the Results

• Avoid making a decision straight after the

interview - allow time for the face to fade.

• Read the notes through at least three times.

• Write out or highlight all the positive and

negative comments.

• Sort the comments into natural groupings -

‘people’, ‘learning’, ‘status’, etc.

• Prepare a competency report and a prediction

of on-the-job behaviour.


Q-Sort Positives and Negatives

�

Being made prefect

Teachers giving me praise

Winning the 400m prize

Being the youngest customer service rep in history of company

Winning the top sales prize

Winning customers off the other reps

Working for an elite company

�

Not being listened to by pupils

Missing out on 1st XV selection

Practical jokes by childish sales reps

Not being invited to Board dinner

Manager trying to tell me how to do my job

Customers kept cancelling orders

Having to explain things several times to clients


Interpreting the Results

• Work on the obvious - don’t read in things that

are not there.

• Be hypercritical - if they can’t suppress negative

traits in a 40 minute interview they won’t stand a

chance on the job.

• Focus on the behaviours demonstrated in the

interview - sarcasm, criticism, talking about people, money, results, etc.


Reporting the Results

Always prepare a written prediction of how the

person will behave in the workplace, even if it is

just for yourself. Dig it out six months later and

see how close you were.

If you are assessing against competencies use the

data you have collected as the basis for the

assessment.

Have someone else review your analysis to ensure consistency


Competency Report

On five occasions referred to people as ‘half-wits’ and idiots. Made six comments about the trouble people had put him to

Not AchievedAwareness of Others

Gave in-depth answers, reads widely and has advanced knowledge

AchievedTechnical Knowledge

Referred to teams once in 60 minutes. Resented team activities at school and in first job

Not achievedTeambuilding

EvidenceAssessmentCompetency


Summary Report

Suitable for genuine, short term contract but not for permanent employment.

Recommend-ation

He will begin well and have some early wins but then start ‘falling out’ with managers and co-workers. Will resign and blame others for his problems

Predicted performance

Initially impressed as open and responsive with good knowledge. Successfully completed three major

projects. As interview progressed started criticising and blaming others. Highly judgemental and confrontational

General Summary

Repetitive, boring work, managers, people of high rank, education system, bureaucracy, being told what to do,

co-workers

Negatives

Attention and support from others, feeling valued, being listened to, pleasant working conditions, freedom to

make his own decisions, relaxed environment

Positives


Light Reading

Blackman MC, Funder DC. (2002). Effective Interview

Practices for Accurately Assessing Counterproductive Traits. International Journal of Selection and Assessment,

10, 109-116.

Campion MA, Palmer DK, Campion JE (1997) A Review of

Structure in the Selection Interview. Personnel Psychology,

50, 655-702

Thompson, D. W.: Managing People: Influencing

Behaviour. St Louis, Miss, 1978, C.V. Mosby. Pp 101-118


Finally – Getting to the Truth


About PPNZPersonnel Psychology NZ Ltd provides a comprehensive range of industrial and

organisational psychology services.

• Specialist training (e.g. ‘The Psychology of Management’, ‘Advanced Interview Skills’ and ‘Managing Organisational Change’)

• In-depth executive appointment screening

• Individual psychological assessment and counselling for executives

• Mentoring, coaching and supervision

• Organisational climate surveys and stress audits

• Psychometric test development ( e.g. Selector PA, Career Step, E Profiler)

• Safety management

The director, Keith McGregor, is an industrial psychologist based in Lower Hutt,

New Zealand. His background includes twelve years as an occupational

psychologist in the Royal New Zealand Air Force and twenty in private practice

working with a wide range of private and public sector organisations in New

Zealand and Australia.

Education

Behavioural event interviewing science or religion