Perceptions and observations of coaching

A.P.B. Alken, E.C.T.H. Tan, J-M. Luursema, C.R.M.G. Fluit, H. van Goor

Radboud University Nijmegen Medical Center, the Netherlands

• The Definitive Surgical Trauma Care course Master class for trauma management

• Learning objective Treatment of life-threatening injuries by operating team

• Six complex scenarios Abdominal and thoracic trauma

• Live tissue training highly realistic

• Coaching by faculty

Introduction – the DSTC course

1. How did trainees and faculty perceive the

coaching?

2. How congruent are these perceptions

with observed findings?

Study purpose

• Faculty self reports vs. trainee questionnaires Faculty report higher satisfaction 1,2

• Perceptions vs structured observations No research

Background – prior research

1. Levinson et al. J Grad Med Educ 2010;2(1):31-36.2. Yarris et al. Acad Emerg Med 2009;16(2):S76–81..

• DSTC training 11 teams; per team: 1 faculty and 2 trainees

• Faculty self reports and trainee questionnaires 12 items 1 – 6 (absolutely not/never - absolutely yes/all the time)

Example:

“I would describe the feedback as being focused on corrections”

“The coaching was directed at the way trainees communicated in the team”

• Observations 45 minutes of identical scenarios

Methods

• Observational instrument 3

Methods – observational instrument

Technical skills Communication skills

Team skills

Reinforced Reinforced Reinforced

Exp

lana

tion

how

Not

at

all

Exp

lana

tion

why

How

and

why

Exp

lana

tion

how

Not

at

all

Exp

lana

tion

why

How

and

why

Exp

lana

tion

how

Not

at

all

Exp

lana

tion

why

3. Presented at ASE congress 2012

Corrective feedback Positive feedback Instruction

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

why

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Not

at

all

How

and

why

Exp

lana

tion

why

Exp

lana

tion

how

Reinforced Reinforced Reinforced

Not

at

all

How

and

why

• 7 of 12 items matched with observation

1. Corrective feedback

2. Positive feedback

3. Technical skills

4. Communication skills

5. Team skills

6. Explanations why correct

7. Explanations why incorrect

Methods – matching variables

• Comparison

• Faculty perceptions – trainee perceptions

• Faculty perceptions – independent observations

• Trainee perceptions – independent observations

• Statistical analysis Mean and median scores Wilcoxon signed rank tests Correlations

Methods - statistics

• Faculty vs. trainees

Results - perceptions

Faculty means Trainee means

Corrective feedback 4.6 5.1*

Positive feedback 5.0* 4.4

Coaching timed well 4.4 5.3*

Reinforcements why correct 4.7 5.2*

Reinforcements why incorrect 3.9 4.9*

• Perceptions vs 45 min observations Mean coaching activities per faculty: 30 times

Faculty vs. observations: coaching on communication skills: ρ = .6

Results – perceptions vs. observations

Faculty ratings6-point scale

Trainee ratings6-point scale

Observed count

Corrective feedback 4.6 5.1 2

Positive feedback 5.0 4.4 2

Instructions 26

Technical skills 4.6 4.8 28

Communication skills 4.4 4.7 2

Team skills 3.8 4.6 0.3

Expl. why correct 4.6 5.2 2

Expl. why incorrect 3.9 4.9 0

Not reinforced or how 28

• Differences in coaching perceptions faculty and trainees

• Discrepancy between observation and perception in type of coaching and level of reinforcement

Conclusions

• Overall satisfaction may have contributed 4

• Faculty rate to the ideal level 5

• Perception questionnaires not always effective

Discussion

4. Palter et al. Surg Endosc 2010; 24: 2830-2834.5. Moorthy K et al. Am J Surg 2006;192:114-118.

Thank you!Thank you!