AMEE presentation 2011 Junior Doctor engagement with elearning tool

Engagement with an e-learning tool during an Emergency Department induction programme: Can it predict junior doctor performance?

Dr. Damian RolandNIHR Doctoral Research Fellow

A question…

Medical students' compliance with simple administrative tasks and success in final examinations

Care Free Hospitals NHS Trust

Identification Badge

Medical Student: T.Ardy

Placement: Paediatrics

Start date: 1st August 2011

Photograph Date: 4th October 2011

Neil Wright, M S Tanner BMJ 2002;324:1554-1555 doi:10.1136/bmj.324.7353.1554

Related ResearchDisorganized junior doctors fail the MRCP

(UK) Stanley, A. Medical Teacher, Vol. 28, No. 1, 2006, pp. e40–e42

Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance

Kibble, J. Adv Physiol Educ 31: 253–260, 2007; doi:10.1152/advan.00027.2007.

The use of formative online quizzes to enhance class preparation and scores on summative exams

Dobson, J. Adv Physiol Educ 32: 297–302, 2008; doi:10.1152/advan.90162.2008.

The REMIT Project

Refining Evaluation Methodologies for IntervenTions that change practice

Junior Doctors Induction36 New Starters

in an Emergency Department

“Un-enforced” Mandatory requirements

Various Metrics applied

Junior Doctors InductionPerformance in

initial Paediatric MCQ

Use of “Spotting the Sick Child”

Number of Patients seen in first two months

Consultant spot-check assessment:

Junior Doctors InductionDarzi

Performing well above expected standard

GSP

Good solid provider

HDU

High Dependency – concern expressed and fed-back to Clinical/Educational Supervisor

ITU

Intensive Care – intervention required

Key FindingsSupervisor

Assessment9 Darzi21 GSP5 HDU1 Didn’t start

Range of Scores 131-171

Spotting the Sick Child Use

24 No log-in

4 No log-in longer than 2 minutes

8 Log-in consistent with some activity

Patients seen in 1st Two months

All Patients 0-15 Years 0-5 Years

Number seen[Min (IQ Range) Max]

115 (179-239) 366

17 (28-56) 117

6 (16-34) 63

Discharge Rate 30.8% - 55.7% 36.8%-91.7% 35.7%-96.2%

All Patients0

50

100

150

200

250

300

350

400

Min ... Age 0-150

20

40

60

80

100

120

140

Age 0-50

10

20

30

40

50

60

70

Risk FactorsDoctors who didn’t undertake

MCQRelative Risk of being HDU/ITU:

8 (CI 2.4 – 27)

Doctors who undertook <2mins SSCNo doctor who was HDU/ITU completed more than 2 mins

Discussion

Trends continue towards engagement being linked in some way to performance

What is a gold standard of “engagement”?

What is the best measure of “performance”?

Can we separate “performance” and “professionalism”?

ThanksDr Ffion DaviesDr. Holger WahlDr. Monica LakhanpaulNic BlackwellProf. Tim CoatsNIHR (the views expressed in this

presentation may not be reflective of the NIHR)

My wife who has let me come to this event