18
Redefining Simulator Redefining Simulator Proficiency Using Proficiency Using Automaticity Theory Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD Scerbo, PhD James R Korndorffer Jr, MD, Daniel J James R Korndorffer Jr, MD, Daniel J Scott, MD Scott, MD Carolinas Medical Center, Charlotte, NC Carolinas Medical Center, Charlotte, NC Old Dominion University, Norfolk, VA Old Dominion University, Norfolk, VA Tulane University, New Orleans, LA Tulane University, New Orleans, LA UT Southwestern Medical Center, Dallas, TX UT Southwestern Medical Center, Dallas, TX

Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Embed Size (px)

Citation preview

Page 1: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Redefining Simulator Redefining Simulator Proficiency Using Proficiency Using

Automaticity Theory Automaticity Theory

Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhDDimitris Stefanidis, MD, PhD, Mark W Scerbo, PhDJames R Korndorffer Jr, MD, Daniel J Scott, MDJames R Korndorffer Jr, MD, Daniel J Scott, MD

Carolinas Medical Center, Charlotte, NCCarolinas Medical Center, Charlotte, NCOld Dominion University, Norfolk, VAOld Dominion University, Norfolk, VATulane University, New Orleans, LA Tulane University, New Orleans, LA

UT Southwestern Medical Center, Dallas, TXUT Southwestern Medical Center, Dallas, TX

Page 2: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

BackgroundBackground Training to proficiency on simulators Training to proficiency on simulators

improves operative performanceimproves operative performance1,21,2

Proficiency levels are derived from Proficiency levels are derived from expert performanceexpert performance

Most common performance metrics Most common performance metrics used are time and errors used are time and errors

Expertise is, however, not well Expertise is, however, not well defined in surgerydefined in surgery33 and other metrics and other metrics may more accurately assess may more accurately assess performanceperformance

1 Korndorffer Jr JR, JACS 20052 Seymour NE, Ann Surg 20023 Ericsson KA, Acad Med 2004

Page 3: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

BackgroundBackground Automaticity refers to the habitual Automaticity refers to the habitual

performance of a task without performance of a task without significant demands on attentionsignificant demands on attention11

Expert performance is characterized Expert performance is characterized by automaticity; it confers them the by automaticity; it confers them the ability to multi-taskability to multi-task

A secondary task performed A secondary task performed simultaneously with the primary task simultaneously with the primary task can be used as an index of expertise can be used as an index of expertise by measuring multi-tasking ability by measuring multi-tasking ability

1 Schiffrin & Schneider, Psychol Rev 1977

Page 4: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

BackgroundBackground

Secondary tasks must be sensitive, Secondary tasks must be sensitive, selective and unobtrusiveselective and unobtrusive11

They must compete for resources They must compete for resources that are common to the primary task that are common to the primary task

Laparoscopy imposes heavy visual-Laparoscopy imposes heavy visual-spatial demands on the surgeonspatial demands on the surgeon

A visual-spatial secondary task that A visual-spatial secondary task that draws from the same attentional draws from the same attentional resources is ideal resources is ideal

1 O’Donnell & Eggemeier, 1986

Page 5: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Study ObjectiveStudy Objective

To assess whether a visual-spatial To assess whether a visual-spatial task that measures attentional task that measures attentional spare capacity can distinguish spare capacity can distinguish among subjects of variable among subjects of variable expertise in laparoscopic suturingexpertise in laparoscopic suturing

Page 6: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

MethodsMethods

IRB- approved protocolIRB- approved protocol 12 Participants: 4 novices, 3 surgery 12 Participants: 4 novices, 3 surgery

residents, 3 laparoscopy experts, and residents, 3 laparoscopy experts, and 2 novices trained to proficiency in 2 novices trained to proficiency in laparoscopic suturinglaparoscopic suturing

Participants had to perform under dual-Participants had to perform under dual-task conditions for 10 minutestask conditions for 10 minutes• Laparoscopic suturing and knot tying on a Laparoscopic suturing and knot tying on a

videotrainer simulator using the FLS modelvideotrainer simulator using the FLS model• Visual-spatial secondary task performanceVisual-spatial secondary task performance

Page 7: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Video - Primary taskVideo - Primary task

ExpertExpert NoviceNovice

Page 8: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Video - Secondary taskVideo - Secondary task

Page 9: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Video - Dual TaskingVideo - Dual Tasking

Page 10: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

MetricsMetrics Laparoscopic SuturingLaparoscopic Suturing

• Objective score (=300-[time+10*errors])Objective score (=300-[time+10*errors]) Time (max 5 min per repetition)Time (max 5 min per repetition) ErrorsErrors

• AccuracyAccuracy• Knot SecurityKnot Security

Secondary taskSecondary task• Percent correct detections Percent correct detections

When performed aloneWhen performed alone When performed simultaneously with suturing When performed simultaneously with suturing

Page 11: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Statistical AnalysisStatistical Analysis

Kruskal-Wallis One Way Kruskal-Wallis One Way Analysis of Variance on Analysis of Variance on Ranks for the four groups Ranks for the four groups with pairwise comparisons with pairwise comparisons (Dunn’s method)(Dunn’s method)

Page 12: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

ResultsResults

Experts:>200 basic and >100 advanced Experts:>200 basic and >100 advanced laparoscopic cases, extensive experience laparoscopic cases, extensive experience with simulatorwith simulator

Novices: no prior laparoscopy or simulator Novices: no prior laparoscopy or simulator experienceexperience

Trained: achieved proficiency on Trained: achieved proficiency on simulator, no operative experiencesimulator, no operative experience

Residents: <100 basic, <30 advanced Residents: <100 basic, <30 advanced laparoscopic cases, familiar with simulator laparoscopic cases, familiar with simulator

Page 13: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

ResultsResults All achieved a perfect score on secondary All achieved a perfect score on secondary

task when performed alone (100% correct task when performed alone (100% correct detections)detections)

Secondary task performance deteriorated Secondary task performance deteriorated in all groups when performed in all groups when performed simultaneously with primary task simultaneously with primary task (p<0.001)(p<0.001)

Experts and trained outperformed Experts and trained outperformed residents and novices on suturing taskresidents and novices on suturing task

While experts did not differ from trained While experts did not differ from trained based on time and errors, they achieved based on time and errors, they achieved better secondary task scores (p=n.s.)better secondary task scores (p=n.s.)

Page 14: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Repetition duration

Experts Trained Residents Novices

Tim

e (s

ec)

0

50

100

150

200

250

300

p<0.001

Page 15: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Primary Task Performance

Experts Trained Residents Novices

Sco

re

0

50

100

150

200

250

300

p<0.001

Page 16: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

Secondary Task Performance

Experts Trained Residents Novices

% C

orre

ct D

etec

tions

0

20

40

60

80

100

p<0.03

Page 17: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

ConclusionsConclusions A visual-spatial secondary task that assesses A visual-spatial secondary task that assesses

attentional spare capacity may distinguish attentional spare capacity may distinguish among levels of laparoscopic expertise among levels of laparoscopic expertise (construct validity) when standard (construct validity) when standard performance measures fail to do soperformance measures fail to do so

Such secondary task metrics may more Such secondary task metrics may more accurately define expert performance for use accurately define expert performance for use as training endpoints during simulator as training endpoints during simulator curricula and possibly for assessment curricula and possibly for assessment purposespurposes

Further validation of secondary task metrics Further validation of secondary task metrics and automaticity is warranted and currently and automaticity is warranted and currently underwayunderway

Page 18: Redefining Simulator Proficiency Using Automaticity Theory Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhD James R Korndorffer Jr, MD, Daniel J Scott,

??