Upload
lorraine-gallagher
View
216
Download
2
Embed Size (px)
Citation preview
Redefining Simulator Redefining Simulator Proficiency Using Proficiency Using
Automaticity Theory Automaticity Theory
Dimitris Stefanidis, MD, PhD, Mark W Scerbo, PhDDimitris Stefanidis, MD, PhD, Mark W Scerbo, PhDJames R Korndorffer Jr, MD, Daniel J Scott, MDJames R Korndorffer Jr, MD, Daniel J Scott, MD
Carolinas Medical Center, Charlotte, NCCarolinas Medical Center, Charlotte, NCOld Dominion University, Norfolk, VAOld Dominion University, Norfolk, VATulane University, New Orleans, LA Tulane University, New Orleans, LA
UT Southwestern Medical Center, Dallas, TXUT Southwestern Medical Center, Dallas, TX
BackgroundBackground Training to proficiency on simulators Training to proficiency on simulators
improves operative performanceimproves operative performance1,21,2
Proficiency levels are derived from Proficiency levels are derived from expert performanceexpert performance
Most common performance metrics Most common performance metrics used are time and errors used are time and errors
Expertise is, however, not well Expertise is, however, not well defined in surgerydefined in surgery33 and other metrics and other metrics may more accurately assess may more accurately assess performanceperformance
1 Korndorffer Jr JR, JACS 20052 Seymour NE, Ann Surg 20023 Ericsson KA, Acad Med 2004
BackgroundBackground Automaticity refers to the habitual Automaticity refers to the habitual
performance of a task without performance of a task without significant demands on attentionsignificant demands on attention11
Expert performance is characterized Expert performance is characterized by automaticity; it confers them the by automaticity; it confers them the ability to multi-taskability to multi-task
A secondary task performed A secondary task performed simultaneously with the primary task simultaneously with the primary task can be used as an index of expertise can be used as an index of expertise by measuring multi-tasking ability by measuring multi-tasking ability
1 Schiffrin & Schneider, Psychol Rev 1977
BackgroundBackground
Secondary tasks must be sensitive, Secondary tasks must be sensitive, selective and unobtrusiveselective and unobtrusive11
They must compete for resources They must compete for resources that are common to the primary task that are common to the primary task
Laparoscopy imposes heavy visual-Laparoscopy imposes heavy visual-spatial demands on the surgeonspatial demands on the surgeon
A visual-spatial secondary task that A visual-spatial secondary task that draws from the same attentional draws from the same attentional resources is ideal resources is ideal
1 O’Donnell & Eggemeier, 1986
Study ObjectiveStudy Objective
To assess whether a visual-spatial To assess whether a visual-spatial task that measures attentional task that measures attentional spare capacity can distinguish spare capacity can distinguish among subjects of variable among subjects of variable expertise in laparoscopic suturingexpertise in laparoscopic suturing
MethodsMethods
IRB- approved protocolIRB- approved protocol 12 Participants: 4 novices, 3 surgery 12 Participants: 4 novices, 3 surgery
residents, 3 laparoscopy experts, and residents, 3 laparoscopy experts, and 2 novices trained to proficiency in 2 novices trained to proficiency in laparoscopic suturinglaparoscopic suturing
Participants had to perform under dual-Participants had to perform under dual-task conditions for 10 minutestask conditions for 10 minutes• Laparoscopic suturing and knot tying on a Laparoscopic suturing and knot tying on a
videotrainer simulator using the FLS modelvideotrainer simulator using the FLS model• Visual-spatial secondary task performanceVisual-spatial secondary task performance
Video - Primary taskVideo - Primary task
ExpertExpert NoviceNovice
Video - Secondary taskVideo - Secondary task
Video - Dual TaskingVideo - Dual Tasking
MetricsMetrics Laparoscopic SuturingLaparoscopic Suturing
• Objective score (=300-[time+10*errors])Objective score (=300-[time+10*errors]) Time (max 5 min per repetition)Time (max 5 min per repetition) ErrorsErrors
• AccuracyAccuracy• Knot SecurityKnot Security
Secondary taskSecondary task• Percent correct detections Percent correct detections
When performed aloneWhen performed alone When performed simultaneously with suturing When performed simultaneously with suturing
Statistical AnalysisStatistical Analysis
Kruskal-Wallis One Way Kruskal-Wallis One Way Analysis of Variance on Analysis of Variance on Ranks for the four groups Ranks for the four groups with pairwise comparisons with pairwise comparisons (Dunn’s method)(Dunn’s method)
ResultsResults
Experts:>200 basic and >100 advanced Experts:>200 basic and >100 advanced laparoscopic cases, extensive experience laparoscopic cases, extensive experience with simulatorwith simulator
Novices: no prior laparoscopy or simulator Novices: no prior laparoscopy or simulator experienceexperience
Trained: achieved proficiency on Trained: achieved proficiency on simulator, no operative experiencesimulator, no operative experience
Residents: <100 basic, <30 advanced Residents: <100 basic, <30 advanced laparoscopic cases, familiar with simulator laparoscopic cases, familiar with simulator
ResultsResults All achieved a perfect score on secondary All achieved a perfect score on secondary
task when performed alone (100% correct task when performed alone (100% correct detections)detections)
Secondary task performance deteriorated Secondary task performance deteriorated in all groups when performed in all groups when performed simultaneously with primary task simultaneously with primary task (p<0.001)(p<0.001)
Experts and trained outperformed Experts and trained outperformed residents and novices on suturing taskresidents and novices on suturing task
While experts did not differ from trained While experts did not differ from trained based on time and errors, they achieved based on time and errors, they achieved better secondary task scores (p=n.s.)better secondary task scores (p=n.s.)
Repetition duration
Experts Trained Residents Novices
Tim
e (s
ec)
0
50
100
150
200
250
300
p<0.001
Primary Task Performance
Experts Trained Residents Novices
Sco
re
0
50
100
150
200
250
300
p<0.001
Secondary Task Performance
Experts Trained Residents Novices
% C
orre
ct D
etec
tions
0
20
40
60
80
100
p<0.03
ConclusionsConclusions A visual-spatial secondary task that assesses A visual-spatial secondary task that assesses
attentional spare capacity may distinguish attentional spare capacity may distinguish among levels of laparoscopic expertise among levels of laparoscopic expertise (construct validity) when standard (construct validity) when standard performance measures fail to do soperformance measures fail to do so
Such secondary task metrics may more Such secondary task metrics may more accurately define expert performance for use accurately define expert performance for use as training endpoints during simulator as training endpoints during simulator curricula and possibly for assessment curricula and possibly for assessment purposespurposes
Further validation of secondary task metrics Further validation of secondary task metrics and automaticity is warranted and currently and automaticity is warranted and currently underwayunderway
??