View
48
Download
3
Category
Preview:
DESCRIPTION
Dr. Steve Yule, STRATUS Center for Medical Simulation, Boston, USA, presented "Developing and Evaluating Performance in Simulator Scenarios" at Simulation in Irish Medical Education: Where Are We, and Where Are We Going? held at NUI Galway on the 2nd July 2012.
Citation preview
Steven Yule PhD STRATUS Center for Medical Simulation
Brigham & Women’s Hospital, Boston
Department of Surgery, Harvard Medical School
2nd July 2012
Developing scenarios and evaluating performance in medical simulation
The challenge: Defining how learners should act, think or feel differently as a result of your simulation program
1. Basics of scenario development
2. Performance evaluation
Basics of scenario development
Checklist for designing a simulation scenario
① Define aim
② Needs assessment & learning objectives
③ Learning outcomes (for evaluation)
④ Develop a matrix
⑤ Work on evaluation strategy
⑥ Identify resources and people required
⑦ Programming and software
⑧ Dry run
⑨ Pilot with real participant
⑩ Run program
Performance evaluation
To successfully evaluate learners, you need to complete these three tasks as part of the scenario design process:
#1 Consider the evaluation options
#2 Review learning objectives
#3 Select strategy
Task # 1: Consider evaluation options • Written record
• Log
• Attendance
• Case Presentations
• Direct Observations
• Written exam
• Oral examination
» Structured (oral patient management problem)
» Short answer
• Videotape and audio tape review
• Simulated Patient
• Objective Structured Clinical Examination
Task # 2: Review learning objectives and select the most appropriate evaluation method according to these criteria:
Practical
Economical
Objective
Comprehensive
Valid
Reliable
Feasibile
Task # 3: Select strategy and implement
• How are the data to be collected?
• By what mechanism or by who?
For example:
• “Leadership behaviors will be assessed during the
scenario from video by blinded expert observers
using the Surgical Leadership Inventory”
• “Confidence in maintaining an airway during an
anaesthetic crisis will be measured by pre- and post-
course survey”
A worked example…
Categories
Elements
Situation Awareness Gathering Information Understanding Information Projecting and anticipating future state
Decision Making Considering options Selecting and communicating option Implementing and reviewing decisions
Communication and Teamwork
Exchanging information Establishing a shared understanding Co-ordinating team
Leadership Setting and maintaining standards Coping with pressure Supporting others
NOTSS skills taxonomy v1.2
Yule et al (2006) Medical Education, 40, 1098-1104
NOTSS rating form
Simulation coaching
NOTSS coaching significantly improved skills in a randomized controlled trial of 16 senior surgical trainees
Simulation-based team training (2012-2015)
Designing good simulation scenarios is impossible without clear learning outcomes and an evaluation strategy: summary
1. Decide what you want people to do differently after your session
2. Use a structured method of scenario design
3. Select most appropriate evaluation method for learning outcomes using criteria
Contact » syule@partners.org » http://scholar.harvard.edu/yule/
Recommended