OVERVIEW OF PROJECT
6 STAGES
WP1 MANAGEMENT WP2 DEVELOPMENT OF IAM WP3 PILOT OF ICT BASED SIM ADOPTION WP4 IMPACT MAXIMISATION WP5 GENERAL DISSEMINATION WP6 QUALITY AND EVALUATION MX
WP2 IMPACT ASSESSMENT MODEL
AUTHORS Ulf-Daniel Ehlers Tatiana Shamarina Heidenreich
41 pages long!!!
The model was designed according to 4 cornerstones1. Identifying criteria to describe impact of simulation training2. Identifying success factors and indicators of this3. Defining learning phases to follow during modelling
activities4. The selection of a set of standards
MUST HAVE A STEP BY STEP DESCRIPTION OF HOW TO USE IT
WP2 IMPACT ASSESSMENT MODEL
1. Identifying criteria to describe impact of simulation training*
2. Identifying success factors and indicators of this3. Defining learning phases to follow during modelling
activities4. The selection of a set of standards
*From literature, partners practice, requirements of model (from they conclude model IAM has to represent teaching
and learning process as a flow of phases)
From, this the PRIME model of learning has been adapted
WP2 IMPACT ASSESSMENT MODEL
They conclude that:
“The adoption of the IAM can improve current organisational practices and support innovation in the health care sector”
“Also, model can be used to understand how medical students and employees develop and exploit their competences”
WP2 IMPACT ASSESSMENT MODEL
This 1st IAM will be refined based on validation results
Objectives of this document:
Define SBT and methods and chose one model for SIMBASE
Analyse parameters which influence success / failure
Define methodology to analyse the way SIM contributes to the learning process
Define quality of learning process
WP2 IMPACT ASSESSMENT MODEL
Current methods available to analyse SIM training
Kilpatrick model Miller model PRIME model Process model ISO/IEC 19796-1 model Portfolio methods Assessment methods
WP2 IMPACT ASSESSMENT MODEL
Kilpatrick model Assesses response and impact of an educational
exercise
Miller Model Pyramid of competence, useful foot mapping
assessment methods
Prime Model The one that they have followed closely
PRIME MODEL
Level 1-7 1 = Participants 2 = Assessment of educational activities 3 = Knowledge acquisition and attitudinal change 4 = Competence 5 = Performance 6 = Patient Health 7 = Community Health
WP2 IMPACT ASSESSMENT MODEL
WP2 IMPACT ASSESSMENT MODEL
Process Model
To establish details of quality development
Model can be used for design and structure of training
Can be divided into: Creating context specific quality profile Specifying the individual process descriptions
WP2 IMPACT ASSESSMENT MODEL
They use this evidence to present 2 models for simulation
Model 1 – Process model, to understand how to plan
Model 2 – PRIME model, how to characterize competency and show different levels of learning different project participants can how
Adapted to the aims of the SIMBASE project
WP2 IMPACT ASSESSMENT MODEL PROCESS adapted model
Planning phase
Design Phase
Implementation phase
Evaluation
WP3 PILOTING GUIDE
Outlines how the IAM can be implemented (43 pages!!)
Aims of WP3
Test IAM as a strategy design and assessment instrument
Define patient simulation training strategies
Implement priority actions from these
Access impact of actions on health care systems
WP3 PILOTING GUIDE
Methodology and timetables for pilots
Tools for recording the results of pilots
Types and methods for analysis
Guidelines for piloting co-ordinators in each location
Detailed success criteria for determining models suitability
WP3 PILOTING GUIDE
BEFORE (Planning & design phase of process model)
DURING (3RD Phase of process model)
AND AFTER TRAINING
WP3 PILOTING GUIDE
Before training:
Detect training requirements
Appoint panel of experts (5-10) Arrange initial meeting Supply panel with details of discussion before
meeting Panel during meeting to be given 5-15
questions Higher the number of participants the fewer the
questions
WP3 PILOTING GUIDE
Before training:
Example questions include:“In your opinion what are the main problems with…?”
“If you had to select three aspects that urgently require
improvement, what would they be”
“What, in your opinion, is the principle requirement of the patients with require to….?”
WP3 PILOTING GUIDE
Before training:
Panel should be experts /have extensive knowledge of subject
The panel should contain 2 different profiles:
Scientific technical profile Administrative profile
Information during meeting should be recorded on audio/video
Results of data gathering should be sent to panel (APPENDIX 1)
WP3 PILOTING GUIDE
Before training:
Identifying Competencies
“Once area that needs improving defined we need to identify competencies and good practises to allow improvement”
WP3 PILOTING GUIDE
Before training:
In order to allow this they recommend:
Semi structured interviews with experts (APPENDIX 2)
Duration should be 30mins-1hr
Minimum number of interviews is 5 (reduce false –ve)
WP3 PILOTING GUIDE
Before training:
Selecting competencies that can be trained & methodology used
To selected competencies the following need to be considered:
Necessity
Impact on health
Impact on patient safety
Response to simulation-based training
Cost-effectiveness of training
WP3 PILOTING GUIDE
To do this need another panel!! (can use previous one)
Identify competencies that can be trained, tools/methods
A maximum of one month should elapse between detection of first training requirements and the formation of this panel
Set up of panel numbers etc same as before
WP3 PILOTING GUIDE
Selecting the most suitable methodology and training scenarios
Should consider:
Cost Applicability Reduction of learning curve Allow the evaluation of the acquisition of competencies High degree of acceptance with trainee Availability of resources
WP3 PILOTING GUIDE
Acceptable types of training to evaluate:
Face-to-face training (Role play, debate analysis, brainstorming, Didactic panels)
E-training methodology
Mixed methodology (face-to-face, e-training methodology)
Staged simulation (using actors, clinical interviews)
Virtual simulation
Robotic simulation
WP3 PILOTING GUIDE
Designing training guide
Once we have detected the training requirements, selected the competencies to be trained, chosen the methodology and scenario the next stage is to:
Design the training and produce the training guide
WP3 PILOTING GUIDE
Design the training and produce the training guide
Panel should consider the following Clear selection process of target students Competencies to be trained Requirements that will be covered Final objectives Methodologies used Schedule Evaluation system Description of feedback procedure to training team EXAMPLES IN APPENDIX 3 AND 4
WP3 PILOTING GUIDE
Selecting the training team
CV screen (teaching ability, training skills etc)Personal interviewPractical testing
Selecting the students need to consider: Profile Professional category Previous experience Ease of transferral Heterogeneity Need to evaluate prior knowledge & motivation of students (APPEN
9 & 10)
WP3 PILOTING GUIDE
DURING THE TRAINING
Really give information about how to run an event!Which WIMAT do all the timeNo strict protocol to follow in this section apart from:
The trainers must:Encourage processes of reflectionObtains and transmits informationInformal communicationTransmits supportSolves problems Be familiar with a range of teaching methods
WP3 PILOTING GUIDE
Should be evaluations should be performed at beginning and
end of training of motivation and expectation of to assess if
teaching had a positive effect
Also satisfaction survey for trainers (APPENDIX 8)
WP3 PILOTING GUIDE
AFTER THE TRAINING
Facilitate feedback channels using questionnaires
(APPENDIX 6, 7 and 8) or using virtual communities
Revision of training according to feedback
Accredit students with certification
WP3 PILOTING GUIDE
AFTER THE TRAINING
Pick one of following: Objective evaluation Open question test On-going evaluation (APPEN 11) In situ test
Evaluate transfer of skills (recommend during this by
self questionnaires to participants) Questionnaires
should be completed within 2/52 to 2/12
IMPACT: To establish if students motivate colleagues to come to course?
SUMMARY OF SKYPE MEETING
Stakeholders information to be completed
Before Portugal meeting
Familiarise with WP2 and WP3 Meeting to discuss courses to pilot