View
230
Download
4
Category
Tags:
Preview:
Citation preview
Workplace assessment
Dr. Kieran Walsh, Editor, BMJ Learning.
Workplace assessment – pluses
• Lots of clinical material available• Skilled clinicians/teachers available• Strong educational impact
Workplace assessment – potential difficulties
• Broad range of problems/patients/procedures required
• Integrated skills • Potential to practice • Specific assessment experience/expertise is
often rare • Small number of trainees in each discipline
in an average sized hospital
Workplace assessment – theory
• Knows• Knows how
• Shows how• Does
Workplace assessment – judgement based on….
Single encounter• Actually observed performance • Case specific Routine performance• Observation over a period on different
occasions on different problems • Reduces problems with case specificity
Workplace assessment – nature of judgement
Did it happen? • Checklist approach • Objective• Structured• Consistent• But not very discriminatory
Workplace assessment – nature of judgement
Was the trainee any good? • Global rating• More valid
Workplace assessment – nature of judgement
Was the trainee good enough for a particular purpose?
Workplace assessment – influence on learning
• Formative assessment and feedback
• Direct observation (in the past many trainees never observed in a patient encounter)
Workplace assessment – faculty development
Workshops needed
• Training in behavioural observation • Performance dimension training• Frame of reference training
Workplace assessment – reliability
• Number of encounters observed
• Number of assessors
• Aspects of performance being assessed
Workplace assessment – is it a fair test?
• Different patients• Different assessors
• Faculty development• Common problem list• Number of assessors for each trainee
Workplace assessment – stakes
• Assessments influenced by what is at stake• Colleagues reluctant to make negative
assessment when stakes are high
BUT• External examiners• Faculty development • Anonymity in peer assessment• Should trainees choose their assessor?
Workplace assessment – costs
• Planning• Time of trainees and assessors • Real patients• Training of both groups• Administration • Material• Collating and processing results• Training • Cost effective?? If used for correct purpose
Workplace assessment – CbD
• Case based discussion • Trainee selects case studies, assessor
chooses one of these • Assessor should ask why this selection• Assesses quality of … diagnosis,
assessment, management, record-keeping• 15 minutes (5 minutes for feedback)
Workplace assessment – MiniPAT
• Peer assessment tool• 8 assessors• Including senior colleagues, nurses, AHPs• Including self assessment • Routine performance• Feedback reviewed with trainee and
supervisor • Agreed action plan
Workplace assessment – MiniCEX
• Faculty member observes a trainee interacting with a patient in a clinical setting
• 15 minutes (+5 minutes for feedback) • Interviewing skills, physical exam,
professionalism, communication skills• Formative feedback
Workplace assessment – MiniCEX
• Not for high stakes exams• Not to rank trainees• Not to compare training programmes• 4-6 times in the year• Different assessor each time• Encounter chosen by the trainee, confirmed
by the assessor
Workplace assessment – DOPS
• Direct observation of procedural skills• Real patients• 15 minutes (+ 5 minutes feedback) • Trainees select from an approved list of
procedures• Indications, communication, technique,
analgesia, asepsis
Workplace assessment – DOPS
• 6 times in the year • Valid • Global rating scales
Workplace assessment – portfolios
• Different types of information• Summary of educational experiences• Reviewed by an assessor • Reflection• Action plan• Purpose must be clear (e.g. for summative use)• Need be specific about what portfolio should
contain
Workplace assessment – screening
• If you do really well on your first 2 MiniCEX’s, how will you get on in your next 4?
Workplace assessment – checklists vs global ratings?
• Scores are broadly correlated
• Checklists – objective, reliable, lower level of expertise
• Global ratings – subjective, more valid
• Only experts can use global ratings• Anyone can be trained to use a checklist
Workplace assessment – references and further reading
Norcini JJ. Workplace assessment. In Understanding Medical Education: Evidence, Theory and Practice. ASME 2010
Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: a systematic review. BMJ. 2010 Sep 24;341
Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE Guide no. 45. Med Teach. 2009 Sep;31(9):790-801.
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007 Nov;29(9):855-71.
Wilkinson TJ. Assessment of clinical performance: gathering evidence. Intern Med J. 2007 Sep;37(9):631-6.
Recommended