Upload
alodie
View
38
Download
0
Embed Size (px)
DESCRIPTION
Clinical Skills Verification Rater Training MODULE 3 Setting the Clinical Skills Verification Examination Climate and Giving Feedback. Karen Broquet , M.D. Eugene Beresin , M.D. Nyapati Rau , M.D. Module 3 Pre-test. - PowerPoint PPT Presentation
Citation preview
Clinical Skills Verification Rater TrainingMODULE 3
Setting the Clinical Skills Verification Examination Climate and Giving Feedback
Karen Broquet , M.D.Eugene Beresin, M.D.
Nyapati Rau , M.D.
Module 3Pre-test
1. Which of the following is assessed in the Clinical Skills Verification Evaluation?
a. Patient Doctor Relationship
b. Clinical Formulationc. Differential Diagnosisd. Treatment Planning
Module 3Pre-test
2. In assessment of clinical skills evaluations, which of the following methods is most likely to demonstrate what a resident actually “does” according to Miller’s pyramid?
a. Arrange Mock Oral Board-style evaluationsb. Use Standardized Patientsc. Perform the exercise during routine workd. Make use of the Hawthorne Effecte. Add a multiple choice test following the interview
Module 3Pre-test
3. Which of the following rating methods is recommended for a documented Clinical Skills Verification Evaluation?
a. Any rating form can be used
b. Process notes provided by the residentc. Faculty should use a rating form he/she most prefersd. Faculty should all use the same rating form
Module 3Pre-test
4. Which of the following is considered formative assessment?
a. Year-end reviews of performance
b. Pass/fail examinationsc. Specific feedback of behavior in real timed. Anonymous 360 degree evaluations sent to the residente. Periodic summative evaluations
CSV Exam MUST Assess…
• Ability to establish doctor-patient relationship
• Conduct a clinical interview• Oral presentation of patient history
and clinical findings
But can (and should) ALSO be….
• Valuable educational experience• Opportunity for
• Direct observation of clinical skills by faculty
• Formal evaluation of clinical skills
• Focused feedback
Feedback is Formative
Process by which the teacher provides the learner with information about his/her performance for the purpose of improving performance (Ende 1983)
• Without specific and focused feedback, learners:• Rely on self-assessment (unreliable!)• May establish undesired behaviors and/or
extinguish desired behaviors
Evaluation is Summative
“Did I Pass the exam!?!”
Feedback = Coach
• Timely, immediate, ongoing
• Direct• Verbal• Face to face• Guiding future
performance
Evaluation = Judge
• Summative• Not ongoing,
occurs at the end of time together
• Assesses whether or not learner met performance standards
Quality of both feedback and evaluation depends on the observer’s detection, perception and recall of the resident’s
performance
How to maximize this:• Incorporate direct observation into clinical
teaching routine• Brief (daily rounds) and not so brief (CSV exam)• Normalizes observation/assessment experience• Hawthorne effect: the more we know we are
assessed, the better we perform• Write it down!
• Specific, accurate feedback is better feedback• Also helps with accurate evaluation
• Prepare for the encounter
Setting Expectations
• How long will the encounter be?• Who will be observing?• What are the goals of the encounter?
• i.e., is this a formal CSV evaluation?• Assessment parameters?• What rating instrument/form will be used?• When will “results” be given?• What are the consequences of “passing” or
“failing” a CSV exam?
What Rating Form/Instrument to Use?
• Multiple forms accepted by the ABPN
• Forms vary by level of detail• All faculty/CSV raters in a program
should be trained on and use the same form
How Long Should the CSV Exam Be?
• ABPN: “about” 30 minutes for interview and 15 minutes for presentation
• AADPRT CS Assessment Task Force: 30-45 minutes for interview, 60-75 minutes total
• Programs should choose one format and use consistently
Guidelines for giving effective feedback – whatever the setting/encounter
Effective/Meaningful Feedback is…
• approached with teacher and learner working as allies with common goals
• given immediately in a private environment• well timed and expected• based on first-hand (observed) data• focused on specific behaviors, not
generalizations• limited to behaviors that are remediable
Effective/Meaningful Feedback is…
• regulated in quantity (no more than 2-3 items for improvement)
• limited to actions and decisions, not assumed intentions or interpretations
• phrased in descriptive, nonjudgmental language
• closed with suggestions for improvement and a mutually agreed upon action plan
Barriers to Effective Feedback
Faculty• Time• Lack of feedback skills• Feels like being critical• Want learners to like us • Gestalt vs specific
observations• Impact of CSV ‘failure”
Resident• Time• Lack of feedback
experience • Feels like being criticized• Don’t want to expose
weaknesses
Residents CRAVE feedback
To minimize barriers, faculty can…
• Understand the profound and beneficial educational value of feedback
• Provide the proper climate and preparation for the experience
• Elicit reciprocal feedback from trainees• Gain the skills for giving both constructive
and reinforcing feedback• Use a consistent, common rating instrument
Cultural Phenomena – International Medical Grads
• Lack of feedback Experience• Diverse cultural backgrounds• Power dynamics between teacher and learner• Accepted communication patterns• Attitudes towards critique• Vulnerability in evaluation • Stress and difficult social situations • Experience with discrimination
Important for all learners – especially IMGs
• Learner centered approach• Explicitly prepare the resident for the
expectations of the exam and the provision of feedback
• The importance of the teacher and the IMG working together cannot be over emphasized.
• Create a safe learning environment and clear expectations and educational objectives
Important for all learners – especially IMGs
• Invite learner to indicate his/her learning needs and provide a self-assessment.
• Normalize feedback as regular educational activity that includes areas to be reinforced and areas requiring improvement.
• Be clear when feedback is formative or summative (i.e evaluation)
Module 3Post-test
1. Which of the following is assessed in the Clinical Skills Verification Evaluation?
a. Patient Doctor Relationship
b. Clinical Formulationc. Differential Diagnosisd. Treatment Planning
Module 5 Post-test
1. Which of the following is assessed in the Clinical Skills Verification Evaluation?
a. Patient Doctor Relationship
Diagnosis, biopsychosocial formulation and treatment planning are not assessed
Module 5Post-test
2. In assessment of clinical skills evaluations, which of the following methods is most likely to demonstrate what a resident actually “does” according to Miller’s pyramid?
a. Arrange Mock Oral Board-style evaluationsb. Use Standardized Patientsc. Perform the exercise during routine workd. Make use of the Hawthorne Effecte. Add a multiple choice test following the interview
Module 5Post-test
2. In assessment of clinical skills evaluations, which of the following methods is most likely to demonstrate what a resident actually “does” according to Miller’s pyramid?
c. Perform the exercise during routine work
Module 5Post-test
3. Which of the following rating methods is recommended for a documented Clinical Skills Verification Evaluation?
a. Any rating form can be used
b. Process notes provided by the residentc. Faculty should use a rating form he/she most prefersd. Faculty should all use the same rating form
Module 5Post-test
3. Which of the following rating methods is recommended for a documented Clinical Skills Verification Evaluation?
d. Faculty should all use the same rating form
Although the ABPN has approved a variety of different rating forms for the CSV exam, consistent use of and feedback regarding the same form by all raters in a program is associated with improved inter-rater reliability
Module 5Post-test
4. Which of the following is considered formative assessment?
a. Year-end reviews of performance
b. Pass/fail examinationsc. Specific feedback of behavior in real timed. Anonymous 360 degree evaluations sent to the residente. Periodic summative evaluations
Module 5Post-test
4. Which of the following is considered formative assessment?
c. Specific feedback of behavior in real time
Annual performance reviews, 360 degree evaluations and pass-fail examinations are all types of summative assessment.