The program will start promptly at 12:00 PM

The program will start promptly at 12:00 PM

For technical assistance please contact Tech Support at 404-969-0387 or email [email protected]

Faculty Development: Part 2 Preceptor Principles of Evaluating

Learners and Giving Feedback

Augusta, GA

December 11, 2008

T. Andrew Albritton, MDT. Andrew Albritton, MD

Christopher White, MDChristopher White, MD

Medical College of GeorgiaMedical College of Georgia

For technical assistance please contact Tech Support at 404-969-0387 or email [email protected]

ObjectivesObjectives

At the end of this part of the presentation, At the end of this part of the presentation, faculty will know:faculty will know:– the difference between formative and summative the difference between formative and summative

evaluationevaluation– the goals of evaluationthe goals of evaluation– the RIME approach to evaluationthe RIME approach to evaluation

If you have any technical issues, please contact 404-969-0387 or email [email protected]

mailto:[email protected]

Objectives

At the end of this part of the At the end of this part of the presentation, faculty will know:presentation, faculty will know:– the stages of the learning of a new skillthe stages of the learning of a new skill

– the characteristics of effective feedback the characteristics of effective feedback

– how to give positive and negative feedback how to give positive and negative feedback effectively effectively

– the importance of observation in providing the importance of observation in providing effective feedbackeffective feedback



Stages of Learning a New SkillStages of Learning a New Skill

Unconscious IncompetenceUnconscious IncompetenceConscious IncompetenceConscious IncompetenceConscious CompetenceConscious CompetenceUnconscious CompetenceUnconscious Competence

Howell WC. Human Performance and Productivity. Vol 2: Information Howell WC. Human Performance and Productivity. Vol 2: Information Processing and Decision Making. Hillsdale, NJ: Erlbaum; 1982.Processing and Decision Making. Hillsdale, NJ: Erlbaum; 1982.



Physicians Are Unconsciously Competent in Many Skills

History-TakingHistory-Taking– VerbalVerbal– Non-verbalNon-verbal– OrganizationOrganization

Physical ExaminationPhysical Examination

Technical SkillsTechnical Skills– Reading X-RaysReading X-Rays– Interpreting most lab studiesInterpreting most lab studies– Surgical scrubSurgical scrub– Knot tyingKnot tying



It’s hard to teach someone a skill that you do automatically!



Therefore, physicians Therefore, physicians who teach medical who teach medical

students have to think students have to think differently, going differently, going

beyond unconscious beyond unconscious competence to a competence to a higher stage of higher stage of

competencecompetence



Stages of Learning a New SkillReflective CompetenceReflective Competence

– Although “second nature, the practitioner can Although “second nature, the practitioner can also articulate the fine details of the skill to also articulate the fine details of the skill to others. others.

– Having fully understood all necessary steps Having fully understood all necessary steps and components of the skill to be learned and and components of the skill to be learned and the manner how they are dynamically the manner how they are dynamically integrated . . . he is thereby able to teach the integrated . . . he is thereby able to teach the skill to others in a manner that is effective skill to others in a manner that is effective and expedient.” and expedient.”

http://www.cognitivedesignsolutions.com/Instruction/TestingEvaluation.htmhttp://www.cognitivedesignsolutions.com/Instruction/TestingEvaluation.htm



Learning a New Skill: A Practical Example

Non-driver:Non-driver:

Beginner:Beginner:

Just passed driving test:Just passed driving test:

Driver who gets to work Driver who gets to work without remembering the without remembering the drivedrive

Consciously safe driverConsciously safe driver

Unconscious incompetenceUnconscious incompetence

Conscious incompetenceConscious incompetence

Conscious competenceConscious competence

Unconscious competenceUnconscious competence

Reflective competenceReflective competence



Evaluation

FormativeFormative: Provide observations to help : Provide observations to help the learner assess strengths and learning the learner assess strengths and learning needs to improve future performance.needs to improve future performance.

SummativeSummative: Assessing a learner’s : Assessing a learner’s performance using specific criteria to performance using specific criteria to determine competency.determine competency.



Goals of Evaluation

Clear and articulate description of Clear and articulate description of performance.performance.

More objective evaluation of core More objective evaluation of core competencies.competencies.

Consistent evaluation of core Consistent evaluation of core competencies.competencies.



Preceptor’s Responsibilities

Treat the patientTreat the patientDiagnose the learnerDiagnose the learnerTeach the learnerTeach the learner



Diagnosing the Learner

It’s easy: Ask 3 simple questions!It’s easy: Ask 3 simple questions!WHAT do you think is going on?WHAT do you think is going on?WHY do you think so?WHY do you think so?WHAT do you want to do?WHAT do you want to do?



RIME* Approach to Evaluation

ReporterReporterInterpreterInterpreterManager/EducatorManager/Educator**Based on the work of Louis N. Pangaro, MDBased on the work of Louis N. Pangaro, MD



Reporter

Accurately gathers information.Accurately gathers information.Basic skills to perform a history and Basic skills to perform a history and

physical examination.physical examination.Distinguishes normal from abnormal.Distinguishes normal from abnormal.Reports information without attempting Reports information without attempting

integration or synthesize.integration or synthesize.



InterpreterDemonstrates the integration and synthesis

of information reported.Prioritizes clinical problems at a basic level.Develops a more patient-oriented

assessment including a differential diagnosis.

Actively participates in patient care.



Manager

Reports, interprets, and applies Reports, interprets, and applies information to the patient.information to the patient.

Makes diagnostic or management Makes diagnostic or management suggestions.suggestions.

Uses textbooks and the primary literature Uses textbooks and the primary literature to better understand the patient.to better understand the patient.



Educator

Goes beyond the basics of self-directed Goes beyond the basics of self-directed learning.learning.

Critical appraisal skills.Critical appraisal skills.Demonstrates insight and explores Demonstrates insight and explores

important questions in more depth.important questions in more depth.Participates in the education of others.Participates in the education of others.



RIMEReporter:Reporter: Passing but requires additional Passing but requires additional

work to address deficiencies. “C” Student.work to address deficiencies. “C” Student. Interpreter:Interpreter: Meeting Expectations. “B” Meeting Expectations. “B”

Student.Student.Manager/Educator:Manager/Educator: Exceeding Exceeding

Expectations. “A” Student.Expectations. “A” Student.



Giving Feedback:

The Most Important Skill Needed The Most Important Skill Needed

to be an Effective Teacherto be an Effective Teacher



Feedback



What is Feedback?What is Feedback?

Feedback occurs when a learner is offered insight into what he or she actually did as well as the consequences of his or her actions.

This insight . . . . Highlights the dissonance between the intended result and the actual result, thereby providing impetus for change.

Ende, JAMA 1983; 250:777-781.



OverviewFeedback can be either:Feedback can be either:

– Positive: reinforcing what was done rightPositive: reinforcing what was done right– Constructive: correcting mistakes or Constructive: correcting mistakes or

misconceptionsmisconceptions

Feedback goals:Feedback goals:– Ensuring the learner improvesEnsuring the learner improves– While keeping his/her self-respect intactWhile keeping his/her self-respect intact



Important Elements of Feedback

RapportRapportClear expectationsClear expectationsAppropriate locationAppropriate locationTimely and regularTimely and regular



Effective Feedback

Descriptive and nonjudgmentalDescriptive and nonjudgmentalBased on direct observationsBased on direct observationsFocused on areas that can be changedFocused on areas that can be changedLimited to what the learner can useLimited to what the learner can useThe learner understands the feedback and The learner understands the feedback and

knows the next stepknows the next step



Feedback ≠ Evaluation

Feedback

– Presents information enabling the student to remain on course to reach a given goal.

Evaluation

– Presents a judgment about how well the student met a given goal




Feedback


Evaluation


– Formative (coaching) –Summative (a grade)




Feedback


Evaluation


– Formative (coaching)

– During or after the fact

– Summative (a grade)– Occurs after the fact



Key Principle for Providing Feedback . . . .


You only see what You only see what you look for . . . .you look for . . . .



You only see what You only see what you look for . . . .you look for . . . .

You only look for You only look for what you what you know . . . .know . . . .


Feedback: What Should Teachers Look For?

Attitudes:– Verbal & Non-verbal– Punctuality

Skills:– Presentations– Written Notes– History & Physical Exam

Knowledge:– Differential diagnosis– Therapy– Management

Observable Behaviors & SkillsObservable Behaviors & Skills


Why Is It So Hard To Give Feedback?

Failure to obtain data (observations)Failure to obtain data (observations)



““Observations are the currency of feedback, Observations are the currency of feedback, and without them the process becomes and without them the process becomes

‘feedback’ in name only”‘feedback’ in name only”

Ende, Ende, JAMA JAMA 1983; 250:777-781.1983; 250:777-781.



It’s hard to teach someone a skill It’s hard to teach someone a skill that you do automatically!that you do automatically!



We readily recognize strong learners and weak learners by looking at how they perform common clinical tasks (Hx, PE, oral presentations, etc)

“Unconscious Competence”

We have difficulty “deconstructing” the individual components of the task in order to identify areas for

feedback . . . .


Examples of Effective Feedback

Ende, JAMA 1983; 250:777-781.

Dr. Dee MeaningDr. Dee Meaning Dr. Anne SpireDr. Anne Spire



Ende, JAMA 1983; 250:777-781.

“Your differential diagnosis for this patient is inadequate”



Ende, JAMA 1983; 250:777-781.


“The differential diagnosis did not include the possibility of TB”



Ende, JAMA 1983; 250:777-781.


“You looked uncomfortable talking about your patient’s cancer”




Ende, JAMA 1983; 250:777-781.


“You looked uncomfortable talking about your patient’s cancer”


“While observing you, I began to feel that you were not comfortable talking about your patient’s cancer”


Giving Positive Feedback


Giving Positive FeedbackGiving Positive Feedback

Not general praise . . . .Not general praise . . . .

““You did a good job with that patient.”You did a good job with that patient.”

““You are doing fine.”You are doing fine.”


Giving Positive FeedbackGiving Positive Feedback Not general praise . . . .Not general praise . . . .

““You did a good job with that patient.”You did a good job with that patient.”

““You are doing fine.”You are doing fine.” Inform the student about a specific behavior or action Inform the student about a specific behavior or action

that was done wellthat was done well

“ “You gave a very good description of your patient’s general You gave a very good description of your patient’s general appearance. Successful clinicians are good observers, appearance. Successful clinicians are good observers, and you have demonstrated that you are developing good and you have demonstrated that you are developing good observation skills as you evaluate patients.observation skills as you evaluate patients.


Giving Constructive FeedbackGiving Constructive FeedbackLabel the feedbackLabel the feedback

Identify a specific behavior or action that was Identify a specific behavior or action that was done wrongdone wrong

Tell the student how to avoid or correct the Tell the student how to avoid or correct the error in the futureerror in the future

Make sure the leaner understands the Make sure the leaner understands the feedbackfeedback


Helpful HintsHelpful HintsAsk for the learner’s thoughts about his/her Ask for the learner’s thoughts about his/her

performance firstperformance firstAvoid vague, harsh, or judgmental Avoid vague, harsh, or judgmental

statementsstatementsFeedback sandwichFeedback sandwich

– Tell what was done wellTell what was done well– Tell what was done wrongTell what was done wrong– Discuss to do next timeDiscuss to do next time


Why Is It So Hard To Give Why Is It So Hard To Give Feedback?Feedback?

““Observations are the currency of feedback, Observations are the currency of feedback, and without them the process becomes and without them the process becomes

‘feedback’ in name only”‘feedback’ in name only”Ende, Ende, JAMA JAMA 1983; 250:777-781.1983; 250:777-781.



Take Home Messages

Assessment drives learningAssessment drives learningObjective evaluation of core competencies Objective evaluation of core competencies

helps the student become a better physicianhelps the student become a better physicianUsing the “3 W’s” helps the student in Using the “3 W’s” helps the student in

developing their interpretation and developing their interpretation and management skillsmanagement skills

Working hard does not necessary mean an Working hard does not necessary mean an “A” performance“A” performance


Questions?Questions?


Please Type your questions in the chat box provided on the screen.

Thank you for attending our program!

Please take the time to fill out the survey:http://www.surveymonkey.com/s.aspx?sm=JawKsSls6lJA5_2fvqZ13m3Q_3d_3d

For more information on HealthTecDL events please visit www.healthtecdl.org

For more information on becoming a Preceptor,please contact your local Area Health Education Center:

http://www.healthtecdl.org/network-organizations/network-overview.cfm

Documents

The program will start promptly at 12:00 PM