The mission of TEACH is to promote learning excellence at Carilion Clinic, Virginia Tech Carilion School of Medicine, Virginia Tech Carilion Research Institute, and Jefferson College by creating a community of educators and fostering their development as
teachers, learners, and education researchers.
2
Looking in the Mirror: self-reflection, self-assessment, and direct observation
Sarah M. Bean, M.D.Associate Professor, Department of PathologyCytopathology Fellowship Program DirectorSurgical Pathology Fellowship Program DirectorClinical Competency Committee Chair
Learning Objectives
1. Define self-regulated learning
2. Discuss the challenges of self-assessment
3. Identify barriers to direct observation
4. Understand the importance of self-reflection
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Learner Differences
When compared to high performers, low performing learners have:
Task valueSelf-efficacy beliefs
Course-related anxietyFrustrationBoredom
Artino AR et al. Acad Med. 86(10):S35-S38, 2011.
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Planning: Goal Setting
Self-efficacy, an individual’s belief in his or her ability to achieve specific goals, is critical to setting and achieving challenging yet attainable goals.
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Learning
• Epistemology (beliefs about knowledge and knowing)
• Learning styles (innate characteristics possessed by learners that underlie and influence their learning)
• Learning strategies
• Principles and methods
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
self-as·sess·ment
• assessment or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard.
www.google.com
Self-Assessment
• Identify learner needs
• Influence learning activity
• Change clinical practice
• Improve patient outcomes
• Be accurate
Colthart I et al. Med Teach. 30:124-145, 2008.
Self-Assessment
• Identify learner needs
• Influence learning activity
• Change clinical practice
• Improve patient outcomes
• Be accurate
Colthart I et al. Med Teach. 30:124-145, 2008.
Kruger and Dunning: Is it funny?
Kruger J & Dunning D. J of Personal and Soc Psychol. 77(6):1121-1134, 1999.
Kruger and Dunning: Logical Reasoning
Kruger J & Dunning D. J of Personal and Soc Psychol. 77(6):1121-1134, 1999.
Self-Assessment
• Physicians have limited ability to accurately self-assess.• Davis et al. JAMA 296(9):1094-1102, 2006.
• Colthart et al. Med Teach. 30:124-145, 2008.
• Regehr G et al. Acad Med. 71(10):S52-54.
• The most confident are the weakest self-assessors.• Eva KW and Regehr G. Self-assessment in the health professions: reformulation and research agenda. Acad Med. 80(10):S46-54.
Contributing factors
• Motivation
• Metacognition
• Selective recall of past behavior
• Tendency to ignore proficiency of others
• Purpose of the self-assessment task (how is it used?)
Improving Self-Assessment
• Provide explicit assessment criteria
• Benchmarking
• External feedback (verbal, video)
Ward et al. Am J Surg. 185: 521-524, 2003.
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Importance of Feedback
“Without feedback, mistakes go uncorrected, good performance is not reinforced, and clinical competence is achieved empirically or not at all.”
- J. Ende, 1983
Ende J. Feedback in Clinical Medical Education. JAMA. 250(6):777-781, 1983.
Guidelines for Giving Feedback
• Teacher and learner are allies
• Timely, expected
• Based on direct observation
• Focused on specific behaviors
• Includes positive behaviors
• Limited to behaviors amenable to change
• Descriptive, constructive
• Quantity is regulated
Direct Observation
• Strongly associated with the perception of accurate knowledge of a learner’s performance
• Perception of accurate knowledge of a learner’s performance is associated with increased feedback
Mazor KM et al. Acad Med 86(10):S63-S68, 2011.
Direct Observation: Does it happen?
• NBME survey reported students were observed more often by a resident than by a faculty member
• 20% indicated that had been observed by a faculty 0-2 times
Direct Observation: does it happen?
Rotation History-Taking Focused Physical Complete Physical
Internal Medicine 59% 51% 74%
Surgery 74% 71% 91%
Pediatrics 49% 47% 77%
Psychiatry 27% 71% 87%
Family Medicine 26% 25% 70%
Ob-GYN 68% 61% 88%
Howley L & Wilson W. Acad Med. 79(3):276-280, 2004.
Direct Observation
• Develop policies that encourage and reward direct observation
• Develop education champions
• Evaluate the process
Holmboe ES. Acad Med. 79(1):16-22, 2004.
Direct Observation
• Define objectives to guide DO
• Formative or summative?
• Use an existing tool
• Culture change
• Faculty development
• Give feedback
Hauer KE et al. Med Teach. 33:27-33, 2011.
• Make a learner action plan
• Orient learners
• Use the tool multiple times per trainee
• Measure outcomes
Hauer KE et al. Med Teach. 33:27-33, 2011.
Direct Observation
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Self-Reflection
• Analyzing, questioning, and reframing an experience for the purposes of learning
– Aronson L. Med Teach. 33:200-205,2011.
• ‘recapture their experience, think about it, mull it over and evaluate it’
– Boud, Keogh and Walker (1985)
Self-Reflection
• Talking
• “How did it go?”
• Root cause analysis
• Journal entry/writing
• Story telling
• Self-reflection exercises
• Be creative (art, twitter, dreams)
Self-Reflection
• Adds meaning to complex situations
• Enables experiential learning
Mann K et al. Adv in Health Sci Educ. 14:595-621,2009.
Encouraging Self-Reflection
• Role model the behavior
• Establish an educational alliance with learner
• Facilitate reflection
Branch & Paranjape. Acad Med. 77(12):1185-1188, 2002.
Setting the Stage
Link past, present, and future
Integrate cognitive and emotional
Consider multiple perspectives
Reframe
Describe lessons learned
Plan for future learning
Aronson L. Med Teach 33(3):200-205,2011.
Self-Regulated Learning
• Epistemology
• Learning styles
• Strategies
• Principles/methods
• Self-assessment
• External feedback
• Goal setting
• Motivation
• Self-reflection
• Attribution
Adjustment Planning
LearningAssessment
Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.
Conclusions
• Self-regulated learning is the theoretical foundation of competency based education.
• Physicians do NOT accurately self-assess.
• External feedback with direct observation is the check and balance of SRL.
• Self-reflection enables experiential learning.