FACULTY AS TEACHERS(FasT)
CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS
ERIK BLACK, PH.D.ASSISTANT PROFESSORPEDIATRICS
September 21, 2011
Faculty Affairs Announcements• Credit for today’s workshop (4)• Educational Development Certificate
Serieshttp://facultyaffairs.med.ufl.edu/
• Online Master’s degree [email protected]
• Most memorable teacher
Today’s session
IntendedLearning Outcomes
Teaching/LearningFormats
AssessmentMethods
inform inform
Outcomes met?
Intended learning outcomes
At the end of this workshop, attendees will be able to:
1. Write effective intended learning outcomes (ILOs)
2. Distinguish between effective and ineffective ILOs
3. Evaluate/critique ILOs
4. Recognize the value – for the learner and the teacher – of effective ILOs
Intended learning outcomes
5. Describe their beliefs about learning
6. Apply those beliefs to their own teaching & learning
7. Distinguish between active and passive learning (theory)
8. Describe passive methodologies and active methodologies (applied)
9. Compare and contrast active and passive learning (advantages and disadvantages of both)
10.Describe the role of assessment in contemporary medical education
Intended learning outcomes
11.Identify methods of assessment
12.Determine which method of assessment is most appropriate in specific scenarios
WRITING EFFECTIVEINTENDED LEARNING OUTCOMES (ILOS)
CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS
September 21, 2011
Elements, Domains, Levels
Elements: A B C D
A = AudienceB = BehaviorC = ConditionD = Degree
4 Elements: A B C D
AudienceThe individuals who are your students
?
4 Elements: A B C D
AudienceThe individuals who are your students
Second year students . . .
Graduates of this program . . .
4 Elements: A B C D
Behavior 1What you expect the student to know
?
4 Elements: A B C D
Behavior 1What you expect the student to know
List and describe the ACGME core competencies . . .
4 Elements: A B C D
Behavior 2What you expect the student to know/do
?
4 Elements: A B C D
Behavior 2What you expect the student to know/do
Palpate the breast . . .
4 Elements: A B C D
Behavior 3What you expect the student to
know/do/feel
?
4 Elements: A B C D
Behavior 3What you expect the student to
know/do/feel
Show compassion . . .
4 Elements: A B C D
ConditionSituation in which student will
demonstrate task
?
4 Elements: A B C D
ConditionSituation in which student will
demonstrate task
In an exercise with a standardized patient . . .
4 Elements: A B C D
DegreeStandard of performance student must
achieve
?
4 Elements: A B C D
DegreeStandard of performance student must
achieve80% correct . . .
[Palpating] all relevant locations . . .
Always washing your hands first . . .
4 Elements: A B C D
In a 15 minute OSCE station, each third-year student will perform a clinical breast examination on a standardized patient, achieving an accuracy rate of at least 70% to pass.
Domains: K S A
K = KnowledgeS = SkillsA = Affect or Attitude
3 Domains
KnowledgeWhat the student will know
3 Domains
SkillsWhat the student will be able to do
3 Domains
AttitudeHow the student will feel
Levels
Handout: Bloom’s
Choosing the right verb
Knowledge (Cognition)
6 Evaluation
5 Synthesis
4 Analysis
3 Application
2 Comprehension
1 Knowledge
high
mid
low
Notebook pages 21-28
Skills (psycho-motor)
7 Origination
6 Adaptation
5 Complex overt response
4 Mechanism
3 Guided response
2 Set
1 Perception
Attitudes/Affect
5 Internalizing values
4 Organization
3 Valuing
2 Respond to phenomena
1 Receive phenomena
Sample Learning Outcome
Domain: Level:
In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.
Sample Learning Outcome
Domain: CognitionLevel: Application (mid)
In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.
Sample Learning Outcome
Domain: CognitionLevel: Application (mid)
In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.
condition
degreebehavior
audience
Sample Learning Outcome
Domain: Level:
In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.
Sample Learning Outcome
Domain: CognitionLevel: Synthesis (high)
In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.
Sample Learning Outcome
Domain: CognitionLevel: Synthesis (high)
In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.
condition
degree behavior
audience
Sample Learning Outcome
Domain: Level:
In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.
Sample Learning Outcome
Domain: SkillsLevel: Mechanism (mid)
In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.
Sample Learning Outcome
Domain: SkillsLevel: Mechanism (mid)
condition
behavior
In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.
degree
audience
Sample Learning Outcome
Domain: SkillsLevel: Mechanism (mid)
condition
behavior
In a 15-minute station with a standardized patient, the student will efficiently and accurately perform 85% of a “head-to-toe” examination.
degree
audience
Is there another degree “hidden”here?
Sample Learning Outcome
Domain: Level:
With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.
Sample Learning Outcome
Domain: AttitudeLevel: Valuing (mid)
With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.
Sample Learning Outcome
Domain: AttitudeLevel: Valuing (mid)
With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.
condition
degree
behavior
audience
Sample Learning Outcome
Domain:Level:
In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.
Sample Learning Outcome
Domain: CognitiveLevel: Synthesis (high)
In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.
Sample Learning Outcome
Domain: CognitiveLevel: Synthesis (high)
In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.
condition
standardbehavior
audience
Sample Learning Outcome
Domain: Level:
In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.
Sample Learning Outcome
Domain: SkillsLevel: Complex Overt Response (high)
In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.
Sample Learning Outcome
Domain: SkillsLevel: Complex Overt Response (high)
In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.
condition
degree behavioraudience
Your turn
• Re-group in small groups• For a course/module that you teach (or
want to teach), please write at least 2 intended learning outcomes.
• Use the handout “Bloom’s” in your notebook as a guide.
Notebook pages 21-28
Summary
Elements1. Audience2. Behavior3. Condition4. Degree
Domains5. Knowledge6. Skills7. Attitudes
Taxonomies (Levels/Verbs)1. Knowledge (Cognition): Bloom2. Skills (Psychomotor): Simpson3. Attitudes (Affect): Bloom
ACTIVE LEARNING
CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS
September 21, 2011
Today
• Learning theory• Research on teaching/learning• Modeling formats (small groups)• Notebook resources
TWO LEARNING THEORIES
Epistemology Inventory (5 minutes)
Your beliefs about teaching/learning
Notebook page 59
What behaviorists generally believe
• The teacher is the expert
• The teacher dispenses information to students
• Learning is a solitary activity
• Emphasis is on shorter-term memorization
• Assessment is primarily through testing
Notebook pages 83-93
What constructivists generally believe
• Learners can learn from each other and teachers can learn from learners
• New knowledge is built upon previous knowledge
• Learning is unique to each individual
• Assessment is often interwoven with teaching
RESEARCH ON LECTURES
A brief quiz
A brief quiz
While teachers are lecturing, students are not attending to what is being said ___% of the time:
a.10%b. 20%c. 30%d. 40%e. 50%
A brief quiz
In the first 10 minutes of lecture, students retain ___% of the information, in the last 10 minutes ___%:
a. 90%, 25%b. 85%, 10%c. 70%, 20%d. 60%, 15%e. 50%, 5%
A brief quiz
Students who took a (lecture-based) introductory psych course knew ___% more than students who had never taken the course*
a. 22%b. 15%c. 10%d. 8%e. 5%
*measured shortly after the course concluded
Research on lectures
• While teachers are lecturing, students are not attending to what is being said 40% of the time (Pollio)
• In the first 10 minutes of (50-minute) lecture, students retain 70% of the information, in the last 10 minutes 20% (McKeachie)
Research on lectures
• Students who took a (lecture-based) introductory psych course knew only 8% more than students who never took the course (Rickard, Rogers, Ellis, Beidleman)
Cone of learning*
Reading
Hearing words
Watching a movieLooking at an exhibit
Watching a demonstrationSeeing it done on location
Participating in a discussionGiving a talk
Doing a dramatic presentationSimulating the real experience
Doing the real thing
After two weeks we tend to remember:
10% of what we read20% of what we hear
30% of what we see
50% of what we seeand hear
70% of what we say
Looking at pictures
90% of what we say and do
PASSIVE
ACTIVE
*Adapted from Edgar Dale
WHAT’S A GOOD LECTURE?
Your turn
What’s a good lecture?
Provides new information based on original research, and not generally found in textbooks
Highlights similarities and differences between key concepts
Communicates enthusiasm of teachers for their subjects
What’s a good lecture?
Models how a discipline deals with questions of evidence, critical analysis, problem solving, etc.
Dramatizes important concepts and share personal insights Organizes subject matter
in an effective way for the class and the course
RESEARCH ON ACTIVE LEARNING
Research on active learning
• In a section of a course where peer learning was integrated: students scored 10% higher on final exams
• Active learning students scored significantly higher than traditional on 2 out of 4 exams (no difference on other two)
• Active learning reduced dropout rates in organic chemistry by 38%
• 94% of students had more confidence in their scientific ability, compared to 56%
Research on active learning
Students who learn how to apply new information are more likely to be able to find and use it when needed
Active approaches are more effective for higher order cognition, skills and attitudes
Working in groups, students can help each other push new knowledge further - do more complex thinking
BARRIERS TO ACTIVE LEARNINGYour turn
Barriers to active learning
• Influence of educational tradition
• Faculty definitions of their roles
• Epistemology (beliefs about effective learning)
• Discomfort with change
Barriers to active learning
• Difficulty in covering assigned material
• Increase in preparation time
• Difficulty with large classes
• Lack of materials, equipment, resources
• Limited incentives to change
ENGAGING THE DATA
Arno Kumagai, M.D.
Data from the Donner Party
The Donner/Reed PartyMay 1846-February 1847
July 28 Party meets Lansford Hastings at Fort
BridgerDonner Lake
James & Margaret Reed
May, 1846: Donner/Reed Party leaves Independence
MO for Sutter’s Fort, CA
Rescued survivors arrive February 1847
Tasks
1. Write down 2-3 observations about the data. What do you see?
2. Suggest explanations (hypotheses) for the data.
3. How would you test your hypotheses?
Review the data sets provided in your packet:
Notebook pages 60-62
EVIDENCE-BASED DISCUSSION/TBL
Mary Kate Worden, Ph.D.
Biology Course at UVa
• Students assigned advance reading• Meet in small groups; given a modified
relevant journal article• Read modified article and answer
questions (in group)• Sample is in your notebook
Notebook pages 63-64
THINK-PAIR-SHARE
Think-Pair-Share
Pair up with someone in your small group:
• Choose one of the intended learning outcomes you wrote this morning
• Discuss and come to agreement on what methods you would use to help your students achieve that objective (refer to Bloom’s Wheel handout) Notebook page 65
• Discuss and come to agreement on how you would assess that students have achieved that outcome
Active learning strategies
Simple
Complex
Pause for Reflection
Interaction with Guest Speaker
Fishbowl Role Playing
Forum Theater
Simulations
Good practice in education*
1. Encourages contact between students and faculty
2. Develops reciprocity and cooperation among students
3. Encourages active learning
4. Provides prompt feedback
* Chickering and Gamson, 1987
Good practice in education*
5. Emphasizes time on task
6. Communicates high expectations
7. Respects diverse talents and ways of learning
* Chickering and Gamson, 1987
AN INTRODUCTION TO EDUCATIONAL ASSESSMENT
Assessment
Why do it?
What it is?
Assessment’s role in course development
Types of assessment
The future of assessment in medicine
Assessment drives student learning
Assessment drives student learning
What is the primary format for MS1 and MS2 student assessments?
What is one of our primary complaints about medical students (and residents)and their knowledge?
Assessment drives student learning
We assess to: Measure impact of
our curriculum Measure impact of
our instructional methods
Measure student achievement of curricular goals and objectives
Continuously improve upon all of the above
What exactly is assessment?
More than just a grade…it’s a means to understand learner and instructor performance
“Assessment is a mechanism for providing instructors with data for improving their teaching methods and for guiding and motivating students to be actively involved in their own learning. As such, assessment provides important feedback to both instructors and students.” (Angelo & Cross, 1993, p.4)
Types of assessment
Direct Student Assessment Techniques
The assessment is based on an analysis of student behaviors or products in which they demonstrate how well they have mastered learning outcomes. (formative v. summative)
Indirect Student Assessment Techniques
The assessment is based on an analysis of reported perceptions about student mastery of learning outcomes. (eg: survey,
Interview, focus group)
Program Assessment Techniques
Characteristics of Good Assessment Valid—directly reflects the learning outcome
being assessed Reliable—especially inter-rater reliability when
subjective judgments are made Actionable—results help faculty identify what
students are learning well and what requires more attention
Efficient and cost-effective in time and money Engaging to students and other respondents—so
they’ll demonstrate the extent of their learning Interesting to faculty and other stakeholders—
they care about results and are willing to act on them
Triangulation—multiple lines of evidence point to the same conclusion
Difference between formative and summative assessment Summative – most common,
approximation of achievement at x time. Mid-Term Examination Final Examination
Formative – Informal, feedback driven. Self-test Background knowledge probe Muddiest point (what is the most unclear
element)
The future of assessment in medical education
Recognition that knowledge-based assessment is one of several tools available (but is your test reliable and valid?)
Focused on application of knowledge
Competency driven
Increasingly qualitative in nature OSCEs SPs Portfolios
Resources
Some Assessment Techniques – CSU Fresno
Epstein – NEJM
Application exercise
• Using the objectives previously developed, convene in small groups to develop a strategy for assessing your objective.
• Share with the larger group. Tell us why the chosen assessment technique is appropriate.
Notebook pages 116-117, 121-122
Summary
At the end of this workshop, you should be able to:1. Write effective intended learning
outcomes (ILOs)2. Distinguish between effective and
ineffective ILOs3. Evaluate/critique ILOs4. Recognize the value – for the
learner and the teacher – of effective ILOs
5. Describe their beliefs about learning 6. Apply those beliefs to their own teaching
& learning7. Distinguish between active and passive
learning (theory)8. Describe passive methodologies and
active methodologies (applied)9. Compare and contrast active and passive
learning (advantages and disadvantages of both)
10.Describe the role of assessment in contemporary medical education
Intended learning outcomes
11.Identify methods of assessment
12.Determine which method of assessment is most appropriate in specific scenarios
Workshop follow-up
Index card: Please write down one thing you will or want to change; leave on the
table
THANK YOU!!!