FACULTY AS TEACHERS (FasT) CASEY WHITE, PH.D. DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMS OFFICE...

FACULTY AS TEACHERS(FasT)

CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS

ERIK BLACK, PH.D.ASSISTANT PROFESSORPEDIATRICS

September 21, 2011

Faculty Affairs Announcements• Credit for today’s workshop (4)• Educational Development Certificate

Serieshttp://facultyaffairs.med.ufl.edu/

• Online Master’s degree programccimino@ufl.edu

• Most memorable teacher

Today’s session

IntendedLearning Outcomes

Teaching/LearningFormats

AssessmentMethods

inform inform

Outcomes met?

Intended learning outcomes

At the end of this workshop, attendees will be able to:

1. Write effective intended learning outcomes (ILOs)

2. Distinguish between effective and ineffective ILOs

3. Evaluate/critique ILOs

4. Recognize the value – for the learner and the teacher – of effective ILOs

Intended learning outcomes

5. Describe their beliefs about learning

6. Apply those beliefs to their own teaching & learning

7. Distinguish between active and passive learning (theory)

8. Describe passive methodologies and active methodologies (applied)

9. Compare and contrast active and passive learning (advantages and disadvantages of both)

10.Describe the role of assessment in contemporary medical education

Intended learning outcomes

11.Identify methods of assessment

12.Determine which method of assessment is most appropriate in specific scenarios

WRITING EFFECTIVEINTENDED LEARNING OUTCOMES (ILOS)

CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS

September 21, 2011

Elements, Domains, Levels

Elements: A B C D

A = AudienceB = BehaviorC = ConditionD = Degree

4 Elements: A B C D

AudienceThe individuals who are your students

?

4 Elements: A B C D

AudienceThe individuals who are your students

Second year students . . .

Graduates of this program . . .

4 Elements: A B C D

Behavior 1What you expect the student to know

?

4 Elements: A B C D

Behavior 1What you expect the student to know

List and describe the ACGME core competencies . . .

4 Elements: A B C D

Behavior 2What you expect the student to know/do

?

4 Elements: A B C D

Behavior 2What you expect the student to know/do

Palpate the breast . . .

4 Elements: A B C D

Behavior 3What you expect the student to

know/do/feel

?

4 Elements: A B C D

Behavior 3What you expect the student to

know/do/feel

Show compassion . . .

4 Elements: A B C D

ConditionSituation in which student will

demonstrate task

?

4 Elements: A B C D

ConditionSituation in which student will

demonstrate task

In an exercise with a standardized patient . . .

4 Elements: A B C D

DegreeStandard of performance student must

achieve

?

4 Elements: A B C D

DegreeStandard of performance student must

achieve80% correct . . .

[Palpating] all relevant locations . . .

Always washing your hands first . . .

4 Elements: A B C D

In a 15 minute OSCE station, each third-year student will perform a clinical breast examination on a standardized patient, achieving an accuracy rate of at least 70% to pass.

Domains: K S A

K = KnowledgeS = SkillsA = Affect or Attitude

3 Domains

KnowledgeWhat the student will know

3 Domains

SkillsWhat the student will be able to do

3 Domains

AttitudeHow the student will feel

Levels

Handout: Bloom’s

Choosing the right verb

Knowledge (Cognition)

6 Evaluation

5 Synthesis

4 Analysis

3 Application

2 Comprehension

1 Knowledge

high

mid

low

Notebook pages 21-28

Skills (psycho-motor)

7 Origination

6 Adaptation

5 Complex overt response

4 Mechanism

3 Guided response

2 Set

1 Perception

Attitudes/Affect

5 Internalizing values

4 Organization

3 Valuing

2 Respond to phenomena

1 Receive phenomena

Sample Learning Outcome

Domain: Level:

In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.

Sample Learning Outcome

Domain: CognitionLevel: Application (mid)

In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.

Sample Learning Outcome

Domain: CognitionLevel: Application (mid)

In an outpatient setting, the student willmodify a treatment plan to incorporatea patient’s religious beliefs.

condition

degreebehavior

audience

Sample Learning Outcome

Domain: Level:

In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.

Sample Learning Outcome

Domain: CognitionLevel: Synthesis (high)

In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.

Sample Learning Outcome

Domain: CognitionLevel: Synthesis (high)

In partnership with the patient, the student will effectively plan for the pharmacologic management of hyperglycemia.

condition

degree behavior

audience

Sample Learning Outcome

Domain: Level:

In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.

Sample Learning Outcome

Domain: SkillsLevel: Mechanism (mid)

In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.

Sample Learning Outcome

Domain: SkillsLevel: Mechanism (mid)

condition

behavior

In a 15-minute station with a standardized patient, the student will efficiently and accurately perform a “head-to-toe” examination.

degree

audience

Sample Learning Outcome

Domain: SkillsLevel: Mechanism (mid)

condition

behavior

In a 15-minute station with a standardized patient, the student will efficiently and accurately perform 85% of a “head-to-toe” examination.

degree

audience

Is there another degree “hidden”here?

Sample Learning Outcome

Domain: Level:

With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.

Sample Learning Outcome

Domain: AttitudeLevel: Valuing (mid)

With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.

Sample Learning Outcome

Domain: AttitudeLevel: Valuing (mid)

With older patients who have hearing challenges, the student will use appropriate communication skills (e.g., speaking slowly, clearly and loudly) to obtain a medical history.

condition

degree

behavior

audience

Sample Learning Outcome

Domain:Level:

In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.

Sample Learning Outcome

Domain: CognitiveLevel: Synthesis (high)

In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.

Sample Learning Outcome

Domain: CognitiveLevel: Synthesis (high)

In response to information obtained from a seminal journal article, the student will accurately adjust a patient’s treatment plan to accommodate new and relevant clinical findings.

condition

standardbehavior

audience

Sample Learning Outcome

Domain: Level:

In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.

Sample Learning Outcome

Domain: SkillsLevel: Complex Overt Response (high)

In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.

Sample Learning Outcome

Domain: SkillsLevel: Complex Overt Response (high)

In a patient with a fixed physical finding, the student will accurately detect and identify a heart murmur.

condition

degree behavioraudience

Your turn

• Re-group in small groups• For a course/module that you teach (or

want to teach), please write at least 2 intended learning outcomes.

• Use the handout “Bloom’s” in your notebook as a guide.

Notebook pages 21-28

Summary

Elements1. Audience2. Behavior3. Condition4. Degree

Domains5. Knowledge6. Skills7. Attitudes

Taxonomies (Levels/Verbs)1. Knowledge (Cognition): Bloom2. Skills (Psychomotor): Simpson3. Attitudes (Affect): Bloom

ACTIVE LEARNING

CASEY WHITE, PH.D.DIRECTOR OF PROFESSIONAL DEVELOPMENT PROGRAMSOFFICE OF FACULTY AFFAIRS

September 21, 2011

Today

• Learning theory• Research on teaching/learning• Modeling formats (small groups)• Notebook resources

TWO LEARNING THEORIES

Epistemology Inventory (5 minutes)

Your beliefs about teaching/learning

Notebook page 59

What behaviorists generally believe

• The teacher is the expert

• The teacher dispenses information to students

• Learning is a solitary activity

• Emphasis is on shorter-term memorization

• Assessment is primarily through testing

Notebook pages 83-93

What constructivists generally believe

• Learners can learn from each other and teachers can learn from learners

• New knowledge is built upon previous knowledge

• Learning is unique to each individual

• Assessment is often interwoven with teaching

RESEARCH ON LECTURES

A brief quiz

A brief quiz

While teachers are lecturing, students are not attending to what is being said ___% of the time:

a.10%b. 20%c. 30%d. 40%e. 50%

A brief quiz

In the first 10 minutes of lecture, students retain ___% of the information, in the last 10 minutes ___%:

a. 90%, 25%b. 85%, 10%c. 70%, 20%d. 60%, 15%e. 50%, 5%

A brief quiz

Students who took a (lecture-based) introductory psych course knew ___% more than students who had never taken the course*

a. 22%b. 15%c. 10%d. 8%e. 5%

*measured shortly after the course concluded

Research on lectures

• While teachers are lecturing, students are not attending to what is being said 40% of the time (Pollio)

• In the first 10 minutes of (50-minute) lecture, students retain 70% of the information, in the last 10 minutes 20% (McKeachie)

Research on lectures

• Students who took a (lecture-based) introductory psych course knew only 8% more than students who never took the course (Rickard, Rogers, Ellis, Beidleman)

Cone of learning*

Reading

Hearing words

Watching a movieLooking at an exhibit

Watching a demonstrationSeeing it done on location

Participating in a discussionGiving a talk

Doing a dramatic presentationSimulating the real experience

Doing the real thing

After two weeks we tend to remember:

10% of what we read20% of what we hear

30% of what we see

50% of what we seeand hear

70% of what we say

Looking at pictures

90% of what we say and do

PASSIVE

ACTIVE

*Adapted from Edgar Dale

WHAT’S A GOOD LECTURE?

Your turn

What’s a good lecture?

Provides new information based on original research, and not generally found in textbooks

Highlights similarities and differences between key concepts

Communicates enthusiasm of teachers for their subjects

What’s a good lecture?

Models how a discipline deals with questions of evidence, critical analysis, problem solving, etc.

Dramatizes important concepts and share personal insights Organizes subject matter

in an effective way for the class and the course

RESEARCH ON ACTIVE LEARNING

Research on active learning

• In a section of a course where peer learning was integrated: students scored 10% higher on final exams

• Active learning students scored significantly higher than traditional on 2 out of 4 exams (no difference on other two)

• Active learning reduced dropout rates in organic chemistry by 38%

• 94% of students had more confidence in their scientific ability, compared to 56%

Research on active learning

Students who learn how to apply new information are more likely to be able to find and use it when needed

Active approaches are more effective for higher order cognition, skills and attitudes

Working in groups, students can help each other push new knowledge further - do more complex thinking

BARRIERS TO ACTIVE LEARNINGYour turn

Barriers to active learning

• Influence of educational tradition

• Faculty definitions of their roles

• Epistemology (beliefs about effective learning)

• Discomfort with change

Barriers to active learning

• Difficulty in covering assigned material

• Increase in preparation time

• Difficulty with large classes

• Lack of materials, equipment, resources

• Limited incentives to change

ENGAGING THE DATA

Arno Kumagai, M.D.

Data from the Donner Party

The Donner/Reed PartyMay 1846-February 1847

July 28 Party meets Lansford Hastings at Fort

BridgerDonner Lake

James & Margaret Reed

May, 1846: Donner/Reed Party leaves Independence

MO for Sutter’s Fort, CA

Rescued survivors arrive February 1847

Tasks

1. Write down 2-3 observations about the data. What do you see?

2. Suggest explanations (hypotheses) for the data.

3. How would you test your hypotheses?

Review the data sets provided in your packet:

Notebook pages 60-62

EVIDENCE-BASED DISCUSSION/TBL

Mary Kate Worden, Ph.D.

Biology Course at UVa

• Students assigned advance reading• Meet in small groups; given a modified

relevant journal article• Read modified article and answer

questions (in group)• Sample is in your notebook

Notebook pages 63-64

THINK-PAIR-SHARE

Think-Pair-Share

Pair up with someone in your small group:

• Choose one of the intended learning outcomes you wrote this morning

• Discuss and come to agreement on what methods you would use to help your students achieve that objective (refer to Bloom’s Wheel handout) Notebook page 65

• Discuss and come to agreement on how you would assess that students have achieved that outcome

Active learning strategies

Simple

Complex

Pause for Reflection

Interaction with Guest Speaker

Fishbowl Role Playing

Forum Theater

Simulations

Good practice in education*

1. Encourages contact between students and faculty

2. Develops reciprocity and cooperation among students

3. Encourages active learning

4. Provides prompt feedback

* Chickering and Gamson, 1987

Good practice in education*

5. Emphasizes time on task

6. Communicates high expectations

7. Respects diverse talents and ways of learning

* Chickering and Gamson, 1987

AN INTRODUCTION TO EDUCATIONAL ASSESSMENT

Assessment

Why do it?

What it is?

Assessment’s role in course development

Types of assessment

The future of assessment in medicine

Assessment drives student learning

Assessment drives student learning

What is the primary format for MS1 and MS2 student assessments?

What is one of our primary complaints about medical students (and residents)and their knowledge?

Assessment drives student learning

We assess to: Measure impact of

our curriculum Measure impact of

our instructional methods

Measure student achievement of curricular goals and objectives

Continuously improve upon all of the above

What exactly is assessment?

More than just a grade…it’s a means to understand learner and instructor performance

“Assessment is a mechanism for providing instructors with data for improving their teaching methods and for guiding and motivating students to be actively involved in their own learning. As such, assessment provides important feedback to both instructors and students.” (Angelo & Cross, 1993, p.4)

Types of assessment

Direct Student Assessment Techniques

The assessment is based on an analysis of student behaviors or products in which they demonstrate how well they have mastered learning outcomes. (formative v. summative)

Indirect Student Assessment Techniques

The assessment is based on an analysis of reported perceptions about student mastery of learning outcomes. (eg: survey,

Interview, focus group)

Program Assessment Techniques

Characteristics of Good Assessment Valid—directly reflects the learning outcome

being assessed Reliable—especially inter-rater reliability when

subjective judgments are made Actionable—results help faculty identify what

students are learning well and what requires more attention

Efficient and cost-effective in time and money Engaging to students and other respondents—so

they’ll demonstrate the extent of their learning Interesting to faculty and other stakeholders—

they care about results and are willing to act on them

Triangulation—multiple lines of evidence point to the same conclusion

Difference between formative and summative assessment Summative – most common,

approximation of achievement at x time. Mid-Term Examination Final Examination

Formative – Informal, feedback driven. Self-test Background knowledge probe Muddiest point (what is the most unclear

element)

The future of assessment in medical education

Recognition that knowledge-based assessment is one of several tools available (but is your test reliable and valid?)

Focused on application of knowledge

Competency driven

Increasingly qualitative in nature OSCEs SPs Portfolios

Resources

Some Assessment Techniques – CSU Fresno

Epstein – NEJM

Application exercise

• Using the objectives previously developed, convene in small groups to develop a strategy for assessing your objective.

• Share with the larger group. Tell us why the chosen assessment technique is appropriate.

Notebook pages 116-117, 121-122

Summary

At the end of this workshop, you should be able to:1. Write effective intended learning

outcomes (ILOs)2. Distinguish between effective and

ineffective ILOs3. Evaluate/critique ILOs4. Recognize the value – for the

learner and the teacher – of effective ILOs

5. Describe their beliefs about learning 6. Apply those beliefs to their own teaching

& learning7. Distinguish between active and passive

learning (theory)8. Describe passive methodologies and

active methodologies (applied)9. Compare and contrast active and passive

learning (advantages and disadvantages of both)

10.Describe the role of assessment in contemporary medical education

Intended learning outcomes

11.Identify methods of assessment

12.Determine which method of assessment is most appropriate in specific scenarios

Workshop follow-up

cwhite@anesth.ufl.edu

Index card: Please write down one thing you will or want to change; leave on the

table

THANK YOU!!!