C O M P E N D I U M O F O R I G I N A L W O R K O F G E N E H A R K L E S S I N T E G R AT E D W I T H

S L I D E S F R O MM A R I LY N H . O E R M A N N, P H D , R N, FA A N,

A N E FP R O F E S S O R A N D C H A I R , A D U LT / G E R I AT R I C

H E A LT HM O E R M A N N @ E M A I L . U N C . E D U

T H E U N I V E R S I T Y O F N O R T H C A R O L I N A AT C H A P E L H I L L

Best Practices For Clinical Teaching and Evaluation

Disclaimer“Education is not a science, it is a

complex set of practices that is grounded and principled but not

rule-governed.” Shulman, 2004

What links all clinical education?

Practical application of decision-making Professionals deal with ill-structured

problems under conditions of uncertainty

"What is not surrounded by uncertainty cannot be the truth.”

Richard Feynman cited by Timothy Ferris in M. Feynman (2005),

Perfectly reasonable deviations from the beaten track: The letters of Richard P. Feynman. New York: Basic

Books.

Research on Clinical Teaching

Studies on qualities of effective clinical teachers = 290

Sufficient research

Stresses of clinical teachers = 15 studies

Multiple demands Heavy workload

Balancing teaching with activities of personal importance

Pressure to maintain clinical competence or a practice…no time

Balancing demands of students, clinical staff, others

Teaching inadequately prepared students

Good Clinical Teaching

Interactional processEstablish climate for

learning and evaluation

Student in clinical setting is learner, not a nurse

Clear expectations Need time to learn

before being evaluated

Students make mistakes

Learn how to prevent those mistakes next time

Do not expect perfection

Students think teacher’s role is to evaluate BUT the primary role is to EDUCATE

Good Clinical Teaching

Give feedback Most important

variable affecting learning

Formative (improve performance)

Specific, informational

Continuous within clinical learning experience

Provide deliberate practice

Repetitive practice of skills (cognitive, motor)

Assessment of performance + feedback

Strong association between extent of practice and performance

Challenges of Clinical Teaching

Stresses of students in clinical practice = 28 studies

Fear of making mistake that would harm patient

Interacting with teacher, other providers, patients, staff

Changing nature of patient conditions

Lack of knowledge and skill

Being unfamiliar with clinical setting

Challenges in Clinical Teaching

Lack of evidence to guide clinical education practicesComplex practice

environments Increased complexity

and acuityContinued shift of care to

communityNew technologies, highly

specialized interventionsFocus on quality and

safety

Traditional model of clinical teaching

Clinical learning dependent on:

Available patients and experiences

When students “there”

Research findings: graduates not well prepared

Practice is Essential

Arthur W, Bennett W, Stanush PL, McNelly TL. Factors that influence skill decay and retention: A quantitative review and analysis. HumanPerformance.1998;11:57-101.

Meta-analysis of 53 studies on skill decay1. Substantial loss of acquired skills

from nonuse or lack of practice2. Skills not practiced or used for 1

year: Average participant performs at less than 92% of original skill

3. Skills taught early in nursing program that are not used are not retained

Patient Assignment Choose variety of clinical learning activitiesPatient care, but not all “complete care” Other learning activitiesFocused on clinical competencies of course

and students’ learning needs

Clinical Teaching Methods

Clinical Teaching Methods

Investigate pain management strategies used on your unit. How do they compare to the literature? To what you learned in class?

What are similarities and differences across patients, and why? Present in conference.

Describe a problem with quality of care. Write a 1-page report about the need for a QI project.

Asking Higher Level Questions

What studies have been done? = 18

Consistent findings: Teachers and

preceptors ask low level questions (knowledge, recall) during clinical practice and in discussions

Most questions seek yes/no response

Guide thinking about patients, practice… What did you notice? What did you expect?

Find?What are other

possibilities? Perspectives? etc.

Explore student’s understanding

Short Integrated Cases

Focus on specific outcomes to be learnedIntegrate concepts, classes, readingsPresent new or different scenarios than prior

examples

Sample Case #1

A patient is transferred to your unit from a community hospital with headache, nausea, and vomiting. The patient’s headache is getting progressively worse, and she is losing vision in her right eye.1. What data are most important and why?2. What are the next steps?3. Prepare a report on this patient for

handoff.

Sample Case #2

You make a home visit to an elderly patient with depression who is having loss of memory and crying spells. The patient is taking a selective serotonin reuptake inhibitor (SSRI). Two weeks later, you visit again, and the patient also has loss of appetite and difficulty sleeping.1. What do you notice in this situation?2. Provide alternate explanations for the

patient’s current symptoms.3. Discuss the case with a peer. Decide on

next steps to be taken.

Unfolding CasesChanging Case Scenarios

Sample Unfolding Case

You are making a home visit to see a 71-year old woman who has a leg ulcer that began after she fell. The patient is coughing and wheezing; she says she “feels terrible.”1. What additional data would you collect in

the initial assessment? Why?2. What actions would you take during this

home visit? Provide a rationale.

Case continues…

In 3 days you visit this patient again. She has shortness of breath, more fatigue, and appears cyanotic around her mouth.1. Does this new information change your

impression of her problems? Why or why not?2. List priority problems for this patient with

a brief rationale.3. What is the pathophysiology of cyanosis?

Case Continues

The patient recovers from that episode, and you are able to visit her one more time. At this last visit, she is still short of breath but otherwise seems improved.1. Write your final report on this patient.

Short Written Assignments

Prevent summarizing what others have written

Promote students’ thinking about patient care

Can be done in clinical conferences and critiqued by peers

Focus on outcomesDescribe how your patient’s treatments and interventions are similar to or different from your readings and why. (1 p.)

Concept Maps

Promote meaningful learning

Are additional resource for learning

Useful to provide feedback to students

Assess learning and performance

Complete from readings to assist students in linking new facts and concepts to their patients

Prepare for clinical practice

Present in pre clinical conference, revise during

clinical practice, discuss in post clinical conference Develop collaboratively

by students in conference

Nursing Care Plans

Enable students to analyze patients’

problemsdesign plans of care select evidence-

based interventions identify outcomes to

measure Should be usable,

realistic

Do they promote problem solving and higher level thinking?No research to support Do students only

summarize from textbook without thinking about information?

ProbablyHow many in a course?

What type?Depends on purpose

Pre Clinical GoalsAre students

prepared for assignments?

Help set priorities for care

Decrease student anxiety

Post Clinical Conference Goals? Reflect on clinical

practice and decisions

Focused discussionsEvidence suggests…

Pre/ Post Clinical Work

What studies have been done? = 193

Evaluation tools and their development

Specific methods, eg, portfolios n=20

What methods work best for particular competencies, settings, etc.?

Survey of 1573 nursing facultyPredominant strategy:

Observation of student performance (n=1289, 93%)

Other clinical evaluation methods: Written assignments (85%), skills testing, conferences, student self-assessments

Most clinical courses pass/fail (n=1116, 83%)

Clinical Evaluation

Clinical Evaluation

Most use clinical evaluation tool (n=1534, 98%)Same tool in all courses but modified to reflect

unique aspects of each course (n=1095, 70%) Process by which judgments are made about

performance in clinical practice Data Judgment

Oermann MH, Yarbrough SS, Ard N, Saewert KJ, Charasika M. Clinical evaluation and grading practices in schools of nursing: National Survey Findings Part II. Nurs Educ Perspect.2009;30: 352-357 .

Clinical Evaluation

Survey of 1573 nursing faculty Predominant strategy:

Observation of student performance (n=1289, 93%)

Other clinical evaluation methods: Written assignments (85%), skills testing, conferences, student self-assessments

Most clinical courses pass/fail (n=1116, 83%)

Most use clinical evaluation tool (n=1534, 98%)

Same tool in all courses but modified to reflect unique aspects of each course (n=1095, 70%)

Oermann MH, Yarbrough SS, Ard N, Saewert KJ, Charasika M. Clinical evaluation and grading practices in schools of nursing: National Survey Findings Part II. Nurs Educ Perspect.2009;30: 352- 357 .

Evaluation

Formative Evaluation Feedback to

learnerProgress toward

meeting outcomesNOT GRADED

Summative EvaluationAchievement of

outcomes, competencies

End-of-instruction evaluation

GRADEDClimate for Clinical Evaluation

Clinical evaluation is a PUBLIC EVENT

Clinical Evaluation Methods

Decisions:1. What is

being assessed?

2. Formative or summative?

3. How many times to evaluate each outcome?

Observation: Notes about

performanceChecklists Standardized

patients /Objective Structured Clinical Examination

Written assignments Rating scales SimulationsReflective journals

Clinical Evaluation Methods

Nursing care plansConcept mapsCase method,

unfolding cases case study

Papers

PortfoliosConferencesCasesProcess recordingGroup projects Self-assessment

Some Problems with Observation

Your values and biases

Research suggests over-reliance on 1st impressions

Window of time“Good data” but

incorrect judgment

So,

Don’t rely on it

Notes about performanceDetect patterns of performanceFormativePDA, netbook, checklistRating scale/clinical evaluation tool Summative

Documenting Observations

Clinical Evaluation Tool

Consistent with outcomes/competencies

ValidDoes tool collect

intended information about performance?

Does tool measure safe, effective practice?

ReliableSame results when

used with different student groups and by different faculty?

Avoid drift away from meaning/interpretation over time

Appropriate number of competencies?

Improving Use of Clinical Evaluation Tool

Have regular discussion of competencies to be rated

Meaning of each competency

How to assess itWhat would

performance look like to pass or fail or achieve each rating level

Use simulation to facilitate discussion

Observe a performance, assess it with rating scale, and discuss rationale for rating

Annual evaluation of tool and process

Observation of Performance

Problems with Observations by Preceptors Their values and biasesTheir standards for

comparison Limited understanding

of meaning of course outcomes and competencies How to judge the

quality of their observations?

Critical Faculty Decision Which clinical

competencies need to be observed and performance rated?

Prepare preceptors to observe and rate performance

Simulation, standardized patients, videotapes

of performance for practice

ePortfolios

Collections of projects by students Demonstrate achievement of outcomes Show work completed over period of time

TypesBest work (graded) Growth and development (formative)

Using Simulation for Clinical Evaluation

Competencies to be assessed?

Are simulations available or need to be developed?

Formative or summative?

Rating forms Timing in course

Objective Structured ClinicalExamination (OSCE)Evidence Students rotate through

series of stations Clinical—interaction with

simulated patientPractical—demonstration

of motor skills, techniques Static—evaluation of

cognitive skillsPerformance assessed Summative

Learning and clinical teaching nuggets….

Clinical teaching requires a process of

“uncovering” the material rather than trying to “cover it all.”

Fam Med 2003;35(3):160-2.

“The single most important thing

influencing learning is what

the learner already knows. Ascertain

this, and then teach her/him accordingly”

Ausubel

Copyright ©2003 BMJ Publishing Group Ltd.

Spencer, J. BMJ 2003;326:591-594

Who, what, and hows of Clinical Teaching

Experience occurs where design and intention collide with chance.

Shulman (2004). Teaching as

Community Property.

Theory for practice

Be aware that…

As practical knowledge develops we typically look back on theoretical preparation and begin to devalue it.

Theory may be more radical and reform-oriented than practice itself.

Theory may pull the “bungee” cord taut but then the recoil is caused by the conservation of habits of practice.

Shulman, 2004

“Responsibility of the developing professional is not simply to apply but

to transform, adapt, merge, synthesize, criticize, to INVENT in order to move from theoretical knowledge to practical knowledge needed to engage

professional work.” Shulman, 2004

KnowledgeApplication

Situated practice

JudgmentTechnical/ Moral

Theory helps…

Copyright ©2003 BMJ Publishing Group Ltd.

Spencer, J. BMJ 2003;326:591-594

Learning context for the educator

More theory help

Stages in skill acquisition

Educational objectives (teacher focus)

versus

Learning outcomes (learner focus)

Three central challenges of learning AKAthe taxonomy of pedago-pathology

AmnesiaI forgot

Illusory understanding

I thought I understood it

Inert ideasUselessness of learning

ACTION!!!!!

So what are you to do?

Skills and strategies

Engaged teaching

“Never underestimate the person's intelligence, but don't overestimate their knowledge.”

Jacobson as quoted in Spencer, BMJ, 2003.

Engaged Clinical Teaching

Avoid MEGO My eyes glaze over

Maintain a flexible stance

Avoid yes/ no questions What do you think is most important in

what I just told you? How would you summarize this case?

http://www.sru.edu/pages/6384.asp Adapted, with permission, from Case, Betty. (1999). Advanced Practice Nurse Preceptor

Workbook. Chicago: Niehoff School of Nursing, University of Chicag

Planning and Preparing

Agree on Daily Goals Limit the Number of Patients That Your Student

Sees Encourage “Just in Time” Learning Debrief and Plan for the Next Session

Maximizing LearningEfficiency

Use the Five Clinical Teaching MicroskillsMake Feedback Routine

Feedback that is based on observation, consistent, fair, routine, and given in a spirit of unconditional positive regard will be accepted and appreciated

Molodysky, E. (2007). Clinical teacher training--maximising the 'ad hoc' teaching encounter. Aust Fam Physician, 36(12), 1044-1046.

Five microskills for the one minute preceptor

Case example Five-step Microskills

Teaching With Patients/ Clients

Develop a cadre of “Teaching Patients”Seize unexpected learning opportunitiesHear presentations with the patient/ family

present.

Using Service Learning

Use the students for select administrative tasks

Let students write notesAllow students to teach patients

O-RIMEDeveloped by Lou Pangaro, 1999

May provide feedback focused on what information the student is sharing with you.

O: Observer (passive)R: Reporter (data gathering)I: Interpreter (differential nursing

diagnosis)M: Manager (formulates a treatment plan)E: Educator (reads up on a topic, teaches the team)

RIME Case Example

Problem learners

Be sure you are solving the right problemSimply share your observations and ask for

the student interpretationIs it a cognitive, affective, or psychomotor

problem?Ask the student to identify factors

contributing to the problem and possible solutions.

Safety

Okay to ask questions

Okay to not know answers

Able to think out loud

Able to make verbal mistakes

Gentle correctionDon’t ask junior

learners questions already missed by senior learners

Faculty admit their own limitations

System safety

Safe for patientsprevent learner

mistakes from causing harm

transparent learner progress that faculty can safely manage

Incorporate patient into learning process

Safe for teachercan trust

information from learners (no

fabricating to appear competent)learners more honest about their

own limitations

Effective feedback

Descriptive and non-judgmental

SpecificBehaviorally

anchoredWell-timed :

frequent, regularPositive as well

as corrective

Anticipatory guidance

Helps learners develop a personal feedback

mechanismPre-requisite for

behavior modification

Feedback

Praise

Criticism

Praise

ASK What did you do well? What did you struggle

with? TELL / TEACH Agree / Disagree

Focused Teaching about General Rules

Scripts Begin to Emerge

ASK WHAT and HOW can we

(you and I) improve? Demonstrate /

Practice / Read

Teaching Effectiveness Instrument

Challenges

Expect ambiguityTake risksLearning is emotionalInfluence of personalitiesImportance of skepticism

Communication challenges

Clinical work requires that clinical decisions be made in the face of complex, contradictory, and often incomplete information.

Ultimately, one of the most important results of

all coursework is to inculcate clinical decision making skills.

Copyright ©2003 BMJ Publishing Group Ltd.

Spencer, J. BMJ 2003;326:591-594

Situating formative and / or summative evaluation

Clinical SkillsSample feedback form

Educating for future clinical practice

Thompson, Kershbaumer, and Krisman-Scott (2001) suggest that clinical educators teach critical thinking skills so that the practitioner is

a detective in taking a thorough and focused history,

reflective about the information gathered from the history and physical and

ultimately effective in assessment, management, and follow-up.

New Tools in Health Profession Education

Learning artifacts

Sophisticated, teaching resources

Websites Blogs Wiki Podcasts Videocasts

http://www.youtube.com/watch?v=dGCJ46vyR9o

http://blogs.usask.ca/medical_education/archive/2007/10/

http://www.biomedcentral.com/1472-6920/6/41

Integrating apprenticeships

Carnegie Foundation for the Advancement of Teaching

An apprenticeship to the ethical standards, ethical comportment, social roles, and responsibilities of the profession, through which the novice is introduced to the meaning of an integrated practice of all dimensions of the profession, grounded in the profession's fundamental purposes

A skill-based apprenticeship of practice, including clinical

judgment.

Intellectual training to learn the academic knowledge base and the capacity to think in ways important to the profession.

http://72.5.117.129/programs/sub.asp?key=1829&subkey=2309&topkey=1829

In closing

“Learning is least useful when it is private and hidden, it is most powerful when it becomes public and communal. Learning flourishes when we take what we think we know and offer it as community property among fellow learners so that it can be tested, examined, challenged, and improved before we internalize it. “ Shulman, 2004

GENE.HARKLESS@UNH.EDU

Thank you