Upload
viet
View
65
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Promoting the Development of Clinical Skills throughout the Continuum of Medical Education University of North Carolina – Chapel Hill School of Medicine November 9, 2011. Ann C. Jobe, MD,MSN Executive Director Clinical Skills Evaluation Collaboration (CSEC). Clinical Skills in Practice. - PowerPoint PPT Presentation
Citation preview
Promoting the Development of Clinical Skills throughout the Continuum of Medical Education
University of North Carolina – Chapel Hill School of MedicineNovember 9, 2011
Ann C. Jobe, MD,MSNExecutive Director
Clinical Skills Evaluation Collaboration (CSEC)
Clinical Skills in Practice
• The physician-patient encounter is central to the identity of physicians in the US
• Clinical skills of trainees and young physicians have been described as deficient since at least the 1970’s
• Good evidence supports the diagnostic and therapeutic value of the clinical encounter but…
• …..Technology, fragmented care, reimbursement, and practice culture affect the clinical encounter
Weiner,A. & Nathonson M; JAMA 1976; 236:852-855Verghese, A et al; Annals Int Med 2011;155:550-553
Clinical Skills in Practice
• The clinical encounter is often buried in process measures, such as HEDIS or other guidelines
• The ritual value of the clinical encounter is important, and must be balanced by its documented utility
• The environment determines most of what and how trainees learn about the clinical examination
Weiner,A. & Nathonson M; JAMA 1976; 236:852-855Verghese, A et al; Annals Int Med 2011;155:550-553
COMMUNICATION
• The essence of the patient-physician relationship
• Includes communicating verbally, non-verbally, as well as actions and interactions during a physical examination
Communication
• It is all about COMMUNICATING with patients and families and health professionals
• It is all about improving communication to improve the quality and safety of health care
Why Assess Communication Skills?
• Essential physician competency • (LCME, ACGME, ABMS, USMLE)
• Clinical outcomes require effective
communication
• Public expectations: need for more
information and supportive interactions.
• Quality measures now incorporate
patient-centeredness
Patient-Centered Communication
• Exploring the patient’s illness experience
• Understanding the patient as a whole person
• Picking up on patient cues
• Involvement of the patient in problem definition• Involvement of the patient in decision-making
• (now >50% expect such involvement)
• Finding common ground regarding management• Enhancing the doctor/patient relationship by
being responsive to the patient IOM,2001; Street,2008
Communication Skills
• Prospective study of 80 medical outpatients with new or previously undiagnosed conditions
• Internists asked to list their differential diagnoses and to estimate their confidence in each diagnostic possibility • after the history, • after the physical examination, and • after the laboratory investigation.
Communication Skills
• In 61 of 80 cases (76%), the leading diagnosis after taking the history agreed with the diagnosis accepted at the time the record was reviewed• The physical examination led to the diagnosis in 10 patients (12%)• The laboratory investigation led to the diagnosis in 9 patients
(11%)
• These data support the concept that most diagnoses are made from the medical history
Communication Skills
• Authors suggest that more time should be devoted to improving history-taking skills during clinical training.
Peterson MC, Holbrook JH, Hales D, Smith NL, Staker LV: Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses.
West J Med 1992 Feb; 156:163-165
Communication Skills
• Numerous publications confirm that poor skills in patient communication are associated with:• Lower levels of patient
satisfaction• Higher rates of complaints• Increased risk of
malpractice claims• Poorer health outcomes
High level skills in “bedside medicine” – “clinical skills” • Ability to elicit a patient’s
story/history• Correct use of evidence-based
PE maneuvers in a focused manner based on history
• Ability to synthesize information gathered
• Ability to communicate and negotiate plans for management
are the cornerstone of patient safety and quality of care
Why Does It Matter?
• Initiatives focused on improving clinical skills, especially communication – through teaching and assessment - will be most successful in improving the quality and outcomes of care provided by health professionals
Comprehensive Program
• Overarching Competencies and Objectives
• Map for addressing teaching and assessing throughout the continuum of education• Course content• Assessment methodologies
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Professionalism• The ability to understand the nature of, and demonstrate
professional and ethical behavior in, the act of medical care.
• Patient Engagement and Communication Skills• The ability to engage and communicate with a patient, develop a
student-patient relationship, and communicate with others in the professional setting
• Biomedical Knowledge Application Skills• The ability to apply scientific knowledge and method to clinical
problem solving.
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• History Taking• The ability to take a clinical history, both focused and
comprehensive.
• Patient Examination• The ability to perform a mental and physical examination
• Clinical Testing• The ability to select, justify and interpret selected clinical tests and
imaging
• Clinical Procedures• The ability to understand and perform a variety of basic clinical
procedures
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Diagnosis• The ability to diagnose and explain clinical problems in terms of
pathogenesis, to develop basic differential diagnosis, andto learn and demonstrate clinical reasoning and problem identification.
• Clinical Information Management• The ability to record, present, research, critique and manage
clinical information
• Clinical Intervention• The ability to understand and select clinical interventions in the
natural history of disease, including basic preventive, curative and palliative strategies
AAMC Recommendations ForClinical Skills Curricula For Undergraduate
Medical Education(2008)
• Prognosis• The ability to understand and formulate a prognosis about the
future events of an individual’s health and illness basedupon an understanding of the patient, the natural history of disease, and upon known intervention alternatives.
• Personalizing Clinical Care• The ability to provide clinical care within the practical context of a
patient’s age, gender, personal preferences, family, healthliteracy, culture, religious perspective, and their economic circumstances
Core Competencies & Assessment
• Patient Care/Clinical Skills• Students must be able
to provide care that is compassionate, appropriate, and effective for treating health problems and promoting health
Core Competencies & Assessment
• Interpersonal & Communication Skills• Students must
demonstrate interpersonal and communication skills that facilitate effective interactions with patients and their families and other health professionals
Developing a Comprehensive Program
• Types of assessments
• Examinees
• Timing of assessments
• Types of assessments• Formative
• Designed to provide feedback to facilitate acquisition of new skills or improvement of performance
• Part of continuous professional development• Part of performance and quality improvement
• Types of assessments• Summative
• “High stakes”• Associated with an important decision – like
graduation, licensure, certification or credentialing• Utilized to distinguish between those who are
competent and those who are not
• Types of assessments• “Snapshot”
• One time assessment
• Longitudinal• Repeated over various periods of time
• Timing of assessments• At planned intervals for promotion decisions• Ongoing for continuous professional
development and/or performance improvement• One-time “snapshot” for initial licensure • Repeat assessment for license renewal• For credentialing or granting privileges• Review for re-entry into practice
Program Elements
• Depend on PURPOSE of the assessment
and• LEVEL of the
examinee
Assessing Skills and Performance
• What is included in an assessment of skills and performance?
• What are some of the assessment methods and how are they assembled?
• How do the methods perform against the criteria for good assessment?
Miller’s Pyramid for Assessing Clinical
Competence
Does
Knows
Shows How
Knows How
Knowledge
Performance
Competence
Action
Kirkpatrick Criteria
4. ResultsChange in organizational practice
Benefits to patients/clients
5. BehaviorTransfer learning to workplace
Learners apply new knowledge and skills
6. LearningChange attitudes/perceptions
Change knowledge/skills
7. ReactionCustomer satisfaction related to participation in
educational activities
Simulation
• Simulation• Real patients are
replaced with realistic but artificial experiences
• Trainee interacts with the re-creations
• Judgments are made about their performance
Simulation
• Methods can be divided according to how faithful they are to reality• Intermediate fidelity
• Task specific models
• Instructor driven models
• High fidelity • Virtual reality
• Standardized patients (SPs)
Method: Task Specific Models
• Designed around a specific task• Venipuncture model• Animal cadavers
• Usually not automated• Relatively inexpensive
Method: Instructor Driven Models
• Physical representation
• Responses driven by an instructor
• Little feedback
• Moderate cost
Method: Virtual Reality Simulators
• Simple physical representation
• Sensing device that informs computer of user actions
• Computer models realistic reactions• 3D imaging• Haptics
Method: Standardized Patients
• Individuals trained to portray a patient• Scripted and standardized
• USMLE Step 2 CS example• Integrated Clinical Encounter
• Data gathering• SP completing checklists
• Written communication• Doctor rating a patient note
• Communication & Interpersonal skills
• SP Rating
• Spoken English• SP Rating
Ideal Assessment of Communication Skills
• Evidence-based construct• Assessment instrument consists of observable
behaviors• Realistic stimuli
• SPs trained to use instrument reliably
• Appropriate scoring decisions
Putting it Together: Objective Structured Clinical Examination
(OSCE)• Multiple stations
• Each focused on a specific aspect of competence
• Stations might include• Manikins
• SPs
• ECG or X-ray interpretation
• Heart sounds
• Animal cadavers• Anastomosis• Laparoscopic vessel ligation
• Simulators
“In a way the OSCE is not an examination method; rather it is an examination format or framework into which many different types of test methods can be incorporated”
Ian Hart, 2001
Putting it Together: OSCE
• Stations are usually short: 10-15 minutes
• Test is composed of 8-25 stations• Round-robin format
• At a bell, examinees rotate to next station
• Can accommodate as many examinees as stations
• Total score is calculated across all stations
Work-based Methods
• Work-based assessment
• Real patient encounters
• Trainees are observed
• Judgments are made about their performance
“When your work speaks for itself, don't interrupt.”
Henry Kaiser
Work-based Assessment
• Foundation Programme (in the UK)• Two-year program
• Bridge between medical school and advanced training
• Series of clinical placements
• Assessment Purpose• Determine fitness to progress to
the next level• Identify trainees in difficulty• Provide feedback• Establish accountability
• Three methods • Mini-Clinical Evaluation
Exercise (mCEX) • Directly Observed
Procedures (DOPs)• Case-Based Discussion
(CbD)
Mini-Clinical Evaluation Exercise (mCEX)
• Process• List of patient problems
• Trainee picks a patient
• Assessor observes the encounter
• Focused clinical task
• Assessor rates:• Hx, PE, Communication, Clinical
Judgment, Professionalism, Organization/Efficiency
• Assessor provides feedback
• Takes 15-20 minutes
Directly Observed Procedures (DOPs)
• Process• List of procedures• Trainee picks a patient• Assessor observes the
encounter • Procedure
• Assessor rates:• Preparation, Sedation,
Asepsis, Technical skill, etc.
• Assessor provides feedback
• Takes 15-20 minutes
Case-Based Discussion (CbD)
• Process• List of patient problems• Trainee picks 2 case records
• Assessor selects one
• Discussion centered on the trainee’s notes
• Assessor rates:• Diagnosis, Treatment,
Planning, Professionalism, etc.
• Assessor provides feedback
• Takes 15-20 minutes
Putting it Together: Work-based Assessment
• An OSCE “on the hoof”• Multiple encounters are
needed• Captured as feasible
during clinical training
• Multiple examiners are needed
• Encounters can be made to conform loosely to a problem list
• Ongoing, longitudinal assessments
Criteria for Judging an Assessment
• How do simulation and work-based assessment perform against the criteria?• Validity • Reliability• Equivalence• Educational effect• Opportunity for feedback• Feasibility
Validity
• What is validity?• Degree to which the
inferences based on scores are correct
• Does the test measure what it is supposed to measure?
• Simulation • Good content coverage
• Rare conditions• Errors cause no harm
• Good fidelity
• Work-based methods• Excellent content
coverage• Includes difficult to
simulate conditions
• High fidelity
Reliability
• What is reliability?• If an assessment process
is repeated with the same trainees, they should get the same scores
• Physician performance varies considerably from patient to patient• The trainee must be
observed with several patients
• Assessors differ in stringency• The trainee must be
evaluated by different examiners
Equivalence
• What is equivalence?• To compare examinees
they must have taken assessments that are equal in difficulty
• Fairness • Comparable meaning
• Simulation• Different examinees can
be given the same items• Security
• Statistical techniques help with different versions
• Work-based methods• Equivalence is a
problem that can be mitigated but not eliminated
Educational Effect
“Students respect what you inspect.”
• Both simulation and work-based methods signal the importance of working with patients• Drives learning
Opportunity for Feedback
• Feedback is critical to learning • General education (Hattie,
1999)• Meta-analysis of 12 meta-
analyses
• Feedback is among the largest influences on achievement
• Medical education (Veloski et al., 2006)
• Feedback alone is effective in 71% of studies
• Simulation• Amount of feedback varies
by method• Depends on deployment• Lower for instructor driven
methods • Higher for model driven
methods
• Work-based methods• Trainees rarely observed• Provides an excellent
opportunity for feedback following observation
Feasibility
• There are significant resource constraints in most educational programs
• Simulation• Purchase, maintenance,
logistics
• Case development
• SP/Observer training
• Work-based methods• Faculty development
• Logistics
Summary: Assessment of Skills and Performance
• Trainees must ‘show how’ • Simulation
• Can produce equivalent scores
• Work-based methods • Cover more patient problems• Can be more feasible
• Both methods• Require multiple patients and
examiners • Have positive educational
effects• Provide opportunities for
feedback
Finding Opportunities
• Seeking out the “best practices” already in place across the organization
• Disseminating and seeding what is working to other areas
• Finding ways to maximize synergy of work already in place
Opportunities Along the Continuum
• Assessment of team member performance
Opportunities Along the Continuum
• Assessment of outcomes of a team’s performance
Opportunities Along the Continuum
• Assessment of individual team members – using “standardized team members”
Opportunities Along the Continuum
• Assessment of teams composed of members of several health professions
Opportunities Along the Continuum
• Standardized Patient assessments/ OSCEs & simulations for:• Incoming residents• Residents moving into
supervisory roles• Residents at completion of
residency• New medical staff –
credentialing review and privileging
• Individuals who are re-entering practice
Opportunities Along the Continuum
• “Secret Shoppers” -standardized patients in clinical settings assessing clinical skills of:• Residents• Faculty• New medical staff –
credentialing review and privileging
• Individuals upon re-entry into practice
Most Important Consideration
• A Comprehensive Program based on • Well defined Purpose and Goals• Overarching Competencies and
Objectives• A detailed “Map” that covers the
timing and methodologies of assessments across the continuum
• Focused efforts on gaps in teaching and assessment
• A well thought out evaluation of the program
• Providing data and evidence supporting the benefit to patients and improvement in care
Why Does It Matter?
• Initiatives focused on improving clinical skills, especially communication – through teaching and assessment - will be most successful in improving the quality and outcomes of care provided by health professionals
THANK YOU
Let us continue on the journey together –
improving how we care for our
patients