Outcomes Methodologies in the Changing Landscape of CME

All Rights Reserved, Duke Medicine 2007This event is powered by BeaconLive Webinars | Webcasts | Online Events

How do youdeliver your CME?

Just like your CME, technology delivery is ever-evolving. Pair your CME with BeaconLive’s cutting edge delivery methods.

BeaconLive | Webinars | Webcasts | Online Events Learn More: BeaconLive.com/Healthcare

Outcomes Methodologies in the Changing Landscape of CMEChitra Subramaniam, Ph.D

Assistant Dean and Director, Continuing Medical EducationAssistant Director, Center for Educational Excellence

This webinar is brought to you by

BeaconLiveWebinars | Webcasts | Online Events

All Rights Reserved, Duke Medicine 2007This event is powered by BeaconLive Webinars | Webcasts | Online EventsThis event is powered by BeaconLive

About Me

Education:• Ph.D in Curriculum and Instruction with a focus on Instructional Technology; Cognate: Measurement and

Evaluation• 10X10 AMIA training in Biomedical Informatics• MS Clinical Biochemistry

Areas of Expertise and its Application:Basic :Curriculum and Instruction

Education Evaluation and Measurement

Information Processing/The learning process

Knowledge Repositories and Instructional Design Systems

Data Visualization

Information/Interaction design; Usability

Applied: Design of Knowledge Systems for Outcomes based Education

Measurement and Evaluation methods; Impact of Learning

Instructional Design Systems; Instructional design strategies, teaching

Integration of Technology in Education

Information Design


Those who have been influenced!• AO Foundation• American College of Cardiology• American Society for Clinical Pathology• American College of Radiology• CDC, Ministry of Health, Tanzania• National Quality Laboratory Training Center, Tanzania• Pearson Custom Solutions, Pearson Publishing• McGraw Hill• Cengage Learning• Thomson/Delmar Publications• Center for Medicare and Medicaid Services• Council for Exceptional Children, Washington D.C• University of South Florida• University of Central Florida• Lincoln Technical Schools• US Education Corp• City College• Kaiser University• Hernando County School Board• Hillsborough County School Board• IBM, AT&T• Learning Mate, India• Chartered Institute for Professionals in Financial Accounting, U.K• American Society for Clinical Pathology


Polling QuestionRank the # 1 challenge in measurement and evaluation you face today

1. Concepts are too abstract and cannot be operationalized

2. Data has too much variability

3. Results are mostly subjective. Objective measurements are a challenge

4. The system does not allow for a streamlined approach to measurement

5. None to very few tools available for implementation

6. Accountability is too high- cannot meet expectation

7. Have little or no access to data; not within a health system


My Perspectives• Outcomes is an afterthought in most organizations both for

education and training. • Strategic and tactical frameworks for achieving outcomes in

most organization is lacking• Lack of clear mapping and alignment of the outcomes

methodology with the instructional design and delivery strategies and learner engagement approaches.

• At a macro-level, the educational program goals do not align with institutional goals.

• The process of learning and factors influencing the learning (individual, group and system level) are not being considered.

• Protocols for programs that include QI/PI, research, collaborative team based learning are not clearly defined.


A Paradigm Shift in Healthcare

From Here To Here

Individualistic Collaboration

Autonomous Integrated

Scholarly Practical

Expert Centered Focus Patient/Service Team

Hierarchical Distributed


Healthcare Environment…..Patient Centered

Personalized Care

Diagnosis, treatment, and management- the process

Behavioral, cognitive, social, economic and cultural factors that impact the physical, mental and social well being of a patient

Integrated system- Its about the experience

Micro and Macro level issues

Accountability/Standardization/health outcomes

Health care costs- Sustainability

Motivated, disciplined patientLife Long Learners


The Challenge

Influencing Factors

Research

Quality Education


Performance Needs

Learning Needs

Business Needs

Learning Objectives

Application Objectives

Impact Objectives Impact and

Consequences

Application Implementation

Learning and

Confidence

Begin with the End In Mind!


Business Needs

Job Performance Needs

Learning Needs

Design

Analyze DevelopEvaluationAssessment

Needs Assessments,Evaluations and survey results,Market ResearchNew developmentsAnd trends, Teaching philosophy, goals and vision, situated learning

Objectives/learning outcomes, content, instructional design, assessment measures, supplemental materials, resources, modalities and technology

Formative/Summative evaluations, resources, support services, supplemental materials

Implement

What does that mean?


SYSTEM NEEDS

Evidence Based Medicine

Quality Improvement

Performance Metrics

Business practices

Delivery Models

Clinical Research

HEALTHCARE DELIVERY

Healthcare Teams

Coordination of Care

Healthcare Disparities

Health Information Technology

Pay for Performance

Life Long Management of

Diseases

CPD/CE

Healthcare Disparities, Episodic to life long treatment, Patient expectations and demands are different, Patients are a part of the team

Learner Centered Education/System Driven Training

Quality

Research Education


Display knowledge, competence, performance at predefined level/benchmarks

Develop contextual competence, adapt to change, recognize patterns in problems within a context

Adapt to change, find, validate and apply new knowledge

Develop competence further through habits of mind, behavior, and wisdom

Continuously add to knowledge, competence and performance through practice and reflection on experience

Assess “the habitual and judicious use of • Communication• Knowledge• Technical skills• Clinical reasoning• Emotions• Values• Reflection in daily practice”.

Epstein, MR., Assessment in Medical Education, N. Engl. J Med Jan 25, 2007. 356;4

Teacher/Instructor

Facilitator

Coach/Curator

FEEDBACK and POSITIVE REINFORCMENTS


Polling QuestionsOf the 5 stages discussed at which level do you think your CE activities are designed and measured?

1. Display knowledge, competence, performance at predefined level/benchmarks

2. Develop contextual competence, adapt to change, recognize patterns in problems within a context

3. Adapt to change, find, validate and apply new knowledge

4. Develop competence further through habits of mind, behavior, and wisdom

5. Continuously add to knowledge, competence and performance through practice and reflection on experience


Performance

Knowledge

Competence

Moore’s Outcomes Levels

Bloom’s levels

Participation, Satisfaction

Performance

Learning- Declarative Procedural

Patient Health

Community Health

Competence

Population Health

KNOWLEDGE

APPLICATION

ANALYSIS

CREATE

EVALUATIONLearner Needs

UNDERSTANDING

Learning FormatsInstructional

StrategiesM & E methods and

tools

Constructive Alignment


Pre-structural Uni-structural Multicultural Relational Extended Abstract

Quantitative Phase Qualitative Phase

Identify

So simple

procedures

Enumerate

Describe

List

Combine

Do Algorithms

Compare

Contrast

Explain

Relate

Analyze

Theorize

Generalize

Hypothesize

Reflect

The Hierarchy of Verbs

Deep Learning


Motivation, stress, time constraints, cognitive overload

Metacognition, critical thinking/reasoning, problem recognition

System/Business

Healthcare Consumers



In case there was a story to tell….

Member Engagement

• At the basic level, any product or service that meets customer needs can derive growth in sales, participation, brand identity and word of mouth.

• Consumption is a form of participation or engagement• Can be measured in money, satisfaction, participation, idea

generation, retention, money invested by customers, number of spin off products.

• Directly links to the professional advancement of the member.

• Product/service level engagement

Educational Value

Activities’ influence on performance, habits and perhaps even outcomes.

Activity integrates into professional practice Activity-satisfaction Activity addresses needs Activity allows for knowledge and skills transfer Format of the activity suitable, convenient and easy to

follow Content relevant to practice and the needs

Return On Investment

A traditional financial measure based on historic data. Trends from financial performance Rate of increase/decrease in revenues Marketing ROI Retention and Acquisition rates

Strategic Effectiveness

Organizational strategy the association chooses to deploy (product leader, most advanced, price leader, cheapest, most customized) based on

o Strategic organizational goalso Member value proposition


A systems approach to desired resultsFramework for Program planning and Evaluation


How do you measure quality?

Improvement Tools • Continuous quality improvement (CQI) –

opportunity for improvement exists in every process on every occasion.

• CQI model emphasizes view healthcare as a process and focuses on the system rather than the individual when considering improvement opportunities.



The FADE Model FOCUS: Define and verify

the process to be improved ANALYZE: Collect and

analyze data to establish baselines, identify root causes and point toward possible solutions

DEVELOP: Based on the data, develop action plans for improvement, including implementation, communication, and measuring/monitoring

EXECUTE: Implement the action plans, on a pilot basis as indicated, and

EVALUATE: Install an ongoing measuring/monitoring (process control) system to ensure success.



PDSA PLAN: Plan a change or test of how something works.

DO: Carry out the plan.

STUDY: Look at the results. What did you find out?

ACT: Decide what actions should be taken to improve.

Repeat as needed until the desired goal is achieved



Six Sigma• Statistical measurement methodology designed to reduce

cost, decrease process variation, and eliminate defects• Sigma is a statistical unit reflecting the number of SDs a

given process is from perfection• Steps

• 1. Define – creation of project charter which defines the customer’s needs, project scope, goals

• 2. Measurement – data collection plan• 3. Analyze – data analysis occurs, deviation from standards is

identified• 4. Improve – create solutions and implementation plans• 5. Control – process is controlled by implementing policies,

guidelines, and error-proofing strategies to make reverting to old process impossible.


Things to consider

• Quality Improvement versus research• QI objective address need of local situation• Research seeks to address problems that will provide

generalizable results• QI can be considered research if:

• the tested intervention involves a deviation from establishedpractices

• individual patients are subject• randomization or blinding is conducted• Participants are subjected to additional risks/burdens beyond

usual practice.


Quality of Care Measurement

Costly to implement Quality Improvement programs Requires good data and evaluation of costs

Method Purpose Data Requirements

Cost-effectiveness analysis

Comparison of costs and health effects of a QI program vs. usual care

Utilization data for healthcare providersLong-term health benefits to patients

Cost-minimization analysis

Comparison of costs of two programs with identical health benefits

Same as above

Cost-benefit analysis

Comparison of program costs and benefits

Health benefits of programCost of programs


HealthCare Matrix

Bingham, J. and Doris, Q et al (2006). Using an Healthcare Matrix to assess patient care in terms of aims for improvement of core competencies. Journal of Quality and Patient Safety, Vol 31 (2).


The Duke Outcomes Matrix

• Practice gap matrix• This is where we started…….• Everything in between……• This is where we are now…….


Polling Question

Do you use tools to design, develop, implement and measure CE activities?

Yes

No


Outcome- Increase pneumococcal vaccination rates in patients at high risk (18- 64) and those 65 and greater

Physicians


Health Educators from the County

Just for us

• Reach out to all seniors and high risk groups through educational sessions within Durham, Wake and Orange Counties

• Communicate the importance of pneumococcal vaccines and the consequences of not obtaining one.

Outcome- Increase pneumococcal vaccination rates in patients at high risk (18- 64) and those 65 and greater

• During home visits for seniors, screen for pneumococcal vaccinations. • Document the screening• Ensure vaccination of seniors who need one and document it.


Outcome- Improve Quality of Life for HIV patients by empowering them and their care givers

Physicians, researchers, social workers/counselors, nurses and patients

Role Goals Measures

Physicians, Specialists Effectively communicate the need to maintain the treatment regimens.

Emphasize the need for mental, emotional and physical well being and empower the patient to pursue the best treatment that addresses their individual needs.

Work with the care giver team members to support the emotional well being of the patient

Implement evidence based strategies to communicate and build trust and connect with patient

During every patient encounter evaluate patient’s adherence to medication plans.

Assess the emotional, physical, and mental well being of the patient.

Document the assessment

Using coaching techniques, offer suggestions and strategies to enable patient self confidence

Present all available options based on the results of the assessment and discuss their advantages and disadvantages.


Roles Goals Measures/Indicators

Social Workers Guide HIV patients regarding their legal rights following disclosure of their status.

Serve as an advocate for the patient.


Implement evidence based strategies to communicate and build trust and connect with patients.

Upon disclosure, inform and document patient’s legal rights using case examples.

List and share all community, state and other resources available to the patient.

Communicate to the physician and other care givers if any changes in the emotional, mental or socio-economic condition is observed.

Follow up with patients on a regular basis to ensure that the support through resources are made available.



Nurses Follow all protocols during patient encounters

Provide guidance and support related to treatment regiments and their management


Implement evidence based strategies to communicate and build trust and connect with patients.

Document all patient vital signs, HIV health factors that are evaluated. Assess medication adherence and communicate to the care team appropriately.

Share with the care team any changes observed in the emotional, and physical well being of the patient.

Communicate with patients, physicians and specialists to ensure proper coordination of care.



Patients Be well informed about the disease, its diagnosis, treatment and management

Contribute to, engage and learn from the broader community

Ask questions of all care givers and communicate concerns and challenges.

Stay motivated and empowered to pursue the best treatment that addresses their individual needs

Participate in educational opportunities offered by the care givers.

Engage in online communities and share/exchange knowledge, experiences and resources.

During visits, clarify concerns and understanding of treatment options.

Participate in activities that lead to empowerment and motivation of oneself to lead a quality life.


Performance Dashboard


Questions and Next Steps

View these on-demand webinars at BeaconLive.com/healthcare-webinar-series

Involve Me! Reflection of Adult Learning Principals

Healthcare Reform 2.0: What it Means for CME and Medical Communication Professionals Automating your CME Enterprise with Learning Management Technologies

BeaconLive’s Healthcare Webinar Series

On-Demand Webinars


Questions and Next Steps

➔ Full Service Webinars➔ Onsite and Remote Webcasts➔ On-Demand Solutions➔ Continuing Education Platform➔ Integrated Registration

Contact BeaconLive today to learn how we work with the healthcare industry

John Craine | National Sales Manager, Healthcare Services [email protected] | 434.927.4122

BeaconLive’s Healthcare Webinar Series

Health & Medicine

Outcomes Methodologies in the Changing Landscape of CME