Welcome and Introduction · •Formative/Summative Evaluations ... Process Evaluation-Grant Management, ... • Certify nurses to use telehealth equipment

2

Welcome and Introduction

Mary Beth Bigley, DrPH, MSN, APRN

Director, Division of Nursing and Public

Health, Bureau of Health Workforce

Health Resources and Services

Administration

Barbara F. Brandt, PhD

Director, National Center for

Interprofessional Practice and

Education

3

Follow the Conversation

Join a real-time conversation about this webinar on Twitter.

#CommunityIPE

4

Featured Speakers

Jennifer Morton DNP, MPH, APHN

University of New England

Susan Kimble, DNP, APRN

University of Missouri, Kansas City

5

Measuring the Impact: Evaluating

Interprofessional Education and Care

in Community Clinical Settings

Jennifer Morton DNP, MPH, APHN, University of New England

Susan Kimble, DNP, APRN, University of Missouri, Kansas City

6

Objectives

• Illustrate how National Center resources can be used to

support IPECP studies

• Share how program plans, including evaluation, translate

across various demographics

• Describe attributes that contribute to a high-functioning

interprofessional team

• Describe how an interprofessional student interface is

beneficial to a clinical partner’s organizational culture

• Showcase the linking of isolated IPECP efforts

7

Team Identity Exercise

What structural(brick) and philosophical(mortar) attributes

contribute to your team identity?

Learning with, from, and about...

8

Innovative Models of Education, Training,

and Service Delivery Health, Resources and Services Administration (HRSA)

• Bureau of Health Professions

• Nurse, Education, Practice, Retention and Quality

(NEPQR), IPE grants, 2012

• University of New England

• University of Missouri, Kansas City

9

A Tale of Two Cities

Portland, Maine

• CHANNELS Project-improving the health of immigrant/refugee

communities through innovations in team based care

• Population = 66,650

• Community of Interest = Immigrant and Refugee (13,000),

resettlement area

• Partnership with local health department and FQHC

1 0

A Tale of Two Cities

Kansas City, Missouri

• Bridging the Disparities Gap: Strengthening Community-based

Healthcare Delivery in an Urban Setting

• Community of Interest=urban underserved including immigrant and refugee at the two community-based urban HPSA clinics

• Urban population living at 50% at or below 200% of the federal poverty level

• Patient population, 41% uninsured, 39% receiving Medicaid (RWJF, 2011)

• Partnership with FQHC and urban faith based clinic

1 1

The University of New England

• Health Sciences College within a University in Portland, Maine

• Educates the largest number of health professions students in the state

• Relies on partnerships for clinical/practice experiences, no academic health center on site

• Robust IPE infrastructure

• “Private university with a public mission”-President Danielle Ripich

1 2

The University of New England, cont.

• Nursing

• Nurse Anesthesia

• Physician Assistant

• Physical Therapy

• Occupational Therapy

• Dental Hygiene

• Social Work

• Athletic Training

• Applied Exercise Science

Other Colleges:

• Osteopathic Medicine

• Pharmacy

• Dental Medicine

1 3

To improve the health of immigrants and

refugees through innovations in

community-based interprofessional care

1 4

University of Missouri, Kansas City

• BSN completion and BSN-pre-

license

• First DNP in state (post-MSN)

• MSN tracks

• CNS, NE, NNP, PMHNP

• BSN to DNP

• AGNP, FNP, PNP, WHNP

• Nursing PhD

• Health Science Campus:

Pharmacy, Medicine, Dentistry,

City Hospital (no academic

medical center)

• Health Science Bachelor of

Science

1 5

1 6

Evaluation Framework: The Institute for

Healthcare Improvement’s Triple Aims

Can innovative models of education,

training and service delivery have

the ultimate outcome of:

• Improving patient/client/population

outcomes?

• Preserving the patient

experience?

• Lowering health care costs?

1 7

Measuring the Difference

The National Center for Interprofessional

Practice and Education

• The Nexus: “A new kind of collaboration and

shared responsibility among education and

practice partners at local, regional and national

levels.”

• http://nexusipe.org/measurement-instruments

http://nexusipe.org/measurement-instruments




1 8

The Case for Standardized and

Individual Measures

Broad Utility

Student/Trainee Outcomes

• Formative/Summative Evaluations

(Reflective Journals)

• Cultural Sensitivity

• Focus Groups

Service Delivery Outcomes

• CAHPS, HCAHPS

• Utilization Reports

• EHR

• Focus Groups (patients, providers)

CHANNELS IPEC STUDENT SURVEY

BEFORE CLASS

www.surveymonkey.com/s/IPECBeforeClass

1 9

The Case for Standardized and

Individual Measures, cont.

Unique to Demographic

UNE

Student/Trainee Outcomes

• Attitudes toward health teams

• Attitudes toward IP education


• TeamSTEPPS

UMKC

Student Trainee Outcomes

• Attitudes toward health teams

• Readiness for IP Learning

• IP Collaboration Scale

• Team Skills Scale


2 1

Process Evaluation-Grant Management

“The need that the grant is attempting to address is

complex, because it isn’t just service delivery

(although that’s a large component of it).

“Its also preparing future providers to deliver services

in a culturally congruent and acceptable way. And it’s

preparing leadership so that such services can be

guided, mentored and sustainable…into the future…

“Its so complex that you need this team of different

types of organizations that do [this] type of work to

really address the problem.”

2 2

Process Evaluation-Grant Management, cont.

Method:

• Polling focus group

Results:

• The steering committee was rated as highly

efficient and effective.

• Much of the committee’s success was attributed

to the strong leadership and dedication among

partners.

2 3

Process Evaluation-Grant Management, cont.

Polling Question Agree Strongly

agree

Members are dedicated to the idea that we can make the CHANNELS grant a

success. 8.3% 83.3%

Members have a clear sense of their roles and responsibilities. 54.6% 45.5%

Members are creating new knowledge or insights together. 16.7% 83.3%

Members have the connections they need to achieve the CHANNELS goals. 16.7% 75%

Members communicate openly with one another. 75% 16.7%

What we are trying to accomplish with the CHANNELS grant would be difficult for

any single organization to accomplish by itself. 9.1% 90.9%

2 4

Student Education Evaluation Methods and

Results

Tools:

• Surveys

• Focus Groups

Existing instruments used:

• Attitudes Towards

Interprofessional Health Care

Teams / Education Scales

• Cultural Competence

Assessment

• TeamSTEPPS

Cohort No. of

Studen

ts

Disciplines

Represented

Summer

2013

94 Dental Hygiene,

Nursing

Fall 2013 97 Dental Hygiene,

Nursing

Winter

2014

28 Nursing, Social Work

Summer

2014

117 Nursing, Occupational

Therapy, Physician

Assistant

Up next:

Fall 2014

~130 Dental Hygiene,

Nursing, Social Work

2 5


Results, cont. Results across cohorts:

• 71% - 82% of students had prior

experience with IPE*

• Students report positive attitudes at

baseline

• 69 - 90% of all attitude measures

received over 70% IPCP-affirming

scores at baseline

• Students find IPCP beneficial to patient

care

• 93 -100% of respondents felt “The

interprofessional approach improves the

quality of care to patients/clients”

* Data for Cohorts 2-4 only

Cohort No. of

Studen

ts

Disciplines

Represented

Summer

2013

94 Dental Hygiene,

Nursing

Fall 2013 97 Dental Hygiene,

Nursing

Winter

2014

28 Nursing, Social Work

Summer

2014

117 Nursing, Occupational

Therapy, Physician

Assistant

Up next:

Fall 2014

~130 Dental Hygiene,

Nursing, Social Work

2 6


Results, cont.

“As OT’s we can’t diagnose, so our referrals are coming

from behavioral doctors and then most likely if they’re

seeing OT’s they’re seeing some other form of therapy and

if you’re in a hospital setting you have to be able to talk to

the doctors and nurses and communicate the care, it’s

essential.”

- UNE Occupational Therapy Student, 2014 Focus Group

2 7

Training Evaluation Methodology and

Results Tools:

• Surveys

• Interviews

• CHOW daily logs


• Attitudes Towards Interprofessional Health Care Teams / Education Scales

• Cultural Competence Assessment

• TeamSTEPPS

• Participant Evaluation

Demographics:

• 3 Cohorts

• Ages 20-60, 41 Participants

• 52% Masters, 26.1% Bachelors, 13% Associate, 4% Doctoral,

4% Secondary

2 8


Results, cont. Results

• Improved TeamSTEPPS learning outcomes

• Improvement in 3 measures related to “Attitudes Towards IP Health Care

Teams”

• Developing an interprofessional patient/client care plan is excessively time

consuming.

• The interprofessional approach permits health professionals to meet the

needs of family caregivers as well as patients.

• Hospital patients who receive interprofessional team care are better

prepared for discharge than other patients.

• No significant findings on “Attitudes towards IP education”

SERVICE - IPCP TRAINING EDUCATION

Population

Focused Nurse

Leader institute

Expanded CHOW

Model

Interprofessional

Team Training -

IPCP Integrated Curriculum

Health Professions

Students

Public Health

Activities

Vision Initiatives

Smile Partners

Oral Health

Program

2 9

CHOW’s

About:

• October 2013 – July 2014

• 2 CHOWs (2 FTE)

• 221 unique clients

• 622 “contacts” logged

Demographics:

• 43% Arabic, 44% Somali, 4%

English,

3 % Other, 5% Missing data

• 90% MaineCare coverage


Population Focused

Nurse Leader institute

Integrated Curriculum

Health Professions

Students

Public Health

Activities

Vision Initiatives

Smile Partners

Oral Health

Program

Expanded CHOW Model

Interprofessional Team

Training - IPCP

3 0

CHOW’s, cont.

Top 3 client barriers identified by clients:

• Cultural issues (91%)

• Does not understand health care system (68%)

• Language (73%)

3 1

Nurse Leader Institute

About:

• 11 Nurses graduated in 2014

• Community Impact Projects:

• Provide customer service training to clinic staff

• Certify nurses to use telehealth equipment

• Create a resource manual for immigrant and refugee

services

• Educate parents about prescription drug abuse

• Hold school-based health fairs



Health Professions

Students

Public Health

Activities

Vision Initiatives

Smile Partners

Oral Health

Program Population Focused


Expanded CHOW Model


Training - IPCP

3 2

Nurse Leader Institute, cont.

Findings:

• Participants felt the opportunity to network with colleagues

was the most helpful aspect of the program

3 3

IPCP Services Methodology

Tools:

• Surveys

• Administrative Data

• CHOW logs

• Electronic Health Records

• National / Statewide Surveillance

• Patient Surveys


• CG-CAHPS 12-Month Survey with Patient-Centered

Medical Home Items (Adult)

• Adapted NIH Vision-related Quality of Life Survey

3 4

Riverton Health Center

About:

• Small health center situated in housing

complex, with:

• Family Nurse Practitioner, 2 CHOWs, 1

Social Worker (MSW) and Students

Demographics:

• 60% MaineCare insurance (Medicaid)

• 35% No insurance

• 5% Medicare or private insurance

Findings:

(From June 2013 – April 2014):

• 384 appointments scheduled

at Riverton Health Clinic

• Overall appointment

adherence: 57.2%

• Increasing number of patients

utilizing the clinic as their

medical home



Health Professions

Students

Population Focused


Expanded CHOW Model


Training - IPCP


Public Health Activities

Vision Initiatives

Smile Partners

Oral Health Program

3 5

Clinician & Group - Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS)

About:

• CAHPS-Gold standard for

measuring patient

satisfaction and informing

decision making

Findings:

Measure Top Box

Score

How often this provider explained things in a

way that was easy to understand (n=15)

73.3%

How often the clerks and receptionists at this

provider’s office as helpful as you thought

they should be (n=15)

86.7%

When you made an appointment for a check-

up or routine care, how often did you got an

appointment as soon as you needed (n=13)

38.5%

How often this provider showed respect for

what you had to say (n=15)

100%



Health Professions

Students



Vision Initiatives

Smile Partners

Oral Health Program

Population Focused


Expanded CHOW Model


Training - IPCP

3 6

Health on the Move

Event:

• Initiative of the Cumberland

District Public Health Council

• CHANNELS-funded CHOWs

provided critical event support

• UNE Nursing students conducted

blood pressure screenings and

prediabetes risk assessments

• UNE pharmacy students

answered questions on

medication safety

Demographics of Attendees:*

• 47% MaineCare coverage

(Medicaid)

• 73% Had a doctor / PCP

• 58% Had been to their PCP in

last year

• 67% Did not have a dentist

• 50% Reported they or their child

needed eye glasses

• Survey languages: 44% English,

24% Arabic, 20% French. 13%

Somali *Results from self-reported data from entrance survey, n=131



Health Professions

Students

Population Focused


Expanded CHOW Model


Training - IPCP



Vision Initiatives

Smile Partners

Oral Health Program

3 7

Public Health Activities – Vision Initiatives

Reach:

• 150+ vision screenings completed

• ~150 free Adlens self-adjusting glasses

distributed

Demographics:

• Ages range: 6-82 years old

• Over 50% had some visual impairment

Findings:

• High demand and need for vision services

Reasons for needing

glasses (July 2014):

• “She will use driving.”

• “Squinting a lot, has

headaches.”

• “Has glasses- not

updated in a long

time: says vision is 0

without glasses.”

• “[Respondent had]

Glaucoma - last eye

exam over a year

ago.”



Health Professions

Students

Population Focused


Expanded CHOW Model


Training - IPCP



Vision Initiatives

Smile Partners

Oral Health Program

3 8

Public Health Activities – Oral Health (“SmilePartners”)

Reach:

• 158 people screened

Demographics:

• Average age: 37

• Language

• 56% Arabic; 33% Somali;

7% French; 4% Other

Preliminary Data February 2013 – September 2014

Screenings 158

Home visits 55

X-rays 42

Dental cleanings 20

Graduated program 12



Health Professions

Students

Population Focused


Expanded CHOW Model


Training - IPCP


Smile Partners

Oral Health Program

Vision Initiatives


3 9


Results Tools:

• Surveys

• Interviews

• CHOW daily logs


• Attitudes Towards Interprofessional Health Care Teams / Education Scales


• TeamSTEPPS

• Participant Evaluation



Health Professions

Students

Population Focused


Expanded CHOW Model


Vision Initiatives

Smile Partners

Oral Health Program


Training - IPCP

4 0


Results, cont. Demographics

• 3 Cohorts

• Ages 20-60, 41 Participants

• 52% Masters, 26.1% Bachelors, 13% Associate,4% Doctoral, 4%

Secondary

Results • Improved TeamSTEPPS learning outcomes

• Improvement in 3 measures related to “Attitudes Towards IP Health

Care Teams”

• Developing an interprofessional patient/client care plan is

excessively time consuming.

• The interprofessional approach permits health professionals to meet

the needs of family caregivers as well as patients.

• Hospital patients who receive interprofessional team care

are better prepared for discharge than other patients.

• No significant findings on “Attitudes towards IP education”

4 1

IPE Evaluation Results

Tools:

• Student Surveys

• Student and Provider Focus Groups

• Patient Satisfaction Surveys


• Readiness for Interprofessional Learning Scale

• Interprofessional Collaboration Scale

• Attitudes Towards Health Care Teams Scale

• Team Skills Scale


Demographics:

• 4 Cohorts

• Ages 22-49 years, 51 Participants

• Graduate nursing, pharmacy, dental students

4 2

IPE Evaluation Results, cont.

Readiness for Interprofessional Learning Scale

(McFadyen et al., 2005)

• 19 items, 1 = strongly disagree to 5 = strongly agree

• Administered at baseline

• Students: n = 49

Students reported

very high readiness

to participate on

IPCP teams, and

openness to learn

from/with other

students. They

reported moderately

high positive

feelings about their

role. Mean

4 3


Interprofessional Collaboration Scale

(Kenaszchuk et al., 2010)

• 14 items, 1 = strongly disagree to 4 = strongly agree

• Administered at end of rotation


Students reported

high levels of

communication

among team

members, high

cooperation and

collaboration, and

high levels of

affiliation with other

professionals

(higher is better on

Isolation scale).

Sum of

Items

4 4


Team Skills Scale

(Hepburn, Tsukuda, & Fasser, 1998)

• Assesses team members’ perceptions of their own skills

• 17 items rated on 5 pt scale, 1=Poor, 2=Fair, 3=Good, 4=Very Good,

and 5=Excellent (possible range 17 – 85)

• Administered at baseline and end of rotation

• Students: n = 32 Students reported a

significant increase

in perceived team

skills

(t = 2.56, p = .02)

Sum of

Items

4 5


Cultural Competence Assessment

(Schim et al., 2004)

• Assesses changes in perceived cultural competence

• Overall perceived cultural competence rated on a scale from 1 = Very

Incompetent to 5 = Very Competent


• Students: n = 32 Students reported a


in perceived cultural

competence

(t = 2.35, p = .03)

Mean

4 6

IPE Evaluation Results, cont. Cultural Competence Assessment

(Schim et al., 2004)

• Assesses changes in perceived cultural competence

• Cultural Competence Behavior Subscale – rating of how often

students perform 10 behaviors related to culturally competent care on

a scale from 1 = Never to 7 = Always



Students reported a


in culturally

competent

behaviors

(t = 4.02, p = .001)

Mean

4 7

Community-based Healthcare Centers

Clinic 1 Small, young community-based health center:

• Teams of NP students, Pharmacy students, Dental students

• 934 patients seen (64% female; 0 - 65+ yrs)

Most Frequent Diagnoses:

• 152 – Essential Hypertension

• 109 – Diabetes Mellitus

• 81 – Disorders of Lipoid Metabolism

• 65 – Overweight, obesity

• 62 – Health supervision infant or child

• 53 – General Medical Exam

• 52 – Asthma

4 8

Community-based Healthcare Centers

Clinic 2 Large, urban federally-qualified healthcare center:

• NP students, and Pharmacy students

• 1,814 patients seen (78% female; 0 - 65+ yrs)

Most Frequent Diagnoses:

• 320 – Normal Pregnancy

• 219 – Essential Hypertension

• 184 – Screening Malignant Neoplasm

• 136 – Diabetes Mellitus

• 134 – Screening Bacterial

• 132 – Contraceptive Management

• 128 – Overweight/obesity

• 117 - Health Supervision of infant/child

• 115 - Disorders of lipoid metabolism

4 9

Qualitative Data

• Focus groups in three populations provided qualitative data:

• Students

• Providers

• Patients (comments and satisfaction surveys)

• This information offered feedback and guidance as the

projects progressed.

5 0

Evaluation Gratitude

Michelle Mitchell, Msc, MS

Partnerships for Health, LCC

Evaluator for the University of New England project

Heather J. Gotham, PhD

Associate Research Professor

Evaluator for the University of Missouri, Kansas City project

5 2

Lessons Learned...Are Nuggets for

Refinement!

• UNE added RIPLS as a measure

• UMKC considering TeamSTEPPS training

• Process evaluation is critical to overall evaluation processes

• We need more time! Triple Aims require longitudinal study

AND patience

• Standardized measures translate across all sorts of projects

(resources and partnerships, methodologies, demographics,

etc)

• National Center excellent resource to better understand tool

validity

5 3

Questions and Discussion

Please type your questions in the chat box.

5 4

Join the IPECP Community

Create a profile

www.nexusipe.org

Add a resource

www.nexusipe.org/resource-exchange

Start a conversation

www.nexusipe.org/forum

Go social

www.twitter.com/nexusipe

http://www.nexusipe.org/

http://www.nexusipe.org/resource-exchange



http://www.nexusipe.org/forum

http://www.twitter.com/nexusipe

Documents

Welcome and Introduction · •Formative/Summative Evaluations ... Process Evaluation-Grant Management, ... • Certify nurses to use telehealth equipment