K n o w l e d g e i n p r a c t i c e... kusp Research Utilization: and its (many) determinants University of Melbourne Melbourne, AU

k n o w l e d g e i n p r a c t i c e. . .

www.ualberta.ca/~kusp

Research Utilization: and its (many) determinants

University of MelbourneMelbourne, AUOctober, 2005

Carole A. EstabrooksProfessor, Faculty of Nursing

&Canada Research Chair

in Knowledge Translation

University of Alberta





Edmonton, Alberta



University of Alberta

Context

FacilitationEvidence

Evidence-based practice

PARiHS Model(Kitson, et al., 1998, QSHC

PARiHS Framework for Research Implementation

Policy Framework

Lomas, 2000

Policy Analysis Framework



First…… what do we know about the determinants of research utilization?



Common barriers encountered in knowledge transfer in hospital settings

Commonly identified barriers include:

• Time• Resources• Support• Knowledge & skills • Adequate evidence• Culture, context,

environment….Less commonly identified barriers include:

• Disincentives• Irrationality of decision-making

environment• Decision-making “errors”• Attitudes & beliefs• Inability to see consequences

Imperfect evidence base for decision makers Many current rigorous evaluations have methodological

weaknesses Poor reporting of study settings, barriers to change,

content and rationale of intervention Generalisability of study findings is frequently uncertain Reminders most consistently observed to be effective Educational outreach only led to modest effects Dissemination of educational materials may lead to modest

but potentially important effects (similar effects to more intensive interventions)

Multifaceted interventions not necessarily more effective than single interventions

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale L et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004. (Available from: http://www.hta.nhsweb.nhs.uk/)

Best strategies for transferring research findings?



From some of our work in KUSP

In nursing…• Most common interventions tend to be

education, protocol & guideline implementation

• Little audit & feedback, this is emphasized more among physicians

• Individual patient information, personal experience, and social interactions are main information sources

• Limited use of journals, textbooks, and other media, including the internet

1



1. Structure of Authority2. Nature of nurses’ work

• location of work • expected level of activity• routinized work • expected ways to get work done

3. Workplace climate • ethical issues • the response to errors • the receptivity to innovation

4. Types of knowledge valued• clinical knowledge (practical, often

tacit) • specialized knowledge (e.g., PICU)• research

Scott-Findlay’s doctoral work on culture and RU

(in progress)



Manifestations

of culture

Creation of values that result in

behavioral expectations

Conformity

Nature of nurses’ work

Hierarchical structure of

authority Types of

knowledge

Workplace ethos

Decision not to

conform

Intervening factor – inconsistency from superiors

that creates unclear expectations

Members resort to doing what they are told to do because of inconsistent and shifting expectations

Through ongoing interpretation of

events and dialogues members

‘learn’ what is important

Scott-Findlay (in progress)



For interventions we also need to consider (from ethnographic case studies)

• Information needs and how these relate to the sources of knowledge used in practice

• How the knowledge sources used bear on the packaging and dissemination of “evidence”

• How dissemination strategies need to focus on what is needed and useful regardless of what we perceive may be the most relevant approach

• A focus on those determinants of research use with the potential to be manipulated in order to develop interventions with potential utility

4

From work in KUSP



nursing control over practice opportunity for nurse-to-nurse collaboration support for innovation facilitation of research use positive culture (characterized by good

leadership, evaluation and performance feedback)

increased time to nurse (-ve) emotional exhaustion (-ve)

Organizationally (causal and multi-level modeling)

From work in KUSP

5

Higher nurse education levels Greater control over practice Greater freedom to make decisions Having a good nursing leader on unit Low levels of emotional exhaustion A sense of personal accomplishment



A. Theorizing the determinants of research utilization (2000 – 2005)

B. KUPI - The knowledge utilization & policy implementation program (2002 – 2007)

C. The Bibliometric study (2004 – 2006)

D. Measuring research utilization (2005 – 2008)

E. New and future studies

KUSP Studies



A. Theorizing the determinants of research utilization (2000 – 2005)

The Determinants of Research Utilization:Pain Management in Adults & Children

Research Utilization Studies

Purpose: To understand how nurses use research in their practice and what influences their ability to use or not use research.



Four hospitals (two in Ontario, two in Alberta)

Adult and pediatric surgical units

Qualitative and quantitative data collected

Six months of participant observation each unit

Nurses, patients, physicians, social workers, physiotherapists, and administrators participated

METHODSEthnographic case studies



• Documents and records

• Interviews (individual & group)

• Participant observations

• Physical artifacts

• Measures (e.g., Research utilization, unit culture, critical thinking, pain, workload, environmental complexity)

Data Sources



Quantitative findings(selected - sources only)

• In this study the patient care unit was unit of analysis

• Our goal: to understand the role of context in shaping RU

• We mapped relationship of contextual factors using correspondence analysis



Quantitative findings (cont’d)

• Units with highest RU clustered

together on factors such as:• Critical thinking• Unit culture

• Workload and people support clustered more closely to units with lowest RU scores



The 7 Units on knowledge sources

Top five sources:

1. Personal experience & individual patient

2. In-services in the workplace

3. Nursing school

4. Discussions with physicians & fellow nurses

5. Intuition



Five least used sources:

5. Nursing journals

4. “Ways nurses have always done it”

3. Nursing research journals

2. Medical journals

1. Media



Number of Sources Used by Nurses

• 20% of nurses use 1 – 51 – 5 sources• 50% of nurses uses 6 – 106 – 10 sources• 25% of nurses use 11-1511-15 sources


www.ualberta.ca/~kusp 7 Unit ComparisonOverall, few differences in the sources used by nurses across units

Information Source Ranking by Means

Unit 1 Unit 2 Unit 3 Unit 4 Unit 5 Unit 6 Unit 7

Individual patient 3 1 1.5 2 2 1 2.5

Intuitions 8 8 4 5 8 7 4.5

Personal experience 1.5 2 1.5 1 1 2 1

Nursing school 1.5 4 3 3 6.5 9 8

Physicians disc’ns w nurses

9 5.5 7 7 6.5 3 9

Physician’s orders 7 7 9.5 7 9.5 8 12.5

Medical journals 15 15 16 14.5 15 14 15

Nursing journals 13 12 13 11 13 10 10

Nursing research journals 14 13 14 16 15.5 13 11

Textbooks 11 9 11 12.5 11 12 14

‘What has worked for years’

6 11 7 7 4.5 11 4.5

‘Ways nurse has always done it

12 14 12 10 12 15 12.5

Fellow nurses 4.5 5.5 7 4 3 6 7

In-services in workplace 4.5 3 5 12.5 4.5 5 2.5

Policy & procedure manuals

10 10 9.5 9 9.5 4 6

The media 16 16 15 14.5 15.5 16 16



Sources of Knowledge Patterns Over Time

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Indiv

idual C

lient

/ Pat

ient

Intu

itions

Perso

nal e

xper

ience

Nursing S

choo

l

Physic

ians d

iscus

sions w

ith n

urse

Physic

ian’s

order

s

Med

ical J

ourn

als

Nursing J

ourn

als

Nursing R

esea

rch

Jour

nals

Textb

ooks

What

has w

orke

d for y

ears

Ways

nurse

has

alw

ays d

one

it

Fellow

Nur

ses

In-s

ervic

es in

wor

kplac

e

Policy

and

pro

cedur

e m

anuals

The m

edia

Sources of Knowledge

Me

an

Sc

ore

Baessler et al., 1994

Estabrooks, 1998

Estabrooks et al., Current

Note. Mean scores were transformed to fit a five-point likert scale (0-4), with 0 as ‘never’ and 4 as ‘always’.



Main Categories of Knowledge Sources

A. Social interactionsB. ExperienceC. DocumentsD. Intra-Personal

Qualitative findings(selected - sources only)

Taxonomy of nurses’ sources of knowledge



Why? TimeNurses typically work not only on multiple schedules; high and frequent levels of resequencing are required

ContextNurses tend to be motivated to seek knowledge when they have context specific patient care situations, problems to deal with

TrustNurses tend to seek information , knowledge from those they trust (knowledgeable, empathetic)

HierarchyAn environment where norms of organizational hierarchy are strictly adhered to may negatively affect knowledge transfer between nurses and CNS’s, clinical leaders, and physicians.



Summary & Conclusions

• Individual patient information, personal experience, and social interactions are main information sources

• Limited use of journals, textbooks, and popular media, including the internet

• The structure and organization of nurses’ work has a major influence on their choice of knowledge sources

• Researchers need to merge nurses’ information-seeking behavior with dissemination strategies



Summary & Conclusions

The nurses in this study overwhelmingly relied on social interactions and experience as their primary sources of practice knowledge

Time, trust, and unit environment played an important role in determining the nurse’s sources of knowledge



B. KUPI: The knowledge utilization & policy implementation program (2002 – 2007)



Context

FacilitationEvidence

Evidence-based practice PARISH Promoting Action on Research Implementation in Health Services

SI = f (E,C,F)Successful implementation is a function of evidence, context and facilitation

Kitson et al, 1998Rycroft-Malone et al,

2002



Knowledge utilization and policy

implementation (KUPI)

A five year research program whose purpose is to develop theory in the knowledge utilization field that increases (research) knowledge use by clinicians and decision-makers across multiple levels - by using existing data



Objectives of the larger KUPI program

To study: how predictors of research use vary

across levels of decision-making how research use is accomplished in

organizations, and how research use shapes policy

implementation



Designed to study factors influencing knowledge use, specifically research utilization (RU), at different levels of decision-making among nurses in Alberta hospitals.

Modeling Sub-Projects



Modeling sub-project: Five studies

1. DV derivation & outcome validation2. A structural equation (SEM) assessment of

the PARiSH (Kitson) model3. Multi-level (HLM) modeling4. Hierarchical structural equation modeling

(H-SEM)5. A methodological evaluation of HLM vs. H-

SEM and their usefulness for the study of research utilization



Example: Multi-level (HLM) study objectives

To determine which nursing factors at the individual, unit, and hospital levels predict research utilization

To measure the contribution of the significant factors to variation in research utilization across units and hospitals



The Alberta Nurse Survey (1998)A census population of all

12,345 registered nurses from 129 hospitals in Alberta, Canada. Useable returns 6526 (52.8%)

A: Employment Characteristics

B: Nursing Work Index (NWI)

C: Maslach Burnout Inventory(MBI)

D: Job characteristics

E: Last shift

F: Demographics

G:Site specific questions



The Research Utilization Survey (1996)

A randomly selected sample of staff nurses in Alberta, Canada. Useable returns 600 (42%)

I. Research utilization

II. Background and daily demands

III. Individual and professional factors

IV. The research itself

V. Your organization



Hospitals Nurses

109 6,526 90 5,228

Selection criteria: 5 nurses per hospital

Final Sample



The 3 level model

Organization factors (hospital level)

Specialty factors (unit level)

Nurse factors (individual level)



Individual level Higher nurse education Control over practice Freedom to make decisions Emotional exhaustion Lack of personal accomplishmentSpecialty (unit) level Leadership Relational capital Consulting opportunity Participate in policy

Organizational level Climate Innovativeness Organizational slack

Significant univariate predictors: Individual, specialty (unit), and hospital

levels



Summary

Higher nurse education levels Greater control over practice Greater freedom to make decisions Having a good nursing leader on unit Low levels of emotional exhaustion A sense of personal accomplishment

Greater research utilization among nurses was predicted in the final model by:



Individual unit hospital

Components (%) of research explained at the individual, specialty (unit), and hospital levels

91.8%

4.5%

3.7%



Summary

Variation in research utilization was mainly due to differences in individual characteristics of nurses, with organizational factors contributing less.

Of the latter, however, having good leadership on the unit was the most significant determinant of research utilization



Structural equation (SEM) modeling objective

to develop a theoretical model of hospital characteristics that predict research use by nurses

Nurse to NurseCollaboration

ResponsiveAdministration

Hospital Size

Relational Capital

Staff DevelopmentPrograms

Control over Practice

Staffing and Support Services

Facilitation

Time to Nurse

Emotional Exhaustion

Innovation

Nursing Education

RU

Internet Use

The Structural Equation Model (SEM)



Positive impact on research use:• nursing control over practice• opportunity for nurse-to-nurse collaboration• support for innovation• facilitation of research use• positive culture (characterized by good

leadership, evaluation and performance feedback)

Negative impact on research use:• Increased time to nurse• emotional exhaustion

Structural Equation Modeling Results







Objectives Map the development of the field of

knowledge utilization Identify the structure of the scientific

community Provide direction to investigators &

decision-makers in the field



Education- evaluation utilization

Business/Org -innovation

diffusion

Health- research utilization

-evidence-based practice

Science & Technology

-technology transfer

Social Sciences & Humanities-knowledge utilization

Terminology Tangle



Pilot study: A bibliometric analysis of the RU literature in nursing

• 600 articles• Findings/Conclusions

• Limited collaborations• Primarily opinion pieces with little research or

theoretical work• UK & US most prolific in the field• Increased productivity since early 1990’s with

change in terminology

Estabrooks, C.A., Winther, C., & Derksen, L. (2004). Mapping the field. A bibliometric analysis of the research utilization literature in nursing. Nursing Research, 53(5), 293-303.





Larger Study: Methods

Overview •Web of Science 1945-2004•Articles related to KU in general

Three case studies related to KU•Guidelines•Policy & management•Nursing



Preliminary findings

Key authors: 15 most cited authors



Key journals

Journal of Evaluation in Clinical Practice Journal of Advanced Nursing British Medical Journal Knowledge: Creation, Diffusion, Utilization Lancet Journal of General Internal Medicine Research Policy Journal of the American Medical

Association Science Communication Social Science & Medicine



D. Measuring research utilization (2005 – 2008)



Existing measures

Systematic Review Findings

Nursing Practice Questionnaire (NPQ) NPQ (Modified) Research Utilization Questionnaire (RUQ) Edmonton Research Orientation Survey (EROS)

Other multi-item measures (N=3) Single item measures (N=19)



Poor construct clarity Lack of theoretical framing Lack of measurement theory Lack of psychometric assessment Presumption of linearity Absence of longitudinal work

Systematic Review Findings

Self-report Recall Social desirability Scaling Unit of analysis



Instrument Development Study

Objectives: 1. to clarify and validate the construct2. to develop observable indicators3. to develop a set of “items” that

measure research use4. to conduct a pilot test of the

instrument



Approach: 1. Construct clarity Series of focus groups

with three sets of expertsa. Local (research team) expert panel

(complete)b. International panel (May 2005)c. Managers and educatorsd. Providers (RNs, LPNs, assistants, aides)

2. Indicator and item development Series of focus groups with two sets of expertsa. Managers and educatorsb. Providers of nursing care (RNs, LPNs, RPNs,

assistants, aides)

3. Pilot test the draft instrument(s)

Instrument Development study



Instrumental Use

“Action”

Non-useA. Informed (non-defensible)

B. Informed (defensible)

Non-use

Task completionRelationalAssessmentScreening

Research Use

Research Non-Use

“Thinking”(i.e., Clinical Decision Making

Clinical Reasoning )

Conceptual Use

Instrumental Use

Task completion

Act

ion

in P

ract

ice

Uninformed

Non-use

Conceptual Use

Conceptual Use



HI

LO

HILO

Instrumental

Con

cep

tual Research utilization



E. New and future studies

Health research utilization in hospitals (start up)

Under review SHARP (Share and House Accessible Research

Products)

TROPIC team grant

(KUSP) Team grant (letter of intent stage): The influence of context on research use (long term care settings)

Objectives of the first (completed) study 1. To describe two groups (researchers and

users)

2. To compare research dissemination behaviour among the researcher sub-groups

3. To compare research utilization behaviour among the users sub-groups

4. To explore which factors contribute to research dissemination (researchers) and research use (users)

Health research utilization in hospitals (start up)(2nd in a series of 3-4 studies)



Key Messages (from the first study)

About Researchers Research infrastructure matters Not all academic researchers are

equal Researchers do contribute to

research use

About Users Organizational capacity matters Intermediaries matter Talking with researchers matters

med - basic other - basic med - app other - app

Group

9.00

10.00

11.00

12.00

13.00

14.00

15.00

16.00

Pla

in D

iss

em

ina

tio

n


Group

8.00

10.00

12.00

14.00

16.00

En

ga

ge

d D

iss

em

ina

tio

n


Group

3.40

3.60

3.80

4.00

4.20

4.40

4.60

Nu

mb

er

of

pu

bli

ca

tio

ns

(s

qu

are

-ro

ot

tra

ns

form

)

Plain Dissemination By Group

Engaged Dissemination By Group

Number of Publication By Group

Correlations

Plain

Dissemination

Engaged Disseminatio

n

Number of Publicatio

ns

Contribution provided by users .507(**) .448(**)

Perceived impact .624(**) .590(**)

Relational capital .561(**) .495(**)

Barriers to uptake of research .241(**) .267(**)

Perceived importance of dissemination activities (A)

.306(**) .164(*)

Perceived importance of dissemination activities (B)

.404(**) .341(**)

Recognition of expertise .618(**) .448(**)

Years of experience (as service provider)

.364(**) .331(**)

Research focus – users’ need .464(**) .542(**)

Research focus – scholarly advancement

-.144(*) -.191(**) .192(**

)

Perceived importance of original studies leading to publication

-.133(*) -.157(*) .316(**

)

Cost of utilization .213(**)

Number of research personnel .247(**)

Years of experience (post grad) .303(**)

** Correlation is significant at the 0.01 level (2-tailed).* Correlation is significant at the 0.05 level (2-tailed).

2 3 5 4 1

user

20.00

21.00

22.00

23.001 Nurse providers2 Physician providers3 Intermediaries4 Nurse Decision-makers5 Other Decision-makers

Summary of Comparisons

• There is significant difference across groups.

• Multiple comparison test revealed 4 homogeneous subsets: nurse providers, nurse decision-makers and other decision-makers, other decision-makers and intermediaries; and intermediaries and physician providers.

• The physicians are significantly different from all nurses and other decision-makers.

Research Utilization Index



Factors Influencing Dissemination and Publication

Plain Dissemination: Contribution by users, Perceived impact, Perceived importance of dissemination activities (B), Recognition of expertise, Research focus – users’ need

Engaged Dissemination: Perceived impact, Perceived importance of dissemination activities (B), Research focus – users’ need

Publication: Number of research personnel, Research focus – scholarly advancement, Academic rank, Cost of utilization



Present Study

Objectives: to determine the feasibility of

undertaking a larger scale study in Western Canada

To validate a typology of providers To validate a revised survey instrument assessing individual and organizational characteristics

To identify predictors of research utilization

To assess provider group differences and similarities



9. Research Outcomes

8. Data Analysis

7. Repurposing

6. Data Discovery

5. Data Archiving

4. Data Dissemination

3. Data Processing

2. Data Collection

1. Study Concept

Data Life Cycle

7

6

1

2

3

4

8

5

9

9. Research Outcomes

8. Data Analysis

7. Repurposing

6. Data Discovery

5. Data Archiving

4. Data Dissemination

3. Data Processing

2. Data Collection

1. Study Concept

Data Life Cycle

7

6

1

2

3

4

8

5

9

The SHARP grant(infrastructure)



Why SHARP?

Preserves data for future analyses

Facilitates researcher collaboration

Increases effectiveness of funding

Provides opportunities for mentoring and education in data management

Reduces duplication of data collection



Purpose: to ameliorate pain in children by narrowing the gap between clinical practice and the research evidence supporting optimal patient care. Using the PARIHS framework

TROPIC

Main applicants:Bonnie Stevens (PI), University of TorontoCeleste Johnston, McGill UniversityShoo Lee, University of AlbertaCarole Estabrooks, University of AlbertaShannon Scott-Findlay, University of AlbertaPatrick McGrath, Dalhousie UniversityChristine Chambers, Dalhousie University



Figure 1. Working model of research utilization(Estabrooks, Scott-Findlay & Cummings, 2005)

Evidence Context

Facilitation

PARIHS model Research Use/Research Uptake (Process Outcomes)

Clinical (Patient) Outcomes

INTERMEDIATE Research Utilization OUTCOMESGeneric measuresResearch useResearch uptakeSpecific measures (i.e. Process outcomes)Completed pain assessmentsAppropriate analgesic administrationImplementation of other nonpharmacologic pain management interventions

Pain intensity



1.The creation of a centralized Canadian Pediatric Pain Research Network Database (e.g., collect demographics and information about pain)

2.Assess organizational context and determine its impact on the implementation of pain research in pediatric clinical practice

3.Evaluate the efficacy and effectiveness of (a) Evidence-based Practice In Change (EPIC; Lee et al., 2002) and (b) Audit and Feedback

Three major TROPIC projects



Culture• OCI, Cooke & Lafferty (1987)

Leadership• Leadership Practice Inventory (LPI) Kouzes, & Posner

(2002)• Leadership Empowering Behaviours (LEB) Hui (1994)

Evaluation• Focused qualitative data

Other overall contextual measures• Maslach Burnout Inventory • Nursing Work Index - Revised (NWI – R)• Environmental Complexity Scale (ECS)• Project Research in Nursing (PRN) Workload

Measurement Instrument

Research Utilization (revised)• Research Utilization Survey (Estabrooks 1997)



A main goal of the analyses will be to assess the role of unit contextual variables on nurse research utilization behaviors

Additionally, to estimate the independent effects of these contextual variables, we have proposed Hierarchical Linear Modeling (HLM) and Structural Equation Modeling (SEM)



Carole Estabrooks (PI, Canada Research Chair)

Lesley Degner (CHSRF Chair) Michael Leiter (Canada Research Chair) Heather Laschinger (UWO, leadership) Greta Cummings (UA, leadership) Joanne Profetto-McGrath (UA, facilitation) Malcolm Smith (Business, Univ of Manitoba) Laurel Strain (Gerontology, sociology)UK Joanne Rycroft-Malone Sue DopsonUSA Anne SalesAnd others tba

(KUSP) Team LOI(tentative and evolving)



Examining the context dimension of the PARIHS framework in LTC settings

Context

Facilitation

Evidence


