Project JOINTS: What factors affectbundle adoption in a voluntaryquality improvement campaign?

Dmitry Khodyakov,1 M Susan Ridgely,1 Christina Huang,1

Katherine O DeBartolo,2 Melony E Sorbero,3 Eric C Schneider4,5,6

▸ Additional material ispublished online only. To viewplease visit the journal online(http://dx.doi.org/10.1136/bmjqs-2014-003169).

1RAND Corporation, SantaMonica, California, USA2Institute for HealthcareImprovement, Cambridge,Massachusetts, USA3RAND Corporation, Pittsburgh,Pennsylvania, USA4RAND Corporation, Boston,Massachusetts, USA5Department of Health Policyand Management, HarvardSchool of Public Health, Boston,MA, USA6Division of General Medicineand Primary Care, Brigham andWomen’s Hospital, HarvardMedical School, Boston, MA,USA

Correspondence toDr Dmitry Khodyakov, RANDCorporation, 1776 Main Street,P.O. Box 2138, Santa Monica,CA 90407-2138, USA;Dmitry_Khodyakov@rand.org

Received 28 April 2014Revised 10 October 2014Accepted 14 October 2014Published Online First5 November 2014

To cite: Khodyakov D,Ridgely MS, Huang C, et al.BMJ Qual Saf 2015;24:38–47.

ABSTRACTBackground Diffusion and adoption of effectiveevidence-based clinical practices can be slow,especially if complex changes are required toimplement new practices.Objective To examine how hospital adherenceto quality improvement (QI) methods andhospital engagement with a large-scale QIcampaign could facilitate the adoption of anenhanced prevention bundle designed to reducesurgical site infection (SSI) rates after orthopaedicsurgery (hip and knee arthroplasty).Methods We conducted telephone interviewswith hospital QI leaders from 73 of the 109hospitals (67% response rate) in five states thatparticipated in Project JOINTS (JoiningOrganizations IN Tackling SSIs), a QI campaign runby Institute for Healthcare Improvement (IHI).Using QI methods grounded in the IHI Model forImprovement, this campaign encouraged hospitalsto implement an enhanced SSI prevention bundle.Hospital QI leaders reported on their hospital’sadherence to the Project JOINTS QI methods; theirlevel of engagement with Project JOINTS activities;and adoption of the SSI prevention bundlecomponents. Interview data were analysedquantitatively and qualitatively.Results Both adherence to the QI methods andhospital engagement were positively associatedwith complete bundle adoption. Hospitalengagement, especially the use of projectmaterials and tools, was also positively associatedwith the initiation of and improved adherence toindividual bundle components.Conclusions Our findings suggest that greateradherence to the QI methods and active hospitalengagement in a QI campaign facilitate adoptionof evidence-based patient safety bundles inorthopaedic practice.

INTRODUCTIONAlthough the field of implementationscience strives to address the recognisedgap between knowledge generated in

clinical trials and routine clinical prac-tice,1 2 research has demonstrated signifi-cant delays in the diffusion and adoptionof effective, evidence-based clinical prac-tices.3 4 Closing this gap still remains achallenge for many health systems.Quality improvement (QI) collabora-

tives,5 such as Institute for HealthcareImprovement’s (IHIs) Breakthrough Seriesin the USA,67 the Health Foundation’sSafer Patients Initiative8–12 and Stroke90:10 QI Collaborative13 in the UK, andEvidence 2 Excellence in Canada,14 15

have been used to accelerate the adoptionof evidence-based care. A collaborative is amultiorganisational effort to design andpromote the adoption of a multicompo-nent QI package.16 17 QI collaborativesfocus on a prespecified topic, offer accessto expert knowledge on clinical and QItopics, provide peer support that can helpan individual hospital identify and imple-ment best practices; rely on multidisciplin-ary implementation teams within hospitals,use a QI methodology and encourage par-ticipants to engage in a series of structuredactivities.18 Collaboratives tend to generateenthusiasm and energy among participat-ing organisations to effect change.7

Although the QI collaborative hasbecome a popular approach, participantsoften have to invest considerable time,effort and financial resources to achieveand sustain even small improvements.7 8

Because collaboratives are resource-intensive, they may be limited in scope anddifficult to scaleup.8 Even though certainaspects of teamwork, data collection andparticipation in collaborative activities havebeen shown to increase the collaborative’schances for success,18 several recent sys-tematic reviews suggest that effectivenessof collaboratives is variable and setting-dependent.17 While it might be desirable

ORIGINAL RESEARCH

38 Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

to remove costly components of a collaborative, theimprecise descriptions of the QI intervention compo-nents make it difficult to identify those components thataffect provider performance and patient outcomes.19

To expedite the process of disseminating evidence-based practices and to increase their reach whiledecreasing the burden on participants, the IHI intro-duced the QI ‘campaign’—a new approach that uses afield infrastructure called the Rapid Spread Networkcomprising local (usually state-based) QI organisa-tions, professional societies and hospital associations,among others, to recruit hospitals and disseminatecampaign information. In campaigns, hospitals are notrequired to attend numerous face-to-face meetingsand provide periodic updates on their implementationefforts and success, which are typical requirements ofIHI collaboratives.20 Instead, participating hospitalsare encouraged to participate in campaign activitiesremotely (primarily via webinar meetings and anemail listserv) and to use educational materials postedon the campaign website. Moreover, participating hos-pitals are encouraged to use a multifaceted QIapproach developed by IHI to motivate and supportchange. The most well-known IHI campaigns are the100 000 Lives21 22 and 5 Million Lives campaigns,23

which were designed to reduce medical harm by pro-moting adoption of evidence-based practices. Theeffectiveness of campaigns as a QI approach, however,is uncertain. Despite the compelling theoretical basis,campaigns remain something of a ‘black box’ in thatthe mechanisms by which they actually produceimprovement are not well described.

THE PROJECT JOINTS INTERVENTIONSurgical site infection (SSI) following hip and kneearthroplasty is devastating for both patients and theircaregivers. Treatment of such infections may requireadditional surgery, removal of the prosthetic joint,replacing the prosthetic with a spacer device and pro-longed systemic antibiotic therapy. Patients unneces-sarily suffer from pain and typically experienceimpaired mobility during recovery. Second and thirdsurgeries to revise the implant are painful, burden-some for the patient and caregivers, and costly.24 25

Funded by the US Department of Health andHuman Services, Project JOINTS (JoiningOrganizations IN Tackling SSIs) was an IHI campaignto spread a bundle of evidence-based practices that canprevent SSI among patients undergoing hip and kneearthroplasty.26 For simplicity of messaging, IHI usedthe term ‘bundle’ to describe what was actually a com-bination of two SSI prevention bundles that involvetwo points of care (the preop clinic and the operatingroom).27 The resulting five-component enhanced SSIprevention bundle included three evidence-based prac-tices that were not widely used prior to 2010:1. use an alcohol-containing antiseptic agent for preopera-

tive skin preparation28 29 because the use of alcohol in

addition to a long-acting antiseptic agent provides super-ior protection against SSIs;

2. instruct patients to bathe or shower with chlorhexidinegluconate (CHG) soap for at least three days beforesurgery30–32 to reduce bacterial colonisation of the skin;

3. screen patients for Staphylococcus aureus (SA) carriageand decolonise SA carriers with 5 days of intranasalmupirocin and at least three days of chlorhexidine soapprior to surgery33 34 because SA nasal colonisation hasbeen shown to be associated with an increased risk forSSI;and two established practices that had become usual

care in many hospitals:1. appropriate use of prophylactic antibiotics;2. appropriate hair removal for the location and

procedure.35

Building on previous IHI campaigns, ProjectJOINTS used a field infrastructure called the RapidSpread Network (see figure 1) that consisted of threesets of actors: IHI; state organisations, such as QualityImprovement Organisations and state hospital associa-tions, known in the parlance of the campaign as‘nodes’; and hospitals.IHI established the agenda for Project JOINTS, pro-

vided recruitment materials designed for specialisedaudiences, disseminated a ‘How to Guide’ describinga tailored set of QI methods recommended for hospi-tals and orthopaedic practices, developed all ProjectJOINTS intervention materials, created and main-tained a project website and email listserv, and offereda range of learning opportunities (including webinarcalls, faculty consultations and town hall meetings,among others) to participating hospitals (see onlinesupplementary appendix A for a description of ProjectJOINTS materials, tools and activities).Nodes were responsible for raising awareness about

Project JOINTS at the state and local levels, recruitinghospitals, disseminating information about the projectand helping IHI organise on-site visits and town hallmeetings, where participating hospital teams from thesame geographical region could come together toopenly and informally share their Project JOINTSexperiences, learn from each other and get help fromIHI faculty in a face-to-face setting. Nodes were notrequired to provide ‘hands-on’ assistance to participat-ing hospitals, there was no required level of ‘node’participation and some of these state-level nodes weremore active than others in Project JOINTS for avariety of reasons, including ongoing participation inother local or national initiatives that might not havebeen aligned with Project JOINTS.Participating Project JOINTS hospitals were granted

access to a password-protected website maintained byIHI, which contained the Project JOINTS print andonline materials about SSI prevention (including a‘How to Guide’ that described the recommendedProject JOINTS QI methods; evidence handouts forsurgeons, patients and family members; and a

Original research

Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169 39

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

‘business case’ for SSI prevention, among others).Hospitals were also encouraged to participate in avariety of interactive Project JOINTS activities, includ-ing a webinar call series and a project listserv. Tofacilitate implementation of as many evidence-basedpractices as possible, hospitals were encouraged to usethree elements of the Model for Improvement asso-ciated with successful QI projects (1) developing a QIplan, (2) putting in place a multidisciplinary imple-mentation team and (3) conducting small tests ofchange.6 No funding was provided to hospitals todefray costs associated with their participation in thiscampaign.Although there are many contextual factors that

affect QI intervention uptake in large-scale initiatives,including leadership support,36 this study wasdesigned to test whether adherence to the ProjectJOINTS QI methods and participation (or ‘engage-ment’) in Project JOINTS activities could facilitate theadoption of the SSI prevention bundle in hospitalsand consequently reduce SSI rates after hip and kneearthroplasty.

STUDY DESIGNStudy hypothesesBecause data on SSI rates (outcomes) at the hospitallevel are not available to us at this time, we focusedon the reported uptake of the SSI bundle as theprimary measure of effectiveness. We tested twohypotheses based on the QI literature and prior IHIexperience. Previous research shows that adherence tothe QI methods (such as the use of Plan-Do-Study-Act

cycles) helps hospitals implement QI initiatives.37

Similarly, prior IHI experience with large-scale QIinitiatives suggests that adherence to QI methods,such as the establishment of a QI plan with clear aims,reliance on multidisciplinary implementation teamsand the conduct of small-scale tests of change, islikely to positively affect implementation of a QI ini-tiative.6 Therefore, we hypothesised that hospitalsthat adhere to the Project JOINTS QI methods will bemore likely to adopt the enhanced SSI preventionbundle than hospitals that do not.Furthermore, results of a recent literature review on

determinants of success of QI collaboratives18 andprevious IHI experience38 suggest that hospitalengagement in QI activities may positively impactintervention implementation. Because the IHI cam-paign infrastructure is designed to help hospitals learnabout the evidence supporting intervention compo-nents and QI methods, as well as benefit from theexperience of IHI faculty and their peers,22 wehypothesised that the level of hospital engagement inProject JOINTS (ie, use of campaign materials andtools and participation in interactive activities) will bepositively associated with bundle adoption.

Data collectionWe tested these hypotheses using self-reported, cross-sectional interview data collected in five states thatparticipated in Project JOINTS. We developed a struc-tured interview protocol to collect self-reported dataon hospital adherence to the Project JOINTS QImethods, hospital engagement in Project JOINTS and

Figure 1 Conceptual model for Project JOINTS. JOINTS, Joining Organizations IN Tackling SSIs; SSI, surgical site infection;IHI, Institute for Healthcare Improvement.

Original research

40 Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

uptake of the five-component enhanced SSI preven-tion bundle. Adherence to Project JOINTS QImethods referred to the development of an implemen-tation plan, the deployment of an implementationteam and the use of small-scale tests of change, suchas conducting a pilot study or implementing theproject for a selected group of patients first. Tomeasure hospital engagement in Project JOINTS, weasked participants to indicate which of the 10 ProjectJOINTS tools and activities their hospital used or par-ticipated in (see figure 2 for a complete list of cam-paign activities, materials and tools). To measurebundle adoption, we asked the following question:‘Are staff at your hospital now doing all five of thecomponents?’We also asked questions about implementation of

each of the five individual bundle components. Weasked whether hospital staff had (a) performed thebundle component consistently prior to ProjectJOINTS; (b) initiated the component as a result ofparticipation in Project JOINTS; (c) performed thecomponent inconsistently before Project JOINTS, butnow were performing it consistently; or (d) other(please specify). We also asked about barriers andfacilitators to bundle adoption. The interview proto-col included both close-ended and open-ended ques-tions, and we encouraged participants to elaborate ontheir answers. The research protocol was reviewedand approved by our Institutional Review Board.The interviews with representatives of hospitals

enrolled in Project JOINTS were conducted by tele-phone between November 2012 and January 2013.Based on the information provided by IHI, we con-tacted all eligible hospitals that had enrolled in ProjectJOINTS in the five intervention states (n=109 hospi-tals). Our recruitment procedures including a firstcontact by email sent to the identified Project JOINTScontact at each participating hospital. We followed upnon-respondents with a second email and then a tele-phone call. Each interview lasted for approximately

45 min; participant responses were entered intoSelectSurvey, a web survey tool.

Quantitative data analysisTo conduct the quantitative analyses, we constructedseveral indices based on the interview items notedabove. We created an index of adherence to the ProjectJOINTS QI methods by adding the number of threeQI methods (range 0–3) a respondent reported thathis/her hospital used within Project JOINTS. Wecreated a hospital engagement index by summing thenumber of Project JOINTS tools and activities eachhospital used (range 0–10). We also measured thetype of resources hospitals used by creating two newindices: number of interactive activities (range 0–4)and number of printed and online Project JOINTSmaterials and tools (range 0–6). Finally, we created adichotomous (yes/no) measure of complete bundleadoption, which we used as a primary measure ofintervention uptake. We also created a variable meas-uring the number of newly initiated (response cat-egory b) or improved (response category c) bundlecomponents, which was used as a secondary measureof intervention uptake.Covariates included the following hospital charac-

teristic variables: hospital size (small, 2–42 beds;medium, 43–115 beds and large, 116–381 beds; cal-culated based on the number of beds tertiles obtainedfrom 2011 American Hospital Association (AHA)data); location (urban vs rural, as defined by partici-pants, and state); volume of hip and knee arthroplasty(above or equal/below 250 annual procedures (medianvalue) using the 2011 Hospital Compare Medicaredata); and self-reported previous experience with IHIcampaigns, other SSI prevention bundles and orga-nised patient safety initiativesi (yes/no to each).

Figure 2 Hospital use of Project JOINTS (Joining Organizations IN Tackling SSIs) resources (N=73).

iThis variable had no significant bivariate associations with theoutcome variables and was not included in the final models.

Original research

Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169 41

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

To test our hypotheses, depending on the outcomemeasures, we used logistic or Poisson regression withrobust SEs to account for potential heteroscedasticityand tested for mediation effects. We first modelled therelationship between adherence to QI methods andtwo measures of intervention uptake—completebundle adoption and initiation of and increasedadherence to bundle components. Second, we mod-elled the relationship between hospital engagementand measures of bundle uptake. Finally, we modelledthe impact of both predictors together to exploremediation effects.

Qualitative data analysisTo identify and summarise themes from responses tothe open-ended questions, we developed a hierarchic-ally organised codebook based on the interview proto-col. We incorporated themes from the QI literatureand previous IHI campaigns, and new unanticipatedthemes that emerged from the interviews. Responsesto open-ended questions (such as the question aboutbarriers to complete bundle adoption) were reviewedand systematically coded (using MAXQDA 10 qualita-tive data management software) into a series of 8–10response categories based on the codebook developedfor each question. Two qualitative researchers codedthe data independently and then reviewed coded textto ensure coding consistency; disagreements were dis-cussed until consensus was reached.39 40

RESULTSOur final sample consisted of representatives of 73 ofthe 109 hospitals participating in Project JOINTS(response rate of 67%). Response rates varied acrossthe five states—from a low of 60% of eligible hospitalsto a high of 83%. Each of our respondents was signifi-cantly involved in Project JOINTS and knew about theproject-related activities in his/her hospital. The major-ity of respondents were infection control practitioners,followed by directors/managers of surgical services anddirectors of clinical quality. Respondents had a rela-tively long tenure at their hospital (mean=15.3 years;SD=11.85; range 5 months–43 years) and had spent,on average, 7.1 years (SD=7.11; range 3 months–30 years) in the current role.Most of the hospitals in our sample were small

(41%) or medium-sized (36%) (see table 1). About anequal number were located in rural and urban settingsand approximately equal numbers had high and lowarthroplasty volume. Most hospitals reported partici-pating in prior IHI campaigns (59%), SSI preventionactivities (63%) and organised patient safety initiatives(90%). More than one-third (36%) of the hospitals inour sample was located in a single state.

Use of QI methods and engagement in project JOINTSMore than two-thirds of responding hospitals (n=50)used at least one of the Project JOINTS QI methods.

Out of the 50, 90% had an implementation team,52% developed an implementation plan and 36%conducted a small test of change (see table 1).Participating hospitals were variably engaged in

Project JOINTS. On average, hospital QI leadersreported using slightly more than half of all availableProject JOINTS resources (5.4 out of 10). Theyreported participating in 2.45 (out of four) interactive

Table 1 Sample descriptions (N=73)

% ormean SD

Hospital size

Large 23%

Medium 36%

Small 41%

Hospital volume

High 48%

Low 52%

Rural location 53%

Previous experience with IHI campaigns (yes=1) 59%

Previous experience with other SSI preventionbundles (yes=1)

63%

Previous experience with patient safety initiatives(yes=1)

90%

State

A 19%

B 22%

C 10%

D 14%

E 36%

Complete bundle adoption (yes=1) 55%

Number of initiated/improved bundle components 1.22 1.16

Number of previously implemented bundlecomponents

2.96 0.96

Adherence to Project JOINTS QI methods* 1.22 1.03

Number of Project JOINTS QI methods used

0 32%

1 27%

2 29%

3 12%

Project JOINTS QI methods used

Implementation plan (yes=1) 52%†

Implementation team (yes=1) 90%†

Small tests of change (yes=1) 36%†

Hospital engagement‡ 5.40 2.63

Number of interactive campaign activitiesparticipated in

2.45 1.09

Number of campaign materials and tools used 2.95 1.99

*Adherence to Project JOINTS QI methods is an index calculated byadding the number of three QI methods a respondent reported that his/herhospital used within Project JOINTS (range 0–3).†The denominator is 50, which is the number of hospitals reporting usingat least one Project JOINTS QI method.‡Hospital engagement is an index calculated by adding the number ofProject JOINTS tools and activities each hospital used (range 0–10).IHI, Institute for Healthcare Improvement; JOINTS, Joining Organizations INTackling SSIs; QI, quality improvement; SSI, surgical site infection.

Original research

42 Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

activities (SD=1.09) and using 2.95 (out of six) onlineor print Project JOINTS materials and tools(SD=1.99). Webinar calls and electronic communica-tions were the most popular interactive activities withmore than four-fifths of hospital QI leaders reportingthat their hospitals had participated in them (figure 2).Among the project materials and tools, the ‘How toGuide’, the ‘improvement tools’ and the surgeon evi-dence handouts were the most popular: roughly three-fifths of respondents reporting using each.

Adoption of the enhanced SSI prevention bundle and itscomponentsThe majority of respondents (55%) reported that theirhospital adopted the complete (five-component)enhanced SSI prevention bundle (table 1). Figure 3suggests, however, that these hospitals were alreadyfamiliar with many of the bundle components: onaverage, they reported using three out of five enhancedSSI prevention bundle components prior to ProjectJOINTS (appropriate hair removal, appropriate use ofprophylactic antibiotic and the use of alcohol-containing antiseptic were the most commonlyreported). Approximately one-third of the hospitalsreported not initiating or increasing adherence to anyof the bundle components during Project JOINTS (seeonline supplementary appendix B). Hospitals that didinitiate or increase adherence tended to focus on twoof the three new components suggested by ProjectJOINTS—the 3-day use of CHG soap (50%) andscreening for SA carriage and decolonisation of carriers(40%) (figure 3).

Does adherence to the QI methods affect bundleadoption?Reported adherence to the QI methods was positivelyassociated with complete bundle adoption (r=0.33,p≤0.001) and initiation of or improved adherence tobundle components (r=0.51, p≤0.001). Controllingfor hospital characteristics and previous

implementation of SSI bundle components, for eachadditional QI method a hospital implemented, theodds of adopting the complete enhanced SSI preven-tion bundle increased; this result was statistically sig-nificant (OR=2.53, 95% CI 1.18 to 5.42, p≤0.05;table 2, model A). However, adherence to the QImethods was not significantly associated with initiationor improved adherence to individual bundle compo-nents (incidence rate ratio (IRR)=1.16, p>0.1; table 3,model A).

Does the level of hospital engagement in Project JOINTSaffect bundle uptake?The level of hospital engagement with ProjectJOINTS was positively and statistically significantlyassociated with adherence to Project JOINTS QImethods (r=0.52, p≤0.01), complete bundle adoption(r=0.23, p≤0.05) and initiation of or improved adher-ence to bundle components (r=0.5, p≤0.001).Controlling for hospital characteristics and previouslyimplemented SSI prevention bundle components,more engaged hospitals had better odds of adoptingthe entire bundle (OR=1.45, 95% CI 1.09 to 1.93,p≤0.05; table 2, model B). The odds of completebundle adoption were 1.45 times higher for each add-itional Project JOINTS tool or activity a hospitalreported using. The association of hospital engage-ment with the number of bundle components initiatedor newly adhered to was statistically significant butsmall (IRR=1.12, 95% CI 1.03 to 1.22, p≤0.01;table 3, model B). Each additional hospital activity ortool used was associated with a 12% increase in thenumber of bundle components a hospital QI leaderreported initiating or improving adherence to.

Which is more important: adherence to the QI methods orhospital engagement?When we entered both predictors into the multipleregression model, adherence to Project JOINTS QImethods (OR=2.05, 95% CI 0.91 to 4.62, p≤0.1;

Figure 3 SSI prevention bundle component adoption (N=73). SSI, surgical site infection; CHG, chlorhexidine gluconate.

Original research

Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169 43

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

table 2, model C) and level of hospital engagement(OR=1.32, 95% CI 0.97 to 1.81, p≤0.1; table 2,model C) both had a positive effect on completebundle adoption. However, only hospital engagement(IRR=1.11, 95% CI 1.02 to 1.21, p≤0.05; table 3,model C), and not adherence to Project JOINTS QImethods (IRR=1.04, 95% CI 0.87 to 1.26, p>0.1;table 3, model C), was associated with initiation of orimproved adherence to individual bundle components.To determine whether the type of engagement in

Project JOINTS is associated with bundle adoption,we differentiated hospital engagement into participa-tion in interactive campaign activities and the use ofprinted and online campaign materials and tools. Wefound that complete bundle adoption was only asso-ciated with adherence to Project JOINTS QI methods(OR=2.01, 95% CI 0.88 to 4.62, p≤0.1; table 2,model D), suggesting that the level rather than type ofengagement, along with the use of QI methods, hadstronger positive associations with the adoption of allbundle components. We also found that initiation of

or improved adherence to bundle components wasstatistically significantly associated only with the useof printed and online materials and tools (IRR=1.13,95% CI 1.01 to 1.27, p≤0.05; table 3, model D).

What are the barriers to complete bundle adoption?Hospital representatives we interviewed cited severalbarriers to adopting all five bundle components.One-third of all respondents in hospitals that had notimplemented the complete bundle reported lack ofphysician buy-in and staff resistance as the main bar-riers. Explained one interviewee,

Our facility is owned half by physicians, and theydon’t belong to the same organization. They are notour employees. So if they decide not to do something,they don’t. We always try to compromise and havetried to come up with a process that will make themhappy and that will serve the patients as well, butfrankly, we have failed. We failed at getting ProjectJOINTS on board here.

Table 3 Poisson regression models predicting initiation/improved adherence to bundle components (N=73)

Model A: QI methodsModel B: hospitalengagement

Model C: QI methodsand hospitalengagement

Model D: QI methodsand type of hospitalengagement

IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI

Adherence to Project JOINTS QI methods 1.16 (0.96 to 1.40) 1.04 (0.87 to 1.26) 1.04 (0.86 to 1.26)

Hospital engagement 1.12 (1.03 to 1.22)** 1.11 (1.02 to 1.21)*

Interactive campaign activities 1.08 (0.91 to 1.30)

Campaign materials and tools 1.13 (1.01 to 1.27)*

Wald χ2 148.05** 133.72** 137.52** 141.89**

McFadden’s R2 0.2299 0.2431 0.2436 0.2439

Results from Poisson regression with robust standard errors. All models controlled for state, rural/urban location, hospital size, surgery volume, previousInstitute for Healthcare Improvement experience, previous surgical site infection experience and number of prior components.The Wald χ2 statistic is used to determine whether at least one of the predictors’ regression coefficient is not equal to zero.R2 indicates the proportion of the variation in the outcome variable due to the independent variables included.*significant at p≤0.05; **significant at p≤0.01.JOINTS, Joining Organizations IN Tackling SSIs; SSI, surgical site infection; QI, quality improvement; IRR, incidence rate ratio.

Table 2 Logistic regression models predicting complete bundle adoption (N=73)

Model A: QI methodsModel B: hospitalengagement

Model C: QI methodsand hospitalengagement

Model D: QI methodsand type of hospitalengagement

OR 95% CI OR 95% CI OR 95% CI OR 95% CI

Adherence to Project JOINTS QI methods 2.53 (1.18 to5.43)* 2.05 (0.91 to 4.62)† 2.01 (0.88 to 4.62)†

Hospital engagement 1.45 (1.09 to 1.93)* 1.32 (0.97 to 1.81)†

Interactive campaign activities 1.20 (0.62 to 2.31)

Campaign materials and tools 1.40 (0.92 to 2.14)

Wald χ2 22.28* 20.15† 26.98* 26.20*

McFadden’s R2 0.2534 0.2441 0.2866 0.2878

Results from logistic regression with robust standard errors. All models controlled for state, rural/urban location, hospital size, surgery volume, previousInstitute for Healthcare Improvement experience, previous surgical site infection experience and number of prior components.The Wald χ2 statistic is used to determine whether at least one of the predictors’ regression coefficient is not equal to zero.R2 indicates the proportion of the variation in the outcome variable due to the independent variables included.*significant at p≤0.05; †significant at p≤0.1.JOINTS, Joining Organizations IN Tackling SSIs; SSI, surgical site infection; QI, quality improvement.

Original research

44 Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

The second and third most commonly cited barriersto complete bundle adoption were the cost of antibio-tics and CHG soap (18%) and the logistics of screen-ing and bathing (18%). Indeed, several individualsnoted that the logistics of implementing some bundlecomponents (eg, screening and bathing) were moredifficult to implement than others (eg, hair removal).Logistical challenges included coordination amongstaff, calls to insurance companies, increased treat-ments and dependence on patients’ actions outside ofa care facility.Finally, 15% cited a perceived lack of evidence sup-

porting some bundle components. As described byone interviewee, ‘I do not see any literature that saysthat three days are necessary’. ‘All the literature I’velooked at says that [bathing/showering one day beforesurgery] is enough. It’s already hard to get this going100% of the time,’ said another. Similarly, another15% reported that their hospital had plans to startusing the bundle components soon, but wanted to seethe results of this evaluation study. Interestingly, 9%of hospital respondents did not feel the need to adoptthe bundle because SSI was ‘not a problem in thehospital’.

DISCUSSIONOur results supported both of our study hypotheses.Both adherence to the QI methods and the level ofhospital engagement in Project JOINTS were positivelyassociated with the adoption of the complete SSI pre-vention bundle. Controlling for hospital characteristicsand previously implemented bundle components, wefound that adherence to more of the QI methods wasassociated with higher odds of adopting the completebundle. This association between adoption ofevidence-based practices and QI methods adherence(developing a plan, forming an implementation teamand conducting small-scale tests of change) is consist-ent with prior literature suggesting that basing a cam-paign on systematic use of QI methods is an importantpathway towards its effectiveness.6 37

In addition, we found that among hospitals partici-pating in Project JOINTS, those that were moreengaged in Project JOINTS were more likely to imple-ment the complete bundle and initiated or improvedadherence to a larger number of the individual bundlecomponents. It is possible that having access to experi-enced faculty members and carefully developedproject materials, as well as learning from peersduring in-person and online interactive activities,helped hospitals address barriers to adopting newevidence-based practices in their own settings.Hospital engagement was also an important factorthat facilitated intervention implementation in the IHIBreakthrough Series collaboratives.38 This result maysuggest that large-scale QI campaigns can be moresuccessful if they focus additional efforts on encour-aging participating hospitals to engage in more

campaign activities and use campaign materials; none-theless, we cannot exclude the possibility that moreengaged hospitals are simply more receptive to QI.All else being equal, hospitals that used more

Project JOINTS printed and online materials andtools were more likely to initiate or improve adher-ence to individual bundle components compared withhospitals that used fewer Project JOINTS resources.This finding may be somewhat surprising given priorresearch suggesting that printed educational materialshave limited effects on professional practice out-comes.41 42 We note, however, that Project JOINTSdid not merely distribute copies of clinical guidelines.Instead, IHI staff created tailored Project JOINTSmaterials for specific uses and audiences (eg, surgeon,family and patient handouts) to facilitate their buy-inand acceptance of Project JOINTS. Tailoring interven-tion materials and ensuring that they are easy-to-useand readily accessible to campaign participants (eg, viaproject website) may be very important. Creation ofhigh-quality customised intervention materials thatclearly describe the evidence base behind interventioncomponents may facilitate adoption of best practicesby specific groups of stakeholders (in this instance,orthopaedic surgeons).More than half of our respondents reported that

their hospitals had adopted the complete bundle.Among those hospitals that did not adopt the com-plete bundle, 36% did not improve or increase adher-ence to any bundle components as part of ProjectJOINTS (data not shown). Lack of surgeon buy-inand staff support for implementing the bundle arewell-known implementation challenges in the use ofQI collaboratives43 and were the two most cited bar-riers in this study.Moreover, Project JOINTS hospitals often struggled

to implement the two components (using CHG soapfor at least three days before the surgery and pre-operative nasal screening and SA decolonisation) thatthey reported to be the most logistically challengingand for which there was still some scepticism aboutthe evidence supporting their effectiveness. Thefinding about logistical challenges illustrates the diffi-culty of changing processes that require substantialcoordination of multiple services and departments,thereby supporting an argument that the effects of aQI intervention ‘may be mediated by the nature of theprocesses being targeted and the amount of controlparticipants have over them’.13 Hospital representa-tives also noted that in general they conduct theirown evaluation of the evidence supporting eachbundle component and implement components onlywhen they are persuaded that the evidence of effect-iveness meets their own threshold. This observationabout claims of inadequate evidence is consistent withthe results of previous research.12 44 Several respon-dents hoped Project JOINTS would produce the evi-dence necessary to justify complete bundle adoption.

Original research

Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169 45

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

This study has several limitations. First, it relies onself-reported, cross-sectional interview data collectedfrom a single representative of each participating hos-pital. Senior personnel participating in our studymight have been misinformed about the implementa-tion of campaign activities. Although frequently usedto evaluate the process and outcomes of QI collabora-tives, self-reported data may also be subject to recallbias. To minimise recall bias, we started collectingthese data within a month after the Project JOINTSprogramme was completed. If additional changes inbundle adoption took longer to occur, our findingsmay underestimate the effect of the intervention.Second, responses may have been affected by social

desirability bias. Senior personnel participating in a QIcampaign may exaggerate the extent of bundle adop-tion. While we cannot quantify the magnitude of thisbias, we note that several participants reported bothpositive and negative feedback about their ProjectJOINTS experiences (36% reported that they did notinitiate or improve adherence to any components ofthe SSI prevention bundle). Nevertheless, it is possiblethat respondents who report higher adherence to QImethods and engagement in Project JOINTS may alsobe more likely to report bundle uptake because theymay believe that if their hospital adhered to QImethods or engaged in Project JOINTS, it must alsohave improved adherence to bundle components (andvice versa). According to cognitive dissonance theory,individuals commonly search for consistency in theirbeliefs and perceptions; they try to avoid inconsisten-cies that may lead to discrepancies between theirbeliefs and behaviours by changing their beliefs andperceptions to reduce or eliminate the dissonance.45

Finally, while we controlled for a number of hos-pital characteristics that describe intervention context,our interview protocol did not include questions onseveral contextual variables, such as organisationalculture, teamwork and leadership support; these mayhave affected the implementation of QI. Although thiswas a large QI campaign, this sample of voluntaryparticipants tended to be smaller and more oftenrural, which may limit generalisability of these resultsto other types of hospitals.Looking inside the ‘black box’ of a QI campaign,

we find that—from the point of view of participants—certain campaign strategies were more helpful thanothers in assisting hospitals to reach their SSI preven-tion goals. Our findings suggest four opportunities toenhance the effectiveness of large-scale QI campaigns.First, actively engaging hospitals by encouraging

them to use campaign resources, such as online andprinted materials/tools and interactive activities, mayhelp hospitals to initiate or improve adherence to newbundle components. Customising materials so thatthey are tailored to the needs of participating hospitalsand their stakeholders can help hospitals address bar-riers to adopting new QI practices in their own setting.

Second, printed and online campaign materials thatare self-explanatory offer a forthright description ofthe latest clinical evidence and explain the advantagesand potential benefits of changing current clinicalpractice appear to be more effective. For surgical QIcampaigns, particular emphasis should be placed onthe materials distributed directly to surgeons; materi-als tailored specifically for this audience could becrucial for engaging them in the campaign and gettingtheir support for clinical practice change. Lack ofsurgeon buy-in was one of the two most cited barriersto implementing the Project JOINTS bundle.Third, hospital engagement in campaign activities

may help participants learn and share experienceswith one another. We found that hospital engagementwas positively associated with both adoption of thecomplete bundle and with initiation and adoption ofindividual components. Local Rapid Spread Networkorganisations may be better positioned to engage hos-pitals in QI activities.Finally, campaigns that promote adherence to specified

QI methods may be more likely to achieve bundle adop-tion. Experienced campaign faculty can help hospitalimplementation teams develop implementation plansand advise on how to conduct small tests of change.Developing a QI plan and conducting small tests ofchange may have been underused in this campaign.In conclusion, our findings provide some support

for the use of QI methods and promotion of activehospital engagement in a QI campaign as a way toincrease adoption of evidence-based, multicomponentpatient safety bundles in orthopaedic practice.

Contributors DK and MSR led the study design. DK supervisedthe team that collected the study data and performed thequalitative analyses. CH helped collect the quantitative data andcarried out the statistical analyses. All authors participated inthe interpretation of the results and drafting and criticalrevision of the manuscript. ECS was the principal investigatorof the study.

Funding This study was funded by a grant (R18 AE000051)from the US Department of Health and Human Services.

Competing interests None.

Ethics approval This study was reviewed and approved by theHuman Subjects’ Protection Committee.

Provenance and peer review Not commissioned; externallypeer reviewed.

REFERENCES1 Beckett M, Quiter E, Ryan G, et al. Bridging the gap between

basic science and clinical practice: the role of organizations inaddressing clinician barriers. Implement Sci 2011;6:35.

2 Kahn K, Ryan G, Beckett M, et al. Bridging the gap betweenbasic science and clinical practice: a role for communityclinicians. Implement Sci 2011;6:34.

3 Dougherty D, Conway PH. The “3T’s” road map to transformUS health care. JAMA 2008;299:2319–21.

4 Penna RD, Martel H, Neuwirth EB, et al. Rapid spread ofcomplex change: a case study in inpatient palliative care. BMCHealth Serv Res 2009;9:245.

Original research

46 Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from

5 Newton PJ, Davidson PM, Halcomb EJ, et al. An introduction tothe collaborative methodology and its potential use for themanagement of heart failure. J Cardiovasc Nurs 2006;21:161–8.

6 Kilo CM. A framework for collaborative improvement: lessonsfrom the Institute for Healthcare Improvement’s BreakthroughSeries. Qual Manag Health Care 1998;6:1–14.

7 Mittman BS. Creating the evidence base for quality improvementcollaboratives. Ann Intern Med 2004;140:897–901.

8 Benn J, Burnett S, Parand A, et al. Factors predicting change inhospital safety climate and capability in a multi-site patientsafety collaborative: a longitudinal survey study. BMJ Qual Saf2012;21:559–68.

9 Parand A, Burnett S, Benn J, et al. Medical engagement inorganisation-wide safety and quality-improvementprogrammes: experience in the UK Safer Patients Initiative.Qual Saf Health Care 2010;19:e44.

10 Rushmer R, Voigt D. MEASURE IT, IMPROVE IT: the SaferPatients Initiative and quality improvement in subcutaneous insulintherapy for hospital in-patients. Diabet Med 2008;25:960–7.

11 Benning A, Dixon-Woods M, Nwulu U, et al. Multiplecomponent patient safety intervention in English hospitals:controlled evaluation of second phase. BMJ 2011;342:d199.

12 Benning A, Ghaleb M, Suokas A, et al. Large scaleorganisational intervention to improve patient safety in fourUK hospitals: mixed method evaluation. BMJ 2011;342:d195.

13 Power M, Tyrrell PJ, Rudd AG, et al. Did a qualityimprovement collaborative make stroke care better? A clusterrandomized trial. Implement Sci 2014;9:40.

14 Marsden J, Kamal N, Lauscher H, et al. Evidence 2 excellence:an emergency medicine quality improvement model utilizingtechnology-enabled interprofessional collaboration in BritishColumbia. In: Ho K, Jarvis-Selinger S, Novak Lauscher H,Cordeiro J, Scott R, eds. Technology enabled knowledgetranslation for eHealth. New York: Springer, 2012:91–116.

15 Ho K, Marsden J, Jarvis-Selinger S, et al. A collaborativequality improvement model and electronic community ofpractice to support sepsis management in emergencydepartments: Investigating care harmonization for provincialknowledge translation. JMIR Res Protoc 2012;1:e6.

16 Øvretveit J, Bate P, Cleary P, et al. Quality collaboratives:lessons from research. Qual Saf Health Care 2002;11:345–51.

17 Schouten LMT, Hulscher MEJL, van Everdingen JJE, et al.Evidence for the impact of quality improvement collaboratives:systematic review. BMJ 2008;336:1491–4.

18 Hulscher MEJL, Schouten LMT, Grol RPTM, et al.Determinants of success of quality improvement collaboratives:what does the literature show? BMJ Qual Saf 2013;22:19–31.

19 Nadeem E, Olin SS, Hill LC, et al. Understanding thecomponents of quality improvement collaboratives: asystematic literature review. Milbank Q 2013;91:354–94.

20 Boushon B, Provost L, Gagnon J, et al. Using a virtualbreakthrough series collaborative to improve access in primarycare. Jt Comm J Qual Patient Saf 2006;32:573–84.

21 Berwick DM, Calkins DR, McCannon CJ, et al. The 100 000lives campaign. JAMA 2006;295:324–7.

22 McCannon CJ, Schall MW, Calkins DR, et al. Saving 100 000lives in US hospitals. BMJ 2006;332:1328–30.

23 McCannon CJ, Hackbarth AD, Griffin FA. Miles to go: anintroduction to the 5 Million Lives Campaign. Jt Comm J QualPatient Saf 2007;33:477–84.

24 Urquhart DM, Hanna FS, Brennan SL, et al. Incidence and riskfactors for deep surgical site infection after primary total hiparthroplasty: a systematic review. J Arthroplasty 2010;25:1216–22.e3.

25 Whitehouse JD, Friedman ND, Kirkland KB, et al. The impactof surgical-site infections following orthopedic surgery at acommunity hospital and a university hospital: adverse quality oflife, excess length of stay, and extra cost. Infect Control HospEpidemiol 2002;23:183–9.

26 Hussaini A, Martin J. Preventing infection after hip and kneereplacements. Healthc Exec 2013;28:68–70.

27 Institute for Healthcare Improvement. How-to guide: preventsurgical site infection for hip and knee arthroplasty. Cambridge,MA: Institute for Healthcare Improvement, 2012.

28 Darouiche RO, Wall MJ Jr, Itani KM, et al. Chlorhexidine–alcohol versus povidone–iodine for surgical-site antisepsis.N Engl J Med 2010;362:18–26.

29 Swenson BR, Hedrick TL, Metzger R, et al. Effects ofpreoperative skin preparation on postoperative woundinfection rates: a prospective study of 3 skin preparationprotocols. Infect Control Hosp Epidemiol 2009;30:964–71.

30 Paulson DS. Efficacy evaluaton of a 4% chlorhexidine gluconateas a full-body shower wash. Am J Infect Control 1993;21:205–9.

31 Byrne D, Napier A, Phillips G, et al. Effects of whole bodydisinfection on skin flora in patients undergoing electivesurgery. J Hosp Infect 1991;17:217–22.

32 Kaiser A, Kernodle D, Barg N, et al. Influence of preoperativeshowers on staphylococcal skin colonization: a comparative trialof antiseptic skin cleansers. Ann Thorac Surg 1988;78:35–8.

33 Bode LG, Kluytmans JA, Wertheim HF, et al. Preventingsurgical-site infections in nasal carriers of Staphylococcusaureus. N Engl J Med 2010;362:9–17.

34 Kim DH, Spencer M, Davidson SM, et al. Institutionalprescreening for detection and eradication of methicillin-resistant Staphylococcus aureus in patients undergoing electiveorthopaedic surgery. J Bone Joint Surg 2010;92:1820–6.

35 Institute for Healthcare Improvement. How-to guide: preventsurgical site infections. Cambridge, MA: Institute forHealthcare Improvement, 2012.

36 Pronovost PJ. Navigating adaptive challenges in qualityimprovement. BMJ Qual Saf 2011;20:560–3.

37 Speroff T, O’Connor GT. Study designs for PDSA qualityimprovement research. Qual Manag Healthc 2004;13:17–32.

38 Nembhard IM. Learning and improving in quality improvementcollaboratives: which collaborative features do participantsvalue most? Health Serv Res 2009;44(2p1):359–78.

39 MacQueen KM, McLellan E, Kay K, et al. Codebookdevelopment for team-based qualitative analysis. CultAnthropol Methods 1998;10:31–6.

40 Khodyakov D, Uscher-Pines L, Lorick SA, et al. A qualitativeanalysis of the impact of healthcare personnel influenzavaccination requirements in California. Vaccine2014;42:288–93.

41 Giguere A, Legare F, Grimshaw J, et al. Printed educationalmaterials: effects on professional practice and healthcareoutcomes. Cochrane Database Syst Rev 2012;10:CD004398.

42 Grimshaw J, Eccles M, Tetroe J. Implementing clinicalguidelines: current evidence and future implications. J ContinEduc Health Prof 2004;24(Suppl 1):S31–7.

43 Montoye CK, Mehta RH, Baker PL, et al. A rapid-cyclecollaborative model to promote guidelines for acute myocardialinfarction. Jt Comm J Qual Patient Saf 2003;29:468–78.

44 Grol R, Grimshaw J. From best evidence to best practice:effective implementation of change in patients’ care. Lancet2003;362:1225–30.

45 Festinger L. A theory of cognitive dissonance. Stanforduniversity press, 1962.

Original research

Khodyakov D, et al. BMJ Qual Saf 2015;24:38–47. doi:10.1136/bmjqs-2014-003169 47

on August 15, 2020 by guest. P

rotected by copyright.http://qualitysafety.bm

j.com/

BM

J Qual S

af: first published as 10.1136/bmjqs-2014-003169 on 5 N

ovember 2014. D

ownloaded from