Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
ABOUT PRACTICE-BASED RESEARCH NETWORKS
Preventing the Voltage Drop: KeepingPractice-based Research Network (PBRN) PracticesEngaged in StudiesBarbara P. Yawn, MD, MSc, Allen Dietrich, MD, MPH, Deborah Graham, MSPH,Susan Bertram, BSN, MSN, Marge Kurland, BSN, Suzanne Madison, MPH,Dawn Littlefield, CSC, Brian Manning, MPH, Craig Smail, MPH,and Wilson Pace, MD
Introduction: Practice-based research continues to evolve and has become a major methodology formany pragmatic studies. While early practice-based network projects were usually short term, currentstudies often introduce or compare practice innovations that require long-term evaluation. That changerequires that practice sites remain engaged in research work for up to 5 years, a time that can allow fora significant “voltage drop,” or decline in active participation.
Methods: Over the past 15 years we have developed and adapted several strategies to facilitate andencourage the continued active engagement of practices in practice-based research network studies ofup to 5 years’ duration. The concepts, details, evaluation, and results (when available) of the strategiesare described.
Results: Eight strategies that enhance practice sites’ attention to enrollment, data collection and con-tinued use of the implemented practice change are described.
Conclusion: The loss of momentum, or “voltage drop,” that happens in longer-term practice-basedresearch network studies can be addressed using multiple strategies. (J Am Board Fam Med 2014;27:123–135.)
Keywords: Methods, Practice-based Research Network, Program Effectiveness, Randomized Controlled ClinicalTrials, Training
Practice-based research network (PBRN) researchhas become a valuable method of identifying andanswering questions relevant to primary care pa-tients and practices. As PBRN research has ma-tured, studies have evolved from simple, short-termobservational studies to longer-duration interven-tional studies that involve all members of the prac-
tice and enrolled patients.1 PBRN methodologyliterature also has progressed from reports of howto begin PBRNs2–6 to publications relevant to con-ducting longer-term and more complex studies inestablished PBRNs.7–12
Practices usually begin a study with significantenthusiasm and willingness to learn and fulfill thestudy requirements.13,14 As time progresses, thereality of everyday life in a busy practice may usurpthe time and effort needed for continued studyparticipation. We have observed this phenomenonin our own studies, and we have labeled this par-ticipation decline as “voltage drop” (A. Dietrich,MD, personal communication).
Regular study visits to local or regional PBRNpractices by practice enhancement assistants, prac-tice enhancement and research coordinators, or re-gional study team members may reduce the loss ofstudy momentum.15,16 Such activities seem to im-
This article was externally peer reviewed.Submitted 14 January 2013; revised 19 July 2013; accepted
12 August 2013.From the Department of Research, Olmsted Medical
Center, Rochester, MN (BPW, SB, MK, SM, DL); theNational Research Network, American Academy of FamilyPhysicians, Leawood, KS (DG, BM, CS, WP); and theDartmouth Medical Center, Dartmouth, NH (AD).
Funding: none.Conflict of interest: none declared.Corresponding author: Barbara P. Yawn, MD, MSc, De-
partment of Research, Olmsted Medical Center, 210 NinthSt. SE, Rochester, MN 55904 (E-mail: [email protected]).
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 123
prove practice outcomes17–19 but are not feasiblefor large, geographically dispersed studies thatwork to achieve a nationally representative sampleof practices.
Over the past 10 years of working with practicesto complete pragmatic trials3 we have identifiedseveral methods to facilitate initial engagement bythe practices and to maintain the initial high levelof enthusiasm and willingness to participate overthe long study period. The purpose of this report isto describe these strategies and their impact withthe purpose of supporting other researchers in im-plementing and enhancing their work in PBRNs.Other PBRNs have been engaged in similar trans-lational and practice change implementation stud-ies, so the strategies presented here may not beunique to our work but to date have not beenpublished in the PBRN literature.
MethodsThe strategies described in this report come frominformation collected over the past 10 years in 4large, federally funded studies, each lasting up to 5years (Table 1). We believe that these 4 long-termstudies are most representative of the difficulties wehave encountered in sustaining the practices’ inter-est and commitment over several years in projectsthat introduced a significant practice change. De-spite the variety of topics and varying study designs,many of the same implementation strategies andsupport techniques proved to be useful across stud-ies. Each of the studies introduced a practicechange: using spirometry to evaluate patients withasthma and chronic obstructive pulmonary disor-der20; introducing screening, diagnosis, and man-agement support for postpartum depression21–24;using new asthma tools (asthma APGAR tools) inmanaging children and adults with asthma25; andassessing the effect of 2 classes of long-acting bron-chodilators on exacerbation rates in black adultswith asthma.
We included 96 practices in the four large prag-matic studies mentioned above. Of the total, 87were family medicine practices, 5 were pulmonol-ogy practices, and 4 were pediatric practices. Foreach study except the spirometry study, we havelost from 1 to 4 practices because of a variety ofproblems, including dissolution of practices, physi-cian illness with prolonged disability, and inabilityto recruit patients (Table 1). The practices included Ta
ble
1.Th
eFo
urPB
RNSt
udie
sIn
clud
edin
this
Stud
y
Stud
yN
ame
Pra
ctic
es(n
)St
udy
Des
ign
Stud
yD
urat
ion
(mon
ths)
Cen
tral
Site
sSt
atus
Fund
ing
Pra
ctic
eC
hang
eD
isco
ntin
ued
Pra
ctic
es(n
)
Intr
oduc
tion
ofSp
irom
etry
into
Pri
mar
yC
are
18B
efor
e/af
ter
obse
rvat
iona
l18
OM
Can
dA
AFP
NR
NC
are
Net
Com
plet
edin
2006
20N
HL
BI
Usi
ngsp
irom
etry
toev
alua
tepa
tient
sw
ithas
thm
aan
dC
OP
D1
(una
ble
toen
roll
patie
nts)
TR
IPP
D(T
rans
latin
gR
esea
rch
into
Pri
mar
yC
are
Pra
ctic
efo
rP
ostp
artu
mD
epre
ssio
n)
28R
ando
miz
edco
ntro
lled
tria
l60
OM
Can
dA
AFP
NR
NC
ompl
eted
in20
1021
,22
AH
RQ
Intr
oduc
ing
scre
enin
g,di
agno
sis,
and
man
agem
ent
supp
ort
for
post
part
umde
pres
sion
4(2
prac
tices
diss
olve
d,1
prac
tice
had
long
-ter
mill
ness
oflo
cal
prim
ary
inve
stig
ator
,1pr
actic
eco
uld
not
mee
ten
rollm
ent
goal
s)T
heA
sthm
aT
ools
Stud
y28
Ran
dom
ized
cont
rolle
dtr
ial
60O
MC
and
AA
FPN
RN
Inpr
ogre
ssA
HR
QU
sing
new
asth
ma
tool
sin
man
agin
gch
ildre
nan
dad
ults
with
asth
ma
1(P
ract
ice
dest
roye
dby
torn
ado,
unab
leto
cont
inue
whi
letr
ying
tore
build
)B
EL
T(B
lack
Ast
hmat
ic’s
Exa
cerb
atio
nsw
ithL
AB
Avs
Tio
trop
ium
)
30R
ando
miz
edco
ntro
lled
tria
l36
OM
C,A
AFP
NR
N,
and
Har
vard
Med
ical
Scho
ol
Inpr
ogre
ss23
AH
RQ
Ass
essi
ngth
eim
pact
onex
acer
batio
nra
tes
of2
clas
ses
oflo
ng-a
ctin
gbr
onch
odila
tors
inbl
ack
adul
tsw
ithas
thm
a
3(1
prac
tice
unab
leto
dow
ork,
2si
tes
unab
leto
mee
ten
rollm
ent
goal
s)
AA
FPN
RN
,Am
eric
anA
cade
my
ofFa
mily
Phy
sici
ans
Nat
iona
lRes
earc
hN
etw
ork;
AH
RQ
,Age
ncy
for
Hea
lthca
reR
esea
rch
and
Qua
lity;
NH
LB
I,N
atio
nalH
eart
,Lun
gan
dB
lood
Inst
itute
;O
MC
,Olm
sted
Med
ical
Cen
ter.
124 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
576 physicians and 52 other clinicians (nurse prac-titioners and physician assistants), and each practiceincluded one or more other practice study leaders,usually a practice nurse, social worker, or medicalassistant.
While no specific evaluation activities were de-signed to rigorously test the value of any of thestrategies, formal and informal evaluation strategieswere used across all studies. Following centraltraining, each attendee was asked to use a Likertscale to answer questions about the perceived valueof the training, timing of the training, specific con-tent related to study procedures, and informationon the practice change. At the close of each of thestudies, qualitative data were collected using semi-structured interviews with the practice leaders and,in some cases, one or more other members of thepractice teams. The interview included questionsabout the support provided by the central site;sessions to help solve implementation issues in thelocal practice; time constraints; which, if any, partsof the study intervention were now a “routine” partof the practice; and which elements would be con-tinued and which elements were not feasible. Theinterviews also included open-ended questions al-lowing the practice leaders to make any other com-ments or suggestions they desired. Data from theinterviews were analyzed by members of the re-search group who were familiar with qualitativeresearch. Informal data sources included anecdotesand comments during case reviews, and calls by theliaison and principal investigator with the site lead-ers during the course of the studies.
ResultsWe report 8 general strategies (Table 2) that ourteam identified as useful for maintaining engage-ment in PBRN studies of up to 5 years’ duration.Each strategy is designed to help prevent or over-come practice study fatigue and disengagement.Table 2 expands on the characteristics of each ofthe strategies beyond the comments presentedhere.
1. Centralized TrainingGarnering sufficient time from busy practice stafffor education about study details is difficult. Tryingto train an entire practice about all the details ofany study is unrealistic. To overcome these prob-lems, we identified 2 study champions or site lead-
ers at each practice: one a physician and the other amember of the nursing staff. These 2 people be-came the study “leads” and attended the centralizedtraining.
Centralized training was usually done over aweekend in conjunction with another primary caremeeting or at a “vacation” site to enhance interestin attendance. The sessions were designed to intro-duce study background and need, study proceduressuch as enrollment and informed consent, and thepractice change or intervention; the format allowedinteraction among site leaders and between siteleaders and the central staff of the study. Fundingfor the central training was included in the budgetfor each study.
Training sessions were led by different membersof the central team to allow the site champions tomeet everyone on the central team. While the first3 to 4 hours were primarily didactic, the rest of thesessions were interactive, practicing informed con-sent or use of the intervention tools.
When appropriate, randomization assignmentswere announced at the training session so thatsupport could be provided to the “usual care” orcontrol group, explaining their important role andreminding them that they will move to the inter-vention arm within 18 months. The control prac-tices often realize quickly that they have a smallerinitial study burden and will get the benefits of anysuccessful interventions after the study interventionis complete.
At the close of the training session, each site wasprovided with a condensed slide set prepared forthe site leaders to use to educate the rest of theirpractice members. Asking the site leaders to leadthe presentation ensured that they understood theproject sufficiently well to present the project totheir colleagues and supported their identificationas the site study leaders and champions.
All materials presented during the central train-ing sessions were provided to each site in a binderthat included all slides and handouts from the cen-tral training. The materials supported study re-freshers between practices and central study teammembers on an as-needed basis.
Immediately after its completion, the centralizedtraining received reviews of “very good” to “excel-lent” for meeting expectations, understanding ofthe intervention, and clear, readable slides. Whenpresenting to their local practices, the practiceleaders repeatedly commented that they were able
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 125
Table 2. Summary of Elements of the Eight Methods
Method Main Elements Subcategories
Centralizedtraining
1. Select an attractive city a. Link to another meetingb. Choose a “vacation” city
2. Brief and focused presentations3. All central team members make presentations4. Introduce all site champion teams and
announce randomizationa. Celebrate with both groupsb. Extra support for usual care initial disappointment
5. Introduce site champions to the central siteliaison
6. Introduce the topic of the protocol and reasonsfor the study
7. Provide detailed information in the slides andhandbooks
a. Continually refer to the handbook when championsreturn home
b. Use the handbook to reinvigorate, remind, andrefresh
8. Introduce the intervention to appropriate sites a. Use practical examples based on patient careb. Let the group brainstorm anticipated problems and
solutionsc. Introduce the concept of implementation flexibility
9. Provide time for each set of site champions todiscuss their local implementation strategieswhile central team members are present toanswer questions.
10. Take and share pictures of site champions andcentral teams
a. It may be harder to ignore calls from someonewhose picture is right over the phone
b. Helps central team to remember more detailsabout each champion
11. Provide time to complete human subjectstraining with central team mentors
a. Reinforce that support is readily availableb. Helps when the champions may have to repeat
training in 2 to 3 years12. Provide concise slide set for site champions to
use to train practices when they go home13. Use humor and interactive, supportive sessions
to model behavior for interactions for the restof the study period
Liaisons 1. Assign to specific practices for the entire study2. Serve as primary contact for practices to the
central team3. Include in all study meetings4. Become an early warning system for practice
problems and “voltage drop”FAQs 1. Develop and use a consistent format with
study logo at top2. Send to sites on a consistent day of the week3. Share by fax or E-mail as sites request4. Use for continuing education, maintaining
same style as central training5. Reinforce complex or apparently confusing
steps in implementation6. Recognize accomplishments such as enrollment
numbers7. Share local sites’ stories a. Successes
b. New strategies or tools developed by sitesc. Occurrence and burden of natural disastersi. Floods, earthquakes, fires, hurricanes, tornadoes
ii. Share interpractice support given8. Use humor and cartoons to illustrate issues
and celebrations
Continued
126 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
Table 2. Continued
Method Main Elements Subcategories
9. Develop and use mini crossword puzzles orother study-related “contests”
10. Share case reviews and questions11. Celebrate holidays
Incentives 1. Foster recognition and acknowledgement a. Press releases for local mediab. Framed certificates of participation for display in
the local office2. Set achievable goals and recognize them when
they are met3. Provide small but fun rewards, eg, boxes of
microwave popcorn4. Use variety to continue encouraging
participation5. Increase the frequency at lulls in enrollment or
other dips in activity6. Provide larger rewards for major achievements
like meeting total enrollment goalsa. Pizza partyb. Ice cream social
7. Provide personalized thank you from the PIfor special recognition
Case reviews 1. Assume sites have important insight to offerand expect to learn from sites
2. Highlight positive aspects of care3. Ask what could be done to make care even
better4. Relate the care questions to the intervention5. Allow site attendees to teach each other6. Present alternatives from other sites or the PI’s
experience or expertise7. Guide problem solving8. End by setting up the next review and perhaps
follow up on the cases presentedAppreciation 1. Respect is mainstay
2. Acknowledge all members of the practice3. Praise the work of the site champions during
case reviews and in FAQs4. Schedule interactions at the convenience of the
sites5. Consider using emails for some interactions—
especially PI to site physician champions6. Give site champions study shirts or mugs7. Give modest holiday gifts for all practice
membersa. Candy or fruit for winter holidaysb. Cookies or gift card for an ice cream party in
summerIRB support 1. Offer to do initial submission
2. Provide support for all IRB interactions a. Serious adverse events reportingb. Annual reports
3. Copy and mail new consent forms requiredannually
4. Offer to provide telephone support for all site-required IRB interactions
5. As requested, teach sites to do its own IRBsubmissions with oversight and support
FAQ, frequently asked question; IRB, institutional review board; PI, primary investigator.
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 127
to do this only because they had attended the cen-tralized training. Site leaders also commented onthe importance and value of interacting with otherPBRN practices; for many sites this was the firstPBRN study in which they had participated. One ofthe most common requests was to have a secondmeeting of practice leaders at the mid-point orclose to the end of the study. We interpreted theserequests as an endorsement of this type of training.The centralized training engages the attendees andhelps them gain an in-depth understanding of theresearch question and methods for the study.
2. Practice LiaisonsEach practice was assigned a central site study liai-son, with each liaison working with 4 to 10 prac-tices, depending on the study complexity and otherduties of the liaison. The liaison maintained weeklyto biweekly contact with the practice leaders andwas the first line of communication between thecentral site and the practice.
The relationship between the liaisons and thepractice leaders was initiated at the centralizedtraining and was instrumental to continued practiceengagement. The liaison worked to develop a long-term personal relationship with the practice to al-low easy and honest communication between thepractices and the central study site. The liaison wasusually the first to notice signs of slowed enroll-ment or failure to use the intervention suggested bydelays in scheduling or completing regular liaisonsite contact. By brainstorming with the site leaders,the liaison, with the assistance of the study centralinvestigators, was able to find reasons for slowedparticipation, disengagement, or “voltage drop”and developed strategies to solve the identifiedproblems.
The importance and value of the site liaisons wasa recurrent theme in the exit interviews for each ofthe completed studies. In all interviews, the prac-tice leaders knew their liaisons by name, often com-mented that they kept her phone number immedi-ately available, and enjoyed having the long-termrelationship.
3. Frequently Asked Questions (FAQs)Frequently asked questions (FAQs) began as justthat—responses to a question asked by one site butwith answers shared with all sites. They evolvedinto a method to communicate weekly with siteswithout phone calls that could be disruptive to the
practice. The FAQs were sent to each participatingpractice on a specific day of the week, using aconsistent format with the study logo at the top ofeach FAQ. FAQ communications evolved fromfaxes to E-mails, which allowed the use of color.Site champions were asked to print the FAQs andpost them in an area where they could be seen by allstaff members. Examples of FAQs are shown inFigures 1 and 2.
FAQ topics varied from updates on study imple-mentation details to reminders of proper tech-niques for specific study procedures such as signinginformed consents or completing enrollment logs.Studies with both intervention and usual care armsoften required arm-specific FAQs at least part ofthe time, for example, when discussing interventionspecific topics. To maintain interest and preventthe FAQs from being seen as nagging, FAQs alsowere used to recognize and congratulate sites fortheir efforts or to celebrate national and faith-basedholidays, recognizing that sometimes the sitesneeded a break from our educational endeavors.
Acknowledgment that sites were aware of receiv-ing the FAQs is illustrated in a comment we re-ceived following a blizzard and power outage at thecentral site that prevented sending of the FAQ onthe assigned day: “We did not get our FAQ—areyou there and OK?” Evidence of reading theFAQ content came when we reported some ofthe study practices’ experiences with tornadoes,floods, earthquakes, and hurricanes. Followingreports of these occurrences in the weekly FAQs,other practices responded with offers of help andemergency supplies.
As we became more creative with our FAQgraphics and used more color, sites responded withrequests to use less color and simpler pictures toconserve color printer ink, demonstrating that thesites were printing out the FAQs. FAQs promptedcalls from practices asking the liaisons to clarifyimprovements requested on the FAQ, such as mak-ing sure the “witness” signs the required lines onthe informed consent. These anecdotal findingsconfirm that the FAQs generated interactions be-tween the central study staff and the practices.
During exit interviews for the 3 studies that havebeen completed, the practice leaders reported thatFAQs were read “almost every week and saved inthe notebook” provided by the central site for FAQcollection. The practice leaders further reportedthat both the “funny” FAQs and the serious ones
128 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
were useful and kept them thinking about the studyand their roles of “identifying and enrolling pa-tients” and “facilitating the intervention.” FAQsthat included case reviews generated discussionamong the physician leaders across practices. Ex-amples included an asthma case that generatedcomments from 50% of the 29 physician practiceleaders and a postpartum depression case that gen-erated responses from 30% of the physicians and70% of the nursing practice leaders.
4. IncentivesThere are many types of incentives that can be usedin PBRN studies. Using the FAQs to acknowledgesites that meet their individual monthly enrollmentgoals can be an incentive. Simple contests thatquery the site leaders about protocol details or evencurrent medical events and reward correct answerswith boxes of microwave popcorn or gift cards ofnominal value to local restaurants are easy to ac-complish. Most incentives are provided to the prac-
tice as a whole to reemphasize that it is the wholepractice and not just the champions who are par-ticipating in the study.
Response rates to the simple contests have beenhigh. In one contest all the practice leaders plusanother 5 practice members answered (110% re-sponse). When enrollment slows, incentives thatchallenge sites to reach their individual monthlyenrollment goals have resulted in increasing enroll-ments (13 of the 16 times such FAQs were sent).Incentives of recognition and praise— for example,an FAQ naming all the “winning” sites in an en-rollment challenge—were also well accepted. Theincentives were in addition to the site payments forstudy participation.
5. Case ReviewsCase reviews evolved from specific patient encoun-ters that prompted discussion in an FAQ as beingof general interest to all practice sites. Case reviewsallowed the central site to better understand how
Figure 1. Example of frequently asked questions.
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 129
Figure 2. Example of frequently asked questions.
130 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
the practice incorporated the intervention and toprovide the sites with an opportunity to ask anexpert about difficult cases. Case reviews were lim-ited to the intervention sites.
Each case review was completed by conferencephone call between the study’s principal investiga-tor (PI), the site’s liaison, and all the members atthe site who were available to attend. The case waspresented by the enrolled patient’s physician, resi-dent, or nurse. The presenters from the site chosetheir own presentation formats, although we devel-oped a written outline for anyone requesting it.Most cases required 3 to 5 minutes for the initialpresentation, with another 10 to 15 minutes fordiscussion. The questions that followed the presen-tation included how the study intervention did ordid not affect the patient’s care, what might be thenext steps in care, and other management issuespertinent to the specific patient, such as immuni-zation status or adherence issues. The participationof the central PI allowed the site practice membersto query about anything from how the investigatorwould have used the intervention tools to furtherdiagnostic testing or alternative therapeutic strate-gies.
Case reviews were not an instant success: manypractices were reluctant to participate in the firstcase reviews. However, almost every practice hadsomeone brave enough to have their patient carediscussed, and participation increased as the prac-tices learned that the case reviews were intended tobe learning sessions rather than judgmental ses-sions.
At the first case review it was often just the studyPI and the presenter interacting. But once the in-teraction was shown to be positive and truly inter-active, such as asking the presenter what they thinkis good and what could be improved in the care ofthe patient, others often joined the discussion. Thediscussions were intended to focus on the practiceor systems changes and tools introduced during thestudy. Asking for additional suggestions often re-vealed other aspects of patient care, such as clari-fying requirements for pneumococcal immuniza-tions in people with asthma or querying about theyearly influenza immunization for the pregnant andpostpartum women.
After the initial 1 or 2 case reviews the casereviews were well attended, especially in residencyprograms, and the practice staff asked questionsabout other complex cases. The PI, central site
staff, and practice staff usually learned at least onenew thing from these interactions.
One of the best indications that the case re-views were valued was the decreasing resistanceto scheduling future case reviews. It was commonat the end of the second or third case review tohave the site staff make recommendations for thedate, time, and patient for the next case. Thepractice’s queries regarding the care of nonen-rolled patients also suggests that the practicesvalued the time and collaborative attitude thatthese sessions exemplified.
6. AppreciationIn health care settings, expressions of appreciationmay be uncommon. Many members of the studypractices reported that they heard little praise butbore the brunt of many complaints. We appliedseveral methods for expressing our appreciation forthe site’s participation in the studies. At the centraltraining session, each site was provided with aframed certificate of appreciation for its participa-tion in the study. These plaques can be displayed inthe office waiting room and are often as simple as aframed page designed and printed at the centralsite. To facilitate acknowledgment of the work ofthe practice in their local community, we providedeach site with tailored press releases to explain thestudy and the local clinic’s role. These were de-signed for the practice leaders to forward to localnewspapers or other media outlets.
The study liaison for each site provided encour-agement, support, and praise during each interac-tion with the site leaders. All interactions with thesites ended with the liaison or investigator thankingthe practice leaders or other practice staff for theircontinued work.
Sites have repeatedly told us they displayed theirstudy participation certificates in their offices orfront lobbies. Certificates from previous studies arefrequently encountered during site visits. Five sitesfrom the postpartum depression study sent us cop-ies of newspaper articles that resulted from pressreleases. Accompanying the articles were the fol-lowing comments: “Our patients really liked seeingus in the newspaper” and “some of our subspecialtycolleagues were impressed with what we are do-ing.” The central site received many E-mails, cards,and notes from the practices, usually addressed tothe practice liaison, acknowledging and thankingthe central site for the small boxes of cookies or
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 131
candy and holiday greetings sent in appreciation ofthe practice’s work.
7. Institutional Review Board Submission SupportInstitutional review board (IRB) submission sup-port from the central study team was very impor-tant since many PBRN practices in our studies hadlimited experience with IRB submissions. The ideaof submitting the protocol application to the localIRB or sending a yearly report to the IRB of recordcan be overwhelming for a staff not accustomed topreparing such reports. Learning to complete IRBreports can take time away from other importantstudy activities such as enrolling patients or sup-porting an intervention. Having one person on thecentral team assigned to manage the IRB submis-sions freed the sites of this concern and assured thecentral team that there would be no lapse in theability to enroll patients.
A proxy outcome of the effectiveness of the IRBsupport provided to the practices was the numberof IRB refusals, queries, and delays that our studiesexperienced. In our submissions and yearly reportsto �55 IRBs for periods up to 5 years each (�200interactions), we have had 3 initial protocol approv-als delayed by 30 to 60 days, 2 yearly reports thatresulted in temporary holds on enrollment, andonly one IRB request for additional data on a pa-tient outcome. Considering the large number ofIRBs and the requirements for up to 5 yearsof approval from each of the IRBs, the number ofdelays and requests for additional informationhas been minimal, and we believe this was thedirect result of the support provided by the cen-tral study team during initial and yearly submis-sions of the reports.
8. Meeting the Academic Needs of Site PersonnelMany family physicians report the need for aca-demic stimulation in their regular practices. Forsome physicians, such as residents and physiciansseeking recertification, the requirements are morespecific.
For example, all family medicine residents arerequired to complete some type of scholarly activityduring their training. In 2 of our studies we havebeen able to provide the support and data for res-idents to complete research projects for this re-quirement that are complementary and parallel tothe main study.
All physicians are required to complete trainingfor maintenance of board certification throughtheir appropriate medical specialty board. We de-veloped a Maintenance of Certification program,sponsored by the American Academy of Pediatrics,based on one study’s intervention by expanding itsquality improvement focus. The cost of the pro-gram was minimal and provided the physician withMaintenance of Certification credit for 3 years.This program has been directly responsible foractivating 2 of the pediatricians who were not prac-tice leaders to incorporate the study interventioninto their daily practices and to add further modi-fications that they believe enhance the interventionin their practices.
One of our studies has been able to support thedoctoral dissertation of the practice’s nonphysicianleader. We have previously published other bene-fits of participation in PBRN studies as reported bythe practices.1
DiscussionWhile each of our individual strategies seems tohave had a favorable impact on re-energizing andsupporting continued activity in one or more of ourstudies, we have used the combined strategies in amulticomponent approach. Waning interest is apotential problem for many types of practice activ-ities and has been described most often in qualityimprovement activities.26–33 Over periods of 18 to60 months, as required by our studies, currentmedical practices are likely to be faced with manyexternal requirements for change, including theintroduction or update of electronic medical re-cords systems, payment restructuring, clinical sys-tems changes, and staff turnover. Designing andimplementing methods to maintain interest andenthusiasm for research requirements and practicechanges seemed to prevent some of the “voltagedrop” for the practices in the 4 studies discussedhere.
The importance of building relationships be-tween central team members and the practices byusing a practice facilitator, practice enhancementassistant, or practice enhancement and research co-ordinator has been reported by others.15,17,18,34,35
While we agree that the ability to send facilitatorsdirectly into the practices can be very helpful, inwidely dispersed practices, such as those in ournational studies, the contact must be continued
132 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
using distance communication formats such as thetelephone, E-mails, and faxes. We also have morerecently used video communication with free orlow-cost shared services such as Skype.
Our attention to the learning and supportiveenvironment was explicitly begun during our 2002spirometry study funded by the National Heart,Lung and Blood Institute, and it might be consid-ered similar to the concept of the formal “learningcollaborative” introduced by the Institute forHealthcare Improvement (IHI) in 2003. However,unlike the early design of IHI’s learning collabor-ative, we found it more feasible to have a singleface-to-face meeting and to work with each site ata distance after that.36,37 Our system perhaps fitsmore closely with the Collaborative NetworkedLearning first described in 1987 by Findley,38
which is designed to occur via electronic dialoguebetween learners and experts sharing a commonpurpose—in our case implementation of practicechanges and study-related activities.38 Our strate-gies of centralized training, regular contact, andinteractive discussion with the site leaders, as wellas the FAQs and case reviews, could be consideredelements of collaborative learning. We consciouslychose not to include some of the elements requiredby the IHI learning collaborative concept, such asthe large time and personnel commitments. TheIHI learning collaborative can require severalmonths to work out a solution for an identifiedproblem. Our approach with busy practices usedmuch shorter time frames to determine how best tointroduce practice tools and systems that had beenpreviously developed and tested. The focus was onflexibility in implementing the tools and systemchanges rather than developing those tools andsystems.39
Quality improvement initiatives also encountersimilar experiences with “voltage drop.” The re-sponse to declining participation in quality im-provement work initially was to declare that “whatgets measured gets done” and develop multiplemetrics to assess quality processes.40,41 Similargoals can be accomplished in research settings withaudit and feedback.42 While some of our studies,such as the Asthma Tools study, use specific prac-tice audit and feedback, we also consider the casereviews to be a form of assessing what is actuallybeing done in the practice followed by an interac-tive discussion between the practices and the studystaff.
Few other specific strategies can be found in themedical literature.26–33 The RE-AIM (Reach, Effi-cacy, Adoption, Implementation, and Maintenance)framework clearly denotes the importance of “main-tenance,” which might be equated with continuedparticipation in our research projects.43 Suggestionsfor supporting maintenance often include recom-mendations for continuing to support and em-power those doing the quality improvement pro-cesses but without methods of operationalizingthose recommendations. Our strategies incorpo-rated support and empowerment while focusingheavily on implementing and sustaining the ele-ments necessary to complete the studies.
The challenge of working with multiple IRBshas been discussed frequently.8 However, little hasbeen published about IRB continuing reviews andthe need to identify a central team member tooversee this work. With many IRBs required foreach study, the need for an organized system be-comes obvious. While some practices assisted withthe IRB submissions, all were happy to have centralsupport. Although this may be considered usualprocedure for any PBRN study, it removes some ofthe time demand from the site and allows greatertime for the core study activities.
Incentives and mini contests were used toheighten awareness of the need for ongoing patientenrollment into the studies and often as a simplefun break from everyday practice routines. Theresponse rate to the incentives and mini contestswas high and usually resulted in a note or picturewhen the incentive or prize was enjoyed by theentire practice.1,10,44
To support the need for health professionals toregularly complete continuing education require-ments,45 we worked to provide continuing medicaleducation credit for attendance at the in-personcentralized training sessions through the AmericanAcademy of Family Physicians. We most recentlyadded an opportunity for pediatricians to complete amodule required for maintenance of certification, andwe assisted family medicine residents in completingtheir required academic activities during their final 2years of training.46,47 Supporting continuing medi-cal education and academic activities has been fa-vorably received by residency directors, residents,and practice physicians involved in our PBRNstudies, and we believe that this has contributed tothe willingness of the physicians to sustain theirinvolvement with the studies.
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 133
ConclusionPBRN, translational, and implementation researchrequires unique methods to assure success, espe-cially as the studies become more complex and oflonger duration. Each practice is a unique researchpartner whose strengths should be highlighted andenhanced. The practices are often inexperiencedpartners who require special attention when iden-tifying and overcoming barriers to consistentlycompleting all the requirements of the study anddoing so over a sustained time period, especially themid-study lag that we have labeled a “voltage drop.”The strategies we have outlined here were successfulin sustaining a high level of practice involvement inour studies, which is essential for high-quality proj-ects producing valid results. We believe that PBRNresearch will continue to be extremely valuable intransforming medical practice, and the use of ourstrategies can contribute to the overall success ofthe PBRN concept.
References1. Yawn BP, Pace W, Dietrich A, et al. Practice benefit
from participating in a practice-based research net-work study of postpartum depression: a NationalResearch Network (NRN) report. J Am Board FamMed 2010;23:455–64.
2. Green LA, Hickner J. A short history of primary carepractice-based research networks: from concept toessential research laboratories. J Am Board Fam Med2006;19:1–10.
3. Woolf SH. The meaning of translational researchand why it matters. JAMA 2008;299:211–3.
4. Hayes H, Parchman ML, Howard R. A logic modelframework for evaluation and planning in a primarycare practice-based research network (PBRN). J AmBoard Fam Med 2011;24:576–82.
5. DeVoe JE, Gold R, Spofford M, et al. Developing anetwork of community health centers with a com-mon electronic health record: description of theSafety Net West Practice-based Research Network(SNW-PBRN). J Am Board Fam Med 2011;24:597–604.
6. DeVoe JE, Likumahuwa S, Eiff MP, et al. Lessonslearned and challenges ahead: report from theOCHIN SafetyNet West practice-based researchnetwork (PBRN). J Am Board Fam Med 2012;25:560–4.
7. Mold JW, Lipman PD, Durako SJ. Coordinatingcenters and multi-practice-based research network(PBRN) research. J Am Board Fam Med 2012;25:577–81.
8. Yawn BP, Graham DG, Bertram SL, et al. Practice-based research network studies and institutional re-
view boards: two new issues. J Am Board Fam Med2009;22:453–60.
9. Yawn BP, Bertram S, Wollan P. Introduction ofAsthma APGAR tools improve asthma managementin primary care practices. J Asthma Allergy 2008;1:1–10.
10. Pace WD, Fagnan LJ, West DR. The Agency forHealthcare Research and Quality (AHRQ) practice-based research network (PBRN) relationship: deliv-ering on an opportunity, challenges, and future di-rections. J Am Board Fam Med 2011;24:489–92.
11. Williams RL, Rhyne RL. No longer simply a prac-tice-based research network (PBRN) health im-provement networks. J Am Board Fam Med 2011;24:485–8.
12. Gilbert GH, Richman JS, Gordan VV, et al. Lessonslearned during the conduct of clinical studies in thedental PBRN. J Dent Educ 2011;75:453–65.
13. Graham DG, Spano MS, Stewart TV, Staton EW,Meers A, Pace WD. Strategies for planning andlaunching PBRN research studies: a project of theAcademy of Family Physicians National ResearchNetwork (AAFP NRN). J Am Board Fam Med 2007;20:220–8.
14. Fagnan LJ, Handley MA, Rollins N, Mold J. Voicesfrom left of the dial: reflections of practice-basedresearchers. J Am Board Fam Med 2010;23:442–51.
15. Nagykaldi Z, Mold JW, Aspy CB. Practice Facilita-tors: a review of the literature. Fam Med 2005;37:581–8.
16. Nagykaldi Z, Mold JW. The role of health informationtechnology in the translation of research into practice:an Oklahoma Physicians Resource/Research Network(OKPRN) study. J Am Board Fam Med 2007;20:188–95.
17. Hogg W, Baskerville N, Nykiforuk C, Mallen D.Improved preventive care in family practices withoutreach facilitation: understanding success and fail-ure. J Health Serv Res Policy 2002;7:195–201.
18. Frijling BD, Lobo CM, Hulscher ME, et al. Multi-faceted support to improve clinical decision makingin diabetes care: a randomized controlled trial ingeneral practice. Diabet Med 2002;19:836–42.
19. Dietrich AJ, O’Connor GT, Keller A, Carney PA,Levy D, Whaley FS. Cancer: improving early detec-tion and prevention. A community practice ran-domised trial. BMJ 1992;304:687–91.
20. Yawn BP, Enright PL, Lemanske RF, et al. Spirom-etry can be done in family physicians’ offices andalters clinical decisions in management of asthmaand COPD. Chest 2007;132:1162–8.
21. Yawn B, Dietrich A, Wollan P, et al. TRIPPD: apractice based network effectiveness study of post-partum depression screening and management. AnnFam Med 2012;10:320–9.
22. Yawn BP, Pace W, Wollan PC, et al. Concordanceof Edinburgh Postnatal Depression Scale (EPDS)and Patient Health Questionnaire (PHQ-9) to assess
134 JABFM January–February 2014 Vol. 27 No. 1 http://www.jabfm.org
increased risk of depression among postpartumwomen. J Am Board Fam Med 2009;22:483–91.
23. Yawn B, Madison S, Bertram S, et al. Automatedpatient and medication payment method for clinicaltrials. J Clin Trials 2012;4:1–9.
24. Yawn B, Dietrich A, Wollan P, et al. TRIPPD: Apractice-based network effectiveness study of post-partum depression screening and management. Annof Fam Med. 2012;10(4).
25. Yawn B, Bertram S, Kurland M, et al. Protocol forthe asthma tools study: a pragmatic practice-basedresearch network trial. Pragmat Observ Res 2013;4:1–12.
26. Hughes RG. Chapter 44. Tools and strategies forquality improvement and patient safety. In: Patientsafety and quality an evidence-based handbook fornurses. Rockville, MD: Agency for Healthcare Re-search and Quality; 2008. Available from: http://www.ahrq.gov/professionals/clinicians-providers/resources/nursing/resources/nurseshdbk/nurseshdbk.pdf. Accessed November 11, 2013.
27. Bryant D. Maintaining your quality improvementachievements. May 7, 2009. Available from: http://ezinearticles.com/?Maintaining-Your-Quality-Improvement-Achievements&id�2312493. Ac-cessed November 11, 2013.
28. Rabinow P, Vilela M. Section 1. Achieving andmaintaining quality performance. In: Nagy K,Berkowitz B, Loewenstein M, editors. Maintainingquality performance. Lawrence: Work Group forCommunity Health and Development, University ofKansas; 2013. Available from: http://ctb.ku.edu/en/tablecontents/sub_section_main_1387.aspx. AccessedNovember 11, 2013.
29. Institute of Medicine. Crossing the quality chasm: anew heath system for the 21st century. Washington,DC: National Academies Press; 2001: 164–80.
30. Wallin L, Bostrom AM, Wikblad K, et al. Sustainabil-ity in changing clinical practice promotes evidence-based nursing care. J Adv Nurs 2003;41:509–18.
31. Shojania K, McDonald K, Wachter R, et al. Closingthe quality gap: a critical analysis of quality improve-ment strategies: volume 1—series overview andmethodology. Technical review, no. 9. AHRQ pub-lication no. 04-0051-1. Rockville, MD: Agency forHealthcare Research and Quality; 2004.
32. Horbar JD, Plsek PE, Leahy K; NIC/Q 2000.NIC/Q 2000: establishing habits for improvement inneonatal intensive care units. Pediatrics 2003;111(4Pt 2):e397–410.
33. Kim GR, Chen AR, Arceci RJ, et al. Error reductionin pediatric chemotherapy: computerized order en-try and failure modes and effects analysis. Arch Pe-diatr Adolesc Med 2006;160:495–8.
34. Cook R. Primary care. Facilitators: looking forward.Health Visit 1994;67:434–5.
35. Nagykaldi Z, Mold JW, Robinson A, Neibauer, FordA. Practice facilitators and practice-based researchnetworks. J Am Board Fam Med 2006;19:506–10.
36. Collaboratives. Cambridge, MA: Institute for Health-care Improvement. Available from: http://www.ihi.org/offerings/MembershipsNetworks/collaboratives/Pages/default.aspx. Accessed November 11, 2013.
37. Wikipedia. Collaborative learning. Available from: http://en.wikipedia.org/w/index.php?title�Collaborative_learning&printable�yes. Accessed November 11, 2013.
38. Quik WH, Wright N. Information sharing to trans-formation: antecedents of collaborative networkedlearning. World Acad Sci Engin Technol 2012;68:1477–86.
39. Roland M, Torgerson DJ. Understanding controlledtrials: what are pragmatic trials? BMJ 1998;316:285.
40. Williamson RM. Available from: http://www.swspitcrew.com/articles/What%20Gets%20Measured%201106.pdf. Accessed December 2, 2013.
41. Six Sigma. Community. Available from: http://www.isixsigma.com/community. Accessed July 10,2013.
42. Ivers N, Jamtvedt G, Flottorp S, et al. Audit andfeedback: effects on professional practice and health-care outcomes. Cochrane Database Syst Rev 2012;(6):CD000259.
43. Glasgow RE, Vogt TM, Boles SM. Evaluating thepublic health impact of health promotion interven-tions: the RE-AIM framework. Am J Public Health1999;89:1322–7.
44. Gibson K, Szilagyi P, Swanger CM, et al. Physicianperspectives on incentives to participate in practice-based research: a greater Rochester practice-basedresearch network (GR-PBRN) study. J Am BoardFam Med 2010;23:452–4.
45. ABFM maintenance of certification for family med-icine. Lexington, KY: American Board of FamilyMedicine. Available from: https://www.theabfm.org/moc/index.aspx. Accessed July 10, 2013.
46. Maintenance of certification (MOC) four-part structure.Chapel Hill, NC: American Board of Pediatrics. Availablefrom: https://www.abp.org/abpwebsite/moc/aboutmoc/maintenanceofcertification(moc)four-partstructure.htm.Accessed July 10, 2013.
47. American College of Graduate Medical Education.Scholarly activity guidelines. Review Committee for Fam-ily Medicine. Available from: https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramResources/120_Family_Medicine_Scholarship_Guidelines.pdf. AccessedJuly 10, 2013.
doi: 10.3122/jabfm.2014.01.130026 Keeping PBRN Practices Engaged in Studies 135