EHP290408

http://ehp.sagepub.com

Professions Evaluation & the Health

DOI: 10.1177/0163278706290408 2006; 29; 302 Eval Health Prof

Louise A. Rohrbach, Rachel Grana, Steve Sussman and Thomas W. Valente Research to Real-World Settings

Type II Translation: Transporting Prevention Interventions From

http://ehp.sagepub.com/cgi/content/abstract/29/3/302 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

at:can be foundEvaluation & the Health Professions Additional services and information for

http://ehp.sagepub.com/cgi/alerts Email Alerts:

http://ehp.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://ehp.sagepub.com/cgi/content/refs/29/3/302SAGE Journals Online and HighWire Press platforms):

(this article cites 64 articles hosted on the Citations

distribution.© 2006 SAGE Publications. All rights reserved. Not for commercial use or unauthorized

by Juan Pardo on November 14, 2007 http://ehp.sagepub.comDownloaded from

http://ehp.sagepub.com/cgi/alerts

http://ehp.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://ehp.sagepub.com/cgi/content/refs/29/3/302


302

AUTHORS’ NOTE: The preparation of thisarticle was supported by grants from theNational Institute on Drug Abuse (Grantnumbers R01-DA-13814 and R01-DA-16090).

EVALUATION & THE HEALTH PROFESSIONS, Vol. 29 No. 3, September 2006 302-333DOI: 10.1177/0163278706290408© 2006 Sage Publications

This article summarizes research on Type IItranslation of prevention interventions aimedat enhancing the adoption of effectiveprograms and practices in communities. Theprimary goal of Type II translation is to insti-tutionalize evidence-based programs, prod-ucts, and services. First, the authors describetheoretical frameworks that are useful toguide Type II translation research. Second,research on prevention program implementa-tion, including fidelity of implementation andfactors that are associated with successfulprogram implementation, is summarized. Theauthors describe interventions designed toenhance the dissemination of preventiveinterventions in community and public healthsettings. Third, they describe strategies usedby prevention program developers who havetaken programs to scale. Fourth, they presenta case example of Project Towards No DrugAbuse (TND), an empirically validated highschool–based substance abuse preventionprogram. They describe ongoing research onthe dissemination of Project TND. Finally,they provide suggestions for future Type IItranslation research.

Keywords: translation; adoption; implemen-tation; dissemination; training;transportability; prevention;effectiveness; diffusion

TYPE II TRANSLATIONTransporting Prevention

Interventions From Research toReal-World Settings

LOUISE A. ROHRBACHRACHEL GRANA

STEVE SUSSMANTHOMAS W. VALENTE

University of Southern California




The National Institutes of Health (NIH) has defined two areas oftranslation research (Office of Behavioral and Social Sciences,

2005). The first, “Type I translation,” applies discoveries generatedthrough laboratory and preclinical research to the development andtesting of treatment and prevention approaches. In other words, thistype of research moves the basic sciences into a realm of social orpersonal relevance (Sussman, Valente, Pentz, Rohrbach, & Skara,2006). The second area of translation research, “Type II translation,”is aimed at enhancing the adoption of effective practices in the com-munity. The primary goal of Type II translation is to institutionalizeeffective programs, products, and services. In this special issue,Ames and McBride (2006) summarize the translation of biomedicalsciences to effective treatment and prevention interventions. In thisarticle, we address the adoption and institutionalization of effectivehealth promotion and disease prevention practices in real-worldsettings.

During the past two decades, the knowledge base in disease pre-vention has been rapidly expanding, resulting in a considerablenumber of evidence-based interventions and programs that arepoised to be disseminated widely. In school-based prevention, forexample, there are empirically validated programs that have beenshown to reduce the onset of drug abuse (Drug Strategies, 1999; Safeand Drug Free Schools Program, 2001), violent behaviors (DrugStrategies, 1998; Elliott, 1997), mental disorders (Greenberg,Domiltrovich, Graczyk, & Zins, 2000), and sexual risk behaviors(Jemmott, Jemmott, & Fong, 1992; Kirby, Barth, Leland, & Fetro,1991; Main et al., 1994). In the field of family-centered preventionscience, a number of interventions have been found to be efficaciousin reducing problem behaviors, enhancing competencies, andimproving interfamilial relationships (Spoth, Kavanagh, & Dishion,2002). Similarly, research on HIV prevention has shown the effec-tiveness of behavioral interventions for preventing HIV among anumber of high-risk populations, such as gay men, women, adoles-cents, injection drug users, and persons with sexually transmitteddiseases (Kelly et al., 2000). Although such programs have demon-strated efficacy when tested under “ideal” conditions, there hasbeen considerably less research focused on their “transportability”(Schoenwald & Hoagwood, 2001) to the real-world environments ofschools, clinics, communities, and other settings. Thus, the sciencerelated to implementing evidence-based programs and practices with

Rohrbach et al. / TYPE II TRANSLATION 303




fidelity and positive outcomes for target populations lags far behindthe science related to developing the programs (Fixsen, Naoom,Blase, Friendman, & Wallace, 2005). Furthermore, little attentionhas been devoted to dissemination research, or studies examining thebest methods for enhancing the widespread adoption and institution-alization of effective prevention programs.

This article summarizes research on Type II translation of pre-vention interventions. First, we describe theoretical frameworks thatare useful to guide Type II translation research. Second, we summa-rize research on factors that influence, and methods to enhance, thedissemination of preventive interventions in community and publichealth settings. Third, we present a case example of dissemination ofProject Towards No Drug Abuse (TND), an empirically validatedhigh school–based substance abuse prevention program (Sussman,Sun, McCuller, & Dent, 2003). Finally, we provide suggestions forfuture Type II translation research.

THEORETICAL FRAMEWORKS FORTRANSLATION RESEARCH

In the next section of the article, we describe several theoreticalframeworks that describe how evidence-based practices are diffused,how organizations make changes such as the adoption of an evidence-based program, and how individuals within organizations makebehavior changes. These theories are useful to understand theprocess of translation and to guide the development of interventionsto promote translation.

DIFFUSION OF INNOVATIONS THEORY

Research on the diffusion of innovations spans diverse fields,from agriculture to manufacturing to medicine and disease preven-tion (Rogers, 2003). Most studies have focused on identifying thetypes of organizations, communities, or individuals that do and donot adopt a particular innovation and the factors that influence theadoption or nonadoption (Schoenwald & Hoagwood, 2001).Diffusion of innovations theory has several components, the first ofwhich is that adoption occurs in stages (Rogers, 2003). In mostapplications, five stages are proposed: (a) gaining knowledge about

304 Evaluation & the Health Professions / September 2006




the innovation, (b) being persuaded to use the innovation, (c) decid-ing to try the innovation, (d) deciding to implement the innovation,and (e) confirming the decision, including integrating it into one’sroutine. When applied, these adoption stages are often referred to asdissemination, adoption (combining persuasion and decision),implementation, and maintenance or sustainability. In the empiricalliterature, the terms dissemination and diffusion often are used inter-changeably, although a purist would reserve dissemination for pur-posive attempts at behavior change and diffusion as the term todescribe the process. Some researchers do not study the entire diffu-sion process but instead focus on the implementation and utilizationof innovations.

A second tenet of diffusion theory that has been broadly appliedis the concept that innovations have perceived attributes that affecttheir rates of adoption. These attributes include relative advantage,compatibility, complexity, trialability, and observability. Studieshave shown that perceptions of these attributes are associated withadoption of evidence-based prevention programs (e.g., Hallfors &Godette, 2002). Other perceived attributes include cost and uncon-ventionality or novelty of the innovation. A third component is thatdiffusion takes time, often a long time. Researchers frequently fail torealize that diffusion is slow and that behavior change is a slow andsometimes discontinuous process.

Diffusion theory has been useful in guiding research on methods toenhance or accelerate diffusion of prevention programs (Valente, 2002).For example, the theory suggests that multiple intervention strategieswill be necessary, tailored to each stage in the diffusion process (e.g.,Feifer et al., 2006; Parcel et al., 1989). The theory also suggests whichfeatures of a program could be marketed to speed its adoption (e.g., it iseasy to use, it has been successful elsewhere).

ORGANIZATIONAL CHANGE THEORIES

Organizational change theories also may be useful heuristics forconceptualizing the translation of empirically validated programsand practices to real-world settings. Most organizational change the-ories posit that change occurs in stages (Bartholomew, Parcel, Kok,& Gottlieb, 2001). One example is the stage theory of organizationalchange, which describes multiple stages that organizations pass throughas they accomplish an institution-wide change such as adoption of an





innovation (Beyer & Trice, 1978). These stages range from identifyingthe problem or need for change, to searching for solutions, to choos-ing a course of action and implementing it, to institutionalizing thechange so that it becomes part of the routine workings of the orga-nization. Other theories suggest specific strategies for cultivatingorganizational change, such as forming a powerful guiding coalitionwithin the organization, developing one vision and empoweringmembers to act on the vision, planning for short-term wins, evaluat-ing the change efforts, and providing feedback to members abouttheir efforts (Kotter, 1998; Senge, 1990). An important tenet of thesetheories is that in hierarchical organizations, innovations oftenrequire change by different levels of employees (e.g., managers,implementers, etc.) at different stages of the innovation adoptionprocess (Bartholomew et al., 2001).

The utility of organizational change theories in guiding the dissem-ination of evidence-based prevention programs may vary by the settingin which the programs are implemented. With regard to school-basedprevention programs, there is some evidence that organizationalchanges occur in stages and different change strategies are appropriateat different stages (e.g., Goodman, Smith, Dawson, & Steckler, 1991;Goodman, Tenney, Smith, & Steckler, 1992). Furthermore, the keyactors involved in change have been found to differ from stageto stage (e.g., Huberman & Miles, 1984). However, empirical studiesof health care systems have shown that organizational change oftenoccurs in unpredictable and nonsequential ways (Denis, Hebert,Langley, Lozeau, & Trottier, 2002; Pettigrew, Woodman, & Cameron,2001). Overall, these findings suggest that there is no one correct strat-egy for promoting organizational change in all settings, hence organi-zations may need to experiment with different strategies andcombinations of strategies to get an innovation institutionalized.

INDIVIDUAL-LEVEL BEHAVIOR CHANGE THEORIES

Interventions to change the behaviors of individuals within theorganizations that are adopting evidence-based practices may beguided by several social-psychological theories that are appliedto individual health behaviors, such as social cognitive theory(Bandura, 1977), the theory of planned behavior (Ajzen, 1991), andthe stages of change model (Prochaska et al., 1994). Social cognitivetheory posits that one’s behavior is determined by one’s self-efficacy,





or the belief that one has the ability to perform the target behavior,and one’s outcome expectations, or his or her judgment of the likelyconsequences that the target behavior will produce. Modeling servesas a major vehicle for learning about a new behavior or innovation(Bandura, 1977). Parcel and colleagues (Brink et al., 1995; Parcelet al., 1989; Parcel et al., 1995) have described the application ofsocial cognitive theory in an intervention designed to enhance dis-semination of an evidence-based smoking prevention program inschools, especially the use of modeling to influence teachers’ self-efficacy, outcome expectations, and skills. The theory of plannedbehavior states that one’s behavior is predicted by one’s intention toperform the behavior, which is influenced by one’s perceived controlof performing the behavior and perceived social expectations for per-forming the behavior (Ajzen, 1991). The stages of change modelposits that behavior change occurs in stages that range from no moti-vation to change, to internalization of the new behavior (Prochaskaet al., 1994). An important contribution of the model is the idea oftailoring of behavior change interventions to include different methodsfor individuals in different stages of change.

These theories provide guideposts that are useful for translatinginterventions, designing behavior change strategies, and evaluatingprogress in increasing adoption of translated programs. For example,Feifer et al. (2006) described the application of organizational andsocial-psychological theories to an intervention they designed toimprove adherence to evidence-based clinical practice guidelines.They used a variety of strategies to influence physicians’ perceivedsocial expectations for adherence, increase physicians’ self-efficacyby providing feedback on performance, and tailor the intervention tothe physician’s level of motivation to change. In the design of inter-ventions, it is important that researchers ascertain where organiza-tions and individuals are in their stage of behavior change. Theadoption of translated programs may require changes in organiza-tional norms and in the structure of incentives for members of theorganization to implement new practices.

PROGRAM IMPLEMENTATION RESEARCH

In the past few years, several publications have highlighted thegap between our knowledge of effective health services and prevention





approaches and what is received by consumers (e.g., Institute ofMedicine, 2001; Ringwalt et al., 2002; U.S. Department of Health& Human Services, 1999). There is broad agreement that the imple-mentation of evidence-based programs and practices is a complexendeavor, one that is more complex than the programs, practices, ortechnologies that are the subject of implementation efforts (Fixsenet al., 2005). Furthermore, there have been numerous calls for moreapplied research to better understand how to improve the efficiencyand effectiveness of health program delivery in medical, community,and school settings. In the following section of this article, we sum-marize research on program implementation, including how it is hasbeen measured and what types of factors predict it.

FIDELITY OF IMPLEMENTATION

Prevention researchers have long recognized the importance ofmeasuring the process of program implementation to understand whyprograms succeed or fail and be able to attribute changes in the out-come variables to the intervention. Various methods have been usedto measure prevention program implementation. In general, there isagreement that fidelity of implementation is critical to implementa-tion success. Fidelity refers to the degree to which program providersimplement programs as intended by the developers (Dusenbury,Brannigan, Falco, & Hansen, 2003). The most commonly used mea-sures of fidelity are adherence to the program, dose (amount of theprogram delivered), quality of program delivery, and participant reac-tions or acceptance (Dane & Schneider, 1998). In addition to fidelity,some researchers have considered other dimensions when determiningimplementation success, such as “buy-in” of program providers and theextent to which the program reaches the target population (Bosworth,Gingiss, Potthoff, & Roberts-Gray, 1999; Glasgow, Lichtenstein, &Marcus, 2003). The twin concerns of fidelity and acceptability lead toone of the conundrums facing translation research: To make programsmore acceptable, adaptability is often built in, yet this adaptabilitymakes fidelity more difficult to achieve.

Research on a wide range of educational programs and mental healthinterventions has shown that when efficacious interventions aretranslated to use in real-world settings, users often modify them to suittheir needs or improve the fit of intervention with local conditions(Rogers, 2003). In contrast to the high levels of fidelity achieved in





efficacy trials of prevention interventions (e.g., Hansen, Graham,Wolkenstein, & Rohrbach, 1991), effectiveness trials have showntremendous variability in the amount and quality of implementationwhen programs are applied to real-world situations (e.g., Botvin,Baker, Dusenbury, Tortu, & Botvin, 1990; Rohrbach, Graham, &Hansen, 1993). In school settings, for example, teachers report elim-inating some of the key curriculum points, objectives, and/or modules(Botvin et al., 1990; Rohrbach et al., 1993; Tappe, Galer-Unti, &Bailey, 1995; Tortu & Botvin, 1989); being less likely to use the inter-active teaching methods that are essential to the program, such asrole-playing and small group exercises (Ennett et al., 2003; Tappeet al., 1995); and generally deviating from the program as written(Pentz et al., 1990; Ringwalt et al., 2003). Furthermore, there issome evidence to suggest that when programs are not implementedwith fidelity, they are less likely to be effective (Botvin et al., 1990;Resnicow, Cross, & Wynder, 1993; Rohrbach et al., 1993).

FACTORS ASSOCIATED WITH IMPLEMENTATION SUCCESS

Observational studies have shown that a combination of factorsis associated with, or predict success of, program implementation.Table 1 lists factors that are related to the implementation of inno-vations in school settings, which may be categorized as factorsrelated to the organizational context in which the program is imple-mented, characteristics of the program itself, factors related to theprogram provider, and adequate training and technical assistance(Mihalic & Irwin, 2003; Rohrbach, D’Onofrio, Backer, &Montgomery, 1996). Implementation is more likely to be successfulin organizations that have strong administrative leadership and sup-port for the program (Berends, Bodilly, & Kirby, 2002; Farrell,Meyer, Kung, & Sullivan, 2001; Fullan, 1991, 1992; Gager & Elias,1997; Kam, Greenberg, & Walls, 2003; Kegler, Steckler, Malek, &McLeroy, 1998; Louis & Miles, 1990; Petersilia, 1990; Rohrbachet al., 1993; Smith, McCormick, Steckler, & McLeroy, 1993), astructure that provides for power sharing and participatory decisionmaking (Bernd, 1992; Boyd, 1992; Stoll & Fink, 1996), communi-cation patterns that are open and clear (Fullan, 1992; Kegler et al.,1998; Palestini, 2000), and stability in terms of resources and per-sonnel (Fullan, 1992; Gottfredson & Gottfredson, 2002). Factorsrelated to the school culture that increase the likelihood that schools





will adopt innovations include a shared vision (i.e., values andbeliefs) and goals, a willingness to initiate change, and a positiveorganizational climate (i.e., higher levels of trust and collaborationamong teachers, administrators, and other staff; Fullan, 1992; Louis& Miles, 1990; Stoll & Fink, 1996). Schoenwald and colleagueshave found that similar organizational and service system factors,such as the intensity of administrative support for the program, orga-nizational structure, policies affecting personnel (e.g., “comp time”and salaries), and organizational culture, influenced implementationof multisystemic therapy by mental health clinicians in usual-caresettings (Henggeler, Schoenwald, Liao, Letourneau, & Edwards,2002; Schoenwald, Sheidow, Letourneau, & Liao, 2003).

With regard to the characteristics of innovative programs, school-based research indicates that teachers are more likely to implementinnovations when they are well specified, include attractive and user-friendly materials, require instructional methods that are familiarto the teacher, and have a perceived advantage over current prac-tices (Gingiss, Gottlieb, & Brink, 1994; Hallfors & Godette, 2002;Huberman & Miles, 1984; Pankratz, Hallfors, & Cho, 2002; Parcelet al., 1995; Perry, Murray, & Griffin, 1990; Rohrbach et al., 1993;Smith et al., 1993; Taggart, Bush, Zuckerman, & Theiss, 1990).

There is some evidence to suggest that several characteristics ofprogram providers are associated with variation in implementation offidelity of prevention programs. Implementation is more likely to besuccessful when program providers are motivated to implement, have apositive attitude toward the prevention program, are comfortable withthe program approach, and have strong self-efficacy to implement(Gingiss et al., 1994; Hunter, Elias, & Norris, 2001; McCormick,Steckler, & McLeroy, 1995; Parcel et al., 1995; Rohrbach et al., 1993;Taggart et al., 1990). Furthermore, providers with a nonauthoritarianteaching style, good overall teaching skills, gregariousness, and stronggroup leadership skills may achieve greater fidelity when implementinghighly interactive prevention programs, relative to providers withoutthese characteristics (Gingiss et al., 1994; Sobol et al., 1989; Tobler,2000; Young et al., 1990). The professional background and other char-acteristics of the provider may be moderating variables that interact withthe program or change the program effects (Glasgow et al., 2003).

A fourth category of factors that affect program implementationincludes the training and technical assistance that program providersreceive prior to and during implementation. For the most part, train-






ing involves face-to-face educational workshops that are conductedprior to prevention program implementation. Although use of theterm technical assistance varies in practice and in publications, oftenit refers to ongoing formal or informal training, which may be deliv-ered via face-to-face, telephone, and/or electronic contacts, andwhich generally involves providing follow-up consultation, support,and reinforcement to program providers during program delivery.Several studies have shown that providers are more likely to imple-ment prevention programs, and they may use them with greaterfidelity, when they receive preimplementation training (Connell,Turner, & Mason, 1985; Fagan & Mihalic, 2003; Flay et al., 1987;Gager & Elias, 1997; Gottfredson & Gottfredson, 2002; Hunter et al.,2001; Perry et al., 1990; Ross, Leupker, Nelson, Saavedra, & Hubbard,

TABLE 1Factors Associated With the Adoption and Implementation

of Innovations in Schools

Organizational levelPrincipalLeadershipCommitment to and support of innovationStructure and processesManagement (decentralized)Participatory decision makingCommunicationStability and adequacy of resources (personnel, etc.)School cultureShared vision and goalsInnovativeness (willingness to initiate change)Positive climate

Provider levelCommitment to trying innovationAttitudes toward innovationComfort with approach of innovationGood teaching skillsSelf-efficacy to implement innovation

Innovation levelWell-specified programProgram that is attractively packaged and easy to useProgram that utilizes teaching methods familiar to provider

TrainingPreimplementation training and technical assistance provided

NOTE: Adapted from Rohrbach, D’Onofrio, Backer, & Montgomery (1996).




1991). However, it is unclear how much training is required to improveimplementation. Although some program developers suggest thattechnical assistance may improve implementation fidelity (e.g.,Durlak, 1998; Gager & Elias, 1997; Gingiss, 1992; Parcel et al., 1991),little attention has been devoted to experimental tests of the effec-tiveness of technical assistance over and above preimplementationtraining.

In summary, research on implementation of effective preventionprograms suggests that implementation represents a complex inter-action between characteristics of the intervention itself, characteris-tics of the implementer, and various aspects of the organizationalcontext in which the program is implemented. Although we summa-rized findings mostly from research on program implementation inschool settings, the factors that inhibit or enhance implementation inschools are very similar to those in other prevention program set-tings, such as health care or community-based organizations (Grol &Grimshaw, 2003; Schoenwald & Hoagwood, 2001). While one maybe inclined to attribute failures in translation to deficiencies in atti-tudes, knowledge, training, and skills of the “frontline” staff that areresponsible for providing the program in any setting, it is importantto view the actions of those responsible for implementation withinthe broader organizational context (i.e., organizational culture, pro-cedures regarding chains of command, mission, etc.) in which theprogram providers work (Sanders, Turner, & Markie-Dadds, 2002).Often staff members who are expected to adopt new programs areunderstandably wary and cautious of new ideas and practices theyhave had little involvement in developing.

EFFORTS TO ENHANCE PROGRAM ADOPTION AND IMPLEMENTATION

Although research has increased our understanding of barriersand facilitators of program implementation, there has been relativelyless application of those findings to interventions to enhance imple-mentation. In the following section, we summarize examples ofefforts to enhance prevention program adoption and implementation.

PROGRAM PROVIDER TRAINING

Most research focused on interventions to promote the broad dis-tribution, or dissemination, of evidence-based health services andprevention approaches has tested approaches to training program or






service providers (Grol & Grimshaw, 2003; Kelly et al., 2000; Parcelet al., 1989; Perry et al., 1990; Rohrbach et al., 1993; Society forPrevention Research [SPR], 2005; Steckler, Goodman, McLeroy,Davis, & Koch, 1992). Observation studies have suggested that themost effective approach to program provider training for mental,physical, and behavioral health education innovations is a workshop,followed by personalized consultation or technical assistance(Holloway & Neufeldt, 1995; Tornatzky, Fergus, Avellar, &Fairweather, 1980; Tortu & Botvin, 1989). Two studies of the rela-tive effectiveness of training workshops versus self-instruction viavideotape failed to show that training improved implementationfidelity (Basen-Engquist et al., 1994) or program outcomes(Cameron et al., 1999). In a randomized trial, Kelly et al. (2000)compared three approaches to training AIDS service organizationstaff to deliver evidence-based prevention programs. The first groupreceived the program materials only (i.e., self-instruction), the sec-ond group received the materials plus an intensive (2-day) work-shop, and the third group received the materials, workshop, andconsultation or technical assistance by telephone during a 6-monthperiod. Results indicated that the third approach produced the high-est rates of program adoption and implementation, suggesting thatworkshops and technical assistance may be critical to successfultranslation.

A recent review of the literature on training health care providersto deliver innovative services and practices showed that no singleapproach to training was superior to others, except that educationalinterventions that used interactive techniques such as discussion,performance feedback provided by peers, and interactive group plan-ning were more effective than noninteractive approaches (Grol &Grimshaw, 2003). Given the small number of studies that havedirectly compared different training models, it is difficult to describethe type of approach that is most effective in preparing frontline staffto implement evidence-based prevention programs. For the trainingof practitioners to deliver innovative health care services, it appearsthat a combination of strategies should be used (Grol & Grimshaw,2003). There is a need for more research on different trainingmodels, particularly because the cost of face-to-face training (e.g.,workshops and coaching) may be a significant barrier to dissemina-tion of evidence-based prevention programs (Cameron et al., 1999).




Evaluations of models for training health care providers to deliverinnovations have shown that practitioners’ behaviors explain only partof the variance in successful implementation. The organizational con-text (i.e., the culture and policies of the health care setting, administra-tive support for the innovation, etc.) has a strong influence on theadoption and implementation fidelity of the innovative practices byhealth care practitioners (Grol & Grimshaw, 2003). Despite the influ-ence of organizational factors on program implementation, there hasbeen little research on methods for increasing organizational capabilityto implement evidence-based programs. In one dissemination trial ofschool-based smoking prevention, the research team used an organiza-tional development technique known as “process consultation” to aidschool districts in their selection of an appropriate evidence-basedprogram and to promote program implementation and sustainability(Steckler et al., 1992). They found no effects on adoption or mainte-nance rates in districts that received, versus those that did not receive,the consultation. Elliott and colleagues (Elliott, 1997; Elliott &Mihalic, 2004) have implemented the “Blueprints ReplicationInitiative,” which identifies evidence-based programs for violence andsubstance abuse prevention among youth and promotes replication ofthe programs nationwide. Their intervention to promote program repli-cation involves working with schools and community settings via sitevisits and telephone contacts, in an attempt to build the organization’scapacity to implement the specific evidence-based program (Elliott &Mihalic, 2004). Although they did not test the efficacy of this approachin a randomized trial, they suggested that helping to build capacityshould be a routine part of dissemination initiatives, to preclude repli-cation failures. Other researchers have proposed using assessment toolsto determine the extent to which an organization is ready to implementan evidence-based prevention or treatment program (e.g., Bosworth etal., 1999; Lehman, Greener, & Simpson, 2002; Simpson, 2002).

Thus, although it appears that organizational change strategiessuch as capacity building may be necessary for prevention programimplementation to succeed, more research is needed to determinewhether and how such approaches are effective. For example, whatlevel of “readiness to change” is sufficient for a change agent toapproach an organization about potentially adopting a new program?Does the decision to adopt a new program need to emanate fromwithin the organization or can it come from the outside?





TAKING EMPIRICALLY VALIDATED PREVENTIONPROGRAMS “TO SCALE”

As was stated above, the science related to the transportability ofefficacious prevention programs to usual-care settings and the dissem-ination of these programs on a broad scale lags far behind the sciencerelated to developing the programs (Fixsen et al., 2005). Guided bytheory and the extant, though limited, information from disseminationtrials, some prevention program developers have begun to distribute ordisseminate their evidence-based programs broadly. Using a market-ing analogy, in the prevention literature these efforts have beenreferred to as “taking the program to scale” or “scaling up” (e.g., Elias,Zins, Graczyk, & Weissberg, 2003; Spoth et al., 2002).

Several publications about efforts to disseminate effective preven-tion programs in schools, health care organizations, and other com-munity settings (e.g., Elliott & Mihalic, 2004; Grol & Grimshaw,2003; Olds, 2002; Sanders et al., 2002; Spoth & Redmond, 2002)have suggested that a combination of strategies may be required foradoption and successful implementation. These strategies include (a)packaging program materials so that they are attractive and userfriendly, (b) establishing a diffusion system, (c) helping adopting sitesto build their organizational capacity for program implementation,(d) providing training and technical assistance to program implementers,and (e) establishing a system for collecting and reporting data onprogram delivery, including feedback to program providers and coor-dinators. Although these suggestions have not been validated empiri-cally, they are useful in guiding interventions to enhancedissemination. Each of these strategies is discussed below.

PACKAGING PROGRAM MATERIALS

An initial task for program developers is to transform writtenprogram materials into program implementation manuals that areeasy to follow and attractive to users. Manuals need to be explicitabout what program implementation requires, and they should bepresented in language and a format that is familiar to implementers.Similarly, developers need to make sure that materials that will bedistributed to program recipients are attractively packaged, appro-priate for all cultural groups that may use them, and reproducible ona large scale. Overall, program developers need to find the appropriate





balance between producing materials that are of high quality, yetavailable to users at a reasonable cost (Sanders et al., 2002).

ESTABLISHING A DIFFUSION SYSTEM

The program developer needs to establish a new, or select anexisting, system for diffusion of the evidence-based program. Thereare several critical ingredients of organized diffusion systems. Theseinclude the system’s ability to disseminate information and respondto inquiries about the program, distribute program materials on awide scale in a timely manner, provide training and technical assis-tance to program users, and link program users to developers toaddress issues related to program implementation, adaptation, andevaluation (Rohrbach et al., 1996).

Some prevention program developers have used existing distribu-tion systems, such as publishing companies (e.g., Project TowardsNo Tobacco Use [TNT], Sussman et al., 1993) and the Departmentof Education’s National Diffusion Network (U.S. Department ofEducation [USDoE], 1980), to disseminate their written program mate-rials. However, recognizing the need to provide training in addition toprogram materials, some program developers have established their owndistribution system in the form of a private business (e.g., Sanders et al.,2002) or service center located with a nonprofit organization such as auniversity (e.g., the National Center for Children, Families, andCommunities at the University of Colorado, established by Olds, 2002).Another model utilizes the existing Cooperative Education System(Halpert & Sharp, 1991) for materials distribution, technical assistance,and staff for program implementation (e.g., Spoth & Redmond, 2002).Probably one of the most common distribution techniques is appearanceat professional conferences, although no research has been conducted todetermine the utility of conference booths. Often it seems that a sign ofa program’s success is having a conference booth.

ENHANCING ORGANIZATIONAL CAPACITY

One of the lessons learned by the Blueprints for Violence PreventionInitiative (Elliott & Mihalic, 2004) and the National Center forChildren, Families, and Communities (Olds, 2002) is the importanceof working with adopting organizations to build their capacity toimplement empirically validated programs, prior to and during the





implementation process. An initial step is to make sure that theadopting organization has identified a clear need for the selectedprogram and a consensus that it represents a good “fit” with the orga-nization’s need, mission, and agenda (Olds, 2002; Roberts-Gray &Scheirer, 1988). Additional factors that appear to be necessary for suc-cessful implementation, and which may need to be built if they arelacking, include upfront and sustainable funding for the program,strong administrative support, a local program “champion” (i.e., a pri-mary promoter and supporter of the program), an adequate supply ofreceptive and skilled program implementers, and strong levels of com-munity support (i.e., support of key stakeholders and the target popu-lation; Elliott & Mihalic, 2004; Olds, 2002; Schoenwald &Hoagwood, 2001; SPR, 2005). Specifically, the capacity-buildingprocess may take up to 9 months; may require on-site visits, in addi-tion to telephone contacts, by a representative of the diffusion system;and should involve the development of a detailed program implemen-tation plan. The process should focus on strategically formingalliances with key administrative and management staff and other keystakeholders to ensure support from administrators and “buy-in” fromprogram implementers (Kam et al., 2003; Rohrbach et al., 1993;Sanders et al., 2002). Furthermore, sites should carefully selectprogram implementers and develop a plan for sustaining programimplementation when there is staff turnover. To build community sup-port for prevention, a formal planning process such as Communitiesthat Care (Hawkins, Catalano, & Arthur, 2002) might be used.

TRAINING PROGRAM PROVIDERS

Developing a cadre of qualified staff for providing training andtechnical assistance is an important element of “going to scale.” Thismay be a challenging effort for the diffusion system, particularly inthe early stages of scaling-up. It is likely that the staff members mostqualified to conduct training are those who assisted in the researchtrials of the specific program and hence have substantial programimplementation experience.1 As widespread dissemination of theprogram begins to take place, additional qualified individuals (whocan travel) need to be recruited and trained. Another model is to uti-lize an existing diffusion system that has a cadre of staff that can pro-vide training on a range of prevention interventions (e.g., Spoth,Greenberg, Bierman, & Redmond, 2004). As discussed above, workshops





are the most common method for training implementers of empiricallyvalidated prevention programs because they provide the opportunityfor interaction, practice of key program activities, and feedback.However, in a climate of declining resources for human service deliv-ery, program developers are exploring the use of less expensive train-ing self-instruction modalities, such as Web sites and videos (e.g.,Sanders et al., 2002). One innovative trial used satellite televisiontraining as a cost-effective method for delivering substance abuse pre-vention programming to community leaders (Pentz et al., 2004).

Another service that diffusion systems should be prepared to pro-vide is technical assistance during program implementation. Thenature of the assistance required will vary by the type of preventionintervention that the system is disseminating. For example, in inter-ventions that are delivered in school settings by mental health practi-tioners, ongoing supervision and support may be more important thanin interventions delivered by regular classroom teachers. For the mostpart, mental health interventions address more severe problem behav-iors among youth (e.g., aggression, disruptive behaviors, etc.) than dohealth education interventions, and they utilize intervention methods,such as individual and group counseling, which may require moreintensive provider training than classroom-based methods (Eliaset al., 2003). In all types of prevention programs, technical assistancerequests may include peer support, coaching, consultation, and men-toring; assistance in process and outcome evaluation; troubleshootingimplementation problems; and ongoing efforts to help the organiza-tion build capacity for program sustainability.

CONDUCTING AND REPORTING PROGRAM EVALUATION

Finally, an important element of disseminating empirically vali-dated prevention programs is collecting and reporting data on imple-mentation fidelity and program outcomes. Routine reporting of thesedata can guide ongoing efforts to improve the quality of programimplementation (Olds, 2002; SPR, 2005) and can foster sustainabil-ity of the program (Elliott & Mihalic, 2004). One approach to thisfunction is to have the diffusion system to develop a centralizedinformation system that gathers and reports information for all localsites (e.g., Olds, 2002). However, such a system is costly and mayrequire additional fund-raising for the dissemination effort. Anotherapproach is to provide technical assistance, such as advising on





evaluation approaches and making program-specific evaluationinstruments available, to enable local sites to collect their own evalu-ation data, as has been done by the Substance Abuse and MentalHealth Services [SAMHSA] Model Programs initiative (Atanda,Podrasky-Mattia, & Benton, 2005; Atkinson, Wilson, & Avula, 2005;SAMHSA, 2005; Wilson, Atanda, Atkinson, & Mulvey, 2005).

Failure to implement each one of these steps and services threatenssuccessful translation of evidence-based programs. Poor materialstranslation, inadequate distribution, poorly trained or uncommittedstaff, lack of technical assistance, and less-than-adequate help withongoing monitoring and evaluation are all threats that potentiallydegrade the effectiveness of evidence-based programs. Moreover, thetime lines involved in achieving all of these steps mean that staff sub-stantial turnover can occur and organizational cultures and climatescan change so much that translators often feel off balance.

CASE EXAMPLE: TAKING PROJECT TOWARDSNO DRUG ABUSE (PROJECT TND) TO SCALE

In the next section of this article, we describe the dissemination ofProject TND, an evidence-based substance abuse prevention program,including our process for taking it to scale and our current research onits dissemination.

Project TND is an effective drug abuse prevention program designedfor high school–aged populations. To date, the program has been iden-tified as a “model” or “exemplary” program by the SAMHSA (2005),Blueprints for Violence Prevention Replication Initiative (Sussman,Rohrbach, & Mihalic, 2004), Health Canada, Office of Juvenile Justice,and numerous other agencies. The program has been evaluated in threerandomized experimental trials and has demonstrated an impact on theuse of alcohol, tobacco, marijuana, and hard drugs at 1-year follow-up(Sussman, Dent, & Stacy, 2002). Program effects have been achieved inalternative high schools, where students are generally at higher risk forsubstance abuse, and regular high schools (Dent, Sussman, & Stacy,2001; Sussman, Dent, Simon, Stacy, & Craig, 1998).

We began disseminating Project TND in 2001, 8 years after thestart of the first experimental trial. Two events took place in 2001that provided the impetus for us to begin taking the program to scale.First, SAMHSA identified Project TND as one of its “model”





school-based programs for substance abuse prevention (SAMHSA,2005). The second event was the passage of the No Child Left BehindAct of 2001 (USDoE, 2005). Title IV of that act specified that schoolsreceiving Safe and Drug Free Schools and Communities funds mustuse them to support programs based on “scientifically based research”that provides evidence that the program reduces violence and substanceuse. The confluence of these events created demand for Project TND,and we began receiving requests for program materials and training.

PROJECT TND DIFFUSION SYSTEM

We carefully weighed the advantages and disadvantages of differ-ent types of diffusion systems for the program. Although ourresearch institute at the University of Southern California (USC) didnot have an existing infrastructure for large-scale production anddistribution of program materials, we wanted to maintain control ofthe distribution process, rather than turn it over to a partner organi-zation. This would enable us to collect information about the sitesthat adopted the program and conduct research on the disseminationprocess. Prior to developing a system for Project TND, Dr. Sussmandisseminated an evidence-based model program that he had devel-oped and evaluated, Project Towards No Tobacco Use (ProjectTNT) (Sussman et al., 1998). For Project TNT, he established a con-tract between USC and a private nonprofit organization devoted totraining and the distribution of educational materials, ETRAssociates. ETR published and distributed Project TNT; however,teacher training, when requested, was provided by program special-ists who had worked on the research trials. Dr. Sussman created arestricted account at USC in which funds from the sale of ProjectTNT and other tobacco and drug abuse prevention and/or cessationprogramming could be deposited. Although the transfer of distribu-tion rights to ETR Associates led to aggressive promotion of theprogram, Dr. Sussman could not track purchasers because of propri-etary and confidentiality rights of the company. Therefore, he initi-ated a new process for Project TND whereby he retained thedistribution rights of the program and utilized outside vendors toreproduce the program materials. The monies generated from salesof Project TNT permitted the base on which to begin reproduction ofProject TND materials. The protocol worked, and sales were generated.Later, Dr. Rohrbach joined Dr. Sussman in this effort, contributing to





revision of Project TND materials, refinement of the training model,and preparation and supervision of trainers, and being the drivingforce behind creation of a project Web site (http://tnd.usc.edu).

The Project TND diffusion system comprised a small staff of full-time university employees, who respond to requests for information,process invoices for program materials, and coordinate the schedulingof training, as well as a cadre of health education specialists who con-duct implementer training workshops on a part-time consulting basis.Our system is different from the university-based center for dissemi-nating the Nurse-Family Partnership program (Olds, 2002), in that theentire cost of the program materials and implementer training that weprovide is paid by the adopting sites. We also offer other programdevelopment and evaluation services on an as-needed consultant basis,negotiating the cost with the adopting organization that requests theservices. In contrast, the Nurse-Family Partnership program offers arange of services to help adopting organizations build their capacityfor implementation (e.g., readiness training, implementer training,consultation, and evaluation), the cost of which is supported, in part,by the university-based distribution center.

PROJECT TND MATERIALS

We utilized outside vendors to reproduce our Project TND mate-rials, which include a teacher’s manual, student workbooks, and avideo. Income from the program materials supports the salaries ofthree full-time project staff, who devote an average of 40% of theirtime to Project TND dissemination activities, and the proportion oftwo faculty members’ time that is devoted to overseeing the dissem-ination effort (i.e., Drs. Sussman and Rohrbach, approximately 20%for each). The income also provides support for other substanceabuse prevention research activities of those investigators. A portionof the income from the program materials (17%) goes to USC,which, in turn, supports the indirect costs of the dissemination activ-ities (e.g., office space, administrative oversight, etc.).

One of the initial steps of going to scale involved modifying theProject TND Teacher’s Manual for use by teachers and other imple-menters. The modifications did not change the essential elements ofthe program lessons; rather, they involved changes in language andformat that made the manual easier to follow. This turned out to be alabor-intensive effort; however, we were able to draw on the expertise





of several health educators who had taught the program in the efficacytrials. In addition, we incorporated suggestions for improving themanual format that were solicited from a group of regular classroomteachers who had implemented the program in the Project TNDeffectiveness trial (Rohrbach, Dent, Skara, & Sussman, 2006; Skara,Rohrbach, Sun, & Sussman, in press).

IMPLEMENTER TRAINING

Some evidence-based school drug abuse prevention programsrequire that potential program users attend training as a condition forprogram purchase and adoption, such as the Life Skills Trainingprogram developed by Botvin and colleagues (Botvin, Baker,Dusenbury, Botvin, & Diaz, 1995) and distributed by Princeton HealthPress, and Project ALERT, developed by Ellickson and colleagues(Ellickson & Bell, 1990) and distributed by the Best Foundation for aDrug Free Tomorrow. We decided to make Project TND materialsavailable with the strong recommendation, but not the requirement,that adopters also obtain certified training for their program imple-menters. Our primary rationale for this decision was that we wanted tomake the program available to schools and other local sites at a rea-sonable cost. We knew that many of the organizations that might useProject TND have limited resources, and the cost of a required train-ing would make program adoption prohibitive, especially for manyschool districts. At present, we are interviewing organizations thathave adopted Project TND to find out how they learned about theprogram, why they chose to adopt it, how they have used it (i.e., howoften it is implemented, to whom, by whom, etc.), and how they payfor program implementation and training. Based on the data we havecollected thus far, it appears that more than 75% of school organiza-tions support training through their Safe and Drug Free Schools andCommunities funds (USDoE, 2005), and more than 75% of community-based organizations use various sources of funds (e.g., state block grants,competitive grants, etc.). These data may change over time.

When we first began going to scale, we were able to meet thedemand for training by sending one of our staff health educators thathad taught Project TND. However, as the demand for training grew,we recruited several additional trainers who conduct training work-shops at the location of the adopting organization. The trainers pro-vide technical assistance to adopting sites when it is requested.






We encourage local sites that adopt Project TND to conduct theirown program evaluation. Recently, we launched a project Web site thatprovides detailed information about the program, including versionsof the student surveys that we adapted for use in program evalua-tion.2 Furthermore, we encourage users to monitor program imple-mentation, and we have published a modification of the instrumentwe have used to assess implementation fidelity (Rohrbach et al.,2006; Sussman et al., 2004).

PROJECT TND DISSEMINATION RESEARCH

At present, we are in the process of conducting a study of ProjectTND dissemination. In 2003, we received a grant from the NationalInstitute on Drug Abuse to conduct a randomized trial that comparesthe relative effectiveness of a standard face-to-face training work-shop (only) versus a comprehensive implementation support model(i.e., training workshop plus Web-based support, coaching, and tech-nical assistance for teachers). Teachers in schools assigned to thecomprehensive-implementation-support condition first participate ina 1-day training workshop. Then, the trainer (one of our programspecialists) visits the teachers’ classroom twice during the programdelivery period to observe delivery of key lessons and provide feed-back and coaching. The trainer provides technical assistance toteachers as requested. The teachers are given access to a “membersonly” section of the Project TND Web site, which contains down-loadable teaching tips for each lesson, supplementary informationthat may aid them in teaching the lessons (e.g., fact sheets), and rel-evant scientific articles. This section of the Web site also links usersto a discussion forum designed to address teachers’ questions aboutprogram implementation as they arise, with comments from thetrainers suggesting solutions to implementation problems. Finally,school and school district administrators in this condition receiveproactive technical assistance designed to promote program sustain-ability. Teachers in schools assigned to the standard-care trainingcondition receive the 1-day training workshop only.

We expect that the comprehensive support system will yieldhigher rates of implementation fidelity and program sustainability,and stronger effects on substance use outcomes. Schools in bothtraining conditions will be compared to those in a delayed-intervention





control condition on measures of students’ use of tobacco, alcohol,marijuana, and other drugs.

In summary, although it was challenging to set up our university-basedsystem for disseminating Project TND, the structure of our systemhas allowed financial support for ongoing prevention research activ-ities. Furthermore, our approach has allowed us to make the programaffordable for a substantial number of schools and community orga-nizations across the country. Finally, maintaining control of infor-mation about the organizations that have adopted the program willprovide us the opportunity to conduct research on all phases of thediffusion process.

SUGGESTIONS FOR FUTURE RESEARCH

The translation research findings discussed in this article haveseveral implications for future research. First, there is a need formore research on the ways in which prevention programs getadapted as they diffuse and how those adaptations affect programoutcomes. Some social scientists have argued that program adapta-tion is inevitable and even desirable to meet the needs of differentprogram providers and recipients (e.g., Backer, 2001), whereasothers suggest that any departure from an exact replication of anevidence-based program will most likely decrease its effectiveness(e.g., Elliott & Mihalic, 2004). Schoenwald and Hoagwood (2001)argued that few community-based settings can implement all fea-tures of evidence-based interventions, thus adaptations to the inter-vention protocols and the practice setting need to occur so thateffective programs can be delivered in real-world settings. To date,there has been little attention paid to these transportability issues onthe part of program developers, and there have been few studies onthe impact of local adaptations on program outcomes. More researchis needed to determine whether and under what conditions adapta-tions might enhance dissemination and outcomes, and under whatconditions they might result in a loss of program effectiveness(Dusenbury et al., 2003; Schoenwald & Hoagwood, 2001).

Second, there is a need for more experimental trials of models fortraining providers to implement evidence-based programs with fidelity.Although a few studies reported in the literature have compared the





relative effectiveness of various training and technical assistancemodels (e.g., Basen-Engquist et al., 1994; Cameron et al., 1999;Kelly et al., 2000), more research is needed to identify the type oftraining and technical assistance package most effective for the suc-cessful transfer of evidence-based interventions to practice and com-munity settings. In particular, training models that might reach greaternumbers of program providers at a lower cost than on-site workshops,with the use of technology such as videotapes or distance learning,should be explored.

Third, one of the lessons suggested by evidence-based programdevelopers currently taking their programs to scale is that adoptingorganizations are more likely to implement programs successfullywhen they build their organizational capacity to deliver the programseffectively (e.g., Elliott & Mihalic, 2004; Olds, 2002). This lesson isconsistent with research that has shown that successful implementa-tion of innovative programs in schools and other settings is associ-ated with organizational factors such as strong employee morale,high levels of participation in decision making, active support for theprogram from managers and administrators, and an organizationalculture that is conducive to change (Rohrbach et al., 1996). To date,few dissemination intervention studies have attempted to promoteorganizational readiness, or create favorable conditions in organiza-tions for prevention program implementation. The prevention fieldcould benefit greatly from trials that compare the effectiveness ofvarious methods for building organizational capacity (SPR, 2005).As discussed above, organizational change theory might provide auseful conceptual framework for designing such interventions.

There are numerous challenges to conducting Type II translationresearch and/or dissemination research. First, it involves researchand service provision, yet the funding for these two sets of activitiesgenerally comes from different governmental agencies that often actwithout coordination (SPR, 2005). To meet the objectives of a dis-semination study, researchers may need to apply for funding to mul-tiple sources that are likely to have different grant cycles and criteria.Second, dissemination studies require multiple sites; therefore, theyare more expensive and take longer than most efficacy trials.Furthermore, these types of studies generally involve building col-laborations between researchers and community sites before theresearch may even begin (e.g., Alexander et al., 2003; Goering,




Butterill, Jacobson, & Sturtevant, 2003), which takes a considerableamount of time. Third, dissemination studies do not garner the sameprestige as efficacy studies, as they are often seen as less likely tostimulate new knowledge or provide new findings.

Recently, prevention researchers have made suggestions for over-coming these barriers to conducting translation research. One group ofsuggestions relates to funding mechanisms. The SPR (2005) proposedthat governmental agencies consider efforts to “braid” the fundingmade available for translation research and prevention service deliv-ery. This would involve coordination between service agencies (e.g.,SAMHSA) and NIH to integrate funding that supports an agendadesigned to increase the reach of effective programs (i.e., funds forservice delivery) and experimental trials related to the translation ofthose programs (i.e., funds for research). In addition, funding systemsshould take into account the greater amount of time required to con-duct translation research.

Finally, some prevention researchers (Glasgow et al., 2003;Green & Glasgow, 2006; Schoenwald & Hoagwood, 2001) havesuggested that it may be time to “rethink the efficacy-to-effectiveness”sequence of research currently used in prevention science, which isbased on the NIH phases of cancer control research first publishedin the 1980s (Greenwald & Cullen, 1985). Glasgow and colleagues(Glasgow, Vogt, & Boles, 1999; Glasgow et al., 2003) argued that itis highly unlikely that interventions that are successful in efficacystudies will do well in real-world applications because most haveunknown generalizability; that is, little is known about the represen-tativeness of the participants, settings, or implementers of most effi-cacious health promotion programs. They suggested that researcherspay increased attention to moderating factors in efficacy and effec-tiveness trials, such as types of settings, implementers, and targetaudience, in an effort to place a greater emphasis on the external valid-ity of prevention interventions. Schoenwald and Hoagwood (2001)suggested that a new type of research, transportability research, needsto be conducted after effectiveness studies and before disseminationstudies (Schoenwald & Hoagwood, 2001). Transportability studiesexamine “who will conduct the intervention in question, under whatcircumstances, and to what effect” and “which aspects of the inter-vention protocols and practice settings require modifications” beforebroad dissemination may occur.






NOTES

1. In some cases, however, these individuals may not be the best ones to conduct trainingsince they may be so familiar with the program that they are overly confident of its superiority.

2. The surveys are available at http://tnd.usc.edu/localeval.php.

REFERENCES

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and HumanDecision Process, 50, 179-211.

Alexander, J. A., Weiner, B. J., Metzger, M. E., Shortell, S. M., Bazzoli, G. J., Hasnain-Wynia, R.,et al. (2003). Sustainability of collaborative capacity in community health partnerships.Medical Care Research and Review, 60(4 Suppl. 1), 130-160.

Ames, S. L., & McBride, C. (2006). Translating genetics, cognitive science and other basicscience research findings into applications for prevention. Evaluation & the HealthProfessions, 29(3), 277-301.

Atanda, R., Podrasky-Mattia, L., & Benton, A. (2005). Developing a data collection system.Evaluation and Program Planning, 28(3), 335-339.

Atkinson, D. D., Wilson, M., & Avula, D. (2005). A participatory approach to building capac-ity of treatment programs to engage in evaluation. Evaluation and Program Planning,28(3), 329-334.

Backer, T. E. (2001). Finding the balance: Program fidelity in substance abuse prevention: Astate-of-the-art review. Rockville, MD: Substance Abuse and Mental Health ServicesAdministration, Center for Substance Abuse Prevention.

Bandura, A. (1977). Social learning theory. New York: General Learning Press.Bartholomew, L. K., Parcel, G. S., Kok, G., & Gottlieb, N. H. (2001). Intervention-mapping:

Designing theory and evidence-based health promotion programs. Mountain View, CA:Mayfield.

Basen-Engquist, K., O’Hara-Tompkins, N., Lovato, C. Y., Lewis, M. J., Parcel, G. S., &Gingiss, P. (1994). The effect of two types of teacher training on implementation of SmartChoices: A tobacco prevention curriculum. Journal of School Health, 64(8), 334-339.

Berends, M., Bodilly, S. J., & Kirby, S. N. (2002). Facing the challenges of whole-schoolreform. New American schools after a decade. Santa Monica, CA: RAND.

Bernd, M. (1992). Shared decision making requires effective instructional leadership. NASSPBulletin, 76(546), 64-69.

Beyer, J. M., & Trice, H. M. (1978). Implementing change: Alcoholism policies in work orga-nization. New York: Free Press.

Bosworth, K., Gingiss, P. M., Potthoff, S., & Roberts-Gray, C. (1999). A Bayesian model topredict the success of the implementation of health and education innovations in school-centered programs. Evaluation and Program Planning, 22(1), 1-11.

Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population.Journal of the American Medical Association, 273(14), 1106-1112.

Botvin, G. J., Baker, E., Dusenbury, L., Tortu, S., & Botvin, E. M. (1990). Preventing adoles-cent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3-yearstudy. Journal of Consulting and Clinical Psychology, 58(4), 437-446.

Boyd, V. (1992). School context: Bridge or barrier to change? (Office of EducationalResearch and Improvement Contract No. RP 1002003). Austin, TX: SouthwestEducational Development Laboratory.




Brink, S. G., Basen-Enquist, K. M., O’Hara-Tompkins, N. M., Parcel, G. S., Gottlieb, N. H.,& Lovato, C. Y. (1995). Diffusion of an effective tobacco prevention program. Part I:Evaluation of the dissemination phase. Health Education Research, 10, 283-295.

Cameron, R., Brown, K. S., Best, J. A., Pelkman, C. L., Madill, C. L., Manske, S. R., et al.(1999). Effectiveness of a social influences smoking prevention program as a function ofprovider type, training method, and school risk. American Journal of Public Health,89(12), 1827-1831.

Connell, D. B., Turner, R. R., & Mason, E. F. (1985). Summary of findings of the SchoolHealth Education Evaluation: Health promotion effectiveness, implementation, and costs.Journal of School Health, 55(8), 316-321.

Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary pre-vention: Are implementation effects out of control? Clinical Psychology Review, 8(1), 23-45.

Denis, J. L., Hebert, Y., Langley, A., Lozeau, D., & Trottier, L.-H. (2002). Explaining diffusionpatterns for complex health care innovations. Health Care Management Review, 27, 60-73.

Dent, C. W., Sussman, S., & Stacy, A. W. (2001). Project Towards No Drug Abuse:Generalizabilty to a general high school sample. Preventive Medicine, 32(6), 514-520.

Drug Strategies. (1998). Safe schools, safe students: A guide to violence prevention strategies.Washington, DC: Author.

Drug Strategies. (1999). Making the grade: A guide to school drug prevention programs.Washington, DC: Author.

Durlak, J. A. (1998). Why program implementation is important. Journal of Prevention andIntervention in the Community, 17, 5-18.

Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research onfidelity of implementation: Implications for drug abuse prevention in school settings.Health Education Research, 18(2), 237-256.

Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003). Implementation, sustain-ability, and scaling up of social-emotional and academic innovations in schools. SchoolPsychology Review, 32, 303-319.

Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudi-nal test. Science, 247, 1299-1305.

Elliott, D. S. (1997). Blueprints for violence prevention (Vols. 1-11). Boulder: University ofColorado, Institute of Behavioral Science Center for the Study and Prevention of Violence.

Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective preven-tion programs. Prevention Science, 5(1), 47-53.

Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A.,et al. (2003). A comparison of current practice in school-based substance use preventionprograms with meta-analysis findings. Prevention Science, 4(1), 1-14.

Fagan, A. A., & Mihalic, S. F. (2003). Strategies for enhancing the adoption of school-basedprevention programs: Lessons learned from the Blueprints for Violence PreventionReplications of the Life Skills Training Program. Journal of Community Psychology,31(3), 235-253.

Farrell, A. D., Meyer, A. L., Kung, E. M., & Sullivan, T. N. (2001). Development and evalua-tion of school-based violence prevention programs. Journal of Clinical Child Psychology,30(2), 207-220.

Feifer, C., Ornstein, S. M., Jenkins, R. G., Wessell, A., Corley, S. T., Nemeth, L. S., et al.(2006). The logic behind a multi-method intervention to improve adherence to clinicalpractice guidelines in a nationwide network of primary care practices. Evaluation & theHealth Professions, 29(1), 65-88.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friendman, R. M., & Wallace, F. (2005).Implementation research: A synthesis of the literature (FMHI Publication #231). Tampa:University of South Florida, Louis de la Parte Florida Mental Health Institute, the NationalImplementation Research Network.





Flay, B. R., Hansen, W. B., Johnson, C. A., Collins, L. M., Dent, C. W., Dwyer, K. M., et al.(1987). Implementation effectiveness trial of a social influences smoking preventionprogram using schools and television. Health Education Research, 2(4), 385-400.

Fullan, M. G. (1991). The new meaning of educational change. New York: Teachers CollegePress.

Fullan, M. G. (1992). Successful school improvement: The implementation perspective andbeyond. Philadelphia: Open University Press.

Gager, P. J., & Elias, M. J. (1997). Implementing prevention programs in high-risk environ-ments: Application of the resiliency paradigm. American Journal of Orthopsychiatry,67(3), 363-373.

Gingiss, P. L. (1992). Enhancing program implementation and maintenance through a multi-phase approach to peer-based staff development. Journal of School Health, 62(5), 161-166.

Gingiss, P. L., Gottlieb, N. H., & Brink, S. G. (1994). Increasing teacher receptivity toward useof tobacco prevention education programs. Journal of Drug Education, 24(2), 163-176.

Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003). Why don’t we see more translationof health promotion research to practice? Rethinking the efficacy-to-effectiveness transi-tion. American Journal of Public Health, 93(8), 1261-1267.

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact ofhealth promotion interventions: The RE-AIM framework. American Journal of PublicHealth, 89(9), 1323-1327.

Goering, P., Butterill, D., Jacobson, N., & Sturtevant, D. (2003). Linkage and exchange at theorganizational level: A model of collaboration between research and policy. Journal ofHealth Services Research and Policy, 8(4 Suppl. 2), 14-19.

Goodman, R. M., Tenney, M., Smith, D. W., & Steckler, A. (1992). The adoption process forhealth curriculum innovations in schools: A case study. Journal of Health Education,23(4), 215-220.

Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based preventionprograms: Results from a national survey. Journal of Research in Crime and Delinquency,39(1), 3-35.

Green, L. W., & Glasgow, R. E. (2006). Evaluating the relevance, generalization, and applic-ability of research: Issues in external validity and translation methodology. Evaluation andthe Health Professions, 29(1), 126-153.

Greenberg, M. T., Domiltrovich, C., Graczyk, P. A., & Zins, J. E. (2000). The study of imple-mentation of school-based prevention research: Theory, research and practice. Report tothe Center for Mental Health Services (SAHMSA). Available at www.prevention.psu.edu/

Greenwald, P., & Cullen, J. W. (1985). The new emphasis in cancer control. Journal of theNational Cancer Institute, 74, 543-551.

Grol, R., & Grimshaw, J. (2003). From best evidence to best practice: Effective implementa-tion of change in patients’ care. Lancet, 362, 1225-1230.

Hallfors, D., & Godette, D. (2002). Will the “principles of effectiveness” improve preventionpractice? Early findings from a diffusion study. Health Education Research: Theory andPractice, 17(4), 461-470.

Halpert, B. P., & Sharp, T. S. (1991). Utilizing cooperative extension services to meet ruralhealth needs. Journal of Rural Health, 7(1), 23-29.

Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Programintegrity as a moderator of prevention program effectiveness: Results for fifth gradestudents in the Adolescent Alcohol Prevention Trial. Journal of Studies on Alcohol, 52(6),568-579.

Hawkins, J. D., Catalano, R. F., & Arthur, M. W. (2002). Promoting science-based preventionin communities. Addictive Behaviors, 27(6), 951-976.

Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002).Transporting efficacious treatments to field settings: The link between supervisory practices





and therapist fidelity in MST programs. Journal of Clinical Child and AdolescentPsychology, 31(2), 155-167.

Holloway, E. L., & Neufeldt, S. A. (1995). Supervision: Its contributions to treatment efficacy.Journal of Consulting and Clinical Psychology, 63(2), 207-213.

Huberman, A. M., & Miles, M. B. (1984). Innovation up close: How school improvementworks. New York: Plenum.

Hunter, L., Elias, M. J., & Norris, J. (2001). School-based violence prevention: Challenges andlessons learned from an action research project. Journal of School Psychology, 39(2), 161-175.

Institute of Medicine—Committee on Quality of Health Care in America. (2001). Crossing thequality chasm: A new health system for the 21st century. Washington, DC: NationalAcademy Press.

Jemmott, J. B., Jemmott, L. S., & Fong, T. (1992). Reduction of HIV risk-associated sexualbehaviors among black male adolescents: Effects of an AIDS prevention intervention.American Journal of Public Health, 82(3), 372-374.

Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation qual-ity in school-based prevention using the PATHS curriculum. Prevention Science, 4(1), 55-63.

Kegler, M. C., Steckler, A., Malek, S. H., & McLeroy, K. (1998). A multiple case study ofimplementation in 10 local Project ASSIST coalitions in North Carolina. Health EducationResearch, 13(2), 225-238.

Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj, L. L., McAuliffe, T. L., Hackl, K. L.,et al. (2000). Bridging the gap between the science and service of HIV prevention:Transferring effective research-based HIV prevention interventions to community AIDSservice providers. American Journal of Public Health, 90(7), 1082-1088.

Kirby, D., Barth, R. P., Leland, N., & Fetro, J. V. (1991). Reducing the risk: Impact of a newcurriculum on sexual risk taking. Family Planning Perspectives, 23(6), 253-263.

Kotter, J. (1998). Winning at change. Leader to Leader, 10, 27-33. Retrieved January 16, 2002,from http://leadertoleader.org/leaderbooks/L2L/fall98/kotter.html

Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readi-ness for change. Journal of Substance Abuse Treatment, 22, 197-209.

Louis, K. S., & Miles, M. B. (1990). Improving the urban high school: What works and why.New York: Teachers College Press.

Main, D. S., Iverson, D. C., McGloin, J., Banspach, S. W., Collins, J. L., Rugg, D. L., et al.(1994). Preventing HIV infection among adolescents: Evaluation of a school-based edu-cation program. Preventive Medicine, 23(4), 409-417.

McCormick, L. K., Steckler, A., & McLeroy, K. R. (1995). Diffusion of innovations inschools: A study of adoption and implementation of school-based tobacco preventioncurricula. American Journal of Health Promotion, 9(3), 210-219.

Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Factors influencing the successful replication of model programs. YouthViolence and Juvenile Justice, 1(4), 307-329.

Office of Behavioral and Social Sciences, National Institutes of Health, Office of the Director.(2005). Strategic plan. Available at http://obssr.od.nih.gov/about/plan.html

Olds, D. L. (2002). Prenatal and infancy home visiting by nurses: From randomized trials tocommunity replication. Prevention Science, 3(3), 153-172.

Palestini, R. H. (2000). Ten steps to educational reform: Making change happen. Lanham,MD: Scarecrow.

Pankratz, M., Hallfors, D., & Cho, H. (2002). Measuring perceptions of innovation adoption:The diffusion of a federal drug prevention policy. Health Educational Research, 17(3),315-326.

Parcel, G., Eriksen, M. P., Lovato, C. Y., Gottlieb, N. H., Brink, S. G., & Green, L. W. (1989).The diffusion of school-based tobacco use prevention programs: Project description andbaseline data. Health Education Research, 4, 111-124.





Parcel, G., Ross, J., Lavin, A., Portnoy, B., Nelson, G., & Winters, F. (1991). Enhancing imple-mentation of the teenage health teaching modules. Journal of School Health, 61, 35-38.

Parcel, G. S., O’Hara-Tompkins, N. M., Harrist, R. B., Basen-Engquist, K. M., McCormick,L. K., Gottlieb, N. H., et al. (1995). Diffusion of an effective tobacco prevention program.Part II: Evaluation of the adoption phase. Health Education Research, 10(3), 297-307.

Pentz, M. A., Chou, C.-P., McClure, M., Bernstein, K., Mann, D., Ross, L., et al. (2004, May).Adoption and early implementation of STEP: A multi-state teleconference-based preven-tion diffusion trial. Paper presented at the Annual Meeting of the Society for PreventionResearch, Quebec City, Quebec.

Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A.,et al. (1990). Effects of program implementation on adolescent drug use behavior. TheMidwestern Prevention Project (MPP). Evaluation Review, 14(3), 264-289.

Perry, C. L., Murray, D. M., & Griffin, G. (1990). Evaluating the statewide dissemination ofsmoking prevention curricula: Factors in teacher compliance. Journal of School Health,60, 501-504.

Petersilia, J. (1990). Conditions that permit intensive supervision programs to survive. Crimeand Delinquency, 36(1), 126-145.

Pettigrew, A. M., Woodman, R. W., & Cameron, K. S. (2001). Studying organizational change anddevelopment: Challenges for future research. Academy of Management Journal, 44(4), 697-713.

Prochaska, J. O., Velicer, W. F., Rossi, J. S., Goldstein, M. G., Marcus, B. H., Rakowski, W.,et al. (1994). Stages of change and decisional balance for 12 problem behaviors. HealthPsychology, 13, 39-46.

Resnicow, K., Cross, D., & Wynder, E. (1993). The Know Your Body Program: A review ofevaluation studies. Bulletin of the New York Academy of Medicine, 70(3), 188-207.

Ringwalt, C. R., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A.,et al. (2003). Factors associated with fidelity to substance use prevention curriculum guidesin the nation’s middle schools. Health Education & Behavior, 30(3), 375-391.

Ringwalt, C. R., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A.(2002). The prevalence of effective substance use prevention curricula in U.S. middleschools. Prevention Science, 3(4), 257-265.

Roberts-Gray, C., & Scheirer, M. A. (1988). Checking the congruence between a program andits organizational environment. New Directions for Program Evaluation, 40, 63-81.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.Rohrbach, L. A., Dent, C. W., Skara, S., & Sussman, S. (2006). Fidelity of implementation in

Project Towards No Drug Abuse (TND): A comparison of classroom teachers and programspecialists. Manuscript submitted for publication.

Rohrbach, L. A., D’Onofrio, C. N., Backer, T. E., & Montgomery, S. B. (1996). Diffusion ofschool-based substance abuse prevention programs. American Behavioral Scientist, 39(7),919-934.

Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based sub-stance abuse prevention program: Predictors of program implementation. PreventiveMedicine, 22, 237-260.

Ross, J. G., Leupker, R. V., Nelson, G. D., Saavedra, P., & Hubbard, B. M. (1991). Teenagehealth teaching modules: Impact of teacher training on implementation and student out-comes. Journal of School Health, 61(1), 31-34.

Safe and Drug Free Schools Program, U.S. Department of Education. (2001). Exemplary andpromising safe, disciplined, and drug-free schools programs 2001: Safe, Disciplined, andDrug-Free Schools Expert Panel. Retrieved July 29, 2005, from www.ed.gov/admins/lead/safety/exemplary01/index.html

Sanders, M. R., Turner, K. M., & Markie-Dadds, C. (2002). The development and disseminationof the Triple P-Positive Parenting Program: A multilevel, evidence-based system of par-enting and family support. Prevention Science, 3(3), 173-189.





Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and disseminationof interventions: What matters when? Psychiatric Services, 52(9), 1190-1197.

Schoenwald, S. K., Sheidow, A. J., Letourneau, E. J., & Liao, J. G. (2003). Transportability ofmultisystemic therapy: Evidence for multilevel influences. Mental Health ServicesResearch, 5(4), 223-239.

Senge, P. (1990). The fifth discipline: The art and practice of the learning organization. NewYork: Currency Doubleday.

Simpson, D. D. (2002). A conceptual framework for transferring research to practice. Journalof Substance Abuse Treatment, 22, 171-182.

Skara, S., Rohrbach, L. A., Sun, P., & Sussman, S. (in press). An evaluation of the fidelity ofimplementation of a school-based drug abuse prevention program: Project Towards NoDrug Abuse (TND). Journal of Drug Education.

Smith, D. W., McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1993). Teachers’ use ofhealth curricula: Implementation of Growing Healthy, Project SMART, and the TeenageHealth Teaching Modules. Journal of School Health, 63(8), 349-354.

Sobol, D. F., Rohrbach, L. A., Dent, C. W., Gleason, L., Brannon, B. R., Johnson, C. A., et al.(1989). The integrity of smoking prevention curriculum delivery. Health EducationResearch, 4, 59-67.

Society for Prevention Research. (2005). A case for braided prevention research and servicefunding. Retrieved September 9, 2005, from www.preventionresearch.org/BraidedFunding.pdf

Spoth, R., Greenberg, M., Bierman, K., & Redmond, C. (2004). PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science, 5(1), 31-39.

Spoth, R. L., Kavanagh, K. A., & Dishion, T. J. (2002). Family-centered preventive interven-tion science: Toward benefits to larger populations of children, youth, and families.Prevention Science, 3(3), 145-152.

Spoth, R. L., & Redmond, C. (2002). Project family prevention trials based in community-universitypartnerships: Toward scaled-up preventive interventions. Prevention Science, 3(3), 203-221.

Steckler, A., Goodman, R. M., McLeroy, K. R., Davis, S., & Koch, G. (1992). Measuring thediffusion of innovative health promotion programs. American Journal of HealthPromotion, 6(3), 214-224.

Stoll, L., & Fink, D. (1996). Changing our schools: Linking school effectiveness and schoolimprovement. Philadelphia: Open University Press.

Substance Abuse and Mental Health Services Administration. (2005). SAMHSA modelprograms. Retrieved November 17, 2005, from www.modelprograms.samhsa.gov

Sussman, S., Dent, C. W., Simon, T. R., Stacy, A., & Craig, S. (1998). One year outcomes ofProject Towards No Drug Abuse. Preventive Medicine, 27, 632-642.

Sussman, S., Dent, C. W., & Stacy, A. W. (2002). Project Towards No Drug Abuse: A review ofthe findings and future directions. American Journal of Health Behavior, 26(5), 354-365.

Sussman, S., Dent, C. W., Stacy, A. W., Hodgson, C. S., Burton, D., & Flay, B. R. (1993).Project Towards No Tobacco Use: Implementation, process and post-test knowledge eval-uation. Health Education Research, 8(1), 109-123.

Sussman, S., Rohrbach, L., & Mihalic, S. (2004). Blueprints for violence prevention, booktwelve: Project Towards No Drug Abuse. Boulder, CO: Center for the Study andPrevention of Violence.

Sussman, S., Sun, P., McCuller, W. J., & Dent, C. W. (2003). Project Towards No Drug Abuse:Two-year outcomes of a trial that compares health educator delivery to self-instruction.Preventive Medicine, 37(2), 155-162.

Sussman, S., Valente, T. W., Pentz, M. A., Rohrbach, L. A., & Skara, S. (2006). Translation in thehealth professions: Converting science into action. Evaluation and the Health Professions,29(1), 7-32.





Taggart, V., Bush, P., Zuckerman, A., & Theiss, P. (1990). A process evaluation of the Districtof Columbia “Know Your Body” project. Journal of School Health, 60, 60-66.

Tappe, M. K., Galer-Unti, R. A., & Bailey, K. C. (1995). Long-term implementation of theTeenage Health Teaching Modules by trained teachers: A case study. Journal of SchoolHealth, 65(10), 411-415.

Tobler, N. S. (2000). Lessons learned. Journal of Primary Prevention, 20(4), 261-274.Tornatzky, L. G., Fergus, E.., Avellar, J., & Fairweather, G. W. (1980). Innovation and social

processes: A national experiment in implementing social technology. New York: Pergamon.Tortu, S., & Botvin, G. (1989). School-based smoking prevention: The teacher training

process. Preventive Medicine, 18(2), 280-289.U.S. Department of Education. (2005). Preliminary overview of programs and changes

included in the No Child Left Behind Act of 2001: Safe and drug-free schools andcommunities (Title IV, Part A). Retrieved October 6, 2005, from www.ed.gov/nclb/overview/intro/progsum

U.S. Department of Education, National Diffusion Network. (1980). Educational programsthat work. San Francisco: Far West Laboratory of Educational Research.

U.S. Department of Health and Human Services. (1999). Mental health: A report of the sur-geon general. Rockville, MD: U.S. Department of Health and Human Services, SubstanceAbuse and Mental Health Services Administration, Center for Mental Health Services,National Institutes of Health, National Institute of Mental Health.

Valente, T. W. (2002). Evaluating health promotion programs. New York: Oxford UniversityPress.

Wilson, M. T., Atanda, R., Atkinson, D. D., & Mulvey, K. (2005). Outcomes from the TargetedCapacity Expansion (TCE) Substance Abuse Treatment Program. Evaluation and ProgramPlanning, 28(3), 341-348.

Young, R. L., deMoor, C., Wilder, M. B., Gully, S., Hovell, M. F., & Elder, J. P. (1990).Correlates of health facilitator performance in a tobacco-use prevention program:Implications for recruitment. Journal of School Health, 60(9), 463-467.





Documents

EHP290408