Use of research results in policy decision-making,.pdf

8/11/2019 Use of research results in policy decision-making,.pdf

1/13

Cad. Sade Pblica, Rio de Janeiro, 22 Sup:S7-S33, 2006

S7

Use of research results in policy decision-making,formulation, and implementation: a reviewof the literature

La utilizacin de los resultados de la investigacinen el proceso de decisin, formulacin yimplementacin de polticas: unarevisin de la literatura

1 Escola Nacional de SadePblica Sergio Arouca,Fundao Oswaldo Cruz,Rio de Janeiro, Brasil. 2 Instituto de la Salud JuanLazarte, Rosario, Argentina.

Correspondence

C. AlmeidaDepartamento de Admi nist rao ePlanejamento em Sade,Escola Nacional de SadePblica Sergio Arouca,Fundao Oswaldo Cruz.Rua Leopoldo Bulhes 1480,Rio de Janeiro, RJ 2104 1-21 0, Br asil [email protected]

Celia Almeida 1

Ernesto Bscolo 2

Abstract

This paper offers a critical review of the theo-retical literature on the relationship between theproduction of scientific knowledge and its usein policy formulation and implementation. Ex-tensive academic output, using a diversity of ap-proaches and analytical frameworks, has sought

to systematize knowledge transfer categoriesand strategies with a view to improving the ap-plication of scientific knowledge. A considerablepart of this thinking addresses the problem froma more traditional perspective, which (explic-itly or implicitly) regards research results as anaccumulable product, depicts the decision-making process simplistically and linearly, andthus restricts strategies to the suiting of researchendeavors to needs. Newer approaches place grea ter importance on the complexity of pol i-cymaking and the knowledge production pro-cess, which are integrated into and explained in

particular political and institutional settings. Although the application of knowledge transferideas to health policy and systems research does generate some interesting contributions, a longprocess of theoretical thinking lies ahead.

Health Policy; Policy Making; Use of ScientificInformation for Health Decision Making

Introduction

Although the debate on the use of research re-sults for policy decision-making and implemen-tation processes is not new and its features havechanged over time, the issue has gained greaterprominence in recent decades following the major processes of world change that increasingly

call for concrete evidence to support or challengethe innovations that are implemented in a va-riety of contexts, including health policies andsystems.

The background to the debate dates to themid-20 th century and overlaps the following: (1)the field of public policy analysis, and thus theformation of political science as a specific dis-cipline in the social sciences and (2) discussionsof the social sciences approach and methods asapplied to various fields of knowledge, includinghealth services research.

The literature on the analysis of health and

health systems public policy does not provide aclear, single definition of either health policy orhealth services and systems. However, there isa clear consensus that what is being analyzed ispublic policy, and thus state policy; there is lessagreement (particularly more recently) as to thescope of such policy and whether health policies(or at least part of their content) belong to the rollof social policies.

Importantly, unlike the English-language lit-erature, which distinguishes between policy

DEBATE DEBATE


2/13

Almeida C, Bscolo ES8


S8

(policy content, a framework of guidelines for ac-tion) and politics (the political decision-mak-ing process, involving a range of loci and actors),Latin American literature (in Spanish or Portu-guese) uses a single term (poltica ), with varioususes and meanings that encompass both senses.

Public policy analysis emerged, particularlyin the United States, as a s cience of action , a con-tribution by experts (analysts) to governmentdecision-making processes. The central concern was to direct research in such a way as to be rel-evant , useful for action. That pragmatic view ofpolitical science came, in turn, as a reaction to aprevailing formalism 1 and against the abstractrationalism of Homo economicus 2.

Underlying such an approach was obviouslyan ingenuous conception that tended to see asimplistic relationship between improved under-standing of action and better government perfor-mance. In the late 1950s and early 60s it was be-lieved that this association between experts (pol-icy analysts) and policymakers would facilitatesolutions to societys problems. This helped focusattention on crafting tools to be made availableto politicians and decision-makers, while theo-retical considerations were relegated to second-ary importance.

This trend was extremely strong in the Unit-ed States in the 1960s and 70s. According tosome authors it led to the production of practi-cal knowledge. This submissive, a-theoreticalview of political science was challenged, thussparking interest in other concerns more fun-damental to this debate, making it possible to

break out of the vicious circle that threatenedto confine public policy analysis to the func-tion of a decision-making aid, [placing expertsin the role of] consultant and princes helper 2 (p. 45). This confusion between research and theoperations approach led to a differentiation (andseparation) of functions between scientists andconsultants.

Paradoxically, however, the resurgence oftheory during the subsequent decades returnedto the same theoretical models that had beenrejected initially, because public policy theoriesare not particularly innovative. Whether for or

against, they are rooted in political philosophyand economic thinking, with the important,original difference that they address what wasthen a little-explored field, i.e., they are based onempirical research, but do not disregard the im-plications for action, thus surmounting both theconfusion between practical usefulness andrecipe for government and the false, radical op-position between theory and practice.

Writing in the context of the process summa-rized above and concerned with the history and

development of health services research as a fieldof inquiry, Greenberg & Choi 3 suggest, based on Anderson 4, that the development of public poli-cy consensus on a particular issue or set of issuesestablishes the basic framework for policy-relat-ed social and economic research, within whichresearch priorities are set.

As Greenberg & Choi3 (p. 4) emphasize, twokey words systematic and instrumental emerge from Andersons thesis, which is not arguing thatno research or data collection occurs prior to policyconsensus, but rather that large-scale, systematiccollection and analysis efforts only emerge (andare funded) after the establishment of consensuson issues to be addressed by particular policies ()[and] the resulting research efforts become primar-ily instrumental in nature. The term instrumen-tal refers to work that is oriented to the solution ofspecific management program or policy problemsand tends to be constrained by very short time frames . Seen in this way, health policy and ser-vices research becomes a problem-oriented fieldand, thus understood, is instrumental.

It is important to remember that these authorsare discussing the health services research field, which was developing as part of a much largertrend towards using applied social science re-search to improve public decision-making. Manyhistorians of health policy and health services re-search argue that this field, as a distinct field ofinquiry, really began in the 1960s 5,6. Before then,the primary focus of health policy had been todevelop a delivery system that would provide eq-uitable access to care, and the perceived role of

federal government was to channel resources forthat purpose. The 1970s witnessed a shift in thepolicy consensus away from access to health careand expansion of coverage and toward cost con-tainment and shrinking the system 3,5,7,8,9,10.The issue of health sector reform has been on theagenda of almost all countries ever since, and newmodels of health systems and services organiza-tion have come into play. New policy instrumentshave become necessary, and these have gener-ated great demand for new knowledge about howto control health systems more effectively.

As part of this process, the health services

research field took on new life in the 1980s, butreceived heavy criticism for its apparent failureto perform as a source of instrumental findings the central issue in debate came to be the useof research results. As a result, there is no generalagreement on how to distinguish between ap-plied social science research in health deliverysystems, health services research, and health pol-icy analysis. Such criticism also raises the issue ofemphasis: whether towards theoretical develop-ment or problem-solving approaches 3.


3/13

USE OF RESEA RCH RESULTS IN POLICY DECISION-MAKING, FOR MULATION, AND IMPLEMENTATIONS9


S9

In order to help ensure that research wouldbe problem-solving oriented and would fo-cus on desired policy issues, the sponsors andfinancers in many cases mainly the state, i.e.,the federal government, and international or-ganizations began to rely more heavily on thecall for proposals approach to award grants,in contrast to the researcher-oriented approachof the past 3,11. The issue of the diffusion of ser-vice innovations gained prominence, as did thesearch for evidence for public health and healthpolicy. Thus, the problems posed by the difficultyin transferring scientific information or researchresults to decision-making have become a subject for academic thinking.

As many authors point out, whether researchresults or scientific evidence have contributedsignificantly to public policy depends largely onhow one defines public policy and the policy de-velopment process. Opposing approaches can beidentified in the recent past.

One view of the policy-making and policy-formation processes sees them as sets of explicit,authoritative decisions by sets of identifiablepublic officials. The other view argues that a morecomplex, general political process is involved,strongly influenced by values, opinions, and ac-tions which move decisions in certain directionsin the political, social, and economic systems. Inthe first approach, the impact of research resultstends to be evaluated in terms of the direct ef-fects on such decisions; in the second, researchperformance should be evaluated in terms ofits overall ability to shape debate and action. Ac-

cordingly, one should not expect any specific re-sult of any specific study to play a central role inany specific decision, but should look toward thenature of the issues being raised and the debatesurrounding theses issues. Greenberg & Choi 3 feel one should at least be able to associate pastand ongoing research with current policy issuesand debates.

In the wake of these discussions and changesin direction, analysis of the health policy processhas gained prominence, and efforts are under way to equip and formalize the health sector de-cision-making process by developing facilitator

tools.Generally speaking, the literature warns thatthe fields of knowledge production and policy for-mulation and implementation are very different:their goals and methods for working and eval-uating results are completely different and noteasily interchangeable. Meanwhile, this lack ofclarity makes it difficult to establish strategiesto effectively draw these two activities closertogether on the basis of greater cooperationand visible, concrete results. In addition, much

of the frustration surrounding attempts to ap-ply research results to policy stems from mis-taken expectations as to what such applicationmeans 12, as well as to a lack of any clear percep-tion of what the decision-making process is reallylike 13. There is also no single way of viewing thatprocess, and depending on which analytical per-spective is used, the entry points for research inthe policy process also shift, as do the variablesthat interact in it 14.

Terms such as informed choice, considereddecision, rational policy, evidence-based pol-icy, strategic research, and essential nationalresearch have been used to express the beliefin the need to build a bridge between researchand policy. National and international seminars,congresses, and scientific meetings have focusedon the research-to-policy issue. A number ofmoves have attempted to facilitate the use of re-search results in policies, including the prepara-tion of tool kits for defining research priorities,demand for (and utilization of) health policy andsystems research, and capacity-building (Train-ing Materials and Tools, Alliance for Health Policyand System Research, http://www.alliance-hpsr.org); or political maps designed to facilitate anunderstanding of the decision-making process15 and mainly involving the national and inter-national financing agencies. Increasingly, suchagencies are requiring that research protocolsstate this link explicitly and discuss specific strat-egies for this purpose. Still, little is known aboutwhere , how , and by whom such a bridge is to bebuilt 12.

The health field provides abundant examplesof the complex relationship between researchresults and policy formulation and of the waysby which scientific evidence influences specificcauses of disease or individual behavior changes.Taking prevention as an example (e.g., smokingand related health damage, or environmentalmeasures), the evidence is that the accumulatedscientific knowledge on a given issue is not whatdetermines significant policy or even behavioralchange 16. The interests involved in these issuesare far more varied and extend far beyond thehealth sector 3,14.

The purpose of this article is to offer a litera-ture review on this subject, based on selected au-thors, and to discuss their arguments and someof the main theoretical approaches to explain orback the relationship between the productionof scientific knowledge and its use in policy for-mulation and implementation. The first sectionpresents several analytical models designed toexplain these relationships. The second reviewsthe issue of the use of research results and poli-cy-making. The third analyzes the literature on


4/13



the interaction between researchers and poli-cymakers. The fourth reviews issues related tothe spread of knowledge, knowledge transfer,and evidence base in health policy and practice.The initial hypothesis is that there is an exces-sive formalization of instruments and pragmaticsimplification in the proposals designed to drawthe two fields (research and policy) closer, whilethe importance of formulating and developinganalytical and explanatory frameworks that per-haps offer more promise in this process is under-estimated.

Some explanatory models for useof research in policy

The term knowledge transfer is increasinglyused to describe a series of activities containedin the process of generating knowledge basedon user needs, disseminating it, building capac-ity for its uptake by decision-makers, and finallytracking its application in specific contexts. Whilethere is increasing interest in knowledge trans-fer for health policy and practice, there is still nodominant explanatory model to guide efforts inthis area, and there is little empirical research on what has worked in specific contexts.

Weiss 17 is cited in the literature as havingpioneered the identification and description ofseven models to illustrate how research is used inpolicy formulation or how it functions as a guidefor the decision-making process. From thosemodels, authors are defining analytical catego-

ries that enable this knowledge to be extended.Trostle et al. 12 summarize these models inthree basic approaches: (1) the rational approach includes the models that Weiss calls knowledge-driven and problem-solving , representing theconventional manner of thinking about this rela-tionship: the policy process is inherently rational, with research results being used when they existand decision-makers calling for research when itis needed; (2) the strategic approach groups themodels Weiss calls political and tactical andviews research as a kind of ammunition in sup-port or critical of certain positions, prompting or

delaying policy action; and (3) the enlightenmentor diffusion approach , comprising Weiss threeremaining models interactive , enlighten-ment , and intellectual enterprise and stress-ing that both the research and decision-makingprocesses take place in parallel with a numberof other social processes and thus play severaldifferent roles.

The knowledge-driven model assumes thatbasic research leads to applied research, to de-velopment, and finally to application of the re-

sults, and the problem-solving model starts witha problem that needs solving, in turn requiringresearch, the results of which lead to action beingtaken. The political and tactical models connectdirectly to executive action; and the last three interactive, enlightenment, and intellectual en-terprise relate to the production of scientificknowledge in a given line of research, fostering abuild-up of knowledge that (it is believed) gradu-ally informs action 13.

In the late 1980s, the concept of use expand-ed to encompass at least three different types ofmeaning: (1) instrumental, as an input to deci-sion-making; (2) conceptual, contributing toimproved understanding of the subject matter,the related problems, or the political interven-tions under study; and (3) strategic, serving topersuade other actors or as a means to attain cer-tain aims 18.

With regard to evaluation of study results,Kirkhart 19 criticized the traditional concept ofuse as being unidirectional, episodic, intended,and instrumental. It failed to adequately describetypes of impact deriving from sources other thanthe results of evaluation, or unintended results orgradual, incremental impact over time. Kirkhartaddressed issues such as the ways by which theresults of an evaluation study are used, includingan examination of the ways (how and to what ex-tent) the evaluator participates, affects, supports,and proposes mechanisms that foster behaviorchanges in people and systems. This places newimportance on the impact of evaluation itself,rather than on any immediate use of its output 20.

From this perspective, it is supposed to be easierto measure the extent of changes in a programor the views of strategic actors than in the use ofstudy results.

In a similar approach, Rich 21 characterizesthe use of research results in the following stages,viewed from a process perspective: informationpick-up, processing, and application.

Kirkhart thus proposes the development of anintegrated theory of influence, enabling the im-pact of research or evaluation to be assessed, by re-placing the category use (with its more restrictedmeaning) with influence, defined as the capac-

ity or power of persons or things to produce effectson others by intangible or indirect means 19 (p. 7).The integrated theory of influence rests on

three dimensions: source, intention, and timing.Source relates to the initial cause of a process ofchange. Sources can arise either in the evalua-tion process (process-based) or the results (re-sults-based). The former are oriented towardsincreasing understanding among the actors,changing their sense of the values and develop-ing new relationships, dialogues, and networks.


5/13



The categories used are: (i) cognitive changes inunderstandings stimulated by the discussion, re-flection, and problem analysis in an evaluationstudy; (ii) affective changes in the individual andcollective feelings of worth and value resultingfrom involvement in a study; and (iii) politicalchanges resulting from the role of evaluation increating new dialogues, drawing attention tosocial problems, or influencing power relation-ships. Evaluation of results is seen in terms of themore conventional categories of instrumental,conceptual, symbolic, and strategic influence.

A second dimension of influence identifiedby Kirkhart 19 (p. 11) is intention , defined as theextent to which evaluation influence is purpose- fully directed, consciously recognized and planful-ly anticipated and characterized by the type, tar-get, and sources of influence (people, processes,and findings). Other complementary features aredimension (intended/unintended), explicitness(manifest/latent), orientation (results-oriented/process-oriented), and direction (positive/nega-tive). Lastly, influence is seen in terms of threecategories of temporal dimension: immediate(during the study), end-of-cycle, and long-term.

Patton 22 identifies a typology of use thatstresses processes rather than products: (1) en-hancing shared understandings, especially aboutresults; (2) supporting and reinforcing the pro-gram through intervention-oriented evaluation;(3) increasing participants engagement, sense ofownership, and self-determination; and (4) pro-gram or organizational development.

Meanwhile, Forss et al. 23 expand on the previ-

ous proposal to identify five types of process-re-lated use: learning to learn; developing networks;creating shared understanding; strengtheningthe project; and boosting morale.

Despite the extensive literature (the major-ity of the studies have not been cited here), thestate-of-the-art on the most effective and effi-cient knowledge transfer strategies is still in itsinfancy.

Use of research results andpolicy-making in health

The authors thinking and models are useful forunderstanding the various ways by which re-search results are used in policy-making . How-ever, Walt & Gilson13 emphasize that the under-lying assumption of many is that both researchand policy-making are logical, rational processes where researchers ask the right questions, planand conduct their studies rigorously, and circu-late their results appropriately, and that deci-sion-makers read research reports, understand

the results and their implications, and act tocorrect their course in the direction indicated.Even admitting to a specific rationality in eachof these processes, the real world is not so lin-ear: new knowledge and information do in factpenetrate the policy sphere and become part ofdecision-makers argumentation and thinking,but in a much more diffuse way that depends onthe accumulation of scientific information on agiven issue, the political environment, and theconjuncture, among numerous other variables.The search to find (and accumulate) evidencethus becomes the other important part of thisequation.

In order to capture this dynamic, Walt & Gil-son 13 suggested the use of other analytical cat-egories to understand the influence of researchon specific concrete cases, namely policy con-tent , social actors , decision-making process , andcontext .

These are basic political science categories.The formulation, implementation, and evalua-tion of social policies are heavily guided by thevalues and concepts of social realities shared bythe leading actors in the various process levels,or by bureaucratic elites. These values and con-cepts provide the terms of the debate on poli-cies, delimiting and circumscribing the policyagenda at any given moment 24. Meanwhile, thepolitical, economic, and institutional context ofthe decision-making process shapes the rangeof available options and affects decision-makerschoices 25, i.e., the ways in which the evidence isused in the policy process are largely determined

by beliefs and values of policymakers, as well asby considerations of timing, economics, costs, andpolitics 27 (p. 601). Besides, the policy formu-lation process is completely different from theimplementation process, so that a proposal forchange rarely retains its original characteristics when implemented, because it alters the statusquo and mobilizes actors to defend their inter-ests. Overall, the central category that emergesfrom such discussions is power , with its innumer-able facets and dimensions.

Brown 14 (p. 20) asserts that the numerousgeneralizations on the role of health systems and

services research in the political process are notparticularly useful, because as a power resource ,knowledge plays innumerable roles, whichchange with place, time, and circumstances. Eachconcrete case also involves different explanatoryvariables for policy change, and each variable im-plies that the research has played a specific role.Some variables are particularly important, asBrown points out on the basis of case studies inthe United States. Trostle et al. 12 find the same intheir Mexican case studies on the same subject.


6/13



There is a certain consensus among authorsconcerning various barriers that hinder or pre-vent research from being used in the decision-making process. These include:a) Ideological problems that constrain politicalrhetoric and the formulation of reform agendas,in addition to a lack of political will or an inabi-lity to formulate and implement more integrated,interactive policies;b) Historical separation between researchers,policymakers, service providers, administrators,managers, etc., allied to a mutual intellectual dis-dain 12;c) Uncertainty caused by scientific divergen-ces among researchers on any given problem(conceptual confusion, methodological proble-ms); by the very changes that scientific and tech-nological progress foster in explanations of anygiven phenomenon, meaning that the scientifictruths of today are challenged tomorrow; andbecause the information is fragmented, with di-fferent definitions of the important problems;d) Different conceptions of risk at the indivi-dual or collective level, and in the same sector oramong various sectors, particularly in multi-sec-tor actions (e.g. environmental control, tobaccouse, alcohol, diet, and nutrition);e) Media interference, which can both confusethe issue by publicizing results inappropriatelyand exploit divergences rather than clarifyingthem;f) Marketing and circulation of research: resear-chers using impenetrable language; inappropria-te means restricted to peer forums and publica-

tions; andg) Research process timeframes out of step withthose of the decision-making process.

Trostle et al. 12, analyzing the Mexican casestudies, identify both barriers and factors thatpromote and facilitate the use of research re-sults in policy in each of the analytical categoriessuggested by Walt & Gilson, which are generallyspecific to each policy area under consideration.However, little headway has been made on pro-posals to surmount these barriers; the questionis whether simply surmounting them will solvethe impasses or whether this approach (although

necessary) is really sufficient to promote greateruse of research in policy formulation and imple-mentation.

Brown 14 argues that some variables are fun-damental, although they should not be consid-ered absolute. In addition, the degree to whichresearch actually influences policy varies in-versely with the complexity of the issue understudy. The first health services research activitythat he identifies with an important influenceon policy formulation and implementation (and

which is not exactly research, but relates to it) isdocumentation : the gathering, cataloguing, andcorrelating of facts depicting the state of affairsthat policy-makers wish to change. In addition,compiling statistical data allows establishingtemporal and spatial correlations.

However, Brown cautions that quantitativeand qualitative databases and statistical indica-tors cannot be considered research results perse, but are the raw material on which researchis shaped and without which it cannot be con-ducted. He also emphasizes that documentationis not always a step toward action; sometimes itstultifies it 14 (p. 28). When the right predispo-sition or political and material conditions exist,information can become a powerful weapon tospur the shift from political rhetoric to concreteaction. However, information alone cannot cre-ate such a predisposition. Likewise, Bardach 27 states that policy analysis theory proposes thatevidence is information that affects existingbeliefs by important persons about significantfeatures of the problem under study and how itmight be solved or mitigated.

A second role of health service research inpolicy design, according to Brown 14, is analyti-cal , i.e. demonstrating what works or does not,and explaining why. In a conflictive reform con-text, research that raises doubts about the omni-present list of alternatives for government inter-vention can prompt policy-makers to take action. Yet it also is a two-edged sword, since it can callattention to aspects unforeseen by policymakersand thus discredit both the policy as formulated

and the proposed alternatives.Bowen & Zwi26 (p. 601) also argue that theway in which research evidence is combined withother forms of information is key to understand-ing the meaning and use of evidence in policy de-velopment and practice. (...) A major challengeto contextualizing evidence for policymaking isrecognition that a broad information base is re-quired [and] () considering the evidence withinthe context in which it will be used is critical foreffective policymaking and practice .

A final role that Brown highlights for healthsystems and services research is prescription . The

political force of empirical evaluation and ana-lytical constructs resides in the scientific char-acter they lend to reform proposals, reinforcingdecision-makers prior positions.

Research can thus play a variety of roles inpolicy. The task of prescription differs greatlyfrom those of documentation and analysis: dem-onstrating how and why the system functions isitself a contribution to the production of knowl-edge, which is the essence of research activity,but on its own that contribution will not teach


7/13



decision-makers how to change the system. Onthe other hand, more narrowly focused studies,directed exclusively to problem-solving, have nobroader implications and do not contribute tothe build-up of knowledge achieved by pursuinga line of research that lasts over time.

Prescription requires understanding not justhow and why actors and institutions behave asthey do under given conditions, but also (andmore importantly) how such behavior can be al-tered under new conditions; and here not eventhe most complete documentation or analysis isenough to ensure any result. In addition, in orderto prescribe solutions, researchers would haveto shed their academic guise and explore ave-nues not entirely authorized by their theoreticaland methodological frameworks, thus runningthe risk of being discredited by actual develop-ments. Seen thus, health systems and servicesresearch has been very erratic as a prescriptiveguide and, in the endeavor, has tended to devi-ate from the field of research and to confuse it with others, such as technocratic and planningactivities. In practice, however, these fields arevery different and operate with distinct concep-tual universes.

In summary, Brown 14 regards research asmost valuable to policy in supporting the docu-mentation that describes the system; most er-ratic in analyzing how policy functions andexplaining what works, and how and why; andconsiderably limited in its prescriptive capacity.Thus, the potential contribution of research tothe decision-making process has less to do with

offering definitive solutions to the problematicissues in debate and more with improving thequality of the terms of the debate. Thus, the abil-ity to change the nature of public debate on a giv-en issue is an important form of power, becausebringing ideas, proposals, and interests intoconfrontation is an important force in changingthe balance of power among the various contest-ing groups. Other authors, like Weiss 17 and Ma- jone 28, reinforce this argument.

The contradiction here is that as the researchfield becomes more developed and promising,the range of results, explanations, arguments,

and recommendations tends to expand, and thefield may thus appear more chaotic to decision-makers. In other words, an active, highly-skilledscientific community in the health systems andservices research field, closely attuned to insti-tutional niches in the state apparatus and civilsociety and with a secure place in the decision-making arena (such as the policy networks 29,30 or the epistemic communities 31,32) is certainto foster better debate, but not necessarily betterpolicy results.

Also distant from the rational approach andcloser to Brown and Walt, some authors recog-nize an incremental dimension to the use ofknowledge 26,33,34, placing new importance onthe complexity of the decision-making process;this dimension identifies conditioning factorscomplementary to scientific knowledge, such asinterests, values, established institutional posi-tions, and personal ambitions. This view includesa political and institutional approach to the de-cision-making process, where identifying andcharacterizing the actors and interrelations is akey dimension to understanding the process andthe use of research results in a framework of po-litical negotiation, rather than restricted to crite-ria sustained by scientific evidence.

Interaction between researchers anddecision-makers as an explanatorydimension conditioning the useof research results

Interrelations between researchers and decision-makers have been considered a prime factor inanalyzing knowledge transfer processes. Analysisof the weaves (or models) of interrelations be-tween researchers and decision-makers is relevant when one realizes that the use of scientific knowl-edge depends largely on certain characteristicsof the actors (researchers behavior and de-cis-ion-makers receptiveness). Various authors ha-ve examined and promoted different ways of im-proving interrelations between researchers and

decision-makers, such as collaborative or alliedresearch 35, or constructivist approaches to eval-uative research 36, including strategies to improvethe knowledge output for decision-making.

This approach fosters a political and institu-tional analysis of relations between actors and or-ganizations in the interconnections between re-search and policy formulation and implementa-tion processes as a conditioning (or independent)variable in the use of knowledge or its impact ondecision-making processes (dependent vari-able). Kothari et al. 37 evaluate the implications ofdifferent interrelations between researchers and

decision-makers for the use of research resultsin health policies and programs. Their concep-tual framework for such interactions is based onRich 21, viewing possible opportunities for con-tact at the following research stages: (i) definingthe research questions; (ii) conducting the re-search, and the research findings; (iii) circulatingresults; and (iv) research utilization, defined asthe ways by which research findings influencedecision-makers. As already mentioned, use ischaracterized by dimensions in which several


8/13



stages play out from a process perspective, sum-marized as receiving and reading; informationprocessing; and application.

Viewed in a less linear light, interaction couldbe defined as a number of disordered interac-tions that take place between researchers andusers, more than a sequence starting with theneeds of researchers or decision-makers.

Lomas 38 has built on this literature and usedthe term push and pull to describe specific po-tential actions to promote knowledge transfer.Push strategies involve transforming the messageaccording to the needs of specific audiences. Thisrepresents a departure from one-size-fits-alldissemination strategies. Pull strategies includeefforts to build capacity at the user end, such as workshops to explain how to access informationand evaluate its scientific rigor.

Meanwhile, Lomas 39, Lavis et al.40, and Landryet al. 41 draw on the literature of action science andparticipatory research to suggest that these contactsituations or interfaces must be leveraged fromthe beginning of the research process to promoteactivities that create linkages. Such activitiesinclude providing opportunities for researchersand users to jointly define the research question,maintain contact during the research process,and (following completion) discuss findings andthe potential implications for policy and practice.

Landry et al. 42 analyzed research use as thedependent variable and its association with a wide range of independent determinants, in-cluding various knowledge transfer activities. Re-search use was measured on a scale of one to six

using Knott & Wildavskys43 self-reported index,as follows:1) Reception: received research pertinent toones work;2) Cognition: read and understood it;3) Discussion: discussed the research in meetings;4) Reference: cited the research in reports/pre-sentations;5) Effort: made an effort to favor the use of re-search; and6) Influence: the research influenced decisions.

In their study of 833 Canadian policy-makers,Landry et al. 42 grouped independent variables

by conceptual paradigm and implicit assump-tions about factors that increase research use.They found that scholarly research was as likelyto be used as applied research, and that highlytheoretical qualitative research was slightly lesslikely to be used than quantitative studies. Theirconclusions can be summarized as follows: Factors deriving from the two-communitiesapproach, such as whether there had been speci-fic efforts to tailor research for users (push) andto build capacity among decision-makers to use

research (pull), were found to be highly related touse; Similarly, the linkages approach, whichposits that more interaction with researchersthroughout the research cycle improves use, pro-ved to be a significant determinant of use; Engineering solutions, cited by those whobelieve research can fix policy problems, pro-pose that the type of research (applied versusscholarly or qualitative versus quantitative) is themost important determinant of use; Organizational factors like structure, size,culture, and policy domain are often posited asimportant determinants, but were not found tobe associated with research use; Among the individual factors often assig-ned importance, like educational level and posi-tion within an institution, education was indeedfound to be associated with higher levels of re-search use, although position within an organi-zation was not.

Looking at the knowledge production and de-cision-making processes suggests several points where contact between these two dimensionsand logics could be useful. The identificationand analysis of these points of contact or inter-faces charts another process: the interrelationsbetween researchers and decision-makers.

Trostle et al. 12 offer a graphic representationof the dynamic relationship between research andpolicy formulation, highlighting that althoughthe processes are usually independent of eachother, there may be connections between them.They call these contacts moments of opportu-

nity, since the actors in one process learn fromor contribute to those in the other. They also em-phasize that the main challenge for applying re-search results to policy is to learn to create or rec-ognize such moments of opportunity, and then toact effectively to take advantage of them. Starting with the left side of a conceptual figure and mov-ing counter-clockwise, the research process in-cludes the stages of idea generation, design, datagathering, analysis, and application (an interre-lationship represented by arrows). Following thearrows, the research results can lead to new ideasand research projects, or may also be applied.

The policy process is similarly represented,starting on the left side of the conceptual figureand moving clockwise. When the needs or prob-lems that arise can be addressed on the basis ofspecific policies, there is an endeavor to gatherinformation on them from a variety of sources. In-terest groups may lobby at various moments andthereby influence definition of priorities amongthe needs, decisions, or policies. As in the previ-ous process, some arrows return to the informa-tion-gathering stage, before policy is designed.


9/13



Some decisions generate a search for additionalinformation and further negotiations, while oth-ers produce policies. New policies can also lead tonew interest groups and new political challenges.

Other authors 44 formulate concepts similar tothe moments of opportunity, containing the ideathat knowledge can be useful in generating chan-ge only when a window of opportunity appears.

It is important that analysis examines the ac-tors in the process and evaluates the sources ofinformation they trust, the type of informationthat interests them, how they evaluate informa-tion, what motivates them to make decisions,and the actors with whom they interact, com-pete, and form alliances.

Trostle et al. 12 and Bronfman et al. 43 alsooffer a graphic representation of relations be-tween actors and context. In their figure, twolarger intersecting circles represent civil societyand the state. Other smaller circles represent theinterest groups real or potential impact on poli-cies. Researchers are one such group. Also showngraphically are the mutual influences betweeninterest groups and decision-makers. Public poli-cies (the focus of their study) are formulated bypublic decision-makers and thus stand at the in-tersection between state and civil society. Actorsare located in one sphere or another, or at theirintersections.

Other studies propose a variant on stakehold-er analysis, featuring researchers as a group withthe ability to intervene in the decision-makingprocess 46 and in the policy formulation processdirectly or through other key stakeholders over

whom they have influence. This interpretationsees the decision-making process as susceptibleto influence not only from the knowledge gen-erated by research, but also from the researchprocess itself. This notion allows analyzing theinfluence of the knowledge produced separatelyfrom the influence of the process by which suchknowledge is produced.

Similarly, the position to be taken towardsthe problem at hand cannot be interpreted sepa-rately from the context and stakeholder analysis.In some situations, objective variables predomi-nate and an instrumental approach may be suf-

ficient. In others, subjective variables predomi-nate, making it possible to opt for a conceptualor symbolic approach.

Diffusion, knowledge transfer,and evidence base in healthpolicy and practice

Many proponents of research use in health carehave drawn on the theory of diffusion of innova-

tion 47. As Bowen & Zwi26 (p. 600) also point out,the contemporary public health effort sees muchdebate about the concepts of evidence and theevidence base, and the usefulness and relevanceof such terms to both policy-making and practice . A key challenge would be to better contextualizeevidence for more effective policy-making andpractice.

Greenhalgh et al. 48 conducted a systematicreview (commissioned by the UK Department ofHealth) of the diffusion of service innovations.Drawing on Rogers overview47 and other empir-ical work, they formulated a unifying conceptualmodel that, instead of serving as a prescriptiveformula, intended to be mainly a mnemonic aidfor considering the different aspects of a complexsituation and their many interactions. These au-thors define innovation in service delivery andorganizations as a novel set of behaviors, routines,and ways of working that are directed at improv-ing health outcomes, administrative efficiency,cost effectiveness, or users experience and thatare implemented by planned and coordinatedactions . The authors distinguish between diffusion (passive spread), dissemination (activeand planned efforts to persuade target groups toadopt an innovation), implementation (activeand planned efforts to mainstream an innova-tion within an organization), and sustainability(making an innovation routine until it reachesobsolescence) 48 (p. 582).

For the purposes of this study, we highlightsome issues emphasized by the authors. Theirview is that the various influences that help

spread the innovation can be thought of as lying ona continuum . In pure diffusion the spread of in-novation is unplanned, informal, decentralized,and largely horizontal or peer-mediated, whileactive dissemination is planned, formal, oftencentralized, and likely to occur more through ver-tical hierarchies. Whereas mass media and otherimpersonal channels may create awareness ofan innovation, interpersonal influence throughsocial networks is the dominant mechanism fordiffusion. The adoption of innovation by indi-viduals is powerfully influenced by the structureand quality of their social networks, and different

social networks also have different uses for differ-ent types of influence 48 (p. 601-2). According to their review, the literature on

diffusion of innovation in health care organiza-tions is vast and complex and contains many well-described themes, such as the useful listof attributes of innovation that predict (but donot guarantee) successful adoption, and the im-portance of social influence and the networksthrough which it operates. They also exposed thelack of empirical evidence for the widely cited


10/13



adopters traits, the limited ability to generalizefrom the empirical work, and the near-absenceof studies focusing primarily on the sustainabil-ity of complex service innovations. Finally, theyemphasize the multiple and often unpredictableinteractions arising in particular contexts andsettings, which determine the success or failureof a dissemination initiative, and clear knowl-edge gaps where further research on diffusion ofinnovation in service organizations should focus.One of the studys most important outputs, re-sulting from their analysis of the literature, was aparsimonious, evidence-based model for consid-ering innovation diffusion in this area 48.

Based on a thorough literature review, Bowen& Zwi26 (p. 600) propose an evidence-informedpolicy and practice pathway to help researchersand policy actors navigate the use of evidence .

The pathway involves three active stages ofprogression, influenced by the policy context:sourcing, using, and implementing the evidence.It also involves decision-making factors and a pro-cess they call adopt, adapt, and act 26 (p. 600).

The authors argue that the term evidence-based policy is used largely for one type of evi-dence (research) using the term evidence-influenced or evidence-informed would reflect theneed to be context-sensitive and concerned witheveryday circumstances and that the types ofevidence that inform the policy process can begrouped as research, knowledge/information,ideas/interests, politics, and economics.

In their attempt to construct a more com-prehensive framework to understand the use of

evidence for policy, Bowen & Zwi26 (p. 602) de-fine the categories of relative advantage, com-patibility with values and past experience, costand flexibility, trialability, and reversibility, policyenvironment and others, besides conductinga critical review of the literature on knowledgetransfer, diffusion, and innovation.

Meanwhile, they emphasize that it is difficult for evidence to remain intact through the process given the policy context, decision-making factors,and the need to adapt , indicating that evidenceinteracts with context before it is fully adopted inpolicy and practice, and/or that different types of

evidence are useful at different times in the policyprocess . Therefore, effective knowledge transferis not a one off event, rather it is a powerful andcontinuous process in which knowledge accumu-lates and influences thinking over time . Thus,the ability to sustain this process and a focuson human interaction is essential . Differences inconceptual understanding, scientific uncertainty,timing, and confusion influence the response toevidence . In summary, understanding knowl-edge utilization in policymaking requires an un-

derstanding of what drives policy... and deter-mining capacity to act on evidence is a neglectedarea of policy analysis and research efforts todate 26 (p. 603-4). In other words, understandinghow evidence informs policy and practice is criti-cal to promoting effective and sustained healthpolicy.

Some final thoughts and remarks

While there is considerable production seekingto systematize knowledge transfer categories andstrategies to improve the application of scientificknowledge, there is still a wide diversity of con-ceptions and analytical frameworks.

According to the traditional view, scientificknowledge is regarded as an accumulable prod-uct which decision-makers can resort to accord-ing to their needs. This conception is generallyallied with a simplified view of the decision-mak-ing process 49, assuming policy formulation andimplementation as a linear process comprisinga chain of rational decisions made by privilegedactors. On this basis the problem is seen to lie inthe difficulties involved in making the right infor-mation available to decision-makers at the rightmoment.

Others studies explore the subject in greaterdepth and highlight the specific features andcomplexity of policy-making, where the actors(researchers, policy-makers, members of thestate structure or civil society) and decisive, spe-cific conjunctures are all fundamental categories

for addressing the problems of knowledge trans-fer. Likewise, researchers and their productionprocess are considered to be actors integratedinto a particular political and institutional con-text, thus reinterpreting the relationship betweenthe subject and the object of study. In this regard,integration between researchers and decision-makers is assigned greater value as a potentialfactor conditioning the ways by which researchresults are used in policies, while the actors or-ganization in social networks is regarded as an-other fundamental variable for facilitating suchinteraction and guaranteeing that specific inno-

vations are incorporated at given conjunctures.The various models and analytical catego-ries discussed in this study describe importantaspects and dimensions in this interaction be-tween research and policy-making, revealing itscomplexity and helping understand it. However,the problem lies in the prescriptive intentionof some proposals, since the decision-makingprocess on any given policy involves numerousintervening variables that can combine in highlyrandom ways (all of which is proper to the po-


11/13



litical process) making it hard to predict the out-come of such interaction.

Research results are seen as having highlyvaried roles in policy formulation, the most ef-fective perhaps being to change the terms of thedebate on a given issue, depending on the ac-tors political power of persuasion and their abil-ity (using politics and lobbying) to keep the spe-cific issue on the policy agenda over time and toimplement the intended changes, as well as theissues importance to a given society at a specificmoment.

The ability to transform technical and scien-tific proposals into policy changes to be imple-mented or in progress (or reform policies) is aprocess involving much more than the actors will or even the technical quality of the scien-tific information recommending such a change,because ideological, political, and conjuncturalfactors are decisive for the proposals formulationand the courses of action chosen to implement it50,51, and the production of scientific knowledgemoves forward precisely by questioning earlierscientific truths, while concrete realities de-mand that the directions of any policy for changebe permanently reviewed and reformulated.Some authors thus warn that it may be easier tomeasure the extent of the changes produced orthe views of the strategic actors towards the usethat was actually made of the research results un-derpinning the change. They also propose thatthe category use should be replaced by influ-ence, i.e., actors ability or power to produce ef-fects in given areas.

In addition, programs or processes of changein the public health field or in health services

cannot be evaluated using the same technolo-gies normally used to evaluate medical care (e.g.,clinical trials), which are the basis for evidence-based medicine so in vogue in recent decades.That is, evaluation of evidence in public healthdepends on other analytical categories, whichHabicht et al. 52 and Victora et al. 53 call adequa-cy, plausibility, and probability, described as three types of scientific inference that are oftenused for making policy decisions in the fields ofhealth 53 (p. 400), because of the complexityof certain interventions and the ways by whichchanges influence the characteristics of the in-stitutions or populations under study, or vice-versa.

In summary, the literature review present-ed here confirmed our initial impression thatthere is an excessive formalization of instru-ments and pragmatic simplification in bothprocesses knowledge production and poli-cymaking in the health sector. Nonetheless,interesting progress can be seen in consideringthe use of research results for policy implemen-tation, particularly regarding the applicationof knowledge transfer-related ideas (diffusion,dissemination, and innovation) in the area ofhealth policy and systems research from a po-litical science perspective. Even so, there is stilla long road ahead in this theoretical reflection,and there seems to be a need for greater invest-ment in empirical research, which although notreproducible, does bring to bear elements ofthe concrete reality that help decipher the ac-knowledged complexity in the issue of the use

of scientific knowledge for policy formulationand implementation.

Resumen

Este texto presenta una revisin crtica de la produc-cin terica dedicada a la reflexin de la relacin entrela produccin del conocimiento cientfico y su utiliza-

cin en el proceso de formulacin e implementacinde polticas. Una extensa produccin acadmica haprocurado una sistematizacin de categoras y estra-tegias de transferencia de conocimiento con el prop-sito de mejorar la aplicacin del conocimiento cien-tfico, desde diversas perspectivas y distintos marcosde anlisis. Parte importante de esa reflexin abordael problema desde una perspectiva ms tradicional,que asume (explcita o implcitamente) una concep-cin de los resultados de la investigacin como unproducto acumulable, mientras el proceso decisoriose presenta en forma simplista y lineal, restringiendo

las estrategias a la adecuacin entre necesidades y losesfuerzos de la investigacin. Con nuevos abordajes, serevaloriza la complejidad de la creacin de polticas

y del proceso de produccin de l conocimiento para laaccin, integrados y explicados en un marco poltico einstitucional particular. A pesar de que la aplicacinde las ideas de transferencia de conocimiento aplica-da al campo de la investigacin en polticas y sistemasde salud genera algunos aportes interesantes, un largocamino de reflexin terica queda pendiente.

Poltica de Salud; Formulacin de Polticas; Uso de laInformacin Cientfica en la Tomada de Decisiones enSalud


12/13



Contributors

The two authors jointly developed this papers under-lying ideas and structure. C. Almeida conducted theliterature inventory and review, summarized the ma-terial, examined the theories and ideas, conceived thestudy structure, wrote the paper, and revised the finaldraft. E. Bscolo also searched the literature, summa-rized material, examined the theories and ideas, com-mented on and wrote in each successive version, andassisted in shaping the paper for publication.

Acknowledgements

The authors thank Patricia Pittman for her support insurveying the literature and for valuable suggestionson the first draft of the paper. Obviously, the authorsassume sole responsibility for the analysis and interpre-tation of the arguments presented here.

References1. White K, editor. Investigaciones sobre servicios

de salud: una antologa. Washington DC: Orga-nizacin Panamericana de la Salud; 1992. (Publi-cacin Cientfica, 534).

2. Meny I, Thoenig J-C. El marco conceptual. In: Me-ny I, Thoenig J-C, editors. Las polticas pblicas.Barcelona: Editorial Ariel; 1992. p. 88-108.

3. Greenberg JN, Choi T. The role of the social sci-ences in the health services research: an overview.In: Choi T, Greenberg JN, editors. Social scienceapproaches to health services research. Ann Arbor:Health Administration Press; 1982. p. 2-20.

4. Anderson OW. Influences of social and economicresearch on public policy in the health field: a re-view. Milbank Mem Fund Q 1966; 44:1-11.

5. Bice T. Social science and health services research:contributions to public policy. Milbank Mem FundQ Health Soc 1980; 58:173-200.

6. Mechanic D. Prospects and problems in healthservices research. Milbank Mem Fund Q HealthSoc 1978; 56:127-39.

7. Arellano OL. La selectividad en la poltica de salud.In: Laurell AC, editor. Nuevas tendencias y alterna-tivas en el sector salud. Mxico DF: Casa Abierta alTiempo/Fundacin Friedrich Ebert Stiftung; 1994.p. 33-60.

8. Almeida CM. As reformas sanitrias nos anos 80:crise ou transio? [Tese de Doutorado]. Rio de Ja-neiro: Escola Nacional de Sade Pblica, Funda-o Oswaldo Cruz; 1995.

9. Chernichovsky D. Health system reforms in indus-trialized democracies: an emerging paradigm. Mil-bank Q 1995; 73:339-72.

10. Lee K, Fustukian S, Buse K. An introduction toglobal health policy. In: Lee K, Buse K, Fustukian S,editors. Health policy in a globalising world. Cam-bridge: Cambridge University Press; 2002. p. 3-17.

11. Kralewski JE, Greene BR. Health services researchand the evolving health systems. In: Levey S, McCar-thy T, editors. Health management for tomorrow.Philadelphia: J. B. Lippincott; 1980. p. 293-307.

12. Trostle J, Bronfman M, Langer A. How do re-searchers influence decision-makers? Case stud-ies of Mexican policies. Health Policy Plan 1999;14:103-14.

13. Walt G, Gilson L. Reforming the health sector indeveloping countries: the central role of policyanalysis. Health Policy Plan 1994; 9:353-70.

14. Brown L. Knowledge and power: health servicesresearch as a political resource. In: Ginzberg E,editor. Health services research: key to healthpolicy. Cambridge: Harvard University Press; 1991.p. 20-45.

15. Reich M. The politics of health sector reform in de-veloping countries: three cases of pharmaceuticalpolicies. Boston: Harvard School of Public Health;1994. (Working Paper, 10).

16. Barreto ML. O conhecimento cientfico e tec-nolgico como evidncia para polticas e ativi-dades regulatrias em sade. Cinc Sade Coletiva2004; 9:329-38.

17. Weiss C. The many meanings of research utiliza-tion. Public Adm Rev 1979; 39:429-31.

18. Shadish WR, Cook TD, Leviton LC. Foundations ofprogram evaluation: theories of practice. Thou-sand Oaks: Sage Publications; 1990.

19. Kirkhart KE. Reconceptualizing evaluation use:an integrated theory of influence. In: Caracelli VJ,Preskill H, editors. The expanding scope of evalua-tion use. San Francisco: Jossey-Bass; 2000. p. 5-24.(New Directions for Evaluation, 88).

20. Henry GT, Rog DJ. A realist theory and analysis ofutilization. In: Henry GT, Julnes G, Mark MM, edi-tors. Realist evaluation: an emerging theory in sup-port of practice. San Francisco: Jossey-Bass; 1998.p. 89-102. (New Directions for Evaluation, 78).

21. Rich RF. Measuring knowledge utilization: pro-cesses and outcomes. Knowledge and Policy: TheInternational Journal of Knowledge Transfer andUtilization 1997; 10:11-24.

22. Patton M. Utilization-focused evaluation. New-bury Park: Sage Publications; 1997.


13/13



23. Forss K, Rebien CC, Carlsson J. Process use of eval-uations: types of use that precede lessons learnedand feedback. Evaluation 2002; 8:29-45.

24. Melo MA. As sete vidas da agenda pblica brasilei-ra. In: Rico EM, organizador. Avaliao de polticassociais: uma questo em debate. So Paulo: Cor-tez Editora/Instituto de Estudos Especiais; 1998.p. 11-28.

25. Lennarson-Greer A. Advances in the study ofdiffusion of innovation in health care organisa-tions. Milbank Mem Fund Q Health Soc 1977; Fall:505-32.

26. Bowen S, Zwi AB. Pathways to evidence-informedpolicy and practice: a framework for action. PLoSMedicine 2005; 2:600-5.

27. Bardach E. A practical guide for policy analysis: theeightfold path to more effective problem solving.New York: Seven Bridges Press; 2000.

28. Majone G. Evidence, argument and persuasionin the policy process. New Haven: Yale UniversityPress; 1989.

29. Davies H, Nutley S, Smith P, editors. What works?Evidence-based policy and practice in public ser-vices. Bristol: The Policy Press; 2000.

30. Nutley S, Webb J. Evidence and the policy pro-cess. In: Davies H, Nutley S, Smith P, editors. What works? Evidence-based policy and practice in pub-lic services. Bristol: The Policy Press; 2000. p. 13-4.

31. Haas PM. Introduction: epistemic communitiesand international policy coordination. Int Organ1992; 46:1-21.

32. Adler E, Hass PM. Conclusion: epistemic commu-nities, world order, and the creation of a reflectiveresearch program. Int Organ 1992; 46:390.

33. Albaeck E. Why all this evaluation? Theoreticalnotes and empirical observations on the func-tions and growth of evaluation, with Denmark asan illustration case. Canadian Journal of ProgramEvaluation 1996; 11:1-34.

34. Hanney S, Gonzalez-Block M, Buxton M, Kogan M.The utilisation of health research in policy-mak-ing: concepts, examples and methods of assess-ment. Health Res Policy Syst 2003; 1:2.

35. Pittman P. Allied research: experimenting withstructures and processes to increase the use of re-search in health policy. In: Global Forum for HealthResearch final documents [CD-ROM]. MexicoDF: Global Forum for Health Research; 2004.

36. Furtado JP. Um mtodo construtivista para aavaliao em sade. Cinc Sade Coletiva 2001;6:165-81.

37. Kothari A, Birch S, Charles C. Interaction and re-search utilization in health policies and programs:does it work? Health Policy 2005; 71:117-25.

38. Lomas J. Improving research dissemination anduptake in the health sector: beyond the sound ofone hand clapping. Ontario: Centre for HealthEconomics and Policy Analysis, McMaster Univer-sity; 1997.

39. Lomas J. Using linkage and exchange to moveresearch into policy at a Canadian foundation.Health Aff (Millwood) 2000; 19:236-40.

40. Lavis JN, Ross SE, Hurley JE. Examining the role ofhealth services research in public policymaking.Milbank Q 2002; 80:125-54.

41. Landry R, Amara N, Lamari M. Climbing the lad-der of research utilization. Sci Commun 2001;9:171-82.

42. Landry R, Lamari M, Amara N. The extent anddeterminants of utilization of university researchin government agencies. Public Adm Rev 2003;63:192-205.

43. Knott J, Wildavsky A. If dissemination is the solu-tion, what is the problem? Knowledge: Creation,Diffusion, Utilization 1980; 1:537-78.

44. Tangcharoensathien V, Wibulpholprasert S, Ni-tayaramphong S. Knowledge-based changes tohealth systems: the Thai experience in policydevelopment. Bull World Health Organ 2004;82:750-6.

45. Bronfman M, Trotsle J, Langer A. De la investi-gacin en salud a la poltica: la difcil traduccin.Mxico DF: Instituto Nacional de Salud Pblica/

Editorial El Manual Moderno; 2000.46. Sauerborn R, Nitayarumphong S, Gerhardus A.

Strategies to enhance the use of health systemsresearch for health sector reform. Trop Med IntHealth 1999; 4:827-35.

47. Rogers E. Diffusion of innovation. 3rd Ed. New York:Free Press; 1988.

48. Greenhalgh T, Robert G, MacFarlane F, Bate P,Kyriakidou O. Diffusion of innovation in serviceorganizations: systematic review and recommen-dations. Milbank Q 2004; 82:581-629.

49. Pellegrini Filho A. Ciencia en pro de la salud. No-tas sobre la organizacin de la actividad cientficapara el desarrollo de la salud en Amrica Latina y el Caribe. Washington DC: Organizacin Pana-

mericana de la Salud; 2000. (Publicacin Cientfica y Tcnica, 578).50. Pgo RA, Almeida CM. Teora y prctica de las re-

formas en los sistemas de salud: los casos de Brasil y Mxico. Cad Sade Pblica 2002; 18:971-89.

51. Pgo RA, Almeida C. mbito y papel de los espe-cialistas en las reformas en los sistemas de salud:los casos de Brasil y Mxico. Notre Dame: The Hel-en Kellogg Institute for International Studies/Uni-versity of Notre Dame; 2002. ( Working Paper, 299).

52. Habicht JP, Victora CG, Vaughan JP. Evaluation de-signs for adequacy, plausibility and probability ofpublic health programmes performance and im-pact. Int J Epidemiol 1999; 28:10-8.

53. Victora CG, Habicht JP, Bryce J. Evidence-basedpublic health: moving beyond randomized trials. Am J Public Health 2004; 94:400-5.

Submitted on 16/Mar/2006 Approved on 20/Mar/2006

Documents

Use of research results in policy decision-making,.pdf