Deconstructing knowledge brokering for commissioned rapid

RESEARCH Open Access

Deconstructing knowledge brokering forcommissioned rapid reviews: anobservational studyG. Moore1,2* , S. Redman1,2, P. Butow3 and A. Haynes1,2

Abstract

Background: Knowledge brokers are increasingly used by policy agencies, yet little is known about how theyengage with policy-makers and facilitate discussions with them about their research needs. This study examinesknowledge brokers’ behaviour in one-off interactions with policy-makers commissioning rapid reviews. It describeshow knowledge brokers engage with policy-makers, build trust and gain agreement about the review’s parameters.

Methods: We observed and transcribed 15 structured knowledge brokering sessions and used line-by-line analysisto derive, test and refine a coding schedule. The final coding schedule was applied to all transcripts. We assigned35 codes to three tasks identified in the data, namely eliciting information, exploring the policy context andnegotiating the content of reviews.

Results: The knowledge brokers we observed were skilled facilitators who built trust by their open stance, neutrality,and knowledge of research and policy contexts. Trust engendered an interplay of expertise in which review questionsand scope were clarified and contextual factors evaluated. Negotiation about the content of the review focused onunderstanding how it would contribute to the policy process, comparing options and assessing feasibility. Keyfunctions of knowledge brokers included eliciting and clarifying information, linking the review questions to thecontext and purpose, moving fluidly between policy and research perspectives, and weighing up review optionsagainst policy objectives. Four knowledge brokering roles were identified, namely diagnostic, facilitative, deliberativeand interpretative.

Conclusions: This study identified ways in which knowledge brokers established rapport with policy-makers whocommissioned reviews, enabled disclosure of essential information and explored contextual factors that affected thereview’s purpose and intended use. Knowledge brokers were competent in the discourse and conventions of bothpolicy and research and were skilled in negotiating complex policy and political environments, assisting policy-makersto evaluate options and craft a review proposal that was targeted, responsive and feasible. Mutuality, respect and aninterplay of expertise were integral to the knowledge brokering process. Future research might usefully examinewhether other rapid review programmes using knowledge brokers have similar results as well as the transferability ofthe four knowledge brokering roles to other contexts and settings.

Keywords: knowledge broker, knowledge brokering, rapid synthesis, rapid review, health policy, research utilisation

* Correspondence: [email protected] Sax Institute, PO Box K617, Haymarket NSW 1240, Ultimo NSW 2007,Australia2School of Public Health, Sydney Medical School, The University of Sydney,Edward Ford Building (A27), Sydney NSW 2006, AustraliaFull list of author information is available at the end of the article

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Moore et al. Health Research Policy and Systems (2018) 16:120 https://doi.org/10.1186/s12961-018-0389-7

http://crossmark.crossref.org/dialog/?doi=10.1186/s12961-018-0389-7&domain=pdf

http://orcid.org/0000-0002-2273-0296

mailto:[email protected]

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/publicdomain/zero/1.0/

IntroductionThere is now substantial literature documenting the bar-riers encountered by policy-makers who seek to access anduse research in decision-making [1, 2]. A number of strat-egies have been posited to address these barriers, includingincreasing access to timely, relevant research [1, 3–6],building capacity to access and use research [7–9], inter-action between policy-makers and researchers [10–13],and coproduction, in which both policy-makers and re-searchers jointly engage in the design, implementation andinterpretation of research [14, 15]. One strategy that isgaining currency is the use of knowledge brokers. Ini-tially described in broad terms [16], knowledge brokers’roles are commonly aligned with one of three maintypes of activity. These are knowledge management,where knowledge brokers develop systems and pro-cesses to access and disseminate research [17–19]; link-age and exchange, where they facilitate interactionbetween policy-makers and researchers [17, 20, 21];and capacity-building, where they provide individua-lised training and one-to-one support [18, 22–26].More recently, there have been attempts to articulate

and differentiate the ways in which knowledge brokersare used. For example, Dobbins et al. [22], among others,described knowledge brokers’ roles as conducting needsassessments, scanning the horizon, building capacity andsupporting organisational change. Dagenais et al. [18]pointed to the importance of brokers’ interaction withpolicy-makers and programme managers through formaland informal networks and their role in relaying infor-mation and new knowledge.Many of these roles seem to be based on an assumption

that knowledge brokers address a gap in policy-makers’ability to find and use research, for example, because indi-vidual policy-makers have insufficient skills or their orga-nisations lack capacity or because policy-makers andresearchers cannot communicate in a meaningful way.However, this assumption is now being challenged. In-deed, many policy-makers have a sophisticated under-standing of research [27] or are themselves researchersand, while policy agency capacity for research may be lim-ited in some instances, there are a number of modelsemerging which address this in considered ways [7, 28].This shift in emphasis points to a different kind of

knowledge brokering role that aligns more closely withnotions of shared expertise than with deficit models.Conklin et al. [17] reported their work in creating rela-tionships, promoting mutual understanding, and facili-tating exchange across social and cultural boundaries.As ‘one-among’ policy-makers and researchers and witha conceptual capacity to move freely within both, know-ledge brokers can engage at a deeper level than is typicalof other facilitative approaches. They are thereforeuniquely positioned to work with policy-makers rather

than stand between them and researchers, straddling adivide [21].When knowledge brokers have an in-depth understand-

ing of the policy domain, they can engage with the com-plexity inherent in policy processes where interests,ideologies, alliances and stakeholders compete [29, 30]. In-deed, policy processes are often contested and knowledgebrokers must be sensitive to the, at times, fraught issues,be able to establish trust, and work in a way that is re-spectful of these circumstances. Knowledge brokers’ activ-ities are therefore highly context dependent. The skillsthat are required in one context are likely to differ signifi-cantly from those needed in another and include facilitat-ing partnerships, clarifying research needs or supportingorganisational change. Therefore, an analysis of knowledgebrokers’ activities needs to be conscious of, and reflect, the‘real world’ environments in which they occur.To date, little information is available about knowledge

brokers’ practices or the mechanisms by which they work[25, 31]. This study therefore aimed to explore in depththe processes used by knowledge brokers and to shed lighton their practices and the ways in which they work, thusdrawing conclusions that may be broadly applicable toother knowledge brokers using similar models. To achievethis, we employed a case study approach and examinedthe operation of knowledge brokers in the New SouthWales Evidence Check programme, which supportspolicy-makers to commission rapid reviews. In thisprogramme, knowledge brokers work with policy-makersto specify the requirements of their review and draw up arapid review proposal that will inform the contracted re-searchers about the scope of work required.While there is little evidence of the effectiveness of

knowledge brokers overall [32, 33], knowledge brokersin the Evidence Check programme have been found tomake a significant difference to the clarity of review pro-posals from the perspective of representative reviewers,that is, previous authors of Evidence Check rapid re-views commissioned using knowledge brokers. Using asample of 60 rapid reviews, representative reviewerswere asked to assess the clarity of review proposals writ-ten before (n = 60) and after (n = 60) knowledge broker-ing while blind to the before and after status of eachproposal. Clarity pertained to the policy problem, the re-view questions, scope, methods and report format. Rep-resentative reviewers were also asked about theirconfidence that they could meet policy-makers’ needsbased on the information contained in the proposal. Theproposals were scored using a 6-point Likert scale andthe scores for all six questions showed significant im-provements following knowledge brokering [34]. More-over, a recent study examining whether rapid reviewscommissioned using Evidence Check were useful forpolicy-makers found that 89% of rapid reviews had been

Moore et al. Health Research Policy and Systems (2018) 16:120 Page 2 of 19

used by the agencies who commissioned them, in multipleand diverse ways [35]. More detailed information aboutthe Evidence Check programme is available elsewhere [36].In the Evidence Check programme, a knowledge broker

meets with the commissioning policy-maker(s) to under-stand why the review is being commissioned and how itwill be used, and to agree on the scope and parameters ofthe review, including what is feasible within the time frameand budget. The knowledge broker has to translate theseparameters in a way that will be understandable to re-searchers and without divulging any confidential informa-tion provided during the discussion with the policy-makers.This study focuses on this complex interpretative process.The specific objectives of this study were to analyse

the ways in which knowledge brokers (1) elicit and clar-ify information, (2) understand the context in which thereview will be used, and (3) gain agreement on the re-view’s final content.

MethodsEnrolment in the studyAll policy teams who commissioned a rapid review fromthe Sax Institute were invited to participate in the study.Enrolment continued until 15 policy teams were en-rolled. A sample size of 15 was chosen based on Guestet al.’s proposal [37] that 12 interviews are likely toachieve a representative sample.Three policy agencies declined to participate in rela-

tion to three knowledge brokering sessions due to thesensitivity of the subject matter; all participants providedinformed consent. The first knowledge brokering sessionwas held in April 2014 and the last in July 2015.Sessions were included in the study consecutively from

the commencement date. Independent of the researchprocess, knowledge brokers were allocated to sessionsaccording to their availability. No other criteria wereused; however, two Evidence Checks were judged to beof a level of complexity that required the input of moreexperienced brokers, thus specific brokers wereapproached to manage these Evidence Checks in accord-ance with the programme’s usual protocols.

Data collectionThe knowledge brokering sessions that formed the basisof this study were conducted face to face, with somemembers participating via teleconference, if needed. Pol-icy agencies were represented at sessions by at least onesenior staff member with responsibility for implementingreview findings and who could therefore make decisionsabout the review’s content. One of the Sax Institute pro-ject managers, who coordinate the review process, wasassigned to each review and attended the knowledgebrokering session.

The lead author (GM) attended all sessions and madeand transcribed audio recordings. Knowledge brokers andpolicy participants completed a standard post-sessionform developed and administered as part of the study(Additional file 1), focusing on changes in understandingabout the policy problem and about the way in which thereview questions were specified. Participants were askedabout the perceived value of the session and were invitedto add any other comments. All data were de-identified.To further de-identify knowledge brokers and partici-pants, we have sometimes altered the names of agenciesor the gender or numbers of participants in the examplestaken from the transcripts.After the sessions, policy participants and knowledge

brokers were contacted up to three times and invited tocomplete a survey. The survey included two free text de-scriptions of their perceptions of what had changed as aresult of the knowledge brokering session and any add-itional comments. Five questions using a 4-point Likertscale examined the degree to which respondents’ reportedchanges to their understanding of the policy problem,changes to the review questions, and the perceived im-portance of these changes. Respondents were also askedto what degree information needed to tailor the review topolicy-makers’ needs had been elicited in the session.

AnalysisGM analysed two transcripts using Charmaz’s groundedtheory approach [38] with word-by-word and line-by-line analysis as an orientation to the text of the tran-scripts and to ensure data coding was based on the dataalone. The results of the analysis were used to develop afirst coding schedule of 38 codes, which was reviewed bytwo authors (SR and PB). The codes were assigned toone of the three knowledge brokering tasks identified inthe data, as eliciting information, exploring the policycontext, and negotiating the content of reviews; thisstructure was used to report the results.The schedule was applied to two new transcripts, the

results were reviewed by SR and PB, and the schedulewas subsequently amended. This second schedule of 15codes was applied to five transcripts using NVIVO 11and 21 new codes were identified. The third and finalcoding schedule of 36 codes was then applied to all tran-scripts until thematic saturation was reached at theninth transcript and tested with the tenth; no newthemes were identified. GM prepared draft findingsbased on the 10 transcripts and the final five transcriptswere coded with a view to testing and amplifying thefindings. Two codes were merged during the analysisresulting in a final set of 35 codes. GM synthesised thedata from the transcripts and the findings were thenreviewed by all authors.


Early analysis suggested that variation in session out-comes was affected by differences in brokers’ levels ofknowledge brokering experience. Consequently, weassigned knowledge brokers to one of three categories ac-cording to their level of experience at the commencementof the study – those who had conducted 1–10 sessions,11–20 sessions, or 21 or more sessions were designated‘less experienced’, ‘experienced’ and ‘very experienced’, re-spectively. NVIVO 11 was used to collate codes occurringacross broker level of experience to explore commonalitiesand differences between their approaches. We also soughtto detail their specific activities in the knowledge broker-ing sessions.To assess the experience of policy participants in know-

ledge brokering sessions, we selected the ‘policy liaisonperson’ for each review to enable a comparison betweentheir experience and that of the knowledge brokers. Thepolicy liaison persons were not necessarily the most seniorpolicy-makers at a knowledge brokering session; however,they liaised directly with the Sax Institute and thereforereceived preliminary information about the roles of policyparticipants and knowledge brokers, the process of pro-posal development, and the conduct and publication ofthe review.We defined policy participants as having ‘some experi-

ence’ if they had attended one or more knowledge bro-kering sessions prior to the study, or as having ‘noexperience’ if they had not attended a session prior tothe study.

ResultsStudy sampleReviews and agenciesWe observed and analysed the interactions of 15 know-ledge brokering sessions concerning 15 Evidence Checkrapid reviews commissioned by six agencies, includingtwo large agencies with multiple divisions. One agencycommissioned five reviews, one commissioned four re-views, two agencies each commissioned two reviews,and two agencies each commissioned one review(Table 1).

SessionsOf the 15 sessions (one per review), 12 were held at thepolicy agency and three at the Sax Institute. All sessionswere conducted face to face, with one person participatingvia teleconference in each of three sessions. The averageduration was 66 min, ranging from 42 to 120 min.

Session participantsKnowledge brokers Five knowledge brokers conductedthe sessions included in this study. Of these, two werevery experienced (Brokers 1 and 3), one was experienced(Broker 5), and two were less experienced (Brokers 2and 4; Table 2). Four knowledge brokers were femaleand one was male. Four knowledge brokers held sub-stantive research positions and one held a senior man-agement position. Coding per broker level of experienceis reported in Table 3.

Policy participants There were 53 policy participants inthe study. The average number of policy participants ata session was 3.5, ranging from 1 to 7. Seven partici-pants were male and 46 were female. Fifteen policy par-ticipants were nominated as liaison persons in the study.One was nominated as policy liaison person for two re-views; this person has been counted once and the de-nominator reduced to 14 persons. Two policy liaisonpersons were male (14%) and 12 (86%) were female(Table 4). Further, three liaison persons (21%) had someexperience of a knowledge brokering session prior to thestudy. However, some participants attended more thanone session during the study period and therefore gainedexperience as they progressed. Two liaison persons(14%) attended two sessions during the study and oneperson (7%) attended three sessions during the study.

Responses to the survey questionsThe response rate for the knowledge broker survey was65% and for the policy participants 26%; therefore, the re-sults must be considered with caution (Tables 5 and 6).

Table 1 Distribution of sessions by type of agency

Agency type No. sessions

Regional statutory health authority 1 2



National non-government agency 1

Regional government agency (health) 4

National government agency (health) 1

15

Table 2 Number of sessions conducted by knowledge brokersbefore and during the study

No. sessionsconductedbefore the study

No. sessionsconductedduring the study

Coded inthe study

Knowledgebroker 1

32 7 Very experienced

Knowledgebroker 3

21 4 Very experienced

Knowledgebroker 5

11 1 Experienced

Knowledgebroker 2

3 2 Less experienced

Knowledgebroker 4

0 1 Less experienced


Overall, there was agreement among the knowledgebroker and policy respondents about changes to the re-view questions (100% and 91%, respectively) and the im-portance of these changes (100% and 85%, respectively).Most knowledge brokers and policy-makers said that theinformation needed to tailor the review to policy-makers’needs was elicited during the session (91% and 92%, re-spectively). Almost all knowledge brokers and policy par-ticipants (91% and 92%, respectively) said they found theknowledge brokering session valuable or very valuable.

Understandably, knowledge brokers and policy respon-dents differed in the degree to which their understandingof the policy problem changed during the knowledge bro-kering session (90% and 46%, respectively) and the im-portance of these changes (100% and 46%, respectively).Further, 46% of policy respondents said their understand-ing of the policy problem had not changed, compared to9% of knowledge brokers.To highlight similarities and differences in knowledge

broker and policy-maker responses, we provide

Table 3 Knowledge broker behaviours coded by level of experience

Knowledge broker behaviours Very experienced brokers Experienced brokers Less experienced brokers No consistent pattern

1 Asking closed questions Yes Yes Yes

2 Asking open ended questions Yes Yes Yes

3 Asking more detail Yes Yes Yes

4 Building trust Yes Yes Yes

5 Checking understanding Yes Yes Yes

6 Managing the session Yes Yes Yes

7 Diagnostics Yes Yes Yes

8 Gaining agreement Yes Yes Yes

9 Managing interludes Yes Yes Yes

10 Letting them talk Yes Yes Yes

11 Open dialogue Yes Yes Yes

12 Pointing to problems or solutions Yes Yes Yes

13 Positive regard Yes Yes Yes

14 Referencing the reviewer Yes Yes Yes

15 Reflecting back Yes Yes Yes

16 Filtering Yes Yes

17 Frank disclosure Yes Yes

18 State re-state Yes Yes

19 Understanding context Yes Yes

20 Using humour Yes Yes

21 Negotiating Yes Yes

22 Setting expectations Yes Yes

23 Explaining Yes Yes

24 Power or ownership Yes Yes

25 Being neutral Yes

26 Personal anecdote Yes

27 Combining expertise Yes

28 Frame-re-frame Yes

29 Interplay of expertise Yes

30 Highlighting inconsistencies Yes

31 Accord and discord Yes

32 Adding value Yes

33 Testing constructs Yes

34 No trust Yes

35 Using examples Yes


examples taken from three knowledge brokering ses-sions (Box 1).

Identification of knowledge broker tasksWorking inductively from our analysis of the data, weidentified three key tasks for knowledge brokers, namely(1) eliciting and clarifying information, (2) exploring theunderlying contextual issues and (3) gaining agreementabout the content of the review. The kinds of activitiesknowledge brokers used in enacting these tasks are sum-marised in Box 2. Across these activities, we found thatbrokers enacted roles that were facilitative, diagnostic,interpretative or deliberative (Table 7). While more thanone role may have been operational at different pointsduring the sessions, we found that they played outroughly in the order in which we have described them,i.e. in the early parts of a session brokers used mostlytheir facilitative and diagnostic roles and in the laterparts of a session, they moved more into their interpret-ative and deliberative roles.

Task 1: Establishing trust and eliciting informationMutual trust and respect have been acknowledged ascritical in establishing and sustaining genuine partner-ships [14, 39]. Yet, knowledge brokers and policy-makersin this study had little time available for the kinds of ex-tended interaction that enable a recognition of thestrengths and expertise in the other. Further, when trustwas not formed, knowledge brokers found it difficult toelicit the kinds of confidential contextual informationthat would enable them to fully understand how thefindings would contribute to a particular process

alongside other inputs to the policy process. Withoutthis understanding, the broker was less able to providethe needed information for the researcher who wouldundertake the review. Trust, therefore, needed to beestablished quite quickly at the commencement of thesession.

Creating an open dialogueIn their facilitative role, knowledge brokers’ initial ob-jective appeared to be to create conditions where anopen dialogue could take place and a sense of safety inwhich the policy team could speak freely about their pol-icy context and the issues at hand.Brokers differed in the way they engaged, and later dis-

engaged, with policy-makers; however, there were com-mon elements in the content of brokers’ openingdiscourse. They acknowledged policy-makers’ work inpreparing the commissioning tool (the form thatpolicy-makers complete describing what they want froma rapid review) as a starting point for the discussion.Most brokers pointed to their own experience as a bro-ker or as a researcher and to their preparation for thesession, such as reading background information aboutthe agency and the review topic, scanning the literatureand thinking through the review questions. They af-firmed their responsiveness to policy-makers’ needs inensuring the review would be fit for purpose. Experi-enced brokers would communicate their neutrality interms of the content of the review:

Broker 5: “My job is not to tell you to do A versus B,I am absolutely agnostic about that. My job is to help

Table 4 Number of sessions attended by policy liaison people before and during the study

No. sessions attended before the study No. sessions attended during the study (in any role) Coded in the study

Policy liaison person 1 0 1 No experience

Policy liaison person 2a 1 3 Some experience




Policy liaison person 6a 1 3 Some experience







Policy liaison person 13 1 1 Some experience


Policy liaison person 15 1 1 Some experienceaPolicy liaison persons 2 and 6 are the same person who has been counted once


you to get what you want out of this evidence review,and basically to make sure that when I write thosetwo, three or four pages for the researchers to look at,they understand what it is that you want.”

Others openly acknowledged the uncertainty policy-makers often felt at the beginning of the session anddescribed the shared nature of that experience. This mu-tuality was characteristic of all sessions and would be anessential aspect throughout the brokering process.

Broker 1: “Let me just assure you that at the end ofthis my brain hurts as much as yours and I think the

tag line for this is that there is no such thing as adumb question or a dumb answer!”

Control of the session varied across participant groupsand could devolve during the session. Experienced bro-kers who took an overt leadership role found it easier tomanage the progression of the session, but somepolicy-makers assumed control. This may relate to thepolicy-maker’s perception about the sensitivity of thesubject matter, to a real or perceived lack of skill on thepart of the knowledge broker, or to other factors such asthe seniority or personality of the policy-maker. Somediscomfort about who was directing the discussion was

Box 1 Examples of knowledge broker and policy-makers’ free text responses

EXAMPLE 1

Knowledge broker

• “[I have a] better understanding of the relative importance of the questions and a better understanding of the practical context in which the

review findings will be used. There were differences in focus amongst the agency representatives (micro vs. bigger picture focus) but there was

overall concordance about the purpose of the rapid review.”

Policy participants

• “Clarification of the questions, scope: these were not major changes, but the questioning allowed us to be more specific about our needs and

to flag some additional information we could provide as background.”

• “It really helped us to clarify what we wanted and to emphasise that whatever [the reviewers] would find would be really useful. I think that

[the broker] gained a better understanding of our requirements, which is actually a relief. It fills me with more confidence that the [reviewers]

will provide us with what we are after.”

EXAMPLE 2

Knowledge broker

• “[I have] a better understanding of the dynamics of the group involved in the session. It was apparent that there was no group consensus

about what was required from the outcomes of a review. This was a difficult session characterised by a limited comprehension of what a

rapid review can and can’t deliver.”

Policy participant

• “I’m not sure the process seemed to clarify things as I expected. The knowledge broker still seems to think that the required review is of a

greater magnitude than I or my colleagues do. It makes me think that the needs are actually still very unclear. I need to put more thought

into exactly what we are asking for, as after the session it felt that things were very ‘woolly’.”

EXAMPLE 3

Knowledge broker

• “The agency appreciation and understanding of the nature, value and process of an independent evidence check [changed]. [There was a]

broadening of agency focus about the reason for the Evidence Check, i.e. not focused on ‘is our programme evidence based?’ A very pleasant

and constructive group to work with, who engaged well with commissioning tool. It really makes a positive difference when the agency is

more interested in doing something thoroughly rather than quickly!”

Policy participants

• “I feel that the brokering session allowed the development of a more detailed and research focussed approach. As someone with limited

research experience, our questions had possibly lacked consideration of the practical process of undertaking research. The knowledge broker’s

input strengthened the independence of the proposal by removing some of the inherent bias of the questions towards our expected outcomes.”

• “The knowledge broker’s external opinion allowed the research questions to be viewed in a slightly different way. I felt that the knowledge

broker was friendly and approachable, and therefore I felt comfortable in challenging some of her ideas where they didn’t fit with our needs

and accepting others.”


common at the beginning of most sessions and the moreexperienced brokers managed this by addressing it expli-citly, negotiating temporary leadership.This power transaction seemed intricately linked to

the establishment of trust, which was not a given, buthad to be won in the early exchange. On a couple of oc-casions, policy-makers asserted their ownership of thecontent of the review – seeking explicit control of thesession may have been a function of their uncertaintyabout the process or perceived loss of authority as thebrokers took leadership. Therefore, brokers often openedwith clear statements about their role in guiding the ses-sion and were explicit about policy-makers’ authority interms of the review’s content.

Broker 3: “Nothing goes anywhere until you haveapproved it. Then the Sax team does their part andtries to attract the right person for the job and you geta say in that as well. So, things don’t happen withoutyour consent.”

Understanding the purpose of the reviewOne of the ways brokers gained trust was by signallingtheir familiarity with the policy context relevant to the re-view in question, and they did this in often quite sophisti-cated and nuanced ways. One experienced broker touchedon the extent of her knowledge by referencing a nationalguideline on an agency’s website that had been recentlyrescinded and pointing to a parallel change in that

agency’s policy direction. Another broker referenced aconceptual framework frequently utilised in the literatureon the specified topic, using its components to structureher questions to the policy team; they immediately recog-nised and responded to the issues she raised. In all butthree of the sessions, the brokers pointed to their know-ledge of the policy context and of the issues likely to occurin a particular health system or policy process.The shared recognition of the complexity of the policy

context enabled the dialogue to move beyond the policy’sdevelopment to the underlying issues that might influenceits direction. In their diagnostic hat, experienced brokersworked to unravel and understand these underlying fac-tors and establish their relationship to the proposed re-view. This kind of deeper exploration only occurred whentrust was established in the early interaction; indeed, it ap-peared that sensitive information was often revealed pre-cisely at the point at which policy-makers determined thatthe broker could be trusted.In one session, a policy-maker was describing the fac-

tors that contributed to staff in emergency departmentsfailing to implement a policy directive when a relativelyrare clinical event occurred. As she described the situ-ation, she broke off and, turning to the broker, showedher discomfort in disclosing information which the bro-ker might perceive in a negative light. The broker imme-diately indicated her understanding of the policy-maker’sdilemma, calling on an experience in which she herselfhad been vulnerable. The exchange, while brief, seemedquite personal, but once the broker had signalled herunderstanding of the circumstances, the policy-makerwas able to continue her analysis of the causes of thepolicy’s failure.

Policy participant 4: “I feel I need to say that it’s ahighly sensitive topic – [You] do understand thesensitivity of what we are talking about.”

Broker 1: “Yes. As I was reading the policy directive,I remembered that as a very junior doctor working in[an emergency department] … you never forget it.”

The broker’s capacity to respond empathically andnon-judgementally both reassured the policy participantand facilitated the smooth progression of the session.In some instances, less experienced brokers moved

directly to clarifying the review questions without ex-ploring the background to the review and the policycontext or without establishing rapport. As a result, par-ticipants received no direction about the process orstructure of the session and no guidance about the issuesto be addressed. The brokers may have assumed that theinformation in the commissioning tool was sufficient orthe value of the early exploration may not have been

Box 2 Knowledge broker activities

Knowledge brokers elicited information by using a range of types

of question (open, closed, hypothetical), drawing out more detail,

using examples to clarify meaning and checking the definition

of key terms. Careful management of the session ensured that

information needed by the brokers was captured and their

understanding of what was required was tested, refined and

confirmed.

Knowledge brokers explored the underlying contextual factors

by creating conditions that enabled an open dialogue, coming

to a shared understanding of the policy environment (including

sensitivities that could not be shared with the researchers), being

attentive to the range of viewpoints from which a problem could

be articulated and evaluating contextual information in light of

the policy objectives.

Knowledge brokers negotiated options for the content of a review

by aligning them with the policy purpose and context, identifying

critical perspectives, highlighting inconsistencies, articulating

alternative review question sets and confirming agreement of the

final review parameters.


Table

5Kn

owledg

ebrokers'respon

sesto

survey

questio

ns

KBSa

No.

Did

theKBSchange

your

unde

rstand

ingof

theprob

lem?

Was

thischange

inun

derstand

ingim

portant?

Did

theKBSchange

theway

the

research

questio

nwas

specified

?Was

thischange

inresearch

questio

nim

portant?

Was

theinform

ationne

eded

totarget

thereview

tothe

policy-makers’ne

edselicited

inthesession?

Overall,didyoufindthe

KBSvaluable?

Question2

Question3

Question4

Question5

Question6

Question7

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

10

10

01

00

01

00

01

00

01

00

01

00

0

20

10

01

00

00

10

00

10

01

00

01

00

0

50

10

01

00

00

10

00

10

01

00

01

00

0

60

01

01

00

01

00

01

00

00

01

00

01

0

7ab

10

00

10

00

10

00

10

00

10

00

10

00

7bb

10

00

10

00

01

00

10

00

10

00

10

00

7cb

10

00

10

00

01

00

10

00

10

00

10

00

80

10

00

10

00

10

00

10

00

10

01

00

0

91

00

01

00

00

10

00

10

01

00

01

00

0

100

10

00

10

00

10

00

10

00

10

00

10

0

141

00

01

00

01

00

01

00

00

10

01

00

0

n5

51

09

20

04

70

06

50

07

31

09

11

0

%45

459

082

180

036

640

055

450

064

279

082

99

0a KBS

know

ledg

ebrok

eringsession

bIn

thissession,

threerelatedrapidreview

swerekn

owledg

ebrok

ered

andthebrok

erprov

ided

respon

sesforeach

one


Table

6Po

licy-makers’respon

sesto

thesurvey

questio

ns

KBSa

No.

Policy

participant

no.

Did

theKBSchange

your

unde

rstand

ingof

the

prob

lem?

Was

thischange

inun

derstand

ingim

portant?

Did

theKBSchange

the

way

theresearch

questio

nwas

specified

?

Was

thischange

inresearch

questio

nim

portant?

Was

theinform

ation

need

edto

target

the

review

tothepo

licy-makers’

need

selicitedin

thesession?

Overall,didyoufind

theKBSvaluable?

Question2

Question3

Question4

Question5

Question6

Question7

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

Yes,

alot

Yes,a

little

No,no

tat

all

Don

’tknow

11

01

00

01

00

01

00

01

00

10

00

10

00

12

01

00

01

00

00

10

00

00

10

00

10

00

37

01

00

01

00

01

00

01

00

01

00

01

00

311

00

01

00

01

01

00

01

00

01

00

01

00

412

00

10

00

00

01

00

10

00

10

00

10

00

413

00

10

00

10

01

00

00

01

10

00

10

00

515

01

00

01

00

01

00

01

00

01

00

01

00

619

00

10

00

00

10

00

10

00

00

01

00

01

724

00

10

00

00

01

00

01

00

01

00

10

00

830

00

10

00

10

10

00

01

00

01

00

10

00

832

01

00

01

00

01

00

10

00

10

00

10

00

933

00

10

00

10

01

00

01

00

10

00

10

00

934

01

00

01

00

01

00

10

00

10

00

10

00

n0

66

10

63

12

101

04

70

17

50

19

30

1

%0

4646

80

4623

815

778

031

540

854

380

869

230

8a KBS

know

ledg

ebrok

eringsession


evident. In addition, participants themselves may havebeen keen to move to the more tangible questions whichseemed to address their policy questions in more con-crete ways. Without a clear understanding about howthe review would be used, brokers found it difficult tonegotiate what would be in or out of the review, andthere was no central focus against which the relevanceof options could be evaluated.In one session, the policy team sought evidence about

prevention in two disease groups with very different tra-jectories and bases in the evidence; with no explorationabout the policy background, the underlying differencesbetween the two main questions were never made explicit.The broker’s questions elicited contradictory responsesand participants were unable to agree on the review ques-tions and scope; the broker was unable to reconcile theseinconsistencies. Because the background to the reviewwas not discussed and the differences in the two ap-proaches to prevention were not explored, assessing theextent to which both could be encompassed in a single re-view became difficult. The broker addressed this to a de-gree in the scope, creating sub-questions relevant to eachspecialty; however, without any clear delineation, the re-view questions became complex and unwieldy.

When sensitive information is withheldWhen sensitive information that could have progressedunderstanding was not disclosed, brokers were left to inferthe focus for the review, based on background informationprovided by the policy-makers in advance of the sessionand on their own knowledge of the policy process.In one review, there was little interaction at the beginning

of the session and little rapport among the policy partici-pants, many of whom had not met before. In the openingminutes of the session, a senior clinician who was part ofthe commissioning team asked the lead policy-makerwhether she could talk about the adverse events that trig-gered the review. The policy-maker said she could not.The knowledge broker accepted that full information

about the problem would not be provided but struggledto reconcile the multiple viewpoints among the policyteam about what was wanted from the review and shehad limited ability to define the research question and

the scope of the review appropriately. Consequently, thebroker drafted a review proposal based on the commis-sioning tool with little information about the actualneeds of the policy team.An absence of interaction and rapport that was obvious

at the beginning of the session may have accounted forthis non-disclosure, at least in part. It may have reflected alack of skill on the part of a less experienced broker in fa-cilitating trust, or the lack of established relationshipsamong the policy participants, some of whom were meet-ing for the first time; however, it may also have reflectedthe lead policy-maker’s concern about tabling for discus-sion an adverse event not yet in the public domain. Never-theless, in most instances, policy-makers readily disclosedsensitive information, allowing the broker to quicklybecome cognisant of the complexities underpinningthe review.

Task 2: Clarifying research needs and understanding contextOne of the most important and complex tasks for know-ledge brokers was to understand the perspective fromwhich the policy-maker viewed the policy problem. Theconcepts and linguistic constructs used by policy-makersseemed to differ quite radically at times from those of re-searchers; that is, the same word could have a particularand shared meaning for policy-makers and have multipleand diverse meanings for researchers. Therefore, for theknowledge brokers, understanding exactly what a policyteam wanted the researchers to focus on required carefulexploration. Implicit in the dialogue was the movement bythe knowledge broker to and from policy and researchperspectives until what was needed was clearly definedand could be conveyed to the researchers who wouldconduct the review. This translational capacity wascharacteristic of experienced knowledge brokers. Tobegin, however, brokers drew on their diagnostic role.

Exploring contextual factors influencing the reviewMost knowledge brokers began by eliciting informationabout the key events that led to the commissioning of thereview, using broad statements that allowed policy-makersto engage in the process and provide information aboutthe policy’s background.

Table 7 Knowledge broker roles

Facilitative role Diagnostic role Interpretative role Deliberative role

• Establishing and maintainingtrust and creating an opendialogue

• Facilitating understandingacross social and culturalboundaries

• Eliciting disclosure ofsensitive information andgaining greater specificity

• Clarifying research needs for aspecific policy process

• Identifying contextual factors thatmay influence research use

• Refining research questions andscope to reflect purpose andcontext

• Moving between and aligningresearch and policy perspectives

• Communicating in the languageand conventions of both policyand research domains

• Conveying what is meant andintended in one domain so itcan be understood and enactedin the other

• Weighing up research optionsin terms of their contributionto a policy objective

• Surfacing inconsistencies orconflicting viewpoints forresolution

• Fostering judgements thatenable action anddecision-making going forward


Broker 5: “I think a good way to start would be for youto tell me a little bit about why we are here. Whatprompted this request? Why now? Because the first thingI am going to do is write a few paragraphs explaining alittle bit about the background and context, for theresearchers.”

Policy-makers were on home ground here and couldreadily provide the kinds of descriptive detail that wouldhelp brokers understand what was required and wouldalign the review to the policy context; however, somepolicy-makers were unsure about exactly what informa-tion the broker needed. Brokers used open-ended ques-tions to explore some of the key issues, such as how thereview would be positioned and the roles of stakeholdersin the policy process.

Broker 3: “So, one of the things I noticed about thecommissioning tool is that you are looking for a reallypractical piece of work that will be used in the real world.Who are these ‘real-world’ people that you had in mind?”

Policy-makers’ narratives were sometimes dense andmulti-layered; in describing the context in which a par-ticular policy decision was unfolding, information wasdisclosed that was not pertinent to the review’s contentor direction. Brokers therefore needed to filter the infor-mation, allowing what was disclosed to stand, but bring-ing the discussion back to the review’s main focus. Thiscould be a delicate task as brokers responded to subtlecues, drawing out what was implicit in what was beingsaid, so that its importance in a particular process couldbe explored and evaluated:

Broker 3: “Now, one of the things I was intrigued about,it says [in the commissioning tool] ‘there is a draft policydeveloped but not progressed’. So, I was wonderingwhether you could sort of fill me in on that? What’s goingon there and why it hasn’t progressed? It could be justsome common-sense reason like lack of time, but therecould be other things as well.”

When needed, knowledge brokers prompted policy-makers to provide more detail, until they were comfort-able they had a clear understanding of what wasrequired and how the review findings would be used inthe particular policy under discussion.

Evaluating contextual informationContested policy environments were a common under-lying concern that policy-makers were sometimes hesi-tant to disclose. Brokers had to find a way to exploresensitive political issues and determine their relationshipto the review in order to clarify the review questions and

scope. One participant expressed the difficulty of negoti-ating an outcome in a fraught political environment.

Policy participant 19: “This is a multi-stakeholderpolicy and there are a lot of very strong supporters.It’s going to be very tricky in terms of who is seen tohave gotten their own way at the end of it.”

One review looked at a multi-agency response to themanagement of an issue involving policy agencies, com-munity services, specialist clinicians and hospital pro-viders. Because the agency facilitating the policy’sdevelopment was a key player in this response, thepolicy-makers were keen to establish their impartialityand avoid a perception of bias towards a particular solu-tion. Further, while the heads of agencies supported ajoint response, there was disagreement among the policyparticipants about how this should play out in practice.The review question reflected something of this ambigu-ity and the broker needed to clarify the premise onwhich the question was based.

Broker 3: “Is there an acceptance that the response isalways going to be multi-agency?”

Policy participant 4: “No, but that’s what we support.”

After further discussion, the broker included some ofthe contested issues in the scope of the review, askingthe reviewers to highlight contextual information thatwould help policy-makers assess the applicability of thefindings, such as information about jurisdictions, agencyroles, governance structures, and barriers and enablersto implementation. While the policy team’s concernabout their perceived partiality was not in itself relevantto the review, the underlying controversial managementissues operating in the background that gave rise to theirconcern, were both pertinent to and appropriate for con-sideration in a rapid review.In their diagnostic role, understanding the underlying

issues enabled brokers to focus the review questions andscope in a way that would be consistent with theagency’s goals. In one session on clinical variation, thebroker proposed outcome measures that would allowthe policy team to advocate for the needs of children inthe adult clinical variation space, as one of the policyteams’ main concerns was to address the assumptionthat solutions to variation in the adult domain couldautomatically be applied in the paediatric domain. Intaking this approach, the broker was also able to reflecther understanding of the policy-makers’ experience.

Broker 1: “You’ve got to understand the particularsof clinical variation in the paediatric space and if


necessary be a strong advocate. Instead of saying as isquite often the case ‘well that really worked well inadults, we will just make it a quarter size’.”

In several sessions there was an unspoken expectationthat research could play a role in resolving an impasse inthe policy process. In one instance, the policy teamwanted to pre-empt a situation where time would be lostin debate between the conflicting views of members of asteering committee about the policy’s direction, whichthe commissioning team predicted would be difficult toresolve. Commissioning a review that examined the evi-dence for the several issues that the policy would ad-dress could both establish the evidence base and set theagenda in a neutral way; however, when the questionswere too many and varied to be addressed in a single re-view, the need for the evidence to inform the policy andprovide a tactical solution meant that policy-makers haddifficulty prioritising among review questions.Exploring the contextual elements enabled the broker to

identify the underlying obstacles and table them for con-sideration. In this instance, the policy participants werethen able to work out a process that allowed them to drawon the committee’s expertise in a neutral way, and the ses-sion could then address the content of the review itself.

Defining the scope of the reviewOnce the knowledge brokers had sufficient informationabout the problem and policy context, they moved to amore diagnostic approach and, using a question and an-swer modality, tested the meaning of the proposed reviewquestions, ruling material in or out of scope, constructinga research question set that was clear and delineated, all ofwhich together made a cohesive whole. Wearing theirdiagnostic hat, brokers checked the alignment of the ques-tions with the policy-makers’ opening narrative, refiningthe language and structure until the question set was de-termined. In this example, Broker 1 and Policy participant38 were specifying the scope of the review.

Broker 1: “Now, in terms of ambulatory care are wetalking about primary care, GPs?” Policy participant38: “No.” Broker 1: “Are we talking about patients?”Policy participant 38: “Yes.” Broker 1: “Are communityservices in your ambit?” Policy participant 38: “I don’tthink so, no.” Broker 1: “So basically outpatients, ED,and admissions are the ones. Ok, alright, that’s veryhelpful.”

A common difficulty was the definition of key terms.Brokers sought to identify the meaning of words andphrases that underpinned the conduct of a review, andrule aspects of these broader concepts in or out of scope.Examples included ‘model of care’, ‘complex systems’, ‘best

practice’, ‘effectiveness’, ‘large-scale change’ and ‘chronicdisease’. Other definitions focused on the specifics of pop-ulations and settings such as ‘home care’, ‘school-based’, ‘in-patient’ or ‘adolescent’. As one broker commented:

Broker 1: “It’s interesting, the use of language.For example, what is ‘effective’ to one person, theiridea of effectiveness, is different to another person.So, I have got to understand what you need.”

Clarifying the intent of the questionKnowledge brokers seemed to find delineating thescope of work relatively straightforward once keyterms were defined and other basic parameters of thereview decided, such as range of publication dates orthe countries from which literature should be drawn.Sometimes, however, understanding what was neededproved more elusive; policy-makers had a clear ideaof what they wanted but could not communicate thisin a way that the knowledge broker could convey topotential reviewers.One review looked at the use of surveys to capture patient

experience. The policy team described the review’s purposeand questions in the commissioning tool given to the know-ledge broker ahead of the session. Their first question was“What are the best ways to assess patient experience?”The question could be understood from a number of

perspectives and each could be a valid and useful ap-proach for a review. In their diagnostic role, brokerswould test the intent of the question and align it with thepolicy-makers’ purpose, delineating the perspectives in away that made their differences clear to the policy team.

Broker 1: “So, I was just interested in the meaning ofthat [question]. Is it to compare what you do [withother agencies], so literally, ‘What we do, what doother people do?’ Or is there a best practice element tothis, ‘What aren’t we doing that we should be doing?’Or are you specifically wanting to harmonise withwhat’s happening in other places?”

The answer would take the reviewers down quite differ-ent pathways each looking at the question from a particularangle, for example, a best practice study of survey design, astudy of the effectiveness of implementation strategies, or astudy of comparative practice, identifying common ele-ments across surveys. While more than one question mayfeasibly have been answered in the given timeframe, thecritical perspective is the one which determines the direc-tion the review will take, namely the logic or line of reason-ing against which the search strategy is constructed, theanalysis undertaken and the results interpreted.Policy-makers often found it difficult to pinpoint the

critical perspective for themselves. In one review on


gendered approaches to interventions for men’s mentalhealth, the policy-makers described the scope of inter-ventions as a continuum from general health and well-being to diagnosed mental illness. To narrow the scopefor the potential reviewers, the broker had prepared alist of ‘categories’ or groups of men who might be ruledin or out of the review such as returning servicespersonnel and elite athletes. The policy-makers insistedthat the review should include all categories and the bro-ker had to explore what they needed in quite a detailedway, before she captured their intent.

Broker 3: “It sounds like you are talking abouthelp-seeking. So it’s not necessarily people who arediagnosed with clinical depression not adhering to aprogramme. This seems to be about that first point ofcontact, and what is a barrier to that, and what helps,and how men recognise it.”

Policy participant 15: “Yes, that’s right!”

The review questions and scope, therefore, took aquite different direction from that prepared by theknowledge broker ahead of the session. Because she wasattentive to perspectives on the problem that had notbeen articulated in the commissioning tool, the brokerwas able to test these with the policy-makers until thecritical perspective was identified and confirmed.

Conveying the researcher’s perspectiveIn their translational role, knowledge brokers explainedproposed changes to the wording of review questions,often positioning the change from the perspective of theresearchers who would undertake the work, so that thepolicy-makers could see the review as these researchersmight see it.

Broker 3: “Not because your words are not good, yourwords are terrific, but they may not be understood sowell by a research team.”

This had a two-fold effect. It explained the reason forthe broker’s proposed changes to the questions andscope and situated the changes in the context of the re-viewer’s task. It also pointed to the quite different re-quirements of a research review process, compared tothose of a policy process.Further, brokers explained the proposed changes in

terms of their benefit to the policy objectives and in lan-guage that was accessible to the policy-makers them-selves. One review was designed to clarify the evidenceabout the effectiveness of a programme that the agencywas implementing, which had been challenged by anumber of stakeholders. The commissioning team

wanted to see how the evidence would align with thecomponents of their programme, and the broker saw arisk in their proposed approach.

Broker 3: “What we need to do is just turn [the question]around so that the reviewer is being asked to do a reviewof a style of programme, rather than the beginning pointbeing your actual programme and then finding evidenceto fit it. Does that make sense? There is a risk of biasthere. You want this review to be seen as independent.‘It’s hands off. We commissioned this review and,independently, this is where the programme stands inrelation to the evidence’.”

It is possible that the policy-makers in this examplewere not familiar with the question of bias that the bro-ker raised, but they could respond to the advantage ofan independent review within a contested environment.

Task 3: Gaining agreement about contentPolicy-makers often found it challenging to narrow thescope of the review, despite a mismatch between whatthey wanted and the limitations of their timeframe andfunding. In addition, some policy participants had a lim-ited understanding of what a research review entailedand therefore of what was possible within a given time-frame. In their diagnostic hat, brokers needed to deter-mine what was feasible within the agency’s constraintsand facilitate a process where policy-makers could makejudgements about what information would be of greatestvalue for a particular policy objective.

Resolving inconsistencies and enabling decisionsOne of the first common tasks brokers encountered wasmanaging diverse perspectives within the policy teamabout the main focus of the review. While the selectionof policy-makers at the session may have been purpose-ful in terms of the policy task, knowledge brokers couldnot assume that they were of one mind.In one review, the policy team had identified a

number of issues their policy would address, namelydevelopmental outcomes, school readiness, models ofcare, workforce roles, technological advances, and costeffectiveness, but time and funding were limited. Thebroker reflected on what she saw as their central con-cern quite early in the discussion, but they continuedto voice the range of issues. The broker’s approachwas to let them ‘talk it out’, allowing them to cometo an understanding about the size of the task andthe breadth of their perspectives. She was then ableto point to what seemed like the two strongest op-tions for the focus of the review, and so narrowedthe discussion:


Broker 1: “So, there are two quite separate directionsI think. [They are] in the same content area broadly,but there is really something around new andemerging areas; and there is something that is muchmore around the models of care and implementation.”

The process was more straightforward when policy-makers had a shared understanding of the focus of thereview before the session commenced. The broker’s taskthen was to confirm rather than negotiate the centralfocus, ensuring that there was a clear and agreed basisagainst which decisions were to be made.In some instances, less experienced brokers seemed to

assume that consensus was in place. One broker statedat the beginning of the session that the main focus wasclear, but did not check her understanding, possibly as-suming that the matter had been agreed ahead of thesession. The lack of explicit agreement meant that, ateach stage of the session, the different perspectives be-came apparent and either agreement was sought at eachinstance, or the senior participant made a decision aboutwhat was or was not to be included in the review. Thiscreated a situation where decisions at any one pointwere not aligned to a central focus, and introduced in-consistencies into the questions and scope. Interestingly,one participant raised the question about the review’smain focus, but his fellow participants could not agree.The broker may have resolved any inconsistencies indrafting the proposal, but several points remained unre-solved at the close of the session.

Weighing up options and fostering judgementsThe most common issue brokers negotiated was thechoice between two review directions that appeared tobe of equal value for the commissioning team, but whichtogether made the review either too large for the avail-able timeframe or funding, or which required distinctkinds of research expertise, so that more than one re-view team would be needed to answer the questions.Policy participants did not find this negotiation easy.

They were keenly aware of the size and complexity ofthe policy problem and that narrowing the scope of thereview to ensure feasibility inevitably meant that not allissues would be addressed. The knowledge brokersneeded to articulate the problem for the policy team andpoint to potential solutions. One approach was to workwith them to prioritise the questions, with a view to ex-cluding those deemed less essential.

Broker 1: “It may be that you feel you have particularquestions around the components of the model and theimplementation of it, you know. Are there reallyimportant developments around that that you want quitea lot of information about? Or is your greatest concern

that there are risk factors that are now well understoodthat were really only in their infancy then? Because eachof these are potentially really quite big questions.”

A second approach was to prompt the policy teamto evaluate which approach would provide best valuein their context. This approach required careful man-agement by the brokers who were experts neither inthe content area nor in the exigencies of the policycontext.A third approach was to answer the policy questions

in a staged way and identify which evidence from re-search would be needed first. This option focused on theunfolding policy process and the likely timing of theneeded information.

Broker 1: “Sometimes in the past it has actually beenmore helpful to use a more staged approach. You mayalready have an idea out of all of these questions what isthe most important thing that you need to know now. If Isaid to you, ‘OK, let’s just say there are three months’, whatwould you pick as the thing that you most need done inorder to inform the activities of your steering group?”

In developing options, knowledge brokers also drewon approaches beyond a standard review tailored to theneeds of the agencies.

Broker 1: “Another option that is a bit outside the usualmodel is a slightly different, longer term relationship,where you have got a bit more interaction with one ortwo key researchers in this area [over time].”

Gaining final agreementMost brokers concluded the knowledge brokering ses-sion by confirming with the policy team the approachthat they would take in drafting the review proposal.This was relatively easily done by reviewing the finalquestion set and parameters.

Broker 5: “Basically, it is going to be four questions:chronic disease; prevention; barriers and factors thataffect benefits for different populations; and thenstrategies to stimulate implementation and adoption:what can we do to make these things work. I think it isa good question set.”

However, agreement about the questions and scope hadnot always been reached by the end of the session. In thiscase, the brokers elected to bring the policy team to anin-principle agreement about the main focus and criticalperspective for the review. Consent was then sought forthe broker to draft a proposal that was consistent withthese minimum parameters. This enabled the brokers to


close the session with the central tenets in place so thatthey were then able to draft a proposal that was feasibleand fit for purpose, on the understanding that the wordingof the questions and scope might need refinement in alimited, iterative exchange post session.

DiscussionThis study examined knowledge brokers in the context ofthe Evidence Check rapid review programme, particularlyin their diagnostic and translational roles. It offers a novelperspective because it analyses actual instances of inter-action between policy-makers and knowledge brokers,identifying and detailing new roles and activities. We an-ticipate that the findings will be applicable to knowledgebrokers with similar models in other contexts.Overall, the exchange between knowledge brokers and

policy-makers in this study is best conceived as an interplayof expertise, where knowledge is pooled and considered interms of its relevance to the policy process and to the pro-posed review, permitting a deeper analysis and generatingnew understandings of policy problems or solutions.This interplay was reflected in the high level of agree-

ment in the survey between knowledge brokers and pol-icy respondents in terms of the overall value of theknowledge brokering session, changes to the reviewquestions and the perceived value of these changes, andtheir perception that the information needed to tailorthe review for policy purposes had been elicited. In oneexample, the policy-maker and knowledge broker agreedthat the review question had changed in important ways,while also agreeing that the needed information had notbeen elicited. Free text comments suggested that neitherfelt they had attained the desired clarity and that bothhad found the experience somewhat difficult.Most importantly, knowledge brokers were usually

able to gain trust. In their ‘facilitative role’, brokers estab-lished trust and respect early in the interaction; this wasgiven form in the shared recognition that the contribu-tions of both knowledge broker and policy-maker wereessential to the task. Knowledge brokers trustedpolicy-makers’ judgement about the policy’s directionand context and policy-makers trusted brokers’ researchskills and assessments of feasibility. The mutuality thatcharacterised all sessions is a departure from ‘deficit’models of knowledge brokering where brokers fill a gapin policy-makers’ research capacity [22, 23, 40, 41] orspan an otherwise unbridgeable divide [20, 21, 42].At the same time, the brokers’ stance in the knowledge

brokering session was one of neutrality. While being re-sponsive to the policy team, they did not align them-selves with a particular perspective. Their focus was onmaximising the potential benefit of a review, allowingpolicy-makers to consider the strengths and limitations

of options, without judgement. It was this neutralitywhich enabled them to explore participants’ context andoptions from an independent perspective. Kislov et al.[25] point to the tension brokers can experience inmaintaining their ‘in-between’ position and they high-light the ambiguity that can be associated with inter-mediary roles. In our study, brokers were at onceengaged and apart, able to interact with the policy team,remain neutral in terms of the selection of options, andbe purposeful in ensuring a review that was both feasibleand matched to policy-makers’ needs.The ‘diagnostic role’ enacted by the knowledge brokers

in this study goes beyond having a generic familiarity withpolicy processes and environments [25]. Discourse aboutthe need to be conscious of the policy context to achieve ashared understanding about an agency’s needs [19, 23, 24]also falls short of capturing the kind of nuanced informa-tion that reviewers require [3, 43]; in any given context,some factors will be more salient to the policy processthan others [44]. The brokers’ task is to work with theagency to refine the review questions and scope so thatthey take into account parallel policy activity and con-straints in such a way that the review captures preciselywhat is needed [34]. Unpacking the relationship betweencontextual factors, review questions and the policy processis therefore a critical function for knowledge brokers.Knowledge brokers had a nuanced understanding of the

ways in which meaning is constructed in policy and in re-search discourse. In their ‘interpretative role’, brokers paidclose attention to the ways in which words and phraseswere used, pinpointing with precision those that would bedifficult to interpret for the researchers and defining andfine-tuning these as the sessions evolved. They drew ontheir knowledge about the way research is accessed andorganised and their experience and understanding abouthow review questions are best framed and structured.The critical act of knowledge translation in this study

lay in the movement from the policy perspective, to thereviewers’ perspective, and again to the policy perspec-tive; this shift in perspectives was inherent in all ses-sions. Knowledge brokers checked that they hadunderstood what policy participants intended, tested thisagainst the logic of the research process, and confirmedthat their research construct was consistent withpolicy-makers’ intent. Policy-makers tended to hold totheir central conviction about what the review shouldaddress but moved with the broker through alternativeconstructs until a shared understanding was achieved.This highlights a ‘deliberative role’ for knowledge bro-

kers. Deliberation as a concept has emerged in responseto the need for a considered exploration of the ways inwhich knowledge from research can or should contributeto policy-making, taking into account features of the localenvironments such as structures, resources, interests,


alliances, ideologies and the potential roles of stakeholders[29, 30]. Deliberative processes have the potential to influ-ence decision-making at local and at organisational levels[11, 45] and can increase the relative weight of research indecision-making [46]. In our study, the deliberative func-tion took the form of a ‘looking forward’ to anticipate howthe potential findings from a review might inform a policyprocess or interact with contextual factors to bring aboutchange – a preview against which choices of review focusmight be made, but many other applications will be pos-sible. The deliberative function of knowledge brokers isalso likely to be relevant to other settings where there aremultiple perspectives on a policy problem or solution,where there are contested policy environments or wherethese perspectives need to be evaluated against a particu-lar policy objective.The facilitative, diagnostic, interpretative and delibera-

tive roles identified in this study add considerable depthto those reported by Bornbaum et al. [33] in their system-atic review of knowledge brokering, namely of knowledgemanagement, linkage and exchange, and capacity-build-ing. The authors describe the multiple activities associatedwith these roles; however, the complex interactions be-tween policy-makers, the policy process and the policy orpolitical environments were not highlighted. The rolesidentified in this study draw attention to the underlyingprocesses knowledge brokers used, such as exploringcomplex interactions, establishing the relationships be-tween factors and their relative importance to the reviewquestion, and testing and reframing their understandingin conventions of both policy and research.The skill complexity highlighted here may provide guid-

ance for the selection and training of knowledge brokers.Knowledge brokers must have the kind of credibility thatengenders trust [39], understand the ways in which re-search is used and have the capacity to explore sensitivepolicy contexts in a neutral way. Further, it would be diffi-cult for a knowledge broker to undertake this work with-out being familiar with policy contexts and with thecomplexities likely to exist in policy processes. This sug-gests that the role may be more suited to senior re-searchers with some policy experience. Knowledge brokertraining may usefully include understanding the perspec-tives from which a policy problem may be viewed, refram-ing these from the perspective of potential reviewers,communicating research concepts in policy-friendly lan-guage and evaluating the potential benefits of optionsagainst policy objectives.While the number of knowledge brokering sessions con-

ducted was used in this study as an indicator of experi-ence, what actually constitutes expertise in knowledgebrokering is rather more complex as outlined above.Nevertheless, some knowledge brokers demonstratedgreater skill complexity than others, showing an ability to

sift dense narratives, identifying issues that would be per-tinent to decisions about the review’s focus and directionand leaving aside others; linguistic and conceptual expert-ise to frame and reframe both policy and research con-cepts in the language of the participants; a genuineneutrality that enables policy participants to actively cri-tique what they want from a review and how this can bestbe achieved; a sensitivity to meaning and intention and tothe ways in which these are signalled; and eliciting andbringing together policy-makers’ expertise in clarifyingand defining problems and solutions.Taken together, these findings point to a knowledge

brokering process that is simultaneously ‘translational’,in that it moves between two disciplines with quite dif-ferent conceptual and linguistic conventions and alignsthem, reframing both policy and research constructs inthe language of the policy participants; ‘diagnostic’, inthat the broker seeks detailed information from the pol-icy participants and hones this recursively throughoutthe session, refining the scope to reflect this informa-tion; ‘deliberative’, in that policy participants are notsimply providing information, but are actively cri-tiquing and revising what they want from the reviewand how this can be achieved; ‘facilitative’ in that thebroker is highly sensitised to cues about meaning,intention, expectations and understandings, and re-sponds to these cues, adjusting their technique to facili-tate trust and maximise progress.Overall, the knowledge of both brokers and policy-

makers is combined to generate new perspectives on thepolicy problem and to open and contextualise a field ofenquiry, extending policy-makers’ understanding of whatis possible from research and what is feasible within theconstraints of a rapid review.Future research might usefully examine the potential

relationship between the use of knowledge brokering atthe commencement of a rapid review process and theextent to which they are used by the policy-makers whocommission them. It would also be of benefit to under-stand whether other rapid review programmes usingknowledge brokers have had similar experiences, and toexamine the transferability of the four knowledge bro-kering roles to other contexts and settings.

ConclusionThis study identified four key roles for knowledge brokers,namely facilitative, diagnostic, interpretative and delibera-tive roles. Knowledge brokers were able to establish rap-port, enable disclosure of essential information andexplore contextual factors, and link these to the review’spurpose and intended use. These brokers were trustedand neutral intermediaries whose knowledge of researchand policy contexts enabled them to move easily between


policy and research perspectives, assist policy-makers toevaluate various options and craft a review proposal thatwas targeted, responsive and feasible. Mutuality, respectand an interplay of expertise were integral to the know-ledge brokering process. Future research might usefullyexamine the transferability of the four knowledge broker-ing roles to other contexts and settings.

Additional file

Additional file 1: Knowledge broker and participant surveys. (DOCX 93 kb)

AcknowledgementsGM was supported by the Outcomes, Services and Policy for the ReproductiveEarly Years (OSPREY) Programme under the NHMRC Capacity Building Grants inPopulation Health and Health Services Research, ID number 573122.

FundingNo funding was obtained for this study.

Availability of data and materialsNot applicable.

Authors’ contributionsGM, SR and PB made a substantial contribution to the conception and design ofthe study and all authors contributed to the analysis or interpretation of the data.GM drafted the manusciprt with critical revision by SR, PB and AH. All authorsapproved the final manuscript and agree to be accountable for the work.

Authors’ informationNot applicable.

Ethics approval and consent to participateEthics approval was obtained from the University of Sydney Human ResearchEthics Committee reference number 2014/021 approved 25 February 2014.Participant consent forms were obtained.

Consent for publicationNot applicable.

Competing interestsThe authors declare that they have no competing interests.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims in publishedmaps and institutional affiliations.

Author details1The Sax Institute, PO Box K617, Haymarket NSW 1240, Ultimo NSW 2007,Australia. 2School of Public Health, Sydney Medical School, The University ofSydney, Edward Ford Building (A27), Sydney NSW 2006, Australia. 3Centre forMedical Psychology & Evidence-based Decision-making (CeMPED), Level 6North, The Lifehouse, The University of Sydney, 119-143 Missenden Rd(C39Z), Sydney NSW 2006, Australia.

Received: 28 March 2018 Accepted: 28 October 2018

References1. Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers perceptions

of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239–44.

2. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review ofbarriers to and facilitators of the use of evidence by policymakers. BMCHealth Serv Res. 2014;14:2.

3. Campbell D, Donald B, Moore G, Frew D. Evidence Check: knowledge brokeringto commission research reviews for policy. Evid Policy. 2011;7(1):97–107.

4. Brennan SE, Cumpston M, Misso ML, McDonald S, Murphy MJ, et al. Designand formative evaluation of the Policy Liaison Initiative: a long-termknowledge translation strategy to encourage and support the use of Cochranesystematic reviews for informing health policy. Evid Policy. 2016;12(1):25–52.

5. Brownson RC, Dodson EA, Stamatakis KA, Casey CM, Elliott MB, et al.Communicating evidence-based information on cancer prevention to state-level policy makers. J Natl Cancer Inst. 2011;103(4):306–16.

6. van der Heide I, van der Noordt M, Proper KI, Schoemaker C, van den BergM, et al. Implementation of a tool to enhance evidence-informed decisionmaking in public health: identifying barriers and facilitating factors. EvidPolicy. 2016;12(2):183–97.

7. Dilworth K, Tao M, Shapiro S, Timmings C. Making health promotion evidenced-informed: an organizational priority. Health Promot Pract. 2013;14(1):139–45.

8. Jansen MW, Hoeijmakers M. A masterclass to teach public health professionals toconduct practice-based research to promote evidence-based practice: a casestudy from the Netherlands. J Public Health Manag Pract. 2013;19(1):83–92.

9. Yost J, Ciliska D, Dobbins M. Evaluating the impact of an intensiveeducation workshop on evidence-informed decision making knowledge,skills, and behaviours: a mixed methods study. BMC Med Educ. 2014;14:13.

10. Morris ZS, Bullock A, Atwell C. Developing engagement, linkage andexchange between health services managers and researchers: Experiencefrom the UK. J Health Serv Res Policy. 2013;18(1_suppl):23–9.

11. Moat KA, Lavis JN, Clancy SJ, El-Jardali F, Pantoja T. Evidence briefs anddeliberative dialogues: perceptions and intentions to act on what waslearnt. Bull World Health Organ. 2013;92:20–8

12. Bullock A, Morris ZS, Atwell C. Collaboration between health servicesmanagers and researchers: making a difference? J Health Serv Res Policy.2012;17(2_suppl):2–10.

13. Wathen CN, Sibbald SL, Jack SM, MacMillan HL. Talk, trust and time: alongitudinal study evaluating knowledge translation and exchange processesfor research on violence against women. Implement Sci. 2011;6:102.

14. Heaton J, Day J, Britten N. Collaborative research and the co-production ofknowledge for practice: an illustrative case study. Implement Sci. 2016;11:20.

15. Rycroft-Malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM, et al.Collaboration and co-production of knowledge in healthcare: opportunitiesand challenges. Int J Health Policy Manag. 2016;5(4):221.

16. Lomas J. The in-between world of knowledge brokering. BMJ. 2007;334(7585):129–32.

17. Conklin J, Lusk E, Harris M, Stolee P. Knowledge brokers in a knowledgenetwork: the case of Seniors Health Research Transfer Network knowledgebrokers. Implement Sci. 2013;8:7.

18. Dagenais C, Laurendeau M-C, Briand-Lamarche M. Knowledge brokering inpublic health: a critical analysis of the results of a qualitative evaluation. EvalProgram Plann. 2015;53:10–7.

19. Ward V, House A, Hamer S. Knowledge brokering: the missing link in theevidence to action chain? Evid Policy. 2009;5(3):267–79.

20. Hammami H, Amara N, Landry R. Organizational climate and its influenceon brokers knowledge transfer activities: a structural equation modeling.IJIM. 2013;33(1):105–18.

21. Long JC, Cunningham FC, Braithwaite J. Bridges, brokers and boundaryspanners in collaborative networks: a systematic review. BMC HealthServ Res. 2013;13:158.

22. Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, et al. A descriptionof a knowledge broker role implemented as part of a randomizedcontrolled trial evaluating three knowledge translation strategies.Implement Sci. 2009;4:23.

23. Traynor R, DeCorby K, Dobbins M. Knowledge brokering in public health: atale of two studies. Public Health. 2014;128(6):533–44.

24. Robeson P, Dobbins M, DeCorby K. Life as a knowledge broker in publichealth. JCHLA/JABSC. 2008;29(3):79–82.

25. Kislov R, Wilson P, Boaden R. The dark side of knowledge brokering. J HealthServ Res Policy. 2017;22(2):107–12.

26. Elueze IN. Evaluating the effectiveness of knowledge brokering in healthresearch: a systematised review with some bibliometric information. HealthInf Libr J. 2015;32(3):168–81.

27. Haynes AS, Gillespie JA, Derrick GE, Hall WD, Redman S, et al. Galvanizers,guides, champions, and shields: the many ways that policymakers usepublic health researchers. Milbank Q. 2011;89(4):564–98.

28. CIPHER Investigators. Supporting Policy In health with Research: anIntervention Trial (SPIRIT)-protocol for a stepped wedge trial. BMJ Open.2014;4(7):e005293.


https://doi.org/10.1186/s12961-018-0389-7

29. Muntaner C, Chung H, Murphy K, Ng E. Barriers to knowledge production,knowledge translation, and urban health policy change: ideological,economic, and political considerations. J Urban Health. 2012;89(6):915–24.

30. Head BW. Three lenses of evidence-based policy. Aust J Publ Adm. 2008;67(1):1–11.

31. Lavis JN, Permanand G. A way to approach knowledge brokering: theBRIDGE framework and criteria. Chapter 2. In: Lavis JN, Catallo C, editors.Bridging the Worlds of Research and Policy in European Health Systems.Copenhagen: WHO Regional Office for Europe on behalf of the EuropeanObservatory on Health Systems and Policies; 2014.

32. Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, et al. A randomizedcontrolled trial evaluating the impact of knowledge translation andexchange strategies. Implement Sci. 2009;4:61.

33. Bornbaum CC, Kornas K, Peirson L, Rosella LC. Exploring the function andeffectiveness of knowledge brokers as facilitators of knowledge translationin health-related settings: a systematic review and thematic analysis.Implement Sci. 2015;10(1):16–2.

34. Moore G, Redman S, D’Este C, Makkar S, Turner T. Does knowledgebrokering improve the quality of rapid review proposals? A before and afterstudy. Syst Rev. 2017;6(1):23.

35. Moore G, Redman S, Rudge S, Haynes A. Do policy-makers findcommissioned rapid reviews useful? Health Res Policy Syst. 2018;16:17.

36. The Sax Institute. Evidence Check Program. 2018. https://www.saxinstitute.org.au/our-work/knowledge-exchange/evidence-check/.[Accessed 27 Mar 2018]

37. Guest G, Bunce A, Johnson L. How many interviews are enough? Anexperiment with data saturation and variability. Field Methods. 2006;18(1):59–82.

38. Charmaz K. Constructing Grounded Theory. Thousand Oaks: Sage; 2014.39. Haynes AS, Derrick GE, Redman S, Hall WD, Gillespie JA, et al. Identifying

trustworthy experts: how do policymakers find and assess public healthresearchers worth consulting or collaborating with? PLoS One. 2012;7(3):e32665.

40. Ward M, Dobbins M, Peirson L. Lessons learnt from implementing anorganizational strategy for evidence-informed decision making. PublicHealth Panor. 2016;2(3):249–400.

41. Armstrong R, Waters E, Dobbins M, Anderson L, Moore L, et al. Knowledgetranslation strategies to improve the use of evidence in public healthdecision making in local government: intervention design andimplementation plan. Implement Sci. 2013;8:121.

42. Knight C, Lyall C. Knowledge brokers: the role of intermediaries inproducing research impact. Evid Policy. 2013;9:309–16.

43. Dwan KM, McInnes PC. Increasing the influence of ones research on policy.Aust Health Rev. 2013;37(2):194–8.

44. Moat KA, Lavis JN, Abelson J. How contexts and issues influence the use ofpolicy-relevant research syntheses: a critical interpretive synthesis. MilbankQ. 2013;91(3):604–48.

45. Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. Deliberative dialoguesas a mechanism for knowledge translation and exchange in health systemsdecision-making. Soc Sci Med. 2012;75(11):1938–45.

46. Flitcroft K, Gillespie J, Salkeld G, Carter S, Trevena L. Getting evidence intopolicy: the need for deliberative strategies? Soc Sci Med. 2011;72(7):1039–46.


https://www.saxinstitute.org.au/our-work/knowledge-exchange/evidence-check/

https://www.saxinstitute.org.au/our-work/knowledge-exchange/evidence-check/

Documents

Deconstructing knowledge brokering for commissioned rapid