The Nigeria Experience - WHO · The Nigeria Experience Knowledge Translation Health Policy & Systems Research Project Ebonyi State University Abakaliki Nigeria . Health Policy & Systems

Operational Manual for Strengthening Institutional Capacity to Employ Evidence in Health Policymaking for Developing Countries The Nigeria Experience

Knowledge

Translation

Platform

Health Policy & Systems Research Project

Ebonyi State University Abakaliki Nigeria


Operational Manual for Strengthening Capacity to Employ Evidence in Policymaking Page 2

Operational Manual for Strengthening Institutional Capacity to Employ Evidence in

Health Policymaking for Developing Countries

The Nigeria Experience

By

Chigozie Jesse Uneke

Abel Abeh Ezeoha

Chinwendu Daniel Ndukwe

Patrick Gold Oyibo

Friday Onwe

Esther Baragha Igbinedion

Peter Nnamdi Chukwu

September 2010



Acknowledgements

The authors are grateful to the World Health Organization for the provision of financial support for

this project through the Alliance for Health Policy and Systems Research (research grant no.

2009/25025-0; PO-No. 2 00072059); for Supporting National Processes for Evidence-informed Policy

in the Health Sector of Developing Countries, 2008.

The Project Team is grateful to The Vice-Chancellor Ebonyi State University (EBSU) Abakaliki

Nigeria, Prof. Francis Idike, for his support, and to the Deputy Vice-Chancellor, Prof Egwu U. Egwu

for his encouragement. The Team wishes to thank all our Mentors from EBSU Abakaliki for their

participation in this phase of the project; these include: Prof. Ogbonnaya Ogbu, Prof Emmanuel Agha,

Dr. Festus Iyare, Dr. Bethrand Ngwu, Dr. Linus Idam, and Dr. Ben Ozor. The Team appreciates all the

policymakers, researchers, and other stakeholders in the Nigeria health sector and their various

organizations and institutions for participating in this interventional phase. The Project Team wishes to

thank Prof. John Lavis, of, McMaster University Canada, for providing valuable information and tools

used for the analysis of the data collected from target population.

Disclaimer

All views expressed in this document are those of the authors and do not necessarily reflects the views

of WHO.



Table of Content

Abbreviations and acronyms……………………………………………………………………..6

Chapter 1. Introduction and general consideration.....................................................................7

1.1 Introduction and background information...................................................................................7

1.2 Project Goals and Objectives.......................................................................................................9

1.3 Purpose and scope of the Manual...............................................................................................12

1.4 Constituting a project team.........................................................................................................12

1.5 Appointment of project mentors.................................................................................................12

1.6 Target Users................................................................................................................................12

1.7 Pilot Test.....................................................................................................................................13

1.8 Organization of the Manual........................................................................................................13

Chapter 2. Overview of the methodological approach…………………………………………14

2.1 Preparatory Phase........................................................................................................................14

2.2 Base line data collection Phase...................................................................................................14

2.3 Intervention/implementation Phase.............................................................................................15

2.4 Monitoring and Evaluation Phase...............................................................................................15

2.5 Theoretical foundation underlying the methodological approach...............................................16

Chapter 3. Procedures for execution of activities of preparatory phase………………………18

3.1 Situational analysis......................................................................................................................18

3.2 Mapping of target populations.....................................................................................................18

3.3 Definition of participant category................................................................................................18

3.4 Researchers/Policy makers‟ interactive forum............................................................................19

3.5 Project monitoring and evaluation design………………………………………………………19

3.6 Data from social research.............................................................................................................21

Chapter 4. Procedures for execution of activities of baseline data collection phase..................24

4.1 Development and production of data collection instruments.......................................................24

4.2 Strategies for the administration of data collection instruments..................................................25

4.3 Analysis of data collected.............................................................................................................25

4.4 Interventional strategies to address the capacity constraints as informed by the

Preparatory and Baseline Data Collection Phases........................................................................25

Chapter 5. Overview of procedures for the intervention/implementation phase.......................36

5.1 Basis for interventional strategies.................................................................................................36

5.2 Training workshop pattern and methods for analysis of data collected........................................37

5.3 Development and production of policy briefs...............................................................................38



Chapter 6. Training workshop on capacity enhancement in information and communication

technology for policymaking...........................................................................................................40

6.1 Methods for Information and Communication Training Workshop............................................40

Chapter 7. Training workshop on capacity enhancement for acquisition, assessment, adaptation

& application of research evidence for policy making and partnership links............................45

7.1 Methods for the Training Workshop............................................................................................45

7.2 Participation of project mentors...................................................................................................47

Chapter 8. Training workshop on capacity enhancement to conceptualize, formalize, implement

and evaluate the research needed for better understanding and improved performance

of the health systems.........................................................................................................................54

8.1 Methods for the Training Workshop............................................................................................54

8.2 Policy dialogue on health systems strengthening.........................................................................55

Chapter 9. Training workshop on enhancing capacity to own and drive the agenda for

strengthening health systems/health sector reforms.....................................................................69

9.1 Methods for Training Workshop.................................................................................................69

9.2 Outcomes evaluation....................................................................................................................71

Chapter 10. Training workshop for capacity enhancement on leadership, governance and

management for health systems strengthening………………………………………………….81

10.1 Methods for Training Workshop...............................................................................................81

10.2 Staff and organizational appraisal/performance assessment.....................................................82

Chapter 11. Project evaluation design..........................................................................................89

11.1 Conceptual design of the evaluation…………………………………………………….…….89

11.2 Data from monitoring/operational research...............................................................................90

11.3 Administering the evaluation questionnaire……………………………………………….….91

Chapter 12. Conclusion and recommendations............................................................................95

References.........................................................................................................................................97



Abbreviations and acronyms

AHPSR Alliance for Health Policy and Systems Research

CHSRF Canadian health services research foundation

EBM Evidence-Based Medicine

EBP Evidence-Based Policymaking

EIP Evidence-Informed Policy making

FGD Focus Group Discussion

GHAIN Global HIV/AIDS Initiatives Network

HPSR Health Policy and Systems Research

HRM Human Resource Management

ICT Information and Communication Technology

KTP Knowledge Translation Platforms

M&E Monitoring and Evaluation

MDGs Millennium Development Goals

MDR Median rating

MNR Mean Rating

MOH Ministry of health

UNDP United Nations Development Programme

WHA World Health Assembly

WHO World Health Organization



CHAPTER 1

INTRODUCTION AND GENERAL CONSIDERATION

1.1 Introduction and background information

In most developing countries of the world, health outcomes have been described as unacceptably low

and at the centre of this human crisis is a failure of health systems (WHO 2007). According to the

World Health Organization (WHO), the health systems comprise of all the organizations, institutions

and resources that are devoted to producing health actions, and have a primary purpose of improving

health (WHO 2000a). In many low income and middle income countries, weak health systems are

impeding the success of the various health intervention programmes being initiated and implemented.

Bowen and Zwi (2005) noted that a key challenge to public health in most developing countries is to

better contextualize evidence for more effective policymaking and practice.

There is therefore increasing recognition that strong and effective health systems that are evidence-

based in their operations are necessary to achieve continued improvement in health outcomes in an

efficient and equitable manner (WHO 2008a; Travis et al. 2004). A number of previous reports have

provided convincing information to prove that evidence from research can enhance the health systems

operations and health policy process and development by identifying new issues for the policy agenda,

informing decisions about policy content and direction, or by evaluating the impact of policy

(Campbell et al. 2009; Dobrow et al. 2004; Hanney et al. 2003; Innvær et al. 2002).

There is an increasing recognition worldwide of the importance and the necessity of the use of more

rigorous processes to ensure that health care recommendations are informed by the best available

research evidence (Gonzalez-Block 2004). There is also increasing awareness of the need for the

production of new knowledge to improve how societies organize themselves to achieve health goals- a

phenomenon described as health policy and systems research HPSR (AHPSR 2007). HPSR aims to

produce reliable and rigorous evidence which helps to inform the many and varied critical decisions

that must be made by ministers of health, senior policy-makers and health service managers about how

to organize the health system and effect changes. Therefore if there must be an improvement in the

health care sector in its entirety particularly in developing countries, HPSR becomes imperative.



Unfortunately, HPSR is a somewhat new phenomenon in the health sector in many developing

countries and most health researchers, health policy makers, health service managers and other major

stakeholders at government and non-governmental levels are completely ignorant of its value in

policy-making and practice (evidence-informed health service management, and evidence-informed

policy making). The major constraint to the use of evidence in policy and practice in developing

countries is the grossly deficient capacity development at the individual and organizational levels,

particularly the lack of formally trained human resources among public health policy makers and

health service managers (Uneke et al. 2009). Health systems research is further burdened with a

number of other such challenges including under-investment, lack of human capacity, lack of public

demand, inadequate utilization, and poor dissemination of results (Uneke et al. 2010).

Health policy and systems research (HPSR) has been shown to have the potential of producing reliable

and rigorous evidence which can help to inform policy development and the policymaking process

(AHPSR 2007). In most developing countries there is a lack of recognition of the importance of HPSR

and there is also little interest in transfer and uptake of research into policy and practice in these

countries (Uneke et al. 2009). A few instances where policymaking involved the use of research

evidence occur mainly on clinical decision-making (evidence-based medicine) and only in a number

of tertiary health institutions such as teaching hospitals. A major factor contributing to this problem is

the lack of both national and sub-national effort to create awareness on the value HPSR in health

policymaking process; this is in addition to the dearth of any active on-going research on HPSR in the

country.

Furthermore, there are generally no financial incentives for doing studies related to HPSR, except for a

number of foreign-funded projects and consultancies, which are usually awarded to a relatively small

and select group of health professionals who may not involve policy makers and other stakeholders.

Moreover little efforts are usually made to widely and effectively disseminate findings from such

studies. There is also insufficient coordination between donors, research community, NGOs and

government bodies as well as weak links between programs, monitoring systems and use of

information.



The World Health Organization like many other international agencies is currently vigorously

supporting the process of contextualizing evidence and translating it to policy through the utility of

health policy and systems research in many developing countries (WHO 2003; AHPSR 2007). This is

in line with the resolution of the World Health Assembly (WHA) held in Geneva in May 2005, which

laid emphasis on how to harness health research more effectively in order to achieve the United

Nations‟ Millennium Development Goals in low- and middle- income countries (WHA 2005).

Capacity constraints at the individual and organizational levels are perceived to be major impediments

in HPSR evidence use in the health policymaking process in most developing countries including

Nigeria (Gonzalez-Block and Mills 2004; Uneke et al. 2009).

According to the World Health Organization, one of the greatest challenges facing the member states

is how to ensure access to safe and effective health services for those population groups most in need;

and strengthening individual and organizational capacities for a more functional health systems is a

core part of this challenge (WHO, 2007). The United Nations Development Programme (UNDP)

defined capacity as the ability of individuals, institutions and societies to perform functions, solve

problems, and set and achieve objectives in a sustainable manner (UNDP, 2006). The capacity to

direct the focus of HPSR is pivotal to shaping evidence-informed national health policies and systems

(AHPSR, 2007). Green and Bennett (2007) had noted that evidence is needed about how capacity

constraints in various countries inhibit evidence-informed health policy, and which strategies are

effective in addressing these constraints.

1.2 Project Goals and Objectives

There is a dearth of information on the status of HPSR evidence use in policymaking at individual and

institutional levels in most developing countries. The scarcity of such baseline information hampers

the following: (1). development effective of strategies to promote the application of HPSR in

policymaking; and (2). development of effective health system strengthening interventions. There was

therefore a dire need to identify the challenges associated with HPSR evidence use in policymaking in

these countries and the potential strategies to address them. There was also the need to assess the

capacity status of health policymaking actors and their organizations, in terms of individual staff skill

and institutional incentives in HPSR evidence use in policymaking. This is with the view to providing

scientific information that can be used to direct policy development and intervention program



implementation on health system strengthening and also provide baseline measurement for monitoring

and evaluating interventions.

This strategies contained in this manual were designed to assist in building sustainable and effective

health systems in developing countries and include the discovery and innovation for health systems,

i.e. in stewardship, governance, financing, resource management and informatics, as well as service

delivery. Thus the strategies are mainly focused on capacity enhancement in health policy and systems

research evidence use in policy making in any low income setting. It is well established that the

capacity to direct the focus of HPSR is pivotal to shaping evidence-informed national health policies

and systems. Consequently, this manual has been designed to enable the research community develop

a platform to work with national and sub-national health policy-makers and capable local health

system managers to promote HPSR. This is with the view to bridging the wide gap that exists between

researchers and policy makers in developing countries. The improvement of the skills of policy

makers in evidence-informed policy making and the establishment of enabling environments and

capacity for HPSR for the Health ministry and the research community are the overriding goals of this

project.

The intervention goal for evidence informed policy that form the basis of the strategies described in

this manual is within the following categories:

(a). Enhancing pull factors: The intervention to be considered includes;

(i) Establishment of rapid response mechanisms or policy advisory units within Ministries of Health

and/or other policy-making bodies or strengthening existing capacity with additional training;

(ii) Technical support and or training to help policy makers to facilitate commissioning of appropriate

research or syntheses; and

(iii) Training policy makers in identifying and applying evidence to decision making.

(b). Supporting exchange efforts: The intervention to be considered includes;

(i) Supporting forums, meetings, workshops, or conferences to facilitate exchange between researchers

and research users;



(ii) Encouraging the establishment of mechanisms which allow for ongoing exchange between

researchers and research users, such as research priority setting committees;

(iii) Supporting secondment of trusted researchers, who already have established links with policy

makers into policy making units.

The specific objectives of this program of work are therefore grouped as follows:

(1). Individual capacity strengthening: Because specific capacities, such as an individual‟s skills,

experience, values and beliefs, leadership, knowledge, resources, organizational support, partnership

links, and participation in networks, have been shown to influence the adoption and adaptation of

evidence in practice, this project is aimed at the enhancement these capacities which will undoubtedly

strengthen evidence use in policy making.

(2). Strengthening skill in developing research initiatives: Another important capacity this project is

designed to enhance is the skill in developing research initiatives on health systems. The capacity to

conceptualize, formalize, implement and evaluate the research needed for better understanding and

improved performance of the country‟s health system would be enhanced in this project. Also

included are technical support and training to help policy makers to facilitate commissioning of

appropriate research or syntheses, establishing stronger relationships with researchers, and also in

identifying, interpreting and applying evidence to decision making and tools to assess the legitimacy

of the filtering and amplification function of diverse organizations.

(3). To build enabling environments and capacity for HPSR: This is specifically targeted at the

health ministry and would include building the capacity to own and drive national/sub-national agenda

for strengthening health systems. Emphasis would be placed on how to enable Ministries of health to

take a strong lead in capacity building for all functions of the health system, including stewardship

(leadership, governance and communications), financing, resource management, informatics, service

delivery and research. The project would introduce strategies that would facilitate the development of

capacity for the generation, dissemination and use of health policy and systems research knowledge

among researchers, policy-makers and other stakeholders.

(4). The institution of a stronger incentive for evidence use: An important capacity that is required

and which is more relevant at organizational level is the institution of a stronger incentive for evidence

use. This would be achieved through the creation of stronger incentives for evidence use.



1.3 Purpose and scope of the Manual

The challenges associated with translating evidence to health policy in developing countries are

enormous and complex. These challenges are compounded by the lack of capacity for evidence

informed policy making at institutional level as well as on the part of policymakers. The purpose of

this manual is to provide a description of the strategies that were used to enhance institutional capacity

for evidence informed policy making in a low income setting; with focus on strengthening the health

systems in the following areas: health service delivery, information and evidence, medical products

and technology, health financing, health workforce, leadership and governance. The strategies are also

designed to strengthen evidence-to-policy link and improve the capacity of policymakers and other

stakeholders in the health policy making process in low and middle income countries. This manual

presents detailed and general descriptions of the procedures that can be used to identify and address

the capacity constraints often encountered at individual and institutional levels in the policymaking

process of developing countries.

1.4 Constituting a project team

To conduct a project of the sort described in this manual, a team must be constituted. This team can be

made up of either policymakers or researchers or preferably a combination of both. Depending on the

scope of coverage (national, regional, sub-regional, district etc), the team cane consist of seven to as

many as fifteen members. The team should be essentially multidisciplinary in composition, making up

of individuals with background in health sciences, social sciences, anthropology/sociology,

management sciences, and information technology, etc.

1.5 Appointment of project mentors

It is important to appoint project mentors who should be essentially senior academics (Professors,

Associate Professors or Senior Lecturers) from university or other tertiary institutions. These

individuals would provide technical support and consultancies to the project and will participate in

some of the workshops as resource persons or to chair/moderate focus group discussions.

1.6 Target Users

This manual is principally designed for use by Ministries of health at federal (national), state (district)

and local (municipal) government levels. The major target end users include the following: health



professionals in charge of the health systems; national, regional, state and local officers of the health

ministry; health professionals working with specific programs in the health ministry who wish to use

HPSR to improve the impact of their strategies; staff and consultants involved in public health issues

within the health ministry; political/legal advisers on health related matters; program/project managers

under the health ministry; researchers and teachers in public health training institutions; leaders and

staff of health-based civil society groups including non-governmental, bilateral or multilateral

organizations; directors and managers of health services in the uniform services.

1.7 Pilot Test

The approach described in this manual has been pilot-tested in Ebonyi State south-eastern Nigeria.

The duration of the project was 32months and the manual also incorporates the feedback from the pilot

test with outlines of recommendation.

1.8 Organization of the Manual

This manual is organized in a way that makes easy to follow the procedures. The manual is divided

into twelve chapters. References and further reading are provided at the end of each chapter. Chapter 1

provides the Introduction and general considerations. Chapter 2 provides an overview of the

methodological approach developed and used in this project. An outline of the procedures for

execution of activities of preparatory phase is shown in Chapter 3. Chapters 4 and 5 provide the

procedures for execution of activities of baseline data collection and for the

intervention/implementation phases respectively. The descriptions of the various training workshops

are provided in Chapters 6-10. Chapter 11 describes the project evaluation design in greater details

while the conclusion and recommendations are outlined in chapter 12. Assessment tools are contained

in relevant chapters where these tools are described.



CHAPTER 2

OVERVIEW OF THE METHODOLOGICAL APPROACH

The methodological approach in this manual was systematically designed to strengthen evidence-to-

policy link. The program of work to strengthen evidence-to-policy link is categorized into four phases.

The first phase is the preparatory phase; the second is the baseline data collection phase; the third

phase is the intervention/implementation phase; while the fourth is the monitoring/evaluation phase.

2.1 Preparatory Phase

This phase of the project focuses on refining the design of the intervention in order to obtain a clearer

picture of the project objectives and interventions. The preparatory phase consists of the following

activities;

(i). Advocacy/consultation visits,

(ii). Situational analysis for evidence use in policy making,

(iii). The design and execution of mechanism that would enhance proper project evaluation at the end

of the intervention, such as the use of process evaluation.

(iv). Identification of relevant training materials and tools for the implementation phase.

2.2 Base line data collection Phase

The baseline data collection is accomplished using three steps as follows:

(i). Data collection using structured pre-tested questionnaire which consists of questions on:

geographical focus of the health ministry, level of health systems research that the health ministry is

engaged in, formal/official collaboration, informal/unofficial collaboration, key activities/objectives of

the health ministry in terms of health policy and systems research and use of evidence in policy

making, key areas of work including key strengths, weaknesses, opportunities, and threats of the

health ministry in HPSR and use of evidence in policy making, level of staff awareness of HPSR and

use of evidence in policy making, computer literacy and skills in the use of information technology

pertaining to health policy and evidence use.

(ii). Key informant interviews of a selected number of individuals in the target group, conducted

face-to-face or by telephone using an interview guide.



(iii). Group discussions with a focus group of 9-12 persons among the target group conducted with a

moderator or two co-moderators using a discussion guide that centred on HPSR and use of evidence

in policy making.

2.3 Intervention/implementation Phase

Two major areas of intervention include:

(i) To strengthen individual staff skills to enhance capacity to use evidence in policy-making: Through

the provision of training and mentoring in use of acquisition, assessment, adaptation and application

of research evidence, e.g. training in the commissioning of research studies and the interpretation of

systematic reviews.

(ii). To strengthen institutional incentives to employ evidence: Through the creation of stronger

incentives for evidence use (e.g. through integrating this dimension into recruitment policies,

performance assessments, and staff appraisals); integration of modules on use of evidence into

leadership training courses for policy makers; and recognition awards - providing financial or in-kind

prizes to policy makers that have shown outstanding performance in applying evidence to policy.

2.4 Monitoring and Evaluation Phase

Process evaluation is useful because it informs learning about the outcomes, and is crucial to the

success of the impact evaluation. It helps to address questions such as: What effect (positive and

negative) is the intervention perceived to have on target participants (and non-participants)? What

effect does the intervention appear to have more widely, for example in terms of cultural change?

What factors appear to underpin differing impacts and outcomes? Are there particular sub-groups

within the target group which do better or worse from the intervention and why? Do particular models

of implementation appear to produce better outcomes? If so, how and why? Are there groups for

which the intervention appears to create more sustainable outcomes?.

Two approaches of the process evaluation include:

(a) Monitoring/operational research

This which involves the use and analysis of data collected other than for the evaluation such as:

(i) The analysis of administrative data about the eligible population

(ii) The analysis of data from Management Information Systems e.g. programme database



(iii) The collection and analysis of performance measurement data about resources (staff and

financial), etc.

(iv) Special monitoring exercises e.g. pro formas attached to specific cases to identify stages and

timing

(v) In-depth area studies

(b) Social research

Designed specifically to collect data for the evaluation and this involves;

(i) large scale quantitative studies using structured questionnaire,

(ii) qualitative studies via individual and paired depth interviews, group discussions, observational

work and documentary analysis,

(iii) case studies which considers individuals from the multiple perspectives of key actors.

2.5 Theoretical foundation underlying the methodological approach

The target participants are health service/policy providers because a supply-driven outcome that would

address capacity constraints in HPSR evidence use in policymaking and policy implementation by

these individuals was anticipated. According to the AHPSR the supply-driven model has been used

extensively to design capacity strengthening initiatives in developing countries and this was based on

the assumption that if the skills of the main actors (researchers and policymakers) are enhanced via

training and enough institutional capacity is built, research outputs will be put to good use (AHPSR

2004). Although argument has been intensified in HPSR circles to focus it on the demand side, taking

the supply-driven approach has a stronger tendency to accomplish a high level of ownership of

policies. This is because health policies are implemented better when those saddled with this

responsibility are made to identify the capacity challenges and the solutions required to address these

challenges.

Capacity constraints at the individual level are perceived to be major impediments in HPSR evidence

use in the health policymaking process in most developing countries (Gonzalez-Block and Mills 2004;

Uneke et al. 2009). Green and Bennett (2007) had noted that evidence is needed about how capacity

constraints particularly among policymakers in various countries inhibit evidence-informed health

policy, and which strategies are effective in addressing these constraints. No other category of

individuals is in the best position to identify the capacity challenges of service/policy providers in



HPSR evidence use in policy than the service/policy providers themselves. This informed the

advocacy of this methodological approach for evidence-to-policy link for low income countries. It is

pertinent to state however, that the intention is not to restrict the evidence-to-policy investigation to

the supply-driven model. The goal is to first generate information using the supply-driven model and

subsequently use it to stimulate the demand driven aspect which is also very vital to achieving

evidence-based policymaking and practice. A number of earlier reports have provided evidence which

proves that supply-side capacity building strategies that do nothing to stimulate the demand for

research are unlikely to achieve expectations, and may actually further distort allocations (Bhagavan

1992; Acemoglu 1997). The essence of this approach is to address the uncoordinated “pushing” of

research results by scientists and “pulling” of research results by market oriented users (AHPSR 2004)

Focus Group Discussion (FGD) is a very useful approach because this it draws upon respondents‟

attitudes, feelings, beliefs, experiences and reactions with respect to capacity constraints in HPSR

evidence use in policymaking. This is the most feasible method of accomplishing this as other

methods such as observation, one-to-one interviewing, and questionnaire surveys do not enhance the

social gathering and the interaction which being in a focus group entails. The approach taken to elicit

information from key informants in the focus groups was described by Kitzinger (1995). The

theoretical foundations underlying this approach are based on the report of Thomas et al. (1995) who

described focus group as „a technique involving the use of in-depth group interviews in which

participants are selected because they are a purposive, although not necessarily representative,

sampling of a specific population, this group being „focused‟ on a given topic‟. Richardson and Rabiee

(2001), noted that individuals participating in a focus group are usually selected based on the fact that

they have a working knowledge of issues addressed, are within the age-range, have similar socio-

demographic characteristics and would be comfortable talking to the interviewer and each other.

According to Burrows & Kendall (see Rabiee 2004), „this approach to selection relates to the concept

of „Applicability‟, in which subjects are selected because of their knowledge of the study area.‟



CHAPTER 3

PROCEDURES FOR EXECUTION OF ACTIVITIES OF PREPARATORY PHASE

3.1 Situational analysis

A comprehensive situational analysis can be conducted as follows;

(i). A preliminary mapping of the potential stakeholders/target group that would be involved in the

intervention phase; this is with the view to identifying eligible individuals (policy makers and/or those

in position to influence the health policy making process).

(ii). A document search can be done at the offices and websites of the various organizations and some

individuals of the target group can be asked to indicate documents and research projects possibly

associated with health policy. The documents should include reports, policy briefs, bulletins, media

communications, etc., which have been produced to inform about research projects and use of

evidence in policy making. Emphasis should be placed on more recent documents/publications. These

documents should be evaluated regarding their congruence with the objectives of the intervention.

(iii). A review of available literature and databases available at the State office of the World Health

Organization (WHO), and at WHO website, and supplemented by Medline and Google search of

journal articles and other literature pertinent to evidence use in HPSR as it relates to the health sector.

3.2 Mapping of target populations

During consultation/advocacy visits, the project team should endeavour to convince heads of the

various organizations to make commitments to approve the participation of policymakers in their

organization in the project. The criteria for inclusion of participants should include;

(i). Participant must be a full-time senior staff or officer in the organization.

(ii). Participant must be a policy maker or in position to influence the policy making process in the

health sector.

3.3 Definition of participant category

Participants should be categorized into two broad groups based on how their job specifications and

nature of influence on the policymaking process as follows:

(1). Participants who have direct influence of policymaking process: These refer to individuals who

receive processed information, data, reports and submissions on health-related issues and



synthesize/translate them into items required for policy drafting. They rarely take part in data

generation/collection activities. They participate in forums/meetings where various types of policy

documents are produced including policy briefs, policy drafts, and principally the main/final policy

documents.

(2). Participants who have indirect influence of policymaking process: These refer to individuals who

are mainly involved in the generation, collection and assembling of relevant information, and

processing of data and reports on health-related issues from the different sectors of the health system

and prepare them into forms that can be submitted for the drafting of policy documents. They may

make inputs during the production of policy briefs and policy drafts but do not usually participate in

the writing of the main/final policy documents.

3.4 Researchers/Policy makers’ interactive forum

As part of strategies to standardize the study protocol, identify priority areas (priority setting), prepare

for the data collection phase, and to make the intervention phase more result oriented, an interactive

forum should be held with target participants at the Ministry of Health. During each forum the

following activities should be performed;

(a). Introduction of project team and participants

(b). An explanatory speech by Project Director

(c). Administration of a simple questionnaire on policy making capacity assessment (Box 3.1)

(d). Questions/answers and interactive session

3.5 Project monitoring and evaluation design-How the preparatory phase helps to inform the

design

The overriding goals of a project of this sort should be to improve the skills of policy makers in health

policy and systems research evidence use in policy making and to assist in building sustainable and

effective health systems in a given low income setting. To achieve the goals of the project, series of

interventional activities should be developed to be implemented and these should be targeted at

strengthening individual staff skills to enhance capacity to use evidence in policy-making, and

enhancing institutional incentives to employ evidence in policy-making. It is intended that the

processes, sets of tools, descriptions, and analyses resulting from the project will provide useful

information to identify potential policy options to strengthen evidence-informed policymaking, which



will facilitate the use of scientific knowledge to improve actions within health systems, and ultimately

improve health and health equity.

One of the major areas of interest should be to determine whether the interventions planned are

delivered as intended to the targeted recipients at the end of the project. To be able to determine this,

the process evaluation design should be adopted which is a sort of implementation assessment

mechanism. The use of process evaluation design mechanism is necessitated by the following facts:

(i) Health policy and systems research evidence use in policy making is a relatively new

programme in most low income settings and further development of the programme appears

very needful before an assessment of the impact of the final model can be done;

(ii) The various health system constituents are under-performing and thus questions have arisen

about the organization, operations, delivery, quality, and success.

Information obtained from interaction with heads of the various organizations, some policymakers

nominated as participants, and assessed the data collection system of the organizations should inform

the evaluation design by helping to improve the quality content of the evaluation and the

implementation processes. Information obtained should aid setting priorities, appropriate evaluation

questions, baselines, quality criteria and thresholds to be determined, with respect to the project

specific objectives. In line with this, certain questions have been developed to assess the extent to

which the programme is being implemented as planned; assess participant and stakeholder experience

and satisfaction with the programme; assess programme reach/output; and improve programme

performance. The key questions are as follows: (a). Is the programme well organized? (b). What are

the main models of programme organization and delivery? (c). Are the activities being implemented as

planned? (d). Is the implementation consistent with quality assurance standards? (e). Are enough of

the policymakers participating? (f). What are participants' experiences of contact with the programme?

(g). Do participants understand the nature of the programme, its intention and its various elements?

(h). Are participants satisfied with the programme? (i). Are there particular groups within the target

population which do not receive the programme and why? and (j). Are there any unintended activities

and processes? These questions were expected to provide information on whether the programme

activities were accomplished; on quality of programme components; on how well programme

activities were implemented; on whether the target population was reached; and how external factors

influenced program implementation.



From the assessment of data collection system of the organizations, the team should gained an

understanding of the type of relevant data available, data being collected, how and where to find

them, how to collect them, and analyze them in a meaningful way. This information will enhance the

development of the monitoring/operational research as well as the social research mechanisms which

are the strategies designed to accomplish the process evaluation design in a project of this sort (see

chapter 11 for details).

The social research mechanism of the process evaluation is described below while the

monitoring/operational research mechanism is described in chapter 11

3.6 Data from social research

The Social research evaluation approaches include;

(a) large scale quantitative studies using structured questionnaire,

(b) qualitative studies via individual and paired depth interviews, group discussions, observational

work and documentary analysis,

(c) case studies which considers individuals from the multiple perspectives of key actors.

All the materials to be used for data collection at the evaluation phase including questionnaires,

interview/group discussions guides etc., should be developed at this preparatory phase and revised as

the project progresses. Information from some publications including the self-evaluation tool

developed by Canadian Health Services research Foundation, and evaluation toolkits/guidelines

developed by CDC Atlanta, WHO, researchers etc., are useful in the standardization of the evaluation

design. Consequently four categories of data to be collected in line with the specific objectives of this

project include;

(i). Data on individual capacity: These include data on specific capacities for evidence use in health

policymaking, such as an individual‟s skills, experience, values and beliefs, leadership, knowledge,

resources, organizational support, partnership links, and participation in networks

(ii). Data on skill in developing research initiatives: These include data on the skill in developing

research initiatives on health systems, capacity to conceptualize, formalize, implement and evaluate

the research needed for better understanding and improved performance of the country‟s health

system, capacity to facilitate commissioning of appropriate research or syntheses, establishing stronger

relationships with researchers, and also in identifying, interpreting and applying evidence to decision



making and tools to assess the legitimacy of the filtering and amplification function of diverse

organizations.

(iii). Data on organizational/institutional capacity for HPSR: These include data on the capacity to

own and drive national/sub-national agenda for strengthening health systems, capacity to take a strong

lead in capacity building for all functions of the health system, including stewardship (leadership,

governance and communications), financing, resource management, informatics, service delivery and

research.

(iv). Data on incentive for evidence use: These include data on organizational/institutional capacity for

institution of a stronger incentive for evidence use.

All data collection tools including structured questionnaires, interview/group discussions guides etc

should be designed to obtain the information. In this manual all the data collection tools described in

Chapters 3-10, are used for social research. The tool for monitoring/operational research is described

in chapter 11.



Box 3.1 Questionnaire for interactive forum with policymakers

HEALTH POLICY AND SYSTEMS RESEARCH PROJECT EVIDENCE-BASED POLICY MAKING CAPACITY ENHANCEMENT QUESTIONNAIRE

The following questions are designed to help us determine your capacity for evidence-based policy making to enable

us channel appropriate interventional strategies to enhance your capacity on the use of research evidence for policy

making in the health sector.

Please answer the questions as correctly as possible. The questionnaire is purely for research purpose only.

1. Designation………………………………………………………………………………

2. Duration in present designation…………………………………………………………

5. Age Category: 25- -

8. Do you use the internet to source fo

9. Do you have an e-

specify)………………………………

14. Do you have any existing health policy working document in your organization? Yes

15. Have you been involved in drafting any health policy document for your organization or the

17. If Yes, what source(s)? Education

18. Have you been involved in any health related research since assumption of your present office?

alth laboratory-

specify)……………………………………….

20. Do you think it is needful to collaborate with researchers from educational institutions in policymaking process?



CHAPTER 4

PROCEDURES FOR EXECUTION OF ACTIVITIES OF BASELINE DATA COLLECTION

PHASE

The baseline data collection phase involves the development and production of data collection

tools/instruments and the design of strategies to reach the target population for the administration of

the instruments.

4.1 Development and production of data collection instruments

The data collection instruments consist of a structured questionnaire, key informant interview guide

and focus group discussion guide. The questionnaire consists of questions which centred on the

following: geographical focus of the organization, level of health systems research that the

organization is engaged in, formal/official collaboration, informal/unofficial collaboration, key health-

related activities/objectives of the organization, in terms of health policy and systems research and use

of evidence in policy making, key areas of work including key strengths, weaknesses, opportunities,

and threats of the organization, in HPSR and use of evidence in policy making, level of staff

awareness of HPSR and use of evidence in policy making, computer literacy and skills in the use of

information technology pertaining to health policy and evidence use. The questionnaire is adapted

from the self-evaluation tool developed by Canadian Health Services Research Foundation [10], but

with some modifications to accommodate the objectives of the project and peculiarities of the low

income health systems. The measurement strategies include the use of the likert scale rating of four

options (1. grossly inadequate, 2. inadequate, 3. fairly adequate, 4. very adequate); yes or no; and

closed end answers. The questionnaire is outlined Box 4.1.

The interview and focus group discussion guide comprise of questions which centred on:

Identification of specific capacity constrains and challenges which impede the development of HPSR

evidence use; identification of potential strategies and solutions that would address capacity constrains

to HPSR evidence use in policy making; identification of critical gaps in HPSR evidence use in policy

making, with a focus on improving public health and; identification of the barriers to, and solutions

for, translating research into policy and practice via evidence use. The guide is outlined in Box 4.2.



4.2 Strategies for the administration of data collection instruments

In a low income setting, it is a lot easier to obtain information from large number of staff of the

ministry by inviting them together to a meeting/workshop. During this meeting the focus group

discussion (FGD) can be held. Focus groups can be constituted usually between 7-12 persons per

group and the FGD can last up to 45 minutes. Copies of the questionnaire can also be sent to other

participants and organizations who cannot be invited to a meeting. Effort should be made for a follow-

up to retrieve completed questionnaires. Selected individuals (particularly heads of health

organizations) should be interviewed (between 35-60 minutes) using a tape recorder and an interview

guide (Box 4.2).

4.3 Analysis of data collected

The data collected via the questionnaire can be analysed using the methods developed at McMaster

University Canada by Johnson and Lavis (2009). The recorded key informant interviews were

transcribed. All the responses from the focus group discussion should be noted. The transcribed

interviews and responses from the focus group discussions can be analyzed based on Giorgi's

phenomenological approach (Giorgi 1985), which is further described by Albert et al. (2007). The

analysis follows the following steps: (i) going over all the textual data to gain an overall

impression,(ii) identifying all comments that appeared significant to the research, extracting these

meaning units, (iii) independent abstracting of the meaning units, followed by discussion and

consensus; (iv) independent categorization and summarization of abstractions into challenges of

Health Policy and System Research evidence use in policymaking and the solutions as perceived by

policy-makers, followed by discussion and consensus; and finally (v) returning to the extracted text to

ensure a good fit with the final list of challenges and solutions.

4.4 Interventional strategies to address the capacity constraints as informed by the Preparatory

and Baseline Data Collection Phases

The Project Preparatory and Baseline Data Collection Phases involve an in-depth assessment of

institutional and individual staff capacity for evidence-based policy making. The findings generated

from the preparatory (first) phase and the base line data collection (second) phase should enable the

Project Team to identify the specific areas within the initial proposed interventions to focus on in

order to achieve the following: (i). Individual capacity strengthening; (ii). Strengthening skills in



developing research initiatives; (iii). Building enabling environments and capacity for HPSR; and (iv).

The institutionalization of a stronger incentive for the use of evidence. Table 4.1 shows an example of

proposed intervention and activities based on the outcome of the study in Nigeria.

Box 4.1 Questionnaire for baseline data collection

HEALTH POLICY AND SYSTEMS RESEARCH PROJECT EVIDENCE-BASED POLICY MAKING CAPACITY ENHANCEMENT QUESTIONNAIRE

The following questions are designed to help us determine your capacity for evidence-based policy making to enable

us channel appropriate interventional strategies to enhance your capacity on the use of research evidence for policy

making in the health sector.


1. BIODATA

- -

(i

2. OFFICIAL DESIGNATION ATTRIBUTES

(i). Name of your organization…………………………………………………………………………

(ii). Designation…………………………………………………………………………………..

(iii). Duration in designation………………………………………………………………………

(vi). Membership of health-

3. KNOWLEDGE & APPLICATION OF NFORMATION/COMMUNICATION

TECHNOLOGY

(ix). Do you have an E-

(x). If yes, please state……………………………………………………………………………

(xi). How often do you use your E-

4. GEOGRAPHICAL FOCUS

(ii). How would you assess your organization‟s capacity/competence to cover your geographical area of operation?



(a). Manpower

(b). Logistics

(c). Funding

(d). Facilit

(iii). How would you describe the accessibility (patronage) of the services provided by your organization

5. HEALTH SYSTEM RESEARCH & EVIDENCE

(A). ACQUISITION OF RESEARCH EVIDENCE

(a). Individual research skill Hints Ratings

(i). How adequate is your present

knowledge about initiating/conduction

research?

Identification of research problems;

construction of research questions;

designing of research methodology;

writing of research

proposals/protocols; analysis &

interpretation of research results;

writing of research reports.

(ii). How effective is your ability to

access and use existing research

evidence?

Journals, internet & library assess;

non journal reports eg. newspapers,

textbooks, reports from national &

international agencies, databases,

websites; works from researchers &

peers.

(b). Institutional/organizational

incentive for research Hints Ratings

(i). How would you describe the

capacity of your organization to carry

out research?

Existence of research programmes,

departments, officers & facilities; any

reputation on specialized areas of

research; research outputs; regularity

of research activities.

(ii). How would you describe the

capacity of your organization to

source for research evidence?

Existence of databases; relationship

with research institutions;

collaboration with researchers &

experts; commissioning of research

projects.

(iii). What is the level of research

incentives available in your

organization?

Availability of library, internet

facilities; availability of research

grants; award of honours/promotions;

in-service training; stipends, bonuses

& per-diem for research works;

provision of research facilities;

sponsorship to conferences/workshops;

institutional subscription of research

materials (periodicals eg. journals),

databases, websites.



(B). ASSESSING THE VALIDITY, QUALITY & APPLICABILITY OF RESEARCH EVIDENCE


(i). How would you describe your

capacity to assess the authenticity,

validity, reliability & high quality of

research evidence?

1. The skill to evaluate & appropriate

the quality of research methodology.

2. The skill to evaluate the reliability

of specific research evidence and to

compare research methods and

results.

(ii). How would you describe your

capacity to assess the relevance and

applicability of research evidence?

1. The skill to identify relevant

similarities and differences between

research evidence.

2. The skill to evaluate the

differences in the research evidences

in the context of your organization.



(i). Does your organization have

necessary incentives for assessment

of the validity, quality and

applicability of research evidence?

Use of consultants; application of

computer software, statistical package;

well-equipped laboratory; existence of

quality control units; promotion of

ethical standards/practices.

(ii). Does your organization have

incentives to encourage the

application of research evidence?

Availability of research evidence

implementation committee; availability

of administrative process for

accepting/implementing research

evidence.

(C). ADAPTING THE FORMAT OF THE RESEARCHER RESULTS TO PROVIDE INFORMATION USEFUL

TO DECISION MAKERS


(i). How would you describe your

ability to summarize results in a user-

friendly way.

1. Present research results concisely in

accessible language

y adequate

2. Synthesize in one document

relevant research as well as

information and analysis from other

sources.

3. Link the research results to key

issues and provide recommendations.

(ii). How would you describe your Use of charts, tables, graphs,



ability to present results of research to

decision makers.

pictogram, bullet/power point

presentations, etc.



(i). Does your organization have

incentives to encourage the provision

of research evidence to decision

makers?

Summarized and easy-to-use research

evidence is routinely brought to the

attention of relevant decision makers

(such as through regular meetings or

reports; or participation by

researchers/analysts in management

meetings to present/discuss evidence).

(D). APPLICATION OF EVIDENCE IN DECISION MAKING

(a). Institutional/organizational


(i). Do we lead by

example and

show we value

research use?

1. Using research is a priority: our

organisation has committed

sufficient people, time, training and

budgets to access, appraise,

adapt and apply research in making

decisions.

2. Our organisation‟s job descriptions

and performance incentives include

enough focus on activities which

encourage using research.

equate

3. Both management and front-line

staff support and participate in

frequent forum where staff and

invitees present and discuss

research evidence related to the

organisation‟s main goals.

ly adequate

4. Management has clearly

communicated corporate strategy and

priority areas for improvement, so that

people creating or monitoring research

evidence know what is needed.

5. Our organisation has effective

communication channels so that

priorities, evidence and ideas are

exchanged across divisions, as

well as between management and front

lines.

adequate

6. Our corporate culture is to value

and reward flexibility, change, and

continuous quality improvement, and

we provide adequate resources at all

levels to support change.

(ii). Do our decision making

processes have a place for research?

1. When we make major decisions, we

usually allow enough time to identify

researchable questions and create/

obtain, analyse and consider research

results and other evidence.

quate



2. Our management team has enough

expertise to evaluate the

feasibility of each option, including

potential impact across the

organisation as well as on its clients,

partners and other

stakeholders.

3. When staff develop or identify high

quality and relevant research,

decision makers will usually give

formal consideration to any

resulting recommendations.

4. Staff and appropriate stakeholders

know when and how major decisions

will be made, how and when they can

contribute evidence and how that

information will be used

e

5. The staff who have provided

evidence and analysis usually

participate in the discussion before a

decision is made and, when possible,

so do relevant non-staff researchers

irly adequate

6. When a decision is made, feedback

to staff and appropriate

stakeholders includes a rationale for

the decision, and review of

how the available evidence influenced

the choices made

airly adequate

6. KEY AREAS OF ACTIVITIES

(A). HEALTH ADMINISTRATION

Specific activities Hints Ratings

(i). Planning. 1. What is the capacity of your

organization to use evidence when

planning?

2. What is the level of external input

in your evidence-based planning

process?

(ii). Performance measurement. 1. What is the capacity of your

organization to make consistent

evidence-based policies?

2. How effective is the performance of

your organization‟s evidence-based

policies?

(iii). Bench marking/best practice. 1. What is the capacity of your

organization to adopt/use

international/national health policy

standard/guidelines?

quate



2. What is the accessibility of

international/national health policy

standard/guidelines to your

organization

(B). SERVICE DELIVERY


(i). Service organization and

management.

1. How functional is your

organization‟s management structure

in the use of evidence in service

delivery?

rossly inadequate

2. How does your service delivery

conform with other evidence-based

health practices?

(ii). Access to health services. 1. How adequate are your

organization‟s health

facilities/services in evidence-based

service delivery?

2. How accessible are the health

facilities/services rendered by your

organization?

(iii). Quality of health services/safety. 1. How efficient is your organization‟s

manpower in evidence-based health

service delivery?

2. To what extent do international

quality and safety guidelines influence

health service delivery by your

organization?

(C). INFORMATION/EVIDENCE


Generation/Strategic use of

information.

1. How do you describe your

organization‟s Information

Technology capacity to source,

generate and manage health related

information?

2. To what extent does your

organization have access to key

sources of health-related information

(media, research findings, educational

materials)?

equate

(D). MEDICAL PRODUCTS AND TECHNOLOGIES


Access to essential medical

products/technologies.

1. How do you describe the capacity

of your organization to acquire and

use essential medical products and

technologies ?

2. How scientifically sound,



efficacious and cost effective are

medical products and technologies

used by your organization?

(E). HEALTH WORKFORCE


(i). Employment and disengagement

of workforce.

1. How do you assess the personal

management practice in your

organization?

2. How do you describe the

availability of qualified health

workers in your organization?

(ii). Training, remuneration and

performance.

1. How do you rate the effectiveness

of your organization‟s policy on

training, remuneration and

performance of health workers?

2. How available and accessible are

training programmes and external

incentives to staff?

(F). HEALTH FINANCING


(i). Health system funding. 1. How do you describe the

affordability of the health services

rendered by your organization?

adequate

2. How adequate is your

organization‟s access to key sources

of health funding?

(ii). Resource management. 1. How do you describe the

budgetary, procurement, accounting

and auditing practice in your

organization?

2. How does external funding

influence your organization‟s policy

on resource allocation and utilization?

(G). LEADERSHIP AND GOVERNANCE


(i). Effective oversight and control. 1. How does your organization adhere

to international and natural regulations

and control mechanisms for health

system management?

2. To what extent do the activities of

regulatory agenci s and legislators

influence leadership and governance



in your organization?

(ii). Attention to health design issues. 1. How do you assess your

organization‟s effort to initiate an

update health system framework?

2. How does global practice influence

the process of development and

adoption of health system framework

in your organization?

te

Box 4.2 Focus group discussion/key informant interview guide

HEALTH POLICY AND SYSTEMS RESEARCH PROJECT EVIDENCE-BASED POLICY MAKING CAPACITY ENHANCEMENT KEY INFORMANT INTERVIEW/FOCUS DROUP DISUSSION GUIDE

PREAMBLE -Introduction of the subject matter by interviewer/moderator of discussion

-Collection of information on the biodata, designation attribute & participant‟s organization

FOCUS A:

Capacity constraints and challenges which impede the development of Health Policy and Systems Research

evidence use.

Questions:

1. What are your specific capacity constraints which impede the development of Health Policy and Systems

Research evidence use?

2. What are your organization‟s major challenges in the development of Health Policy and Systems Research

evidence use?

FOCUS B:

Identification of potential strategies and solutions that would address capacity constraints and challenges of Health

Policy and Systems Research evidence use.

Questions:

1. What do you think are the possible solutions to the identified capacity constraints and challenges on Health Policy

and Systems Research evidence use?

2. What strategies do you think your organization can adopt in the Health Policy and Systems Research evidence use

capacity constraints and challenges?

FOCUS C:

Identification of critical gaps in Health Policy and Systems Research evidence use with a focus on improving public

health.

Questions:

1. What do you think are the critical gaps in Health Policy and Systems Research evidence use?

2. How do you think these factors (gaps) affect evidence-based health care delivery?

FOCUS D:

Identification of barriers to and solution for translating research into policy and practice via evidence use.

Questions:

1. What do you think are the barriers to the use of research evidence in policy making process and practice?

2. What do you think can be done to facilitate the process of translating research evidence into policy and practice?



Table 4.1 Proposed intervention and activities for strengthening capacity for evidence-based

health policymaking based on the outcome of the preparatory and baseline data collection

phases in the Nigeria study. Interventions proposed Specific areas of focus identified

from phases 1&2

Activities in line with proposed

intervention

Remarks

1. Individual capacity

strengthening

(a). Improving information and

communication technology

(computer) skill of participants.

Note: Most participants lacked

adequate ICT(computer) skill, a

very vital requirement for access

to information necessary for

policymaking.

Training on Computer appreciation;

Computer application (use and

application of Microsoft Office

Word; Microsoft Office

PowerPoint); Use of the Internet.

To be

accomplished

during the

first training

workshop.

(b). Improving capacity for

acquisition, assessment,

adaptation & application of

research evidence.

Note: Most participants lacked

adequate knowledge of what

evidence is, how to acquire it,

assess it, adapt it and apply it, in

the policymaking process.

Although most of the capacity

ratings ranged from inadequate to

fairly adequate, our goal is to

achieve a very adequate capacity

for participants.

Training on what evidence is; types

& sources of evidence; how to

identify most relevant evidence;

constructing search protocol;

developing search; gathering the

evidence; conducting searches and

documenting the process; selecting

relevant evidence; assessing the

quality of the evidence; extracting,

synthesizing and presenting

evidence; deriving evidence

statements; and assessing

applicability.

Training to, and also in identifying,

interpreting and applying evidence

to decision making and tools to

assess the legitimacy of the filtering

and amplification function of

diverse organizations.

To be

accomplished

during the

second

training

workshop.

(c). Enhancing partnership links,

and participation in networks.

Note: Most participants do not

know about partnerships and

networks and how these can be

useful to them in the

policymaking. Response from

focus group discussion and

informant interview confirmed

this.

Introducing health-related

partnership & networks; their

importance & benefits; how to

establish partnership links and

participate in networks. Establishing

good working relationship between

policy makers and researchers.

To be

accomplished

during the

second

training

workshop.

2. Strengthening skills in

developing research

initiatives

Enhancing the capacity to

conceptualize, formalize,

implement and evaluate the

research needed for better

understanding and improved

performance of the country‟s

health system.

Note: Apart from participants

with doctorate degree who had

very adequate capacity, most of

Training on research methods and

the usefulness of research findings

in the policy process, and how to

facilitate commissioning of

appropriate research or syntheses

and the interpretation of systematic

reviews.

Training to develop research

capacity of individuals in relation to

HSPR evidence use in policy

To be

accomplished

during the

third training

workshop.



the other participants had capacity

ratings ranging from grossly

inadequate to fairly adequate, in

terms of the ability for research

initiative (conceptualize,

formalize, implement and

evaluate research).

Response from focus group

discussion and informant

interview confirmed this.

making; knowledge translation

programs and processes.

3. Building enabling

environments and

capacity for HPSR

Enhancing capacity to own and

drive the agenda for strengthening

health systems. Enhancing

organizational capacity to take

strong lead in capacity building

for all functions of the health

system.

Note: Results of the baseline data

collection indicate ratings for

key areas of health system

activities (especially health

financing, health workforce &

information/evidence) as

inadequate. Response from focus

group discussion and informant


Training on strategies that would

enable participating organizations to

establish capacity development

mechanism for the health system

(including stewardship [leadership

&governance], communications,

financing, resource management,

informatics, service delivery and

research).

Training on strategies that would

facilitate the development of

organizational capacity for the

generation, dissemination and use of

health policy and systems research

knowledge among researchers,

policy-makers and other

stakeholders.

To be

accomplished

during the

fourth

training

workshops.

(4). The

institutionalization of a

stronger incentive for the

use of evidence.

Enhancing organizational

capacity for the institution of

stronger incentive for research

and use of evidence in

policymaking.

Note: Results of the baseline data

collection indicate ratings for

organizational capacity for

research incentive mostly ranging

from grossly inadequate to

inadequate. Response from focus

group discussion and informant


A leadership training workshop to

be conducted. Exclusively for

policy makers at helm of affairs of

health ministry such as

commissioners, permanent

secretaries, political/legal advisers

on health related matters;

program/project managers under the

health ministry and top officers of

NGOs. Focus on strengthening

institutional incentives to employ

evidence in policymaking.

To be

accomplished

during the

fifth training

workshops.



CHAPTER 5

OVERVIEW OF PROCEDURES FOR THE INTERVENTION/IMPLEMENTATION PHASE

5.1 Basis for interventional strategies

Findings from the preparatory and the baseline data collection phases inform the categorization of the

interventional strategies into two broad areas. Training workshops and special mentoring programme

consist the major components of the intervention phase. The description of the intervention activities

under the two broad areas is as follows:

5.1.1 Strategies to strengthen individual staff skills to enhance capacity to use evidence in policy-

making: This should be executed through the provision of training and mentoring in use of

acquisition, assessment, adaptation and application of research evidence, e.g. training in the

commissioning of research studies and the interpretation of systematic reviews. The training programs

should consist of the development of research capacity of individuals in relation to HSPR evidence use

in policy making; knowledge translation programs and processes; comparative HPSR studies with

emphasis on evidence use, especially in the areas of research utilization, social health insurance, health

systems research on emerging infectious diseases, health issues during crisis periods such as natural

disasters, and equity. Knowledge sharing and networking should be fostered through the internet, and

the feasibility of establishing an HPSR „observatory‟ for knowledge syntheses and evidence-informed

policy recommendations should be explored.

Emphasis should be placed on the development of key HPSR objectives including priority

setting, knowledge generation, advocacy/communications, knowledge translation and capacity

building. Training should also focus on the health systems operations with emphasis on the building

blocks of health systems including service delivery; information and evidence; medical products and

technologies; health financing; health workforce; and leadership/governance (stewardship). Policy

briefs should be produced on these health systems building blocks, followed by policy dialogues.

Other key research areas affecting the health systems such as health sector reforms, human resource

development, equity, and poverty reduction should also be explored. A fellowship/mentoring scheme

that would enable the participating policy makers to take leave to spend time in a research

environment should be developed to commence after all interventional workshops. Arrangements

should be made participating policy makers to be mentored by an expert under whose supervision they

would be required to undertake a research project on HPSR as it pertains to evidence use in policy



making. The mentor in this case should be a senior researcher from the university (A Professor, an

Associate Professor or a Senior Lecturer).

5.1.2 Strategies to strengthen institutional incentives to employ evidence: This can be achieved

through the initiation of the process that would enhance the creation of stronger incentives for

evidence use (e.g. through integrating this dimension into recruitment policies, performance

assessments, and staff appraisals) in the participating organizations; integration of modules on use of

evidence into leadership training courses for policy makers; and recognition awards - providing

financial or in-kind prizes to policy makers that have shown outstanding performance in applying

evidence to policy. The components that can be considered should include: identification and building

the right team; identification and initiating relevant partnerships and linkages; promoting commitment

from key stakeholders; setting priorities on content topics to be tackled; planning of realistic

milestones for evaluation; and identification of local financial and in-kind support.

5.2 Training workshop pattern and methods for analysis of data collected

The major aim of the workshops should be principally to train policy makers and other stakeholders to

enhance their capacity for evidence-based health policymaking. Formal letters of invitation should be

sent to target participants at least three weeks to the date of each workshop, followed by an SMS

reminder via mobile phones a day or two before each workshop. At the workshops each participant

should be registered and given a participant identification number. Practical approaches to

policymaking process should be emphasized in each of the training activity. A pre-workshop

assessment questionnaire should be administered prior to actual training to assess the level of

knowledge, attitude, practices and perception of the participants on the specific topics to be covered

within the theme of each workshop. Space should also be provided for participants general comments

in the questionnaire. After the administration of the pre-workshop questionnaire the training should

commence and should be facilitated by resource persons who could be senior researchers drawn from

the academic institutions and senior policymakers from the health ministry.

All teaching sessions should be done using power-point presentation and handouts on each

topic and should be produced and distributed to all participants. It should be made mandatory for all

lectures to be delivered in simplified, practical and easily comprehensible patterns, with little or no

emphasis on complex mathematical or scientific computations/models for the benefit of non-

specialists who usually constitute the majority of the participants. Key informant interviews, focus



group discussions, dialogues, question/answer sessions, role play, demonstration, simulations, group

work and presentations from participants are some methods that can be used during the workshops.

Room should be created for personal/private interactions among participants and with

Facilitators/Resource persons. At the end each workshop, a post-workshop assessment questionnaire

should be administered to the participants to evaluate the impact of the workshop and also to obtain

the impression of the participants about the training and the materials used for the training.

The data collected via the questionnaire can be analysed using the methods developed at

McMaster University Canada by Johnson and Lavis (2009). The analysis is based on mean rating

(MNR), median rating (MDR) and range (R). For instance the figures represent Likert rating scale of

1-4 points, where 1point=grossly inadequate; 2 points=inadequate; 3 points=fairly adequate; and 4

points=very adequate. In terms of analysis, values ranging from 1.00-2.49 points are considered low,

whereas values ranging from 2.50-4.00 points considered high.

All recorded interviews should be transcribed. Also all the responses from the focus group

discussions and group works should be noted. The transcribed interviews and responses from the focus

group discussions and group works should be analyzed based on Giorgi's phenomenological approach

(Giorgi 1985), which is further described by Albert et al. (2007). The analysis follows the following

steps: (i) going over all the textual data to gain an overall impression,(ii) identifying all comments that

appeared significant to the research, extracting these meaning units, (iii) independent abstracting of the

meaning units, followed by discussion and consensus; (iv) independent categorization and

summarization of abstractions into challenges of Health Policy and System Research evidence use in

policymaking and the solutions as perceived by policy-makers, followed by discussion and consensus;

and finally (v) returning to the extracted text to ensure a good fit with the final list of challenges and

solutions.

In the Nigeria study a total of six training workshops were organized. The following chapters

provide detailed descriptions of the methods that can be used in the training in the workshops.

5.3 Development and production of policy briefs

The policy briefs should be prepared using the techniques outlined by Lavis et al. (2009a); Jones and

Walsh (2008) and GHAIN (2008). The briefs should be subjected by both internal and external review

to ensure it meets up with the minimum acceptable standard. The policy briefs should focus on the



specific areas of the health systems where interventions are mostly required. In the Nigeria Study the

following are the titles of the policy briefs produced are as follows:

1. Strengthening the generation/strategic use of health information and evidence for health

systems operations and policymaking in Nigeria.

2. Ensuring availability and equitable access to essential medical products, equipment and

technologies in Nigeria.

3. Improving access, availability and efficiency in the use of health financial resources.

4. Strengthening the organization, access and quality of health service delivery in Nigeria.

5. Enhancing the distribution, efficiency and performance of health workforce in Nigeria.

6. Ensuring effective leadership and governance to enhance the performance of the health

systems in Nigeria.

Similar titles can be adopted depending on the area of the health systems where intervention is

required.



CHAPTER 6

TRAINING WORKSHOP ON CAPACITY ENHANCEMENT IN INFORMATION AND

COMMUNICATION TECHNOLOGY FOR POLICYMAKING

6.1 Methods for Information and Communication Training Workshop

An Information and Communication Technology (ICT) training workshop can be organized for

participating policy makers at an ICT Centre with facilities to accommodate such training workshop.

This could be the first in the series of the training workshops for the interventional phase of the project

as it was the case in the Nigeria study. In Nigeria, the training was organized at the ICT Centre of

Ebonyi State University Abakaliki. The theme of the workshop in the Nigeria study was: “Enhancing

the capacity for evidence-based health policy making in Nigeria through information and

communication technology.” Similar theme can be adopted depending on the local needs and the

outcome of the baseline data collection. The workshop can commence with the administration of a

pre-workshop pretested structured questionnaire (see Box 6.1). Using the questionnaire, an assessment

can be conducted on participants‟ Knowledge and application of information/communication

technology particularly in the following areas: (i). Computer appreciation; (ii). Microsoft word; (iii).

Power point; and (iv). Internet use. After the questionnaire administration, the training should

commence and should be focused on the development of the capacity of the policymakers on

computer appreciation and application.

A training manual should be developed and made available to participants. The training

manual should cover topics in computer appreciation; use of Microsoft word; use of power point and

use of internet. Participants should also be taught the importance of ICT to the health sector

(improving the functioning of health care systems; improving the delivery of health care; and

improving communication about health), and how to gain access to health information from the

internet.

This aspect of the training should focus on the following:

(a). Benefits of engaging ICT in running the health sector;

(b). The search protocol for information;

(c). Identification of and search strategies of a wide range of electronic resources, in addition to

the traditional scientific and clinical databases;

(d). Information audit to accompany search strategies.



At the end of the training, a post-workshop questionnaire should be administered (See Box 6.2) to

assess the impact of the workshop and participants impression about the workshop.

Box 6.1 Pre-ICT training workshop questionnaire

HEALTH POLICY AND SYSTEMS RESEARCH PROJECT

EVIDENCE-BASED POLICY MAKING CAPACITY ENHANCEMENT

PRE-ICT TRAINING QUESTIONNAIRE

Please answer the questions as correctly as possible. The questionnaire is purely for research purpose

only.

1. BIODATA

- -44 yea



(ii). Designation…………………………………………………………………………………..


3. KNOWLEDGE & APPLICATION OF INFORMATION/COMMUNICATION TECHNOLOGY

A. COMPUTER APPRECIATION

(i). Rate your ability to identify the different basic components of the computer.

(ii). What is your level of knowledge on the functions of basic computer components?

y adequate

(iii). What is your ability to differentiate between the input and output computer devices?

(iv). How would you describe your ability to set up a computer workplace?

(v). How would you describe your ability to boot and shut down a computer?

(vi). Rate your present ability to identify different computer storage devices.

(vii). What is your knowledge of basic care and safety procedures in computer handling/usage?

B. MICROSOFT(MS) WORD

(i). Rate your ability to link up to MS word page from a computer window

(ii). What is your level of knowledge on the identification and functions of the menu/standard tool

bars in an MS word page?

(iii). How would you describe your ability to create and save MS word files and folders?



(iv). How would you describe your ability to access/assess existing files and folders in MS word?

(v). How would you describe your ability to type and format texts/paragraphs in MS word?

C. POWER POINT

(i). How would rate your knowledge of the following power point functions?

(a). Opening and inserting slides:

(b). Bullet presentation:

(c). Copying/pasting from MS word to power point:

Very adequate

(d).Use of graphics/charts/tables in power point:

(e). Use of animations in power point:

(ii). Rate your ability to use power point in conference/workshop presentations and meetings

quate

(iii). How would you describe your present ability to identify and apply basic devices for power point

D. INTERNET USE

(i). How would you describe your knowledge on the importance and benefits of the internet?

(ii). Do you now have an e-

(iii). How would you describe your ability to create and use e-mail address

(iv). How is your ability to locate information on the internet?

(v). How is your ability to locate and access websites of different organizations?

(vi). How is your level of knowledge on the types and use of major search engines?

(vii). How is your ability to locate and access relevant databases?

Comments:…………………………………………………………………………………………………………

……………………………………………………………………………………………………………………

…………………………………………………………………………………………………………

PARTICIPANT No…………………

Thank you very much for your participation.



Box 6.2 Post-ICT training workshop questionnaire



POST-ICT TRAINING QUESTIONNAIRE


only.

1. BIODATA

(i)

- -



(ii). Designation…………………………………………………………………………………..


3. KNOWLEDGE & APPLICATION OF INFORMATION/COMMUNICATION TECHNOLOGY

A. GENERAL QUESTIONS ON ICT TRAINING

(iii). If yes when? (month/year)……………………

(iv). Is the present training t

(v). How adequate was this training in relation to your job description?

(vi). How would you rate the quality of the training?

(vii). How would you rate the impact of the training?

B. COMPUTER APPRECIATION

(i). Rate your present ability to identify the different basic components of the computer.

(ii). What is your present level of knowledge on the functions of basic computer components?

(iii). What is your present ability to differentiate between the input and output computer devices?

(iv). How would you describe your present ability to set up a computer workplace?

(v). How would you describe your present ability to boot and shut down a computer?

(vi). Rate your present ability to identify different computer storage devices.

(vii). What is your present knowledge of basic care and safety procedures in computer handling/usage?

C. MICROSOFT(MS) WORD

(i). Rate your present ability to link up to MS word page from a computer window



(ii). What is your present level of knowledge on the identification and functions of the menu/standard tool

bars in an MS word page?

(iii). How would you describe your present ability to create and save MS word files and folders?

(iv). How would you describe your present ability to access/assess existing files and folders in MS word?

(v). How would you describe your present ability to type and format texts/paragraphs in MS word?

D. POWER POINT

(i). How would rate your present knowledge of the following power point functions?

(a). Opening and inserting slides:

(b). Bullet presentation:

(c). Copying/pasting from MS word to power point:

(d).Use of graphics/charts/tables in power point:

(e). Use of animations in power point:

(ii). Rate your present ability to use power point in conference/workshop presentations and meetings

(iii). How would you describe your present ability to identify and apply basic devices for power point

E. INTERNET USE

(i). How would you describe your present knowledge on the importance and benefits of the internet?

(ii). Do you now have an e-

(iii). How would you describe your present ability to create and use e-mail address

(iv). How is your present ability to locate information on the internet?

(v). How is your present ability to locate and access websites of different organizations?

(vi). How is your present level of knowledge on the types and use of major search engines?

(vii). How is your present ability to locate and access relevant databases?

Comments:…………………………………………………………………………………………………………

……………………………………………………………………………………………………………………

…………………………………………………………………………………………………............................





CHAPTER 7

TRAINING WORKSHOP ON CAPACITY ENHANCEMENT FOR ACQUISITION,

ASSESSMENT, ADAPTATION & APPLICATION OF RESEARCH EVIDENCE FOR

POLICY MAKING AND PARTNERSHIP LINKS

7.1 Methods for the Training Workshop

This training workshop should be the second in the series of training workshops to be organized for

capacity enhancement. In the Nigeria study the theme of the workshop was: “Improving capacity for

acquisition, assessment, adaptation & application of research evidence for policy making &

enhancing partnership links.” Participants comprised of health policy makers, stakeholders and

researchers and were drawn from the health ministry at the State and local government levels, non-

governmental organizations, and civil society groups such as medical and health workers associations

of physicians, nurses, pharmacists, laboratory scientists, and hospital administrators. Also represented

were heads of health divisions of the uniform services such as the police, army, prisons, and road

safety commission.

Similar theme and participant selection can be adopted depending on the local needs and the outcome

of the baseline data collection. The project mentors should be invited to participate in this workshop

because of the role they will play in building partnership link which is also a focus of this workshop.

The workshop should commence with the administration of a pre-workshop pretested structured

questionnaire (see Box 7.1). At the end of the training sessions a post-workshop pretested structured

questionnaire (see Box 7.2) should be administered. Prior to the administration of the post-workshop

questionnaire a focus group discussion (FGD) can be conducted on partnership links/bridging the gap

between policy makers and researchers (see Box 7.3 for FGD guide). The lecture sessions should

focus on the following topic areas:

7.1.1 Introduction to health policy and systems research/ building evidence-informed policy

environments

(i). Definition and scope of health policy and systems research (HPSR); HPSR aim and focus;

(ii). The building block health systems (health service delivery, information and evidence, medical

products and technology, health financing, health workforce, leadership and governance).

(iii). Building evidence-informed policy environments, understanding policy contexts, the role of

interests, ideology and values.



(iv). Research–policy interface and the policy-making processes

(v). Stage of the policy process: Agenda setting; Policy formulation; Implementation and Evaluation

(vi). Policy making and use of evidence

(vii). Types of evidence: systematic reviews; single research studies; pilot studies and case studies;

information available on the internet; and experts‟ opinion.

7.1.2 Acquisition, assessment, adaptation & application of evidence in health policy making

(i). Definition of evidence?

(ii). Evidence in policymaking context: The core issues and Types of evidence

(iii) Scientific evidence (Context-free scientific evidence and Context-sensitive scientific evidence):

(iv). Colloquial evidence (Expert testimony and Consultation with stakeholders):

(v). Identifying evidence for policy making and issues to consider when searching for public health

literature

(vi). Search Procedure (Constructing the search protocol; Developing the search strategy; Gathering

the evidence, conducting searches and documenting the process):

(vii). Additional searches: „Snowballing‟ to find citations, A search of the grey literature,

Journal/publication hand-searches especially use of libraries and other information databases,

Making contact with experts and stakeholders.

(viii). Documenting the search process & Submission of evidence for policymaking

ongoing research; studies that have only been published as abstracts; data on adverse effects

relevant to the interventions being studied; economic models; studies of public or professional

views and experiences; confidentiality information.

7.1.3 Building effective linkage, partnership & exchange between health policy-makers and

researchers

(i). The policy environment

(ii). Actors in Health Policy Making

(iii). Need for Partnership in the Health Policy Making Process

(iv). The „Two Communities‟ Model of Researchers and Policy-Makers (University Researchers and

Government Officials)

(v). The Policy Making Process: The policy process and how networks may affect it



Stage of the policy process (Analysis of the Policy Making Process; Setting the Agenda;

Improving the characteristics of research results)

(v). Adopting the Communication Process and Products to Stakeholders‟ needs

(vi). Building Evidence into the Policy Making Process (Engineering view, Enlightenment view,

Elective Affinity view)

(vii). Models of Researchers versus Policy Makers Linkage

(viii). Difficulties in Charting Effective Researcher/Policy-Maker Linkage

7.1.4 Capacity Development and Enhancement for Evidence-Informed Health Policy-Making

(i). Definition of Capacity

(ii). How to Identify Lack of Capacity

(iii). Focus of Capacity Building

(iv). Steps in Building capacity for Evidence-Based Health Policy Research

(v). The capacity Pyramid

(vi). Elements of Capacity Building Capacity development

(vii). The link between the Need for capacity Development and Evidence Informed Health Policy

Making

(viii). Elements of Evidence-Informed National Policy

7.2 Participation of project mentors

During the workshop the project mentors should complete a Mentor‟s Research-Policy Link

questionnaire (Box 7.4), which is designed to assess the mentors‟ involvement into research/policy

process. Each project mentor should serve as chair of each focus group discussion. The discussion

lasted up to 40minutes and the senior policymaker from each group presented group‟s resolution to all

participants. The following were the components of the discussion: (i). The need for

partnership/collaboration between researchers and policymakers; (ii). Types of partnership; (iii).

Causes of existing gaps between researchers and policymakers in health policymaking; (iv).

Implications of the gaps between researchers and policymakers in health policymaking; and (v).

Strategies for effective partnership/collaboration between researchers and policymakers



Box7.1 Pre-Workshop Questionnaire: Improving Capacity for Acquisition, Assessment,

Adaptation & Application of Research Evidence for Policy Making HEALTH POLICY AND SYSTEMS RESEARCH PROJECT


PRE WORKSHOP QUESTIONNAIRE


A. BIODATA

(i). Gende

- -

B. OFFICIAL DESIGNATION ATTRIBUTES


(ii). Designation…………………………………………………………………………………..


C. POLICY MAKING PROCESS

1. To what extent are you involved in the policymaking process in your organization?

2. What is your level of knowledge of the meaning of policy?

3. How would you rate your understanding of policy context?

4. What is your level of understanding of policy environment?

5. How would you rate your understanding of different types of policies?

6. What is the level of your knowledge about stakeholders‟ and various actors‟ involvement in policy making?

7. Rate your level of understanding of policy making process.

8. How is your level of understanding of the meaning of priority setting/policy agenda in policy making?

9. What is your level of understanding of the meaning of a policy brief?

10. Rate your level of understanding of what a policy dialogue is.

11. What is your level of understanding of the meaning of a policy draft?

12. What is the level of your knowledge on the role of researchers in policy making?



D. CAPACITY FOR USE OF EVIDENCE

1. What is your level of understanding on what evidence is in policy making context

2. Rate your knowledge on the types of evidence that can be used for policy making?

3. What is your level of knowledge on the sources of evidence used for policy making?

4. How would you rate the effectiveness of your ability to source for evidence used in policy making?

5. How would you rate your capacity to identify/select relevant evidence for policy making?

6. How would you rate your ability to adapt (extract, synthesize, & present) evidence used for policy making?

7. How would you rate your ability to transform evidence into policy useable form?

8. How is your knowledge of filtration and amplification function (optimal use of relevant evidence) in policy

making?

Comments:………………………………………………………………………………………………………………

……………………………………………………………………………………………………………………………

……………………………………………………………………………………………





Box 7.2 Pre-Workshop Questionnaire: Improving Capacity for Acquisition, Assessment,

Adaptation & Application of Research Evidence for Policy Making HEALTH POLICY AND SYSTEMS RESEARCH PROJECT


POST WORKSHOP QUESTIONNAIRE


A. BIODATA

- -



(ii). Designation…………………………………………………………………………………..


C. GENERAL QUESTIONS ON POLICY & EVIDENCE

(i). Are capacity enhancement/strengthening necessary for evidence based policy making in your work?

(ii). Have you had any training programme/workshop on capacity enhancement/strengthening for evidence

based policy m

(iii). If yes when? (month/year)……………………

(iv). Is the present training the first formal training you have had on capacity enhancement/strengthening for

(v). How adequate was this training in relation to your job description?

(vi). How would you rate the quality of the training?

(vii). How would you rate the impact of the training?

D. POLICY MAKING PROCESS

1. What is now your level of knowledge of the meaning of policy?

2. How would you rate your present understanding of policy context?

equate

3. What is your present level of understanding of policy environment?

4. How would you rate your present understanding of different types of policies?

5. What is the level of your present knowledge about stakeholders‟ and various actors‟ involvement in policy

making?



6. Rate your present level of understanding of policy making process.

7. How is your present level of understanding of the meaning of priority setting/policy agenda in policy

making?

8. What is your present level of understanding of the meaning of a policy brief?

9. Rate your present level of understanding of what a policy dialogue is.

10. What is your present level of understanding of the meaning of a policy draft?

11. What is the present level of your knowledge on the role of researchers in policy making?

E. CAPACITY FOR USE OF EVIDENCE

1. What is your present level of understanding on what evidence is in policy making context?

dequate

2. Rate your present knowledge on the types of evidence that can be used for policy making?

3. What is your present level of knowledge on the sources of evidence used for policy making?

4. How would you rate the effectiveness of your ability to source for evidence used in policy making?

5. How would you rate your present capacity to identify/select relevant evidence for policy making?

6. How would you rate your present ability to adapt (extract, synthesize, & present) evidence used for policy

making?

7. How would you rate your present ability to transform evidence into policy useable form?

8. How is your present knowledge of filtration and amplification function (optimal use of relevant evidence)

in policy making?

Comments:………………………………………………………………………………………………………………

……………………………………………………………………………………………………………………………

……………………………………………………………………………………………





Box 7.3 Focus group discussion guide on partnership links/bridging the gap between policy

makers and researchers


FOCUS GROUP DISCUSSION GUIDE

Topic: BRIDGING THE GAP BETWEEN HEALTH POLICY MAKERS & RESEARCHERS

1. Preamble: Introduction of the subject matter by moderator.

2. Discussion on the need for partnership/collaboration between researchers and policymakers.

3. Types of partnership (eg. formal/informal, use of joint committees of researchers/policymakers, use of

consultants/knowledge brokers)

4. Causes of existing gaps between researchers and policymakers in health policymaking.

5. Implications of the gaps between researchers and policymakers in health policymaking.

6. Challenges associated with partnership between researchers and policymakers.

7. Strategies for effective partnership/collaboration between researchers and policymakers.



Box 7.4 Project Mentors’ questionnaire on assessment of research- policy link HEALTH POLICY AND SYSTEMS RESEARCH PROJECT


MENTOR QUESTIONNAIRE

A. INSTITUTIONAL AFFILIATION

(i). Name of your institution……………………………………………………………..

(ii). Faculty/department……………………………………………………………………..

B. RESEARCH/POLICY LINK

1. To what extent is research relevant to health policy making in Nigeria

2. How do you rate the level of partnership presently existing between researchers and

health policy makers in Nigeria.

3. How frequent have your previous research outcomes being made available to policy

makers?

4. How adequate are the following factors in researchers/health policy makers partnership

in Nigeria?

(i). Time factor (in terms of time taken to initiate, execute & make available research

evidence ie. Information/data):

(ii). Resources:

(iii). Incentives:

5. Rate the level of influence of the following factors in researchers/health policy makers

partnership in Nigeria?

(i). Research methodology:

(ii). Politics:

(iii). Ownership/Intellectual property:

6. Have you held any policy making position outside the academic environment in the

pas

7. Have you participated in any policy making process outside the academic

8. If yes, to what extent did you use research evidence in policy making process?

9. How often are you being consulted by policy makers for research evidence?

10. Rate your preparedness to partner with policymakers in the policymaking process.




CHAPTER 8

TRAINING WORKSHOP ON CAPACITY ENHANCEMENT TO CONCEPTUALIZE,

FORMALIZE, IMPLEMENT AND EVALUATE THE RESEARCH NEEDED FOR BETTER

UNDERSTANDING AND IMPROVED PERFORMANCE OF THE HEALTH SYSTEMS

8.1 Methods for the Training Workshop

This workshop should be the third in the series of interventional workshops. Participants should

comprise of health policy makers, stakeholders and researchers and were drawn from the health

ministry at the State and local government levels, non-governmental organizations, and civil society

groups such as medical and health workers associations of physicians, nurses, pharmacists, laboratory

scientists, and hospital administrators. Also to be represented are heads of health divisions of the

uniform services such as the police, army, and prisons. In the Nigeria study the theme of the workshop

was “Enhancing the capacity to conceptualize, formalize, implement and evaluate research for better

understanding and improved performance of health systems.” A similar theme can be adopted

depending on the local needs and the outcome of the baseline data collection. The project mentors

should be invited to participate in this workshop.

The workshop should have four main objectives as follows:

(i). To enhance the capacity for the development of effective health systems policy;

(ii). To develop research capacity of individuals in relation to HSPR evidence use in policy making;

knowledge translation programs and processes;

(iii). To facilitate commissioning of appropriate research or syntheses and the interpretation of

systematic reviews;

(iv). To enhance the knowledge of participants on the value of policy briefs and policy dialogues in the

policy making process.

A policy dialogue should be held during this workshop, the detail of the procedure is described in

section 8.2. The workshop should commence with the administration of a pre-workshop pre-tested

structured questionnaire which focuses on the assessment of collaborative initiatives, health policy

briefs and health policy dialogues (see Box 8.1). This should be followed by lectures in the following

topic area:



8.1.1 Framework for Development of Effective Health Systems Policy

(i). Building blocks‟ of health systems (Service delivery; Information and evidence; Medical products

and technologies; Health workforce; Health financing; Leadership and governance)

(ii). Evidence-based health policy yields strong and effective health systems

(iii). Improving evidence-based policy-making for health (Strengthening demand from policy-makers;

Creating formal processes to facilitate dialogue; Improving packaging of evidence; Enhancing

technical capacity; Implementing incentives for researchers; Recognizing the role of informal

relationships)

(iv). Practical steps to developing evidence-based health policy

(v). Establishing collaborative health policy initiative

(vi). Principles and framework for collaborative initiative (setting the vision for collaboration; setting

out the guiding principles for collaboration; defining and setting goals and objectives; understanding

and defining roles and responsibilities; implementation requirements; evaluation)

(vii). Policy Design

(viii). Agenda/Priority-setting

(ix) Policy Briefs (what a policy brief is; preparation of policy briefs; key ingredients of effective

policy briefs; structure/outline of a policy brief; value of policy briefs)

(x). Policy Dialogue (what is policy dialogue is; importance of policy dialogue; organising and using

policy dialogues to support evidence-informed policymaking; characteristics of a policy dialogue)

(xi) Preparation of policy draft and submission to relevant authorities/government

(xii) Engaging the public in evidence-informed policymaking

(xiii) Monitoring and evaluation of policies

8.2 Policy dialogue on health systems strengthening

Strengthening the health systems is a major focus of this project. Therefore the policy briefs produced

in the six health systems building block areas should be subjected to a policy dialogue during this

workshop. Table 8.1 shows the titles of the policy briefs produced in the Nigeria study:



Table 8.1 Policy briefs produced for the Nigeria study Policy

Brief

No.

Health Systems

Building Block

Title of Policy Brief

1. Information &

Evidence

Strengthening the Generation/Strategic Use of Health Information and Evidence

for Health Systems Operations and Policymaking in Nigeria.

2. Medical Products

& Technology

Ensuring Availability and Equitable Access to Essential Medical Products,

Equipment and Technologies in Nigeria.

3. Health Financing Improving Access, Availability and Efficiency in the Use of Health Financial

Resources in Nigeria.

4. Service Delivery Strengthening the Organization, Access and Quality of Health Service Delivery in

Nigeria.

5. Health

Workforce

Enhancing the Distribution, Efficiency and Performance of Health Workforce in

Nigeria.

6. Leadership &

Governance

Ensuring Effective Leadership and Governance to Enhance the Performance of

the Health Systems in Nigeria.

As a part of the preparation for the policy dialogue, a letter of invitation and a policy brief that is most

relevant to each invited participant‟s designation/job description should be sent to the policymakers

two weeks before the workshop. This is to give ample time for participants to study the policy briefs

and prepare their inputs; and also to enable the Project Team to obtain participants‟ assessment of the

policy briefs. Participants should be advised in the accompanying letter of invitation to prepare inputs

on policy options in relation to the policy briefs they have received. During the workshop, before the

commencement of the dialogue, a policy brief evaluation questionnaire developed by Johnson and

Lavis (2009) (see Box 8.2) should be administered. This policy brief questionnaire is designed to

provide an in-depth evaluation of the participant‟s views about how the policy brief was produced and

designed; overall assessment of the policy brief; and views about what can be done better or

differently. After this participants should be grouped into six with each group undertaking a policy

dialogue on each of the six policy briefs. Participants should be assigned to the groups depending on

each person‟s designation/job description; for instance those at management positions including the

Permanent Secretary of the health ministry should be placed in the group that will undertake the

dialogue on the policy brief that centred on leadership and governance. Those in service delivery-

related positions eg., Director of Hospital Management, Director of Medical Services, Director of

Nursing services, etc., should undertake the dialogue on the policy brief focusing on health service

delivery. Each of the groups should consist of 7-12 individuals and should be chaired by a senior

researcher (Professor/Associate Professor/Senior Lecturer) who is a project mentor from the university

and a senior policymaker should serve as a deputy chair. A policy dialogue guideline should be



provided for participants (see Box 8.3). The guideline includes a description on how to evaluate the

policy brief documents in terms of content quality and relevance, as well as the policy issues

presented, the magnitude of the problems to be addressed and how actionable the policy options

recommended are. The dialogue should last up to one hour and each group should document its

resolutions and submit same to the Project Team. Thereafter a policy dialogue evaluation

questionnaire developed by Johnson and Lavis (2009) (see Box 8.4) should be administered. The

purpose of the questionnaire is to evaluate the participant‟s views about how the policy dialogue was

designed; overall assessment of the policy dialogue; views about what can be done better or

differently; and views about using research evidence more generally.

Box 8.1 Pre-workshop questionnaire: Enhancing the capacity to conceptualize, formalize,

implement and evaluate the research needed for better understanding and improved

performance of health systems HEALTH POLICY AND SYSTEMS RESEARCH PROJECT


PRE WORKSHOP QUESTIONNAIRE


only.

A. BIODATA

- -



(ii). Designation…………………………………………………………………………………..


C. GENERAL QUESTIONS ON COLLABORATIVE INITIATIVE

(i). How would you rate your understanding of collaborative initiative in health policymaking?

(ii). Are collaborative initiatives necessary for evidence based health policy making in your work?

(iii). How would you rate the level of involvement of your organization in collaborative initiative in

policymaking?

(iv). Have you ever been involved in any collaborative initiative in health policymaking before this project?

(v). If yes how many times in the last two years? ……………………

(v). Is the present project your first formal involvement in collaborative initiative for evidence based policy



(vi). How would you rate the collaborative initiative in this project since your involvement as participant?

D. GENERAL QUESTIONS ON POLICY BREIFS

(i). How would you rate your understanding of policy briefs in health policymaking?

(ii). Are policy briefs necessary for evidence based policy making in your work?

(iii). How would you rate the level of involvement of your organization in the production of policy briefs in

(iv). Have you had any training programme/workshop on policy briefs for evidence based policy making


(vii). If yes how many times in the last two years? ……………………

(viii). Is the present training workshop your first formal training on policy briefs for evidence based policy

E. GENERAL QUESTIONS ON POLICY DIALOGUES

(i). How would you rate your understanding of policy dialogues in health policymaking?

(ii). Are policy dialogues necessary for evidence based policy making in your work?

(iii). How would you rate the level of involvement of your organization in policy dialogues in

(iv). Have you had any training programme/workshop on policy dialogues for evidence based policy making


(vi). Have you ever participated in a policy dial

(vii). If yes how many times in the last two years? ……………………

(viii). Is the present training workshop your first formal training on policy dialogues for

Comments:……………………………………………………………………………………………....





Box 8.2 Policy brief evaluation questionnaire


Evaluation – Policy Brief

Please circle the number that corresponds to your answer and (if you wish) offer any suggestions about how the policy brief

can be improved.

Several questions make reference to "stakeholders." The term "stakeholders" includes: staff or members of civil society groups;

staff or members of health professional associations or groups; staff of donor agencies (e.g., European Community, Swedish

International Development Agency) or international organizations (e.g., World Health Organization); and staff of

pharmaceutical or other biotechnology companies.

Section A – Views about how the policy brief was produced and designed

1. The policy brief described the context for the issue being addressed. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7

How could the policy brief be improved in this regard? _______________________________________

____________________________________________________________________________________

____________________________________________________________________________________

2. The policy brief described different features of the problem, including (where possible) how it affects particular

groups. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


____________________________________________________________________________________

____________________________________________________________________________________

3. The policy brief described three options for addressing the problem. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7

How could the policy brief be improved in this regard? ______________________________________

___________________________________________________________________________________

___________________________________________________________________________________

4. The policy brief described what is known, based on synthesized research evidence, about each of the three options and

where there are gaps in what is known. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

5. The policy brief described key implementation considerations. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful



1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

6. The policy brief employed systematic and transparent methods to identify, select, and assess synthesized research

evidence. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

7. The policy brief took quality considerations into account when discussing the research evidence. How useful did you

find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

8. The policy brief took local applicability considerations into account when discussing the research evidence. How

helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

9. The policy brief took equity considerations into account when discussing the research evidence. How helpful did you

find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

10. The policy brief did not conclude with particular recommendations. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

11. The policy brief employed a graded-entry format (e.g., a list of key messages and a full report). How helpful did you

find this approach?



Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

12. The policy brief included a reference list for those who wanted to read more about a particular systematic review or

research study. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

13. The policy brief was subjected to a review by at least one policymaker, at least one stakeholder, and at least one

researcher (called a “merit” review process to distinguish it from “peer” review, which would typically only involve

researchers in the review). How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

Section B – Overall assessment of the policy brief

14. The purpose of the policy brief was to present the available research evidence on a high-priority policy issue in order

to inform a policy dialogue where research evidence would be just one input to the discussion. How well did the

policy brief achieve its purpose?

Failed Moderately

failed

Slightly

failed

Neutral Slightly

achieved

Moderately

achieved

Achieved

1 2 3 4 5 6 7

Comments: _________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

Section C – Views about what can be done better or differently

15. Reflecting on your reading of the policy brief, please list at least one element of how the policy brief was produced

and designed that should be retained in future policy briefs.

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

16. Reflecting on your reading of the policy brief, please list any element(s) of how the policy brief was produced and

designed that should be changed in future policy briefs.

_______________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________



17. Reflecting on what you learned from reading the policy brief, please list at least one important action that

policymakers, stakeholders, and/or researchers can do better or differently to address the featured policy issue.

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

18. Reflecting on what you learned from reading the policy brief, please list at least one important action that you

personally can do better or differently to address the featured policy issue.

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

Section D – Role and background

19. I am a (please tick (√ ) single most appropriate role category):

Broad

role category

Specific role category

Tick

(√)

single

most

appro-

priate

Policymaker Public policymaker (i.e., elected official, political staff, or civil servant) in the national

government

Public policymaker (i.e., elected official, political staff, or civil servant) in a sub-

national government (e.g., province/state or a district if the latter has independent

policymaking authority)

Manager in a district/region (if it does not have independent policymaking authority)

Manager in a healthcare institution (e.g., hospital)

Manager in a non-governmental organization (NGO)

Stakeholder Staff/member of a civil society group/community-based NGO

Staff/member of a health professional association or group

Staff of a donor agency (e.g., European Community, Swedish International

Development Agency) or international organization (e.g., World Health Organization)

Staff of a pharmaceutical or other biotechnology company

Representative of another stakeholder group

Researcher Researcher in a national research institution

Researcher in a university

Researcher in another institution

Other

20. I have been working in my current position for _____ years.

21. If you identified yourself as a policymaker, stakeholder, or "other," please indicate if you have training and/or

extensive experience as a researcher (circle one):



Yes / No

22. If you identified yourself as a researcher, stakeholder, or "other," please indicate if you have experience as a

policymaker (circle one):

Yes / No

Thank you!

ID #: __________ (Your responses will be kept confidential and data will not be reported in ways that could potentially

identify you or your organization.)

Box 8.3 Policy dialogue guideline

HEALTH POLICY AND SYSTEMS RESEARCH PROJECT Policy Dialogue Guidelines

1. General assessment of the policy brief documents:

►1. Does the policy brief present research findings, policy options and recommendations

appropriately?

►2. Does Policy brief synthesise a large amount of complex information and present

findings and recommendations in a format that enables the reader to easily and quickly

understand an issue?

►3. Was the Policy brief written in clear, jargon-free language, and pitched towards educated

non-specialists in the topic?

►4. Are there other features lacking and needed to be included in the policy brief?

2. The policy issues:

►1. Does the policy brief address a high-priority issue and describe the relevant context of

the issue?

► 2. Does the policy brief provide adequate background information on the policy issue?

► 3. Does the background information reflect the true scenario in our local context?

►4. Are there other aspects of the background information lacking and needed to be

included in the policy brief?

3. Magnitude of the problem:

►1. Does the policy brief provide definition and a short overview of the root causes of the

problem such that its features can be understood; a clear statement on the policy

implications of the issue; shortcomings of the current approach?

► 2. Does the policy brief describe the problem, costs and consequences of past options to

address the problem, and the inadequacies or failures of past policies?

►3. Are there other aspects of the problem lacking and needed to be included in the policy

brief?

4. Policy options ►1. Does the policy brief provide policy recommendations that are actionable and clearly

connected to specific decision-making junctures in the policy-making process?

►2. Was the implementation considerations taken into account?

►3. Are there other policy recommendations lacking and needed to be included

in the policy brief?

5. General comments



Box 8.4 Policy dialogue evaluation questionnaire


Evaluation – Policy Brief

Please circle the number that corresponds to your answer and (if you wish) offer any suggestions about how the policy brief

can be improved.

Several questions make reference to "stakeholders." The term "stakeholders" includes: staff or members of civil society groups;

staff or members of health professional associations or groups; staff of donor agencies (e.g., European Community, Swedish

International Development Agency) or international organizations (e.g., World Health Organization); and staff of

pharmaceutical or other biotechnology companies.

Section A – Views about how the policy brief was produced and designed

1. The policy brief described the context for the issue being addressed. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


____________________________________________________________________________________

____________________________________________________________________________________

2. The policy brief described different features of the problem, including (where possible) how it affects particular

groups. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


____________________________________________________________________________________

____________________________________________________________________________________

3. The policy brief described three options for addressing the problem. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

4. The policy brief described what is known, based on synthesized research evidence, about each of the three options and

where there are gaps in what is known. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

5. The policy brief described key implementation considerations. How helpful did you find this approach?

Very Moderately Slightly Neutral Slightly Moderately Very helpful



unhelpful unhelpful unhelpful helpful helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

6. The policy brief employed systematic and transparent methods to identify, select, and assess synthesized research

evidence. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

7. The policy brief took quality considerations into account when discussing the research evidence. How useful did you

find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

8. The policy brief took local applicability considerations into account when discussing the research evidence. How

helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

9. The policy brief took equity considerations into account when discussing the research evidence. How helpful did you

find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

20. The policy brief did not conclude with particular recommendations. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

21. The policy brief employed a graded-entry format (e.g., a list of key messages and a full report). How helpful did you

find this approach?



Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

22. The policy brief included a reference list for those who wanted to read more about a particular systematic review or

research study. How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

23. The policy brief was subjected to a review by at least one policymaker, at least one stakeholder, and at least one

researcher (called a “merit” review process to distinguish it from “peer” review, which would typically only involve

researchers in the review). How helpful did you find this approach?

Very

unhelpful

Moderately

unhelpful

Slightly

unhelpful

Neutral Slightly

helpful

Moderately

helpful

Very helpful

1 2 3 4 5 6 7


___________________________________________________________________________________

___________________________________________________________________________________

Section B – Overall assessment of the policy brief

24. The purpose of the policy brief was to present the available research evidence on a high-priority policy issue in order

to inform a policy dialogue where research evidence would be just one input to the discussion. How well did the

policy brief achieve its purpose?

Failed Moderately

failed

Slightly

failed

Neutral Slightly

achieved

Moderately

achieved

Achieved

1 2 3 4 5 6 7

Comments: _________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

Section C – Views about what can be done better or differently

25. Reflecting on your reading of the policy brief, please list at least one element of how the policy brief was produced

and designed that should be retained in future policy briefs.

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

26. Reflecting on your reading of the policy brief, please list any element(s) of how the policy brief was produced and

designed that should be changed in future policy briefs.

_______________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________



27. Reflecting on what you learned from reading the policy brief, please list at least one important action that

policymakers, stakeholders, and/or researchers can do better or differently to address the featured policy issue.

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

28. Reflecting on what you learned from reading the policy brief, please list at least one important action that you

personally can do better or differently to address the featured policy issue.

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

Section D – Role and background

29. I am a (please tick (√ ) single most appropriate role category):

Broad

role category


Tick

(√)

single

most

appro-

priate


government

Public policymaker (i.e., elected official, political staff, or civil servant) in a sub-

national government (e.g., province/state or a district if the latter has independent

policymaking authority)













Other

20. I have been working in my current position for _____ years.

23. If you identified yourself as a policymaker, stakeholder, or "other," please indicate if you have training and/or



extensive experience as a researcher (circle one):

Yes / No

24. If you identified yourself as a researcher, stakeholder, or "other," please indicate if you have experience as a

policymaker (circle one):

Yes / No

Thank you!





CHAPTER 9

TRAINING WORKSHOP ON ENHANCING CAPACITY TO OWN AND DRIVE THE

AGENDA FOR STRENGTHENING HEALTH SYSTEMS/HEALTH SECTOR REFORMS

9.1 Methods for Training Workshop

This workshop should be the fourth of the series of interventional training workshops. Efforts should

be made to retain the same participants from the first training workshop till this fourth workshop.

The objectives of the workshop should be to enhance participants‟ capacity and knowledge on: (i).

health sector reforms, equity and evidence-informed policy recommendations; (ii). Knowledge

generation, advocacy, communications, and knowledge translation programs and processes; (iii).

Comparative HPSR studies in the areas of research utilization, social health insurance and poverty

reduction; (iv). Emergency preparedness and health systems research on emerging infectious diseases

and health issues during crisis periods such as natural disasters. A pre-workshop questionnaire on the

assessment of health systems performance (see Box 9.1) should be administered.

To achieve the objectives of the workshop lectures should focus on the following topic areas:

(1). Health sector policy reforms, equity and evidence-informed policy recommendations.

(2). Health policy and systems studies in the area of research utilization, social health insurance and

poverty reduction.

(3). Health systems research on prevalent and emerging infectious diseases.

Each lecture could take up to 35 minutes and should be accompanied by an interactive session which

can involve deliberations, comments, questions and answers. A breakdown of the contents of each

lecture is described below:

9.1.1 Health sector policy reforms, equity and evidence-informed policy recommendations

(i). Health Sector Reforms

(ii). Components of health sector reform programmes in developing countries (improving the

performance of the civil service, decentralization, improving the functioning of ministries, broadening

choices for health financing, introducing managed competition, working with the private sector)

(iii). Equity in Health

(iv). Equity in health and health care

(v). Gender equity



(vi). Conceptual framework to analyze health sector reform and equity (equity in access to health care

services, equity in the utilization of health care services, equity in resource allocation, equity in the

delivery of quality services, equity in the delivery of effective services)

(v). Engaging the public in evidence-informed policymaking.

(vi). Monitoring and evaluation of policies

9.1.2 Health policy and systems studies in the area of research utilization, social health insurance

and poverty reduction

(i). Building research infrastructure for Health Policy Formulation and implementation with respect to

developing a viable health insurance scheme and scaling up poverty reduction.

(ii). The compelling reasons to improve the design and implementation of efficient health policy data

gathering and empirical analysis management and development

(iii). Addressing the problems of availability of empirical health (social insurance and poverty

reduction inclusive)

(iv). Macro data for policy origination, implementation and monitoring of national health policies.

(V). Philosophy and objectives of national health policy and system

(vi). Health policies and health objectives

(vii). The challenges of public health policy and systems research

9.1.3 Health systems research on prevalent and emerging infectious diseases

(i). Types of health systems research needed to address prevalent and emerging infectious diseases

(efficacy and cost-benefit research, disease patterns research, emerging infectious disease research,

immunology and vaccine research, disease-specific research, health promotion and communications

research, safe food and water research, knowledge translation research)

(ii). Capacity-building for health systems research (attraction and retention, education and training,

collaboration and networks, funding)

(iii). Communication: dissemination of research findings and obtaining public health input

(iv). Surveillance, informatics and databases

(v). Agenda for public health systems research on emergency preparedness against infectious diseases

(vi). Elements of emergency preparedness (operations-ready workers and volunteers, countermeasures

and mitigation strategies, health risk assessment, psychosocial outcomes and community resilience,



mass health care, public information and communication, testing operational capabilities, interagency

communication and coordination).

9.2 Outcomes evaluation

The tool to use for the outcomes evaluation was developed at McMaster University Canada by

Johnson and Lavis (2009). According to Johnson and Lavis (2009), the outcomes evaluation assess

three areas in which knowledge translation (KT) platforms anticipate achieving particular outcomes as

follows:

(i). the availability of health research evidence about high-priority policy issues;

(ii). the strength of relationships among policymakers, researchers, and knowledge-translation

specialists (i.e., individuals, other than journalists, whose expertise and role relate to

supporting the use of research evidence outside the research community); and

(iii). the strength of policymakers‟ and researchers‟ capacity to support the use of health

research evidence in health systems policymaking.

Specifically, the outcomes evaluation surveys policymakers', stakeholders', and researchers' views

about each of the above – with the focus being on policymakers' access to evidence and perceptions of

the utility of available evidence, as well as their interactions with researchers, and their participation in

training courses related to acquiring, using, presenting, and promoting evidence. The outcomes

evaluation questionnaire developed by Johnson and Lavis (2009) is shown in Box 9.2. This should be

administered at the end of the training sessions during the fourth training workshop.



Box 9.1 Users’ perception of health systems operations questionnaire HEALTH POLICY AND SYSTEMS RESEARCH PROJECT

Questionnaire on public & user’s perception on health systems operations

Please answer the questions correctly. The questionnaire is purely for research purpose only.

A. SOCIO-DEMOGRAPHIC DATA

- - -

(iii)

B. ASSESSMENT OF HEALTH SYSTEMS OPERATIONS

1. Health Information & Evidence

(a). To what extent do you have access to health information (eg. about new drugs, technologies, disease

prevention & health programmes)?

(b). How can you rate your confidence (reliability) on available health information?

(c). What is the level of sustainability of sources of health information (eg. media, health institutions,

government & non-government agencies)?

(d). How adequate is the role of government in the provision of health information?

2. Medical Products and Technologies

(a). How would you rate the availability of and access to essential medical products and technologies (eg.,

drugs, vaccines, bednets, chemicals) to the public?

(b). How would you rate the quality of medical products and technologies available to the public?

(c). How consistent are the availability of essential medical products and technologies to the public?

(d). How would you rate the effort of the government in ensuring the availability of medical products and

technologies to the public?

3. Health Finance

(a). How adequate is your personal financial resources for meeting your arising health needs?

(b). How would you rate the level of public confidence on health financial management policies and

practice (budgeting, expenditure, accounting & auditing) in the health sector?

(c). How would you rate the sustainability of the private and government health financing mechanisms?

(d). How would you rate the adequacy of government health finance policies (eg., National health

insurance scheme NHIS) in meeting the health needs of the general public.

4. Health Service Delivery

(a). How would you rate the availability and affordability of quality health service delivery to the public?



(b). How would you rate the level of public confidence on the process of effective health service delivery?

(c). How would you rate the sustainability of health service delivery mechanism available to the public?

(d). How would you rate the efforts of the government to improve efficiency of health service delivery to

the public?

5. Health Workforce

(a). How would you rate the availability of and distribution of qualified health personnel?

(b). How would you rate the level of public confidence on the available health personnel to deliver quality

health services?

(c). How would you rate the level of incentives (eg., salaries, allowances) available to health personnel?

(d). How would you rate the sustainability of supply of qualified health personnel?

(e). How would you rate the efforts of the government in training, development and motivation of the

health workforce?

6. Leadership and Governance

(a). How would you rate the level of public and user‟s involvement by the government in the management

of the health system?

(b). How would you rate the public confidence in the capacity of the government to formulate and

implement functional health policies?

(c). How would you rate the effort of the government to maintain stable political environment that

promotes functional and sustainable health system?

(d). How would you rate the role of the government to ensure efficiency in accountability, transparency

and regulatory frameworks in the health system?

Comments (optional) How do you think the health systems can be improved in Nigeria?.................................................................

...................................................................................................................................................................................

................................................................................................................................ ...................................................

...................................................................................................................................................................................

...................................................................................................................................................................................

...........................................................................................................................................

Thank you for your participation



Box 9.2 Evaluation- outcomes questionnaire


Evaluation – Outcomes Questionnaire

Please circle the number that corresponds to your answer and (if you wish) offer specific comments on any issues raised in particular

questions by identifying the question by number and adding your comments in the space provided on the final page of the

questionnaire.

In this questionnaire we refer to "the KT platform's jurisdiction." In your case this is...........

A number of questions offer "Don't know" as a response option. "Don't know" should be selected only if you feel that you do not

have sufficient information or knowledge to form a view.

Part 1: Views about evidence availability, relationships, and capacity

Section 1A – Availability of health research evidence about high-priority policy issues

1) Please indicate how often, in your view, the following activities or events took place in the KT platform's jurisdiction

(in light of how often it was feasible for each activity or event to take place) over the last two years (or for a shorter period if

you became involved in policymaking more recently).

Never Very rarely Rarely Occasionally Frequently Very

frequently

Always

1 2 3 4 5 6 7

a. Researchers were apprised of high-priority policy issues so that they could orient

their research accordingly

1 2 3 4 5 6 7 Don't

know

b. Primary research was undertaken on high-priority policy issues 1 2 3 4 5 6 7 Don't

know

c.

Systematic reviews of the research literature (i.e., reviews of the research literature

that follow explicit rules to reduce bias in searching the literature, identifying

eligible articles, extracting data, etc.) were conducted or updated on high-priority

policy issues

1 2 3 4 5 6 7 Don't

know

d.

Policy briefs that described evidence about a problem, options for addressing the

problem, and key implementation considerations were produced on high-priority

policy issues

1 2 3 4 5 6 7 Don't

know

e. Brief summaries of articles or reports about primary research (i.e., research

involving original data collection) were produced on high-priority policy issues

1 2 3 4 5 6 7 Don't

know

f. Brief summaries of systematic reviews were produced on high-priority policy

issues

1 2 3 4 5 6 7 Don't

know


(in light of how often it was feasible for each activity or event to take place) over the last two years

(or for a shorter period if you became involved in policymaking more recently).


frequently

Always

1 2 3 4 5 6 7

a. Copies of articles or reports about primary research on high-priority policy issues

were widely disseminated to policymakers working on these issues

1 2 3 4 5 6 7 Don't

know

b. Systematic reviews of the research literature on high-priority policy issues were

widely disseminated to policymakers working on these issues

1 2 3 4 5 6 7 Don't

know

c.

Policy briefs that described evidence about a problem, options for addressing the

problem, and key implementation considerations were widely disseminated to

policymakers working on these issues

1 2 3 4 5 6 7 Don't

know

d. Brief summaries of articles or reports about primary research on high-priority

policy issues were widely disseminated to policymakers working on these issues

1 2 3 4 5 6 7 Don't

know



e. Brief summaries of systematic reviews on high-priority policy issues were widely

disseminated to policymakers working on these issues

1 2 3 4 5 6 7 Don't

know

f. Newsletters that highlight research evidence on high-priority policy issues were

widely disseminated to policymakers working on these issues

1 2 3 4 5 6 7 Don't

know

3) Please indicate how often, in your view, the following activities or events took place in the KT

platform's jurisdiction (in light of how often it was feasible for each activity or event to take place)

over the last two years (or for a shorter period if you became involved in policymaking more recently).


frequently

Always

1 2 3 4 5 6 7

a. Policymakers had access to a personal computer with a functional internet

connection

1 2 3 4 5 6 7 Don't

know

b. Policymakers had access to research evidence on high-priority policy issues

through a website focused on these issues

1 2 3 4 5 6 7 Don't

know

c. Policymakers had access to research evidence on high-priority policy issues

through a searchable database focused on these issues

1 2 3 4 5 6 7 Don't

know

d. Policymakers had access to research evidence on high-priority policy issues

through a list-serve focused on these issues

1 2 3 4 5 6 7 Don't

know

e.

Policymakers had access to research evidence on high-priority policy issues

through a network of researchers or research institutions who could respond in an

ad hoc way to questions about these issues

1 2 3 4 5 6 7 Don't

know

f.

Policymakers had access to research evidence on high-priority policy issues

through a network of researchers or research institutions who maintained some

reserve capacity (i.e., financial and/or human resources that can be redirected when

required) to respond in a systematic way to questions about these issues

1 2 3 4 5 6 7 Don't

know

4) Please indicate the extent to which you agree or disagree with the following statements concerning the

research evidence that was available on high-priority policy issues in the KT platform's jurisdiction over the last two years


Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree

Agree Strongly agree

1 2 3 4 5 6 7

a. Policymakers consider that the available research evidence had little practical

policy application

1 2 3 4 5 6 7 Don't

know

b. Policymakers consider that the available quantitative empirical research evidence

was not helpful in policymaking about health systems

1 2 3 4 5 6 7 Don't

know

c. Policymakers consider that the available research evidence lacked credibility 1 2 3 4 5 6 7 Don't

know

d. Policymakers consider that the available qualitative empirical research was helpful

in policymaking about health systems

1 2 3 4 5 6 7 Don't

know

e. Policymakers consider that the available conceptual (i.e., non-empirical) research

evidence was not helpful in policymaking about health systems

1 2 3 4 5 6 7 Don't

know

Section 1B - Relationships among policymakers and researchers


(in light of how often it was feasible for each activity or event to take place) over the last two years



frequently

Always

1 2 3 4 5 6 7

a.

Policymakers interacted with researchers as part of a priority-setting process to

identify high-priority policy issues for which primary research and systematic

reviews are needed

1 2 3 4 5 6 7 Don't

know

b. Policymakers interacted with researchers as part of primary research or systematic 1 2 3 4 5 6 7 Don't



reviews about high-priority policy issues that they commissioned know

c.

Policymakers interacted with researchers as part of primary research or systematic

reviews about high-priority policy issues with which they were involved as a co-

investigator

1 2 3 4 5 6 7 Don't

know

d. Policymakers interacted with researchers to provide assistance with undertaking

primary research or systematic reviews about high-priority policy issues

1 2 3 4 5 6 7 Don't

know

e.

Policymakers interacted with researchers to provide assistance with interpreting the

findings from primary research or systematic reviews about high-priority policy

issues

1 2 3 4 5 6 7 Don't

know

f.

Policymakers interacted with researchers to provide assistance with designing and

executing strategies to support policymakers‟ use of the findings from primary

research and systematic reviews about high-priority policy issues

1 2 3 4 5 6 7 Don't

know

g. Policymakers interacted with researchers to obtain assistance with acquiring

existing research evidence about high-priority policy issues

1 2 3 4 5 6 7 Don't

know

h.

Policymakers interacted with researchers to obtain assistance with assessing the

quality and local applicability of existing research evidence about high-priority

policy issues

1 2 3 4 5 6 7 Don't

know

i.

Policymakers interacted with researchers to obtain assistance with presenting

existing research evidence about high-priority policy issues to other policymakers

in a useful way

1 2 3 4 5 6 7 Don't

know

j. Policymakers interacted with researchers through legislative committee testimonies

and government-sponsored expert committees or public hearings

1 2 3 4 5 6 7 Don't

know

k. Policymakers interacted with researchers through a rapid-response service designed

to support policymaking processes

1 2 3 4 5 6 7 Don't

know

l.

Policymakers interacted with researchers through policy dialogues designed to

discuss high-priority policy issues and how research evidence can inform how to

address these issues

1 2 3 4 5 6 7 Don't

know

m. Policymakers interacted with researchers through research conferences 1 2 3 4 5 6 7 Don't

know

n. Policymakers interacted with researchers through informal conversations with

personal contacts

1 2 3 4 5 6 7 Don't

know

o. Policymakers interacted with researchers through long-term partnerships (e.g.,

through an advisory board)

1 2 3 4 5 6 7 Don't

know

Section 1C – Policymakers’ capacity to

support the use of health research evidence in health systems policymaking

6) Please indicate how often, in your view, the following activities or events took place in the KT platform's

jurisdiction (in light of how often it was feasible for each activity or event to take place) over the last two years



frequently

Always

1 2 3 4 5 6 7

a.

Policymakers participated in training courses, workshops, seminars, or briefings to

develop their capacity to acquire existing research evidence about high-priority

policy issues

1 2 3 4 5 6 7 Don't

know

b.


develop their capacity to assess the quality and local applicability of existing

research evidence about high-priority policy issues

1 2 3 4 5 6 7 Don't

know

c.


develop their capacity to present existing research evidence about high-priority

policy issues

1 2 3 4 5 6 7 Don't

know

d.


develop their capacity to promote a climate that supports research use and ensure

policymaking processes have a place for research evidence

1 2 3 4 5 6 7 Don't

know



Section 1D – KT platform’s contributions

7) Please indicate the extent to which you agree or disagree with these statements about the KT platform‟s contributions

over the last two years (or for a shorter period if you became involved in policymaking more recently).

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree

Agree Strongly

agree

1 2 3 4 5 6 7

a. The KT platform has contributed to the availability of research evidence on high

priority issues

1 2 3 4 5 6 7 Don't

know

b.

The KT platform has contributed to strengthening relationships among

policymakers, researchers, and knowledge-translation specialists (i.e., individuals,

other than journalists, whose expertise and role relate to supporting the use of

research evidence outside the research community)

1 2 3 4 5 6 7 Don't

know

c.

The KT platform has contributed to strengthening policymakers‟ and researchers‟

capacity to support the use of health research evidence in health systems

policymaking.

1 2 3 4 5 6 7 Don't

know

Part 2: Assessment of your unit’s or department’s capacity to acquire, assess, adapt, and apply health research

evidence on high-priority policy issues

Section 2A - Acquiring health research evidence

8) Can your unit/department find and obtain research evidence on high-priority policy issues?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a. We have skilled staff to find and obtain research evidence on high-priority policy

issues.

1 2 3 4 5 6 7 Don't

know

b. Our staff has enough time to find and obtain research evidence on high-priority

policy issues.

1 2 3 4 5 6 7 Don't

know

c. Our staff has the incentive to find and obtain research evidence on high-priority

policy issues (e.g., it is used in the policymaking process).

1 2 3 4 5 6 7 Don't

know

d. Our staff has the resources to find and obtain research evidence on high-priority

policy issues.

1 2 3 4 5 6 7 Don't

know

e. We have arrangements with external experts who find and obtain research evidence

on high-priority policy issues.

1 2 3 4 5 6 7 Don't

know

9) Is your unit/department looking for research evidence in the right places?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a.

We routinely look for research in journals that we can access by subscription,

online, or through a library. An example is the Bulletin of the World Health

Organization.

1 2 3 4 5 6 7 Don't

know

b.

We routinely look for research in non-journal reports (i.e., grey literature) that we

can access online, through a library, or by direct mail. An example is a program

evaluation report that was prepared for a donor.

1 2 3 4 5 6 7 Don't

know

c.

We routinely look for research in databases that we can access by subscription, on

CD, or online. Examples are The Cochrane Library and the Reproductive Health

Library.

1 2 3 4 5 6 7 Don't

know

d. We routinely look for information on websites that collate and/or evaluate sources

of research evidence. An example is The Reproductive Health Library.

1 2 3 4 5 6 7 Don't

know

e. We routinely interact with researchers through formal and informal networking

meetings.

1 2 3 4 5 6 7 Don't

know

f. We routinely get involved with researchers as a host for their research activities,

funding sponsor, or co-investigator.

1 2 3 4 5 6 7 Don't

know

g. We routinely learn from our peers through formal and informal networks to

exchange ideas, experiences, and best practices.

1 2 3 4 5 6 7 Don't

know



Section 2B - Assessing health research evidence

10) Can your unit/department tell if the available research evidence warrants serious consideration?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a. Staff in our unit/department has skills and tools for evaluating the quality of

research evidence.

1 2 3 4 5 6 7 Don't

know

b. Staff in our unit/department has skills and tools for taking into account equity

considerations in evaluating the research evidence.

1 2 3 4 5 6 7 Don't

know

c. Staff in our unit/department has the skills and tools for evaluating the local

applicability of research evidence that was conducted in other settings.

1 2 3 4 5 6 7 Don't

know

d. Staff in our unit/department has the skills and tools for taking into account scaling

up considerations in evaluating the research evidence.

1 2 3 4 5 6 7 Don't

know

e.

Staff in our unit/department has the skills and tools to identify the relevant

similarities and differences between what we do and what the research evidence

says.

1 2 3 4 5 6 7 Don't

know

f.

Our unit/department has arrangements with external experts who use critical

appraisal skills and tools to help us to assess if the research evidence warrants

serious consideration.

1 2 3 4 5 6 7 Don't

know

Section 2C - Adapting health research evidence

11) Can your unit/department present research evidence to policymakers in a useful way?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a.

Our unit/department has enough skilled staff with time, incentive, and resources

who use research communications skills to present research evidence concisely and

in accessible language.

1 2 3 4 5 6 7 Don't

know

b.


who use research communications skills to synthesize in one document all relevant

research evidence, along with information and analyses from other sources.

1 2 3 4 5 6 7 Don't

know

c.


who use research communications skills to link research evidence to key policy

issues facing policymakers.

1 2 3 4 5 6 7 Don't

know

d.


who use research communications skills to provide specific policy options to

policymakers.

1 2 3 4 5 6 7 Don't

know

e.

Our unit/department has arrangements with external experts who use research

communications skills to help us to present research evidence to policymakers in a

useful way.

1 2 3 4 5 6 7 Don't

know

Section 2D - Applying health research evidence

12) Does your unit/department lead by example and show how it values the use of research evidence?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a. Using research evidence is a priority in our unit/department. 1 2 3 4 5 6 7 Don't

know

b. Our unit/department has committed resources to ensure research evidence is

acquired, assessed, adapted, and applied in policymaking processes.

1 2 3 4 5 6 7 Don't

know

c. Our unit/department ensures staff is involved in discussions about how research

evidence relates to our main goals.

1 2 3 4 5 6 7 Don't

know

d.

The management of our unit/department has clearly communicated our strategy and

policy priorities so those conducting or monitoring research know what is needed

to support our goals.

1 2 3 4 5 6 7 Don't

know



e. We communicate internally in a way that ensures there is information exchanged

across the entire unit/department.

1 2 3 4 5 6 7 Don't

know

f. Our corporate culture values and rewards flexibility, change, and continuous

quality improvement with resources to support these values.

1 2 3 4 5 6 7 Don't

know

13) Do your unit's/department's policymaking processes have a place for research evidence?

Strongly

disagree

Disagree Somewhat

disagree

Neither agree

nor disagree

Somewhat

agree


1 2 3 4 5 6 7

a.

When we develop or change a policy, we usually allow enough time to identify

researchable questions and acquire, assess, and consider research evidence that

addresses these questions.

1 2 3 4 5 6 7 Don't

know

b.

Our management team has enough expertise to evaluate the feasibility of each

policy option, including potential impact across the unit/department and on

patients/clients, healthcare professionals, development partners, and other

stakeholders.

1 2 3 4 5 6 7 Don't

know

c.

Policymakers in our unit/department usually give formal consideration to any

perspectives on a policy issue brought forward by staff who have identified high

quality, locally applicable research evidence on the policy issue.

1 2 3 4 5 6 7 Don't

know

d. Staff and appropriate stakeholders know when and how major policies will be

developed or changed.

1 2 3 4 5 6 7 Don't

know

e. Staff and appropriate stakeholders know how and when they can contribute

research evidence and how that information will be used.

1 2 3 4 5 6 7 Don't

know

f. Staff who have provided research evidence usually participate in policymaking

discussions.

1 2 3 4 5 6 7 Don't

know

g. Relevant on-staff researchers are part of policymaking discussions. 1 2 3 4 5 6 7 Don't

know

h. Staff and appropriate stakeholders receive feedback on policymaking processes

with a rationale for the new or changed policy.

1 2 3 4 5 6 7 Don't

know

i. Staff and appropriate stakeholders are informed of how available research evidence

influenced the choices that were made in our unit/department.

1 2 3 4 5 6 7 Don't

know

Part 3: Role and background

14) I am a (please tick (√ ) single most appropriate role category):

Broad

role category


Tick

(√)

single most

appro-priate


government

Public policymaker (i.e., elected official, political staff, or civil servant) in a

su national government (e.g., province/state or a district if the latter has

independent policymaking authority)















Other

15) I have been working in my current position for _____ years.

16) If you identified yourself as a policymaker, stakeholder, or "other," please indicate if you have training

and/or extensive experience as a researcher (circle one):

Yes / No

17) If you identified yourself as a researcher, stakeholder, or "other," please indicate if you have experience as a policymaker

(circle one):

Yes / No

Thank you!



Additional Thoughts (Optional)

Do you have any comments regarding issues raised in particular questions?

(If the space provided is insufficient to accommodate all your ideas, please feel free to attach additional pages.)

_______________________________________________________________________________________



CHAPTER 10

TRAINING WORKSHOP FOR CAPACITY ENHANCEMENT ON LEADERSHIP,

GOVERNANCE AND MANAGEMENT FOR HEALTH SYSTEMS STRENGTHENING

10.1 Methods for Training Workshop

This workshop should be the fifth and final of the series of interventional training workshops. As in

previous workshops efforts should be made to retain the same participants from the first training

workshop till this fifth workshop. This fifth workshop should focus on enhancing the capacity for

effective leadership, governance and the management for the strengthening of the health systems. In

the Nigeria study the theme of the workshop was: “Leadership, Governance and Management:

Critical Competencies for Health Systems strengthening”. The lecture should be centered on the

following topic area:

10.1.1 Leadership, Governance and Management: Critical Competencies for Health Systems

strengthening

(i). Leadership and governance factors that ensure functionality of the health systems (Policy

guidance; Intelligence and oversight; Collaboration and coalition building; Regulation; System design;

Accountability

(ii). Impact of weak leadership and governance

(iii). Policy options and implementation strategies for addressing leadership and governance problem

(iv). Accountability of the key actors in the health system to the beneficiaries; a policy process that

enables the interplay of the key competing interest groups to influence policymaking; sufficient state

capacity, power, and legitimacy to manage and regulate the policymaking process; effective

engagement of non-state actors in the policy arena; instituting performance measurement system;

establishing health system research mechanism

(v). Leading/Managing and Health Systems Strengthening

(vi). Leadership and management strategies to improved health outcomes (scanning; focusing;

aligning and mobilizing; inspiring; planning; organizing; implementing; monitoring and evaluation)

(vii). Promoting Good Governance in Public and Private Health Organizations

(viii). Dimensions of governance in the health sector (information and assessment capacity; policy

formulation and planning; social participation and system responsiveness; accountability,

transparency, and regulation).



(ix). Improved Health System Performance issues (Equity; Access; Quality; Efficiency;

Sustainability).

10.2 Staff and organizational appraisal/performance assessment

Staff and organizational appraisal and performance assessment are major ways to evaluate the

effectiveness of leadership and governance of a health ministry in strengthening the health systems.

People-centered health systems cannot be strengthened without good management and leadership.

Good leadership aims to manage and lead better so that teams, units, or organizations can fully use and

continuously develop their potential to transform human and financial resources and other inputs into

improved services and, ultimately, improved health outcomes. The questionnaire in Box 10.1 is

designed to evaluate human resource management in terms of of recruitment policies, performance

assessments, and staff appraisals. The questionnaire is also a useful tool for organizational self-

assessment on the use research to make informed decision. The questionnaire was based on the

following assessment tools:

(i). Huddart (2005) developed for the Jordan Human Resources Development Project Report No. 5.

(ii). Canadian health services research foundation (CHSRF) self-assessment tool and discussion guide

for Health services management and policy organizations.



Table 10.1 Staff and organizational appraisal/performance assessment questionnaire HEALTH POLICY AND SYSTEMS RESEARCH PROJECT

STAFF AND ORGANIZATIONAL APPRAISAL/PERFORMANCE ASSESSEMENT QUESTIONNAIRE

HR Components Stages of Human Resource Management Development and Characteristics Comments

1 2 3 4

HR Management Capacity

HR Budget There is no

budget

allocated for

HR staff

or HR activity

within the

organization.

There is limited

money available to

fund an HR

position or to

conduct HR

activities (e.g.

training, systems

development,

performance

planning and

evaluation.

Budget is allocated

for HR staff and

related activities.

Allocation is

irregular and

cannot be relied

upon for any useful

long-range

planning or the

development of

HR systems.

Money for HR staff and

related activities is a

permanent budget item,

reviewed annually and

adjusted if possible.

HR Staff There are no

staff

specifically

charged

with

responsibility-

y for

HR functions.

There are HRM

staff in the

organization, but

they have limited

experience related

to this field

(personnel,

recruitment,

management) and

/or have other

functions in the

organization as well

as HRM.

There are trained

HRM staff in the

organization, but

only at a level to

maintain basic

procedures and

record-keeping

functions.

There are experienced

HRM staff in the

organization who

maintain HR functions.

They participate in long-

range planning for the

organization.

Human Resource Planning

Organizational

Mission and

Goals

No formal

mission

statement or

organization

goals

exist.

Mission/goals exist

but are not formally

linked to HR

planning (e.g.

staffing plan,

training, etc.)

Mission/goals

linked to annual

HRD planning and

also used for

forecasting

long-range staffing

and recruitment

needs.

Mission/goals linked to

annual HR planning and

also used for forecasting

long-range staffing and

recruitment needs.

HR Planning No annual HR

plan

exists.

Annual HR plan

exists, but is not

based on a formal

assessment of the

mission,

organizational

goals, staffing

needs, training

outputs or existing

employee data.

Annual HR plan

exists based on

organizational

goals, staffing

needs, training and

employee data, but

it is not further

evaluated for

effectiveness.

Annual HR plan based

on organizational goals

and training outputs

exists. It is implemented,

evaluated and used for

long-range strategic

planning.

Personnel Policy and Practice

Job

Classification

System

No formal

system

exists to

classify jobs

and the skills

There is some

attempt to classify

jobs, but it is

uneven and

incomplete.

A job classification

system exists, but

it is not used as a

basis for other

HRM functions

A job classification

system exists and is used

in a formal manner for

other HR planning and

staffing functions.



and

qualifications

required for

each position.

(e.g. job

descriptions,

hiring, salary and

benefits).

Compensation

& Benefits

System

No formal

system exists

for

determining

the salary

scale and

benefits

provided for

each position

within the job

classification.

A formal system

exists, but it is not

used in a routine

manner.

A formal system

exists, is

understood by all

employees and

used in a consistent

manner.

A formal system exists

and is used consistently.

It is also used to

determine salary upgrades

and merit awards.

Recruitment,

Hiring,

Transfer and

Promotion

No formal

process

exists for

recruiting,

hiring,

transfer and

promotion

according

to job

descriptions.

There are systems

for hiring, etc. but

they are not

followed.

There are formal

systems, based on

established criteria,

but they are not

used consistently.

There are formal systems,

monitored and used in all

hiring, transfer and

promotion decisions.

Orientation

Program

There is no

formal

orientation

program for

new

employees.

There is a program,

but it is not

implemented on a

regular basis.

Orientation is

offered in a routine

manner, but does

not emphasize the

mission, goals and

performance

standards expected

by the

organization.

Orientation is offered to

all new employees,

emphasizes the mission,

goals and performance

standards expected, and

also makes people fees

welcomed and valued.

Policy Manual

(Org‟al chart,

work hours,

policy, discipline,

grievances,

benefits, travel,

etc.)

No policy

manual exists.

Policy manual does

exist, but it is out of

date and does not

include all of the

relevant

information.

A current policy

manual does exist

but it is not

available to all

employees and is

not always used as

a basis for

personnel

decisions.

An upgraded policy

manual does exist and is

available to all

employees. It serves

as a reference guide to all

questions about

employment in the

organization and is

reviewed and updated

regularly.

Termination

& Grievance

Procedures

No formal

procedures

exist.

Formal procedures

do exist, but they

are not clearly

related to

performance

standards.

Formal procedures

based on

performance

standards exist, but

they are not

followed in any

consistent manner.

Formal procedures based

on performance standards

are known to all

employees and used

consistently.

Relationship

with Unions

(if appropriate)

There is no

link between

HRM,

management

and the

Links exist between

HRM, management

and

union, but roles are

not clear.

Management

involves HR union

issues, but on an

irregular basis.

Management, HRM and

the union work together

to resolve issues and

prevent problems.



union/s.

Labor Law

Compliance

There is no

review of

HR policies to

ensure

compliance

with local

and/or

national labor

law.

There is some effort

to review labor law,

but it

is not done on a

regular basis.

A review of the

labor law is done

regularly as a

formal part of the

HR function, but

policy is not

always adjusted to

ensure compliance.

HR policy and practice is

adjusted as needed to be

in compliance with the

local and/or national

labor law.

Human Resource Data

Employee Data

(No , Where

deployed, skill

level, gender,

cadre, year of

hire, etc.)

None of this

data is

collected on

any kind of

systematic

basis.

Most of this data is

collected, but not

maintained or kept

up to date.

All of this data is

available and up to

date, but data is not

formally used in

HR planning or

forecasting.

All of this data is

available and up to date.

Systems are in place.

Data is formally used in

HR planning and

forecasting.

Computerisation

of Data

There are no

computers or

data systems

available to

the

organization,

externally or

internally.

There are

computers in

place, but no

resources to

develop systems for

data management.

Computers and

data

management

systems are

available, but HR

information is not

produced regularly

for management

decision making.

Computers and data

management systems are

in place and data files are

up to date. HR

information is regularly

produced and used for

decision-making.

Personnel

Files

No individual

employee

records exist.

Limited employee

personnel files are

maintained but not

regularly updated.

Personnel files for

all employees are

maintained & kept

up to date but there

is no policy for

employee access or

use of this data.

Updated personnel files

for all employees exist &

also policies for

appropriate use

(e.g. confidentiality,

employee access).

Performance Management

Job Descriptions

No job

descriptions

are developed.

Some staff have job

descriptions, but

they are not always

up to date and/or

are very

general, lacking job

responsibilities and

supervision.

All staff have job

descriptions, but

they are not all

complete or

up to date with

specific duties and

lines of

supervision.

Complete job descriptions

exist for every employee

and are kept up to date

through a regular process

of review. Specific duties

are clearly stated.

Staff

Supervision

There is no

clear

system of

supervision.

Lines of authority

are unclear. Staff

are not recognized

for their

achievements.

There are

established lines of

authority, but the

supervisor‟s role &

function is not

understood and

little supervision

takes place.

Limited staff

recognition.

Supervisors understand

their roles & lines of

authority and meet

regularly with their

employees to develop

work plans, evaluate

performance and publicly

recognize staff for their

achievements.



Work

Planning and

Performance

Review

There is no

work

planning and

performance

review system

in place.

A work planning

and performance

review system is in

place, but it

is informal and

does not include

work plans and

performance

objectives

developed jointly

with staff.

There is a formal

system and

supervisors are

required to develop

work plans and

performance

objectives with

each employee and

to review

performance in the

past, but this is not

done on a

consistent basis.

Supervisors and

employees develop work

plans jointly and

performance reviews are

con-ducted on a regular

basis. Orientation

sessions and a manual are

provided to all staff.

Reviews are used for

personnel decisions.

Training

Staff Training There is no

staff training

plan.

Training is offered

on an ad hoc basis,

but is not based on

a formal process of

assessing staff

needs nor is it

linked to the

organization‟s key

priorities and

changes in the

health sector and

health practices.

Training is a

formal

component of the

organization and

linked to staff &

organizational

needs, but it is not

available for all

staff, nor is it

evaluated for

results.

Training is a valued part

of the organization and

opportunities are

developed for staff based

on their needs and also on

those of the organization.

Management

& Leadership

Development

There is no

policy or

philosophy

regarding

the

importance of

developing

strong

management

capacity

and future

leaders for

the

organization.

There is an

emphasis on

developing

management

capacity but it is no

done on a regular

basis.

The organization

makes an effort to

develop managers

and future leaders

through training,

and also through

mentoring and

challenging job

assignments, but

participation is

selective.

A plan for management

and leadership develop-

ment is in place and there

is an opportunity for

everyone to participate

based on performance

and other established

criteria.

Links to

External Pre-

Service

Training

There is no

formal link

with the pre-

service

training

institutions

which train

employees for

the health

sector.

There is a loose

relation-ship

between the

organization and

pre-service training

institutions but it is

not used in any

formal way for

workforce training

and development.

The organization

and pre-service

training institutions

work together to

ensure that the

curriculum is based

on skills,

knowledge and

attitudes required

in the

workplace.

The organization and pre-

service training

institutions also offer

regular in-service

training for staff already

in the workplace to

upgrade their skills and

knowledge (e.g. mgmt.

training).



Organizational Self-assessment on the use research to make informed decisions

Based on the self-assessment, our organization should work on the following areas so that we can use research

better to make informed decisions that help meet our goals and objectives: Comments

1. Establish research as a

priority in our organization: I

feel research in our organization

should have: (Check one)

(a). Much

higher

priority

(b).

Somewhat

higher

priority

(c). The same

priority

(d). Somewhat

lower priority

(e). Much

lower priority

2. Integrate the use of

research into the work of

people in our organization: I

feel we need to: (Check one)

(a).

Integrate

research

much

more

often

(b).

Integrate

research

slightly

more often

(c). Maintain

our current

level of

integrating

research

(d). Integrate

research

slightly less

often

(e). Integrate

research

much less

often

3. Encourage the use of

research by our decision

makers: I feel our decision

makers: (Check one)

(a). Do

not use

research

at all

(b). Use

research

poorly

(c). Use

research

inconsistently

(d). Use

research with

some

consistency

(e). Use

research

well/enough

4. Increase our capacity for

research:

(Check all that apply. If you

have more than one answer,

please rate your needs from 1 to

5, with 1 being the highest

priority.) We need: (Check one)

(a).

Skilled

staff

(b).

Resources

(c). Time

(d). Incentives

(e).

Arrangements

with external

experts

5. Acquisition of research: We

need better access to: (Check all

that apply. If you have more

than one answer, please rate

your needs from 1 to 6, with 1

being the highest priority.)

(a).

Journals

& Non-

journal

reports

(grey

literature)

(b).

Databases

(c). Web sites

(d).

Opportunities

to work with

researchers

(e). Learning

from peers

6. Assessment of research: We

need to:

(Check the one that is most

appropriate or best describes

your situation.)

(a). Begin

to assess

and adapt

research

(b). Assess

and adapt

research

more often

(c). Maintain

our current

ability to

assess and

adapt

research

(d). Adapt and

assess research

a bit less often

(e). Adapt and

assess

research

much less

often

7. Linking of research results

to key issues facing our

decision makers: Our decision

makers need to: (Check the one

that is most appropriate or best

describes your situation.)

(a). Begin

to

consider

research

in making

decisions

(b).

Consider

research

more often

in making

decisions

(c). Maintain

our current

frequency of

considering

research

(d). Consider

research a bit

less often

(e). Consider

research

much less

often

PARTICIPANT BIODATA

(i). Gender: Male

(ii). Age category: 25- -

OFFICIAL DESIGNATION ATTRIBUTES



(i). Name of your organization:…………………………………………………………………………..............................

(ii). Designation:………………………………………………………………………………….........................................

(iii). Duration in designation:………………………………………………………………………....................................

(iv). Influence on policymaking process: Direct

(vi). Membership of health-

Participant Workshop No……………………



CHAPTER 11

PROJECT EVALUATION DESIGN

11.1 Conceptual design of the evaluation

The conceptual design of the planned process evaluation takes into consideration the various

constituents of the health system. The constituents of the health systems have been summarized into eight

priority areas in line with the WHO‟s Framework for Action on health systems WHO (2007). These

priority areas include health administration, service delivery, information and evidence, medical

products and technologies, health work force, health financing, population and health interventions,

leadership and governance. These priority areas, which have also been described, by WHO (2007) as

health system building blocks, have been used to categorize the administrative data/data from

management information system in this manual (Table 11.1). Under each category the main data

components/elements, composite indicators and the specific indicators to be measured have been

identified. Evaluating the performance of the major constituents of the health system using a

combination of monitoring/operational research and social research techniques will provide useful

information on the effects of the intervention to be implemented. While the monitoring/operational

research involves the assessment of data collected by target organizations for purposes other than the

evaluation proper (eg. administrative data), the social research involves the collection of data specifically

for the evaluation (to be accomplished via qualitative and quantitative survey). The social research

evaluation instruments are designed in such a way that they would be consistent with administrative data

elements. Evidence from these two techniques is used to triangulate, verify, substantiate and qualify

findings.

This evaluation design is based on the well proven fact that a relationship exists between the health system

and health policymaking process (Gonzalez-Block 2004; Green and Benneth 2007). Therefore any change

in the health policymaking process is most likely to affect the health systems. The aim is to effect a

positive change in the health policymaking process through the promotion of HPSR evidence use, which is

expected to have an impact on the health system. Thus, the findings from the measurement of performance

of the various components of the health system will indicate the extent the interventions have satisfied the

project goals and purposes. This is the main concept of this evaluation design. The specific indicators of

the administrative data/data from management information system (MIS) to be measured are clearly shown

in Table 11.1. These specific indicators are adapted from a WHO previous report (Sadana et al. 2006), and

were modified to capture the requirements of this project. The methods of measurement include document



reviews, key informant interviews and re-analysis of existing data in line with an earlier report

(Sadana and Pang 2004). One or a combination of these methods is used for measurement depending

on the type of indicator being considered. The measurement strategy includes the use of the likert

scale rating of four options (eg. 1. grossly inadequate, 2. inadequate, 3. fairly adequate, 4. Adequate, 5.

very adequate); yes or no; proportions; percentages; etc. The administrative data/data from MIS is

obtained from the administrative records/data bases of target organizations, although the major target

organization would be the ministry of health. Some of the indicators may not be applicable to all the

target organizations.

From the assessment of data collection system of the organizations, the team should gained an

understanding of the type of relevant data available, data being collected, how and where to find

them, how to collect them, and analyze them in a meaningful way. This information will enhance the

development of the monitoring/operational research as well as the social research mechanisms which

are the strategies designed to accomplish the process evaluation design in a project of this sort.

11.2 Data from monitoring/operational research

Because monitoring/operational research involves the use and analysis of data collected other than for

the evaluation, efforts should be made to sensitize the heads of the organizations and their nominees

on the importance of such data so that proper handling, recording and documentation of the data are

ensured. The four categories of data to be collected which are relevant to the monitoring/operational

research include: (i). administrative data, (ii). data from Management Information Systems (MIS) e.g.

programme database, (iii). performance measurement data about resources (staff and financial), etc.,

(iv). Data from special monitoring exercises e.g. pro forma attached to specific cases to identify stages

and timing. These sources of evidence would be used to triangulate, verify, substantiate and qualify

findings. Emphasis was placed on the administrative data and data from Management Information

Systems because the other two categories can be classified under them.

The use of administrative data and data from MIS is vital to the evaluation phase because they are

data collected in the course of programmatic activities particularly in the Health Ministry for the

purposes of program operation, service provision, or decision-making-essentially; they are not

necessarily collected for research activities. Therefore in consonance with the objectives of this type of

project, the Project Team should concentrate on administrative data and data from MIS supplying the

following information (Table 11.1); (i). Health administration- planning, and administration,



performance measurement, benchmarking, and best practice promotion. (ii). Service delivery – service

organization and management, access, quality, safety, and continuity of care across health conditions,

across health facilities and over time. (iii) Information and evidence – the generation and strategic use

of information, evidence and research on health and health systems. (iv). Medical products and

technologies – access to essential medical products and technologies of assured quality, safety,

efficacy and cost-effectiveness, and their scientifically sound and cost-effective use. (v). Health

workforce – entry into and exits from the health workforce, distribution and performance of existing

health workers, staff training. (vi). Health financing – health system funding, utilization and resource

allocation. (vii). Population and health interventions – characteristics of people and communities,

health affecting interventions. (viii). Leadership and governance – strategic policy frameworks,

effective oversight and control, coalition-building, regulation, attention to health-system design issues,

promotion of accountability.

11.3 Administering the evaluation questionnaire

The process evaluation questionnaire should be administered to individuals who are directly in charge

of the operations of the health ministry e.g., the permanent secretary, or any other person who is in the

position to provide detailed accurate information on the measurement indicators outlined in Table

11.1. Information derived from completed questionnaire should provide the evaluation outcome.

Table 11.1 Questionnaire for project evaluation from Administrative data/ Data from

management information system.

Category of

Administrative

data/Data from

Management

Information

System

Main data

components/elements

Composite

indicators

Specific indicators/measurement strategy

1. Health

administration

(i). Planning Use of evidence in

planning, past record

use, capacity of

planners or

policymakers

(experience in health

policy, use of

evidence).

(a). existence of a policy on health research

involving all key stakeholders? (*yes/no). (b). are

stakeholders‟ views defined and integrated within a

policy on health research? (yes/no). (c). existence of

a forum or process to coordinate the setting of

health research priorities? (yes/no). (d). extent of

use of research done by others: (gi,in,fa,ad,va)*

(e). extent of use of research initiated/done by the

organization: (gi,in,fa,ad,va). (f). use of data

collected routinely or by survey: (gi,in,fa,ad,va).

(g). relevance of evidence used: (gir,ir,fr,re,vr)*.

(h). proportion of policymakers with following



qualifications: BSc/BA( ); MSc/MA ( ); PhD ( ).

(i). number of policy documents made by

policymakers in the last 5years ( ).

(ii). Performance

measurement

Number of health

policies made,

functionality of

health programmes,

coverage, public

acceptability, project

outcome

(a). number of health policies made or updated in

the last 5 years ( ). (b). functionality of health

programmes (eg. immunization): % coverage ( );

number of beneficiaries over number of relevant

population ( ); number/types of relevant facilities or

equipment acquired ( ); number of people

trained/employed ( ).

(iii). Bench

marking/best practice

Adherence to

international/national

health policy

standards/guidelines.

(a). availability of ethical unit (yes/no).

(b). availability of document on health research

ethics (yes/no). (c). availability of document on

bench marking/best practice (yes/no). (d). degree of

adherence to guidelines on ethics/ bench

marking/best practice (gi,in,fa,ad,va).

2. Service

delivery

(i). Service

organization and

management

Organizational/mana-

gement structure,

relationship/linkages

to health agencies,

conflicts, service

overlap.

(a). availability of organizational/management

structure (yes/no). (b). any relationship/partnership

with other agencies (yes/no); any counterpart

funding involvement (yes/no); number of MoU ( );

number of letters of collaboration/co-operation ( ).

(c). conflicts-number of reported cases of clash of

duties, tasks, responsibilities per year in the last 5

years ( ). (d). overlap-number of known

organization doing the same task, service/duties ( ).

(ii). Access to health

services

Distribution of health

facilities/services,

affordability,

acceptability,

attendance, hospital

bed occupancy rate.

(a). number of available health facilities-primary

health care centres ( ); secondary health facilities( );

tertiary health facilities ( ). (b). availability of

health intervention programmes (yes/no); if

available what types? (…).

(c). affordability/acceptance-rate of patronage of

available health facilities ( ). (d). hospital bed

occupancy rates in the secondary health facility ( );

in the tertiary health facility ( ).

(iii). Quality of health

services

Manpower

competence, type and

nature of service,

birth rate, death rate,

existence of

surveillance unit.

(a). proportion of doctors ( ), nurses ( ),pharmacists

( ), per health facility. (b). recorded birth rate. (c).

recorded death rate. (d). Availability of functional

disease surveillance unit (yes/no).

(iv). Safety Availability/quality

of safety manuals,

storage/sterilization

facilities.

(a). availability of safety guidelines/manual

(yes/no). (b). availability of storage/sterilization

facilities (yes/no).

(v). Continuity of

health care across

health condition and

across health facilities

Availability of drugs,

consistency of

disease control

programmes

(immunization,

malaria control, HIV

control etc),

availability of regular

field officers.

(a). regularity of drug supply: (gi,in,fa,ad,va). (b).

consistency of disease control programmes:

(gi,in,fa,ad,va). (c). availability and regularity of

health field officers: (gi,in,fa,ad,va).

3. Information (i). Generation and Availability and (a). proportion of policymakers with internet facility



and evidence strategic use of

information

types of information

technology facilities,

access to media,

educational

materials/research

reports, availability

of record keeping and

storage system, vital

statistics.

in their office ( ). (b). number of different

periodicals subscribed by the organization ( ).

(c). availability of organizational information data

base (yes/no). (d). number of health research output

published per year ( ) (e). availability of quality &

peer review mechanisms (yes/no). (e). existence of

vital statistics (yes/no).

(ii). Evidence and

research on health

systems

Availability and

access to research

findings, number of

research projects

initiated/executed.

(a). number of research projects initiated/executed

in the last 5 years ( ). (b). number of active health

researchers in the organization ( ). (c). number of

different health- related academic/professional

journals subscribed by the organization ( ). (d).

availability of mechanisms to review primary

research outputs (yes/no). (e). number of systematic

reviews produced per year ( ).

4. Medical

products and

technologies

(i). Access to essential

medical products and

technologies

Availability of

essential drugs/basic

medical equipment,

availability/use of

modern technologies.

(a). degree of availability of essential drugs:

(gi,in,fa,ad,va). (b). degree of availability of basic

medical equipment: (gi,in,fa,ad,va).

(ii). Scientifically

sound, safety, efficacy,

and cost effectiveness

Availability of

quality assurance

/control mechanism,

nature of

procurement system.

(a). availability of quality assurance/control

mechanism (yes/no). (b). degree of adherence to

quality assurance/control mechanism:

(gi,in,fa,ad,va). (c). nature of procurement system

(direct/indirect)

5. Health

workforce

(i). Entry into and exit

from health workforce

Employment/retireme

nt/disciplinary

policies.

(a). availability of policy document on

employment/retirement/discipline (yes/no).

(ii). Distribution and

performance of health

worker

Staff posting, staff

appraisal, tenure,

promotion system.

(a). availability of policy document on staff posting/

staff appraisal/ tenure/ promotion (yes/no).

(iii). Staff training Availability/regularit

y of training,

sponsorship

programmes, training

fields/specialties.

(a). availability of training/sponsorship programme

(yes/no). (b). degree of regularity of

training/sponsorship programme: (gi,in,fa,ad,va).

(c). relevance of training fields/specialties:

(gir,ir,fr,re,vr)

(iv). Remuneration

and compensation

Incentives/wages

(salaries, allowances,

benefits), payment

process.

(a). adequacy of incentives/wages: (gi,in,fa,ad,va).

(b). efficiency of payment process: (gi,in,fa,ad,va).

6. Health

financing

(i). Health system

funding

Types/sources/availa

bility of funding, user

fees,

subsidies/waivers,

health insurance

scheme.

(a). sources/proportion of funding: external grants

( ); government subventions ( ); loans ( ); income

( ). (b). amount of resources allocated in

accordance with nationally stated priorities? ( )

(c). availability of health insurance scheme

(yes/no). (d). availability of subsidies/waivers for

service users (yes/no). (e). implementation of user

fees (yes/no)

(ii). Resource

allocation, utilization,

and accountability

Budgetary system,

accounting/audit

system.

(a). existence of budgetary guidelines (yes/no). (b).

extent of adherence to budgetary guidelines:

(gi,in,fa,ad,va). (c). existence of audit unit



(yes/no). (d). ability of audit unit to identify frauds,

errors and lapses in accounting process:

(gi,in,fa,ad,va).

7. Population

and health

interventions

(i). Characteristics of

people and

communities

Environmental/socio-

demographic/cultural

/economic factors.

(a). availability of records/data on: environmental

factors-environmental temperature (yes/no); rainfall

(yes/no); natural disaster (flooding) (yes/no).

(b). availability of records/data on socio-economic

factors-population distribution (yes/no);

demography (age, gender, occupation, marital

status, religion, literacy level) (yes/no); income

levels (low, middle, high) (yes/no).

(ii). Health affecting

interventions

Epidemiological data,

essential drugs,

public health

services, transport

and other logistics,

infrastructure,

community services.

(a). availability of records /data on-epidemiological

information (yes/no); essential drugs (yes/no);

public health services (yes/no); transport (yes/no);

infrastructure (yes/no); community service (yes/no).

8. Leadership

and governance

(i). Strategic policy

framework

Organogram, conflict

resolution, research

and development.

(a). effectiveness of organizational structure:

(gi,in,fa,ad,va). (b). existence of mechanism for

conflict resolution (yes/no). (c). existence of

programmes/policy on research and development

(yes/no).

(ii). Effective

oversight and control

Use of committees,

regulatory

mechanisms, role of

legislators

(a). existence of boards, ethical committees, and

consultancy for oversight function (yes/no).

(b). extent of adherence to regulatory

provisions/guidelines (yes/no): (gi,in,fa,ad,va).

(c). existence of monitoring and evaluation

activities clearly linked with strengthening health

system (yes/no).

(ii). Coalition-building Internationally/extern

ally initiated

collaborations and

partnerships.

(a). level of collaboration/partnership with external

(international) agencies: (gi,in,fa,ad,va).

(iii). Attention to

health system design

issues

Initiation/updates/rev

ision of health system

framework.

(a). regularity of formulation/revision of health

system framework/policies: (gi,in,fa,ad,va).

(gi,in,fa,ad,va)* gi= grossly inadequate; in= inadequate; fa= fairly adequate; ad= adequate; va= very adequate

(gir,ir,fr,re,vr)* gir= grossly irrelevant; ir= irrelevant; fr= fairly relevant; vr= very relevant

(*yes)= if yes what date did it come into effect?



CHAPTER 12

CONCLUSION AND RECOMMENDATIONS

Although health policy and systems research HPSR as well as evidence-based policymaking EBP will

appear as new concepts to many policymakers in developing countries, experience in Nigeria has

shown that there was a high level of acceptance of the concepts by all actors in the policymaking

process. A similar trend can be anticipated in any developing country where HPSR and EBP will be

systematically introduced following the procedures outlined in this manual. Throughout the

developing world, particularly under democratic settings, government officials and health

policymakers are under intense pressure to make the health sector functional and deliver democracy

dividends to the public or risk losing popularity. Such pressures have left policy actors with no choice

but to be “open-minded” to any possible solution that can make the health sector more result oriented

and meet the expectation of the populace. Gonzalez-Block and Mills (2003) noted earlier that social

and political pressure towards health system equity and efficiency has stimulated increase in the

prominence of HPSR in low and middle income countries. This is why effective capacity building in

the health sectors of developing countries is essential (Potter and Brough, 2004). There is continuing

trend towards evidence-based policy formation and this has increased the demand for research outputs

that can provide clear, concise policy-relevant findings (WHO 2000b). Thus strengthening research

capacity in developing countries can enhance the generation and translation of knowledge into policy

decisions to improve health systems and increase equitable access to health services in the population

(Pang et al. 2003).

In this manual, as part of the interventional strategies towards improving the ability of the

policymakers and other stakeholders to to acquire, assess, adapt and apply research evidence

effectively, training workshops are recommended. Training workshops have many strategic benefits.

HIFA (2008) noted that workshops (when used as in-service training) are effective in doing what they

are supposed to do: presenting new information to groups of people, practicing new skills and

allowing health workers to share experiences and insights. Furthermore teaching delivered to groups is

often thought to be economical and another advantage is that it is thought that adults learn best by

sharing of experience, discussion and doing. Gates (2002) had earlier indicated that using training

workshop a significant impact on skill difference was gained on providers‟ skills in assessing and

managing HIV/AIDS patients. Methods used in the workshops such as the administration of pre-



workshop questionnaire and post-workshop questionnaire; focus group discussions; group works and

short presentations have been shown to be very effective in a previous study by Poulos et al.(2007).

Therefore the procedures described in this manual are capable of producing the following outcomes

1. The documentation of the actual situational analysis of HPSR evidence use in policy making (i.e.,

knowledge, attitudes and practices).

2. Increased awareness of the importance and value of HPSR evidence use in health policy making

and practice in a developing economy.

3. Identification of specific capacity constrains and challenges which impede the development of

HPSR evidence use in policymaking.

4. Identification of potential strategies and solutions that would address capacity constrains to HPSR

evidence use in policy making.

5. Identification of critical gaps in HPSR evidence use in policy making, with a focus on improving

public health.

6. Identification of the barriers to, and solutions for, translating research into policy and practice via

evidence use, and the formulation of general recommendations for HPSR collaboration and

coordination.

7. Improved staff skills in the acquisition, assessment, adaptation and application of research

evidence, e.g. the commissioning of research studies or the interpretation of systematic reviews.

8. Acquisition of skills in data processing and computer/information technology.

9. Creation of stronger incentives for evidence use in the health ministry (e.g. through integrating

this dimension into recruitment policies, performance assessments, and staff appraisals).

10. Development and integration of modules on use of evidence into leadership training courses for

target group.

11. Development of special training materials.



References

Acemoglu D (1997). Training and Innovation in an Imperfect Labor Market. Review of Economic

Studies 64:445-464.

Albert MA, Fretheim A, Maïga D (2007). Factors influencing the utilization of research findings by

health policy-makers in a developing country: the selection of Mali's essential medicines. Health

Research and Policy System 5: 2.

Alliance for Health Policy and Systems Research (AHPSR) (2004). Strengthening health systems:

the role and promise of policy and systems research. 130p. Available at:

http://www.who.int/alliance-hpsr/resources/Strengthening_complet.pdf

Alliance for Health Policy and Systems Research (AHPSR) (2007). Sound choices: enhancing

capacity for evidence-informed health policy. World Health Organization.

Bowen S, Zwi AB (2005). Pathways to „evidence-informed‟ policy and practice: a framework for

action. PLoS Medicine, 2(7): e166.

Bhagavan M. (1992). The SAREC model: institutional cooperation and the strengthening of national

research capacity in developing countries. Stockholm SAREC, 1992.

Canadian Health Services Research Foundation (CHRF) (2010). Self-assessment tool (Is research

working for you? A self-assessment tool and discussion guide for health services management

and policy organizations). Available online at:

http://www.chsrf.ca/other_documents/working_e.php.

Campbell DM, Redman S, Jorm L, Cooke M, Zwi AB, Rychetnik L (2009). Increasing the use of

evidence in health policy: practice and views of policy makers and researchers. Australia and New

Zealand Health Policy 6:21.

Dobrow MJ, Goel V, Upshur REG (2004). Evidence-based health policy: context and utilisation‟,

Social Science & Medicine, 58:207-217.

Gates JD (2002).Comparison of three types of training on providers skills in assessing and managing

HIV/AIDS patients. Int Conf AIDS. 2002 Jul 7-12; 14: abstract no. MoPeB3160.

Giorgi A (1985). Sketch of a psychological phenomenological method. In A. Giorgi, ed.

Phenomenology and psychological research: essays. Pittsburgh, Pa.: Duquesne University Press.

Global HIV/AIDS Initiatives Network (GHAIN) (2008). Guidelines for writing policy briefs

http://communication-resources.wikispaces.com/file/view/Guidelines+for+writing+policybriefs.pdf

Gonzalez Block, MA, Mills A (2003). Assessing capacity for health policy and systems research in

low and middle income countries. Health Research Policy and Systems 1:1.

http://www.who.int/alliance-hpsr/resources/Strengthening_complet.pdf

http://www.chsrf.ca/other_documents/working_e.php

http://communication-resources.wikispaces.com/file/view/Guidelines+for+writing+policybriefs.pdf



Gonzalez-Block MA. (2004). Health policy and systems research agendas in developing countries.

Health Research Policy System 5;2(1):6.

Green A, Bennett S (editors) (2007). Sound choices: enhancing capacity for evidence-

informed health policy. World Health Organization, Geneva. p172.

Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M (2003). The utilization of health research in

policy-making: concepts, examples and methods of assessment. Health Research and Policy System

1:2-29.

Healthcare for all (HIFA) (2008). CHILD2015 Summary: Are workshops effective? Available at: http://www.hifa2015.org/wp-content/uploads/2008/09/training_workshops_are_they_effective.pdf

Huddart J (2005). Assessment Of Ministry Of Health Human Resource Management Policies

And Practices Jordan Human Resources Development Project Report No. 5. Initiatives Inc.

Innvær S, Vist G, Trommald M, Oxman A (2002). Health policy-makers' perceptions of their use of

evidence: a systematic review. Journal of Health Services and Research Policy 7:239-44.

Johnson NA, Lavis JN (2009). Procedures Manual for the "Evaluating Knowledge-Translation

Platforms in Low- and Middle-Income Countries" Study. Hamilton, Canada: McMaster University

Program in Policy Decision-Making, 17 June 2009.

Jones N, Walsh C (2008) Policy briefs as a communication tool for development research

Background Note, Overseas Development Institute: London. Available at:

http://www.odi.org.uk/publications/background-notes/2008/policy-briefs-communication-tool-

development-research.pdf

Kitzinger J (1995). Qualitative Research: Introducing Focus Groups. British Medical Journal 311:

299–302. Retrieved June 23, 2010. <http://www.bmj.com/cgi/content/extract/311/7000/299>.

Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A (2009a). SUPPORT Tools for evidence-

informed health Policymaking (STP) 13: Preparing and using policy briefs to support evidence-

informed Policymaking. Health Research Policy and Systems, 7(Suppl 1):S13.

Lavis JN, Boyko JA, Oxman AD, Lewin S, Fretheim A (2009b). SUPPORT Tools for evidence-

informed health Policymaking (STP) 14: Organising and using policy dialogues to support evidence-

informed policymaking. Health Research Policy and System, 7(Suppl 1):S14.

Pang T, Sadana R, Hanney S, Bhutta ZA, Hyder AA, Simon J (2003). Knowledge for 1. better health –

a conceptual framework and foundation for health research systems. Bulletin of World Health

Organization 81:815-820.

Poulos RG, Zwi AB, Lord SR (2007). Towards enhancing national capacity for evidence informed

policy and practice in falls management: a role for a "Translation Task Group"?

Australian and New Zealand Health Policy 31; 4:6.

http://www.ncbi.nlm.nih.gov/pubmed/15296509





Potter C & Brough R (2004). Systemic capacity building: a hierarchy of needs. Health Policy

and Planning, 19:336–345.

Rabiee F. 2004. “Focus-group interview and data analysis”. Proceedings of the Nutrition Society 63:

655–660

Richardson CA, Rabiee F (2001). A Question of Access – an exploration of the factors influencing the

health of young males aged 15–19 living in Corby and their use of health care

Services. Health Education Journal 60: 3–6.

Sadana R, Lee-Martin S, Lee J, HRSA Network (2006). Health Research System Analysis

(HRSA) Initiative: Methods for Collecting Benchmarks and Systems Analysis Toolkit. Tool

#1. A brief overview of WHO Health Research System Analysis initiative and an overview of

core indicators and descriptive variables. WHO/EIP/HRSA/06.1. Geneva, World Health

Organization.

Sadana R, Pang T (2004). Current approaches to national health research systems analysis: a brief

overview of the WHO health research system analysis initiative . Ciência & Saúde Coletiva,

9(2):351-362, 2004

Thomas L, MacMillan J, McColl E, Hale C, Bond S (1995). Comparison of focus group and

individual interview methodology in examining patient satisfaction with nursing care”. Social Sciences

in Health 1: 206–219.

Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, Pielemeier NR, Mills A, Evans T (2004).

Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet

364(9437):900-6.

Uneke CJ, Ogbonna A, Ezeoha A, Oyibo PG, Onwe F, Ngwu BAF & Innovative Health

Research Group (2009). Health System Research and Policy Development in Nigeria: the

challenges and way forward.. The Internet Journal of World Health and Societal Politics 6:2.

Uneke CJ, Ngwu BAF, Ogbonna A, Ezeoha A, Oyibo PG, Onwe F (2010). Strategies to

enhance the capacity for evidence-informed health policymaking in Nigeria. The Internet Journal of

Healthcare Administration. 2010 Volume 7 Number 1

United Nations Development Programme (UNDP). (2006). Capacity development practice note. New

York, United Nations Development Programme. Available at: http://content.undp.org/go/cms-

service/download/asset/?asset_id=1654154

World Health Organization (WHO) (2000a). The World health report 2000: health systems: improving

performance. Geneva: World Health Organization.

World Health Organization (WHO) WHO (2000b): Promoting Evidence-based Reproductive Health.

Progress In Reproductive Health Research, No. 54, Part 1.

http://content.undp.org/go/cms-service/download/asset/?asset_id=1654154

http://content.undp.org/go/cms-service/download/asset/?asset_id=1654154



World Health Organization (WHO). 2003. Global Programme on Evidence for Health Policy.

Guidelines for WHO Guidelines. EIP/GPE/EQC/2003.1. Geneva: World Health Organization.

World Health Assembly (WHA). 2005. World Health Assembly concludes: adopts key resolutions

affecting global public health. Fifty-eighth Session. Available at: http://www.who.int/mediacentre/news/releases/2005/pr_wha06/en/print.html

World Health Organization (WHO) (2007) Everybody‟s business: strengthening health systems to

improve health outcomes. WHO‟s Framework for Action. Geneva, World Health Organization.

World Health Organization (WHO). (2008a). Report on meeting on health systems strengthening and

primary health care. Report Series No.: RS/2008/GE/35(PHL). Regional Office for the Western

Pacific Manila, Philippines: World Health Organization.

http://www.who.int/mediacentre/news/releases/2005/pr_wha06/en/print.html



Knowledge

Translation

Platform


Ebonyi State University PMB 053 Abakaliki Nigeria

Copyright 2010

Documents

The Nigeria Experience - WHO · The Nigeria Experience Knowledge Translation Health Policy & Systems Research Project Ebonyi State University Abakaliki Nigeria . Health Policy & Systems