Incorporating Quality of Care Investments into a Health System Pay-for-Performance Framework:

Process Evaluation of Zimbabwe’s Continuous Quality Improvement Innovation

Combined Qualitative and Quantitative Process Evaluation Report

FINAL

1

This report was prepared by Jed Friedman (Senior Economist/Principal Investigator), Ronald Mutasa (Senior Health Specialist/Team Leader), Tamar Gotsadze (Lead Qualitative Research Consultant), Anna Gage (Consultant), Felicia Takavarasha (Consultant) and Crecentia Gandidzanwa (Consultant). The study and report were conceptualized, conducted and analyzed with technical input from Son Nam Nguyen (Lead Health Specialist), Christine Lao Pena (Senior Economist/ Zimbabwe Health Sector Development Support Project Team Leader), Aditya Khaparde (Consultant), Mildred Pepukai (Consultant) and Chenjerai Sisimayi (Health Specialist). Leah Jones (Knowledge Management Consultant), Farai Sekeramayi (Program Assistant) and Yvette Atkins (Senior Program Assistant). The team benefited from the general guidance of Ernest Massiah (Practice Manager), Monique Vledder (Practice Manager for the Global Financing Facility), Magnus Lindelow (Practice Manager) and Mukami Kariuki (Country Manager, Zimbabwe). The technical leadership of the Ministry of Health and Child Care, Zimbabwe in conceptualizing and implementing the CQI innovation and in guiding the process evaluation is duly acknowledged. Cordaid’s role in conceptualizing the CQI initiative and supporting the roll-out, mentorship of MOHCC staff at various levels is greatly acknowledged. Cordaid’s contribution to the process evaluation is equally acknowledged.

The financial support and leadership of the Global Financing Facility and Ministry of Finance is duly acknowledged.

Executive Summary

Introduction

This report provides findings and recommendations based on the Process Monitoring Evaluation of the Continuous Quality Improvement (CQI) intervention being implemented under the World Bank financed Health Sector Development Support (HSDS) Project (P125229). This report includes both qualitative and quantitative components of the process evaluation to understand the extent to which the CQI initiative has improved quality of care structures and processes and facilities, and to identify the enablers and barriers to change.

The CQI initiative was piloted in 2016 as an arm of the Zimbabwe RBF program in order to improve quality of maternal and child health care services. The CQI intervention in Zimbabwe is designed to build capacity of facility health care teams, district managers, and supervisors to continuously improve care and thereby strengthen the capacity of the system to deliver high quality care. CQI regularly monitors quality outcomes to track progress against quality targets set by health facility teams at primary and secondary levels of care and their supervisors at district and provincial levels, a course of action is devised to improve these targets, and necessary actions are taken to achieve these targets.

Key components of the CQI intervention at facility and district management levels include formation of and regular support to CQI teams in participating facilities in order to: i) Define measurable improvement aims for priority best practices; ii) Discuss, review, and prioritize changes to routine care processes to improve adherence with incentivized best practices; iii) Collect and analyze monthly clinical quality measures used in the RBF project to assess progress against defined improvement aims; and iv) Ensure regular posting of results in relevant patient care areas for public information.

In addition, a component of capacity building and mentoring of provincial and district managers and supervisors was designed to: i) Support facility quality improvement teams to regularly carry out CQI activities; ii) Ensure regular competency-based and refresher training of skilled providers; iii) Oversee local supply chain (in close communication with facility managers) to prevent stock outs of essential commodities; iv) Audit medical records and observe simulated performance of priority clinical procedures to assess quality of care during supervision visits; and v) Track facility-specific results for priority clinical quality indicators (incentivized in the RBF program) to tailor support to facilities and clinical areas demonstrating poor performance on priority indicators.

MethodologyThis report addresses three primary research questions on the CQI intervention:

1. What is the effect of a quality improvement model on quality of care?2. What are the factors that influenced observed changes?3. What is the effect of a quality improvement model on how health facility teams

organize to improve quality of care?

2

Quantitative methods were used to explore the first question, while qualitative work addresses the second and third questions.

CQI is being piloted in five of the sixteen RBF districts, namely: Mashonaland Central Province-Centenary District, Manicaland Province-Chipinge District, Masvingo Province-Mwenezi District, Matabeleland North Province-Binga District and Matabeleland South Province-Mangwe District. In each of the five districts, one District Hospital and five health clinics were purposively selected for initial CQI implementation.

The qualitative process evaluation was implemented in 2 phases. In the first phase the evaluation team i) reviewed project related documents, ii) developed process evaluation methodology and tools, iii) sampled health facilities for site visits and collected qualitative information using various data collection methods, iv) constructed stakeholder map and produced inception report covering evaluation methodology, sampling of provinces, facilities and respondents, implementation phases, and schedule and draft outline of the final evaluation report. The second phase was subject to the availability of quantitative PME results collected separately from the qualitative. In this phase, the research team reviewed preliminary results of the quantitative process evaluation and jointly with the quantitative team elaborated exploratory research questions, respective data collection tools, and research protocols.

For the field data collection, the research team collected qualitative information and shared preliminary findings and recommendations with key stakeholders. Data collection included a mix of desk-based research, key stakeholder in-depth interviews (IDIs) and Focus Group Discussions (FGD). Based on the desk review findings, information gaps were identified, and related questions included in the IDI and FGD guides. IDIs were implemented face to face for key informants from central, provincial, district, and facility levels. IDI interview topic guides based on research questions, helped ensure systematic coverage of research questions and issues. FGDs were carried out for hospital CQI committees. FGD guides included questions specific to the FGD participant group and included 8-10 participants per each group. In health care centers with small number of staffed the FGDs were replaced by small group interviews. 232 in depth interviews were conducted at national, province, district and community levels in selected three provinces at both CQI and non-CQI facilities. Apart from the IDIs, researchers also carried out 15 FGDs and 22 group interviews.

Qualitative data analysis focused on synthesizing and triangulating information from the various data sources and evaluation methods. The evaluation took an iterative approach based on grounded theory that allows themes and findings to emerge from the data.

The quantitative evaluation compares quality-of-care outcomes in CQI intervention and control facilities. Data from the CQI quality checklist was used for the quantitative process evaluation. A baseline quality checklist was collected for all health facilities in the 18 study districts in the last quarter of 2016, and then the intervention began in the first quarter of 2017. The primary healthcare checklist contained data on 153 individual quality-of-care related outcomes. These were grouped into thirteen aggregate measures assessing the quality of inputs and seven aggregate measures assessing the quality of processes. The hospital checklist contained data on 253 indicators, which were grouped into 17 aggregate measures assessing the quality of inputs and nine aggregate measures assessing the quality of processes. All aggregate measures were calculated as the percentage of the underlying individual indicators (equally weighted) achieved by the facility. As the CQI intervention was intended to affect process quality more than inputs, the aggregate process quality indicators are presented as the main results.

For both the primary healthcare and hospital outcomes, an average of the four quarters in 2018 was used as the post-intervention timepoint for most analyses in order to (a) allow for the intervention, begun at the start of 2017, to take hold and achieve its intended aims, and (b) smooth variability, including possible seasonal variability, between quarters.

3

ANCOVA models were used to assess the effect of CQI on the quality outcomes, comparing CQI versus non-CQI facilities while controlling for the facility’s baseline quality. As this process evaluation is quasi-experimental in nature, two comparison groups are used for the primary healthcare analysis. The first comparison group is facilities that did not receive the CQI intervention within implementation districts, while the second comprises of facilities from the other thirteen RBF districts that did not implement CQI.

Qualitative results

The CQI pilot is perceived as one of the most important initiatives for improving the quality of MNCH services in Zimbabwe. There is a clear agreement among actors involved in CQI pilot about the significance of quality improvement as a fundamental tool to improve health outcomes. According to respondents, the CQI pilot was one of the first and few initiatives that ensured staff exposure to quality improvement, and which strategically and deliberately included a component of local problem analysis and planning of corrective measures.

The evaluation team noted several important factors facilitating and or impeding achievement of CQI program results. Enabling factors included:

1. GoZ political will to promote quality improvement in the health sector set an enabling environment.

2. The CQI intervention increased awareness of and participation in CQI activities and processes, and enabled CQI facilities to accelerate improvements in service provision and clinical quality.

3. External supportive supervision was viewed as important and useful in guiding the health facilities to focus on areas requiring attention and contributed to staff capacity building and knowledge sharing.

4. CQI fostered leadership, teamwork, and joint decision-making at intervention sites, and QI teams use selected CQI tools in their daily practice.

5. Staff pay for performance had marginal effect on staff motivation to improve care quality at both intervention and non-intervention sites. Rather, improved working environment and external supportive supervision had positive effect on staff motivation.

6. CQI encouraged a virtuous cycle in which improved performance encourages staff to further advance their performance.

7. Addition key components that supported staff knowledge, empowerment and motivation included peer review and peer support practices, knowledge exchange platforms and stronger linkages between facilities and their communities.

Impeding factors included both external and internal factors. External factors included:

1. Fragmentation of national and provincial level QA/QI results in inefficiencies and ineffectiveness of interventions. This fragmentation is mirrored at province, district, and health facility levels and CQI is treated as one among other government programs.

2. Attempts to integrate all QI interventions into one comprehensive QI plan failed at province, district, and district hospital levels.

3. National monetary policies introduced in 2019 inhibit effective implementation of CQI at intervention and non-intervention sites.

4. Delays in payment of RBF subsidies result in reduction of purchasing power and shortages in the procurement of goods for quality improvement.

5. Human resource for health planning along with a government freeze of employment resulted in staff shortages and adversely affected quality of care.

4

Internal factors included:

6. The effectiveness of CQI trainings was challenged by the training content, intensity, and mode of delivery. There was a lack of balance between theoretical knowledge transfer and practical skill development during the training course. Furthermore, due to high turnover staff trained in CQI moved to other facilities and/or locations, and their replacements lacked the knowledge of CQI. The lack of clinical training opportunities to complement the CQI training impedes MNCH quality improvement.

7. The effectiveness of supportive supervision and mentoring/coaching from provincial and district levels was confronted by a number of challenges, including staff shortages at provincial and district levels and limited time spent at the health facilities.

Quantitative results

At baseline, most facilities performed well on the input measures but had poorer performance on the process measures. This is likely due, at least in part, to the participation of all study districts in the RBF program. For example, across all facilities, facilities had on average 79% of the general equipment items such as radio, mosquito nets and adult weighing scales. Primary health centres by far performed the worst on obstetric complications, with facilities only completing 3% of the items on average. The majority of summary outcomes were balanced between CQI and both groups of control facilities at baseline. Where the indicators differ, CQI facilities generally performed better than the control facilities in other districts.

CQI was associated with improvement in the primary health centres for two of the seven process measures: postnatal care (PNC) and maternal care. The intervention was associated with completing one half of an additional PNC process item correctly and one additional maternal process item. The PNC improvement is driven primarily by better assessment in the CQI facilities: facilities were 25 percentage points more likely to have all women checked for their general condition, pulse rate and temperature in comparison to the control facilities. Among the maternal care process indicators, monitoring for women and newborns in the 4th stage of labor, administering immediate postpartum oxytocin, and monitoring women using partographs were the indicators that improved the most between CQI and control facilities.

When comparing the primary care intervention facilities against control facilities in their districts maternal care quality improvement is no longer significant at standard levels, although the coefficients are positive and of similar magnitude. Post-natal care quality still shows significant improvement, though the magnitude of the improvement is smaller than what was observed using the other districts as comparison. Given the robustness checks conducted, it suggests that the CQI intervention did improve post-natal care quality in the intervention facilities, and likely maternal care-quality as well. However, it did not improve the other process quality measures including ANC, PMTCT, sick child assessment, sick child treatment, or obstetric complications quality. In addition, as expected given the focus of the CQI activities, no improvement in input quality measures was observed.

5

Trend in PNC process quality over study period in PHC CQI and control facilities in 13 other districts

Five hospitals implemented CQI, and data was available for at least some indicators from sixteen other hospitals. Hospitals overall performed better on the quality of inputs measures than the quality of processes at baseline. Process quality was generally lower than input quality in 2016, although facilities still scored highly on both maternal care processes and paedatric care processes. Hospital performance was generally higher than primary health centre performance at baseline, although the measures are not directly comparable due to different indicators included in the summary measures. There were few differences in outcomes between CQI hospitals and control hospitals in 2016 but where there were, CQI hospitals had better performance.

The study is not powered to find significant associations among the limited number of hospitals, and so unsurprisingly there are no significant associations. Even given the limited sample size, however, many outcomes have CQI coefficients close to zero, suggesting that there may not be significant effects even with larger sample sizes. The largest positive coefficient is for sick child treatment quality. Despite the lack of associations in the summary outcomes, CQI was significantly associated with a few of the individual input quality outcomes. For example, CQI was associated with a 31 percentage point increased likelihood of having an appropriate waste pit, 19 percentage point increased likelihood of having second line paediatric ART medications, and 47 percentage points more likely to have a functioning ultrasound machine. However, CQI was also associated with a 36 percentage point decreased likelihood of correct treatment for sepsis, and had no significant associations for the vast majority of indicators.

Conclusions

There is some evidence that CQI improved maternal care and postnatal care processes in PHCs, but no evidence of improvement in any inputs, as expected, or any of the hospital processes. Improvements were limited when considering the full breadth of potential outcomes but arise in certain areas of focus of the CQI program.

This absence of broad-based improvement may be due to several factors. First, CQI facilities were already performing better than control facilities at baseline on many indicators. Second, many structural indicators were already very high at baseline. In combination, these suggests that there

6

may have been ceiling effects where CQI facilities were unable to improve anymore but control facilities were. Third, the qualitative results suggest that there was mixed success among different CQI facilities as the intervention interacted differently with the environment, the service, the quality improvement process and the stakeholders. The overall grouping of CQI and control facilities may have masked heterogeneity of improvement, with some facilities performing better.

The mixed success among different CQI facilities is associated with the interaction of features of the environment, the service, the quality improvement process, and the stakeholders, which operated together to produce a set of circumstances that either inhibited or enabled the process of change. Performance improvements would have been impossible without strong leadership of management teams who drive the process of quality improvement and establishment of teamwork culture. Availability of skilled health workforce—and building their capacity with training for continuous knowledge and skills development along with supportive supervision—are the most influential factors behind care quality improvements.

General health system related bottlenecks, including high staff turn-overs and staffing shortages, challenged effectiveness of the CQI pilot. Furthermore, a sub-optimal medical and nursing education system to produce professional health workforce and limited continuous medical education opportunities. Specific challenges to the implementation of CQI included fragmentation of QA/QI functions, gaps in building health worker knowledge and skills, suboptimal supervision and delays in RBF payments.

The qualitative results indicate that the CQI facilities saw greater leadership, teamwork, ownership of clinical quality improvement, engagement, and staff motivation. These could be important outcomes in and of themselves but may not necessarily translate into higher quality care. This reveals a major limitation of the CQI approach: some items are not within the power of the facility staff to solve through continuous quality improvement. For example, the CQI methodology cannot address the challenges with staff shortages, high staff turnover and high workload. These problems may greatly affect care processes but need to be addressed at the national or district levels. The quantitative results also show the extremely low performance of the PHCs on managing obstetric and newborn complications. Improved staff motivation and teamwork will not overcome challenges of lack of training or practice on these skills, rather it suggests that to improve quality, only hospitals should handle these complications.

These findings align with a broader literature indicating that micro strategies to improve quality at the level of providers or facilities will likely be insufficient by themselves. Rather, macro and meso level strategies are likely needed at the national, provincial and district levels in order to improve overall health system quality. The large gaps observed in the quality of care cannot be addressed through targeted quality management when the root causes may be high workload, large turnover, inadequate clinical training, or simply providing a service at an inappropriate level of the system. Macro or meso level strategies to address these challenges would include addressing gaps in governance, reorganizing services to be provided at the appropriate level (i.e. moving deliveries to hospitals that can manage obstetric complications), strengthening pre-service clinical education, or establishing facility learning networks. While CQI may contribute to a better working environment and spur improvement in routine care practices that the staff are already well trained in, it should be seen as one tool in a broader health systems quality improvement strategy.

Table of Contents

7

1. Introduction..............................................................................................................11

2. Country Context........................................................................................................11

2.1 Zimbabwe’s Health Sector...................................................................................................112.1.2 Quality of care challenges..................................................................................................................12

2.2 Health delivery system........................................................................................................13

2.3 Quality assurance and improvement in Zimbabwe..............................................................13

3. Continuous Quality Improvement..............................................................................14

3.1 Summary of Evidence on PBF and CQI.................................................................................16

3.2 CQI in Zimbabwe.................................................................................................................17

3.3 A theory of change for quality improvements at the primary clinic level..............................18

3.4 Intervention questions and objectives.................................................................................19

4. Methodology............................................................................................................19

4.1 CQI Implementation............................................................................................................19

4.2 Selection of CQI implementing health facilities....................................................................20

4.3 Qualitative process evaluation............................................................................................224.3.1 Qualitative data collection.................................................................................................................234.3.2 Qualitative sampling..........................................................................................................................234.3.2 Qualitative data analysis and triangulation........................................................................................244.3.3 Qualitative research ethics................................................................................................................244.3.4 Qualitative study limitations..............................................................................................................25

4.4 Quantitative process evaluation..........................................................................................254.4.1 Quantitative data collection and extraction.......................................................................................254.4.2 Quantitative outcomes adopted in the analysis.................................................................................254.4.3 Quantitative data analysis..................................................................................................................26

5. Qualitative Results.................................................................................................28

5.1 Quality improvement results...............................................................................................28

5.2 Factors influencing observed changes.................................................................................305.2.1 Enabling factors.................................................................................................................................305.2.1 Impeding factors................................................................................................................................35

6. Quantitative results......................................................................................................41

6.1 Primary healthcare facility results.......................................................................................41

6.2 Hospital results...................................................................................................................46

7. Conclusions and lessons learned................................................................................51

7.1 Drivers of change................................................................................................................52

7.2 Barriers to change...............................................................................................................53

7.3 Role of CQI interventions....................................................................................................56

7.4 Lessons Learned..................................................................................................................57

8

8. Recommended Approach to CQI Scale-up and Institutionalization.............................58

8.1 Essential elements for vertical CQI scale-up and institutionalization....................................58

8.2 Recommendations for horizontal CQI scale-up....................................................................61

Annex 1: RBF Package of quantity services.......................................................................63

Annex 2: Exploratory Research Questions........................................................................64

Annex 3: Additional tables & figures................................................................................66

Annex 4. Continuous Quality Intervention checklists for PHCs and Hospitals...................116

9

1. IntroductionIn July 2011, the Government of Zimbabwe (GOZ) piloted a health sector results-based financing

(RBF) program with grant support from the Health Results Innovation Trust Fund1 and with Ministry of Finance and Economic Development (MOFED) co-funding. The RBF program supports the Ministry of Health and Child Care (MOHCC) to increase the availability, accessibility, and utilization of quality health care to improve maternal, neonatal, and child health. The program piloted in two districts in July 2011 and expanded to 415 health facilities across 16 additional districts in March 2012. In 2014, the MOHCC scaled up RBF to cover 44 rural districts and the two urban districts of Harare and Bulawayo with Health Transition Fund2 support. The program has three components: (i) results-based contracting; (ii) management and capacity building; and (iii) monitoring. Under the first component, a portion of financing received by health facilities depends on the quantity and quality of services, with a focus on maternal and child health (MCH). Annex 1 summarizes the package of services linked to RBF incentives. In addition to introducing incentives, the program abolished user fees on a package of services in each district, with the aim of improving access to care.

Improving the quality of care helps to avert adverse health outcomes among targeted populations and may attract more households and patients to seek health care, thereby increasing utilization of services3. Empirical evidence from the first phase of the Zimbabwe RBF impact evaluation in 2016 showed RBF’s positive impact, but also its mixed effects on quality of care, signaling policy makers and development partners to explore innovations to improve quality of care in line with Zimbabwe’s National Quality Strategy 2016 vision.

In 2016, following results of the first phase of the RBF program impact evaluation, the MOHCC requested World Bank support to design and pilot a continuous quality improvement (CQI) innovation to be rolled out in select facilities implementing the RBF program. The CQI innovation was introduced with the goal of improving overall quality of care outcomes—in particular clinical process measures—that had not been very responsive to the RBF intervention.

As part of the Process Evaluation for Zimbabwe’s RBF program, the World Bank commissioned an assessment of the CQI innovation. This assessment was undertaken with technical input and guidance from the MOHCC and Cordaid. This overview report—which incorporates both qualitative and quantitative evidence on the impact of CQI on health care delivery—was developed at the request of MOHCC senior management. The purpose of the report is to inform future directions of the CQI intervention by documenting implementation factors that might contribute to its success or challenges and by offering a synthesis of the quantitative evidence on the effect of the CQI intervention.

2. Country Context2.1 Zimbabwe’s Health Sector

Zimbabwe’s population is estimated at 13,061,2314, with a life expectancy at birth (LEB) of 61 years in 2017. The total fertility rate is 4.1 children per woman. According to the WHO 2015 country burden of disease profile5, at least three quarters of Zimbabwe’s annual deaths can be attributed to communicable, maternal, perinatal, and nutritional illness. A major area of concern for

1 This is a multidonor trust fund administered by the World Bank and funded by the Governments of Norway and United Kingdom to pilot innovative approaches to accelerating progress in maternal, neonatal and child health outcomes.2 This is a multidonor trust fund administered by UNICEF established to support the recovery of the health sector in Zimbabwe.3 Shroff CZ, Bigdeli M, Meissen B. From scheme to system (part 2): Findings from ten countries on the policy evolution of results-based financing in health systems. Health Systems & Reform. 2017;3:137-474 Zimbabwe Population Census 2012, National Report, ZimStat5 WHO Country Profile: http://www.who.int/nmh/countries/zwe_en.pdf?ua=1

10

Zimbabwe has been the high maternal mortality ratio (MMR), which reached 960 deaths per 100,000 live births in 2010-2011 but dropped to 462 deaths per live births by 2019 6. However, the pace of progress is not fast enough to achieve Sustainable Development Goal (SDG) targets. In addition, despite some progress since 2009, infant and under-5 mortality rates were estimated at 52 per 1,0001 live births and 72 per 1,0001 live births, respectively. One-third of children under five are stunted, and there has been little improvement in these figures over the last decade. In addition, the country has in the last decade been increasingly challenged by the dual burden of communicable and non-communicable diseases.

A major financial burden of health care falls on households in the form of out-of-pocket payments—rendering the health system inequitable and inefficient. Access to health services and subsequent outcomes have largely been inequitable with poor and rural populations shouldering a disproportionate burden of disease and health risks.

Cycles of fragility and macroeconomic challenges, compounded by climatic shocks in the last few years have led to cash shortages, affecting the prices of goods and services and, in turn, health providers’ ability to execute planned activities. The worsening economic conditions have also strained the health workforce, a key pillar of the health system. Salaries of the health workforce have been eroded due to inflation, resulting in a demotivated health workforce leading to poor health service delivery and uncertainties in retention of staff. In the last half of 2019, the country experienced the most protracted industrial action by medical doctors it has ever faced, with over 500 junior doctors absconding from duty for over three months citing incapacitation.

Current patterns in the utilisation of health services and quality of care reflect a heightened risk of reversal towards poor health outcomes reminiscent of the pre-2010 era. Although results from the MICS 2019 indicate a continued positive trajectory for the maternal mortality ratio, an increase in the neonatal mortality rate signifies continued gaps in utilization in quality. The underlying patterns of health service utilisation and quality of care in the reference period for the study (2015-2018) are different from those pertaining to 2019 and likely going forward.

The response to the current challenges is limited as both domestic and external funding remain constrained. Whilst the share of Government public health financing has been significant to total health spending, the contributions have been below regional and international thresholds. The sector has continued to experience a financing gap relative to the costed needs, and current forecasts of domestic and external funding streams as reflected in resource mapping data are indicative of limited prospects to address this gap in the short term. In the absence of a coordinated response to address some key priority high impact interventions, the system will fail to activate the essential structural pillars necessary to sustain the gains witnessed to date, let alone withhold the downward spiral in the availability and quality of the critical health services. The sector can leverage on a number of low hanging fruits amongst key interventions. However, the current macro-economic environment and extent of the deterioration in health service delivery suggest modest improvements, at the very best a rebounding to levels to sustain the gains witnessed in the last decade, should these interventions be implemented.

2.1.2 Quality of care challengesMajor gaps in quality of health care services in Zimbabwe are substantiated by several

household and facility surveys (DHS 2005, 2010/11, 2016, USAID MCHIP study 2012). A 2016 study of the quality of MCH services demonstrated important quality gaps for both routine maternal new-born services and for the leading causes of maternal, newborn, and child mortality (i.e. post-partum haemorrhage [PPH], eclampsia, maternal and newborn sepsis, newborn asphyxia, newborn

6 2019 Multiple Indicator Cluster Survey, Zimbabwe National Statistics Agency

11

prematurity, child pneumonia, diarrhoea, and acute malnutrition)7. In addition to being overstretched in understaffed facilities, providers often do not have the skills or the confidence to manage common life-threatening complications. District and regional administrative supervisors often lack up-to-date clinical knowledge and skills to assess and support provider competence in priority clinical areas. There is also no system of clinical certification maintenance.

In many clinics and hospitals, medical records and registers do not document core elements of basic quality care, such as vital signs and physical examination findings. For example, outpatient pediatric and adult registers often include only the patient name, main complaint, diagnosis, and medication prescribed without any results of vital signs, physical exam, or duration and severity of symptoms, etc. Indeed, providers face the burden of completing as many as ten to 15 different clinical registers to meet reporting requirements, yet there is very little integration of essential data. As in many countries, Zimbabwe’s health management information system (HMIS) primarily records coverage information with regard to health care services, with infrequent inclusion of clinical quality measures even for high-impact clinical interventions, such as immediate post-partum oxytocin for prevention of PPH. There is limited MOHCC capacity to support quality improvement (QI) activities at any level of the health system and rare use of QI to influence care in clinics and hospitals.

2.2 Health delivery systemZimbabwe’s public health delivery system, which takes a primary health care centered

approach, is the responsibility of the MOHCC, local governments, and not-for-profit faith-based organizations. Zimbabwe’s current health system delivery has four levels.

(i) Primary level: A total of 1,634 primary care facilities provide basic health promotion, prevention, and curative and rehabilitation services to their catchment populations. These facilities are also responsible for environmental sanitation, water supplies, hygiene, waste disposal systems, control of communicable diseases, nutrition, mental health, and disabilities. They are also the point of reference for village health workers (VHWs). Primary health facilities normally have at least two nurses, one of whom should be a skilled midwife. The staff complement should also include an environmental health technician (EHT). Normally, primary health facilities refer complicated cases to district hospitals.

(ii) Secondary level: This level consists of district hospitals, which take referrals from primary care facilities. These hospitals offer medical interventions that include Caesarean sections and blood transfusions.

(iii) Tertiary level: Zimbabwe has eight provincial hospitals, which comprise the tertiary level of care. One provincial hospital is run and owned by the church, while the GOZ owns and runs the other seven. Provincial hospitals provide health services to patients referred from district hospitals.

(iv) Quaternary level: The quaternary level offers highly specialized hospital services; these hospitals are only found in Harare and Bulawayo. Normally referred to as central hospitals, these provide services to patients with complications as referred from the provincial hospitals.

While in rural areas mission facilities play a considerable service, in urban areas private health providers are major providers of health services. Nationally, mission hospitals and private clinics account for 35% of health care delivery, while about 65% of health care services are provided by the public sector.

7 Hill, K, Quality of Maternal, New Born and Child Health in Zimbabwe (World Bank Report, 2016).

12

2.3 Quality assurance and improvement in ZimbabweEquity and quality are fundamental MOHCC healthcare principles. In 2010, the MOHCC

established a Quality Assurance and Quality Improvement (QA/QI) Directorate under the division of Policy, Planning, and Monitoring and Evaluation. The department was designed to oversee the MOHCC’s healthcare quality assurance and improvement efforts, including creation and/or adherence to quality policies, guidelines, and standards.

Health sector quality improvement in Zimbabwe is regulated by a QA and QI Policy 8

approved by the MOHCC in 2015. The strategy takes a systems approach to improving and sustaining high-quality health services in Zimbabwe, highlighting cross-cutting priority intervention areas related to the World Health Organization’s (WHO) health system building blocks and priority quality domains (e.g. safety, clinical effectiveness), and priority interventions in vertical disease control programs targeting major causes of morbidity and mortality. The policy guides the process of ensuring quality of care as well as CQI in both public and private health sectors; it also provides guidance for capacity building and leadership for QA/QI processes at all levels of the health system through adoption of proven change management methodologies.

Mechanisms for implementing the QA/QI policy include: creation of a National Advisory Committee and technical working groups at MOHCC; creation and support of provincial and district QI focal points; regular monitoring of priority quality measures tailored to specific health system stakeholders; as well as provisions for monitoring of the policy implementation.

At national level, a National QI Committee–with representation from senior management within the MOHCC and other major stakeholders—oversees and guides institutionalization of QA/QI processes to support health care service delivery. The National QI Committee drives quality improvement and ensures that QI becomes a nationwide continuous system-wide approach.

The National QI Committee has five main areas of responsibility:

1. Strategic Planning: Prioritizing goals so the most critical areas are addressed first.

2. Overseeing QI implementation: Ensuring that all quality improvement activities are performed effectively in line with key quality priorities.

3. Guidance: Overseeing activities to ensure that they are on track and responsive to staff, clients, and partners during the improvement process. This guidance includes support and encouragement of the provincial level to maintain QI momentum.

4. Resource mobilization: The national QI committee is responsible for building sustainable infrastructure that fosters a culture of quality service delivery.

5. Enforcing accountability: The committee tracks and reports on implementation progress, using a set of agreed indicators and adjusting implementation course when necessary.

Implementation of the QI strategy utilizes existing structures and systems, as much as possible, enabling QI to take off with minimal additional resources, and encouraging QI activities to be embedded and integrated within the normal MOHCC operational environment.

3. Continuous Quality Improvement In 2014, the MOHCC approved a National QI Strategy, which provided a basis and rationale

for introducing the Continuous Quality Improvement (CQI) innovation embedded within the performance-based financing scheme (Box 1). This approach enabled the GOZ to further strengthen improvements in quality of care, building on the widely successful effects of the RBF mechanism to strengthen coverage and structural quality of services provided in the country. The CQI component of the RBF project was designed to build the capacity of facility health care teams, district managers

8 Quality Assurance and Quality Improvement Policy, 2015, Ministry of Health and Child Care Zimbabwe

13

and supervisors, to continuously improve care and thereby strengthen the capacity of the system to deliver high quality care. The CQI component of the National QI Strategy defined a set of priority conditions for which it seeks to improve the quality of care. Conditions were prioritized based on their contribution to the burden of mortality and morbidity, as well as on evidence of effective health care interventions (preventive and curative). Quality of care measures were integrated into established checklists for various aspects of maternal, newborn, and child care.

Following MOHCC approval of the National QI Strategy, the CQI initiative was piloted in 2016 as an arm of the Zimbabwe RBF program. Continuous quality improvement is “a management philosophy that organizations use to reduce waste, increase efficiency, and increase internal (meaning employee) and external (meaning customer) satisfaction. It is an ongoing process that evaluates how an organization works and ways to improve its processes.”9 CQI regularly monitors quality outcomes to track progress against quality targets set by health facility teams at primary and secondary levels of care and their supervisors at district and provincial levels, a course of action is devised to improve these targets, and necessary actions are taken to achieve these targets.

Box 1: The Continuous Quality Improvement Model

The premise of the CQI model is that a change or series of changes is needed to improve quality of health care services. Managing the change process is therefore central and involves four steps of the Plan-Do-Study-Act (PDSA) cycle. In each cycle, a CQI team collaborates on a series of steps:

Plan: The team plans a change to improve the quality of services it offers

Do: The team executes the plan and documents what is working and any unexpected developments

Study: The team analyzes the results in relation to original objectives

Act: The team decides on next steps. If the actions were successful, the team may introduce it at a larger scale; if they are not successful, the team may decide to discard the model or adapt changes to make it work more successfully.

The summative effect of the CQI team testing changes, identifying effective ones, and adopting solutions should help improve quality and strengthen the health system overall. CQI teams are made up of supervisors, front-line health care workers, and staff with the knowledge of their local systems necessary to be able to identify and test feasible and sustainable changes to “usual processes” to improve care in their local setting. CQI teams regularly monitor performance with adherence to clinical best practices in the areas of improvement targeted by the project and track the improvement of quality outcomes against a quality target.

This model has two types of CQI clinical audits: (i) audit of maternal-neonatal deaths and near-miss 10 events

9 https://study.com/academy/lesson/what-is-continuous-quality-improvement-definition-process-methodologies.html10 Near miss (World Health Organization): Women who survive life-threatening conditions arising from complications related to pregnancy and childbirth have many common aspects with those who die of such complications. This similarity led to the development of the near-miss concept in maternal health. Exploring the similarities, the differences and the relationship between women who died and those who survived life-threatening conditions provides a more complete assessment of quality of maternal health care.

14

allowing teams to reflect on, understand, and learn from rare, catastrophic (or near-catastrophic) events through peer review of cases that caused concern, affected patient safety, or resulted in an unfortunate outcome; and (ii) monthly clinical audits, which are systematic reviews of patient charts to determine care given in relation to the standard of care. These two types are carried out internally by site for monthly monitoring and externally for data validation.

3.1 Summary of Evidence on PBF and CQIWhile performance-based financing schemes are primarily aimed at improving demand and

access to health care, PBF schemes implicitly or explicitly address quality of care, especially through their payment formulas11. Studies on PBF schemes in low- and middle-income countries (LMICs) show that quality of care is often included directly or indirectly in program implementation or design12. Quality improvement mechanisms also help prevent health facilities from comprising health care quality as they aim to meet their quantity targets. At the same time, targets set in the SDGs—specifically SDG 3, which measures health using outcome quality indicators—drove the need for health financing programs to prioritize quality of care in health delivery systems13.

Among LMICs, many PBF programs have begun integrating quality into their programs, but not yet in a uniform or systematic manner14 15. While PBF has been linked to mixed results in workers’ morale and demand of health services, there is a general lack of conclusive data to measure the improvement of quality within PBF programs. Evidence on the impact of PBF on quality is scarce as there is no uniformity of documentation of quality in PBF programs. Standardization of data collection around PBF would help researchers better interpret impact estimates and provide practical guidance to policy makers. This dearth of knowledge and evidence on quality improvement, coupled with limited documented success of the PBF in improving quality of care in LMICs, motivated the CQI pilot in Zimbabwe.

One study in Ghana suggests that CQI improved health outcomes. CQI helped to diagnose problems, recommended solutions, and charter progress on a color-coded scoring system. Staff were provided regular feedback on performance targets and achievements. This initiative showed a 34% decrease in maternal mortality despite increased patient admissions to the study hospital. There was a reduction in case fatality rates for pre-eclampsia and hemorrhage from 3.1% to 1.1% (p<0.05) and from 14.8% to 1.9% (p<0.001), respectively. Stillbirths declined by 36% (p<0.05) 16. Over a two-year period, the MMR decreased from 496 per 100,000 live births to 328 per 100,000. In South Africa, CQI—specifically of HIV/AIDS services—increased antenatal HIV testing by almost 10 percentage points, i.e., from 89% to 98%. CD4 testing of HIV positive mothers increased from 40% to 97%, maternal nevirapine from 57% to 96%, and infant nevirapine from 15% to 68%17. In Kenya, CQI improved

11 Fritsche GB, Sr, Meessen B. Performance-Based Financing Toolkit. Washington, DC: World Bank; 2014. https://openknowledge.worldbank.org/handle/10986/17194 Accessed 23 December 2019.12 Quality of Care in Performance-Based Financing: How It Is Incorporated in 32 Programs Across 28 Countries. Jessica Gergen, Erik Josephson, Martha Coe, Samantha Ski, Supriya Madhavan, Sebastian Bauhoff Glob Health Sci Pract. 2017 Mar 24; 5(1): 90–107. Published online 2017 Mar 1513 Sustainable Development Goals. Goal 3: Ensure healthy lives and promote well-being for all at all ages. United Nations; http://www.un.org/sustainabledevelopment/health/ Accessed December 23, 2019.14 Witter S, Fretheim A, Kessy FL, Lindahl AK. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev. 2012;(2):CD007899. 10.1002/14651858.CD007899.pub2.15 Meessen B, van Heteren G, Soeters R, Fritsche G, van Damme W. Time for innovative dialogue on health systems research. Bull World Health Organ. 2012; 90(10):715–715A. 10.2471/BLT.12.112326.16 E.K. Srofenyoh ,N.J. Kassebaum, D.M. Goodman, A.J. Olufolabi, M.D. Owen. Measuring the impact of a quality improvement collaboration to decrease maternal mortality in a Ghanaian regional hospital. International Journal of Gynecology and Obstetrics 134 (2016) 181–18517 Bhardwaj S, Barron P, Pillay Y, et al. Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up using quality improvement. S Afr Med J. 2014;104(3 Suppl 1):239–43.

15

adolescents and young adults’ knowledge on HIV prevention and transmission knowledge, and a trend towards improved intent to retest at one clinic18.

3.2 CQI in ZimbabweFor all facilities participating in the Zimbabwe RBF program, quality supervision checklists

were revised in 2016 to include clinical quality of care indicators related to delivery of proven best practices for improving MCH in Zimbabwe. These revised quality checklists were linked with incentives in all RBF facilities, including both treatment and control group facilities in the process evaluation, and hence constitute part of the control against which the CQI interventions are assessed.

The CQI intervention study, implemented in addition to the quality of care changes above, included the following steps: (a) introducing CQI processes to MOHCC-selected facilities and districts; and (b) further incentivizing selected CQI process measures to improve use of standardized checklists, charts, and forms by health workers to document the health worker/patient interface. Through implementation of high-impact quality improvement interventions, the CQI initiative aims to decrease morbidity and mortality of: pregnant women during labor and delivery and postpartum; neonates during the early neonatal period; and under-5s from common conditions. CQI was also designed to build capacity of facility health care teams, district managers, and supervisors. By continuously improving care CQI intends to strengthen the capacity of the health system to deliver quality care.

Figure 3.1: Design of the RBF Project with CQI component

Key components (Figure 3.1) of the CQI intervention at facility and district management level include the formation of, and regular support to, CQI teams in participating facilities in order to:

Measure improvement for priority best practices (e.g. essential newborn care, prevention/management newborn sepsis and newborn asphyxia, PMTCT priority services)

Discuss, review, and prioritize changes to routine care processes to improve adherence to incentivized best practices, i.e., quality measures (e.g. division of essential tasks among staff;

18 Wagner AD, Mugo C, Bluemer-Miroite S, et al. Continuous quality improvement intervention for adolescent and young adult HIV testing services in Kenya improves HIV knowledge. AIDS. 2017;31 Suppl 3(Suppl 3):S243–S252. doi:10.1097/QAD.0000000000001531

16

re-organization of services to be more efficient; posting of job aids; regular clinical update sessions; and modification of patient records/registers to capture essential information)

Routinely collect and analyze clinical quality measures used in the RBF program to assess progress against defined improvement aims

Regularly post results in relevant patient care areas for public information Conduct a structured audit of every maternal, newborn, or child death and near miss

In addition, a component of capacity building and mentoring by district/municipal managers and supervisors was designed in order to:

Support facility CQI teams to regularly carry out CQI activities (see above) Ensure regular competency-based training and refresher training of skilled providers Oversee local supply chain (in close communication with facility managers) to prevent stock-

outs of essential commodities (e.g., Gentamycin, functional neonatal bag and mask, rapid HIV diagnostic tests)

Audit medical records and observe simulated performance of priority clinical procedures (e.g., newborn resuscitation using structured observation checklist) to assess quality of care during supervision visits

Track facility-specific results for priority clinical quality indicators (incentivized in RBF program) to tailor support to facilities and clinical areas demonstrating poor performance on priority indicators

3.3 A theory of change for quality improvements at the primary clinic level

The theory of change for the CQI intervention is based on the premise that the continuous quality improvement model will lead to enhanced quality of care and client satisfaction (Figure 3.1). At facility level, the intervention was supported by a multi-departmental QI team, which included facility managers and health providers. Supervisors at the district level built QI team capacity in clinical and management competences through training, review of records, and direct observations of the priority clinical quality indicators. The clinical competences consisted of adherence to the national standards of care for select MCH services, e.g., review of timely completion of partograms as stipulated by the MOHCC and international health guidelines, HIV-infected infants initiated on treatment before age two, newborns breastfed within one hour of birth, and child outpatient diarrhea cases correctly treated. Through mentoring and coaching by supervisors along with facilitation of the supply chain, it was assumed that the capacity and motivation of the health workers would be enhanced, which in turn would enable regular implementation of the CQI cycle (i.e., identify services for quality improvement, plan, analyze, and review, implement, and share the results). Regular CQI implementation would result in adherence to standards and hence improved clinical indicators. Finally, better clinical indicators lead to improved quality of care and client satisfaction. Improved clinical indicators also lead to higher incentive payments through the RBF mechanism that will further motivate the health teams.

17

Figure 3.2: Theory of change for the quality improvement intervention

Source: World Bank 2017

3.4 Evaluation questions and objectivesWhile there is an emergent body of evidence of the positive impact of CQI (and related

quality improvement models) on quality of care, little is known about the effectiveness of incentivizing the implementation of CQI per se. There is limited evidence on the effect of combining RBF interventions with CQI in low-middle income country (LMIC) settings. Therefore, the overarching goal of the qualitative CQI PME study is to help the MOHCC, the Ministry of Finance and Economic Development (MOFED), Cordaid, the World Bank (WB) Task Team and interested local and international stakeholders learn from the CQI implementation process; make course adjustment to the technical design and operational processes; and enhance evidence-based project management decision making by exploring opportunities for the CQI component scale-up.

The primary research questions guiding the process evaluation of the CQI intervention were as follows:

1. What is the effect of a quality improvement model on quality of care?2. What are the factors that influenced observed changes?3. What is the effect of a quality improvement model on how health facility teams

organize to improve quality of care?

Quantitative analyses explore the first question, while qualitative work addresses the second and third questions.

4. Methodology4.1 CQI Implementation

MOHCC QA/QI team staffing was strengthened through Cordaid seconding seasoned international technical staff to work in the MOHCC national QA/QI directorate to support national, provincial, and district level implementation of quality aspects of the RBF program, with added emphasis on the CQI innovation. An MOHCC technical team developed the CQI guidelines with the support of international consultants from Health Quality International. The “how to” guidelines supported the standardization of CQI activities conducted by health facility teams.

18

A selected team of national and provincial focal persons was trained in the initial phase by a team of international and local CQI experts from Health Quality International and from the University of Zimbabwe. The focal persons were trained in QI methods to serve as trainers-of-trainers (TOT) and further cascade the capacity building process to the districts and facilities. A small technical team from the MOHCC QA/QI Directorate and provincial/district focal points was responsible for the roll-out of the CQI innovation to facilities in selected districts during this initial phase. This process was designed to ensure fidelity of CQI implementation across intervention sites.

The CQI project collected data on MCH service-related indicators from the quality supervision checklists; quality assessments are conducted by Provincial Health Executives (PHEs) and District Health Executives (DHEs). PHEs and DHEs were provided with necessary tools and upskilled in CQI through a training-of-trainers model.

PHE and DHE teams were given a four-day basic training in CQI. The training focused on performance measurement and quality improvement techniques to strengthen: quarterly supervision visits; extraction of data; data analysis and interpretation, including conducting root cause analysis; prioritization of health facility issues; and mentorship and coaching. Cordaid was contracted to train identified MOHCC personnel. To facilitate learning and sharing across facilities and to enhance the skills and knowledge of coaches and quality improvement teams, refresher trainings were planned for a minimum of once a year. Quarterly coaching of coaches by MOHCC central coaches to the provincial and district coaches was integrated into the design.

4.2 Selection of CQI implementing health facilities Eighteen districts, two in each of eight Zimbabwean provinces, have been implementing the

RBF program and constituted the study population. Stratified by province, 9 of these 18 districts were initially randomly selected to participate in the CQI pilot program. However, resource limitations further restricted the pilot to five districts. Therefore five of these nine districts were purposively selected in the following manner: the nine districts were ranked on the basis of baseline quality scores for 2014 and 2015 (taken from the balanced scorecard used in the RBF program in those years) as well as the average catchment population for all facilities. To encompass a range of district situations, the three districts with the lowest ranked average, and the two with the highest composite scores were selected. These five districts received the additional quality improvement efforts through the CQI approach.

The two study arms are:

Arm 1: Facilities receiving performance-based incentives for quantity and quality, with quality dimensions determined through a revised Balanced Score Card

Arm 2: Arm 1 activities plus CQI measures and training on management of quality (i.e., quality improvement model)

The five selected districts are listed in Table 4.1 and depicted in the map in Figure 4.2.

19

Figure 4.2 Map of CQI districts studied

Table 4.1: Details of provinces and districts under CQI

ARM 1

Province DistrictDistrict Population Number of Clinics Number of Nurses

Mashonaland Central Centenary 131,219 12 38

Masvingo Mwenezi 166,263 16 50

Manicaland Chipinge 326,467 34 76

Matabeleland North Binga 138,074 14 38

Matabeleland South Mangwe 78,665 11 42

In the original process evaluation plan, 9 randomly selected PHCs and the district hospital that were surveyed in the first round RBF evaluation facility survey were selected to participate in the pilot CQI program. However, resource constraints subsequently limited, before CQI implementation, the number of participating PHCs to five per district (as well as the district hospital). The five study PHCs were purposively chosen from the nine originally sampled facilities, with greater weight given to larger facilities without prior QI initiatives, and greater weight given to facilities that were closer to other participating facilities in order to make the CQI pilot implementation more tractable. Consequently, the supervising district hospital and the five purposively selected rural health centers (RHCs) were chosen for initial implementation.

Figure 4.3 below shows the CQI timeline.

20

Figure 4.3: CQI timeline

4.3 Qualitative process evaluationThe qualitative process evaluation was implemented in 2 phases (Error: Reference source

not found.3). In the first phase (steps 1-4) the evaluation team i) reviewed project related documents, ii) developed process evaluation methodology and tools, iii) sampled health facilities for site visits and collected qualitative information using various data collection methods (described in Section 2.2.3), iv) constructed stakeholder map and produced inception report (steps 1 & 2) covering evaluation methodology, sampling of provinces, facilities and respondents, implementation phases, and schedule and draft outline of the final evaluation report. The inception report incorporated comments solicited from the MOHCC, the World Bank Task Team, and the Principle Investigator (PI). Final data collection tools were submitted to the MOHCC for Ethical Review Committee approval.

Figure 4.3: Phases of the Qualitative Process Evaluation implementation

For the field data collection (step 3) the research team (one international and one national consultant) visited Zimbabwe for two weeks, collected qualitative information, and shared preliminary findings and recommendations with key stakeholders. Following data collection, researchers completed data analysis and triangulation and produced a draft of the CQI process

evaluation report (step 4). The draft CQI report has been shared with the World Bank for review and comments.

The Initiation of the second phase (steps 6-9) of the evaluation was subject to the availability of quantitative PME results collected separately from the qualitative process evaluation (step 5). In this phase, the research team, which comprised one international and one national consultant, reviewed preliminary results of the quantitative PME and jointly with the quantitative evaluation team elaborated exploratory research questions (Annex 2: Exploratory Research Questions) respective data collection tools, and research protocols (steps 6 and 7). For the field data collection

21

(step 3) the research team (one international and one national consultant) visited Zimbabwe for 2 weeks, collected qualitative information and shared preliminary findings and recommendations with key stakeholders. Following data collection step, researchers completed data analysis and triangulation and produced draft CQI process evaluation report (step 4). The draft CQI report has been shared with the World Bank and comments solicited.

At the end of phase 2, the research team visit the country for the second time to ensure collection of exploratory data in the field, analyze and incorporate findings into the final CQI process evaluation report (steps 8-9). The team presented preliminary findings and recommendations to the MOHCC and the World Bank by end of the country mission.

4.3.1 Qualitative data collectionThe qualitative process evaluation methodology comprised a mix of desk-based research,

key stakeholder in-depth interviews (IDI), Focus Group Discussions (FGD) and review of quantitative process evaluation results (in phase 2). Review of documents was a major part of the assignment during the inception phase and largely informed PME methodology. The research team consulted with and obtained necessary documents from the Bank and the MOHCC. Based on the desk review findings, information gaps were identified, and related questions included in the IDI and FGD guides.

IDIs are an important source of evidence for many exploratory research questions and aided researchers to: a) understand the range of contextual and operational challenges and opportunities; b) continue analysis started by the review of quantitative PME results for deeper analysis; c) generate findings and lessons learned; d) explore implementation of different strategies, inputs, and tools by health facilities for improvement of the care quality; and e) identify different results/pathways of contribution where feasible. IDIs were implemented face to face for key informants from central, provincial, district, and facility levels. IDI interview topic guides based on research questions, helped ensure systematic coverage of research questions and issues.

FGDs were carried out for hospital CQI committees. FGD guides included questions specific to the FGD participant group and included 8-10 participants per each group. In health care centers with small number of staffed the FGDs were replaced by small Group Interviews.

During Phase 2, the research team reviewed the preliminary results of the qualitative process evaluation and formulated exploratory research questions that guided the team in the development of the data collection methodology and tools and carried out IDIs of FGDs.

4.3.2 Qualitative samplingA multistage sampling approach was used to select the provinces and facilities. A cascade

sampling enabled selection of three of the five provinces in which CQI was piloted. The selection criteria were based on geographic spread to ensure representation from north and south geo-regions and a remoteness from the capital Harare. Then within each selected province, facilities were sorted by two groups: CQI and non-CQI facilities.

22

Figure 4.4: Provinces, districts, and facilities sampled

Finally, within each group, facilities were sampled ensuring the presence of all types of facilities (hospital, clinic, and rural health center) per each facility group; and quality supportive supervision results to assess facility performance from the 1st quarter of 2018. Sampling for CQI group facilities involved the selection of high and low performing (based on quality scores) facilities, whereas for the comparison non-CQI group, facilities with the highest and lowest quality scores were selected. As a result, in each selected province, six facilities were selected (three CQI group facilities and three comparison group facilities) (Figure 4.4).

Qualitative process evaluation findings are based on the information collected from 232 individuals at national, province, district and community levels in selected three provinces. Overall, apart from individual face-to-face IDIs, researchers also carried out 15 FGDs and 22 group interviews.

4.3.2 Qualitative data analysis and triangulationThe study team entered and analyzed the qualitative data using NVivo 10™ software19.

Analysis focused on synthesizing and triangulating information from the various data sources and evaluation methods. The evaluation took an iterative approach based on grounded theory that allows themes and findings to emerge from the data. In general, this evaluation considered data or evidence to be more valid, and therefore gave it more weight, when the analysis identified convergence in opinions and experiences across multiple sources. However, the evaluation team recognizes that CQI implementation will vary in different contexts and therefore have also reflected opinions and experiences that are not widely shared but are illustrative of a particular situation or consideration.

4.3.3 Qualitative research ethicsThe study obtained ethical permission from the Medical Research Council of Zimbabwe for

the Qualitative process evaluation study as part of the overall RBF impact evaluation research. Before the interviews, participants were informed about the objectives of the study, why they had been chosen, and the risks and benefits of participating in the study. Participants were afforded the

19 NVivo is a qualitative data analysis (QDA) computer software package produced by QSR International. It has been designed for qualitative researchers working with very rich text-based and/or multimedia information, where deep levels of analysis on small or large volumes of data are required. The software allows users to classify, sort and arrange information; examine relationships in the data; and combine analysis with linking, shaping, searching and modeling.

23

opportunity to ask questions or make comments and then decide if they wanted to participate. A consent form which participants were asked to read before the interview, provided further information. Participants then decided to participate voluntarily and were asked to sign two consent forms. The participants were assured that the whole process would uphold the highest standards of confidentiality and that their participation would be anonymized.

4.3.4 Qualitative study limitationsIt is important to acknowledge some limitations associated with the process evaluation.

- At the time of the qualitative process evaluation, the CQI pilot was yet at an early stage of implementation and the findings of the study may not accurately reflect potential attainments of the pilot. More time would have been desired to see medium to long term effects.

- The definition of terms “coaching” and “mentoring” were not well understood by respondents and were used interchangeably during IDIs and FGDs. Thus, the report refers to term “coaching/mentoring” to accurately reflect information collected.

4.4 Quantitative process evaluationThe quantitative evaluation compares quality-of-care outcomes in CQI intervention and

control facilities. All other facilities that did not receive the CQI intervention within the same districts as CQI intervention facilities constituted one possible comparison group of facilities. A second comparison group was made up of facilities in the 13 other study districts that did not receive the CQI pilot.

4.4.1 Quantitative data collection and extractionA baseline survey on quality standards in the form of the revised quality checklist was

collected for all health facilities in the 18 study districts in the last quarter of 2016, and then the intervention began. The quality checklists for both primary and secondary facilities, used for all data capture, are presented in Annex 4.

Data from health facilities’ quality assessments was extracted from the online database. Initially data was captured using the ONA platform, however, starting in 2018, data capture switched to the District Health Information System 2 (DHIS2) database platform. The DHIS2 RBF database is a national database that stores data on most key indicators and disease surveillance for Zimbabwe.

While the RBF Database makes use of the MOHCC HMIS (DHIS2), it has additional data on key RBF indicators and other key variables. Hospital and primary facility level data was extracted from the entire checklist totaling 153 PHC-level indicators and 253 hospital level indicators. Data from these forms included both CQI and non-CQI facilities to enable comparability of results. Data cleaning involved removal of duplications and merging of data from the 15 different forms into one single dataset. The final dataset contained data for the period between October 2016 and December 2018 – i.e., one quarter before the start of implementation and eight quarters of the implementation period. While Cordaid managed and oversaw the maintenance of the DHIS2, the DHE’s carried out data collection. Data was collected using real time data collection tools on mobile devices. However, due to lack of validation rules in the data collection tool, reporting of complete quality indicators varied among DHEs.

4.4.2 Quantitative outcomes adopted in the analysisThe PHC checklist contained data on 153 individual quality-of-care related outcomes. These

were grouped into thirteen aggregate measures assessing the quality of inputs and seven aggregate measures assessing the quality of processes. All aggregate measures were calculated as the percentage of the underlying individual indicators (equally weighted) achieved by the facility. For

24

example, facility infrastructure quality was calculated as the percent of the following eleven items that a facility had: visible sign post when arriving at the clinic, fence/wall in good condition, external appropriate wall finishing, roof intact with well-maintained rain gutters, minimum of three toilets, hand washing facility with soap available near the toilets, electricity for 24 hours a day and 7 days a week, fire extinguishers available and accessible and fire safety plan, appropriate drainage of waste water, shower with either running water or container of at least 100 liters and waiting area with adequate ventilation of waiting area. As the CQI intervention was intended to affect process quality more than inputs, the aggregate process quality indicators are presented as the main results, while the inputs are presented in Annex 3. All individual indicators are presented in their aggregate groups in the Annex 3 Table A1.

The hospital checklist contained data on 253 indicators, which were grouped into 17 aggregate measures assessing the quality of inputs and nine aggregate measures assessing the quality of processes. In addition to the aggregate measures available in the PHC checklist, hospital outcomes include patient amenities; human resources management; surgical, lab and radiology equipment; quality improvement strategy; paediatric care processes; paediatric complications processes, and surgical safety. PMTCT input data was not available from the hospitals.

For both the PHC and hospital outcomes, an average of the four quarters in 2018 was used as the post-intervention timepoint for most analyses in order to (a) allow for the intervention, begun at the start of 2017, to take hold and achieve its intended aims, and (b) smooth variability, including possible seasonal variability, between quarters. A sensitivity check using only the fourth quarter of 2018 to reflect this variability and assess the reliability of the effects is presented in Annex 3.

4.4.3 Quantitative data analysisThe balance of the outcome measures was first assessed prior to the start of the CQI

intervention in the fourth quarter of 2016. A T test with unequal variance was used to identify differences between the treated and control facilities.

To estimate the effect of the CQI intervention on the input and process outcomes, the ANCOVA model20 below was used:

y i2018=β0+ β1 CQ I i+ β2 y i

2016+ϵi

Where y2018 is the value of the outcome in 2018 in facility i, CQI is an indicator for whether the facility implemented the CQI intervention, and y2016 is the value of the outcome in 2016. Standard errors were clustered by district.

As this process evaluation is quasi-experimental in nature, two comparison groups are used for the PHC analysis. In addition, two estimation approaches to the above equation are explored for robustness purposes. First, an unadjusted model is presented that compares the intervention facilities against all PHCs in the 13 non-CQI comparator districts, as well as a model using coarsened exact matching (CEM) weights. CEM is a method that corrects for imbalances between treatment and control facilities by coarsening a set of facility covariates into bins, creating a stratum per bin and assigning observations to the strata, then dropping any observation whose stratum does not contain at least one treated and one control unit.21 CEM weights are then included in the ANCOVA regression. The covariates used for matching include distance from the PHC to the district hospital, facility designation (clinic versus rural health centre), and volume of new patients prior to the

20 McKenzie D. Beyond baseline and follow-up: The case for more T in experiments. Journal of development Economics 2012; 99(2): 210-21. Also see, Frison, Lars, and Stuart J Pocock. 1992. “Repeated Measures in Clinical Trials: Analysis Using Mean Summary Statistics and Its Implications for Design.” Statistics in Medicine 11 (13): 1685–1704.21 Blackwell M, Iacus S, King G, Porro G. cem: Coarsened exact matching in Stata. The Stata Journal 2009; 9(4): 524-46.

25

intervention in 2016. The hospital model is unmatched given the already small number of treated and control facilities (5 and 13 respectively).

The second PHC comparator group contrasts CQI facilities against non-intervention facilities in the same districts. With this comparator, the ANCOVA is again estimated first unweighted and then with CEM weights. Difference and difference models are presented as robustness checks in Annex 3. The PHC difference in difference model includes district fixed effects, when compared with facilities in the same districts, and the CEM weights used in the ANCOVA model. In all models, facilities that do not have data in both 2016 and 2018 are removed case-wise. As the process evaluation is exploratory, P-values are not adjusted for multiple hypothesis testing.22

5.

22 Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology 1990: 43-6.

26

5. Qualitative Results This section of the report summarizes evaluation findings in relation to:

1. Quality improvements of MCH services in intervention health facilities;2. Factors influencing observed changes, particularly examining CQI system structure, content,

delivery modes, reorganization of the facility teams, effectiveness of CQI trainings, and effects of RBF payments on CQI system advancement; and

3. Options and opportunities for CQI system scale-up.

5.1 Quality improvement resultsThe CQI pilot is perceived as one of the most important initiatives for improving the

quality of MNCH services in Zimbabwe. There is a clear agreement among actors involved in CQI pilot about the significance of quality improvement as a fundamental tool to improve health outcomes. According to respondents, the CQI pilot was one of the first and few initiatives that ensured staff exposure to quality improvement, and which strategically and deliberately included a component of local problem analysis and planning of corrective measures. However, there were some limitations of implementation, including variation in the quality of the CQI training and supervision.

CQI facilities demonstrate improved quality of care compared to non-CQI facilities. To perform comparative analysis of the quality improvements at intervention and non-intervention facilities, results of quantitative process evaluation were summarized according three quality domains (structure, process, and outcomes).

CQI facilities demonstrated better results in improvement of MCH outcomes and enhancement of service delivery processes along with advancement of the structural quality than non-CQI facilities. There are a handful of best practices recorded (see Text box).

Both types of facilities (hospitals and health clinics) show an increased proportion of children under five with diarrhoea being correctly treated and the potential for serious bacterial sepsis among neonates managed as per national protocols. Decreases in post-partum hemorrhage rates at CQI health clinics were more evident than in non-CQI health clinics. Although positive results are observed at intervention hospitals and health clinics, higher achievements were recorded at hospitals compared to health clinics (see results of quantitative study). Hospitals demonstrated higher results in improving process and structural quality (see Figure 5.1).

27

- When CQI came it was one of the first few initiatives in terms of improving quality…

- For most health workers it was their first exposure to quality improvements cycles …

- I think it facilitated finding local solutions to local problems…

Quotes from Key Informants at Province and District CQI Facilities

Figure 5.1: Quality improvement by quality domains at CQI facilities

The majority of women interviewed were satisfied with the quality of services they received during childbirth. To learn about their experiences with childbirth and their satisfaction with the quality of care received, the study interviewed women in the postdelivery departments at intervention and non-intervention sites. No significant difference was recorded between CQI and comparison facilities.

Overall, women interviewed for each group of hospitals are satisfied with services received and commended overall good conditions in delivery and postdelivery departments, good hygiene, timely provision of services, and teaching young mothers how to breastfeed their babies. Interviewees unanimously expressed readiness to advise their peers to seek delivery services in these hospitals. Respondents at CQI facilities more frequently highlighted the benevolent attitudes of health personnel, but these results cannot be firmly attributed to only CQI interventions due to the drawbacks of real time patient satisfaction survey methodology used by researchers.

Text Box 1: Selected best practices

Improving immunization coverage: Per month we are supposed to cover 16 kids in each vaccine dose. When we analyzed our immunization coverage, we found it to be below our target of 90%. It’s very difficult to reach out to those children who are not vaccinated. To address this shortcoming, every month we mobilize the community using the village health workers (VHW). VHWs are requested to go door-by door and find children who are due to be immunized. For such children, they write a referral letter to our clinic. Whilst VHW involvement helped to upsurge coverage rates, we were still below the target as not every parent was bringing a child to the clinic for vaccination. Next strategy elaborated by the CQI committee was the EPI outreach and vaccinate children in the communities. At present we mobilize the community to make sure that we have a vehicle to drive to the outreach points and immunize children with the help of VHWs. Addressing late booking for antenatal services and increasing institutional deliveries: We managed to improve the quality of antenatal services and decrease the share of mothers who deliver at home. To address late presentation for antenatal services and decrease home deliveries, the clinic introduced gift packages with soap, napkins, and Vaseline for those who book for antenatal services before 16 weeks of pregnancy, complete six antenatal visits and deliver at the facility. Management of delivery and post delivery period: To improve the quality of delivery services, we assessed whether our staff follow standards and found out that partographs are used during deliveries only in 40% of cases, oxytocin is administered later and mothers after delivery are not regularly monitored due to the

28

- Staff during the whole process was very polite, examined regularly...

- I liked staff attitude, they are considerate and responsive…

- During childbirth, the attitude of personnel was very good and caring…

- After I delivered the baby the nurse would come and check me every other hour…

- I am satisfied with the services received …Quotes from Key Informants at CQI Facility

- I am very satisfied with the services received at the hospital…

- They have blankets, medicines and meals…Quotes from Key Informants at Non-CQI Facility

shortage of staff and high workload. To resolve this issue, we trained our staff in using portograms, placed filled in portogram on a wall as a reminder and regularly monitored its use. This helped us to reach 60% within 5 months. Timely administration of oxytocin was identified as one of the main deviations from the standard. When we completed root cause analysis, we found out that oxytocin is not always readily available in the delivery room, which causes delays. To improve on this, there was a need to ensure availability of the medicine next to the delivery bed of women. By this intervention we assured that in 98% of cases, oxytocin is administered within the first minute after delivery. In order to improve regular monitoring of women during the delivery, we procured wall watches with alarms and assigned a nurse. Alarms are set with 30 minutes interval and remind the nurse to check foetal heartbeat, contraction and pulse. Improving the quality of childhood diseases: We work on improving management of childhood illnesses, by trying to improve correct diagnosis and treatment of cases of diarrhea and pneumonia. Before CQI training we did not know how to assess these variables correctly. When we analyzed only 50% of diarrhoea and 8% of pneumonia cases were managed according to IMNCH guidelines. We had a meeting and strategized how to improve on these indicators. As a result, we provided peer- to peer support and mentoring using those staff who underwent the IMNCH training. We placed flow charts on the wall, that served as a reminder to staff. For the management of pneumonia, we procured timers to accurately measure respiration, oriented new staff and supplied IMNCHI registers and forms. We regularly supervise our staff to ensure that guidelines are strictly followed. By combination of these efforts we ensured 100% compliance with IMNCH protocols. Quotes from key informants at CQI facilities

5.2 Factors influencing observed changesThe evaluation team noted several important factors facilitating and or impeding

achievement of CQI program results. These factors are discussed in following sections of the report.

5.2.1 Enabling factorsThe achievements in improved clinical outcomes and quality of care improvements observed

at CQI facilities were facilitated by a number of factors discussed in detail below.

GoZ political will to promote quality improvement in the health sector set an enabling environment to pilot CQI in selected districts and health facilities that resulted in improved quality. Enabling a QA/QI policy environment and political will of the MOHCC to promote quality improvement in the health sector allowed the initiation of the CQI pilot. The pilot demonstrated improvements in the quality of MNCH services, though achievements are not uniform across all targeted facilities.

Availability of additional funding (RBF subsidy) enabled all RBF facilities to improve quality. Both CQI and comparison facilities had equal access to additional funding (RBF subsidies) based on the facility performance. Availability of additional resources allowed CQI and non-CQI facilities to improve infrastructure, procure some basic medical equipment, as well as maintain a stock of essential medicines and supplies for MNCH services, albeit at various degrees across types of facilities.

Volume of additional funding received by hospitals was higher compared to health clinics due to the higher case load and volume of provided services. This allowed hospitals to perform

29

- With RBF subsidies we improved our facility. We procured some basic equipment although we are yet short of some equipment…

- Subsidies that we receive based on our performance often is not sufficient to cover all needs of infrastructure improvement, procurement of medicines and supplies and/or equipment… Compared to hospitals our earning is smaller because of service volumes…

Quotes from Key informants

- Sometimes we used RBF payments to procure some medicines…

- RBF subsidies help when we are short of medicines… We use these funds to procure medicines when there are delays from the government…

Quotes from Key informants

better by improving structural quality than health clinics though these improvements were marginal compared to non-intervention facilities. CQI facilities were more strategic in using available additional funding for infrastructure improvements that potentially helped to enhance their performance in the field of MNCH, whereas comparison facilities thinly spread resources for general hospital priorities.

RBF subsidies occasionally served as bridge funding to ensure availability of government supplied medicines when delays were observed at both intervention and non-intervention sites.

CQI intervention has increased awareness of and participation in CQI activities and processes, and enabled CQI facilities to accelerate improvements in service provision and clinical quality. Nearly all health service managers and staff interviewed at intervention sites were aware of the concept of CQI, could articulate what it meant, and were able to provide examples of CQI activities in which they had participated. Most informants were able to provide examples of changes to practices they had made as a result of recent engagement in CQI processes. Whilst RBF subsidies were found to be important to structural quality improvement, CQI interventions empowered CQI facilities to accelerate improvements in processes and clinical quality (see Figure 5.1).

Staff capacity building in CQI principles and methods improved quality at CQI facilities. The CQI component of the RBF project supported cascade training of staff in principles and techniques of CQI at provincial, district and facility levels. PHEs and DHEs were trained as coaches in quality improvement principles and how to teach facility CQI committees in MNCH quality; performance measurement and using data for improvement; how to apply quality improvement in practice through the systematic use of simple tools to monitor, assess, and improve the care. Each trained individual had to train others in the health facilities to transfer knowledge and skills.

External Supportive Supervision (ESS) is viewed as important and useful in guiding the health facilities to focus on areas requiring attention. ESS was noted to be helpful in improving performance. CQI facilities reported ESS teams undertaking supportive supervision by reviewing facility data reported per each CQI indicator against primary source of information, examining progress or lack of achieving the target and provided feedback to staff on findings of the supervision, and advising on ways of improvement. Respondents appreciated ESS using a standardized approach to performance assessment, specifically, being assessed against predefined indicators via check lists. Supervisor feedback on results, discussions on identified weaknesses, and recommendations for improvement after each assessment were considered beneficial. Supportive supervision encouraged health facility management and staff member to openly acknowledge weaknesses and mistakes and to work as a team to overcome them.

While non-CQI facilities also received ESS, they lacked mentoring/coaching opportunities. As per the original project design, non-CQI facilities were not supported with mentoring and coaching from their supervisors. Hence, compared to intervention facilities, respondents from

30

- We discuss area of weaknesses and challenges and the next time we visit we track and check we monitor…

- For proper interventions we also advised them to go to the “5 Whys”, process maps and trying to get the root causes. Because without the root cause analysis you will not able to develop proper interventions…

- The supervisors follow on the checklist and after supervision they sit down and tell us how well we have done, the gaps and how we were supposed to do things…

- When they come for the supervision they come as team and analyze our work. then they give us a feedback and give us a %. They call all the staff and we have a short meeting and that is when they give us a feedback on the areas that we are not doing well and also praise us for what we are doing well…

Quotes from Key Informants at CQI facilities

- Nobody coaches and mentors us...- We do not know what is involved in CQI and what is

supposed to be done...- We are trying our best, but we do not have enough

information on what needs to be done to improve quality…

- In CQI facilities quality becomes central to their service delivery as “people who are trained train others”, so there is more knowledge transfer

Quotes from Key Informant at non-CQI facilities

non-CQI facilities reported having no or only vague information about CQI and not receiving any coaching or mentoring from their supervisors.

CQI facilities were shown to be more equipped with skills and knowledge to improve processes (e.g. carrying out root cause analysis), whereas teams at non-CQI facilities mostly depend on how innovative the teams are at defining problems and finding solutions. Respondents from non-CQI sites agreed on the need (i) to be trained in quality improvement and (ii) for mentoring from supervisors who on a quarterly basis supervise their facility and check the accuracy of reported indicators.

CQI fostered leadership, teamwork, and joint decision-making at intervention sites. Strengthened health facility management has been observed in all health facilities since the introduction of RBF. leadership at facility, department/unit, and individual level were more evident at CQI facilities than at non-CQI sites.

Respondents were appreciative that management now consults and engages staff in monitoring results, identifying root causes, planning, and decision making. Health facility QI teams intensified internal supervision and met once a month. As part of regular internal supervision, the QI committee examines each clinical record (clinical audit) sampled and discusses if and how it complied or not with standards of care for prenatal, delivery, immediate postpartum, newborn care, management of main obstetric complications, and management of diarrhoea and pneumonia cases. This process is not only a monitoring procedure, but also a form of collective supervision over the quality of care. The QI committee determines the results of improvement actions implemented and identifies actions required to solve problems detected in weak areas. Based on these discussions, the QI committee decides if the intervention is successful and deserves to be implemented at a larger scale in the health facility or area, or whether it is preferable to modify the intervention, or choose a new healthcare process for improvement.

QI teams use selected CQI tools in their daily practice. “5S”23 was the most commonly used technique by CQI facilities. This tool focuses on establishing an organized workplace, cleanliness, and standardization to improve service profitability, efficiency, and safety. The evaluation found that 5S has been the starting point for health-care quality improvement at reviewed intervention sites. Workplace

23 5S-CQI (KAIZEN)-TQM, using the 5S principle (sort, set, shine, standardize and sustain)

31

- Internal supervision here at the hospitals actually assist us…

- We do our surveillance data analysis and discuss. After assessing the progress, we go back to the ward assessed and report. Alternatively, we discuss via “WhatsApp” mobile application. We also send pictures to people to see where their problems are...

- CQI helps to find local solutions to local problems…Quotes from Key informants at CQI facilities

- Good leader makes things happen…- I think basically what makes things work is leadership and

teamwork because as we sit down as a team, we identify our gaps and find solutions…

- Teamwork is number one in improving CQI…- Last year matron mentored us in using partogram, but we

achieved only 10%. Since January 2018, a new nurse has been assigned to our ward. She strictly follows on use of partogram, trains us. Under her leadership in May 2018 we achieved 87% on use of partogram. Leadership is important to improve quality…

Quotes from Key Informants at CQI facility

- With the 5S you see changes in the department because items are sorted for convenience. It creates a sense of orderliness and give a sign so that that you should not mess up with each other work to be conscious. The 5S makes CQI work…

- We have been sensitized about the 5S on quality and we have removed useless things and retained only needed items and equipment. We learnt a lot from this. We had kept so many things in our department over so many years, old chairs, and old BP machines. We repaired some and disposed others. We filed our material and covered our files. Things are now in order…

- Using the 5S we cleaned up the department. Before the program there was no labelling... We also removed old registers that we did not need…

Quotes from Key Informants at CQI facilities

environment was characterized as a bottleneck in providing adequate services. Respondents noted that 5S was initially the easiest tool for starting improvement of quality exercises. Almost all CQI facilities visited reported improving workplace organization and getting rid of unnecessary equipment, papers, etc. by using 5S. Respondents believe that 5S helps and stimulates working on the quality improvement.

The “5S” technique was used in combination with other CQI tools in pilot facilities, however, experience and knowledge of tools introduced by the pilot are not spread adequately across CQI health facilities. QI committees often apply flow charts, “Why Tree” root cause analyses, and fishbone diagrams, and follow PDSA cycle. However, some respondents also noted having inadequate knowledge to effectively use these tools. The evaluation was not able to observe robust evidence on the regular application of PDSA cycle in studied facilities. Respondents indicated need for additional training and more intense mentoring from PHEs and DHEs.

Employee motivation and satisfaction differs across health facilities. Motivation at work is widely believed to be a key factor for performance of individuals and organizations24. RBF rewards health facilities with supplemental payments based on facility performance, 25% of which is used to motivate personnel and the remaining funds to cover other needs for improving performance. The CQI pilot assumes that financial incentives on top of low salaries will increase staff motivation and promote CQI.

The evaluation attempted to understand how staff financial incentives motivates personnel in improving the quality of MNCH services.

Staff pay for performance had marginal effect on staff motivation to improve care quality at both intervention and non-intervention sites. Views of respondents differed between their peers from the same health facilities and across facilities from different study groups. While few respondents considered financial incentive as a motivation to perform better, a majority thought that financial incentives had marginal effect. Staff at both intervention and non-intervention sites believe that incentives were not sufficient given the increased workload—and that the amount received was insignificant compared to their base salaries. Several facilities reported collectively making decisions on how to use staff performance pay for structural improvements.

Health worker views also varied in relation to the staff incentive distribution formula. All health facilities reported distributing financial incentives to all, clinical and non-clinical personnel using “staff incentive

24 Hornby, P. & Sidney E. (1988). Motivation and Health Systems Performance. WHO, WHO/EDUC/88.196

32

- The root cause analysis is very helpful, because with the root cause you can plan now...

- To identify causes, we use „5 Whys“. This helps to undestand where are the problems and to plan measures...

- PDSA really assist to test out the issues and design interventions…

- We have over 100 staff members and a portion of payment we receive from RBF project, is distributed among all staff based on staff grades. We share this little cake and staff get small incentives. Staff in smaller facilities get more incentives than us… This is discouraging…

- We also do not want it to be equal because of the differences in roles and responsibilities…

- We are not happy. It’s too little…- Partly its fair considering that the differences are not much and

partly unfair because these people directly concerned with the RBF issues working hard have sleepless nights are on the lower rate…

- If everyone is not paid it ends up creating frictions and they are not happy to work with others. It demotivates…

Quotes from Key Informant Interviews

- I believe quality should be internal. One should not be motivated by financial incentives to deliver the best quality services. Of course, incentives motivate, but quality should be part of our daily work…

- Of course, the financial benefit is there but whatever I do, I do it to my best capacity and knowledge…

Quotes from Key informants at CQI and comparison facilities- To ensure electricity supply for the facility, we had to purchase

electricity meters. As RBF subsidies were not sufficient to procure electricity meters in addition to already planned procurements, we collectively decided to seek DHE approval for using the budget dedicated for staff incentives...

calculator” that considers type, level/grade of staff member, experience, and attendance. Findings suggest that because hospitals have more staff than health clinics, the portion of RBF supplemental payments for staff motivation are thinly spread across all staff and are too small to motivate personnel. Few respondents expressed their concerns regarding the fairness of the “staff incentive calculator”. On the other hand, some personnel consider that paying all staff is important to encourage and promote team-work for quality improvement. In summary, while a few respondents considered the financial incentive as motivating them to perform better, the majority thought that financial incentives had marginal effect.

To underpin the causal pathways of marginal effect of financial incentives on staff motivation, the evaluation attempted to understand other factors that powered staff motivation for improved care quality.

Improved working environment had positive effect on staff motivation. Respondents from all studied facilities talked about experiencing changes in their working conditions since the introduction of RBF, including staff dynamics, structural changes due to the RBF subsidies, and changes in supervision and training. A majority of respondents considered improvements in physical working conditions to be the most crucial change with the introduction of RBF. With the subsidies received, facilities in both study groups had improved infrastructure, availability of required essential medicines, and equipment. In addition, facilities were also able to improve sanitary conditions and hygiene. These improvements in working conditions had a positive influence on staff morale and ability to improve performance.

ESS contributed to staff capacity building, knowledge sharing and consequently to personnel motivation. The ESS approach introduced and encouraged by the RBF project influenced capacity of facility management and staff. The benefits of ESS—with its enhanced mentoring/coaching element—are widely recognized by CQI facility respondents that helped to understand how to improve internal supervision practices, apply different CQI tools, and find effective ways to process improvements. ESS is perceived as an influential tool contributing towards knowledge and skills building, thus motivating staff performance.

Improved performance in its term encourages staff to further advance their performance. Attaining better results was found to be another factor encouraging facilities to further improve their performance. Accomplishments on targets encouraged facilities to attend to other priority areas. Improvements resulting from implementation of streamlined procedures led by individual champions, are another

33

- The DHE comes and supervise us. We sit together and discuss… They do not force us to do things, rather encouraging us to work harder. This motivates us as a team…

- It’s not a fault-finding mission. They are coming to support and to motivate us in a good way…

Quotes from key informant interviews

- We have over 100 staff members and a portion of payment we receive from RBF project, is distributed among all staff based on staff grades. We share this little cake and staff get small incentives. Staff in smaller facilities get more incentives than us… This is discouraging…

- We also do not want it to be equal because of the differences in roles and responsibilities…

- We are not happy. It’s too little…- Partly its fair considering that the differences are not much and

partly unfair because these people directly concerned with the RBF issues working hard have sleepless nights are on the lower rate…

- If everyone is not paid it ends up creating frictions and they are not happy to work with others. It demotivates…

Quotes from Key Informant Interviews

- Our patients are happy with improvements we introduced at our facility… Patient satisfaction motivates us to continuously improve…

- When we see that our results improve from quarter to quarter, we do not stop there, but try to achieve higher results in coming months…

- At out facility we achieved 100% use of partographs and administration of oxytocin. Now we will shift to the next priority to improve management of post-partum period…

- I am proud of being responsible to lead the work on these indicators and will continue to closely monitor these indicators to make sure that they do not deteriorate…

Quotes from key informant Interviews

- We do enjoy having clean working are where all items are easily accessible…

- Availability of medicines and supplies is encouraging… Now we can provide better quality to women and children… This brings more self-satisfaction and motivates…

- RBF subsidies help to introduce innovative approaches… Now we can afford to organize outreach immunization sessions, vaccinate more children and show better results… This is really encouraging…we are enthusiastic to explore other innovative ways how to improve quality…

Quotes from Key Informant interviews

motivating factor increasing staff confidence and self-esteem, and promoting personnel dedication and buy-in.

Peer review and peer support practices observed in some CQI facilities contributed to staff empowerment and motivation. Intra-hospital cross-departmental peer assessment of the performance was found to be mutually beneficial for participating hospital departments, catalyzing collective learning, teamwork, staff empowerment, and motivation. Examples of peer support between senior and junior staff and/or between best and poorly performing staff members were found to be another effective practice to capacitate staff and motivate behavior for better performance. Lessons learned from these facilities demonstrate a need for system-wide institutionalization of these peer review and support practices.

Knowledge exchange platforms created by CQI districts promoted staff knowledge and motivation. To inspire CQI, knowledge exchange platforms were established at provincial and district levels. Monthly CQI meetings, held at province level, are attended by all DHEs, where issues related to performance are discussed, common challenges identified, remedial measures planned, and best practices presented. At the district level, the social media platform “WhatsApp” is used for knowledge sharing, peer support, and collective problem solving. Respondents noted benefits of opportunities to: discuss issues with their peers and the DHE, learn how other facilities are performing and/or resolve common problems, and advise and support others when needed. There is a handful of good examples observed in CQI districts. The formation of a social interaction network as an unintended consequence of the CQI pilot is an important aspect for health facility employees because it fosters knowledge exchange required for a particular task. This model also serves as an important way to connect often remote and isolated facilities and providers to the wider health system.

Enhanced social control from the communities is another influential factor keeping health facility management and personnel devoted to continuous quality improvement. The CQI pilot fostered stronger linkages between health facilities and community-based organizations (CBOs) by engaging CBOs in the assessment of patient satisfaction, communicating community concerns to QI improvement committees, and in some cases, participating in joint planning of remedial measures. As reported by CQI facilities and CBO representatives interviewed, the given arrangement empowers a sense of community ownership and creates an obligation and motivation to effectively respond to community needs. While this is a commendable development, the evaluation cannot conclude unanimously that social control mechanisms equally influenced facility and staff motivation and overall facility performance at all intervention and non-intervention sites.

5.2.1 Impeding factorsWhile there is progress in improving clinical quality of MNCH services at CQI

facilities, achievement was undermined by external and internal barriers discussed below.

External Factors:

34

- FCH staff come to maternity ward and maternity goes to FCH and we help each other…

- In the team we use peer reviews. e.g. nurse in charge after analysis of partographs, identifies the best performer and calls other nurses to come and see how partograph has to be filled in…

Quotes from Key informants at CQI facilities

- Close relationship with CBOs helps to understand what we are not doing well, as they regularly undertake patient satisfaction exist surveys… They share main issues with us, and we seek ways of improvements together…

- Even if we do not prioritize some aspect of care, CBOs demand to address them as a first priority… This keeps us alerted to quality improvement…

Quotes from Key Informants at CQI facilities

Fragmentation of national and provincial level QA/QI results in inefficiencies and ineffectiveness of interventions. At the national level, QAD operations are undermined by a combination of different factors such as fragmentation of QA/QI functions among MOHCC Divisions and lack of horizontal coordination, insufficient staffing, and absence of operational budget. The QI strategy aimed at establishing the QAD with the responsibility for overall operational oversight and coordination for QI, but in reality, the QI function is distributed to different MOHCC structural units such as the Curative Care Division (CCD), the Preventive Care Division (PCD), and QAD—with weak horizontal coordination among them. The CCD and PCD are heavily supported by development partners, whose program/project provides technical and financial support to these divisions, as well as assist in operationalization of QI plans for vertical programs (e.g. HIV/TB, EPI, MCH QA/QI initiatives). Apart from external financial support, these divisions are also well resourced by the MOHCC budget.

In contrast, there is limited support available for QAD, which in the MOHCC hierarchy falls below the CCD and PCD (See Figure 2). Whilst these divisions directly report to Permanent Secretary (PS), QAD reports to Chief Director of Policy, Planning, Monitoring and Evaluation Division and lacks an assigned operational budget.

QAD staffing is another factor limiting QAD to administer its assigned functions. At present, the department has three full time staff: Department Head; Deputy Head; and Secretary, and one part time international consultant seconded to QAD and fully financed by the World Bank project. With the given staffing ratio, location in the hierarchy of the MOHCC, and absence of an operational budget, QAD is not well positioned to administer leadership function in improving health services quality. An imbalance of high responsibility with low power underpins the shortcomings of the QAD’s performance.

According to the Quality Improvement Policy25, each MOHCC technical department is requested to work in collaboration with relevant professional associations and councils as well as the QAD to define a 5-year operational improvement strategy in their respective technical areas, which are aligned to MOHCC strategic objectives. Individual thematic TWGs had to develop a detailed operational plan that prioritizes and sequences specific areas of improvement, indicators, training, and supervisory support for the high-burden conditions in their technical area. However, at the time of the process evaluation, Zimbabwe lacks a comprehensive 5-year quality improvement plan that should be guiding MOHCC operations in improving service quality.

Fragmentation of QA/QI functions at the MOHCC level is mirrored at province, district, and health facility levels and CQI is treated as one among other government programs. At each level there are different teams and QI committees

25 Quality Assurance and Quality Improvement Policy, 2015, Ministry of Health and Child Care Zimbabwe

35

- Each Division have its own QI plan and budget for implementation. Although we tried to coordinate and elaborate a comprehensive QI plan and budget, where all divisions are assigned to particular tasks, it did not work out. In summary, we fail to have one, streamlined QI plan to guide QI process within the system…Other divisions are supported by development partners and programs allowing these divisions to have budget in support of vertical QI initiatives…

- Being layer below other divisions, undermines our importance and influencing power…

- We are only 2 full-time staff and one part-time quality specialist…- The department does not have budget. In order to work on

clinical protocols, we mostly rely on DP support… We cannot travel regularly for M&E to see how CQI functions at province and district levels, what are the challenges and best practices…

Quotes from Key informants

- At the province level we have number of programs which have its own QI component. Supervision visits are paid to facilities separately by different program teams using program specific QI check-lists…

- For CQI there is a team of three individuals who only check performance against CQI indicators…”

Quotes from Key informants

overseeing quality of services within different government programs such as TB, HIV/AIDS, PMTCT, OI, MNCH, etc.

CQI is perceived as another government program, rather than approach to service quality improvement for which there is a separate group of people and committee working on CQI issues. Consequently, the presence of several QI committees increases the administrative burden and makes both the system and processes inefficient and ineffective.

Attempts to integrate all QI interventions into one comprehensive QI plan failed at province, district, and district hospital levels. Inefficiency of QI committee operations at all levels is well-acknowledged. Integration of all QI interventions into one comprehensive QI plan using CQI has been discussed and probed at province, district, and CQI facility levels, but without success. Respondents noted having limited authority to make changes. Without adjustments of QI structure at the central level and standardization of approaches and processes, they are not in a position to ensure improvements in QI committee performance.

New national monetary policies inhibit effective implementation of CQI at intervention and non-intervention sites. Before 2019, financial transactions using USD were permitted and RBF project used USD for disbursing RBF subsidies to health facilities. In June 2019, the Reserve Bank of Zimbabwe abolished the multiple currency system and replaced it with a new Zimbabwe dollar (RTGS) Dollar26 and set exchange rate 1 RTGS to 1 US$. The new monetary policy greatly affected the volume of RBF subsidies received by facilities and restricted their ability to ensure adequate supply of medicines and/or implementation of planned quality improvement activities at the initial phase of policy implementation. Though the GoZ’s approach has changed lately to use inter-bank exchange rate instead of original proposed one, but at the time of the qualitative process evaluation (Phase 2) facilities have not yet received their RBF funding.

Delays in payment of RBF subsidies result in reduction of purchasing power and shortages in the procurement of goods for quality improvement. In January 2018, responsibility for the payment of RBF subsidies was handed to the MOHCC from Cordaid. Respondents at CQI and non-CQI facilities alike flagged problems related to the delays in performance-based payments as well as lower levels of payments due to the recently introduced government monetary policy.

Another impediment was decreasing purchasing power of the subsidies due to observed delays in payments by the MOHCC. Estimation of RBF quarterly subsidies is performed by the MOFED based on the intrabank exchange rate (starting from the quarter 2, 2019) on the day of budget preparation. Variations in the exchange rate until subsidies reach the facility negatively affect purchasing power of subsidies

26 The RTGS Dollar, is Zimbabwe’s   new currency since 21 February 2019 and came into effect after the February 2019 Monetary Policy by the Governor of the Reserve Bank of Zimbabwe.  The currency is made up of bond coins, bond notes and RTGS balances. The original bond notes came in to become part of Zimbabwe's multicurrency system which was introduced in 2009. They were renamed RTGS dollars in 2019, and in June 2019 became the only legal currency in Zimbabwe, replacing the multicurrency system

36

- CQI is a vertical program and at times integration is not possible because some programs are better paying, and some more are willing to spend only an hour at a health facility…

- Everything should start from the top. Everything we are doing as a way for measuring CQI just ends there… That’s how QI is organized at central level and cascaded to levels below…

- TB only looks at mentoring and spends less time at a facility. So, the vertical disease specific QI initiatives cannot be integrated with CQI because by the time CQI is finished the TB team will have completed their work already...

Quotes from Key Informants at Provincial, District and CQI facilities

- Subsidies were calculated at exchange rate of 3 RTGS:1 US$, but when funds reached us the exchange rate was already 10.4 RTGS to 1 US$

- We often fail to complete our procurement process, because although we collect 3 quotations, at the date funds are received, prices are changed by suppliers and we have to start the whole process again. This delays procurement of medicines and supplies and affects our performance …

Quotes from Key informants

allocated for the procurement of commodities in the quantities planned. The situation is further worsened by changing prices of health commodities in the market due to the frequent fluctuation of RTGS/US$ exchange rate. Observed challenges adversely affect their performance in the next reporting period. The latter ultimately results in lower pay and further limitations for quality improvement.

Human resource for health planning along with a government freeze of employment resulted in staff shortages and adversely affected quality of care. The evaluation observed improvement of staff posting in both groups of studied health facilities. However, the evaluation discovered that post establishments were set around the time of independence in the country and have never been revised since that time. Along with inadequate staff establishment, other challenges are the maldistribution of health workers and the associated difficulty of filling health worker posts, particularly in rural areas, which has long been recognized as a serious challenge to the equitable provision of healthcare.

Health sector staffing was further affected by recruitment freeze introduced in 2010. In response to deteriorated economic growth and in search of efficiency gains, an employment freeze was introduced to control the public service wage bill. Though the freeze was occasionally temporarily lifted and short windows of hiring were provided between 2011 and 2013, staffing levels continued to be inadequate. The combination of low establishments and staff shortages caused managers of studied health facilities to complain of the difficulty of managing the workload with insufficient staff. The recent amnesty for nurses has slightly improved workloads as posts have been filled by returning nurses and/or junior nurses (though most junior nurses lack practical experience). The gradual loss of skills among returning nurses and insufficient knowledge and lack of hands-on experience of juniors, has required more mentoring and training from already practicing senior staff.

Internal Factors:

The effectiveness of CQI trainings was challenged by the training content, intensity, and mode of delivery. While cascade trainings have been beneficial in the early stages of the CQI implementation as a mechanism for getting CQI ‘off the ground’, supporting staff at all levels to become familiar with CQI concepts and tools and increasing awareness and buy-in to CQI was hampered by:

Content of the training – Respondents commented on the lack of balance between theoretical knowledge transfer and practical skill development during the training course. Training participants also acknowledged complexity of issues and recommended simplification of the training and/or extend the duration from the current five days.

Inability and/or unwillingness to transfer knowledge – Considering the complexity of the training, not all trained professionals were able to fully absorb knowledge delivered during the training and/or are able or willing to further transfer it to other staff members.

37

- Training was more theoretical than practical. The practical side was lacking…

- The course is very difficult to understand…- The training is too congested and theoretical delivered in 5

days. Because of its intensity after the training we could only remember the 5s. I think we need to have to redesign the training programme to be easily digested because this becomes complicated…

Quotes from Key informants

- After CQI training we came back and give feedback, but it was very difficult for people to grasp the concepts…

- We were trained in CQI but it’s very difficult to get someone who will explain everything to you, because that person also has her own way of understanding…

Quotes from Key informants

High staff turnover – Staff trained in CQI moved to other facilities and/or locations, and their replacements lacked the knowledge of CQI, hence challenging effective quality improvement.

High staff workload – In the context of observed staff shortages, staff have high workloads that limit transfer of CQI knowledge to newcomers. The evaluation found that at the time of the qualitative process evaluation, most of the newly deployed staff have not yet been trained in CQI.

Lack of clinical training opportunities impedes MNCH quality improvement. Provision of the clinical training was not originally planned under the CQI pilot. Nonetheless, some health workers at CQI facilities have attended different training courses supported by the MOHCC with financial support from developing partners. Respondents noted a need for more trainings due to the high staff turnover and sub-optimal in-house transfer of knowledge by trained staff. A need for more clinical trainings was also mentioned by respondent of non-CQI facilities.

The effectiveness of supportive supervision and mentoring/coaching from provincial and district levels was confronted by a number of challenges: According to respondents, supervision is not always followed by mentoring/coaching of health facilities. The effectiveness of supervision from provincial and district levels is hampered by a number of factors:

Staff shortages at provincial and district levels and high workload limit effective supervision, coaching, and mentoring by provincial and district health authorities. Respondents highlighted a difficulty in assembling a team for CQI supervision visits. Often there is a shortage of staff and high staff turnover, and those deployed have a high workload.

Time spent at health facilities restricts mentoring and coaching. Time spent at health facilities during the supervision is defined by the PHE/DHE teams’ plan on number of facilities they have to visit that day. PHEs and DHEs report that on average they spent 3-4 hours per facility. Time required for checking data and meeting with the staff for feedback often does not leave the room for coaching and mentoring, as teams have to move to another health facility. Supervisory teams acknowledge that they are supposed to supervise every department and coach staff but are not always able to allocate sufficient time for the given task. A need for more mentoring and coaching was also confirmed by key informants at CQI facilities. The majority noted that supervisory teams mostly come for supervision, but not for coaching and mentoring.

Observed delays in RBF funded performance payments impede quality improvement at CQI facilities. Respondents at CQI and non-CQI facilities complained about delays in performance-based

38

- Two health workers from each CQI health facility were trained. But because of staff turnover some of the health facilities now have none of its staff trained…

- I have not been trained for the CQI yet. I have been here for about four months…

- When the team from the district came, we did a little verification that is when I picked one or two things…

- There are now 5 nurses trained in the clinic and the other nurses are not trained. This compromises on quality…

- We need training. Quotes from Key informants at CQI facility

- Duration of supervisory visit is based on the CQI needs of the facility. it’s just estimated time…

- We suppose to supervise department by department, but we fail due to time limitations…

Quotes from Key informants at PHE, DHE- Most of the time the PHE team comes for supervision but there is

little time left for mentoring…- Out mentors come, supervise, provide feedback and advise us

how to improve. But we need more mentoring and coaching to increase our knowledge and skills in applying CQI…

- They don’t make time for coaching…Quotes from Key Informants at CQI facilities

- Due to staff shortages and workload it takes a while to have a full complement of the team for supportive supervision hence usually when we go out the team fails to have all representatives and at most only three members go…

- There is need to add more members to the team. There are times when the DMO and DNO are not there and the pharm tech is on leave…

Quotes from key Informants at CQI facilities

payments. Absence of sufficient funds to ensure quality improvements adversely affects their performance in the next reporting period. The latter ultimately results in lower pay and further limitations for quality improvement.

6.

39

6. Quantitative results 6.1 Primary healthcare facility results

Available data on at least some indicators in 2016 and 2018 covered 24 PHC facilities that received the CQI intervention, 230 control facilities in the 13 comparator districts, and 63 control facilities in the same districts. At baseline, most facilities performed well on the input measures but had poorer performance on the process measures. This is likely due, at least in part, to the participation of all study districts in the RBF program. For example, across all facilities, facilities had on average 79% of the general equipment items such as radio, mosquito nets and adult weighing scales. Of the input measures, the facilities performed the worst on the HMIS completion and correctness indicators and the highest on the privacy indicators. Performance on process measures was generally worse than input measures at baseline. For example, all facilities completed only 62% of ANC processes on average. PHC facilities by far performed the worst on obstetric complications, with facilities only completing 3% of the items on average. There were several individual indicators, including correct management of intra- or post-partum sepsis and resuscitation of newborns with APGAR scores below 5 that no PHCs had completed in the fourth quarter of 2016 (Annex 3). This suggests PHCs should not be handling obstetric complications.

Table 6.1: Balance of PHC summary outcomes in 2016

Outcome CQI Control 1

Control 2

T testp-value 1

T test p-value 2

N CQI

N control 1

N control

2

Input measures

General infrastructure: % of 11 items 73% 69% 63% 0.23 0.02 24 230 63

Cleanliness: % of 6 items 88% 87% 85% 0.60 0.41 24 211 62

Safety: % of 4 items 80% 66% 80% 0.01 0.93 24 230 63

Privacy: % of 3 items 96% 95% 93% 0.73 0.33 24 205 60

General equipment: % of 7 items 82% 78% 81% 0.17 0.74 24 214 62

Pharmacy management: % of 9 items 79% 76% 84% 0.35 0.17 24 203 61

Pharmacy stock: % of 25 items 78% 79% 80% 0.77 0.47 24 213 61

Availability of guidelines: % of 6 items 90% 90% 88% 1.00 0.57 24 204 60

Vaccine management: % of 17 items 86% 88% 88% 0.51 0.66 19 178 37

Maternity supplies: % of 11 items 77% 70% 80% 0.08 0.45 23 168 59

HMIS: % of 6 items 66% 62% 68% 0.51 0.77 23 209 59

OPD triaging knowledge: % of 4 items 95% 87% 96% 0.07 0.74 24 207 60

TB screening knowledge: % of 5 items 96% 92% 96% 0.25 0.95 19 182 50

Processes measures

ANC process quality: % of 5 items 65% 60% 69% 0.25 0.47 19 217 46

PNC process quality: % of 5 items 69% 71% 66% 0.59 0.61 20 226 48

PMTCT process quality: % of 4 items 46% 38% 42% 0.36 0.71 17 212 49

Sick child assessment quality: % of 3 items 57% 38% 73% 0.05 0.12 17 225 44

40

Sick child treatment quality: % of 5 items 32% 35% 42% 0.53 0.08 17 225 44

Maternal process quality: % of 10 items 68% 59% 65% 0.11 0.74 16 203 46

Obstetric complications quality: % of 7 items 2% 3% 1% 0.59 0.32 19 211 47

Where Control 1 are PHC facilities in the 13 non-intervention districts and Control 2 are non-intervention facilities within the CQI districts

The majority of summary outcomes were balanced between CQI and both groups of control facilities at baseline. Where the indicators differ, CQI facilities generally performed better than the control facilities in other districts (Control 1). For example, CQI facilities on average scored 80% on safety indicators in comparison to 66% in control facilities from other districts. Similarly, CQI facilities on average scored 57% on assessing sick child care for pneumonia, diarrhea and malaria, while control facilities from other districts scored 38%. Differences between the CQI facilities and control facilities in their own districts (Control 2) were minor for most summary measures, although general infrastructure was higher in CQI facilities at baseline. There are select differences between treatment and control facilities in other districts at the individual outcome level, in these cases the CQI facilities also tend to score better than the control facilities (Annex 3 Table A1). For example, 96% of CQI facilities had handwashing facilities with soap available for outpatients, in comparison with 76% of control facilities. Taken overall, there are few differences between treatment and comparison facilities, especially with regard to the process measures which are the focus of the CQI activities.

CQI and control facilities were largely balanced at baseline on the identified covariates used in the matching estimator (Annex 3 Table A2). A greater percentage of control facilities are clinics (versus RHCs) than CQI facilities. Both CQI and control facilities are a mean distance of 41 km from the district hospital and had similar volumes of new patients in 2016.

The unadjusted estimates for the effect of the CQI intervention on the process quality measures using the ANCOVA model and the other 13 districts as comparison are presented in Table 5.2. CQI was associated with improvement for two of the seven process measures: postnatal care and maternal care. The intervention was associated with completing one half of an additional PNC process item correctly and one additional maternal process item. The individual indicator results reveal that the PNC improvement is driven primarily by better assessment in the CQI facilities: facilities were 25 percentage points more likely to have all women checked for their general condition, pulse rate and temperature in comparison to the control facilities. Among the maternal care process indicators, monitoring for women and newborns in the 4 th stage of labor, administering immediate postpartum oxytocin, and monitoring women using partographs were the indicators that improved the most between CQI and control facilities (Annex 3 Table A10). As maternal and post-natal care were two focus areas of the CQI program, these results suggest that CQI related efforts may have improved process quality of care in these areas. As expected, the intervention was not associated with improvements of any of the input quality measures (Annex 3 Tables A3 and A4).

Table 6.2: Unadjusted PHC summary ANCOVA results

CQI Baseline Total N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 5 items -0.03 0.35 0.17 0.04 236

PNC process quality: % of 5 items 0.11 0.02 0.21 0.00 246

PMTCT process quality: % of 4 items -0.03 0.75 0.12 0.03 229

41

Sick child assessment quality: % of 3 items 0.02 0.75 0.33 0.00 242

Sick child treatment quality: % of 5 items 0.11 0.16 0.34 0.00 242

Maternal process quality: % of 10 items 0.12 0.00 0.32 0.00 218

Obstetric complications quality: % of 7 items -0.03 0.07 0.27 0.08 230

After matching on the facility covariates, CQI is still significantly associated with improvement in these two indicators (at a p-value of .10 or less), suggesting that the results are robust to covariate balancing and a narrowing of the comparison set. As before, no other process indicator is significantly improved in the CEM ANCOVA analysis.

Table 6.3: Adjusted PHC summary ANCOVA results

CQI Baseline N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 5 items -0.03 0.44 0.11 0.22 105

PNC process quality: % of 5 items 0.10 0.03 0.26 0.00 109

PMTCT process quality: % of 4 items -0.07 0.44 0.12 0.37 105

Sick child assessment quality: % of 3 items -0.02 0.75 0.35 0.01 107

Sick child treatment quality: % of 5 items 0.13 0.13 0.27 0.03 107

Maternal process quality: % of 10 items 0.10 0.06 0.22 0.17 98

Obstetric complications quality: % of 7 items -0.02 0.13 0.12 0.34 103

Figure 6.1. Trend in maternal process quality over study period in PHC CQI and control facilities in 13 other districts

42

In the difference in differences model, the PNC process result was robust, but the maternal quality process improvement was no longer statistically significant. Standard errors are typically higher in difference-in-difference models, especially in the presence of measurement error in the differenced variable. In addition, when using the difference in differences model, assessment of sick child care was worse among CQI facilities in comparison to the control facilities. The graph of this trend (Annex 3 graphs) suggests this is due to particular poor performance in sick child assessment in control facilities at baseline rather than poor performance among CQI facilities.

When comparing the intervention facilities against control facilities in their districts (Tables 6.4 and 6.5), maternal care quality improvement is no longer significant at standard levels, although the coefficients are positive, of similar magnitude, and attain p-values of .13 (unmatched) and .15 (matched). Post-natal care quality still shows significant improvement, though the magnitude of the improvement is smaller than what was observed using the other districts as comparison. Sick child treatment quality is improved in the unmatched analysis, but the coefficient is very small and this association disappears in the matched analysis. There is no change in any of the other process measures. Given all of these robustness checks, it suggests that the CQI intervention did improve post-natal care quality in the intervention facilities, and likely maternal care-quality as well, but no other process quality measures. In addition, as expected given the focus of the CQI activities, no improvement in input quality measures was observed.

Table 6.4. Unadjusted PHC summary ANCOVA results using same-district comparators

CQI Baseline N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 5 items 0.02 0.54 -0.06 0.53 65

PNC process quality: % of 5 items 0.05 0.06 0.12 0.12 68

PMTCT process quality: % of 4 items -0.05 0.50 0.08 0.33 66

Sick child assessment quality: % of 3 items 0.06 0.17 0.04 0.32 61

Sick child treatment quality: % of 5 items 0.02 0.03 0.30 0.01 61

Maternal process quality: % of 10 items 0.12 0.13 0.52 0.01 62

Obstetric complications quality: % of 7 items -0.02 0.32 -0.08 0.60 66

Table 6.5. Adjusted PHC summary ANCOVA results using same-district comparators

CQI Baseline N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 5 items 0.05 0.17 -0.10 0.28 52

PNC process quality: % of 5 items 0.06 0.03 0.07 0.10 55

PMTCT process quality: % of 4 items -0.05 0.22 0.07 0.28 53

Sick child assessment quality: % of 3 items 0.06 0.24 0.08 0.32 50

Sick child treatment quality: % of 5 items 0.01 0.90 0.50 0.03 50

Maternal process quality: % of 10 items 0.08 0.15 0.20 0.10 51

Obstetric complications quality: % of 7 items -0.02 0.35 0.04 0.82 55

43

When looking at the descriptive trends of input or process quality measures, there appears to be large heterogeneity by quarter rather than smooth trends in many of the summary indicators (Annex 3 graphs). PNC process quality seems to improve in both CQI and control facilities just after the CQI intervention began but then revert down by the end of 2018. CQI facilities did not consistently perform better than control facilities over this period.

Figure 6.2 Trend in PNC process quality over study period in PHC CQI and control facilities in 13 other districts

The qualitative section of the report found that the funding through RBF was important in improving inputs or structural items in both CQI and control facilities, while the CQI intervention was thought to have a larger impact on processes of care. These quantitative results are partially consistent with this conclusion. Although the CQI intervention did improve maternal and post-natal care processes, these are limited when considering the full breadth of potential MNCH process quality items.

6.2 Hospital resultsFive hospitals implemented CQI, however data was unavailable from one of the facilities in

Q4 of 2016, and it was therefore dropped from the analysis. Data was available for at least some indicators from sixteen other hospitals. Similar to the PHCs, hospitals overall performed better on the quality of inputs measures than the quality of processes at baseline. For example, nearly all facilities had all appropriate guidelines available and staff scored an average of 97% on knowledge of outpatient triaging. Process quality was generally lower than input quality in 2016, although facilities still scored highly on both maternal care processes and paediatric care processes. Hospital performance was generally higher than PHC performance at baseline, although the measures are not directly comparable due to different indicators included in the summary measures.

44

Table 6.6: Balance of hospital summary outcomes in 2016

Outcome CQI ControlT test p-value N CQI

N Control

Inputs

General infrastructure: % of 18 items 83% 84% 0.82 4 14

Cleanliness: % of 7 items 86% 88% 0.58 4 14

Safety: % of 24 items 89% 78% 0.14 4 16

Privacy: % of 4 items 94% 81% 0.15 4 13

Patient amenities: % of 6 items 95% 78% 0.06 4 13

General equipment: % of 19 items 81% 79% 0.72 4 15

HR management: % of 13 items 88% 83% 0.47 4 17

Surgical, lab and radiology equipment: % of 16 items 88% 84% 0.68 3 15

Pharmacy management: % of 16 items 75% 78% 0.86 3 15

Pharmacy stock: % of 21 items 76% 77% 0.97 4 16

Guidelines availability: % of 8 items 100% 96% 0.19 3 10

Vaccine management: % of 16 items 97% 83% 0.05 2 10

Maternity supplies: % of 17 items 88% 82% 0.46 2 13

HMIS indicators: % of 9 items 83% 84% 0.90 4 16

Quality improvement strategy: % of 22 items 71% 48% 0.15 3 10

OPD triaging knowledge: % of 2 items 100% 90% 0.17 3 10

TB screening knowledge: % of 10 items 80% 91% 0.18 3 9

Processes

ANC process quality: % of 7 items 76% 43% 0.09 3 12

PNC process quality: % of 5 items 80% 70% 0.54 3 14

Sick child assessment quality: % of 3 items 67% 39% 0.51 3 11

Sick child treatment quality: % of 5 items 53% 51% 0.91 3 11

Maternal process quality: % of 10 items 87% 73% 0.16 3 14

Obstetric complications quality: % of 8 items 79% 61% 0.34 3 14

Paediatric care processes: % of 4 items 100% 69% 0.02 3 12

Paediatric complications processes: % of 4 items 75% 52% 0.47 3 11

Surgical safety: % of 2 items 83% 59% 0.31 3 11

Also similar to PHCs, there were few differences in outcomes between CQI hospitals and control hospitals in 2016 but where there were, CQI hospitals had better performance. All CQI hospitals completed all of the recommended basic paediatric care items, while control facilities scored 69% on average. CQI facilities also had better maternal care quality than control facilities at baseline. Although not statistically significant, it is worth noting that quality improvement plans and

45

strategies were already higher in CQI facilities before the start of the intervention, with an average of 71% of the 22 quality improvement items in comparison with 48% in control facilities.

The estimates for the effect of the CQI intervention on the outcome measures in hospitals using the ANCOVA model are presented in Table 6.7. The study is not powered to find significant associations among the limited number of hospitals, and so unsurprisingly there are no significant associations. Even given the limited sample size, however, many outcomes have CQI coefficients close to zero, suggesting that there may not be significant effects even with larger sample sizes. The largest positive coefficient is for sick child treatment quality. The coefficient for CQI on paediatric care processes was large and negative, however that is due to the high performance of CQI facilities on this outcome at baseline as described above. The intervention was also not associated with any of the input quality measures in hospitals (Annex 3 Table A11).

Table 6.7: Hospital summary ANCOVA results

CQI Baseline N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 7 items -0.01 0.86 0.15 0.18 15

PNC process quality: % of 5 items -0.05 0.73 0.05 0.77 17

Sick child assessment quality: % of 3 items 0.07 0.52 0.12 0.33 14

Sick child treatment quality: % of 5 items 0.23 0.18 0.25 0.42 14

Maternal process quality: % of 10 items 0.02 0.78 0.01 0.94 17

Obstetric complications quality: % of 8 items -0.01 0.95 -0.15 0.47 17

Paediatric care processes: % of 4 items -0.22 0.15 0.38 0.04 15

Paediatric complications processes: % of 4 items -0.12 0.45 -0.05 0.78 14

Surgical safety: % of 2 items -0.01 0.98 0.42 0.07 14

Despite the lack of associations in the summary outcomes, CQI was significantly associated with a few of the individual input quality outcomes (Annex 3 Table A14). For example, CQI was associated with a 31 percentage point increased likelihood of having an appropriate waste pit, 18 percentage point increased likelihood of having second line paediatric ART medications, and 47 percentage points more likely to have a functioning ultrasound machine. However, CQI was also associated with a 34 percentage point decreased likelihood of correct treatment for intra-partum or post-partum sepsis, and had no significant associations for the vast majority of indicators.

The results are robust to using the fourth quarter of 2018 as the outcome instead of all of 2018 (Annex 3 Table A12). The difference in differences model does reveal several significant associations, however, all of them show that CQI was negatively associated with the outcome. ANC process quality and paediatric complications processes were both negatively associated with the CQI intervention. The number of ANC items completed correctly decreased by about 2 out of 7 items as a result of the CQI intervention. The graph of these trends shows that although ANC process quality was improving in both CQI and control hospitals, control facilities had more rapid improvement. In contrast, paediatric complications processes increased slightly in control facilities but fell in CQI facilities, particularly in the 4th quarter of 2018. Given large heterogeneity in the data from month to month, it is possible that these results are spurious.

46

Figure 6.3 Trend in ANC process quality over study period in CQI and control hospitals

Figure 6.4 Trend in paediatric complications processes over study period in CQI and control hospitals

The hospital results also differ somewhat than those presented in the qualitative section of the report. First, the qualitative section suggests better MNCH related process quality among CQI facilities. However, the results here suggest that the CQI intervention did not result in better quality improvement processes in hospitals, in part because these processes were already better in CQI facilities prior to start of the intervention. The difference in difference analysis shows that quality improvement processes improved by 32 percentage points in control facilities over the study period, in comparison the CQI facilities improved by a much smaller amount. Second, the qualitative section reported structural improvements in both CQI and control facilities particularly in hospitals due to the higher RBF payments. However, the trends of structural measures in Annex 3 reveal that many inputs actually declined in both CQI and control facilities over the study period. General infrastructure, cleanliness, patient amenities, human resources management, and maternity supplies all saw broad declines from 2016-2018. The structural indicators that saw improvement, including

47

pharmacy management, HMIS indicators and quality improvement, do not require large increases in funding. Finally, the qualitative results report that hospitals demonstrate greater results in improving process and structural quality than PHCs. This finding is not directly comparable in the quantitative results because of the differences in indicators and power to detect differences, however many of the coefficients appear to be the same size or smaller than those seen in the PHC analysis.

48

7. Conclusions and lessons learned

The Zimbabwe health system recently piloted a CQI intervention in select PHCs and district hospitals in five rural districts that had already been implementing a related RBF program over the four previous years. This process evaluation leverages both qualitative and quantitative data to understand whether CQI had an impact on quality of care structures and processes, and factors that contributed to or inhibited its success. The qualitative section drew on key stakeholder in-depth interviews and focus group discussions to identify and understand enablers and barriers to change from the CQI intervention. The quantitative section uses routine data collected by the RBF program in both CQI and comparison facilities to explore possible differential gains in process quality measures in the facilities that participate in CQI activities.

Uptake of CQI is a complex process that engages multiple stakeholders in new relationships that can operationalize key concepts and tools, develop and embed new practices into service systems and routines. At the time of the process evaluation, the CQI pilot is in its experimental phase of implementation. Hence, results achieved so far should be treated with caution.

The progress of internalization of CQI is slowly gaining momentum and has exhibited positive changes in management processes but has shown limited improvement of quality of care processes. Table 7.1 below schematically presents both qualitative and quantitative process evaluation findings on the changes in quality processes, management processes, structural quality and client satisfaction. The pilot demonstrated some improvements in the quality of MNCH services, though achievements are not uniform across all targeted facilities nor all services.

Table 7.1: Comparison between CQI and non-CQI facilities

Domains` CQI Facilities Non-CQI Facilities

District

Hospital

PHC District

Hospital

PHC

Quality processes

ANC quality

PNC quality

PMTCT quality (PHC only)

Sick child care quality

Routine maternal care quality

Obstetric complications quality

Management processes

Leadership

Teamwork

MNCH QI processes

Ownership of clinical quality improvement

Staff engagement

Capacity building

Staff motivation (financial)

Staff motivation (non-financial)

Structure

Client Satisfaction

49

There is some evidence that CQI improved maternal care and postnatal care processes in PHCs, but no evidence of improvement in any inputs, as expected, or any of the hospital processes. Improvements were limited when considering the full breadth of potential outcomes but arise in certain areas of focus of the CQI program.

This absence of broad-based improvement may be due to several factors. First, CQI facilities were already performing better than control facilities at baseline on many indicators. Second, many structural indicators were already very high at baseline. In combination, these suggests that there may have been ceiling effects where CQI facilities were unable to improve anymore but control facilities were. Third, the qualitative results suggest that there was mixed success among different CQI facilities as the intervention interacted differently with the environment, the service, the quality improvement process and the stakeholders. The overall grouping of CQI and control facilities may have masked heterogeneity of improvement, with some facilities performing better.

The limited improvement may also be due to limitations in the checklist data: First, there was a change in measurement system in Q2-Q4 2018. Measurement likely improved non-differentially, but ANCOVA results do not fully adjust for baseline outcomes if there was measurement error. Second, missingness was high, particularly for vaccine management and maternity supplies forms. We assume that missingness was completely at random, however missingness could be related to data management ability. Regardless of bias, it limits the power to detect differences.

The mixed success among different CQI facilities is associated with the interaction of features of the environment, the service, the quality improvement process, and the stakeholders, which operated together to produce a set of circumstances that either inhibited or enabled the process of change. The following section of the report summarizes enablers and barriers influencing effectiveness of the CQI pilot from the qualitative analysis.

7.1 Drivers of changeTo summarize key drivers of change at health facility level, as seen by health service

providers, the evaluation explored perceptions of respondents on the internal key factors influencing care quality. Respondents were asked to name factors by priority and answers were recorded on a five-point Likert scale and transformed into unidirectional measures with the highest level of influence “5” and the lowest “1”. Results of this exercise are schematically presented in Figure 1.1 below.

Figure 1.1: Prioritization of factors influencing quality improvements

50

Performance improvements would have been impossible without strong leadership of management teams who drive the process of quality improvement and establishment of teamwork culture. The findings suggest that the extent to which top management becomes directly involved in quality improvement activities determines the degree of quality improvement implementation. Management teams at CQI facilities proactively lead the quality improvement initiatives and demonstrated tangible results. Despite the widespread recognition of the importance of effective leadership among respondents, there are some considerable barriers to participation in leadership. Such barriers are noted extensively and include shortage of confidence and poor preparation for leadership/managerial roles. Deeper analysis is required to inform future strategies addressing these barriers to enhance the quality of management and leadership. The findings suggest that a “teamwork” culture, introduced by CQI pilot, provides an important foundation for implementing a quality improvement initiative. CQI facilities, by internalizing teamwork culture, show some quality improvement results compared to non-CQI facilities.

Availability of skilled health workforce—and building their capacity with training for continuous knowledge and skills development along with supportive supervision—are the most influential factors behind care quality improvements. Evidence suggests that with adequate staffing, tailored training, and regular supportive supervision, health care quality can improve even without additional funding. Evidence-based best practices are the foundation upon which successful quality improvement initiatives are built. Developing and integrating evidence-based best practices is not enough; healthcare organizations also need to have ways to measure how consistently best practices are being used and their impact on outcomes. External and internal supportive supervision system introduced in these health facilities have crucial roles in continuous quality improvement. On the other hand, systematic staff shortages undermined attainment of better results.

Monitoring of quality of care is an important and legitimate part of the government’s stewardship role enabling quality improvements at health facilities. Organizational processes to ensure quality and continuous improvement, such as supervision from PHEs and DHEs and within health organizations, is essential to promoting quality improvement at CQI facilities. According to key informants, the CQI pilot improved communication and cooperation between facility management and PHEs and DHEs facilitated adoption of a shared goal of improving facility performance. This has helped to promote a localized level of stewardship for service delivery and improved data capturing and analysis, and implementation of remedial actions.

Staff financial incentives are considered to have less impact on performance. Health workers interviewed ranked staff financial incentives as the lowest priority for improvement in performance. While they unanimously agree that higher staff remuneration is needed to incentivize staff, they believe that non-financial incentives are more powerful to influence staff motivation for CQI. Informants reported to be ‘awakened’ by improved working environments, routine supportive supervision, opportunities to improve knowledge and skills, changed relationships among staff and managers, more cohesive teams, and availability of essential drugs and equipment.

The core to sustained quality improvement lies in good leadership, teamwork, and availability of skilled labor. All respondents agree that adequate funding allowing regular maintenance of facilities, procurement of medical equipment, medicines and consumables is important, but without sound leadership, teamwork, and skilled staff, availability of equipment, medicines, and supplies cannot ensure quality improvement.

7.2 Barriers to changeAttainment of better results was hampered by external and internal factors detailed below

and schematically presented on Figure 7.2.

51

Figure 7.2: External and Internal factors negatively affecting effectiveness of CQI

External Barriers

General health system related bottlenecks challenged effectiveness of the CQI pilot. These include:

- Freeze on employment: The government-wide policy of freezing of posts in 2010, initiated by the MOFED, compounded the problem of the insufficient quantity of established posts. The freeze affected vacancy rates for doctors and Environmental Health Professionals negatively, in spite of the three episodes of the temporary unfreezing in 2011, 2012, and 2013. For nursing posts, vacancy rates improved as a result of the temporary unfreezing of posts.

- Staff to population ratio, staff shortage, high staff turn-over and high workload: Staff deployment to some facilities has not been consistent with the catchment population and staffing plans. The staffing situation in both the government-run facilities and faith-based health facilities is the result of the number of posts that the government funds and the availability of staff willing and able to take up the posts. As in other sectors, the number of funded – or ‘established’ – posts per health facility has been reduced in line with overall spending cuts, rather than increased in line with health service expansion or the upgrading of facilities – for example, of health centers to hospitals. In the absence of adequate staff, facility personnel are overstretched, and high workload often limits them to focus on the quality improvement. Frequent staff movement is another challenge faced by the system. Specifically, respondents reported gradually losing people knowledgeable in CQI and/or clinical practices at province, district, and facility levels that evidently affect continuity in quality improvement. Addressing structural factors affecting health human resource supply is definitely beyond the project reach as it requires multi-sectoral and bold government action. The challenge, by far, is not unique to Zimbabwe; most countries around the globe face similar problems,27 which has led the WHO to propose a global strategy on human resources for health28. Consequently, the evaluation team thinks, it would not be fair to expect RBF project to resolve this problem.

- Sub-optimal medical and nursing education system to produce professional health workforce and limited continuous medical education opportunities: Inadequate knowledge and skills of

27 Evans TG, Correia Araujo E, Herbst C, Pannenborg. Transforming Health Workers’ Education for Universal Health Coverage: Global Challenges and Recommendations. World Health & Population 17(3) September 2017: 70-80. 28 WHO 2016. Global strategy on human resources for health: workforce 2030

52

new graduates entering the system slows quality improvements. The latter is further aggravated by limited professional development and medical education opportunities enabling filling in the knowledge gap. To a large extent, access to such trainings is mostly granted by donor financed projects.

- Many facilities, including CQI facilities, have space shortages, which undermines quality of care. While health clinics were able to expand space availability through construction of new wings using RBF funding, not all hospitals could afford new construction.

- New monetary policy with observed delays in RBF funded performance payments along with the lowered costs of performance indicators impede quality improvement at CQI facilities. Since the transfer of the RBF subsidy payment function from Cordaid to the MOHCC, respondents in both groups of studied health facilities flagged problems related to delays in performance-based payments, as well as lower payments due to the recently introduced government monetary policy. Although the government adjusted the exchange rate, at the time of the study, PME facilities still awaited subsidies denominated in US$ and recalculated based on the intra-bank exchange rate. Another important impediment extensively discussed by respondents was the decreasing purchasing power of the subsidies due to observed delays in payments by the MOHCC and instability of commodity prices due to variation of exchange rate. These resulted in the absence of sufficient funds to ensure quality improvements, which adversely affected their performance for the next reporting period. The latter ultimately resulted in lower pay and further limited quality improvements.

Internal Barriers:

- Inadequate leadership from the central level in developing management structures, flow of resources, monitoring and evaluation, and providing adequate guidance to the CQI pilot challenged attainment of better results. Fragmentation of QA/QI function among different divisions and directorates of the MOHCC, a less empowered QAD due to insufficient staffing and lack of operational budget, impede effectiveness and efficiency of the QA/QI system. The country lacks an integrated health quality improvement program and plan that translates QA/QI strategy into action. Shortage of staff and absence of an operational budget restrict the QAD from adequate monitoring, developing/updating clinical guidelines, or providing training and regular support to QI structures at provincial, district, and facility levels.

- Fragmentation of QA/QI functions at the central level is mirrored at sub-national and local levels and reduces the effectiveness and efficiency of supervision. Supervision is another important planned intervention under CQI pilot. However, fragmentation of teams and QI committees overseeing quality of services within different government programs hinders regularity of supervision visits and decreases time spent at the facility for coaching and mentoring.

- Gaps in implementation of effective strategies to build health worker knowledge and skills obstructs CQI. Large numbers of health workers, a short time period, and financial restrictions forced the GoZ to opt for cascade training strategy to introduce CQI. This strategy undermined the quality of services delivered at the local levels. CQI training mostly relies on didactic teaching, with limited on-the-job follow-up and practical skills-building. However, didactic training does not effectively ensure the translation of theoretical knowledge into practice or address system-level barriers29, neither does the cascade training approach promote transfer of knowledge and skills. The given approach failed to ensure transfer of knowledge and development of skills in CQI and undermined achievement of better results. For example,

29 Pariyo GW, Gouws E, Bryce J, Burnham G, Uganda IMCI Impact Study Team. Improving facility-based care for sick children in Uganda: training is not enough. Health Policy Plan. 2005 Dec; 20 Suppl 1():i58-i68.

53

insufficient knowledge in data analysis and planning has been named as a challenge to quality improvement at CQI facilities.

- The quality of supervision has been also affected by deficient knowledge and hands-on experience of the supervisors to mentor and coach. Problem solving, clinical supervision, and community involvement were less commonly part of the quarterly supervision. Coaching and mentoring require skills that the project tried to develop through training. However, not all supervisors attained knowledge and skills required for quality supervision. In a few instances, respondents reported confusing or even useless advice from supervisors.

- Observed delays in RBF funded performance payments along with the lowered costs of performance indicators impeded quality improvement at CQI facilities from the start of the project. CQI and non-CQI facilities complained about delays in RBF subsidy payments experienced from the start of the project (when RBF payments were processed by Cordaid), which limited their ability to carry out planned quality improvement activities. These delays unfavorably affected their performance for the next quarter and resulted in lower pay for the next reporting period, consequently constraining further quality improvement. Other than late payments, respondents also mentioned lowered costs of performance indicators affecting volume of funds received by health facility - funds normally used for quality improvement and staff motivation purposes.

7.3 Role of CQI interventionsThe qualitative results indicate that the CQI facilities saw greater leadership, teamwork,

ownership of clinical quality improvement, engagement, and staff motivation. These could be important outcomes in and of themselves but may not necessarily translate into higher quality care. This reveals a major limitation of the CQI approach: some items are not within the power of the facility staff to solve through continuous quality improvement. For example, the CQI methodology cannot address the challenges with staff shortages, high staff turnover and high workload. These problems may greatly affect care processes but need to be addressed at the national or district levels. The quantitative results also show the extremely low performance of the PHCs on managing obstetric and newborn complications. Improved staff motivation and teamwork will not overcome challenges of lack of training or practice on these skills, rather it suggests that to improve quality, only hospitals should handle these complications.

These findings align with the conclusions of the recent Lancet Global Health Commission on High Quality Health Systems.30 The Commission argues that micro-level strategies to improve quality at the level of providers or facilities will likely be insufficient by themselves. Rather, macro and meso level strategies are likely needed at the national, provincial and district levels in order to improve overall health system quality. The large gaps observed in the quality of care cannot be addressed through targeted quality management when the root causes may be high workload, large turnover, inadequate clinical training, or simply providing a service at an inappropriate level of the system. Macro or meso level strategies to address these challenges would include addressing gaps in governance, reorganizing services to be provided at the appropriate level (i.e. moving deliveries to hospitals that can manage obstetric complications), strengthening pre-service clinical education, or establishing facility learning networks. While CQI may contribute to a better working environment and spur improvement in routine care practices that the staff are already well trained in, it should be seen as one tool in a broader health systems quality improvement strategy.

30 Kruk, M. E., Gage, A. D., Arsenault, C., Jordan, K., Leslie, H. H., Roder-DeWan, S., ... & English, M. (2018). High-quality health systems in the Sustainable Development Goals era: time for a revolution. The Lancet Global Health, 6(11), e1196-e1252.

54

7.4 Lessons LearnedThere are several key lessons learned that can be useful to any country who plans to introduce and/or enhance QA/QI system in the health sector.

Lesson 1: Optimal CQI benefits to overall quality of care are attained through a health systems approach that uses macro and meso level strategies in addition to CQI.

Lesson 2: Tangible improvements in quality of care are dependent on a streamlined, yet adequately staffed and well-budgeted CQI organizational structure at all levels, along with well-defined and enforced procedures.

Lesson 3: Supervision produces greater improvements. However, issues relating to frequency, intensity (only supervision vs. supervision and coaching), depth, duration and quality of mentoring/coaching (including capacity of mentors and coaches) must be addressed to optimize the return on the CQI investment.

Lesson 4: Development of mentoring and coaching capabilities at all levels of the health care system should be integral to the design of QA/QI initiatives. The selection of people with the right knowledge and skills for the mentor/coach role -and strategic plans to train and support them - are important to consider when initiating and especially when scaling up a CQI intervention.

55

8. Recommended Approach to CQI Scale-up and Institutionalization

Experience has demonstrated that technical interventions shown to be effective at a small scale under tightly controlled conditions cannot naturally be assumed to be successful at scale-up, despite intervention scale-up being essential for population-level impact. Scale-up is challenging, and not always successful31. CQI scale-up and institutionalization is an ongoing process where activities related to defining, measuring, and improving quality become formally and philosophically integrated into the structure and functioning of a healthcare organization and system. It is not a linear process, but rather a fluid one in which the essential elements may mature in sequence or in a less coordinated fashion. There is no one formula or set of steps an organization should follow to successfully institutionalize CQI. Nevertheless, considering Zimbabwe’s current context and factors that impeded attainment of the better results in CQI pilot sites, the process evaluation team suggests that the Government applies a two-prong CQI strategy should it wish to scale up CQI implementation: vertical scale-up followed by horizontal scale-up (see Figure 8.1).

Figure 8.1: Strategies for CQI pilot scale-up and institutionalization

The vertical scaling up strategy involves institutionalization of CQI through policy, legal, budgetary or other health systems change, whereas the horizontal scaling up strategy implies expansion or replication of the CQI approach in different geographic locations and/or healthcare facilities and/or roll-out of CQI pilot

to departments other than MNCH wards of the pilot facilities.

8.1 Essential elements for vertical CQI scale-up and institutionalizationMany factors affect a health system’s and organization’s ability to institutionalize CQI and a culture of quality, but there are key elements defined essential for implementing and sustaining the core quality improvement activities32 grouped into three main categories, depicted on Figure 8.2 and detailed below. Consequently, recommendations for the scale-up and institutionalization of the CQI are grouped per each category of essential elements for implementing and sustaining the core quality improvement activities.

The category “Enabling Environment” refers to the necessary conditions within the health system’s and organization’s internal environment, which benefit and facilitate the process of CQI institutionalization. The specific essential factors to be considered in this category are: a) institutional policies which are conducive and help guide the process; b) development of leadership which establishes priorities and guides the staff; c) availability of human, financial, and material resources for the implementation of CQI; and d) values that emphasize quality

31 Jeffrey Michael Smith et al, scaling up high-impact interventions: How is it done? International Journal of Gynecology and Obstetrics, 2015, 130, S4-S1032 ibid

56

and improvement. More specifically, for enhancement of the enabling environment, the MOHCC is advised to consider the following recommendations.

Figure 8.2: Essential elements for CQI institutionalization

Recommendation 1: Enforce integrated health Quality programming, planning, and implementation. The absence of the integrated quality improvement program along with fragmentation of CQI processes impede potential gains of CQI. Hence, it is strongly recommended that a cohesive health quality program and plan that integrates and harmonizes all quality programs of every

health service across vertical programs is developed and enforced.

Recommendation 2: Develop CQI operational manual and standard operating procedures. To ensure consistency in application of the CQI approach during the scale-up, the MOHCC should facilitate the development of the CQI operational manual and standard operating procedures (SOP) to guide national, sub-national, and health facility staff in CQI.

Recommendation 3: Enforce legislation supporting health facility autonomy. The PME revealed that health facilities with flexibility to plan for quality improvements along with access to funds at their disposal to be used as needed, contributed to improvement of quality. While everybody appreciates the benefits of autonomy, they also realize that autonomy is granted only for the RBF project and will be lost after the project closure. To ensure that quality improvements are sustained, the government is advised to enforce legislation supporting health facility autonomy.

Recommendation 4: Streamline and strengthen CQI governance and leadership at national, provincial and district levels to drive change. Streamlining and strengthening CQI governance at national level should be given a priority before the scale-up. To ensure adequate leadership of the CQI scale-up and institutionalization, it is recommended that the MOHCC: i) Revisits placement of the QA Directorate in the organizational structure of the MOHCC: It is highly recommended that QAD is elevated to the level of division to report to the Permanent Secretary. Right placement of the QA/QI function will give more power and leverage to administer stewardship function; ii) Considers expanding the staffing of the QA/QI division (currently Quality Assurance Department) to allow provision of needed guidance and oversight to central and local levels; and iii) Allocates sufficient resources for effective operation of the QAD. Streamlining and strengthening governance is also required at provincial, district, and facility levels. It is believed that health organizations equipped with CQI department or dedicated staff responsible for CQI would find it much easier to conduct CQI activities.

Recommendation 6: Advancement of supportive supervision mechanisms. The effectiveness of the CQI scale-up process will largely depend on the quality of supervision, which incorporates mentoring/coaching. Enhancement of mentoring and coaching skills of supervisors can transform traditional supervision into a more effective intervention to improve care quality. Furthermore, effective delivery of mentoring and coaching can potentially fill the knowledge gap faced by the health organizations and service providers at the initial stage of CQI scale-up.

57

Recommendation 7: Build requisite clinical and competitive capabilities. The gap in implementation of effective strategies to build staff skills and knowledge, identified by PME, reflect a need to define and invest in effective approaches to better train and support health workers to deliver quality care. The cascade model, a mechanism delivering training messages from trainers at the central level to trainees at the local level through several layers, is largely used for in-service training, as it can deliver many trained health workers quickly and economically33. However, despite these advantages, this model is often criticized for its ineffectiveness. Its main weakness is the distortion of the messages transferred during the training, because they are passed down through many different levels of personnel. The intended messages are often altered, and their effects are diluted through miscommunication and different interpretations of the same messages34,35,36. Therefore, where possible and resources permit, opportunities for more effective training modalities should be explored by the GoZ.

To ensure that health workforce capacity building efforts are sustained, the GoZ is advised to incorporate these trainings into the pre- and post-service and continuous medical education programs. It is also important to upskill trainers so that they can teach quality improvement methods robustly. More specifically, the PME recommends: i) Integrating CQI principles and tools in the pre- and post-service training programs to generate a cadre of health workers equipped with basic knowledge of CQI; ii) Refining the training syllabus developed as part of the CQI pilot to balance theoretical knowledge with practical skills development and assign a clinical base where trainees can practice acquired knowledge and practical skills; and iii) Integrating CQI training into the continuous medical education program as a mandatory module for all health workers. Capacity building should include, but not be limited to only, CQI principles and tools, clinical training, as well as training of health administrators (particularly at health facility level) in basic management and strategic planning. Particular emphasis should be placed on developing specific training programs for incapacitating supervisors at central, provincial, district, and facility levels in mentoring and coaching.

Recommendation 8: Enhance knowledge exchange. In support of CQI scale-up and institutionalization, develop and institutionalize effective knowledge exchange platforms and encourage collection and dissemination of best practices. Capitalize on already available best practices and institutionalize system-wide.

Recommendation 9: Develop appropriate and affordable incentive structures and mechanisms. The development of incentive structures for the health facilities and within individual health-care institutions is important to influence organizational strategy and individual decision making. Therefore, the MOHCC is advised to develop incentive structures, which, in addition to financial incentives, will also consider modalities of non-financial staff motivation.

Recommendation 10: Establish a national evaluation, quality monitoring, measurement, and reporting system. The country should have a well-established QA/QI evaluation, monitoring, and measurement system to inform on the strength and weakness of the current system. Evaluation refers to the formal way in which information is gathered, interpreted, and evaluated in relation to the set standards and criteria and requires monitoring and measurement. Thus, the MOHCC is advised to develop: i) a QA/QI monitoring framework that is adequately reflected in the National Health Sector Strategy M&E Plan; ii) monitoring instruments; iii) people to ensure collection of the data that shows a true reflection of the

33 L. A.Dove, Teachers and Teacher Education in Developing Countries, New Hampshire: Croom Helm, 1986. 34 Takako SUZUKI, The Effectiveness of the Cascade Model for In-service Teacher Training in Nepal, Graduate School of International Cooperation Studies, Kobe University 2-1 Rokko-dai, Nada-ku, Kobe, 657-8501 Japan35 Department of Education and Science, A critique of the implementation of the cascade model used to provide inset for teachers in preparation for the introduction of the general certificate of secondary education, Stanmore, Middlesex, 198836 J.Mezirow, Transformative Dimensions of Adult Learning, San Francisco, Jossey-Bass Publishers, 1991

58

reality in terms of the performance being monitored ; iv) supporting system in the form of technology and information system, as well as the manpower to enable the transmission of the collected data in the most logical and workable manner, as well as the analysis of the data in the most objective manner to prevent any distortion of the data, complying with all the principles of trustworthiness; and v) a system which enables adequate reporting and feedback of the results to the health facilities and practitioners concerned.

8.2 Recommendations for horizontal CQI scale-up Should the Government decide to horizontally expand CQI to additional facilities, it is recommended to use a phased approach.

Phase 1: Integration of MNCH, EPI, HIV, TB CQI approaches in pilot facilities. State-level programs have operational QI components in the studied health facilities, albeit in silos. It is advised to: i) integrate the QI components of all state programs into one “extended” QI component; ii) develop an extended list of indicators; iii) harmonize monitoring tools; iv) provide training of staff at provincial, district, and facility levels; and v) monitor performance of facilities.

Phase 2: Phased scale-up of integrated (MNCH, EPI, HIV, TB) CQI approaches to all facilities in the pilot provinces. When institutional policies and regulatory mechanisms are developed so that quality assurance mechanisms are integrated into the health system’s normal operations at all levels, the MOHCC can consider scaling up integrated CQI to all facilities in original pilot districts.

Phase 3: Phased scale-up of integrated (MNCH, EPI, HIV, TB) CQI approaches to other provinces. When integrated CQI is formally integrated into the structure and functions of the health care system and health organizations, the model can be rolled out to other provinces.

Phase 4: Expansion of the CQI to other government priority programs and phased scale -up to all facilities in the pilot provinces. At the last instance, the CQI approach can be extended to other government priorities in the health sector, e.g., non-communicable diseases, oncology, surgery, etc.

To make the scale-up/expansion process smooth, the GoZ is encouraged to adhere to the steps detailed below.

Table 8.1: Steps in the CQI scaling up process37

Action DescriptionStep1: Develop a scaling up plan: outline a vision of scale-up and a compelling case for action1.1 Rationale for scale-up

Draw up a rationale for scaling up from the information in Step 1, noting that further investigation and analysis may be necessary to provide a compelling case for action.

1.2 Description of the intervention

Describe ‘what’ will be scaled up and where possible the original intervention should be simplified and streamlined.

1.3 Situation and stakeholder analysis

Map the social, political, and organizational environment(s) in which the intervention will be scaled up and identify potential barriers and enablers to scale-up.

1.4 Role and function of stakeholders in the scale-up process

Consider who might perform key functions when the intervention is scaled up by mapping key functions and matching them to those who will potentially be involved.

1.5 Approach to scale up

There are two main scale-up approaches. A vertical approach involves the introduction of an intervention simultaneously across a whole system and results in institutional change through policy, regulation, financing, or health

37 Adapted from Andrew J Milat et al, A guide to scaling up population health interventions, 2016, Public Health Research and Practice, January 2016; Vol. 26(1):e2611604

59

systems change. A horizontal approach involves the introduction of an intervention across different sites or groups in a phased manner. These approaches are not mutually exclusive, and a combination of approaches can be used.

1.6 Monitoring and Evaluation framework of the intervention scale-up

Determine what variables are important to measure over time and determine feasibility and associated cost of these systems.

1.7 Resources required for scale-up

Estimate the human, technical, and financial resources needed to scale up the intervention.

1.8 Scale up plan The plan should present a clear and concise case for scaling up the intervention, as well as an overview of how this will be brought about, including a vision of what scaling up will look like if successfully completed.

Step 2. Prepare for scale-up: secure resources and build a foundation of legitimacy for the scaling up plan2.1 Consult with stakeholders

Assess the appropriateness and acceptability of the intervention and the scaling up plan and use this information to design advocacy and communication strategies.

2.2 Legitimize change

Gain the support of decision makers who must be convinced that scaling up the intervention is: a credible and superior solution to a pressing problem; for a priority population; and is affordable.

2.3 Build a constituency

Mobilize the broader community required to successfully scale up an intervention.

2.4 Realign and mobilize resources

Mobilize financial resources through existing channels or through new funding streams. Ensure that resources are directed to address skill and other capacity deficits in delivery organizations.

Step 3. Scale up the intervention: implement the scale-up plan, making necessary adjustments based on performance data3.1 Modify and strengthen organizations

Manage organizational change through processes such as staff re-training, mentoring, leadership development and coaching

3.2 Coordinate action and governance

Develop and implement concrete and detailed agreements about how, when, where and by whom resources are to be used, and the governance structures that will be used to identify issues and resolve any disputes that may arise.

3.3 Monitor performance and efficiency

Develop systems that have an ongoing focus on measuring effectiveness, reach, fidelity, acceptability, and costs, with a particular focus on the efficiency of intervention delivery.

3.4 Ensure sustainability

Implement organizational and cultural changes to institutionalize an intervention so that it becomes part of routine practice.

60

Annex 1: RBF Package of quantity services

61

District Hospital

1. Normal deliveries in district hospital2. Deliveries with complications (caesareans excluded) and postpartum

complications3. Caesareans performed4. Family planning: Tuba Ligations5. Counter referral note arrives at RHC

Rural Health Center1. OPD new consultations2. First ANC visit during the first 16 weeks of pregnancy (October 2012)3. Ante natal care 4 visits completed4. Post-natal care 2 or more 5. Normal deliveries6. HIV VCT in ANC7. Syphilis RPR test8. IPT (x2 doses)9. Tetanus TT2+10. ARVs to HIV+ preg. Women (PMTCT)11. Family planning short and long term methods12. High risk perinatal referrals13. Vitamin A supplementation14. Children fully immunized15. Growth monitoring, children < 5yrs

Annex 2: Exploratory Research Questions

The qualitative research in the Phase 2 focused on gaining insights into different performance observed between hospitals and Health Centers at intervention sites and between intervention and non-intervention sites. More specifically, the PME attempts to answer the following questions:

1. Consistency of potential gains and staff perceptions that received CQI intervention:1.1. What were the general perceived benefits of the CQI intervention?1.2. Are these potential gains consistent with the qualitative perceptions of the PHC staff

that received the CQI intervention?1.3. Have the staff noticed more organized care/more attention to detail/more feedback

and supervision related to these issues? 1.4. What were the perceived costs/detriments if any? 1.5. How has the provision of care changed under the CQI intervention? 1.6. What are the constraints that prevent further improvement in the quality of care? 1.7. In comparison facilities , has there been any attempt in the previous two years to

improve the clinical quality of care and how? 1.8. What are the constraints that prevent further improvement in the quality of care?

2. Structural quality2.1. What are some of the reasons that account for the mixed results on structural

quality? 2.2. What are the main structural quality changes that health facility teams ascribe to the

CQI (qualitatively)? 2.3. What implementation or design changes do health facility teams recommend to

further enhance the CQI’s effects on structural quality? 2.4. For non-CQI facilities (PHC and hospital levels), has there been any attempt in the

previous two years to improve the structure of care? 2.5. What are the constraints that prevent further improvement in the quality of care?

3. Health Facility Team Organization and Management: 3.1. How did teams in CQI facilities organize themselves differently to deliver better

quality services? 3.2. What worked under the new team set up/organization, if any? 3.3. What would CQI facility teams do differently to achieve better results in the future?

4. Mentorship/coaching: 4.1. Extent to which coaching and mentoring influenced attainment of better results? 4.2. What are the main differences between CQI facilities and non-CQI facilities in terms

of access to mentorship and coaching? 4.3. How do health facility teams in CQI facilities perceive the nature and quality of this

mentorship by supervisors? 4.4. How do district and provincial managers perceive the nature and quality of the

mentorship? 4.5. What could be done differently from a health systems management perspective

going forward?

5. Health Systems Management: 5.1. How do field level CQI mechanisms and processes (from Province to facility level)

influence management decisions related to health system elements (refer across the pillars)?

62

5.2. What CQI related factors play a dominant role in determining decision making by district and provincial managers and follow up on those decisions at health facility level?

5.3. What are the dynamics between Health Facility CQI Teams, Facility Management and District/Province supervisors in the process of facilitating changes at facilities?

6. Staff capacity building and motivation: 6.1. How well and in what way is attained knowledge and skill to assess quality and

improve clinical processes internally transferred amongst staff (from the nuclei- CQI Teams to other staff and departments)?

6.2. To what extent staff capacity building and motivation played a role in improvement or not improving the care quality?

6.3. Extent to which RBF performance payments influenced care quality improvement and how?

7. Factors influencing care quality improvement: 7.1. What were the factors that supported or inhibited improvement of care quality across

system, process and outcome levels (external and internal)?7.2. What influenced better performance of non-CQI facilities across all three domains

(where applicable)?7.3. Extent to which CQI content and delivery modes were appropriate to achieve care

quality improvement at CQI facilities?

8. Scale -up: What are the critical elements to be considered for the scale-up?

63

Annex 3: Additional tables & figures

Table A1. Organization of individual PHC outcomes into aggregate measures and balance in 2016Summary measure Indicator CQI Control

T test p-value N CQI

N control

General infrastructure

Visible sign post when arriving at the clinic 92% 96% 0.52 24 201Fence/wall in good condition and gate with clearly written emergency contact num 42% 58% 0.14 24 201External appropriate wall finishing 100% 99% 0.08 24 201Roof intact, presence of well-maintained rain gutters 54% 64% 0.37 24 201Minimum of 3 toilets 100% 96% 0.00 24 201Hand washing facility with soap available near the toilets 75% 50% 0.02 24 201Electricity for 24 hours a day, and 7 days a week 63% 83% 0.07 24 201Fire extinguishers available, accessible, functional and serviced AND a clearly 29% 25% 0.69 24 230Appropriate drainage of waste water 79% 88% 0.35 24 201Shower with either running water or container of at least 100 litres 75% 81% 0.56 24 201Waiting area with adequate ventilation of waiting area 96% 96% 0.95 24 205

Cleanliness Ground clean with no litters and/or stagnant water and the grass cut 83% 72% 0.17 24 201No waste and dangerous objects in courtyard such as needles – syringes –gloves – 96% 99% 0.54 24 201Recently clean without visible fecal material and without smell 83% 90% 0.41 24 201Bin with lid accessible to clients 88% 85% 0.69 24 201Building clean and with good ventilation without bad smell? 100% 97% 0.01 24 205Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) 79% 80% 0.88 24 205

Safety Bin with plastic liners + sharps containers available and Bin not more than ¾ fu 75% 80% 0.64 24 205Hand washing facilities with soap available at accessible points for outpatient 96% 76% 0.00 24 197Safe drinking water is accessible 92% 80% 0.07 24 197Ottoway pit or rubbish pit and incinerator 58% 51% 0.52 24 230

Privacy Doors with locks and closing properly 96% 95% 0.79 24 205Curtains on windows or non- transparent glass and screen 96% 95% 0.87 24 205Sufficient space between beds (at least 1m between beds) 96% 95% 0.84 24 197

General equipment

Radio or mobile phone with airtime or landline for communication 92% 91% 0.92 24 201Transport plan for emergency referrals and/or contingency plan

83% 85% 0.83 24 201

64

Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 88% 84% 0.66 24 197Mosquito nets available and in good state? 96% 64% 0.00 24 197Adult Weighing Scale and standard pediatric scale available 96% 96% 0.96 24 204BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 54% 54% 0.98 24 204

Pharmacy management

Important Accessories: cannula, giving sets, syringes and needles, drip stand, s 67% 69% 0.81 24 204Monthly physical counts conducted with min, max and emergency order levels recor 67% 66% 0.91 24 203The physical stock level corresponds with that on the stock card: supply on stoc 83% 85% 0.82 24 203Staff completes and send MIS forms to district each month 75% 58% 0.09 24 203Stored correctly in a locked secured storeroom 100% 96% 0.00 24 203Clean place, well ventilated with cupboards, labelled shelves, no incident light 92% 81% 0.11 24 203Medicines stored in alphabetical order also observing the First Expiry First Out 88% 62% 0.00 24 203Expired Medicine Register available 42% 52% 0.34 24 203No expired products in stock: supervisor verifies randomly 3 medicines and 2 con 79% 95% 0.08 24 203Lockable trolleys available with working lock 88% 88% 0.93 24 203

Pharmacy stock

Doxycycline capsules 100mg 88% 80% 0.34 24 203Ciprofloxacin tablets 500mg 38% 27% 0.33 24 203Metronidazole tablets 200mg Oral 71% 73% 0.84 24 203Diazepam injection 5mg/ml 63% 55% 0.50 24 203Benzathine Penicillin injection 33% 28% 0.61 24 203Benzyl Penicillin 88% 88% 0.98 24 203Amoxycillin suspension 125mg/5ml (dispersible tablets) 75% 81% 0.51 24 203Ferrous sulphate tablets/Folic Acid 67% 82% 0.14 24 203Zinc Sulphate tabs 83% 87% 0.68 24 203Paracetamol 500mg tablets 92% 97% 0.42 24 203Paracetamol syrup or dispersible tablets 92% 86% 0.39 24 203Dispensing envelopes 79% 72% 0.43 24 203Latex gloves 96% 97% 0.78 24 203Oxytocin 10IU/ml Injection, 1ml Ampoule. 88% 86% 0.86 24 203Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 96% 97% 0.78 24 203Gentamycin 40mg/ml Injection, 2ml Ampoule 88% 72% 0.05 24 203Artemether 20mg + Lumefantrine 120mg Tablets, 24's 88% 95% 0.29 24 203Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial 100% 98% 0.02 24 203Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 83% 94% 0.19 24 203Lignocaine 96% 88% 0.11 24 203

65

PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami 71% 87% 0.11 24 203Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 100% 100% 0.32 24 203Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( 96% 94% 0.62 24 203Emergency tray with all the necessary un expired medicines available 38% 44% 0.57 24 204Un expired ringer lactate, 5all dextrose, normal saline available 42% 63% 0.06 24 204

Guidelines available

Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of 88% 90% 0.76 24 204PEP policy and guidelines: Available in OPD: posted on the wall and up to date 83% 89% 0.47 24 204Opportunistic Infection and ART guidelines: Available, accessible in all consult 92% 96% 0.51 24 204STI Management protocol: Displayed in all consultation rooms and up to date 96% 89% 0.17 24 204IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 92% 82% 0.13 24 204Focused ANC protocol: Displayed I all consultation rooms an up to date 92% 96% 0.46 24 204

Vaccine management

Surveillance line listing and case definitions displayed 68% 73% 0.69 19 178Updated EPI schedule, and a contingency plan displayed 74% 81% 0.48 19 178EPI graphs showing trends displayed and staff member is able to interpret the g 79% 79% 0.98 19 178EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI 100% 84% 0.00 19 178Fridge with a temperature booklet available and filled twice a day 89% 94% 0.52 19 178The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S 84% 79% 0.55 19 178The following Antigens are available: BCG, MR (measles and Rubella), polio, Pen 79% 92% 0.19 19 178The physical stock and the amount in the stock cards match ( Supervisor verifies 63% 76% 0.30 19 178Vaccines correctly stored in fridge with compartments as follows in fridges with 95% 96% 0.89 19 178No expired vaccines 89% 97% 0.34 19 178The Vaccine Vial Monitor (VVM) status is kept 95% 94% 0.95 19 178There are readable labels on vials with matching diluents 79% 95% 0.12 19 178The number of syringes available matches the number of vaccines in the stock car 89% 92% 0.78 19 178Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 100% 98% 0.05 19 178Vaccine carriers, cold box, gas regulator, gas cylinder and scissors 100% 95% 0.00 19 178AEFI investigation forms, case investigation forms for 89% 83% 0.42 19 178

66

EPI targeted diseases anVaccine order forms and stock cards available 89% 94% 0.57 19 178

Maternity supplies

IV fluids and giving sets 74% 68% 0.55 23 168Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and 57% 33% 0.04 23 168Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 91% 85% 0.36 23 168PPH kits available and complete 70% 51% 0.09 23 168Eclampsia kit available and complete 48% 47% 0.94 23 168Uterotonic Medicines stored at correct temperature available in delivery room or 87% 87% 0.99 23 168Fetoscope, baby blanket Baby scale and tape measure 91% 83% 0.21 23 168Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 83% 68% 0.11 23 168Penguin suction, neonatal bag and mask and suction tube in delivery room 78% 91% 0.17 23 168At least 2 obstetric sterilized delivery standard packs 74% 76% 0.87 23 168At least 5 pairs of sterile gloves should be available 96% 86% 0.06 23 168

HMIS Standard referral forms (at least 10) and register available and properly filled 87% 77% 0.19 23 209A referral feedback documented in the register for every referral made and/or re 52% 53% 0.96 21 183The following T Series forms available and fully completed: T1, T2, T3, T5, T6, 87% 82% 0.51 23 209T5 completed and sent timely (by the 7th of the following month) for previous mo 83% 94% 0.17 23 209All selected registers complete and correct 39% 34% 0.67 23 209T5 correct according to HMIS 48% 34% 0.24 23 209

OPD triaging knowledge

Consultations being done by appropriately qualified staff: PCN and/or RGN 100% 86% 0.00 24 204Triage: Emergency signs requiring immediate attention 83% 90% 0.70 6 82Triage: Priority signs (requiring priority in the queue) 67% 89% 0.34 6 82Triage: Non-urgent cases 67% 90% 0.32 6 82

TB screening knowledge

TB sign: Weight loss 100% 95% 0.00 19 182TB sign: Fever for more than 3 weeks 95% 85% 0.12 19 182TB sign: Cough for more than 14 days 100% 98% 0.05 19 182TB sign: Night sweats 95% 95% 0.97 19 182TB sign: TB contact exposure. 89% 86% 0.68 19 182

ANC process quality

all of first visit ANC bookings who had documented: BP, Height, Weight measurement 79% 65% 0.17 19 217all of first visit ANC bookings who received the standard laboratory test accord 37% 42% 0.67 19 217all of first ANC visits who received TT vaccine 100% 94% 0.00 19 217all of first ANC visits who received iron supplementation 100% 90% 0.00 19 217all of first visit ANC who had documented pregnancy test results 11% 12% 0.90 19 217

67

PNC process quality

all PNC visits with assessment: General condition of the infant, NAD recorded if 58% 76% 0.15 19 217all PNC visits with assessment: General condition ,Pulse rate, B/P and temperature 32% 37% 0.62 19 217all PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 95% 96% 0.77 19 217all women post-partum counselled and offered any of the modern FP method at fol 63% 72% 0.47 19 217all facility births seen for day 3 PNC visit in any one month in the last quarte 94% 74% 0.01 16 203

PMTCT process quality

all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 59% 58% 0.98 17 212all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b 59% 67% 0.55 17 212all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o 29% 23% 0.57 17 212all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 35% 5% 0.02 17 212

Sick child assessment

all children treated as outpatient for pneumonia in any one of month in last qu 65% 38% 0.05 17 225all children with diarrhoea correctly assessed for signs of dehydration), persi 71% 48% 0.07 17 225all children diagnosed with malaria that have RDT + or laboratory confirmation i 35% 29% 0.61 17 225

Sick child treatment

all children correctly treated as an outpatient for pneumonia in any one of mont 59% 59% 0.99 17 225all children correctly treated as an outpatient (ambulatory) for diarrhoea in an 47% 64% 0.20 17 225all Children with uncomplicated malaria correctly treated according to national 35% 24% 0.37 17 225all Children with severe malaria correctly treated according to national guideli 0% 4% 0.00 17 225all of 6-59 months old children with un complicated severe acute malnutrition (S 18% 24% 0.53 17 225

Maternity process quality

all women who delivered in the facility monitored using partographs as per crite 19% 14% 0.67 16 203all deliveries performed by skilled personnel in any one month in the last quart 100% 92% 0.00 16 203all total births in any one month in the last quarter documenting administration 88% 66% 0.03 16 203all births with placental status documented at birth in any one month in the las 100% 93% 0.00 16 203all newborns BF within one hour of birth in any one month in the last quarter 69% 75% 0.60 16 203all newborns received Vitamin K in the any one month in the last quarter 63% 40% 0.10 16 203all newborns received eye care (Tetracycline) in the any one month in the las 81% 86% 0.67 16 203all newborns received first vaccination (BCG) in the any one month in the last q

50% 71% 0.13 16 203

68

all women monitored in early post-partum period (4th stage) per guideline (birth 63% 24% 0.01 16 203all newborns monitored in early post-partum period per guideline (birth to disch 44% 25% 0.18 16 203

Obstetric complications

all women with prolonged labour in last quarter referred to higher level 5% 11% 0.33 19 211all women with prolonged labor or Rupture of Membranes and without chorioamnioni 0% 1% 0.16 19 211all women with PPH managed per guideline in the last quarter 0% 6% 0.00 19 211all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch 0% 0% 19 211all pregnant women with severe pre-eclampsia and/or eclampsia managed according 5% 0% 0.33 19 211all of neonates who had an APGAR score of ≤5 in the first minute after birth for 0% 0% 19 211all neonates with possible serious bacterial sepsis managed per standard in last 5% 3% 0.66 19 211

Table A2. Balance between treated and control PHCs on covariates

CQI ControlP-value of T test N CQI

N control

Percent Clinic (in comparison to RHC) 46% 70% 24 230Distance from district hospital 41 41 0.94 24 2062016 Total New Patients 4068 4480 0.61 24 230

Table A3. Unadjusted PHC summary ANCOVA results on input quality measures CQI Baseline N Coef. p-val Coef. p-val Inputs

General infrastructure: % of 11 items 0.07 0.11 0.31 0.00 254Cleanliness: % of 6 items 0.03 0.35 0.16 0.16 235Safety: % of 4 items 0.04 0.29 0.26 0.00 254Privacy: % of 3 items 0.01 0.63 0.07 0.08 229General equipment: % of 7 items 0.04 0.46 0.02 0.81 238Pharmacy management: % of 9 items 0.02 0.54 0.32 0.00 226Pharmacy stock: % of 25 items -0.03 0.33 0.12 0.02 237Availability of guidelines: % of 6 items -0.01 0.79 0.16 0.09 228Vaccine management: % of 17 items -0.02 0.65 0.15 0.09 197Maternity supplies: % of 11 items 0.05 0.29 0.03 0.57 190HMIS: % of 6 items 0.01 0.93 0.20 0.00 231OPD triaging knowledge: % of 4 items -0.04 0.33 0.02 0.19 231TB screening knowledge: % of 5 items -0.01 0.65 0.08 0.18 201

Table A4. Adjusted PHC summary ANCOVA results on input quality measures CQI Baseline N Coef. p-val Coef. p-val

69

Inputs General infrastructure: % of 11 items 0.08 0.10 0.25 0.03 114Cleanliness: % of 6 items 0.05 0.23 0.34 0.09 106Safety: % of 4 items 0.04 0.42 0.19 0.11 114Privacy: % of 3 items 0.04 0.20 0.25 0.02 103General equipment: % of 7 items 0.04 0.50 -0.01 0.96 106Pharmacy management: % of 9 items 0.02 0.67 0.32 0.02 102Pharmacy stock: % of 25 items -0.01 0.65 0.22 0.00 106Availability of guidelines: % of 6 items -0.01 0.63 0.22 0.04 105Vaccine management: % of 17 items 0.00 0.97 0.10 0.39 92Maternity supplies: % of 11 items 0.06 0.32 0.02 0.79 91HMIS: % of 6 items -0.01 0.88 0.26 0.03 102OPD triaging knowledge: % of 4 items -0.07 0.20 0.05 0.18 105TB screening knowledge: % of 5 items 0.01 0.62 0.02 0.83 89

Table A5. Adjusted PHC summary ANCOVA results using Q4 2018 as outcome CQI Baseline N Coef. p-val Coef. p-val General infrastructure: % of 11 items -0.03 0.57 0.06 0.53 114Cleanliness: % of 6 items -0.03 0.57 0.26 0.09 106Safety: % of 4 items -0.02 0.71 0.05 0.34 114Privacy: % of 3 items -0.03 0.43 0.14 0.33 97General equipment: % of 7 items 0.01 0.82 -0.09 0.47 106Pharmacy management: % of 9 items -0.03 0.66 0.32 0.14 96Pharmacy stock: % of 25 items -0.06 0.18 0.06 0.48 102Availability of guidelines: % of 6 items -0.01 0.78 0.18 0.10 98Vaccine management: % of 17 items -0.08 0.27 -0.07 0.50 91Maternity supplies: % of 11 items 0.05 0.26 0.07 0.59 86HMIS: % of 6 items 0.06 0.21 0.43 0.00 95OPD triaging knowledge: % of 4 items -0.02 0.71 0.07 0.04 98TB screening knowledge: % of 5 items 0.03 0.46 0.02 0.88 84ANC process quality: % of 5 items -0.03 0.53 0.11 0.29 96PNC process quality: % of 5 items 0.06 0.33 0.32 0.02 105PMTCT process quality: % of 4 items -0.01 0.98 -0.06 0.72 101Sick child assessment quality: % of 3 items 0.01 0.87 0.38 0.00 90Sick child treatment quality: % of 5 items 0.11 0.37 0.25 0.15 90Maternal process quality: % of 10 items 0.03 0.46 -0.01 0.83 93Obstetric complications quality: % of 7 items -0.02 0.45 0.04 0.75 79

70

Table A6. Adjusted PHC summary difference in differences results DnD estimator Post N Coef. p-val Coef. p-val Input measures

General infrastructure: % of 11 items 0.01 0.87 0.04 0.48 182Cleanliness: % of 6 items 0.03 0.41 -0.02 0.59 176Safety: % of 4 items -0.09 0.33 0.10 0.26 182Privacy: % of 3 items 0.06 0.09 -0.04 0.06 173General equipment: % of 7 items 0.00 0.95 -0.05 0.27 176Pharmacy management: % of 9 items 0.03 0.66 0.03 0.35 174Pharmacy stock: % of 25 items 0.01 0.79 0.03 0.29 176Availability of guidelines: % of 6 items -0.06 0.21 0.06 0.01 175Vaccine management: % of 17 items 0.03 0.58 -0.01 0.81 167Maternity supplies: % of 11 items -0.02 0.83 -0.01 0.92 168HMIS: % of 6 items -0.07 0.32 0.06 0.03 171OPD triaging knowledge: % of 4 items -0.11 0.20 0.03 0.58 175TB screening knowledge: % of 5 items 0.00 1.00 0.01 0.82 162

Process measuresANC process quality: % of 5 items -0.08 0.30 0.12 0.00 174PNC process quality: % of 5 items 0.15 0.02 -0.01 0.84 177PMTCT process quality: % of 4 items -0.10 0.20 0.17 0.01 174Sick child assessment quality: % of 3 items -0.15 0.05 0.22 0.00 174Sick child treatment quality: % of 5 items 0.12 0.16 0.08 0.24 174Maternal process quality: % of 10 items 0.03 0.73 0.16 0.01 168Obstetric complications quality: % of 7 items -0.01 0.73 0.00 0.89 174

Table A7. Adjusted PHC ANCOVA results for all facility outcomes CQI Baseline N Coef. p-val Coef. p-val

Visible sign post when arriving at the clinic 0.02 0.71 0.13 0.07 100Fence/wall in good condition and gate with clearly written emergency contact num 0.08 0.49 -0.20 0.05 100External appropriate wall finishing 0.00 0.97 -0.05 0.14 84Roof intact, presence of well-maintained rain gutters 0.11 0.34 0.14 0.34 100Minimum of 3 toilets 0.04 0.24 0.66 0.00 100Hand washing facility with soap available near the toilets 0.10 0.08 -0.02 0.57 100Electricity for 24 hours a day, and 7 days a week 0.07 0.37 0.19 0.01 100Fire extinguishers available, accessible, functional and serviced AND a clearly 0.09 0.27 0.04 0.62 114Appropriate drainage of waste water 0.03 0.51 0.24 0.00 100Shower with either running water or container of at least 100 litres 0.02 0.80 0.18 0.01 100Waiting area with adequate ventilation of waiting area 0.01 0.80 -0.04 0.10 103Ground clean with no litters and/or stagnant water and the grass cut 0.02 0.66 0.13 0.10 100No waste and dangerous objects in courtyard such as needles – syringes –gloves – 0.08 0.06 0.39 0.02 100Recently clean without visible fecal material and without smell 0.02 0.73 0.02 0.84 100Bin with lid accessible to clients 0.07 0.15 0.03 0.60 100

71

Building clean and with good ventilation without bad smell? 0.02 0.02 0.07 0.42 103Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) -0.06 0.44 0.18 0.06 103Bin with plastic liners + sharps containers available and Bin not more than ¾ fu -0.01 0.85 0.04 0.59 103Hand washing facilities with soap available at accessible points for outpatient -0.05 0.51 -0.05 0.18 101Safe drinking water is accessible -0.10 0.29 -0.02 0.68 99Ottoway pit or rubbish pit and incinerator 0.13 0.20 0.24 0.02 114Doors with locks and closing properly -0.02 0.39 0.26 0.05 103Curtains on windows or non- transparent glass and screen 0.02 0.27 -0.03 0.05 103Sufficient space between beds (at least 1m between beds) 0.09 0.26 0.34 0.13 99Radio or mobile phone with airtime or landline for communication 0.02 0.74 0.23 0.09 100Transport plan for emergency referrals and/or contingency plan 0.01 0.94 0.14 0.01 100Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 0.02 0.77 0.07 0.32 99Mosquito nets available and in good state? 0.00 0.98 0.18 0.05 99Adult Weighing Scale and standard pediatric scale available 0.00 0.87 -0.02 0.03 105BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 0.17 0.14 -0.06 0.39 105Important Accessories: cannula, giving sets, syringes and needles, drip stand, s -0.01 0.91 0.02 0.82 105Monthly physical counts conducted with min, max and emergency order levels recor 0.07 0.46 0.04 0.61 102The physical stock level corresponds with that on the stock card: supply on stoc -0.09 0.20 -0.01 0.89 102Staff completes and send MIS forms to district each month 0.03 0.77 0.34 0.01 102Stored correctly in a locked secured storeroom 0.02 0.15 0.17 0.01 102Clean place, well ventilated with cupboards, labelled shelves, no incident light 0.00 1.00 0.43 0.01 102Medicines stored in alphabetical order also observing the First Expiry First Out -0.01 0.85 0.13 0.25 102Expired Medicine Register available -0.02 0.81 0.15 0.06 90No expired products in stock: supervisor verifies randomly 3 medicines and 2 con -0.08 0.26 -0.12 0.04 102Lockable trolleys available with working lock 0.05 0.33 -0.08 0.16 102Doxycycline capsules 100mg 0.01 0.76 -0.02 0.63 102Ciprofloxacin tablets 500mg 0.00 0.97 0.17 0.00 102Metronidazole tablets 200mg Oral 0.01 0.79 0.04 0.17 102Diazepam injection 5mg/ml -0.09 0.34 0.12 0.22 102Benzathine Penicillin injection 0.01 0.91 -0.05 0.61 90Benzyl Penicillin -0.05 0.44 0.13 0.46 102Amoxycillin suspension 125mg/5ml (dispersible tablets) -0.02 0.72 0.02 0.67 102Ferrous sulphate tablets/Folic Acid -0.02 0.12 -0.02 0.01 102Zinc Sulphate tabs -0.01 0.89 -0.05 0.10 102Paracetamol 500mg tablets -0.11 0.29 0.04 0.73 102Paracetamol syrup or dispersible tablets 0.00 0.99 -0.20 0.04 102Dispensing envelopes -0.04 0.25 0.07 0.16 102

72

Latex gloves 0.00 0.92 0.14 0.21 102Oxytocin 10IU/ml Injection, 1ml Ampoule. -0.03 0.61 0.05 0.51 102Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 0.09 0.37 -0.06 0.37 102Gentamycin 40mg/ml Injection, 2ml Ampoule -0.01 0.85 0.07 0.15 102Artemether 20mg + Lumefantrine 120mg Tablets, 24's 0.01 0.64 -0.02 0.13 102Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial -0.01 0.71 0.53 0.01 102Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 0.06 0.39 0.14 0.31 102Lignocaine -0.02 0.66 -0.05 0.05 102PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami -0.10 0.38 0.18 0.45 102Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 0.01 0.10 0.00 102Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( 0.00 0.96 0.12 0.04 102Emergency tray with all the necessary un expired medicines available -0.03 0.77 0.02 0.84 105Un expired ringer lactate, 5all dextrose, normal saline available -0.07 0.59 -0.11 0.12 105Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of 0.01 0.77 -0.02 0.23 105PEP policy and guidelines: Available in OPD: posted on the wall and up to date -0.08 0.34 0.03 0.67 105Opportunistic Infection and ART guidelines: Available, accessible in all consult -0.01 0.74 -0.03 0.05 105STI Management protocol: Displayed in all consultation rooms and up to date 0.07 0.05 0.14 0.23 105IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 0.02 0.16 0.16 0.00 105Focused ANC protocol: Displayed I all consultation rooms an up to date -0.03 0.32 0.07 0.38 105Surveillance line listing and case definitions displayed 0.03 0.37 0.12 0.03 92Updated EPI schedule, and a contingency plan displayed -0.07 0.40 -0.02 0.58 92EPI graphs showing trends displayed and staff member is able to interpret the g 0.06 0.54 -0.01 0.93 92EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI -0.13 0.28 0.26 0.13 92Fridge with a temperature booklet available and filled twice a day 0.02 0.74 0.09 0.27 92The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S 0.03 0.66 0.16 0.33 92The following Antigens are available: BCG, MR ( measles and Rubella), polio, Pen -0.17 0.01 -0.04 0.64 92The physical stock and the amount in the stock cards match ( Supervisor verifies 0.00 0.96 0.18 0.10 92Vaccines correctly stored in fridge with compartments as follows in fridges with 0.00 0.88 -0.06 0.00 92No expired vaccines -0.02 0.70 0.00 0.95 92The Vaccine Vial Monitor (VVM) status is kept 0.04 0.38 -0.07 0.03 92There are readable labels on vials with matching diluents -0.06 0.43 0.07 0.35 92The number of syringes available matches the number of vaccines in the stock car 0.10 0.22 0.13 0.36 92Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.02 0.08 -0.02 0.08 92

73

Vaccine carriers, cold box, gas regulator, gas cylinder and scissors -0.02 0.75 -0.07 0.20 92AEFI investigation forms, case investigation forms for EPI targeted diseases an 0.09 0.07 0.09 0.19 92Vaccine order forms and stock cards available 0.02 0.60 -0.05 0.01 92IV fluids and giving sets 0.08 0.38 -0.04 0.41 91Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and 0.02 0.69 0.00 0.94 91Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant -0.01 0.92 0.12 0.22 91PPH kits available and complete 0.24 0.02 0.04 0.67 91Eclampsia kit available and complete 0.22 0.07 0.10 0.26 91Uterotonic Medicines stored at correct temperature available in delivery room or 0.06 0.08 0.08 0.11 91Fetoscope, baby blanket Baby scale and tape measure -0.04 0.48 0.14 0.20 91Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 0.05 0.71 -0.03 0.70 91Penguin suction, neonatal bag and mask and suction tube in delivery room -0.06 0.44 0.12 0.30 91At least 2 obstetric sterilized delivery standard packs 0.10 0.52 0.08 0.32 91At least 5 pairs of sterile gloves should be available -0.05 0.31 0.02 0.58 91Standard referral forms (at least 10) and register available and properly filled -0.05 0.48 -0.01 0.79 102A referral feedback documented in the register for every referral made and/or re -0.10 0.40 0.17 0.08 92The following T Series forms available and fully completed: T1, T2, T3, T5, T6, 0.02 0.85 0.29 0.10 102T5 completed and sent timely (by the 7th of the following month) for previous mo -0.03 0.57 0.26 0.01 102All selected registers complete and correct 0.03 0.77 0.06 0.25 102T5 correct according to HMIS 0.13 0.37 0.01 0.87 102Consultations being done by appropriately qualified staff: PCN and/or RGN -0.05 0.41 0.10 0.01 105Triage: Emergency signs requiring immediate attention 0.05 0.34 -0.05 0.34 33Triage: Priority signs (requiring priority in the queue) 0.02 0.34 -0.02 0.34 33Triage: Non-urgent cases 0.00 0.00 33TB sign: Weight loss -0.01 0.58 -0.01 0.25 89TB sign: Fever for more than 3 weeks 0.01 0.85 -0.03 0.32 89TB sign: Cough for more than 14 days 0.02 0.53 -0.03 0.18 89TB sign: Night sweats 0.02 0.22 0.02 0.48 89TB sign: TB contact exposure. 0.02 0.51 0.00 0.99 89all of first visit ANC bookings who had documented: BP, Height, Weight measureme -0.02 0.83 0.00 0.96 105all of first visit ANC bookings who received the standard laboratory test accord 0.03 0.56 0.07 0.10 105all of first ANC visits who received TT vaccine -0.05 0.27 0.04 0.32 105all of first ANC visits who received iron supplementation 0.00 0.98 -0.02 0.09 105all of first visit ANC who had documented pregnancy test results -0.09 0.24 -0.11 0.18 105all PNC visits with assessment: General condition of the infant, NAD recorded if 0.18 0.11 0.21 0.04 105all PNC visits with assessment: General condition ,Pulse rate, B/P and 0.25 0.02 0.15 0.15 105

74

temperatuall PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 0.03 0.22 -0.03 0.29 105all women post-partum counselled and offered any of the modern FP method at fol 0.00 0.98 0.07 0.26 105all facility births seen for day 3 PNC visit in any one month in the last quarte 0.01 0.76 0.15 0.06 98all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 0.02 0.84 0.17 0.02 105all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b -0.15 0.09 0.02 0.70 105all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o -0.08 0.57 0.06 0.49 105all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 -0.07 0.59 0.02 0.84 105all children treated as outpatient for pneumonia in any one of month in last qu 0.02 0.79 0.19 0.07 107all children with diarrhoea correctly assessed for signs of dehydration), persi -0.07 0.37 0.10 0.29 107all children diagnosed with malaria that have RDT + or laboratory confirmation i 0.08 0.52 0.49 0.00 107all children correctly treated as an outpatient for pneumonia in any one of mont 0.09 0.34 0.25 0.00 107all children correctly treated as an outpatient (ambulatory) for diarrhoea in an 0.13 0.31 0.07 0.41 107all Children with uncomplicated malaria correctly treated according to national 0.11 0.47 0.38 0.00 107all Children with severe malaria correctly treated according to national guideli 0.13 0.41 -0.05 0.68 107all of 6-59 months old children with un complicated severe acute malnutrition (S 0.13 0.19 0.07 0.39 107all women who delivered in the facility monitored using partographs as per crite 0.15 0.14 -0.19 0.26 98all deliveries performed by skilled personnel in any one month in the last quart 0.01 0.13 0.58 0.01 98all total births in any one month in the last quarter documenting administration 0.13 0.00 0.04 0.29 98all births with placental status documented at birth in any one month in the las 0.03 0.01 0.68 0.00 98all newborns BF within one hour of birth in any one month in the last quarter 0.05 0.47 0.12 0.33 98all newborns received Vitamin K in the any one month in the last quarter 0.11 0.36 0.20 0.03 98all newborns received eye care (Tetracycline) in the any one month in the las 0.07 0.11 0.30 0.11 98all newborns received first vaccination (BCG) in the any one month in the last q 0.03 0.51 0.09 0.12 98all women monitored in early post-partum period (4th stage) per guideline (birth 0.18 0.09 0.09 0.21 98all newborns monitored in early post-partum period per guideline (birth to disch 0.28 0.01 0.01 0.93 98all women with prolonged labour in last quarter referred to higher level -0.05 0.10 0.11 0.32 103

75

all women with prolonged labor or Rupture of Membranes and without chorioamnioni -0.03 0.19 -0.05 0.03 103all women with PPH managed per guideline in the last quarter 0.00 0.93 -0.03 0.03 103all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch 0.00 0.17 0.00 103all pregnant women with severe pre-eclampsia and/or eclampsia managed according 0.01 0.44 -0.01 0.25 103all of neonates who had an APGAR score of ≤5 in the first minute after birth for -0.04 0.05 0.00 103all neonates with possible serious bacterial sepsis managed per standard in last -0.04 0.36 -0.06 0.02 103

Table A8. PHC unmatched ANCOVA using same-district comparators on input quality measures CQI Baseline N Coef. p-val Coef. p-val Input measures

General infrastructure: % of 11 items 0.00 0.99 0.19 0.07 87Cleanliness: % of 6 items -0.01 0.79 0.10 0.06 86Safety: % of 4 items 0.01 0.76 -0.01 0.91 87Privacy: % of 3 items 0.00 0.86 0.06 0.45 84General equipment: % of 7 items 0.03 0.39 0.09 0.36 86Pharmacy management: % of 9 items 0.02 0.09 0.11 0.06 85Pharmacy stock: % of 25 items -0.02 0.42 0.09 0.36 85Availability of guidelines: % of 6 items -0.02 0.58 0.09 0.10 84Vaccine management: % of 17 items -0.01 0.64 -0.05 0.39 56Maternity supplies: % of 11 items 0.03 0.60 0.22 0.03 82HMIS: % of 6 items -0.01 0.65 0.17 0.15 82OPD triaging knowledge: % of 4 items -0.04 0.33 -0.03 0.78 84TB screening knowledge: % of 5 items 0.01 0.46 0.04 0.64 69

Table A9. PHC matched ANCOVA using same-district comparators on input quality measures CQI Baseline N Coef. p-val Coef. p-val Input measures

General infrastructure: % of 11 items 0.00 0.74 0.25 0.04 71Cleanliness: % of 6 items 0.00 0.81 0.07 0.42 70Safety: % of 4 items -0.01 0.62 0.08 0.48 71Privacy: % of 3 items -0.01 0.63 0.05 0.57 69General equipment: % of 7 items 0.04 0.31 0.13 0.20 70Pharmacy management: % of 9 items 0.03 0.24 0.16 0.06 69Pharmacy stock: % of 25 items -0.01 0.62 0.18 0.26 69Availability of guidelines: % of 6 items -0.01 0.85 0.09 0.16 69Vaccine management: % of 17 items -0.01 0.73 0.01 0.84 45Maternity supplies: % of 11 items 0.01 0.76 0.27 0.03 68HMIS: % of 6 items -0.02 0.65 0.19 0.32 67OPD triaging knowledge: % of 4 items -0.06 0.33 0.07 0.63 69TB screening knowledge: % of 5 items 0.01 0.42 -0.02 0.06 56

76

Table A10. PHC matched ANCOVA using same-district comparators on individual indicators CQI Baseline N Coef. p-val Coef. p-val

Visible sign post when arriving at the clinic 0.00 0.96 0.30 0.01 70Fence/wall in good condition and gate with clearly written emergency contact num 0.11 0.24 0.19 0.09 70External appropriate wall finishing 0.16 0.02 0.34 0.29 67Roof intact, presence of well-maintained rain gutters 0.06 0.42 0.41 0.01 70Minimum of 3 toilets -0.03 0.26 0.10 0.18 70Hand washing facility with soap available near the toilets 0.05 0.08 0.03 0.33 70Electricity for 24 hours a day, and 7 days a week -0.01 0.73 0.07 0.54 70Fire extinguishers available, accessible, functional and serviced AND a clearly 0.17 0.01 0.06 0.38 71Appropriate drainage of waste water -0.02 0.58 0.18 0.01 70Shower with either running water or container of at least 100 litres -0.04 0.72 0.20 0.08 70Waiting area with adequate ventilation of waiting area -0.02 0.44 -0.01 0.12 69Ground clean with no litters and/or stagnant water and the grass cut 0.02 0.75 -0.02 0.69 70No waste and dangerous objects in courtyard such as needles – syringes –gloves – 0.02 0.35 -0.01 0.87 70Recently clean without visible fecal material and without smell -0.03 0.26 0.05 0.58 70Bin with lid accessible to clients 0.03 0.17 0.15 0.13 70Building clean and with good ventilation without bad smell? 0.02 0.23 0.02 0.61 69Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) -0.03 0.48 0.09 0.18 69Bin with plastic liners + sharps containers available and Bin not more than ¾ fu 0.05 0.31 0.10 0.10 69Hand washing facilities with soap available at accessible points for outpatient -0.06 0.41 0.21 0.01 69Safe drinking water is accessible 0.08 0.10 0.01 0.87 69(Ottoway pit OR Rubbish pit) AND Incinerator 0.05 0.39 0.29 0.09 71Doors with locks and closing properly -0.03 0.26 0.09 0.48 69Curtains on windows or non- transparent glass and screen 0.03 0.37 -0.04 0.13 69Sufficient space between beds (at least 1m between beds) -0.01 0.66 0.04 0.39 69Radio or mobile phone with airtime or landline for communication 0.01 0.76 0.15 0.32 70Transport plan for emergency referrals and/or contingency plan -0.08 0.08 0.06 0.05 70Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 0.12 0.16 0.20 0.06 69Mosquito nets available and in good state? 0.13 0.04 0.20 0.33 69Adult Weighing Scale and standard pediatric scale available 0.00 0.72 -0.03 0.12 69BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 0.08 0.50 0.10 0.09 69Important Accessories: cannula, giving sets, syringes and needles, drip stand, s 0.04 0.53 -0.06 0.02 69Monthly physical counts conducted with min, max and emergency order levels recor 0.07 0.08 0.16 0.04 69The physical stock level corresponds with that on the stock card: supply on stoc 0.01 0.92 0.07 0.29 69Staff completes and send MIS forms to district each month 0.04 0.74 -0.13 0.09 69

77

Stored correctly in a locked secured storeroom 0.01 0.12 -0.01 0.12 69Clean place, well ventilated with cupboards, labelled shelves, no incident light 0.03 0.68 0.04 0.59 69Medicines stored in alphabetical order also observing the First Expiry First Out 0.00 0.99 0.06 0.43 69Expired Medicine Register available -0.08 0.14 0.19 0.05 55No expired products in stock: supervisor verifies randomly 3 medicines and 2 con -0.02 0.82 -0.06 0.38 69Lockable trolleys available with working lock 0.05 0.08 0.31 0.07 69Doxycycline capsules 100mg 0.03 0.32 -0.01 0.88 69Ciprofloxacin tablets 500mg 0.05 0.46 0.10 0.20 69Metronidazole tablets 200mg Oral 0.03 0.50 -0.06 0.14 69Diazepam injection 5mg/ml -0.08 0.39 0.00 0.97 69Benzathine Penicillin injection 0.20 0.18 0.12 0.04 55Benzyl Penicillin 0.02 0.50 -0.04 0.42 69Amoxycillin suspension 125mg/5ml (dispersible tablets) 0.00 0.99 -0.03 0.46 69Ferrous sulphate tablets/Folic Acid -0.02 0.21 -0.02 0.10 69Zinc Sulphate tabs -0.02 0.54 -0.04 0.02 69Paracetamol 500mg tablets -0.02 0.52 0.01 0.89 69Paracetamol syrup or dispersible tablets -0.11 0.33 0.00 0.99 69Dispensing envelopes -0.02 0.59 0.07 0.18 69Latex gloves 0.00 0.62 -0.01 0.36 69Oxytocin 10IU/ml Injection, 1ml Ampoule. -0.02 0.66 -0.06 0.08 69Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 0.01 0.94 -0.11 0.29 69Gentamycin 40mg/ml Injection, 2ml Ampoule 0.03 0.63 0.19 0.40 69Artemether 20mg + Lumefantrine 120mg Tablets, 24's 0.02 0.34 -0.02 0.13 69Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial 0.05 0.02 0.44 0.31 69Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 0.01 0.91 0.08 0.63 69Lignocaine -0.03 0.56 -0.04 0.03 69PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami -0.08 0.37 0.16 0.42 69Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 0.01 0.44 -0.01 0.44 69Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( -0.04 0.24 -0.07 0.05 69Emergency tray with all the necessary un expired medicines available 0.01 0.83 0.02 0.76 69Un expired ringer lactate, 5all dextrose, normal saline available 0.04 0.68 0.02 0.85 69Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of -0.01 0.89 0.05 0.29 69PEP policy and guidelines: Available in OPD: posted on the wall and up to date -0.08 0.51 0.06 0.55 69Opportunistic Infection and ART guidelines: Available, accessible in all consult 0.01 0.37 0.11 0.20 69STI Management protocol: Displayed in all consultation rooms and up to date 0.02 0.63 -0.01 0.92 69IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 0.03 0.22 0.04 0.37 69Focused ANC protocol: Displayed I all consultation rooms an up to date -0.01 0.77 0.20 0.01 69

78

Surveillance line listing and case definitions displayed 0.11 0.17 0.00 0.97 45Updated EPI schedule, and a contingency plan displayed 0.02 0.81 0.01 0.89 45EPI graphs showing trends displayed and staff member is able to interpret the g -0.03 0.42 -0.04 0.73 45EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI -0.15 0.23 0.11 0.19 45Fridge with a temperature booklet available and filled twice a day -0.01 0.87 0.03 0.57 45The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S -0.07 0.18 0.00 0.89 45The following Antigens are available: BCG, MR ( measles and Rubella), polio, Pen -0.03 0.48 0.09 0.53 45The physical stock and the amount in the stock cards match ( Supervisor verifies -0.04 0.46 0.09 0.37 45Vaccines correctly stored in fridge with compartments as follows in fridges with 0.01 0.72 0.01 0.89 45No expired vaccines -0.04 0.35 0.04 0.69 45The Vaccine Vial Monitor (VVM) status is kept -0.02 0.23 -0.04 0.18 45There are readable labels on vials with matching diluents -0.09 0.28 -0.05 0.26 45The number of syringes available matches the number of vaccines in the stock car 0.00 0.95 -0.10 0.06 45Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.00 0.00 45Vaccine carriers, cold box, gas regulator, gas cylinder and scissors 0.01 0.84 0.00 45AEFI investigation forms, case investigation forms for EPI targeted diseases an -0.04 0.62 0.02 0.60 45Vaccine order forms and stock cards available 0.04 0.48 0.18 0.03 45IV fluids and giving sets -0.05 0.36 0.02 0.76 68Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and -0.04 0.43 0.14 0.05 68Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 0.03 0.74 0.04 0.70 68PPH kits available and complete 0.03 0.56 0.05 0.52 68Eclampsia kit available and complete 0.02 0.86 0.09 0.44 68Uterotonic Medicines stored at correct temperature available in delivery room or 0.04 0.07 0.05 0.43 68Fetoscope, baby blanket Baby scale and tape measure 0.00 0.98 -0.11 0.16 68Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 0.08 0.30 -0.19 0.04 68Penguin suction, neonatal bag and mask and suction tube in delivery room -0.03 0.65 0.05 0.71 68At least 2 obstetric sterilized delivery standard packs 0.08 0.28 0.07 0.44 68At least 5 pairs of sterile gloves should be available 0.01 0.81 -0.05 0.21 68Standard referral forms (at least 10) and register available and properly filled 0.07 0.30 0.13 0.27 67A referral feedback documented in the register for every referral made and/or re -0.04 0.63 0.41 0.06 63The following T Series forms available and fully completed: T1, T2, T3, T5, T6, -0.04 0.38 0.13 0.16 67T5 completed and sent timely (by the 7th of the following month) for previous mo -0.03 0.38 0.12 0.05 67All selected registers complete and correct -0.18 0.02 0.03 0.72 67

79

T5 correct according to HMIS -0.07 0.58 0.09 0.38 67Consultations being done by appropriately qualified staff: PCN and/or RGN -0.08 0.34 -0.01 0.22 69Triage: Emergency signs requiring immediate attention 0.00 0.00 10Triage: Priority signs (requiring priority in the queue) 0.00 0.00 10Triage: Non-urgent cases 0.00 0.00 10TB sign: Weight loss -0.01 0.71 0.00 56TB sign: Fever for more than 3 weeks -0.02 0.58 -0.07 0.24 56TB sign: Cough for more than 14 days -0.01 0.34 0.00 56TB sign: Night sweats 0.02 0.07 -0.01 0.06 56TB sign: TB contact exposure. 0.01 0.80 -0.07 0.12 56all of first visit ANC bookings who had documented: BP, Height, Weight measureme 0.08 0.45 0.00 1.00 52all of first visit ANC bookings who received the standard laboratory test accord -0.01 0.78 0.12 0.01 52all of first ANC visits who received TT vaccine -0.04 0.35 0.00 52all of first ANC visits who received iron supplementation 0.02 0.23 -0.03 0.04 52all of first visit ANC who had documented pregnancy test results 0.06 0.58 -0.33 0.13 52all PNC visits with assessment: General condition of the infant, NAD recorded if 0.02 0.69 0.02 0.78 52all PNC visits with assessment: General condition ,Pulse rate, B/P and temperatu 0.15 0.05 0.03 0.78 52all PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 0.04 0.29 0.04 0.69 52all women post-partum counselled and offered any of the modern FP method at fol 0.14 0.04 -0.03 0.65 52all facility births seen for day 3 PNC visit in any one month in the last quarte 0.00 0.90 -0.07 0.12 51all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 0.10 0.10 0.18 0.01 53all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b -0.09 0.27 0.08 0.36 53all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o 0.06 0.56 0.00 0.98 53all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 -0.02 0.72 0.12 0.31 53all children treated as outpatient for pneumonia in any one of month in last qu 0.09 0.27 0.15 0.29 50all children with diarrhoea correctly assessed for signs of dehydration), persi 0.03 0.42 0.21 0.06 50all children diagnosed with malaria that have RDT + or laboratory confirmation i -0.01 0.96 0.56 0.05 50all children correctly treated as an outpatient for pneumonia in any one of mont -0.02 0.87 0.16 0.09 50all children correctly treated as an outpatient (ambulatory) for diarrhoea in an -0.02 0.85 0.07 0.01 50all Children with uncomplicated malaria correctly treated according to national -0.03 0.63 0.56 0.04 50all Children with severe malaria correctly treated according to national guideli -0.10 0.11 -0.16 0.27 50

80

all of 6-59 months old children with un complicated severe acute malnutrition (S 0.04 0.58 0.12 0.47 50all women who delivered in the facility monitored using partographs as per crite 0.11 0.27 0.01 0.86 51all deliveries performed by skilled personnel in any one month in the last quart 0.01 0.41 0.00 51all total births in any one month in the last quarter documenting administration 0.06 0.03 -0.02 0.34 51all births with placental status documented at birth in any one month in the las 0.06 0.40 0.03 0.45 51all newborns BF within one hour of birth in any one month in the last quarter 0.09 0.06 0.10 0.35 51all newborns received Vitamin K in the any one month in the last quarter 0.11 0.26 0.11 0.07 51all newborns received eye care (Tetracycline) in the any one month in the las 0.07 0.05 0.09 0.12 51all newborns received first vaccination (BCG) in the any one month in the last q 0.01 0.83 0.07 0.09 51all women monitored in early post-partum period (4th stage) per guideline (birth 0.04 0.50 0.17 0.08 51all newborns monitored in early post-partum period per guideline (birth to disch 0.08 0.30 0.20 0.09 51all women with prolonged labour in last quarter referred to higher level -0.06 0.03 0.15 0.38 55all women with prolonged labor or Rupture of Membranes and without chorioamnioni -0.03 0.12 0.00 55all women with PPH managed per guideline in the last quarter -0.01 0.79 0.00 55all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch -0.01 0.11 0.00 55all pregnant women with severe pre-eclampsia and/or eclampsia managed according 0.00 0.84 -0.01 0.34 55all of neonates who had an APGAR score of ≤5 in the first minute after birth for -0.02 0.24 0.00 55all neonates with possible serious bacterial sepsis managed per standard in last 0.02 0.53 -0.03 0.37 55

Table A11. Hospital summary ANCOVA results of input quality measures CQI Baseline N Coef. p-val Coef. p-val Input measures

General infrastructure: % of 18 items 0.01 0.80 0.52 0.01 18Cleanliness: % of 7 items -0.09 0.25 0.07 0.80 18Safety: % of 24 items 0.00 0.91 0.27 0.01 20Privacy: % of 4 items -0.02 0.69 0.04 0.75 17Patient amenities: % of 6 items -0.01 0.78 -0.01 0.90 17General equipment: % of 19 items -0.04 0.36 0.16 0.28 19HR management: % of 13 items 0.05 0.38 0.09 0.56 21Surgical, lab and radiology equipment: % of 16

items 0.11 0.20 0.35 0.05 18Pharmacy management: % of 16 items -0.02 0.70 0.16 0.16 18Pharmacy stock: % of 21 items 0.04 0.30 0.16 0.17 20

81

Guidelines availability: % of 8 items 0.00 1.00 0.17 0.27 13Vaccine management: % of 16 items -0.01 0.83 0.05 0.59 12Maternity supplies: % of 17 items -0.06 0.31 0.38 0.02 15HMIS indicators: % of 9 items 0.01 0.73 0.24 0.03 20Quality improvement strategy: % of 22 items -0.06 0.45 0.29 0.04 13OPD triaging knowledge: % of 2 items 0.02 0.63 0.09 0.23 13TB screening knowledge: % of 10 items 0.03 0.53 0.36 0.07 12

Table A12. Hospital summary ANCOVA results using Q4 2018 as outcome CQI Baseline N Coef. p-val Coef. p-val Input measures

General infrastructure: % of 18 items 0.05 0.52 0.70 0.05 18Cleanliness: % of 7 items -0.16 0.25 -0.46 0.37 18Safety: % of 24 items -0.01 0.85 0.36 0.12 20Privacy: % of 4 items -0.09 0.38 -0.06 0.85 15Patient amenities: % of 6 items -0.13 0.12 0.31 0.07 15General equipment: % of 19 items 0.01 0.92 0.02 0.94 19HR management: % of 13 items 0.03 0.81 -0.01 0.97 21Surgical, lab and radiology equipment: % of

16 items 0.14 0.22 0.07 0.74 17Pharmacy management: % of 16 items 0.06 0.38 -0.04 0.79 18Pharmacy stock: % of 21 items 0.05 0.36 -0.18 0.23 20Guidelines availability: % of 8 items -0.01 0.80 1.66 0.00 13Vaccine management: % of 16 items 0.02 0.79 0.05 0.75 12Maternity supplies: % of 17 items -0.13 0.28 0.62 0.05 15HMIS indicators: % of 9 items 0.06 0.43 0.09 0.62 20Quality improvement strategy: % of 22 items -0.20 0.25 0.20 0.45 12OPD triaging knowledge: % of 2 items 13TB screening knowledge: % of 10 items -0.02 0.82 0.05 0.90 12

Process measures ANC process quality: % of 7 items -0.13 0.27 0.16 0.34 14PNC process quality: % of 5 items -0.42 0.20 -0.38 0.43 15Sick child assessment quality: % of 3 items 0.09 0.53 0.05 0.73 13Sick child treatment quality: % of 5 items 0.33 0.05 -0.56 0.08 13Maternal process quality: % of 10 items -0.20 0.29 0.05 0.92 15Obstetric complications quality: % of 8 items 0.03 0.84 -0.31 0.29 17Paediatric care processes: % of 4 items -0.29 0.15 0.38 0.12 14Paediatric complications processes: % of 4

items -0.23 0.42 0.36 0.31 13Surgical safety: % of 2 items -0.24 0.37 0.50 0.10 13

Table A13. Hospital summary difference in differences results DnD estimator Post CQI N Coef. p-val Coef. p-val Coef. p-val Inputs

General infrastructure: % of 18 items 0.05 0.29 -0.11 0.00 -0.01 0.81 39

82

Cleanliness: % of 7 items -0.14 0.24 -0.16 0.00 -0.02 0.43 39Safety: % of 24 items -0.09 0.18 0.01 0.91 0.11 0.07 41Privacy: % of 4 items -0.23 0.01 0.10 0.04 0.13 0.09 35Patient amenities: % of 6 items -0.24 0.00 -0.12 0.00 0.17 0.03 35General equipment: % of 19 items -0.01 0.92 0.00 0.99 0.03 0.70 40HR management: % of 13 items -0.03 0.70 -0.07 0.41 0.06 0.46 42Surgical, lab and radiology equipment:

% of 16 items 0.10 0.21 -0.10 0.09 0.04 0.61 38Pharmacy management: % of 16 items 0.08 0.56 0.09 0.06 -0.03 0.83 39Pharmacy stock: % of 21 items 0.05 0.62 -0.04 0.62 0.00 0.96 41Guidelines availability: % of 8 items 0.01 0.73 -0.04 0.04 0.04 0.32 34Vaccine management: % of 16 items -0.10 0.04 0.05 0.22 0.14 0.01 33Maternity supplies: % of 17 items -0.08 0.37 -0.01 0.76 0.06 0.29 36HMIS indicators: % of 9 items 0.07 0.54 0.05 0.10 -0.02 0.88 41Quality improvement strategy: % of 22

items -0.29 0.08 0.32 0.00 0.23 0.07 32OPD triaging knowledge: % of 2 items -0.10 0.14 0.10 0.14 0.10 0.14 33TB screening knowledge: % of 10 items 0.08 0.40 0.06 0.31 -0.11 0.13 31

Processes ANC process quality: % of 7 items -0.40 0.01 0.43 0.00 0.34 0.04 35PNC process quality: % of 5 items -0.38 0.27 -0.17 0.46 0.10 0.52 36Sick child assessment quality: % of 3

items -0.27 0.44 0.11 0.54 0.27 0.44 33Sick child treatment quality: % of 5

items 0.03 0.95 -0.31 0.02 0.02 0.89 33Maternal process quality: % of 10

items -0.23 0.21 -0.04 0.79 0.14 0.15 36Obstetric complications quality: % of 8

items -0.10 0.56 -0.20 0.11 0.18 0.22 38Paediatric care processes: % of 4 items -0.42 0.08 -0.01 0.91 0.31 0.04 32Paediatric complications processes: %

of 4 items -0.57 0.03 0.07 0.49 0.23 0.37 32Surgical safety: % of 2 items -0.28 0.19 0.08 0.58 0.24 0.20 33

83

Table A14. Hospital ANCOVA results for all facility outcomes CQI Baseline N Coef. p-val Coef. p-val Visible sign post -0.02 0.68 0.00 17Fence/wall without holes and a gate that can be closed -0.07 0.45 0.63 0.00 17Functional guard room with boom gate and functional light at the gate 0.17 0.27 0.34 0.02 17Direction signs with visiting times displayed -0.09 0.55 0.15 0.36 17Clean ground and grass cut -0.22 0.09 0.00 0.99 17Garden well maintained -0.02 0.91 0.39 0.06 17Clearly marked parking area for staff and clients 0.11 0.25 0.01 0.88 17Connected to local sewage system or septic tank 0.02 0.60 0.00 17Available incinerator, functional, fenced and being used 0.18 0.49 0.20 0.38 17Waste pit with hole 0.31 0.06 0.13 0.40 17Electricity for at 24 hours a day, and 7 days a week 0.02 0.60 0.00 17The hospital has a functional infection control committee 0.04 0.68 -0.11 0.40 17National Infection control guideline available 0.04 0.66 -0.10 0.54 17The committee assesses implementation of infection control guidelines 0.02 0.82 0.17 0.08 17Functional Autoclave/ Steam steriliser available 0.04 0.42 0.96 0.00 17Sensitive tape available on sterilized packs and cords not used to tie packs 0.00 0.00 17Are the following SoPs available in each department/service area? Hand Hygiene, 0.04 0.77 0.17 0.24 17Kitchen staff members receiving quarterly in house training on food handling 0.01 0.95 0.08 0.60 17health workers trained on IPC 0.00 0.99 -0.08 0.39 17Duty roster fully completed for medical and support staff on call -0.02 0.93 -0.23 0.49 13Response time for staff on call displayed and a review of calls made to ascertai -0.08 0.64 0.32 0.05 13Suction machine, adult and paediatric laryngoscope, bag and mask (ambubag) (adul -0.27 0.16 0.19 0.32 13If not functional was a report made? -0.13 0.26 -0.04 0.83 13Emergency tray with all the necessary drugs (as from EDLIZ). 50% dextrose, adren 0.01 0.94 0.15 0.36 13Emergency tray book up to date, signed daily, no expired drugs -0.03 0.73 -0.06 0.66 13Emercency tray is labelled. -0.02 0.90 0.00 13Cannula, syringes and needles, specimen bottles, -0.14 0.04 0.00 13Swabs, strapping, disinfectant, gloves, face mask -0.10 0.53 0.35 0.18 13Functional laryngoscope -0.05 0.65 0.21 0.06 13Ringer lactate, 5% dextrose, and normal saline available and not expired and giv -0.22 0.20 -0.07 0.70 13Blood and blood products 0.06 0.65 0.05 0.69 13OR: Walls of durable material and easily washable walls, Non transparent windows 0.20 0.29 0.35 0.08 14OR: Good appropriate air ventilation system -0.34 0.03 0.07 0.59 14Surgical register available and up to date 0.00 0.67 14Operating table: In good state with easy to clean mattress 0.08 0.47 0.26 0.13 14

84

covered with waterproOperating table: Functional hand rests with handcuffs & stir ups? 0.20 0.33 0.47 0.16 14Operating table: The table can be tilted and raised 0.12 0.45 0.22 0.38 14Anaesthetic machine with: Patient monitor, ECG and ETCO2, Ambu Bag, Laryncoscope 0.09 0.43 0.53 0.00 14Air way equipment: Endotracheal tubes size 3.5-8.5, Laryngeal mask sizes 2-5, Or 0.26 0.21 0.12 0.60 14Efficient suction machine, fluid warmer, pressure pump infusion gadget, all rang 0.05 0.84 0.04 0.86 14Medicines: ketamin, pethidine/morphine, metclorpromide, suxamethanium, atracuriu -0.15 0.43 0.10 0.54 14Emergency drugs: adrenaline, atropine, NaHCO3, hydrocortisone, promethazine -0.17 0.46 0.21 0.25 14Inhalation anaesthetic agents: halothane and/or isoflurane, N2O and O2 cylinder 0.22 0.12 0.15 0.35 14Available trolley with working locks? 0.06 0.73 0.35 0.13 14Dangerous drugs cupboard (double locked) each with a different lock, with DDA re 0.11 0.23 0.89 0.00 14At least 5 kits for each of the packs i.e. general and caesarean available -0.11 0.51 0.57 0.01 14Available and has adequate washing and scrubbing space? 0.00 1.00 0.50 0.00 14Has Pedal or elbow tap with disinfection device? -0.08 0.33 0.17 0.05 14Running water and anti-septic available? -0.02 0.93 0.01 0.96 14Surgical blouse, trousers, masks, hats, sandals and gumboots, goggles and gownin -0.13 0.31 0.30 0.06 14Qualified (certified) staff (laboratory technician and/or lab scientist)? 0.09 0.61 0.53 0.02 16Functional and available for emergencies after working hours? 0.02 0.63 -0.02 0.77 16Laboratory register correctly and completely filled -0.03 0.73 0.00 16Lab personnel wash dirty pipes in containers with antiseptic (except disposable 0.02 0.65 0.00 16Internal and external quality controls done 0.03 0.87 0.41 0.12 16Blood smear: Vivax, Oval, Falciparum, Malariae 0.00 0.98 0.13 0.48 16Stools: Ascaris, entamoebae, ankylostome, schistosome 0.04 0.82 0.00 16Available & functional microscope 0.00 0.00 16Centrifuge, full blood count machine, chemistry analyser machine, incubator and 0.03 0.90 -0.03 0.87 16Reagents for microscope an centrifuge -0.13 0.21 -0.12 0.14 16No expired reagents -0.03 0.87 -0.11 0.61 16Expiry and disposal reagents register available -0.14 0.47 0.16 0.33 16Gloves, specimen containers, appropriate protective clothing in laboratory 0.06 0.07 0.15 0.00 16Pharmacy being manned by qualified staff 0.09 0.66 0.53 0.01 18Specimen signatures for prescribers available at pharmacy 0.01 0.97 0.00 18Medicines and Allied Substances Act available 0.06 0.64 0.08 0.36 18Dangerous Drugs Act available 0.08 0.57 0.11 0.32 18Magnesium sulphate -0.02 0.92 -0.18 0.51 18Gentamycin, Amoxicillin: select one and check its availability 0.05 0.52 0.00 1.00 18Oxytocin 0.05 0.43 0.00 18

85

Contraceptives (implant, injectable, post-operative IUD, progestone- oral contra 0.09 0.36 0.66 0.00 18RHZE: rifampicin + isoniazid + pyrazinamide+Ethambutol -0.23 0.10 -0.02 0.89 18Hydrochlorthiazide, Metformin, Insulin: select one and check its availability 0.08 0.36 -0.01 0.90 18Paediatric first line ART: 0.05 0.70 0.09 0.48 18Adult Second line ART -0.06 0.39 -0.08 0.40 18Paediatric second line ART: 0.18 0.06 0.12 0.28 18V Medicines available at 100% in the last three months -0.11 0.40 0.08 0.47 18E Medicines available at 80% in the last three months -0.03 0.84 0.13 0.24 18N Medicines available at 60% in the last three months -0.01 0.95 0.03 0.84 18Monthly physical counts conducted with min, max and emergency order levels reco -0.01 0.95 -0.02 0.94 18The physical stock level corresponds with that on the stock card -0.04 0.73 -0.10 0.45 18Stored correctly in a locked secured storeroom (e.g burglar bars on windows and 0.00 0.00 18Clean place, well ventilated with cupboards, labelled shelves, no incident light -0.08 0.49 0.23 0.12 18Medicines stored in alphabetical order also observing the First Expiry First Out 0.12 0.24 0.21 0.04 18Expired medical items disposed according to guidelines -0.15 0.37 0.08 0.61 18Prescriptions made according to latest edition of EDLIZ 0.00 0.33 18The hospital sent adverse drug reaction report to PHE and/or MCAZ/MoHCC HQ in th -0.07 0.67 0.18 0.17 18The average number of antibiotics prescribed to a patient is less than two-three 0.00 0.00 18Radiological department manned by qualified staff 0.33 0.40 0.01 0.97 12Radiology department registered with radiation authority of Zimbabwe -0.13 0.65 0.35 0.12 12Staffs monitored for radiological exposure 0.35 0.34 0.28 0.32 12X-ray machine Available and working 0.00 0.00 12Ultrasound scan machine available and working 0.47 0.05 0.57 0.01 12Protective clothing available for each X-Ray room and in place 0.16 0.42 -0.16 0.42 12X-ray films 0.09 0.52 0.24 0.23 12X ray fixers available 0.20 0.36 0.13 0.55 12X ray developers available 0.04 0.85 0.04 0.85 12OPD consultations are done by appropriately qualified staff NURSE (RGN)/DOCTOR 0.00 0.00 8National Malaria guidelines for diagnosis and treatment of uncomplicated and sev 0.10 0.43 0.00 8PEP policy and guidelines 0.08 0.42 0.00 8Opportunistic Infection and ART guidelines 0.10 0.43 0.00 8STI Management protocol -0.03 0.84 0.00 8IMNCI guidelines -0.06 0.67 0.18 0.33 8Adult weighing scale and Standard Paediatric Weighing 0.00 0.25 8BMI calculator, glucometers and strips, peak flow meter, ophthalmoscope, otoscop -0.38 0.20 0.25 0.48 8Functional thermometer and OPD register in place 0.00 0.00 8PEP kit readily available in the event of a needle stick injury or 0.18 0.21 0.15 0.41 8

86

other accidenFocused ANC protocol in ANC care area 0.05 0.44 0.00 13PMTCT guidelines and charts 0.06 0.42 -0.06 0.61 13PMTCT medicine -0.08 0.08 0.00 1.00 13Standard Paediatric Weighing (SALTER) Scale, length/height board and MUAC tape -0.08 0.08 0.00 1.00 13OTP register and case sheet 0.06 0.74 0.45 0.07 13RUTF (ready to use therapeutic food) (adequate for at least three months: based -0.18 0.38 0.41 0.11 13Trained health worker on IMAM (Integrated Management of Acute Malnutrition) -0.08 0.30 0.64 0.00 13EPI surveillance line listing and case definitions displayed 0.02 0.92 0.10 0.54 12Updated EPI schedule, and a contingency plan displayed 0.04 0.66 -0.04 0.75 12EPI graphs showing trends displayed and staff member is able to interpret the g 0.12 0.35 0.03 0.72 12EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI 0.03 0.82 0.00 12Fridge with a temperature booklet available and filled twice a day 0.00 0.00 12The temperature is within the recommended range of + 2 and+ 8 degrees Celsius 0.00 0.00 12The following antigens are available: BCG, MR (measles and Rubella), polio, Pent -0.03 0.86 0.00 12The physical vaccine stock and the amount in the stock cards match 0.15 0.40 0.05 0.71 12Vaccines correctly stored in fridge 0.00 0.00 12No expired vaccines 0.00 0.00 12The Vaccine Vial Monitor (VVM )status is kept 0.08 0.68 0.00 12There are readable labels on vaccine vials with matching diluents 0.03 0.66 0.22 0.03 12The number of syringes available matches the number of vaccines in the stock car -0.32 0.11 0.07 0.66 12Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.03 0.66 -0.03 0.75 12Vaccine carriers, cold box, gas regulator, gas cylinder and scissors -0.20 0.26 -0.30 0.23 12AEFI investigation forms, case investigation forms for EPI targeted diseases an 0.07 0.61 0.10 0.43 12Vaccine order forms and stock cards available -0.10 0.24 -0.03 0.79 12IV fluids (ringer lactate, 5% dextrose, normal saline) and giving sets 0.03 0.81 0.05 0.61 15Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate, cal 0.07 0.70 0.25 0.05 15Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 0.08 0.65 0.50 0.04 15At least 10 pairs of sterile gloves available, face mask, specimen bottles -0.06 0.68 0.02 0.92 15PPH kits available and complete -0.03 0.86 0.28 0.16 15Eclampsia kit available and complete -0.15 0.50 0.32 0.11 15Fetoscope, baby blanket, Baby scale and tape measure -0.21 0.20 0.04 0.85 15Sterile cord clamps/ties for umbilical cord, Eye ointment ( Tetracycline) 0.06 0.69 0.10 0.36 15Neonatal bag and mask, penguin suction, resuscitator and suction -0.08 0.32 -0.04 0.71 15

87

bulb (at leastElectric heater, and wall clock 0.13 0.33 -0.13 0.47 15At least 5 obstetric sterilized delivery standard packs 0.18 0.35 -0.09 0.61 15All beds in the maternity ward/delivery room are in good state (not broken, matt -0.25 0.17 -0.12 0.50 15KMC bed 0.05 0.68 -0.05 0.68 15KMC heat source -0.10 0.16 -0.03 0.66 15KMC wrap for baby ( mbereko) -0.05 0.74 0.10 0.40 15KMC Clothes for baby ( hat, nappy and socks) 0.15 0.40 -0.27 0.13 15KMC register 0.00 0.00 15Standard referral forms (at least 10) available 0.03 0.61 -0.03 0.51 15Referral register available and properly filled -0.33 0.40 0.02 0.95 13The T Series forms are available and fully completed -0.12 0.32 -0.05 0.71 15The T5 and HS3/5completed and sent timely 0.05 0.65 0.20 0.05 15N complete registers -0.05 0.89 -0.01 0.95 15Are the information in each column of the selected registers complete and correc 0.08 0.77 0.22 0.34 15N correct registers 0.85 0.10 0.53 0.00 15Are the figures reported in any one month of the last quarter correct according 0.40 0.18 0.53 0.02 15Triaging of patients at OPD waiting area during all clinic shift 0.05 0.57 0.08 0.44 12TB screening: weight loss 0.00 0.00 12TB screening: fever -0.21 0.05 -0.04 0.78 12TB screening: cough 0.03 0.66 0.00 1.00 12TB screening: night sweats 0.06 0.51 0.00 1.00 12TB screening: TB contact exposure -0.38 0.08 -0.22 0.35 11% of TB presumptive (TB symptom positive) that have sputum results documented in -0.13 0.78 -0.50 0.48 11% of TB patients (SS + and SS-) that have HIV test results documented in any mo 0.17 0.49 -0.17 0.54 11% TB patients diagnosed in any one month in the last quarter that are receiving 0.29 0.36 -0.29 0.54 11% TB patients diagnosed in any one month in the last quarter that were TB notifi 0.00 1.00 0.50 0.04 11% TB patients diagnosed in any one month in the last quarter that TB contact tra 0.22 0.49 0.84 0.04 11Total OPD and TB -3.75 0.25 0.44 0.11 12ANC assessment -0.07 0.65 0.12 0.42 10ANC lab tests -0.29 0.46 0.00 9ANC iptp 0.11 0.49 -0.11 0.60 10ANC TT vaccine 0.03 0.65 0.00 10ANC iron 0.09 0.35 0.13 0.17 10ANC pregnancy test -0.46 0.04 0.25 0.35 10PNC newborn assessment 0.00 0.00 9PNC mother assessment -0.50 0.24 0.67 0.10 9PNC infant feeding 0.00 1.00 9PNC family planning -0.48 0.06 -0.23 0.23 9subtotal_points_scored_2c -5.65 0.38 -0.35 0.54 9

88

Maternity waiting home monitoring -0.25 0.06 0.90 0.00 12group_ge5cd64/complete_records_5c1 0.55 0.44 0.14 0.35 14Pneumonia assessment -0.05 0.81 0.21 0.27 14group_jk5pk73/complete_records_5c2 0.00 0.00 14Pneumonia treatment 0.09 0.66 -0.09 0.66 13group_ez5ry68/complete_records_5c3 0.00 0.00 14Diarrhoea assessment 0.07 0.75 0.13 0.45 13group_qq8do67/complete_records_5c4 0.00 0.00 13Diarrhoea treatment 0.15 0.39 0.35 0.06 13group_ad1im39/complete_records_5c5 0.00 0.00 13Malaria diagnosis 0.24 0.23 0.13 0.48 14group_yc2gx93/complete_records_5c6 0.00 0.00 14Uncomplicated malaria treatment 0.23 0.24 0.23 0.24 14group_tx3ii64/complete_records_5c7 0.00 0.00 14Severe malaria treatment -0.15 0.39 0.35 0.06 13group_oq0on51/complete_records_5c8 0.00 0.00 14Acute malnutrition treatemtn 0.03 0.93 0.21 0.50 13Total Sick child care 4.25 0.62 0.48 0.10 14deliveries performed by skilled personnel in any one month in the last quarter -0.07 0.23 0.00 17women who delivered in the facility monitored using partographs as per criteria -0.02 0.92 0.22 0.19 17total births in any one month in the last quarter documenting administration of -0.18 0.20 -0.15 0.49 17births with placental status documented at birth in any one month in the last qu 0.01 0.90 -0.10 0.57 17newborns BF within one hour of birth in any one month in the last quarter 0.04 0.76 0.00 17newborns received Vitamin K in the any one month in the last quarter -0.04 0.89 0.20 0.35 17newborns received eye care (Tetracycline) in the any one month in the last quart 0.10 0.30 -0.10 0.25 17newborns received first vaccination (BCG) in the any one month in the last quart -0.01 0.96 -0.06 0.70 17women delivered monitored in early post-partum period (4th stage) per guideline 0.23 0.37 0.11 0.58 17newborns monitored in early post-partum period per guideline (birth to discharge 0.14 0.52 0.14 0.41 17facility births seen for day 3 PNC visit in any one month in the last quarter -0.13 0.58 -0.11 0.53 17Total maternity care 2.31 0.60 0.19 0.26 15women with prolonged labor or Rupture of Membranes and without chorioamnionitis -0.16 0.43 0.03 0.86 17women with PPH managed per guideline last quarter -0.03 0.85 -0.27 0.07 17women with signs of intra- or post-partum sepsis that were treated per standard -0.34 0.05 0.18 0.21 17pregnant women with severe pre-eclampsia and/or eclampsia managed according to -0.08 0.60 0.24 0.05 17neonates who had an APGAR score of ≤5 in the first minute after 0.08 0.57 -0.47 0.00 17

89

birth for whom rneonates who had an APGAR score of ≤5/10 at 1 minute after delivery and were suc 0.20 0.21 -0.18 0.18 17neonates with possible serious bacterial sepsis managed per standard in last qua 0.12 0.57 0.29 0.10 17low birth weight (LBW) newborns admitted to KMC unit in the last quarter -0.06 0.69 0.08 0.55 17Total complications -6.43 0.48 -0.29 0.36 17hospitalized patients with correctly completed admission medical record any one -0.15 0.60 0.19 0.42 15Written record of administration of patient medications up to date any one month -0.14 0.35 0.31 0.05 15hospitalized patients with documentation of vital signs every 6 hours and every -0.19 0.15 0.25 0.07 15hospitalized patients with documentation of daily progress note by doctor any on -0.26 0.15 0.43 0.05 15Total: pediatric best practices -2.79 0.25 0.36 0.02 15hospitalized children treated for pneumonia any one month in the last quarter wh -0.04 0.80 -0.03 0.78 14hospitalized children treated correctly (all criteria met) for pneumonia among t -0.09 0.59 -0.02 0.90 14hospitalized children treated for diarrhoea in any one month in the last quarter -0.33 0.11 -0.08 0.65 14hospitalized children treated for diarrhoea correctly (all criteria met) in any -0.02 0.93 -0.10 0.56 14Total: pediatric complications -0.55 0.83 -0.08 0.50 14Surgical safety checklist utilization rate -0.08 0.73 0.58 0.01 14Surgical site infection 0.07 0.75 0.26 0.27 14Total: surgery safety 0.90 0.81 0.73 0.05 14Does the QIC have terms of reference (ToR) with the following components 0.00 1.00 0.33 0.20 13Are all service areas, including administration, represented in the QIC committe 0.03 0.82 0.14 0.47 13QI plan: Action plan for improvement 0.08 0.65 0.35 0.05 13QI plan: Targets 0.02 0.94 0.25 0.20 13QI plan: Areas for improvement -0.08 0.72 0.29 0.15 13Are reports on QI activities sent to PMDs quarterly? -0.28 0.12 0.09 0.63 13Are processes like waiting time, appointment system, and patient flow discussed 0.23 0.28 0.04 0.82 13Is the QIC meeting quarterly? 0.04 0.71 0.00 1.00 13Are status of QI plan and other improvement plans discussed during the meeting? -0.04 0.78 0.13 0.33 13Completeness and correctness of the information in the reviewed patient files 0.17 0.35 0.15 0.33 13Comprehensive patient management given to patients in the reviewed files -0.12 0.49 0.28 0.08 13Consistency of information between the reviewed patient files and registers -0.08 0.65 0.40 0.02 13Does the QIC receive feedback from the supervision? -0.18 0.24 0.03 0.81 13Has the QIC developed action plan for improvement to address 0.02 0.92 -0.01 0.95 13

90

the identified gapsQuarterly client satisfaction surveys: Does the QIC have survey tool? 0.05 0.81 0.19 0.34 13Does the QIC conduct quarterly surveys? -0.05 0.82 0.45 0.07 13Are survey analysis reports available? -0.15 0.39 0.47 0.02 13Does the facility have labelled suggestion boxes? -0.07 0.62 0.03 0.85 13Is instruction on how to use the boxes posted on or above the suggestion boxes? -0.04 0.84 0.07 0.66 13Does the facility analyse the findings of suggestion boxes -0.16 0.45 0.01 0.94 13Does the hospital conduct audit meetings or maternal mortality review meetings a 0.00 0.00 13Is guideline for audit/mortality meetings available? -0.22 0.20 0.04 0.83 13

91

PHC summary outcome figures

92

93

94

95

96

97

98

99

100

101

Hospital summary outcome figures

102

103

104

105

106

107

108

109

110

111

112

113

114

Annex 4. Continuous Quality Intervention checklists for PHCs and Hospitals

HOSPITAL QUALITY SUPERVISION CHECKLIST

February 2016

115

HOSPITAL QUALITY SUPERVISION CHECKLIST

Questionnaire for a Provincial/District/Mission Hospital Quality Review

Province: ___________________________________________________________________________District: ___________________________________________________________________________Hospital: ___________________________________________________________________________Number of beds: ________ Catchment area population: __________________________________________Date of supervision: ______________________________________________________________________

Name of supervisors and designationNo. Name of supervisor Designation12345678

116

Date received by ____________________________________

For RBF use:% Structural score (35%Weight): …………....% Management & planning score

% Clinical care score (65% weight): ………....

Final Combined Score from Database: ……....

I. Hospital Staffing

STAFF Establishment

In post

Vacant

Duration of vacancy

District Medical OfficerGovernment Medical OfficerClinical OfficerHealth Promotion OfficerDistrict Nursing OfficerMatronSister in ChargeSister in Charge CommunityPrincipal State Certified NurseSister GeneralState Certified NurseDistrict TB CoordinatorDistrict Environmental Health OfficerEnvironmental Health OfficerEnvironmental Health TechnicianPharmacistPharmacy TechnicianDispensary AssistantNutritionistAssistant NutritionistRadiographerX-ray OperatorDark Room AssistantPhysiotherapistMedical Laboratory ScientistDistrict Health Service AdministratorHuman Resource OfficerHealth Information AssistantAccountantAccounting AssistantGovernment Dental OfficerDental TherapistDental Surgery AssistantRehabilitation TechnicianCCSD PackerNurses with midwiferyUp skilled PCNPCNOperating theatre nursesNurse anaesthetistNurse aidesGeneral hands

117

STAFF Establishment

In post

Vacant

Duration of vacancy

Non-medical staff or unqualified staff : Cook, , Non-medical staff or unqualified staff : driverNon-medical staff or unqualified staff : Laundry Hand , Senior Hand)

118

ASSESSMENT SUMMARY

I. STRUCTURAL SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

General Structure and auxiliary services

11 6

Structure in clinical departments

44 18

TOTAL 55 24

II, MANAGEMENT & PLANNING SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

Administration, finance, planning

17 8

Infection control 8 4Emergency services 12 5Operating theatre 19 6Laboratory 13 10Pharmacy 28 11Radiological services 9 4Outpatient department (OPD)

10 3

Family and Child Health (FCH)

7 2

Extended Program Immunization (EPI)

17 7

Maternity ward 17 7HMIS 14 4

TOTAL 171 71

III. CLINICAL MANAGEMENT SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

OPD/ consultation area 18 3TB management 24 4ANC-PNC Best practices 54 9Maternity waiting home 6 1

HIV-PMTCT 30 5Ambulatory management diarrhoea, pneumonia

48

8Delivery best practices 66 11Management obstetric complications

548

Inpatient Pediatric best practices 28 4Inpatient management diarrhea, pneumonia

284

Postoperative infection control 12 2Quality Improvement/Assurance

227

TOTAL 384 66

120

STRUCTURAL SECTION

I.1 Structural indicators in the general compound of the Hospital, mortuary, Operating theatre and maternity waiting home

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recordedN/A: not applicable

1S Outside appearance (when arriving at hospital):

1S.1 -Visible sign post

1S.2 -Fence/wall without holes and a gate that can be closed

1S.3 -Functional guard room with boom gate and functional light at the gate

1S.4 -Direction signs with visiting times displayed

2S Maintenance of the ground:

2S.1 -Clean ground and grass cut

2S.2 -Garden well maintained with flower beds, trees or lawn, resting places (benches in shade),and with no animal excrement, no litter or dangerous objects such as needles, syringes, gloves, used cotton wool, etc.,

2S.3 -Clearly marked parking area for staff and clients

3S Waste water drainage system

3S.1 -Connected to local sewage system (if not, septic tank must be available)

4S Incinerator within the premises:

4S.1 -Available, functional, fenced and being used

5S Waste pit for non-contaminated objects:

5S.1

-Waste pit with hole of minimum 3 metres depth fenced, with no contaminated and non-decomposable (non – biodegradable) objects available(Waste pit is only required in hospitals where the city municipality is not collecting the non-contaminated objects)

6S Lighting system

6S.1-Electricity for at 24 hours a day, and 7 days a week(Source of electricity: ZESA with backup system of either generator or solar energy and/or inventors).Total points this quarter (MAXIMUM AVAILABLE Points: 11 POINTS)

121

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators*if you write N/A for an indicator in a selected department, please deduct the points for the indicator from the maximum available points

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panes

8S Inside appearance of building and its cleanliness

8S.1

-Walls and Floors- Clean, without cracks and floors-polished ( if required) and Ceiling / roof intact without leaks neither cobwebs or mould growth

8S.2 -Doors with locks and closing properly

9S Firefighting System:

9S.1-Functional and serviced fire extinguishers with or without water hoses available and accessible

9S.2

-Fire exits clearly marked and is there a clearly marked firefighting assembly area and evacuation plan: Ask staff member on firefighting plan

10S Garbage bins in ground

10S.1

-Labelled Bins with lids and plastic lining available and not more than ¾ full without hazardous waste (e.g. sharps, used cotton wool e.t.c) at accessible points

11S Presence of sufficient well-maintained latrines/toilets

11S.1 -One water closet for every 6 inpatients

122

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panesor One water closet for every 20 outpatients

11S.2-The toilet entries should be clearly marked for each sex and the male toilets should also have a urinal

11S.3 -Recently cleaned without visible faecal matter or urine

12S Water and soap for hand washing:

12S.1-Hand washing facilities with running water and soap available at accessible points

13S Waste collection

13S.1-Waste collected and disposed daily or within 4 to 5 hours ( only for maternity ward) (Ask staff member)

13S.2

Waste segregated with colour coding: -Red/yellow for non-sharp infectious waste-Yellow or other puncture proof containers for sharp objects-Black for communal non-sharp non-infectious wastesand/or service area*the bins and/or sharp boxes should not be more than ¾ full

14S Protective clothing and disinfectant use by cleaners: * Ask the available cleaners during the assessment period

14S.1 -Cleaners have appropriate protective clothing (Heavy duty gloves, Uniforms,

123

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panesDustcoats, Plastic Aprons, Gumboots, Face Mask)

14S.2

-Cleaners know how to appropriately use disinfectants? Soap with water for general cleaning and 1 part jik to 4 parts for spillages and/or depending on the concentration of the bleach i.e. blood and mainly body fluids (after containing and cleaning the spills): Ask the available cleaners during the day of assessment

15S Professional staff appropriately dressed :

15S.1 -Clean with standard uniform, identification tag and lace up shoes.

16S Staff duty roster, staff leave calendar and clock in register (per department) including maintenance and cleaning duty roster with timeline and signature column for cleaner and supervisor:

16S.1

-Duty roster clearly visible including assigned nurse(s) and doctors(s) covering each ward for each shift, including night and weekend coverage

16S.2- The DOCTOR, NURSES actually present in the ward (Check the person/s on duty on the day of visit)

16S.3-Staff leave calendar complete and up to date, and displayed where all staff can see

16S.4 -Clock in register including maintenance and cleaning with timeline and supervisor signature

124

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panescolumnSub-total points (maximum available sub-total points: 20)

Please select one department among the following two departments ( encircle the selected department) and assess indicators 17S-20S1. OPD2. FCH

17S Good conditions in waiting area, meeting minimum standards:

17S.1 -With sufficient benches and / or chairs (according to daily attendance): calculate the average daily attendance from the weekly attendance

17S.2 - Well/adequately ventilated with sufficient light adequate ventilation of waiting area:

If Open space with a shade or roof supported by brick or metal pillars or

If closed space: windows should measure at least 1/10 of floor area and at least ½ of window area should be openable.

18S Consultancy rooms in good condition, meeting minimum standards

18S.1 -Windows non-transparent glass and screen around bed

18S.2 -Functional doors with lock

125

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panes

18S.3 -Furniture (at least one chair and table for nurse/doctor, one chair for patient) positioned to reduce transmission of infection from patient

18S.4 -Hand washing facility with running water and/or alcohol based hand rub

18S .5 -Examination bed in good condition and covered with clean linen

19S OPD fees, and medical aid companies:

19S1

-OPD fees and list of accepted medical aid companies accepted displayed in local vernacular (when applicable) and easily visible for patients

20S Hygienic and aseptic conditions in wound dressing:

20S.1 -Bench and foot rest covered with Macintosh available

20S.2 -Bins for infected and contaminated objects with lid, plastic lining and foot pedal available and not more than ¾ full

20S.3 -Sharp box well positioned and not more than ¾ full?Sub- total points (maximum sub-total points: 11)

Please select one department among the following two departments ( encircle the selected department) and assess indicators 21S-26S1. Maternity Ward

126

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panes

2. Paediatric Ward

21S Availability and status of furniture and other items

21S.1 -Beds, and mattresses with sheets, blankets, bedside lockers, benches on bed side available and in good state (not broken, torn and clean)

21S.2 -Mosquito nets (in malaria endemic areas) available and in good state ( Not Applicable in Non-Malaria Endemic area)

21S.3 -Fan and heater available when required and cleaned on a regular basis

22S Bucket or basin for dirty linen:

22S.1 -Bins covered with a lid and not overflowing

22 S.2 - SOP for linen segregation available as per standard National IPC guideline

23S Hygienic condition, access to water and space:

23S.1 -Clean and regular cleaned ( ask staff member or look at the weekly plan for cleaning)

23S.2 -Safe drinking water available and accessible

23S.3 -Enough space between beds (at least 1m between beds)

23S.4 -Well ventilated without bad smells and/or all windows operational and open

127

I.2Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below.Please vary the department to be selected quarterly*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S1. OPD2. FCH3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panes

23S.5 -Shower with running water, and/ or container with at least 100 litres for patients to bath with hot water available during winter?

24S Movable lockable drug trolleys:

24S.1 -Available with working locks?

24S.2 -Dangerous drugs cupboard (double locked) each with a different lock available

24S.3 -DDA registers available

Sub-total ( maximum sub-total available points: 13)Total points this quarter (MAXIMUM AVAILABLE Points: 44 POINTS)

128

MANAGEMENT AND PLANNING: STAFF, POLICY, GUIDELINES, Medicines & SUPPLIES and Vaccines

II.1 ADMINISTRATION, FINANCE AND PLANNING Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

Indicators

*Assess at District Medical officer’s office/Administration and/or Finance Department /Matron office*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

1M Mission statement, Vision, Values and patient charter 1M.1 -Displayed in public places and clearly visible2M Catchment area map, spot map, monitoring graphs, demographic data:

* Assess the indicator in the office of the DMO/Matron or administrator office2M.1 -Catchment area maps with current catchment population target

population for services calculated correctly and displayed2M.2 -Spot map showing recent or suspected out breaks with clear markings

displayed2M.3 -Monitoring graphs for different services displayed3M Documentation of activities/ Operational Plan3M.1 -Staff minute book/file available, well filed and up to date3M.2 -Quarterly review reports, annual operational plan and annual progress

report ( of previous year) available, well filed and up to date4M Management book, inventory register, maintenance book, returns( human, material and finance)4M.1 -Available and up to date?5M Service and maintenance plan for hospital equipment and vehicles5M.1 -Plan available and being followed (Ask for reports and cross check with

the plan)6M Purchasing of medicines, equipment and consumables6M.1 -MoHCC tender and procurement procedure documents available?6M.2 -Functional CBU and PTC available6M.3 -Medicines, equipment and consumables purchased as per the procedure?

(ask the procurement committee to provide the documents for an item which was purchased and cross check whether it was bought as per the MoHCC procedures)

7M Finance and accounting system7M.1 -Financial and accounting documents available and well-kept in clearly

labelled files including bank statements, payment vouchers with attached support documents.

7M.2 -Document showing budget and revenues (GoZ, HSF, RBF, Donations) available

7M.4 -Managed by qualified staff (accountant/ accounts assistants)? 7M.5 -Financial reports which show expenditure of proper utilization of funds

according to statutory requirements and minutes of finance meetings available

8M Ambulance and communication systems8M.1 -The hospital has a functional (24/7) ambulance8M.2 -The hospital has a functional communication system (land line/cell

phone/radio)

129

Total points this quarter: (Maximum Available Points: 17)

II.2 . INFECTION CONTROL*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators*If N/A, please deduct the points for the indicator from the maximum available points

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

9M Infection control committee *Assess the indicator by asking the infection control committee/focal person

9M.1 -The hospital has a functional infection control committee (stand alone or as part of the quality improvement committee) ( check for the minutes of quarterly meetings)

9M.2 -National Infection control guideline available with a facility risk assessment tool including TBIC

9M.3 -The committee assesses implementation of infection control guidelines and takes measures according to the guideline? (check in reports/minutes)

10M Sterilization of instruments:Assess the indicator at CSSD (Central Steam Sterilizing Department) as all sterilizations are done in this department.

10M.1 -Functional Autoclave/ Steam steriliser available

10M.2 Sensitive tape available on sterilized packs and cords not used to tie packs ( check at least two packs)

11M Basic Infection Prevention and Control (IPC) SOPS

11M.1 -Are the following SoPs available in each department/service area? Hand Hygiene, Environmental Cleaning; waste segregation and management; decontamination. Please assess in one selected department quarterly.

12M Regular training of kitchen staffs on food handling and health workers on IPC: Check for training reports from Hospital Food Services Supervisor and IPC focal person/Committee chair and ask two kitchen staff members and/or health workers

12M.1 -Kitchen staff members receiving regular in-house training (quarterly) on food handling

12 M.2 -health workers trained on IPC

Total points this quarter: (Maximum Available Points: 8)

II.3 EMERGENCY SERVICES

Indicators *Please give a score for each of the criteria under each indicator as

per the criteria in the right column*N.B. The items highlighted in bold are the indicators*

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

130

13M. Staff Duty Roster for On call Medical doctor and support staff

13M.1 -Duty roster fully completed and posted in public place for medical and support staff on call through the month (doctor, lab, x-ray, anaesthetist, theatre nurse and observed on the day of the visit)?

13M.2 -Response time for staff on call displayed and a review of calls made to ascertain response timeSUPPLIES

14M Are the following emergency airway equipment available and functional?

14M.1 -Suction machine, adult and paediatric laryngoscope, bag and mask (ambubag) (adult -and paediatric), oxygen.

14M.2 - If not functional was a report made? *Equipment should be easily accessible not stored under lock and key.

15M Emergency tray:

15M.1 -With all the necessary drugs (as from EDLIZ). 50% dextrose, adrenaline, lignocaine, diazepam, atropine,

15M.2 -Emergency tray book up to date, signed daily, no expired drugs

15M.3 -The tray is labelled. (should be including the drugs and accessories, with clearly legible and durable labels)

16M Important Accessories:

16M.1 -Cannula, syringes and needles, specimen bottles,

16M.2 -Swabs, strapping, disinfectant, gloves, face mask

16M.3 -Functional laryngoscope

17M IV fluids and blood:

17M.1 -Ringer lactate, 5% dextrose, and normal saline available and not expired and giving sets, drip stand

17M.2 Blood and blood products

Total points this quarter: (Maximum Available Points: 12)

II.4. OPERATING THEATRE

Indicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators*If N/A, please deduct the points for the indicator from the maximum available points

Score:1: if all criterion have beenmet/ recorded0: if all criterion have not been met/ not recordedN/A: not applicable

18M Structure and Ventilation system, and register18M.1 -Walls of durable material and easily washable walls, Non transparent

windows and functional doors with floor paved with vinyl/ceramic tiles without cracks, ceiling in good state

18M.2 -Good appropriate air ventilation system: small meshed windows to let

131

air in ( if only natural ventilation is used) and/or air conditioner fitted without/with sealed windows ( if mechanical ventilation is used), and doors to the aisle closed

18M.3 -Surgical register available and up to date?SUPPLIES

19M Operating table19M.1 -In good state with easy to clean mattress covered with waterproof

material?19M.2 - Functional hand rests with handcuffs & stir ups?19M.3 -The table can be tilted and raised

20M Basic equipment and consumables:-Are the following basic equipment & consumables available?

20M.1 Anaesthetic machine with: Patient monitor, ECG and ETCO2, Ambu Bag, Laryncoscope size 0-4 and O2 cylinder for back up, functional failure alarm and7 ventilator

20M.2 Air way equipment: Endotracheal tubes size 3.5-8.5, Laryngeal mask sizes 2-5, Oral air way sizes 00-6, Intubating intruder and all range sizes of face masks

20M.3 Efficient suction machine, fluid warmer, pressure pump infusion gadget, all range sizes of cannulas, syringes and needles and defibrillator

20M.4 Medicines: ketamin, pethidine/morphine, metclorpromide, suxamethanium, atracurium, ephedrine, neostigmine, diclophenac IM/supporitory

20M.5 Emergency drugs: adrenaline, atropine, NaHCO3, hydrocortisone, promethazine

20M.6 Inhalation anaesthetic agents: halothane and/or isoflurane, N2O and O2 cylinder

21M Movable lockable drug trolleys21M.1 -Available with working locks?

21M.2 -Dangerous drugs cupboard (double locked) each with a different lock, with DDA registers available?

22M Emergency surgical packs (general and caesarean ):

22M.1 -At least 5 kits for each of the packs i.e. general and caesarean available

23M Gowning area and theatre clothing:23M.1 -Available and has adequate washing and scrubbing space?23M.2 -Has Pedal or elbow tap with disinfection device?

23M.3 - Running water and anti-septic available?23M.4 -Surgical blouse, trousers, masks, hats, sandals and gumboots, goggles

and gowning packs availableTotal points this quarter: (Maximum Available Points: 19)

II.5. LABORATORY SERVICESIndicat *Please give a score for each of the criteria under each indicator as Score:

132

ors per the criteria in the right column*N.B. The items highlighted in bold are the indicators

1: if all criterion have beenmet/ recorded0: if all criterion have notbeen met/ not recorded

24M Staffing of Laboratory:24M.1 -Qualified (certified) staff (laboratory technician and/or lab scientist)?25M Functionality of Laboratory after working hours:-

25M.1 -Functional and available for emergencies after working hours? * Verify after hour activities in the laboratory register for the last quarter

26M Recording of results:26M.1 -Laboratory register correctly and completely filled:

Check record of any month in the last quarter27M Washing dirty pipettes:

27M.1 -Lab personnel wash dirty pipes in containers with antiseptic (except disposable pipettes)?

28M Internal and External quality assurance services:28M.1 -Internal and external quality controls done

*Check for copy of report of internal and External quality assurance assessment report (by ZINQAP)

cSUPPLIES

29M Parasite demonstration on plastic paper, in a colour book, or put on wall

29M.1 Blood smear: Vivax, Oval, Falciparum, Malariae

29M.2 Stools: Ascaris, entamoebae, ankylostome, schistosome

30M Microscope:30M.1 -Available and in working condition (functional) with functional

objectives - immersion oil – mirror or electricity and – blades, cover glass, slides, GIEMSA

31M Equipment:

31M.1 -Centrifuge, full blood count machine, chemistry analyser machine, incubator and fume cup board (also serviced) available and serviced and functional:( Check in the maintenance register if the assessments were done monthly and signed for)

32M Reagents32M.1 -Available for the equipment mentioned under 30M.1 and 31M.1?

(Check stocks against minimum levels)32M.2 -No expired stocks

32M.3 -Expiry and disposal register available

33M Are the following items available?33M.1 -Gloves, specimen containers, appropriate protective clothing

Total points this quarter: (Maximum Available Points: 13)

133

II.6. PHARMACY (MEDICINES AND SUNDRIES STOCK MANAGEMENT)

Indicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

34M Staffing and Specimen Signature:34M.1 -Pharmacy being manned by qualified staff (Pharmacist and/or

Pharmacy technician)? 34M.2 -Specimen signatures for prescribers available at pharmacy?35M Statutory instruments:35M.1 -Medicines and Allied Substances Act available35M.2 -Dangerous Drugs Act available36M Availability of essential medicines: -Availability for at least 90 days ( Minimum stock) 36M.1 Magnesium sulphate 36M.2 Gentamycin, Amoxicillin: select one and check its availability 36M.3 Oxytocin 36M.4 Contraceptives (implant, injectable, post-operative IUD, progestone-

oral contraceptive, combined oral contraceptive pills)* select one among the list and check its availability

36M.5 RHZE: rifampicin + isoniazid + pyrazinamide+Ethambutol36M.6 Hydrochlorthiazide, Metformin, Insulin: select one and check its

availability37M HIV and AIDS medicines: Availability for at least 90 days.37M.1 Adult first line ART:

Preferred: TDF +3TC+EFV, alternate: TDF+3TC+NVP or ZDV+3TC+EFV/NVP ( could available in Dual or triple FDCs)

37M.2 Paediatric first line ART: Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( <3 yrs)

37M.3 Adult Second line:AZT + 3TC + ATV/r or LPV/r or TDF + 3TC + ATV/r or LPV/r

37M.4 Paediatric Second line:ABC+3TC+LPV/r

38M VEN medicines-Please ask for the MIS report for any one month in the last quarter and then check in the report whether the VEN medicines were available as required

38M.1 V Medicines available at 100% in the last three months38M.2 E Medicines available at 80% in the last three months38M.3 N Medicines available at 60% in the last three months

39M Stock Cards:39M.1 -Monthly physical counts conducted with min, max and emergency

order levels recorded and updated39M.2 - The physical stock level corresponds with that on the stock card40M Storage of drugs:

134

40M.1 -Stored correctly in a locked secured storeroom (e.g burglar bars on windows and doors)?

40M.2 -Clean place, well ventilated with cupboards, labelled shelves, no incident light

41M.3 -Medicines stored in alphabetical order also observing the First Expiry First Out rule

41M Expired products:41M.1 -Separated from stock41M.2 -Expired medical items disposed according to guidelines(Check for the

presence of expired medicine register disposal register and certificate. verify randomly 3 medicines and 2 consumables (check stock cards)

42M Prescriptions: * check last 3 prescriptions made during the day of assessment from OPD register and comparewith EDLIZ

42M.1 -Prescriptions made according to latest edition of EDLIZ43M Adverse events report:-

*Ask the DMO or at the pharmacy43M.1 -The hospital sent adverse drug reaction report to PHE and/or

MCAZ/MoHCC HQ in the last quarter (Check for presence of copy of adverse event) ( if there was no adverse event, check for the presence of the reporting forms)

44M Average number of antibiotics prescribed to a patient •Assessment: measured by considering the previous 30 patients/prescription/T12 (pharmacy register) and then tallying the number of antibiotics per prescription and divide by 30. (Acceptable range is 2-3) *Source of data: Pharmacy register and/or T12

44M.1 -The average number of antibiotics prescribed to a patient is less than two-three antibiotics in any one month in the last quarterTotal points this quarter: (Maximum Available Points: 28)

II.7 RADIOLOGICAL SERVICES

Indicators *Please give a score for each of the criteria under each indicator as

per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

45M Staffing of Radiological department:45M.1 -Manned by qualified staff? (radiographer or x-ray operator46M Registration and monitoring of staff for exposure:- 46M.1 -Radiology department registered with radiation authority of Zimbabwe46M.2 -Staffs monitored for radiological exposure

SUPPLIES47M Radiological equipment:47M.1 - X-ray machine Available and working 47M.2 -Ultrasound scan machine available and working ( if it is available in

maternity ward, consider it as available, but check its functionality)

135

48M Protective clothing and necessary safety precautions:48M.1 -Available for each X-Ray room and in place49M Consumables : minimum level49M.1 -X-ray films 49M.2 X ray fixers available49M.3 X ray developers available

Total points this quarter: (Maximum Available Points: 9)

II.8. OUTPATIENT DEPARTMENT (OPD)

Indicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

50M Staffing50M.1 -Consultations are done by appropriately qualified staff NURSE

(RGN)/DOCTOR51M Guidelines/protocols51M.1 National Malaria guidelines for diagnosis and treatment of

uncomplicated and severe malaria -On wall, accessible to staff and up to date

51M.2 PEP policy and guidelines:-Available in OPD

51M.3 Opportunistic Infection and ART guidelines:-AVAILABLE AND ACCESSILBE in all consultation rooms

51M.4 STI Management protocol:-Displayed in all consultation rooms and up to date

51M.5 IMNCI guidelines:-Available and flowcharts displayed in all consultation areasSUPPLIES

52M Equipment and PEP kit52M.1 Adult weighing scale and Standard Paediatric Weighing (SALTER)

Scale available and functional , height meter, 52M.2 BMI calculator, glucometers and strips, peak flow meter,

ophthalmoscope, otoscope, stethoscope, otoscope, sphygmomanometer, HC meter

52M.3 Functional thermometer and OPD register in place52M.4 PEP kit readily available in the event of a needle stick injury or other

accidentsTotal points this quarter: (Maximum Available Points: 10)

II.9 FAMILY AND CHILD HEALTH (FCH)53M Guidelines/protocols, medicines and equipment53M.1 Focused ANC protocol in ANC care area:

-Available, displayed and up to date53M.2 PMTCT guidelines and charts:

-Available and accessible

136

53M.3 PMTCT medicine:-Available according to guidelines?

53M.4 Standard Paediatric Weighing (SALTER) Scale, length/height board and MUAC tape-Available and functional

54 M Availability of functional OTP equipped with (outpatient therapeutic center at health facility )54M.1 OTP register and case sheet 54M.2 RUTF (ready to use therapeutic food) (adequate for at least three

months: based on previous records/admissions/utilization)54M.3 Trained health worker on IMAM (Integrated Management of Acute

Malnutrition)Total points this quarter: (Maximum Available Points:7)

II.10. EXPANDED PROGRAM ON IMMUNIZATION (EPI)

Indicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators* N.B. Please assess all the indicators requiring opening of the refrigerator at once in order to avoid frequent opening of the refrigerator

Score:1: if all criterion have been met/ recorded0: if all criterion have notbeen met/ not recorded

55M POLICY & GUIDELINES55M.1 -Surveillance line listing and case definitions displayed55M.2 -Updated EPI schedule, and a contingency plan displayed55M.3 -EPI graphs showing trends displayed and staff member is able to

interpret the graphs55M.4 -EPI reference materials: EPI Policy, (e.g. multi dose vial policy

(MDVP) and EPI modules available and easily accessibleSUPPLIES & STORAGE

56M Cold Chain Mechanism: 56M.1 -Fridge with a temperature booklet available and filled twice a day56M.2 -The temperature is within the recommended range of + 2 and+ 8

degrees Celsius (Supervisor should verify functionality of thermometer)57M Availability of vaccines:57M.1 -The following antigens are available: BCG, MR (measles and Rubella),

polio, Penta, tetanus, pneumococcal and rota virus vaccine57M.2 -The physical stock and the amount in the stock cards match (Supervisor

verifies physical stock in the fridge by selecting three different vaccines quarterly)

58M Vaccines storage58M.1 -Correctly stored in fridge with compartments as follows in fridges with

compartments:-Freezing compartment: ice packs well frozen-None freezing compartment: top shelf BCG, OPV, measles -Lower shelf: DPT+HEPB, TT, etc N.B. the new type of refrigerator i.e. Domestic fridge do not have compartments and the live vaccines are stored in the lower tray (colder zone)

137

58M.2 -No expired vaccines58M.3 -The Vaccine Vial Monitor (VVM )status is kept58M.4 -There are readable labels on vials with matching diluents 59M Syringes:59M.1 -The number of syringes available matches the number of vaccines in

the stock cards60M Sharps boxes:60M.1 -Sharps boxes available in immunisation room/corner/area and not more

than 3/4 full)61M EPI accessories: the following EPT accessories should be available and functional61M.1 -Vaccine carriers, cold box, gas regulator, gas cylinder and scissors62M Forms:62M.1 - AEFI investigation forms, case investigation forms for EPI targeted

diseases and vaccine wastage monitoring forms available 62M.2 -Vaccine order forms and stock cards available

Total points this quarter: (Maximum Available Points: 17)

II.11 MATERNITY Paediatric Services ; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER AND NEWBORS and Children

Indicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/ recorded0: if all criterion have not been met/ not recorded

63M Medicines on Emergency tray:Are the following medicines available on the emergency tray and not expired?

63M.1 -IV fluids (ringer lactate, 5% dextrose, normal saline) and giving sets

63M.2 -Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate, calcium gluconate

63M.3 -Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant

63M.4 -At least 10 pairs of sterile gloves available, face mask, specimen bottles

64M PPH kit (please open one kit and check for its completeness)64M.1 -PPH kits available and complete:

please refer to the annex section in the checklists guideline for list of items that should be available in the PPH kit*refer annex section in the checklist guideline for the list of items in PPH kit

65M Eclampsia kit (please open one kit and check for its completeness)65M.1 Eclampsia kit available and complete:

please refer to the annex section in the checklists guideline for list of items that should be available in the eclampsia kit*refer annex section in the checklist guideline for the list of items

138

in Eclampsia kit66M Equipment/supplies for care of newborn and monitoring FHB: Are the following

equipment available?66M.1 -Fetoscope, baby blanket, Baby scale and tape measure66M.2 -Sterile cord clamps/ties for umbilical cord, Eye ointment

( Tetracycline)66M.3 -Neonatal bag and mask, penguin suction, resuscitator and suction

bulb (at least two sets) in a “ready newborn resuscitation” area next to delivery bed

66M.4 Electric heater, and wall clock:67M Obstetric sterilised delivery packs: ( open one pack to see whether all the items are

present and check for expiry date )67M.1 -At least 5 obstetric sterilized delivery standard packs with -2

wrapping towels, 6 drapes, A galipot with 10 swabs, 5 gauze swabs, A receiver, 2 Artery Forceps, Cord Scissor, Episiotomy Scissor, Drying towel for hands, Gown, Cord ties, sanitary pads available

68M Delivery bed:68M.1 -All beds in the maternity ward/delivery room are in good state

(not broken, mattress not torn) and covered with a clean sheet69M Availability of functional equipped KMC ( Kangaroo Mother Care) unit 69M.1 KMC bed69M.2 Heat source69M.3 KMC wrap for baby (mbereko)69M.4 Clothes for baby ( hat, nappy and socks)69M.5 KMC register

Total points this quarter: (Maximum Available Points: 17)

II12 HEALTH INFORMATION MANAGEMENT SYSTEMIndicators

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score: 1: if all criterion have been met/ recorded 0: if all criterion have not beenmet/ not recorded N/A: Not applicable

70M Referral and feedback system: * review referral made in any one month in the last quarter ( if there was no referral made in the selected month, extend the period of assessment to any of the two months in the last quarter)

70M.1 -Standard referral forms (at least 10) available

70M.2 - Referral register available and properly filled ( Applicable only if there was referral in the last quarter)

71M T Series forms and timely reporting : * check the following two items in in any one month in the last quarter

71M.1 -The T Series forms are available and fully completed (T1, T3, T5, T6, T11, and T12)

71M.2 -The T5 and HS3/5completed and sent timely (by the 21st of the following month) for previous months

139

For the following two indicators requiring review of registers and/or reported figures/indicators:Score each register/reported figure as: 1: if all criterion that have been met/ recorded 0: if the criteria has not been met/ not recorded And then give an overall score as shown below:5 Points: if 5 (100%) of registers/reported figures are complete and/or correct3 Points: If 3-4( 60-80%) of registers are complete and/or correct0 Point: if ≤2 (≤40%) of registers are complete and/or correct

72M Completeness and correctness of information in registers :*Randomly select 5 registers to assess the indicators listed below *Please review the annex section of the checklist guideline for the list of registers available in a hospital setting

72M.1 Are the information in each column of the selected registers complete and correct in any one month in the last quarter (Select different registers quarterly)?

R1 R2 R3 R4 R5 Complete registers

Overall score

73M Correctness of reported figures:73M.1 Are the figures reported in any one month of the

last quarter correct according to the HMIS age groups in the T5 and HS3/5?* Randomly select five indicators, verify for accuracy and correctness(selected different indicators quarterly)

i1 i2 i3 i4 i5 Accurate and correct figures

Overallscore

Total points this quarter: (Maximum Available Points: 14)

140

CLINICAL MANAGEMENT PRIORITY AREAS

1C OPD/CONSULTATION AREA

For indicator 1C1: assess by reviewing 5 files of patient who visited the clinic during the day of assessment. If there are no enough records, Score each file as 1or 0 as per the criterion and then give an over score/points for the indicator. If there are no records for review, please assess the indicator by asking the nurse at OPD who is doing the triaging during the day of assessment (( if she/he is able to identify the three group, please give 100% ( 6 points; if not, please give 0 points)For indicator 1C2: assess by asking at one of the nurses on duty during the day of assessment. Score each criterion as 1 or 0 as per the response of the health care provider and then give an over score/point for the indicator.For indicator 1C3: assess by reviewing OPD register and/or TB presumptive register

PATIENT’S RECORDSOr TB Symptom screening1: if all criterion have been met/ recorded

0: if all criterion have not been met/ not recorded N/A: not applicable if there is no record for review

5 records /symptoms (100%): 6 points4 records /symptoms (80%): 4 points3 records /symptoms (60%): 2 points≤2 records /symptom (≤40%): 0 points

1 2 3 4 5 Complete records

POINTS

1C1 Triaging of patients at OPD waiting area during all clinic shift:-Patients are classified into three groups and given due attention accordingly:Assess by reviewing patient files if patients are available at OPD during the day of assessment. If not, assess by asking the nurse on how she/he conducts triaging of patients (her/his answers should match with the points listed below)Emergency signs requiring immediate attention Priority signs (requiring priority in the queue)Non-urgent cases

1C2 % of adult TB screening symptoms correctly stated by health center/clinic OPD provider (Select one provider randomly on shift during the day of assessment) • Assessment: Ask one health care provider at OPD to name criteria for TB testing(please observe whether her/his answers match with the list mentioned below

141

and give score accordingly):1) Weight loss 2) fever for more than 3 weeks 3) cough for more than 14 days 4) Night sweats 5) TB contact exposure.

1C3 % of TB presumptive (TB symptom positive) that have sputum results documented in any month in the last quarterAMBULATORY MANAGEMENT OF TB*Source of data: TB Register*Review TB register and select 5 cases with TB in the any one month in the last quarter. If more than 5 cases found, randomly select 5 cases for each disorder. If the number of cases is not enough extend the search to the last quarter to gather 5 cases for each disorder. If there were less than 5 cases in the last quarter assess the cases found*Write Not Applicable (N/A) if there are no TB cases for review* Indicators 1C4-7 should be assessed at TB clinic and/or OI/ART clinic

1C4 % of TB patients (SS + and SS-) that have HIV test results documented in any month in the last quarter

1C5 % TB patients diagnosed in any one month in the last quarter that are receiving correct treatment with DOTS,

1C6 % TB patients diagnosed in any one month in the last quarter that were TB notified

1C7 % TB patients diagnosed in any one month in the last quarter that TB contact tracing was conductedSUBTOTAL Points – ( Maximum available points:42 )

2C FAMILY AND CHILD HEALTH (FCH)AMBULATORY (ANC, PNC) BEST PRACTICESSource of Data: *ANC register for ANC Best Practise indicators* PNC register/follow up notes for PNC/Postpartum best practise indicators*Assess 10 cases/records in any one month in the last quarter. *If there are no enough cases for review, please extend the review period to a quarter. *If there are less than 10 cases for review after extending the review period to a quarter, assess the available cases/records.

PATIENT’S RECORDS/Registers1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review*Score each case/record as 1 or 0 and then give a score for items per patient record, when applicable.

9-10 records (≥90%): 6 points8 records (80%): 4 points7 records (70%): 2 points≤6

142

* If there are more than 10 cases for review, select 10 cases by using either simple/systematic random sampling*For indicators 2C1.6 i.e. pregnancy test, resampling or extending the review period to a quarter may be required as the number of women with ≤16 weeks of gestations is usually low.*If there are no records for review/the indicator is not applicable in the set up being assessed, please do not assess and not score the indicator; rather write N/A and deduct the available points for the indicator from the maximum available points.

* At last, please write the number of records with complete information as required and give an over score/points as the per the criteria in the right column

records (≤60%):0 points

2C1 ANC BEST PRACTICES 1 2 3 4 5 6 7 8 9 10

Complete records

2C1.1 % of first visit ANC bookings in any one month in last quarter who had documented:BPHeightWeight measurements Fundal height measurements (if pregnancy >16 weeks of gestation)ALL ITEMS PER PATIENT RECORD

2C1.2 % of first visit ANC bookings in any one month in last quarter who received the standard laboratory test according to the ANC guideline: Blood group and RHHIV testHaemoglobinRPR (Rapid plasma regain for syphilis diagnosis)Urine analysisALL ITEMS PER PATIENT RECORD

2C1.3 % of first ANC visits in any one month in last quarter who received IPTp (if pregnancy >16 weeks of gestation if women living in malaria area)* Write Not-Applicable (N/A) if it is not a malaria endemic area

2C1.4 % of first ANC visits in any one month in last quarter who received TT vaccine

2C1.5 % of first ANC visits in any one month in last quarter who received iron supplementation

2C1.6 % of first visit ANC bookings with ≤ 16 weeks of gestation in any one month in last quarter who had documented pregnancy test results

2C2 POSTNATAL AND/OR POSTPARTUM BEST PRACTICES*Source of data: PNC register and/or post-partum follow up notes/sheet/registerIndicator 2C2.4 needs a separate sampling since

1 2 3 4 5 6 7 8 9 10

Complete records

143

the denominator is women who are 6 weeks post- partum during the assessment period.

2C2.1 % PNC visits in any one month in last quarter documenting assessment for the following conditions of the infant:General condition of the infant;NAD recorded if abnormality was not detectedALL ITEMS PER PATIENT RECORD

2C2.2 % PNC visits in any one month in last quarter documenting assessment for the following conditions of the mother:General condition, Pulse rate, B/P and temperatureNAD recorded if abnormality was not detectedALL ITEMS PER PATIENT RECORD

2C2.3 % PNC visits in any one month in last quarter documenting infant feeding (BF) status (exclusive, mixed or not BF)

2C2.4 % women post-partum counselled and offered any of the modern FP method (below)at follow up PNC visit within 6 weeks of delivery in any one month within the last quarter( this indicator should be assessed at 6 weeks post-partum visit)POP (progesterone-only contraceptive safe with BF)injectable, ImplantIUCDTubal ligationDeclineSUBTOTAL: ( Maximum available points: 60)

3C MATERNITY WAITING HOME

Follow up of pregnant mothers in maternity waiting home *Write Non-Applicable (N/A) in clinics without maternity waiting homes and/or if there are no mothers in maternity waiting homes during the assessment period and deduct the available points for the indicator from the maximum available points.*If there are less than 5 mothers in the maternity waiting home,, assess indicator with mothers available during the day of assessment

PATIENT’S RECORDS1: All criterion have been met/ recorded): 0: if all criterion have not been met/ recordedN/A: if there are no mothers in the

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

144

maternity waiting home

ANC Best Practices: follow up of pregnant mothers in maternity waiting home

1 2 3 4 5 Complete record

POINTS

3C1 % of mothers in maternity waiting homes monitored for BP, FHR, and assessed for danger signs daily*Source of data: ANC cards of pregnant mothers*Danger signs: vaginal bleeding, headache/blurred vision, fetal movement, sudden release of water from vagina* if there was no danger signs, it should be recorded as no danger signsSUBTOTAL; ( Maximum available points: 6)

4C HIV–PMTCT

Source of data: ANC, ART, Delivery and DNA PCR register*Review ANC register and select 5 newly identified HIV women for indicator 4C1. *Review ART/ANC register to identify pregnant women initiated on ART before 6 months or indicator 4C2*Review delivery register and select 5 HIV exposed new-borns for indicators 4C2-4C4 in the last quarter. *If more than 5 cases found, select 5 cases using simple/systematic random sampling for each condition. If less than 5 cases in the last quarter, assess all the cases found. *N/A (Not Applicable) if there are no HIV+/HIV exposed cases for review and deduct the available points for the indicator from the maximum available points

PATIENT’S RECORDS

1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%)2 points≤2 records(≤40%):0 points

1 2 3 4 5Complete records

4C1 % NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE SAME DAY in the last quarter*Source: ANC and ART register

4C2 % of HIV+ women retained on ART 6 months after initiation in ANC in the last quarter*Source: ART register

4C3 % of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of birth in the last quarter*source of data: delivery register, PNC register and DNA PCR register

4C4 % of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks of age and received results within one month in last quarter *Source of data: DNA PCR register

4C5 % of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 DAYS OF RECEIPT OF RESULTS*Source of data: DNA PCR register and ART register

145

SUBTOTAL: ( Maximum available points: 30)

5C AMBULATORY MANAGEMENT OF DIARRHEA, PNEUMONIA, MALARIA and Severe Acute Malnutrition in CHILDREN*Source of data: OPD/ IMCI register/CMAM register*Review OPD/ IMCI register and select 5 cases with pneumonia, 5 cases with diarrhea and 5 cases with malaria in the last month. If more than 5 cases found, randomly select 5 cases for each disorder. If the number of cases is not enough extend the search to a quarter to gather 5 cases for each disorder. If there were less than 5 cases in the last quarter assess the cases found*Not Applicable (N/A) if there are no cases for review

PATIENT’S RECORDSYES (all criterion that have been met/ recorded):

1No (if the criteria has not been met/ not recorded) :

0NA (not applicable): N/A

5 records (100%): 6 points4 records (80%): 4 points3 records (60%): 2 points≤2 records(≤40%): 0 points

1 2 3 4 5 Complete records

POINTS

5C1 % children treated as outpatient for pneumonia in any one of month in last quarter who were correctly assessed *Source of data: OPD/ IMNCI registerAbsence of general danger signs recorded: able to drink/feed, vomiting, consciousnessDuration of fever and cough/difficult breathing and child’s age recordedRespiratory rate, and presence/absence of chest in drawing, stridor and wheezing recordedClassified CorrectlyALL ITEMS PER PATIENT RECORD

5C2 % children correctly treated as an outpatient for (ambulatory) pneumonia in any one of month of the last quarter among those correctly assessedTreatment: Oral Amoxicillin 50mg/kg divided thrice per day x 5 days; caretaker counselling and follow up or admitted into hospital

5C3 % children with diarrhoea correctly assessed for signs of dehydration), persistent diarrhoea and dysentery in any one of month of the last quarter Assessment of dehydration: Using IMNCI guidelines (Integrated Management of Neonatal and Childhood Illnesses) IMNCI Flow diagram available and applied,Duration of diarrhoea and presence of blood recordedGeneral condition of the child recorded: lethargy, consciousness and/or restless or irritabilityPresence of sunken eyes, drinking status ( thirsty/drinking eagerly or un able to drink/drink poorly) and skin pinchClassified correctly

146

ALL ITEMS PER PATIENT RECORD5C4 % Children correctly treated as an outpatient (ambulatory) for

diarrhoea in any one month of the last quarter among those correctly assessedTreatment : ORS, Zinc supplements and continued feeding and advise when to return

5C5 % children diagnosed with malaria that have RDT + or laboratory confirmation in any one month of the last quarter

5C6 % Children with uncomplicated malaria correctly treated according to national guidelines in any one month of the last quarterTreatment: ARTEMETHER (20mg)-LUMEFANTRINE (120mg)(C0ARTEMETHER) during 3 days (See treatment protocol in appendix 2 of the checklist guideline)

5C7 % Children with severe malaria correctly treated according to national guidelines in any one month in the last quarter Treatment: PARENTERAL ARTESUNATE IS THE MEDICINE OF CHOICE at a Dose of 2.4mg/kg body weight for 7 days(See treatment protocol in appendix 2 of the checklist guideline)

5C8 % of 6-59 months old children with un complicated severe acute malnutrition (SAM) who were managed as per the national protocol in any month in the last quarterA 6 to 59 months old child with any one of the following criteria is classified as SAM :Weight for height <-3SD (WHO)MUAC <115mmMUAC <125mm and HIV positiveBilateral pitting oedemaOut Patient management of SAM:RUTFRoutine MedicineHealth and nutrition counseling and continued follow up*see annex section of checklist guideline for treatment details

SUBTOTAL: (Maximum available points: 48)

6CMATERNITY SERVICES; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER AND NEWBORNDELIVERY BEST PRACTICES*Source of data: delivery register and partographs *Review delivery register and randomly select 10 deliveries in any month in last quarter. If the number of deliveries is not enough extend the search to the last quarter to gather 10 deliveries. If there were less than 10 deliveries in the last quarter assess the cases found. If there were more than 10 cases in a month/quarter then randomly select 10 deliveries and assess the partographs for the deliveries selected to assess the following indicators

9-10 records (≥90%): 6 points8 records (80%): 4 points7 records (70%): 2 points≤6 records (≤60%): 0 points

147

* Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

1 2 3 4 5 6 7 8 9 10 Complete records

6C1

% deliveries performed by skilled personnel in any one month in the last quarter •Assessment: Identification of the nurse/ midwife by names in the delivery register

6C2

% women who delivered in the facility monitored using partographs as per criteria *Please take into account the cervical dilatation at admission while assessing the following itemsAssumption: Partograph should be opened for all pregnant women in labor unless they come with fully dilated cervix with head visible. Women who present in advanced labor should have at least one measurement of the items mentioned below. If urine analysis was not done, the reason should be noted/documented i.e. lack of supplies/women did not produce urine.Fetal heart tones plotted every 30 minutesState of membranes every 4 hours presence/absence meconiumDescent of presenting part every 4 hoursContractions plotted every 30 minutesMaternal BP every 4 hoursMaternal pulse every 30 minutesMaternal temperature every 4 hoursUrinalysis documented at admissionALL ITEMS PER PATIENT RECORD

6C3

% total births in any one month in the last quarter documenting administration of immediate postpartum oxytocin 10 units IM (within one minute of delivery of baby) (AMSTL: Active management of third stage of labour)

6C4% births with placental status documented at birth in any one month in the last quarter•Assessment: complete or ragged, retained placenta

6C5% newborns BF within one hour of birth in any one month in the last quarter•Assessment: Time of BF initiation documented

6C6 % newborns received Vitamin K in the any one month in the last quarter

6C7 % newborns received eye care (Tetracycline) in the any one month in the last quarter

6C8% newborns received first vaccination (BCG) in the any one month in the last quarter*source of data: Delivery and PNC register

6C9% women delivered monitored in early post-partum period (4th stage) per guideline (birth to discharge) in the any one month in the last quarter*Source of data: partographs and/or post-partum follow up notes/sheets/registerVaginal bleeding, at least every 30 minutes 1st 2 hrs

148

after birth and then four hourly until dischargeUterine contraction at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeBP at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargePulse at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeTemperature at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeALL ITEMS PER PATIENT RECORD

6C10

% newborns monitored in early post-partum period per guideline (birth to discharge) in the any one month in the last quarter*source of data: partographs and/or post-partum follow up notes/sheets/register Temperature documented at least every 30 minute first 2 hours after birth then four hourly until dischargeRespiratory Rate documented at least every 30 minute first 2 hours after birth then four hourly until dischargeBreast feeding status documented at least every 30 minute first 2 hours after birth then four hourly until dischargeColour documented at least every 30 minute first 2 hours after birth then four hourly until dischargeALL ITEMS PER PATIENT RECORD

6C11% facility births seen for day 3 PNC visit in any one month in the last quarter*Source of data: PNC register

SUBTOTAL: ( Maximum available points: 66)

7C OBSTETRIC, NEONATAL and Childhood COMPLICATIONS

*Source of data: in patient and/or delivery registers.*Review in patient and/or delivery register in maternity ward and randomly select 5 cases of patients with the following conditions in the last quarter:PROM; PPH; Postpartum sepsis; and severe Pre-eclampsia/eclampsia, *Review in patient and/or delivery register in paediatric and randomly select 5 cases of patients with the following conditions in the last quarter:Neonatal asphyxia; neonatal sepsis; low birth weight and severe acute malnutrition*Then review their records and assess whether they were managed as per the national protocol..

PATIENT’S RECORDS1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

149

* Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

review

1 2 3 4 5 Complete records

POINTS

7C1 % women with prolonged labor or Rupture of Membranes and without chorioamnionitis that were administered antibiotics as per protocol in the last quarterTreatment with oral erythromycin (or amoxicillin) if ROM > 6 hours or active labor > 12 hours without signs of chorio-amnionitis; first dose antibiotic *Review in patient register and select those in which rupture of membrane documented > 6 hours (at any time in course of labour and delivery) or activelabour >12 hours (at any time) without documentation of other signs of maternal sepsis in the last quarter(maternal fever or foul-smelling discharge) is documented

7C2 % women with PPH managed per guidelinelast quarter*Review in patient register and select 5 cases of PPH fulfilling the following criteria:PPH documented (EBL > 500 cc or VB and tachycardia > 100 bpm or hypotension SBP < 100 or DBP <50) and check whether they were managed according to the guideline and check whether the following three items listed below were done for each identified case

•Assessment: See annex section of checklist guideline for specific audit criteria and management of PPH1. PPH Cause documented (atony, tear, retained placenta, other)2. secure two IV lines with two 16 G cannulas or any large size available, and run normal saline (NS) or ringer lactate (RL)3. Management according to the cause:-Uterine atony: 60 IU Oxytocin in 1L NS or RL solution at 60 drops/minute until uterus is firmly contracted or-Retained placenta: controlled cord traction. If failed, manual removal or-Vaginal/cervical laceration: suturedALL ITEMS PER PATIENT RECORD

7C3 % women with signs of intra- or post-partum sepsis :fever temperature ≥38⁰C, foul-smelling discharge, ≥38⁰C) or Membranes were ruptured for ≥18 hours before delivery that were treated per standard in last quarter

*Treatment with triple antibiotic given IV:Benzyl penicillin 5 mega units IV stat, then 2.5 mega units IV hourly, Metronidazole 500mg IV 8 hourly and Chloramphenicol 500mg IV 6 hourly until patient is fever-free

150

for 48 hours ** see annex section of hospital checklist guidelines for maternal sepsis(chorioamnionitis/puerperal sepsis) case managementALL ITEMS PER PATIENT RECORD

7C4 % pregnant women with severe pre-eclampsia and/or eclampsia managed according to the guideline in last quarter

Severe Pre-eclampsia: -Diastolic BP 100mm HG or more-proteinuria 3+ or more

Eclampsia: -Unconsciousness or Convulsions (fits)-dBP 110 mmHg or more-Proteinuria 2+ or more in a pregnant woman or a woman who has recently given birth

-Check whether the following three items listed below were at least done for each identified case:*refer Annex section of the checklist guideline for details on the management of severe pre-eclampsia/eclampsia1- blood pressure checked every 5 minutes until diastolic BP is90 - 100mmHg2-Blood pressure monitored (if diastolic blood pressure (dBP) is ≥110 mmHg, Nifedipine 10 mg provided. If inadequate response after 20 minutes following first dose:Repeat 10mg dose orally every 20 to 30 minutes until adequate dBP response is achieved, to a maximum of 40 mg given. Then 10-20 mg orally every 4-6 hours to maintain dBP 90-100 mmHg orHydralazine 5mg IV slowly every 20 minutes for a maximum of 20 mg* applicable only if dBP was ≥110mm Hg3-Magnesium sulphate 20% solution, 4gm IV over 5 minutes given. Followed promptly with 10g of 50% magnesium sulphate solution, 5gm in each buttock as deep IM injection with 1 ml of 2% lignocaine in the same syringe.4. Deliver within 12hours for Severe Pre-Eclampsia and 6 hours for EclampsiaALL ITEMS PER PATIENT RECORD

7C5 % of neonates who had an APGAR score of ≤5 in the first minute after birth for whom resuscitation was initiated in last quarter*Source of data: partographs and notes*refer checklist guideline for details

7C6 % neonates who had an APGAR score of ≤5/10 at 1 minute after delivery and were successfully resuscitated i.e. 5th

151

minute APGAR score ≥6/10 in last quarter *refer checklist guideline for details

7C7 % neonates with possible serious bacterial sepsis managed per standard in last quarter•Assessment of possible neonatal sepsis: Review all cases of newborn sepsis (pre-discharge or re-admitted to paediatric ward) in in patient register in last quarter; and select 5 records for review that meet any of following probable sepsis criteria:-if documented temperature >380 C or < 250 C (and not warming);- RR > 60 or <30 breaths per minute; -chest in-drawing or convulsion; -no movement on stimulation;- poor feeding/sucking or -umbilical redness, *see annex section of checklist guideline criteria for chart audit and for treatment details

7C8 % of low birth weight (LBW) newborns admitted to KMC unit in the last quarterDefinition of LBW: infant with birth weight lower than 2500gregardless of gestational age Criteria for providing Kangaroo Mother Care (KMC): LBW neonates weighing >1500 and <2500g Baby‘s condition is stable to permit KMCThe mother is in good health to start KMC

SUBTOTAL: ( Maximum Available Points: 48)

8C PAEDIATRIC WARD

PAEDIATRIC BEST PRACTICES

*Select 5 cases from registers (randomly if there are more than 5 cases) and then obtain and review patient files

* Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

PATIENT’S RECORDS1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

1 2 3 4 5 Complete records

POINTS

8C1 % hospitalized patients with correctly completed admission medical record any one month in the last quarterAdmission medical record for hospitalisations available and documenting at a minimum:Vital signs (RR, HR, BP, temperature);history of illness;

152

physical exam;laboratory/radiology results (if applicable);admission diagnosis and treatmentALL ITEMS PER PATIENT RECORD

8C2 % Written record of administration of patient medications up to date any one month in the last quarter

8C3 % hospitalized patients with documentation of vital signs every 6 hours and every half hour for critical patients any one month in the last quarter

8C4 % hospitalized patients with documentation of daily progress note by doctor any one month in the last quarter.SUBTOTAL - ( Maximum available points 24 )

9C PAEDIATRIC COMPLICATIONS: PNEUMONIA, DIARRHEA,Review 5 cases in registers with the diagnosis of pneumonia, and 5 with diagnosis of diarrhoea. * increase the assessment period to a quarter if there are less than 5 cases in the selected month* Not Applicable (N/A) if there are no cases and deduct the available points for the indicator from the maximum points.

PATIENT’S RECORDS1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

1 2 3 4 5 Complete records

POINTS

9C1 % hospitalized children treated for pneumonia any one month in the last quarter who were correctly assessed for pneumonia in any one month in the last quarterVitals: child’s age recorded; weight recorded; Temperature, respiratory rateSymptoms & duration recorded (at a minimum absence/presence and duration of fever, cough, ability to drink/feed)Pulmonary exam results recorded stridor, wheezes, chest in-drawingALL ITEMS PER PATIENT RECORD

9C2 % hospitalized children treated correctly (all criteria met) for pneumonia among those correctly assessed in any one month in the last quarter•Assessment: see checklist guideline for criteria IV ceftriaxone 50 mg/Kg per day OR oral amoxicillin 50 mg/Kg divided TID x 7 days (if taking fluids and no severe respiratory distressAntipyretic for fever controlOxygen (per nasal cannula or paediatric mask) if : saturation < 94% Or breathing (intercostal retractions and/or respiration rate > 50, if 2months to 1 year; > 40 if 1 yr or older)

153

9C3 % hospitalized children treated for diarrhoea in any one month in the last quarter correctly assessed for signs of severe dehydration

Was general condition (abnormally sleepy or difficult to wake up, restless and irritable, or well and alert) assessed and documented?Were eyes checked for dehydration signs and documented (such as sunken and dry, sunken, normal); Was thirst assessed by offering fluid (drinks poorly or not able to drink, drinks eagerly - thirsty, drinks normally not thirsty)?Was skin turgor assessed by pinch of abdomen or thigh (goes back very slowly - longer than 2 seconds? ALL ITEMS PER PATIENT RECORD

9C4 % hospitalized children treated for diarrhoea correctly (all criteria met) in any one month in the last quarter among those correctly assessed. See checklist guideline for criteria If able to drink: Low osmolarity Oral Rehydration Solution (continue breastfeeding and feeding)If unable to drink: NS (or Ringer Lactate if NS not available) IV Or If unable to drink and unable to star IVF, administer ORS via NGTZinc 10-20 mg/kg/day x 10 days given

ALL ITEMS PER PATIENT RECORD

SUBTOTAL - ( Maximum available points 24 )

154

10 C POST OPERATIVE INFECTION CONTROLReview operating Theatre register and randomly select 5 patients who have undergone major surgical procedures in the last quarter and review their files to assess indicators10C1-2*If there are less than 5 cases, assess the indicators for the available number of cases. But if there are more than 5 cases for review in the last quarter, select 5 cases for review using either simple random/systematic random sampling** Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

*See the WHO surgical safety checklist and the definitions of surgical site infections in annex section of the checklist guideline

PATIENT’S RECORDS1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

1 2 3 4 5 Complete records

10C1 Surgical safety checklist utilization rate: -% of patients with major surgical procedures on whom safe surgical checklist was completed in the last quarter

10C2 Surgical site infection: -% of post major surgical procedures free of surgical site infections in the last quarterSUBTOTAL: ( Maximum available points: 12)

QUALITY PROCESS MANAGEMENT11C Quality Improvement/Assurance:

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators* Assess asking the QI focal person and/or QIC chairperson

Score:1: if all criterion have been met/ recorded 0: if all criterion havenot been met/not recorded

11C1 Presence of a Quality Improvement/Assurance Committee/team (QIC) with clear structure and responsibilities:-Does the QIC have terms of reference (ToR) with the following components: Structure & leadership, known responsibilities, meeting frequency, reporting system and list of committee members?-Are all service areas, including administration, represented in the committee?

11C2 Presence of quality improvement plan:-Is a quality improvement (QI) plan with the following items present, as part of the overall plan of the hospital?-Action plan for improvement-Targets-Areas for improvement

11C3 Quarterly committee meetings and reports to PMDs:-Are reports on QI activities sent to PMDs quarterly? (Check copy of report)?- Are processes like waiting time, appointment system, and patient flow discussed during the meeting? (Check in the minute)-Is the QIC meeting quarterly? (Check for the presence of minutes)

155

- Are status of QI plan and other improvement plans discussed during the meeting? (Check in the minute)

11C4 Quarterly review of patient files: * Check in the minute and/report of QIC meetings.Does the QIC review at least 10 patient files from at least each of three service areas and assess the following items? :-Completeness and correctness of the information in the reviewed patient files-Comprehensive patient management given to patients in the reviewed files-Consistency of information between the reviewed patient files and registers

11C5 Feedback and action on quality supervision checklist assessment findings: -Does the QIC receive feedback from the supervision? (Check for the presence of copy of feedback of previous quarter supervision)-Has the QIC developed action plan for improvement to address the identified gaps from the quality supervision? ( Check for the presence of action plan for improvement)

11C6 Feedback mechanism from ClientsQuarterly client satisfaction surveys:-Does the QIC have survey tool? ( Check for the presence of the survey tool)-Does the QIC conduct quarterly surveys? ( check for copies of filled questionnaires )-Are survey analysis reports available?( check for the presence of analysis report)Suggestion box:-Does the facility have labelled suggestion boxes?-Is instruction on how to use the boxes posted on or above the boxes?- Does the facility analyse the findings? (Check for the presence of analysis report of previous quarter)

11C7 Clinical audit and/or maternal-perinatal mortality audit meetings:-Does the hospital conduct audit meetings or maternal mortality review meetings at least once in a quarter? ( check for minutes/report)-Is guideline for audit/mortality meetings available?( check for the presence of guidelines for clinical audit meetings)Total Points- ( Maximum available points: 22)

VERIFY THAT ALL QUESTIONS ARE FILLED IN Supervisor thanks the staff

Signature:

PHE ………………………………………………………..

DMO/MED. Sup/MATRON…………………….........................

Counter verification………………………………………...ASSESTMENT FEEDBACK

I. Summary Comments on Results. Please note any trends, problems, exceptional or creative

156

changes and results that you saw during your visit assessment

II. Noteworthy Improvement. Please note any improvement and include a few details of what they are doing and why it is unique

III. Difficulties/ Challenges. Please note any assessment area that seem to be having an especially difficult time in improving. Please include a few details about the problem, how it might be solved, and who might be involved

IV. Recommendations and suggestions for improvement. Please note that the feedback is more effective when emphasizes features of the clinical task to be performed (e.g. specifies a target performance, presents information on how target performance can be attained, and address change in performance observed since previous feedback

V. Follow up. Please review previous recommendations provided and assess if they were followed or not

157

HEALTH CENTER/CLINIC QUALITY SUPERVISION CHECKLIST

February 2016

158

HEALTH CENTER/CLINIC QUALITY SUPERVISION CHECKLIST

Province: ______________________________________________________________________________District: ______________________________________________________________________________Health facility: ___________________________________________________________________________Number of beds: ________ Catchment area population: __________________________________________Date of supervision: ______________________________________________________________________

Name of supervisors and designationNo. Name of supervisor Designation123456

Facility Staffing

STAFF Establishment In post

Vacant Duration of vacancyNumber of Months Number of

YearsRGNRGN with midwifery PCN EHTNurse aidesGeneral handsOther (Non-medical staff or unqualified staff)

159

ASSESSMENT SUMMARY

I. STRUCTURAL SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

Structural indicators in general compound of the clinic

20 11

Structure indicators in OPD, Labour ward/Maternity and Inpatient/Observation departments

29 8

TOTAL 49 19

II, MANAGEMENT & PLANNING SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

Administration, finance and planning

14 6

Community services 5 3Environmental health services

9 4

Infection control and waste management

10 5

Pharmacy 32 4Outpatient department (OPD)

12 4

Extended Program Immunization (EPI)

17 8

Maternity ward 11 7Observation/in patients services

1 1

Health Information Management System

14 4

TOTAL 125 46

III. CLINICAL MANAGEMENT SECTION

Available 0Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

1C.OPD/ consultation area 18 32C. ANC-PNC Best practices 60 103C. Maternity waiting 6 1

160

home4C. HIV-PMTCT 24 45C. Ambulatory management diarrhoea, pneumonia

488

6C. Delivery best practices 66 117C. Management obstetric complications 42 7TOTAL 264 44

161

STRUCTURAL SECTION

Indicators

I.1. Structural indicators in the general compound of the clinic

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/recorded0: if all criterion have not been met/ not recordedN/A: not applicable

1S Outside appearance (when arriving at the clinic ):

1S.1 -Visible sign post

1S.2 -Fence/wall: in good condition and gate with clearly written emergency contact number/s and service hours

2S Maintenance of the ground:

2S.1 -Ground clean with no litters and/or stagnant water and the grass cut

2S.2 -No waste and dangerous objects in courtyard such as needles – syringes –gloves – used cotton wool, etc

3S Outside appearance of buildings:

3S.1 -External appropriate wall finishing (painting/bricks/rough plastering)

3S.2 -Roof intact, presence of well-maintained rain gutters

4S Availability of a garbage bin in ground:

4S.1 -Bin with lid accessible to clients and not more than ¾ full

5S Presence of sufficient and clean latrines/toilets:

5S.1 -Minimum of 3 toilets: 1 male, 1 female, 1 staff offering privacy

5S.2 -Recently clean without visible fecal material and without smell

5S.3 -Hand washing facility with soap available near the toilets

6S Lighting system

6S1 -Electricity for 24 hours a day, and 7 days a week:Source of electricity: ZESA with backup system of either generator or solar energy and/or inventors.

7S Firefighting System:7S.1 -Fire extinguishers available, accessible, functional and

serviced7S.2 -A clearly marked firefighting assembly point and procedure

( well understood by a staff member): Ask staff member on firefighting procedure

8S Waste Management system: Fenced and lockable disposal area

162

Indicators

I.1. Structural indicators in the general compound of the clinic

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score:1: if all criterion have been met/recorded0: if all criterion have not been met/ not recordedN/A: not applicable

N.B. Ottoway pit and rubbish pit are not applicable for facilities where the rubbish is collected by city municipality

8S.1 -Ottoway pit with lid, not full and functional

8S.2 -Incinerator ( lined with bricks) and functional

8S.3 -Rubbish pit: 2-3 metres deep without infected non decomposable objects (non – biodegradable)

9S Bathing facilities and waste water drainage system

9S.1 Appropriate drainage of waste water (presence of septic tank or connected to local sewage)

9S.2 Shower with either running water or container of at least 100 litres

10S Referral service-Availability of means of communication:

10S.1 -Radio or mobile phone with airtime or landline for communication

11S Transport plan for emergency referrals:

11S.1 -Transport plan for emergency referrals and/or contingency plan (in case of unavailability of ambulances from hospitals) included in the planTOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 20)

Indicators

I.2. STRUCTURAL Indicators In OPD, Maternity/Labour ward and Inpatient/Observation Departments

Score 1: if all criterion have been met/recorded0: if all criteria have not been met/not recorded

N/A: Not Applicable*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

OPD

12S Good conditions in waiting area, meeting minimum standards:

12S.1 -With sufficient benches and / or chairs (according to average daily attendance calculated using attendance over a week)

12S.2 -Adequate ventilation of waiting area:

163

Indicators

I.2. STRUCTURAL Indicators In OPD, Maternity/Labour ward and Inpatient/Observation Departments

Score 1: if all criterion have been met/recorded0: if all criteria have not been met/not recorded

N/A: Not Applicable*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

OPD

If open space: with a shade or roof supported by brick or metal pillars orIf closed space: windows should measure at least 1/10 of floor area and at least ½ of window area should be openable.

13S Displaying of free service sign and/or service fee charges:

13S.1 -Free services sign or service fee charges (if any) displayed and easily visible for patients

13S.2 -Displayed in local vernacular?

OPD Labour ward

Inpatient

14S Inside appearance of building and its cleanliness14S.1 -Walls, Floors and Ceiling / roof clean and in good condition (no

leaks, cobwebs) and floors polished ( when applicable)14S.2 -Doors with locks and closing properly14S.3 -Curtains on windows or non- transparent glass and screen

14S.4 -Clean and with good ventilation without bad smell?

OPD Labour ward

15S Availability of waste management supplies and their utilization:

15S.1 -Bin with plastic liners + puncture proof sharps containers available and not more than ¾ full

16S Hand washing facilities:

16S.1 -Hand washing facilities with soap available at accessible points for outpatient and/or a functional water point and/or at least 50 litres reserve with soap available in delivery room

Labour ward17S Delivery bed:-

17S.1 -In good state (not broken, mattress not torn) and covered with a clean sheet and Macintosh

Labour ward Inpatient

164

Indicators

I.2. STRUCTURAL Indicators In OPD, Maternity/Labour ward and Inpatient/Observation Departments

Score 1: if all criterion have been met/recorded0: if all criteria have not been met/not recorded

N/A: Not Applicable*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

OPD

18S Availability and status of furniture:

18S.1 -Beds, mattresses (covered in plastic), bed sheet and bed side lockers available and in good state

18S.2 -Mosquito nets available and in good state? (In malaria endemic areas)?N/A in non- malaria endemic area

19S Access to drinking water and space between beds:

19S.1 -Safe drinking water is accessible

19S.2 -Sufficient space between beds (at least 1m between beds)

\ TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 29)

MANAGEMENT AND PLANNING: STAFF, POLICY, GUIDELINES, MEDICINES & SUPPLIES & Vaccines

Indicators II.1. ADMINISTRATION, FINANCE AND PLANNING*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score: 1: if all criterion have been met/recorded 0: if all criteria have not been met/ not recorded

1M Mission statement, vision, values and patient charter

1M.1 -Displayed in accessible area and easily readable ( should be health facility specific)

2M Catchment area map, spot map, monitoring graphs, demographic data and list and mapping of community based workers:

2M.1 -Catchment area maps with current catchment population target population for services calculated correctly and displayed

2M.2 -Up to date monitoring graphs for different services showing trends displayed and health care providers know their interpretations ( ask the health care provider on duty)

2M.3 -Up to date list of community based workers (VHWs, HBC givers) showing those that are active available and displayed

3M Finance, accounting and procurement

3M.1 -Bank statements, receipts, invoices, etc available and filed in clearly labelled files

165

3M.2 -Procurement procedures being followed while purchasing items (Ask for one purchased item and check whether it was purchased according to the procedure)

3M.3 -Management book, inventory/asset register, maintenance book available and up to date

4M Staff duty roster, current practising certificates (nurses and EHT), staff leave calendar and clock in register:

4M.1 -Staff duty roster, staff leave calendar and clock in register complete, up to date and displayed on the wall where all staff can see

4M.2 -Current practising certificates for nurses and EHTs available in the HR files

5M Documentation of activities/ Operational Plan

5M.1 -Staff minute book/file and HCC meetings available, well filed and up to date

5M.2 -Quarterly review and annual/operational plan and annual progress report available and up to date

6M Health education plan/ diary, and reports at the facility

6M.1 -Health education plan/diary with clear topics, target audience (in which number of audience are clearly stated and disaggregated by sex) and dates when they will be presented available

6M.2 -Topics related to current health problems

6M.3 -Report/s on health education session delivered available

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 14)

Indicators II.2 COMMUNITY SERVICES

*Please give a score for each of the criteria under each indicator as per the criteria in the right column*N.B. The items highlighted in bold are the indicators

Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded

7M School health programme:

7M.1 -School health programme available: check for the availability of the plan

7M.2 -Reports and data on activities conducted available

8M VHW, HBC activity reports:

8M.1 -Monthly reports and minutes of meetings available

9M Domiciliary visits to patients:

9M.1 -Reports on visits available in register

9M.2 -Visits categorized according to patients’ condition (e.g. chronically ill,

166

PNC and ANC)TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 5)

II.3 ENVIRONMENTAL HEALTH SERVICES

10M Participatory community health activities

10M.1 -Availability of plan

11M Hygiene education activities

11M.2 -Report of any sessions of education conducted

12M Epidemics and disease surveillance:

12M.1 -Contact tracing forms, and disease surveillance protocols available

12M.2 -Spot map showing recent or suspected out breaks with clear markings displayed

12M.3 -Follow ups and contact tracing (look in registers) on possible or confirmed outbreaks made: Compare reports with weekly statistics (RDNS)

13M Inspections for public premises (including clinics and hospitals) and trading places:

13M.1 -Inspections done (Check availability of inspection reports)

14M Water and sanitation activities:

14M.1 -Coverage statistics displayed

14M.2 -Community health clubs formed

14M.3 - Were water tests conducted: check for reports

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 9)

Indicators II 4. INFECTION CONTROL AND WASTE MANAGEMENT Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/not recorded

15M Infection control guideline:

15M.1 -Available and staff members know about it: Ask a staff member about it

16M Sterilisation of instruments:

16M.1 -Functioning steam steriliser

16M.2 -Guidelines for sterilization of instruments available

167

16M.3 -Sensitive tape available on sterilized packs and cords not used to tie packs

17M Hygienic and aseptic conditions in wound dressing and injection room:

17M.1 -Bench and foot rest covered with macintosh

17M.2 -Labelled Bins for infected and contaminated objects with lid, plastic lining and foot pedal available and not more than ¾ full

17M.3 -Puncture proof Sharps boxes well positioned and not more than ¾ full

18M Protective clothing and disinfectant use by cleaners:

18M.1 -Cleaners have appropriate protective clothing (Heavy duty gloves, Uniforms, Dustcoats, Gumboots, Plastic Apron, Face Mask)

18M.2 - Cleaners know how to appropriately use disinfectants? Soap with water for general cleaning and 1 part jik to 4 parts for spillages and/or depending on the concentration of the bleach i.e. blood and mainly body fluids (after containing and cleaning the spills): Ask the available cleaners during the day of assessment

19M Appropriate dressing of consulting staff:

19M.1 -Dressed with clean standard uniform (princess liner, Brown lace up shoes), and functioning nurses watch

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 10)

Indicators II 5. PHARMACY (MEDICINES AND SUNDRIES STOCK MANAGEMENT)

Score: 1: if all criterion have been met/ recorded 0: if the criterion have not been met/ not recorded

20M Stock Management:20M.1 -Monthly physical counts conducted with min, max and emergency

order levels recorded and updated in stock cards20M.2 -The physical stock level corresponds with that on the stock card:

supply on stock card corresponds with physical count, (use sample of three medicines)

20M.3 -Staff completes and send MIS forms to district each month (check for copies remaining at institution)

21M Storage of drugs:21M.1 -Stored correctly in a locked secured storeroom (e.g burglar bars on

windows and doors)

168

Indicators II 5. PHARMACY (MEDICINES AND SUNDRIES STOCK MANAGEMENT)

Score: 1: if all criterion have been met/ recorded 0: if the criterion have not been met/ not recorded

21M.2 -Clean place, well ventilated with cupboards, labelled shelves, no incident light

21M.3 -Medicines stored in alphabetical order also observing the First Expiry First Out rule

22M Expired products:

22M.1 -Expired Medicine Register available

22M.2 -No expired products in stock: supervisor verifies randomly 3 medicines and 2 consumables (check stock cards)

23M VEN Medicines (according to EDLIZ) adequately stocked 3-6 months’ supply23M.1 Doxycycline capsules 100mg23M.2 Ciprofloxacin tablets 500mg23M.3 Metronidazole tablets 200mg Oral23M.4 Diazepam injection 5mg/ml23M.5 Benzathine Penicillin injection23M.6 Benzyl Penicillin 23M.7 Amoxycillin suspension 125mg/5ml ( dispersible tablets)23M.8 Ferrous sulphate tablets/Folic Acid23M.9 Zinc Sulphate tabs23M.10 Paracetamol 500mg tablets23M.11 Paracetamol syrup or dispersible tablets23M.12 Dispensing envelopes23M.13 Latex gloves23M.14 Oxytocin 10IU/ml Injection, 1ml Ampoule.23M.15 Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule23M.16 Gentamycin 40mg/ml Injection, 2ml Ampoule23M.17 Artemether 20mg + Lumefantrine 120mg Tablets, 24's23M.18 Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial23M.19 Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100

Cycles

169

Indicators II 5. PHARMACY (MEDICINES AND SUNDRIES STOCK MANAGEMENT)

Score: 1: if all criterion have been met/ recorded 0: if the criterion have not been met/ not recorded

23M.20 Lignocaine 23M.21 PEP kits available and accessible:

Contents of PEP kit: Zidovudine 300 mg + lamivudine 300 mg +Atazanavir (300 mg)/Ritonavir (100mg)

23M.22 Lockable trolleys available with working lock23M.23 Adult first line ART:

Preferred: TDF +3TC+EFV, alternate: TDF+3TC+NVP or ZDV+3TC+EFV/NVP ( could available in Dual or triple FDCs)

23M.24 Paediatric first line ART: Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( <3 yrs)

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 32)

170

Indicators II6. OUTPATIENT DEPARTMENT (OPD) Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded

24M Consultations:24M.1 -Consultations being done by appropriately qualified staff: PCN and/or

RGN25M Guidelines/protocols25M.1 National Malaria guidelines for diagnosis and treatment of

uncomplicated and severe malaria - Posted on the wall, accessible to staff and up to date

25M.2 PEP policy and guidelines:-Available in OPD: posted on the wall and up to date

25M.3 Opportunistic Infection and ART guidelines:-Available, accessible in all consultation rooms and up to date

25M.4 STI Management protocol:-Displayed in all consultation rooms and up to date

25M.5 IMNCI guidelines:-Flowcharts displayed in all consultation areas and up to date

25M.6 Focused ANC protocol:-Displayed I all consultation rooms an up to date

26M SUPPLIES: Medicines in Emergency tray and Accessories26M.1 -Emergency tray with all the necessary un expired medicines (as from

EDLIZ) available: adrenaline, lignocaine, diazepam, MgSO4, atropine?26M.2 -Un expired ringer lactate, 5% dextrose, normal saline available26M.3 -Important Accessories: cannula, giving sets, syringes and needles, drip

stand, swabs, strapping, disinfectant, gloves, face mask, specimen bottles available, as part of emergency service

27M Availability of equipment at OPD27M.1 -Adult Weighing Scale and standard pediatric scale available and

functional: inspect in comparison with a known weight, after weighing the indicator should come to zero and height meter

27M.2 -BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmomanometer, thermometer, tape measure, fetoscope TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 12)

171

Indicators II 7. EXPANDED PROGRAM ON IMMUNIZATION (EPI);N.B. Please assess all the indicators requiring opening of the refrigerator at once in order to avoid frequent opening of the refrigerator

Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded

28M POLICY & GUIDELINES28M.1 -Surveillance line listing and case definitions displayed

28M.2 -Updated EPI schedule, and a contingency plan displayed

28M.3 -EPI graphs showing trends displayed and staff member is able to interpret the graphs

28M.4 -EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI modules available and easily accessible

29M Cold Chain Mechanism: 29M.1 -Fridge with a temperature booklet available and filled twice a day

29M.2 -The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (Supervisor should verify functionality of thermometer)

30M Availability of vaccines:30M.1 -The following antigens are available: BCG, MR ( measles and

Rubella), polio, Penta, tetanus, pneumococcal and rota virus vaccine30M.2 -The physical stock and the amount in the stock cards match

( Supervisor verifies physical stock in the fridge by selecting three different vaccines quarterly)

31M Vaccines storage31M.1 -Correctly stored in fridge with compartments as follows in fridges

with compartments:-Freezing compartment: ice packs well frozen-None freezing compartment: top shelf BCG, OPV, measles -Lower shelf: DPT+HEPB, TT, etc N.B. the new type of refrigerator i.e. Dometic fridge do not have compartments and the live vaccines are stored in the lower tray ( colder zone)

31M.2 -No expired vaccines

31M.3 -The Vaccine Vial Monitor (VVM )status is kept

31M.4 -There are readable labels on vials with matching diluents

32M Syringes:32M.1 -The number of syringes available matches the number of vaccines in

the stock cards33M Sharps boxes:33M.1 -Sharps boxes available in immunisation room/corner/area and not

more than 3/4 full)34M EPI accessories: the following EPT accessories should be available and functional34M.1 -Vaccine carriers, cold box, gas regulator, gas cylinder and scissors.

172

35M Forms35M.1 - AEFI investigation forms, case investigation forms for EPI targeted

diseases and vaccine wastage monitoring forms available 35M.2 -Vaccine order forms and stock cards available

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 17)

Indicators

II8. MATERNITY SERVICES; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER AND NEWBORN

Score: 1: if all criterion have beenmet/ recorded 0: if all criterion have not been met/ not recorded

35M Medicines on Emergency tray: the following medicines available on the emergency tray and not expired(Check expiry date when applicable)

35M.1 -IV fluids (ringer lactate, 5% dextrose, normal saline) and giving sets

35M.2 -Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and calcium gluconate

35M.3 -Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant

36M PPH kit ( please open one kit and check for its completeness)-PPH kits available and complete: *please refer to the annex section in the checklists guideline for list of items that should be available in the PPH kit

37 M Eclampsia kit ( please open one kit and check for its completeness)37M.1 Eclampsia kit available and complete:

*please refer to the annex section in the checklists guideline for list of items that should be available in the eclampsia kit

38M Uterotonic Medicines:38M.1 -Stored at correct temperature available in delivery room or immediate

vicinity to delivery room: oxytocin requiring refrigeration at 2-8 oC and the ones that do not require refrigeration below 25 oC

39M Equipment/supplies for care of newborn and monitoring of FHB: Are the following equipment available?

39M.1 -Fetoscope, baby blanket Baby scale and tape measure

39M.2 -Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord

39M.3 -Penguin suction, neonatal bag and mask and suction tube in delivery room

40M Obstetric sterilised delivery packs: ( open one pack to see whether all the items are present and check for expiry date )

40M.1 -At least 2 obstetric sterilized delivery standard packs with -2 wrapping towels, 6 drapes, A galipot with 10 swabs, 5 gauze swabs, A receiver, 2 Artery Forceps, Cord Scissor, Episiotomy Scissor, Drying towel for hands, Gown, Cord ties (2: if no cord clamps), sanitary pads available

41M Sterile gloves:41M.1 -At least 5 pairs of sterile gloves should be available

173

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 11)II9. OBSERVATION/INPATIENT SERVICES

40M In-patient register:40M.1 -Proper or improvised in patient register available and well maintained:

check whether observations documented, identity and hospital bed daysTOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 1)

Indicators

II.10. HEALTH INFORMATION MANAGEMENT SYSTEM Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded N/A: Not applicable

41M Referral and feedback system: review referral made in any one month in the last quarter ( if there was no referral made in the selectedmonth, extend the period of assessment to any of the two months in the last quarter)

41M.1 -Standard referral forms (at least 10) and register available and properly filled

41M.2 - A referral feedback documented in the register for every referral made and/or referral feedback notes available ( Applicable only if there was referral in the last quarter)

42M T Series forms and timely reporting : check the following two items in in any one month in the last quarter

42M.1 -The following T Series forms available and fully completed: T1, T3, T5, T6, and T12

42M.2 -T5 completed and sent timely (by the 7th of the following month) for previous month/sFor the following two indicators requiring review of registers and/or reported figures/indicators:Score each register/reported figure as: 1: if all criterion that have been met/ recorded 0: if the criteria has not been met/ not recorded And then give an overall score as shown below:5 Points: if 5 (100%) of registers/reported figures are complete and/or correct3 Points: If 3-4( 60-80%) of registers/reported figures are complete and/or correct0 Point: if ≤2 (≤40%) of registers/reported figures are complete and/or correct

43M Completeness and correctness of information in registers :Randomly select 5 registers to assess the indicators below *Please review the annex section of the checklist guideline for the list of registers available in a Health Center setting

43M.1 -The information in each column of the selected registers is complete and correct in any one month in the last quarter: select different registers quarterly

R1

R2

R3

R4

R5

Complete registers

Overall score

44M Correctness of reported figures:

174

44M.1 Are the figures reported for the last month of the last quarter correct according to the HMIS age groups in the T5?* Randomly select five indicators , verify for accuracy and correctness(selected different indicators quarterly)

i1 i2 i3 i4 i5 Accurate and correct figures

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 14)

175

CLINICAL MANAGEMENT PRIORITY AREAS

1C OPD/CONSULTATION AREA

AMBULATORY MANAGEMENT :

For indicator 1C1: assess by reviewing 5 files of patient who visited the clinic during the day of assessment. If there are no enough records, Score each file as 1or 0 as per the criterion and then give an over score/points for the indicator. If there are no records for review, please assess the indicator by asking the nurse at OPD ( if she/he is able to identify the three group, please give 100% ( 6 points; if not, please give 0 points)For indicator 1C2: assess by asking one PCN/RGN on duty during the day of assessment. Score each criterion as 1 or 0 as per the response of the health care provider and then give an over score/point for the indicator.

PATIENT’S RECORDSOr TB Symptom screening1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded N/A: not applicable if there is no record for review

5 records/symptoms (100%): 6 points4 records/symptoms (80%): 4 points3 records/symptoms (60%): 2 points≤2 records/symptom (≤40%): 0 points

1 2 3 4 5 Complete records /No. of symptoms correctly identified

POINTS

1C1 Triaging of patients at OPD waiting area during all clinic shift:-Patients are classified into three groups and given due attention accordingly:Assess by reviewing patient files if patients are available at OPD during the day of assessment. If not, assess by asking the nurse on duty on how they conduct triaging of patients ( her/his answers should match with the points listed below)Emergency signs requiring immediate attention Priority signs (requiring priority in the queueNon-urgent cases

1C2 % of adult TB screening symptoms correctly stated by health center/clinic OPD provider (Select one provider randomly on shift during the day of assessment) • Assessment: Ask the nurse at OPD to name criteria for TB screening (please observe whether her/his answers match with the list mentioned below and give score accordingly) 1) Weight loss

176

2) fever for more than 3 weeks 3) cough for more than 14 days 4) Night sweats 5) TB contact exposure.

1C3 % of TB presumptive (TB symptom positive) patients that have sputum results documented in any month in the last quarter*source of data: OPD register/TB suspect register

SUBTOTAL Points – ( Maximum Available points:18)

2C FAMILY AND CHILD HEALTH (FCH)AMBULATORY (ANC, PNC) BEST PRACTICESSource of Data: *ANC register for ANC Best Practise indicators* PNC register/follow up notes for PNC/Postpartum best practise indicators*Assess 10 cases/records in any one month in the last quarter. *If there are no enough cases for review, please extend the review period to a quarter. *If there are less than 10 cases for review after extending the review period to a quarter, assess the available cases/records.* If there are more than 10 cases for review, select 10 cases by using either simple/systematic random sampling*For indicators 2C1.6 i.e. pregnancy test, resampling or extending the review period to a quarter may be required as the number of women with ≤16 weeks of gestations is usually low.*If there no records for review/the indicator is not applicable in the set up being assessed, please do not assess and not score the indicator; rather write N/A and deduct the available points for the indicator from the maximum available points.

PATIENT’S RECORDS/Registers1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review*Score each case/record as 1 or 0 and then give a score for items per patient record, when applicable. * At last, please write the number of records with complete information as required and give an overall score/points as the per the criteria in the left column

9-10 records (≥90%): 6 points8 records (80%): 4 points7 records (70%): 2 points≤6 records (≤60%): 0 points

2C1 ANC BEST PRACTICES: 1 2 3 4 5 6 7 8 9 10

Complete records

POINTS

2C1.1

% of first visit ANC bookings in any one month in last quarter who had documented:BPHeightWeight measurements Fundal height measurements (Applicable only if pregnancy >16 weeks of gestation)

177

ALL ITEMS PER PATIENT RECORD2C1.2

% of first visit ANC bookings in any one month in last quarter who received the standard laboratory test according to the FNAC protocol: Blood group and RHHIV testHaemoglobinRPR (Rapid plasma regain for syphilis diagnosis)Urine analysisALL ITEMS PER PATIENT RECORD

2C1.3

% of first ANC visits in any one month in last quarter who received TT vaccine

2C1.4

% of first ANC visits in any one month in last quarter who received iron supplementation

2C1.5

% of first ANC visits in any one month in last quarter who received IPTp (if pregnancy >16 weeks of gestation and women living in malaria area)* Write Non-Applicable (N/A) if it is not a malaria endemic area

2C1.6

% of first visit ANC bookings with ≤ 16 weeks of gestation in any one month in last quarter who had documented pregnancy test results

2C2 POSTNATAL AND/OR POSTPARTUM BEST PRACTICES*Source of data: PNC register and/or post-partum follow up notes/sheet/registerIndicator 2C2.4 needs a separate sampling since the denominator is women who are 6 weeks post- partum during the assessment period.

1 2 3 4 5 6 7 8 9 10

Complete records

POINTS

2C2.1

% PNC visits in any one month in last quarter documenting assessment for the following conditions of the infant:General condition of the infant;NAD recorded if abnormality was not detected ALL ITEMS PER PATIENT RECORD

2C2.2

% PNC visits in any one month in last quarter documenting assessment for the following conditions of the mother:General condition ,Pulse rate, B/P and temperatureNAD recorded if abnormality was not detectedALL ITEMS PER PATIENT RECORD

2C2.3

% PNC visits in any one month in last quarter documenting infant feeding (BF) status (exclusive, mixed or not BF)

2C2 % women post-partum counselled and

178

.4 offered any of the modern FP method (below)at follow up PNC visit within 6 weeks of delivery in any one month within the last quarter ( this indicator should be assessed at 6 weeks post-partum visit)POP (progesterone-only contraceptive safe with BF)injectable, ImplantIUCDTubal ligationDeclineSUBTOTAL: ( Maximum available points: 60)

3C

MATERNITY WAITING HOME

Follow up of pregnant mothers in maternity waiting home *Write Non-Applicable (N/A) in clinics without maternity waiting homes and/or if there are no mothers in maternity waiting homes during the assessment period and deduct the available points for the indicator from the maximum available points.*If there are less than 5 mothers in the maternity waiting home,, assess indicator with mothers available during the day of assessment

PATIENT’S RECORDS

1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no mothers in the maternity waiting home or if the facility does not have maternity waiting home

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

3C1

ANC Best Practices: follow up of pregnant mothers in maternity waiting home

1 2 3 4 5 Complete Records

POINTS

3C1.1

% of mothers in maternity waiting homes monitored for BP, FHR, and assessed for danger signs daily*Source of data: ANC cards of pregnant mothers* Danger signs: vaginal bleeding, headache/blurred vision, fetal movement, sudden release of water from vagina*if there was no danger signs, it should be recorded as no danger signs

179

SUBTOTAL; ( Maximum Available Points:6)

4C HIV–PMTCT

Source of data: ANC, ART, Delivery and DNA PCR register*Review ANC register and select 5 newly identified HIV women for indicator 4C1. *Review delivery register and select 5 HIV exposed new-borns for indicators 4C2-4C4 in the last quarter. *If more than 5 cases found, select 5 cases using simple/systematic random sampling for each condition. If less than 5 cases in the last quarter, assess all the cases found. *N/A (Not Applicable) if there are no HIV+/HIV exposed cases for review and deduct the available points for the indicator from the maximum available points.

PATIENT’S RECORDS

1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

1 2 3 4 5

Complete records

4C1 % NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE SAME DAY in the last quarter*Source: ANC and ART register

4C2 % of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of birth in the last quarter*source of data: delivery register, PNC and DNA PCR registers

4C3 % of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks of age and received results within one month in last quarter *Source of data: DNA PCR register

4C4 % of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 DAYS OF RECEIPT OF RESULTS*Source of data: DNA PCR register and ART register

SUBTOTAL: ( Maximum available points: 24)5C AMBULATORY MANAGEMENT OF DIARRHEA,

PNEUMONIA ,MALARIA and Un complicated Severe Acute Malnutrition IN < 5 CHILDREN*Source of data: OPD/ IMNCI/CMAM registers*Review OPD/ IMCI register and select 5 cases with pneumonia, 5 cases with diarrhea, 5 cases with malaria, 5 Cases of SAM in any one month in the last quarter. If more than 5 cases found, randomly select 5 cases for each dis order. If the number of cases is not enough, extend the search period to a quarter to gather 5 cases for each disorder. If there are less than 5 cases in the last quarter, assess the indicator based on the available cases.

PATIENT’S RECORDS

1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if

5 5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%): 0 points

180

*Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

there are no records for review

1 2 3 4 5 Complete Records

POINTS

5C1 % children treated as outpatient for pneumonia in any one of month in last quarter who were correctly assessed *Source of data: OPD/ IMNCI registerAbsence of general danger signs recorded: able to drink/feed, vomiting, consciousnessDuration of cough/difficult breathing and child’s age recordedRespiratory rate, and presence/absence of chest in drawing, stridor and wheezing recordedCorrectly ClassifiedALL ITEMS PER PATIENT RECORD

5C2 % children correctly treated as an outpatient for pneumonia in any one of month in the last quarter among those correctly assessed

Treatment: Oral Amoxicillin 50mg/kg divided thrice per day x 5 days; caretaker counselling and follow up specified or admitted into hospital

5C3 % children with diarrhoea correctly assessed for signs of dehydration), persistent diarrhoea and dysentery in any one of month in the last quarter Assessment of dehydration: Using IMNCI guidelines (Integrated Management of Neonatal and Childhood Illnesses) Duration of diarrhoea and presence of blood recordedGeneral condition of the child recorded: lethargy, consciousness and/or restless or irritabilityPresence of sunken eyes, drinking status ( thirsty/drinking eagerly or un able to drink/drink poorly) and skin pinchCorrectly ClassifiedALL ITEMS PER PATIENT RECORD

5C4 % children correctly treated as an outpatient (ambulatory) for diarrhoea in any one month in the last quarter among those correctly assessedTreatment :-ORS, Zinc supplements and continued feeding and advise when to return

5C5 % children diagnosed with malaria that have RDT + or laboratory confirmation in any one month in the last quarter

5C6 % Children with uncomplicated malaria correctly treated according to national guidelines in any one month in the last quarterTreatment: ARTEMETHER (20mg)-LUMEFANTRINE (120mg)(C0ARTEMETHER) during 3 days (See treatment protocol in appendix 2 of the checklist guideline)

181

5C7 % Children with severe malaria correctly treated according to national guidelines in any one month in the last quarter (See treatment protocol in the Annex section of the checklist guideline)

5C8 % of 6-59 months old children with un complicated severe acute malnutrition (SAM) who were managed as per the national protocol in any one month in the last quarter

A 6 to 59 months old child with any one of the following criteria is classified as SAM :Weight for height <-3SD (WHO)MUAC <115mmMUAC <125mm and HIV positiveBilateral pitting oedemaOut Patient management of SAM:RUTFRoutine MedicineHealth and nutrition counseling and continued follow up*see annex section of checklist guideline for treatment details of SAMSUBTOTAL points: ( Maximum available points: 48)

6CMATERNITY SERVICES; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER AND NEWBORNDELIVERY BEST PRACTICES*Source of data: delivery register and partographs *Review delivery register and randomly select 10 deliveries in any month in last quarter. If the number of deliveries is not enough extend the search to the last quarter to gather 10 deliveries. If there were less than 10 deliveries in the last quarter assess the cases found. If more than 10 cases in a month/quarter then randomly select 10 deliveries and assess the partographs for the deliveries selected to assess the following indicators * Write Not Applicable (N/A) if there are no cases for review and deduct the available points for the indicator from the maximum available points.

PATIENT’S RECORDS1: if all criterion have been met/ recorded): 0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

9-10 records (≥90%): 6 points8 records (80%): 4 points7 records (70%): 2 points≤6 records (≤60%): 0 points

1 2 3 4 5 6 7 8 9 10

Complete records

POINTS

6C1

% deliveries performed by skilled personnel in any one month in the last quarter •Skilled provider: RGN with mid wifery training/RGN/PCN

182

6C2

% women who delivered in the facility monitored using partographs as per criteria *Please take into account the cervical dilatation at admission while assessing the following itemsAssumption: Partograph should be opened for all pregnant women in labor unless they come with fully dilated cervix with head visible. Women who present in advanced labor should have at least one measurement of the items mentioned below. If urine analysis was not done, the reason should be noted/documented i.e. lack of supplies/women did not produce urine.Fetal heart rate plotted every 30 minutesState of membranes every 4 hours presence/absence meconium ( if ruptured membrane)Cervical dilatation every 4 hoursDescent of presenting part every 4 hoursContractions plotted every 30 minutesMaternal BP every 4 hoursMaternal pulse every 30 minutesMaternal temperature every 4 hoursUrinalysis documented at admissionALL ITEMS PER PATIENT RECORD

6C3

% total births in any one month in the last quarter documenting administration of immediate postpartum oxytocin 10 units IM (within one minute of delivery of baby) (AMSTL: Active management of third stage of labour)* administration oxytocin 10 units IM within one minute of delivery of fetus (or misoprostol or ergomertrine, if BP normal, and oxytocin unavailable)

6C4

% births with placental status documented at birth in any one month in the last quarter•Assessment: complete or ragged , retained placenta

6C5

% newborns BF within one hour of birth in any one month in the last quarter•Assessment: Time of BF initiation documented

6C6 % newborns received Vitamin K in the any one month in the last quarter

6C7% newborns received eye care (Tetracycline) in the any one month in the last quarter

6C8

% newborns received first vaccination (BCG) in the any one month in the last quarter*source of data: delivery and/or PNC registers

6C9 % women monitored in early post-partum period (4th stage) per guideline (birth to discharge)

183

in any one month in the last quarter. *source of data: partographs and/or post-partum follow up notes/sheets/registerVaginal bleeding, at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeUterine contraction at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeBP at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargePulse at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeTemperature at least every 30 minutes 1st 2 hrs after birth and then four hourly until dischargeALL ITEMS PER PATIENT RECORD

6C10

% newborns monitored in early post-partum period per guideline (birth to discharge) in the any one month in the last quarter *source of data: partographs and/or post-partum follow up notes/sheets/registerTemperature documented at least every 30 minute first 2 hours after birth then four hourly until dischargeRespiratory Rate documented at least every 30 minute first 2 hours after birth then four hourly until dischargeBreast feeding status documented at least every 30 minute first 2 hours after birth then four hourly until dischargeColour documented at least every 30 minute first 2 hours after birth then four hourly until dischargeALL ITEMS PER PATIENT RECORD

6C11

% facility births seen for day 3 PNC visit in any one month in the last quarter*Source of data: Delivery register ( to identify list of deliveries) and PNC registerSUBTOTAL: (Maximum available points: 66)

7C OBSTETRIC & NEONATAL COMPLICATIONS

*Source of data: delivery register and/or partographs.*Review partographs/follow up notes and randomly select 5 cases with PROM, 5 cases with PPH, and 5 cases with Postpartum sepsis, 5 cases with Pre-eclampsia/eclampsia, 5 with neonatal asphyxia and 5 with neonatal sepsis in the last

PATIENT’S RECORDS

PATIENT’S RECORDS1: if all criterion have been met/ recorded):

5 records (100%): 6 points4 records (80%) : 4 points3 records (60%) : 2 points≤2 records (≤40%):

184

quarter. If there were less than 5 for review in the last quarter, assess the cases found. If there were more than 5 cases in a quarter then randomly select 5 cases to assess the following indicators*Write Not Applicable (N/A) if there are no cases for review

0: if all criterion have not been met/ not recordedN/A: not applicable: if there are no records for review

0 points

* 1 2 3 4 5 Complete Records

POINTS

7C1 % women with prolonged labour in last quarter referred to higher level facility -obstructed labour criteria in any one month in last quarter: active labour > 12 hours (from admission at minimum 4 cm dilation or per patient-reported labour onset if admitted > 4 cm)* Source of data: partographs and/or referral register or referral notes/

7C2 % women with prolonged labor or Rupture of Membranes and without chorioamnionitis that were administered antibiotics as per protocolTreatment with oral erythromycin (or amoxicillin) if ROM > 6 hours or active labor > 12 hours without signs of chorio-amnionitis; first dose antibiotic and referral to hospital if signs of sepsis (intra partum fever, foul-smelling discharge) *Review all partographs of women who delivered in last quarter and select those in which rupture of membrane documented > 6 hours (at any time in course of labour and delivery) or activelabour >12 hours (at any time) without documentation of other signs of maternal sepsis(maternal fever or foul-smelling discharge) is documented

7C3 % women with PPH managed per guideline in thelast quarter*Review all partographs of women who delivered in last quarter and select those partographs fulfilling the following criteria: PPH documented (EBL > 500 cc or VB and tachycardia > 100 bpm or hypotension SBP < 100 or DBP <50) and check whether the following three items listed below were done for each identified case- See the annex section of the checklist guideline for the details -PPH cause documented (atony, tear, retained placenta, other)-Resuscitation for all PPH cases irrespective of the cause: secure two IV lines with two 14/16 G cannulas or any large size available, and run normal saline (NS) or ringer lactate (RL) -Management according to the cause:-Uterine atony: add 40 IU Oxytocin in 1L NS or RL solution and run at 60 drops/minute until uterus is firmly contracted or insert misoprostol 600-1000 mcg rectally or-Retained placenta: controlled cord traction. If failed, manual

185

removal or-Vaginal/cervical laceration: sutured -Referral to hospital with IV access irrespective of the causeALL ITEMS PER PATIENT RECORD

7C4 % women with signs of intra- or post-partum sepsis (fever, foul-smelling discharge) that were given pre-referral treatment and referred to hospital in last quarterPre-referral treatment: X-pen 5 mega units IV immediately, then 2.5 units IV 6 hourly, plus metronidazole 500 mg IV 8 hourly and chloramphenicol 500mg IV 6 hourly (Use ampicillin 1g IV stat, then 500mg IV 6 hourly instead of X-pen if available)

7C5 % pregnant women with severe pre-eclampsia and/or eclampsia managed according to the guideline in last quarterReview partographs of women who delivered in the last quarter and select those who fulfill the following criterion

Severe Pre-eclampsia: -Diastolic BP 100mm HG or more-proteinuria 3+ or more

Eclampsia: -Unconsciousness or Convulsions (fits)-dBP 110 mmHg or more-Proteinuria or 2+ or more in a pregnant woman or a woman who has recently given birth

-Check whether the following three items listed below were done for each identified case:*see the annex section in the checklist guideline for details on management of severe pre-eclampsia and/or eclampsia- blood pressure checked every 5 minutes until diastolic BP is90 - 100mmHg-Blood pressure monitored (if diastolic blood pressure (dBP) is ≥110 mmHg give:Nifedipine 10 mg provided. If inadequate response after 20 minutes following first dose and repeat 10mg dose orally every 20 to 30 minutes until adequate dBP response is achieved, to a maximum of 40 mg given. Then 10-20 mg orally every 4-6 hours to maintain dBP 90-100 mmHg orHydralazine 5mg IV slowly every 20 minutes for a maximum of 20 mg* applicable only if dBP was ≥110mm Hg-Magnesium sulphate 20% solution, 4gm IV over 5 minutes given. Followed promptly with 10g of 50% magnesium sulphate solution, 5gm in each buttock as deep IM injection with 1 ml of 2% lignocaine in the same syringe.-Progress of labour including fetal well being monitored

-referral after stabilizing the motherALL ITEMS PER PATIENT RECORD

7C6 % of neonates who had an APGAR score of ≤5 in the first

186

minute after birth for whom resuscitation was initiated in last quarter*Source of data: partographs and notes*see the annex section in checklist guideline for details

7C7 % neonates with possible serious bacterial sepsis managed per standard in last quarter•Assessment of possible neonatal sepsis: Review all cases of newborn sepsis in last quarter from partographs and/or PNC register; and select 5 records for review that meet any of following probable sepsis criteria. *see checklist guideline criteria for chart audit-if documented temperature >380 C or < 250 C (and not warming);- RR > 60 or <30 breaths per minute; -chest in-drawing or convulsion; -no movement on stimulation;- poor feeding/sucking or -umbilical redness, newborn treated with stat dose antibiotics and referred to hospital: Treatment: Benzyl Penicillin 100,000 units / and Gentamycin 5 mg/kg first dose antibiotic and referred to hospital*Source of data: partographs and/or PNC registerSUBTOTAL: ( Maximum available points: 42)

VERIFY THAT ALL QUESTIONS ARE FILLED IN Supervisor thanks the staff

Signature:

DMO/Representative………………………………………………………..

Nurse in Charge…………………….........................

Counter verification………………………………………...

187

ASSESTMENT FEEDBACK

I. Summary Comments on Results. Please note any trends, problems, exceptional or creative changes and results that you saw during your visit assessment

II. Noteworthy Improvement. Please note any improvement and include a few details of what they are doing and why it is unique

III. Difficulties/ Challenges. Please note any assessment area that seem to be having an especially difficult time in improving. Please include a few details about the problem, how it might be solved, and who might be involved

IV. Recommendations and suggestions for improvement. Please note that the feedback is more effective when emphasizes features of the clinical task to be performed (e.g. specifies a target performance, presents information on how target performance can be attained, and address change in performance observed since previous feedback

V. Follow up. Please review previous recommendations provided and assess if they were followed or not

188