Development Scorecard

7/27/2019 Development Scorecard

1/5

Geoffrey Wright

POLS 30595-Essay #2

November 10, 2013

Practical Considerations for the Development of a Community Scorecard

With the increasing demand for improved accountability and governance in healthcare systems,

especially those of developing countries, the need for reliable performance measures that allow

governments and other stakeholders to identify deficiencies in the current system and to assess the

impact of health system strengthening initiatives on service delivery and, ultimately, health outcomes is

critical. One such method, the balanced scorecard (BSC) system, has evolved from its origins in the

business sector and is emerging as an invaluable tool for governments and health organizations in the

developing world as they look to improve quality and access of healthcare services. While the power of

the BSC is well-established in the business world, its application to healthcare systems of low- and

middle-income countries is limited, albeit promising. A pioneering BSC system carried out in Afghanistan

in 2004 (Edward 2011) and CARE Internationals innovative community scorecard (CSC) approach in

Malawi (Wild & Harris 2012) have both proven effective in monitoring and evaluating the progress of

health service capacity and delivery. However, due to the diverse and complex nature of health systems

and the fact that issues are addressed differently at the national, district, and community levels, no

single scorecard method has emerged as a universal solution to address the need for better

accountability and governance. Efforts to combine current models with novel ideas are needed to help

better the strategic management of resource-limited health systems worldwide. Although terminology

varies, researchers implementing the scorecard commonly emphasized the importance of developing

relevant, site-specific performance indicators with measurable and achievable benchmarks while, at the

same time, understanding the contextual perspectives in relation to scorecard implementation. This

paper begins to analyze several of the methodologies used in the development and implementation of a


2/5

scorecard system, with the intent of applying a revised version to future health systems strengthening

initiatives (HSSI) in Zambia.

Developing Relevant Indicators

In order for researchers and policy makers to maximize the benefits of a community scorecard, they

must first ensure that the indicators are 1) measurable and given achievable benchmarks based on the

realities of the country (Edward et al. 2011) and 2) are relevant to the population being served. Setting

high, but achievable benchmarks not only gives hospitals and other health settings being evaluated a

standard to strive for, but it also makes it easier to compare performance across hospitals/districts and

identify areas that are lacking, say, in leadership, drug availability, or financial support. The power of the

scorecard system lies in its ability to identify these deficiencies and observe the effects of resulting

interventions. To see this in practice, the BSC study in Afghanistan utilized upper benchmarks (termed

the green zone) set at a level that, in 2004, were being achieved by the six best performing provinces

(out of 33 total). The red zone, or the bottom quintile, represented those provinces performing the

worst in a particular sector, thus emphasizing areas needing special attention. These results provided

much needed accountability and transparency for struggling health systems at the provincial level in

Afghanistan. However, there is a recognized need to continue breaking down this scorecard system even

further, into community and facility levels, if the most marginalized groups are to be reached.

With this in mind, and with the possible future implementation of a community scorecard

system specific to Zambia, how do researchers go about developing relevant and robust indicators? It is

possible for indicators to be adopted from previously used scorecards. In the case of Zambia, for

example, it may be useful to borrow items from the scorecard formulated by Mutale et al. (2013) for

their research of the health facilities in three rural districts of Zambia. Their performance indicators fell

into one of six domains: patient and community, human resources, service provision, and health

systems preparedness for equipment, essential commodities and infrastructure (Mutale et al. 2013).


3/5

However, developers must be resolute in ensuring that the indicators are relevant to the particular

population being served and a deep contextual knowledge of both the community and the service

providers is necessary for the assessment to be most effective. Community involvement helps those

developing the scorecard to better assess the realities of the country at the micro-level and ensures

that the population they intend to serve has a voice. Furthermore, this involvement may be beneficial in

fostering a more inclusive and participatory culture when implementing the scorecard, something

Rabbani et al. (2011) highlights is a vital component to a successful and iterative scorecard system.

The World Bank outlines a general methodology for developing a community generated

performance scorecard (Singh & Shah):

Divide Into Focus Groups:After gathering the community, participants will be classified into focus

groups, ensuring that a heterogeneous mix of members is included in each group, based on age,

gender, and occupation, among other things, in order to stimulate a discussion that allows for

marginalized groups in the community to be represented and give any input that may have otherwise

been ignored.

Develop Performance Indicators: The focus groups brainstorm to develop criteria with which to evaluate

the health facilities or services in question. The facilitator then leads the group in organizing the

suggestions into measurable or observable indicators. Participation is crucial in this step to bring out a

critical mass of objective criteria (Singh & Shah).

Finalize Indicators: Indicators will be prioritized and reduced, perhaps through corroboration with the

MOH or other stakeholders. In the Afghanistan BSC, 340 potential indicators were analyzed for face

validity and importance, reliability, completeness, outlying values, and variation (Peters et al. 2007) and

ultimately reduced to only 29 in the final BSC. The final number of indicators varies from scorecard to

scorecard, but, as Peters et al. explained in their methodology, the final scorecard should have a limited


4/5

number of easily understood and robust indicators that represent the most important aspects of service

delivery (2007).

Ask Groups to Score Each Indicator: In order to get a relative idea of where the community stands on

the issues decided upon, participants will be asked to score each of the performance criteria, typically

on a scale from 1-100.

Ask Groups to Explain High/Low Scores: This step is important in drawing out peoples perceptions by

having them explain their reasoning for high or low scores. This can help researchers to explain outliers

and can provide them with anecdotes regarding service delivery that can be useful when thinking about

future interventions.

The document adds that suggestions from the community as a whole regarding the new performance

criteria they helped to establish would be insightful to conclude the process and move on to

implementation. Once the scorecard has been finalized and implemented for a baseline study, the

World Bank proposes a unique strategy that was initially carried out by CARE International in Malawi in

2002. Their innovative concept of conducting interface meetings in which the beneficiaries and service

providers meet to discuss the results of the scorecard assessment has been shown to be effective in

ensuring that community feedback is taken into account and that deficiencies in service delivery are

acknowledged.

The development of relevant indicators that are both measurable and achievable is a vital piece

in establishing a successful community scorecard system. Setting upper and lower benchmarks are

useful when comparing across facilities or communities and when analyzing the effectiveness of

interventions over time. The central ideas surrounding the development of a scorecard, though, involve

flexibility and valuedinput from the community. These factors are crucial in ensuring an inclusive and

participatory culture, giving the scorecard a contextual basis that allows it to serve as an evidence-based

decision-making tool for low- and middle-income communities.


5/5

Works Cited

Edward A, Kumar B, Kakar F, Salehi AS, Burnham G, et al. (2011). Configuring Balanced Scorecards for

Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan. PLoS

Med 8(7): e1001066. doi:10.1371/journal.pmed.1001066

Mutale W, et al. "Measuring Health System Strengthening: Application of the Balanced Scorecard

Approach to Rank the Baseline Performance of Three Rural Districts in Zambia." PLoS

OneE58650 8.3 (2013): 1-11. Print.

Peters D. "A Balanced Scorecard for Health Services in Afghanistan." Bulletin of the World Health

Organization85.2 (2007): 146-51. Print.

Rabbani, F. "Understanding the context of balanced scorecard implementation: A hospital-based case

study in Pakistan." Implementation Science 6 (2011).

Singh J, and Shah P. "Community Score Card Process: A Short Note on the General Methodology for

Implementation." The World Bank. Web. 11 Nov. 2013.

.

Wild, L., and D. Harris. 2012. "More than Just 'demand': Malawi's Public-service Community Scorecard."

Project Briefing: 69. Overseas Development Institute. Print.

Documents

Development Scorecard