Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Capacity Needs Assessment of Monitoring and Evaluation
Systems in KwaZulu-Natal Province
By
Dr. Nelson Kamoga and Dr. Faniel Habtemichael
Final Report : 24 January 2010
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal ii
Table of Contents 1. INTRODUCTION ............................................................................................................. 1
1.1. Context ............................................................................................................................... 1
1.2. Objectives of assessment ................................................................................................... 6
1.3. Methodology ...................................................................................................................... 6
1.4. Limitations of methodology ............................................................................................... 9 2. Overview of the M&E System in KZN ........................................................................... 10
2.1. M&E units in the province ................................................................................................ 10
2.1.1. Office of the Premier ........................................................................................ 12
2.1.2. Agriculture and Environmental Affairs ............................................................. 13
2.1.3. Arts & Culture ................................................................................................... 14
2.1.4. Community Safety and Liaison ......................................................................... 15
2.1.5. Economic Development & Tourism ................................................................. 16
2.1.6. Education .......................................................................................................... 17
2.1.7. Health ............................................................................................................... 18
2.1.8. Human Settlements .......................................................................................... 19
2.1.9. Local Government & Traditional Affairs ........................................................... 19
2.1.10. Public Works ..................................................................................................... 20
2.1.11. Social Development .......................................................................................... 21
2.1.12. Sports & Recreation ......................................................................................... 21
2.1.13. Transport .......................................................................................................... 22
2.1.14. Treasury ............................................................................................................ 23
2.1.15. Sisonke District Municipality ............................................................................ 24
2.1.16. Msunduzi Local Municipality ............................................................................ 24
2.1.17. Public Service Training Academy...................................................................... 25 3. OVERALL ASSESSMENT FINDINGS ................................................................................ 27
3.1. Institutional arrangements for monitoring and evaluation ............................................. 27
3.1.1. Legal framework, policies and procedures to implement an M&E systems ... 27
3.1.2. Overall system used to perform M&E tasks .................................................... 29
3.1.3. Roles and responsibilities to manage and implement M&E system ............... 29
3.1.4. Level of coordination on management and implementation of M&E system 30
3.1.5. Donors and other external stakeholder involvement in M&E capacity building30
3.1.6. Mechanisms to engage civil society, public and the private sector in M&E.... 31
3.2. Monitoring and evaluation advocacy ............................................................................... 31
3.2.1. Leadership influence on development of an M&E system .............................. 32
3.2.2. Use of M&E information for planning, resource allocation and policy making32
3.2.3. Management feedback on quality of M&E system .......................................... 33
3.2.4. Management capacity to perform and use inferential analyses on M&E data33
3.3. Monitoring and evaluation knowledge and skills ............................................................. 33
3.3.1. Capacity of the M&E system to generate and disseminate good quality data 33
3.3.2. Electronic system to collate data for the M&E system .................................... 34
3.3.3. Availability and quality of recent Monitoring and evaluation reports ............ 35
3.3.4. Availability of M&E system to generate data on the MDG indicators ............. 35
3.4. Reporting mechanisms ..................................................................................................... 36
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal iii
3.4.1. Existence of an effective reporting mechanisms ............................................. 36
3.4.2. Adequacy of monitoring system to generate disaggregated data reports ...... 36
3.4.3. Availability of mechanisms to disseminate M&E reports to stakeholders ...... 37
3.4.4. Availability of mechanism to link M&E data and performance ....................... 37 4. RECOMMENDATIONS ON M&E CAPACITY BUILDING .................................................... 38
4.1. Substantially increase demand for M&E by provincial governing structures .................. 38
4.2. Elevate awareness of incentives for M&E across the province ....................................... 38
4.3. Perform detailed diagnosis of department and municipality M&E systems ................... 40
4.4. Re-arrange organizational structures to strengthen the role of M&E ............................. 40
4.5. Increase quantity and quality of M&E supply .................................................................. 42
4.6. Strengthen structures and process to support M&E systems ......................................... 44
4.6.1. Increase financial resources ............................................................................. 44
4.6.2. Develop reliable data systems ......................................................................... 44
4.6.2.1. Good quality performance indicators: ............................................................. 44
4.6.2.2. Reliable interoperable information systems .................................................... 45
4.6.2.3. Coordination with civil society and community ............................................... 47 5. CONCLUSION ............................................................................................................... 48
5.1. Way forward ..................................................................................................................... 48
5.2. Action plan ........................................................................................................................ 51 Appendix 1: M&E assessment questionnaire ........................................................................ 53 Appendix 2a: Assessment summary findings ........................................................................ 58 Appendix 2b: Assessment summary findings ........................................................................ 59 References .......................................................................................................................... 60
Tables and Figures
Page Table 1 M&E units: Year of establishment and available staff………………………………. 11 Table 2 M&E capacity building results framework..……………………………………………… 50 Table 3 Provincial action plan……………………………………………………………………………….. 51 Table 4 Public Service Training Academy action plan……………………………………………. 52 Figures 1-16 Organograms……………………………………………………………………………………………. 13 - 26 Figure 17 M&E unit restructuring – Option 1…………………………………………………………… 41 Figure 18 M&E unit restructuring – Option 2a…………………………………………………………. 42 Figure 19 M&E unit restructuring – Option 2b…………………………………………………………. 42 Figure 20 Performance indicators……………………………………………………………………………. 45 Figure 21 M&E capacity building framework …………………………………………………………… 49
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 1
1. INTRODUCTION
1.1. Context
Being a young democracy, South Africa has had to deal with the colonial and apartheid legacy of a
poor and fragmented public service. Over the last 15 years, the government of South Africa has
built institutions and developed policies of change to ensure equality and improvement in the
well-being of the previously disadvantaged majority. Although the country has made some
progress towards several intersectoral goals such as primary school enrolment and gender parity
in education, progress has been insufficient or even reversed for many of the Millennium
Development Goals (MDGs). For example, since 1994, life expectancy has fallen by more than 10
years – mainly because of the rise in HIV-related mortality. In the middle of 2009, South Africa was
hit by a wave of service delivery protests in peri-urban communities1. According to the Municipal
IQ, these protests are due to poor service delivery and due to poorly managed state structures,
one of which include performance monitoring and accountability 2. There is an increasing
acceptance by government and citizens that the objectives of delivering quality services have not
being met3. The reasons include lack of political will, inadequate leadership, management
weaknesses, inappropriate institutional design, misaligned decision rights and absence of a strong
performance culture with effective rewards and sanctions.
World-wide, a growing number of governments are working to improve their performance by
creating systems to measure the quantity, quality, and targeting of the services that the
government provides and to measure the outcomes and impacts resulting from such outputs.
Such systems also facilitate an understanding of the causes of good and poor performance. One
of the key reasons performance systems have become critical is the increasing fiscal pressure to
spend resources more effectively and efficiently as the public demands quick redress of
community needs that the government is expected to provide. In 2008 the Auditor-General4
reported that the Departments of Health and Education had inadequate reconciliations,
insufficient controls and monitoring, as well as the lack of supporting documentation.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 2
Since 1994 there has been growing emphasis on M&E, largely to support democracy and transit
the country to a more transparent state. From 1995 there was emergence of M&E, in some
departments, but not within a coherent and overarching framework. This monitoring was largely
driven by the need to report on financial accountability, due to requirements of the Auditor-
General. However, there was widespread failure to address issues of impact or effectiveness
because the focus was more on activities and outputs, rather than outcomes. Until 2005, only
individual staff performance evaluations were institutionalised and regularly carried out in the
South African government. Policy programme monitoring and evaluation, however, were
undertaken sporadically by line function departments for purposes of their annual departmental
reports (Cloete5, 2009).
It is in this context that the government of South Africa has recently engaged in a process to
strengthen and institutionalise monitoring and evaluation (M&E) as part of the key structures to
continuously and more accurately track the process of service delivery to the people of South
Africa. In 2004, the Cabinet initiated plans for a monitoring and evaluation system for government,
and the Presidency subsequently developed the Government-wide Monitoring and Evaluation
Framework (GWMEF). The principles of the Policy Framework for the GWMEF6 are that M&E
should: contribute to improved governance (Transparency, Accountability, Participation,
Inclusion), be rights based, be development-oriented – nationally, institutionally and locally be
undertaken ethically and with integrity, be utilisation oriented, be methodologically sound and be
operationally effective.
According to Cloete7 (2009) the following considerations inter alia motivated a Cabinet decision in
2005 to develop the GWMEF:
A need for regular national government report-backs to the International UN Millennium
Goals Initiative on the progress with halving poverty in South Africa by 2014;
The fact that South Africa was the host of the World Summit on Sustainable Development
in 2002 and at that time did not have any national M&E system to assess sustainable
development as required by the Rio Convention of 1992 and reiterated at the
Johannesburg Summit where South Africa was the host country;
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 3
The undertaking by the South Africa president to regularly inform citizens about progress
with the Government’s National Programme of Action (POA);
The fact that donors are increasingly requiring systematic M&E of projects and
programmes that they fund, in order to protect their investments; and
The fact that institutionalising national M&E systems has proved to be an international
good governance practice.
The South African M&E system draws its mandate from several pieces of legislative framework
including the Constitution (Act 108 of 1996), Public Finance Management Amendment Act No 29
of 1999, Municipal Finance Management Act (2003), Public Service Act (1994 as amended), Batho
Pele principles, National Treasury guidelines and frameworks , and Cabinet decisions. The South
African Constitution calls for high standard of professional ethics; efficient, economic and effective
use of resources; development oriented public administration, provision of services in an
impartial, fair, equitable and unbiased manner; accountability; transparency; good human
resources management and career development practices to maximise human potential; and
representation. This creates clarity for departments on what is required to meet good governance
standards and helps them to focus in their efforts. The Public Finance Management Amendment
Act No 29 of 1999 requires public entities to control financial risk and promote internal control.
This act gives the National Treasury power to monitor the implementation of provincial budgets. It
also gives Accounting Officers the responsibility for the effective, efficient and transparent use of
the resources. The Municipal Finance Management Act (2003) requires municipalities to properly
monitor revenue and expenditure, indicate service delivery targets and performance indicators for
each quarter; assess the performance of the entity and monitor the operational functions of their
entities. The public service act (Amended in 1994) emphasizes the need for monitoring and
evaluation systems to ensure effective and efficient public service performance. The Eight Batho
Pele principles were developed to serve as acceptable policy and legislative framework regarding
service delivery in the public service. They lay the ground for M&E by requiring efficient and
effective utilisation of resources. These principles reinforce the need for benchmarks to constantly
measure the extent to which citizens are satisfied with the service or products they receive from
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 4
departments. They also play a critical role in the development of service delivery improvement
plans to ensure a better life for all South Africans.
National Treasury guidelines set monitoring and evaluation plan to examine whether objectives
have been met. According to the guidelines public entities, state-owned enterprises and
constitutional institutions are required to submit their expenditure estimates and related
performance outputs to inform the determination of appropriation requirements. The guidelines
require departments to critically evaluate their performance against stated targets and ensure
that outputs are achieved and measurable.
The current National Strategic Planning Green Paper8 emphasises the linkage and feedback loop
that exists between planning, and performance monitoring and evaluation. The processes and
results of monitoring and evaluation will be critical to planning and may result in modified
sequencing of programmes. The other discussion paper on improving Government Performance9
(2009) that describes the Presidency’s approach to management monitoring and evaluation
concentrates on the 10 priority outcomes of the government’s Medium Term Strategic
Framework. The paper proposes the need to use outcome and output measures to promote a
change in behaviour, and create a culture of accountability. It further states that its focus areas are
management of outcomes through ministerial accountability for improving delivery, performance,
institutionalising the Government-wide Monitoring and Evaluation system, and unblocking service
delivery.
The GWMEF draws from three data terrains namely programme performance information,
evaluations, and social, economic and demographic statistics.
In the case of the program performance information, the government, through the Department of
National Treasury, developed the Programme Performance Information Framework (PPIF) in May
2007 to monitor both financial and non-financial information about government services and
activities10. The Department of National Treasury’s mandate is informed by sections 215 and 216
of the Constitution, and other legislation such as the Public Finance Management Act (PFMA) of
1999 and the Municipal Finance Management Act (MFMA) of 2003. The Treasury’s engagement
with the GWMEF revolves around ensuring that information on inputs, activities, outputs and
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 5
outcomes underpins planning, budgeting, implementation management and accountability
reporting to promote economy, efficiency, effectiveness and equity, as well as transparency and
expenditure control. In the whole, the PPIF goal is to ensure that performance information is
available to national, provincial and local government managers at each stage of the planning,
budgeting and reporting cycle, and a results-based approach to managing and responding to
service delivery would become institutionalised. The lead institution responsible for the
implementation of the PPIF is the National Treasury. Other institutional stakeholders of the
Government-wide monitoring and evaluation system include the Department of Public Service
and Administration (To monitor staff performance evaluations), Department of Cooperative
Governance and Traditional Affairs (To assess policy programme performances of provinces and
local authorities), Department of Environmental Affairs (To assess state of the environment and
sustainable development), Public Service Commission (To promote the values and principles
governing public administration listed in section 195 of the Constitution), Statistics South Africa
(Responsible for data collection, storage and quality control), Public Administration Leadership
and Management Academy (To provide monitoring and evaluating capacity-building through
training). All national departments, provinces, and local governments are therefore expected to
develop M&E systems for performance monitoring and evaluation at their level.
The second data terrain of the GWMEF is the social, economic and demographic statistics focusing
on information generated by Statistics South Africa (Stats SA), government institutions, private
sector, research institutions and NGOs. As the agency responsible for collecting and reporting of
official statistics, Stats SA has a central role of ensuring that such statistics are fit to use. The
Statistics Act (No. 6 of 1999) mandates the Statistician General (SG) to put in place a framework to
enable evaluation of statistics generated by organs of the state. In addition, the Act empowers the
SG to designate as official any statistics produced by Stats SA or any organ of the state and
authorises the Statistician-General to comment on the quality of national statistics produced by
another organ of state; and to publish such other department’s statistics. It is in this regard that
the Stats SA has developed the South African Statistical Quality Assessment Framework
(SASQAF)11. Through SASQAF, Stats SA has enhanced transparency in data monitoring and
evaluation and thus promotes data quality maintenance within a decentralised system of statistics
production.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 6
The third data terrain of the GWMEF is the evaluation, and focuses on the standards, processes,
and techniques of planning and conducting evaluations and communicating the results of
evaluations of government programmes and policies. The Presidency will be developing an
Evaluation Framework and other guidelines and support material to facilitate the overall
implementation of evaluation systems across the three spheres of government. The Evaluations
Framework aims to encourage government institutions to evaluate their programmes on a regular
basis, provide guidance on the general approach to be adopted when conducting evaluations and
provide for the publication of the results of evaluations.
1.2. Objectives of assessment
The overall goal of the assessment was to perform a rapid appraisal to identify the challenges
facing the institutionalization of M&E in the province of KwaZulu-Natal (KZN) and the capacity of
the Provincial Public Service Training Academy to provide relevant results based provincial M&E
training. Specific objectives of the assessment therefore included:
Determining strengths and weaknesses of the current provincial M&E systems
across departments in the province
Develop strategies to improve capacity of provincial M&E systems based on
opportunities and threats identified from the assessment
1.3. Methodology
Before the field work to assess the capacity of the M&E in the province started, a stakeholders
meeting comprising the M&E Chief Directorate of the Office of the Premier, representatives from
the M&E units of the provincial departments, two municipalities, UNDP representative, and two
consultants was held in the Premier’s Office in Pietermaritzburg, KwaZulu-Natal (KZN) on the 1st
of October 2009.
The purpose of the meeting was to initiate the assessment process, and explain to the
stakeholders the UNDP’s capacity development framework, methodology, approach of the UNDP–
commissioned task, and get an overview of the challenges and scope of the need regarding M&E
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 7
systems. It was also an opportunity to elicit important inputs from participants on the conduct of
the assessment process. The meeting identified some key challenges including sensitivities around
the current M&E systems and suggested ways of handling them during the assessment process.
Institutions that were represented at the meeting included:
i. Office of the Premier (OTP)
ii. Departments
a) Agriculture & Environmental Affairs
b) Arts & Culture
c) Community Safety & Liaison
d) Economic Development & Tourism
e) Education
f) Health
g) Human Settlements
h) Local Government & Traditional Affairs
i) Public Works
j) Social Development
k) Sports & Recreation
l) Transport
iii. Public Service Training Academy (KwaZulu-Natal)
iv. Municipalities
m) Sisonke (district)
n) Msunduzi (local)
v. UNDP representative
vi. Consultants
The OTP, UNDP, and the consultants met first with the Public Service Training Academy, and later
with the departments and municipalities. Key issues discussed included:
The OTP role in the capacity assessment was to organize and facilitate stakeholders’
meetings, validate tools of assessment, identify those who will engage in the interviews,
assist the consultants regarding some logistical issues.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 8
The post assessment core function of the OTP will be to assist other departments to build
their M&E systems based on the assessment findings.
The UNDP engagement is based on demand for support for M&E capacity building by the
OTP. UNDP is committed to providing technical support in this regard. For example UNDP
will provide support to develop an E-Library on M&E and other development literature that
will be linked to the UN and other information sources
The role of the Provincial Public Service Training Academy will be crucial in the process of
institutionalization of the M&E system. The assessment would determine human and
institutional capacity of the Provincial Public Service Training Academy to provide relevant
M&E training to provincial departments and other public institutions. Training and other
capacity building activities by the academy will be guided by needs as identified from the
current assessment findings and future departmental M&E system assessments.
Unless there is an environment that supports the M&E system, the impact of technical
experts is minimal. For example the involvement of politicians in the M&E capacity building,
as such individuals are major drivers of demand for M&E.
The assessment involved two-week interviews with key informants responsible for M&E in
fourteen (14) provincial departments and two (2) municipalities (District and Local):
A questionnaire was designed to determine adequacy of available M&E systems. Interviewees
would respond to a question by scoring whether a specific aspect of M&E was adequate or not
using a scale of 1-5 with 5 equating to highly adequate and 1 inexistent at all. Interviewees would
elaborate on the rationale for giving a specific score thus giving clarity to the responses. See
Appendix 1 for a copy of the questionnaire. Departments were also asked to provide any
documentation depicting the relationship of the M&E unit to the rest of the department or
municipality structure.
The data from the interview findings were summarized into frequency tables as shown in
Appendix 2a and Appendix 2b. Section 2 provides a brief overview of institutional status of M&E in
departments, municipalities and the training Academy. Section 3 discusses in more detail
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 9
interviewee findings from the assessment and sections 4 and 5 provide detailed recommendations
and a summary action plan for provincial M&E capacity building.
1.4. Limitations of methodology
The assessment was based on key informants providing their perception on the current level of
adequacy of M&E systems that they use in provincial departments and local governments. This
perception may therefore be considered subjective and not truly representative of what is actually
happening on the ground. The diagnosis did not involve an audit of M&E systems to determine
quality of specific data items and information reported by departments, how it is used and the
impact (negative or positive) of using available data. Secondly, key informants were in most cases
department staff involved in generating and reporting information from the M&E systems. Thus
the system diagnostic was based on responses from suppliers of M&E and not those that demand
and therefore drive the need for M&E like politicians and higher department and local
government managers.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 10
2. Overview of the M&E System in KZN
KwaZulu-Natal is a provincial region with an area of 94,361 square kilometers and a population of
about 10 million inhabitants (South Africa's largest provincial population, i.e. about 21%) on the
eastern sea-board of South Africa12. KZN is one of the country's most popular tourist destinations.
And its port, Durban, is the busiest in South Africa and one of the 10 largest in the world.
Agriculture is central to the economy supported by mining, vehicle-manufacturing, vehicle-
manufacturing, and tourism. The province has ten districts, one metropolitan and 61 local
municipalities. The main languages spoken are isiZulu (81%), English (13%), and Afrikaans (1.5%).
Remnants of British colonialism and a mix of Zulu, Indian and Afrikaans traditions give the province
a rich cultural diversity. Fifty four percent of this population lives in rural areas, which are
characterized by extreme poverty and poor infrastructure (roads, transport and
telecommunication). This poses a huge challenge towards ensuring effective delivery of basic
services such as health, water and sanitation. Poverty, over-consumption, poor quality of- and
poor access to education and limited livelihood options, drought and flooding (linked to climate
change), spread of HIV/AIDS (highest prevalence in South Africa) are some of the major
sustainable development challenges in the province.
According to Ndebele13 the public service in KZN, as in many other provinces is characterized by
the amalgamation of different administrations and these are the Natal Provincial Administration,
KwaZulu-Natal Government and the Mass Democratic Movement. Each of these had its own
baggage, culture and attitude with regard to the concept of public service.
Prior to the development of M&E in KZN, the provincial departmental programs contained
financial information which relate mainly to inputs such as government expenditure and income
patterns. Reporting on performance came after a reform agenda since 1999 with the introduction
of the Public finance Management Act (No 1 of 1999).
2.1. M&E units in the province
This section provides an overview of the M&E units in the province including the training Academy.
The accompanying organograms give an indication of the position of the M&E units in the
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 11
Table 1 M&E units
Year of establishment and available staff
Provincial departments and municipalities Year M&E Unit established
Number of key staff available
Agriculture & Environmental Affairs 2006 3
Arts & Culture 2007 01
Community Safety & Liaison2 2004 10
Economic Development & Tourism 2006 4
Education 20073 2
Health 2007 6
Human Settlements 20024 1
Local Government & Traditional Affairs 2007 2
Public Service Training Academy 2008 1
Public Works 20095 0
Social Development 20086 1
Sports & Recreation 2009 1
Transport 2000 3
Treasury7
Sisonke District Municipality8 1
Msunduzi Local Municipality9 2003
organizational structure especially in relation to the planning units. The organograms are not
meant to give the whole organizational structures of the departments. Most of the M&E units of
the provincial departments are only 2-3 years old as shown in Table 1. This is with the exception of
the Department of Transport which has been in existence since 2000.
1 There is only 1 manager for corporate strategy
2 Its M&E is only external, monitoring the Police service 3 It featured in the organizational only in 2008 4 Staff were in and out up to 2009 5 Policy was approved in 2009, but no staff is assigned so far 6 It was available on paper in 2007 7 It monitors other provincial departments, it does not have M&E unit for its internal activities. 8 It does not have M&E unit per se, but some of its units have performance management functions, there is no dedicated unit
performing the activity. 9 It does not have M&E unit per se, it is a Performance Management Unit that is functioning as M&E.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 12
2.1.1. Office of the Premier
Although both the provincial and local government had made inroads in bringing services to
different communities and also facilitating poverty eradication programs, it is still not clear
whether of these initiatives had brought any improvement in the life of local people. The
Monitoring and Evaluation (M&E) Directorate in the Office of the Premier (OTP) was instituted in
2005 to provide key information for strategic decision-making and better government service
delivery based on the GWM&EF system that will collect, analyze and synthesize service delivery
data. The OTP holds an oversight role in the provincial government and as such is mandated to be
reliably informed of the status of the government. This information should create the single view
of the province—a view transcending organizational structures and creating a reliable, timely and
accurate picture of the status of the province. The M&E directorate launched the Provincial Nerve
Centre (NC) on 27 August 2008 to generate such information for the OTP. The NC provides an
automated and integrated information management system complete with reporting and analysis,
dashboards and GIS mapping to monitor and evaluate government’s key performance indicators
and to promote transparency and anti-corruption. The key objectives of the Nerve Centre include
bridging the gap between strategic provincial objectives and execution thereof. The system will
help to link objectives to tangible outcomes in terms of information on key performance indicators
and program outputs across departments and local government institutions. The NC will allow for
a single view of the province to assess progress within certain key areas of concern in the province,
along cluster requirements. It will replace the current mostly manual reporting systems, integrate
relevant data into the system, and track the progress of the KZN province growth and
development strategy, among others. Currently the Nerve Centre is operating on silo basis and
data is collected on demand, and does not regularly flow to it as planned. The Nerve Center does
evaluations with insufficient baseline data and relies on secondary sources like Statistics South
Africa. The system is expected to be full-fledged by March 2010.
Although the directorate has, with the assistance of UNDP, laid the foundation for full
implementation of the provincial M&E system there have been some challenges. Key challenges
articulated by the directorate include
Absence of a fully functional National Monitoring & evaluation system;
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 13
Lack of public sector capacity in monitoring and evaluation;
Absence of results based management paradigm;
Galvanizing consensus on impact indicators;
Lack of integrated management information system; and
Need to establish a roadmap to achieve MDG’s
Figure 1: Office of the Premier, M& unit.
2.1.2. Agriculture and Environmental Affairs
The M&E entity in the Department of Agriculture & Environmental Affairs was created in the
financial year 2006-2007 but its policies are still in the draft stage. Amendments to the M&E
framework are still outstanding department waits for central government changes to GWMEF. The
unit is better resourced in terms of finances compared to other departments. A recent audit by
the office of the auditor general determined that the unit was not well equipped to implement
M&E functions. In the whole there is still misconception about the role of M&E in the department.
The M&E unit is positioned low in the hierarchy at a sub-directorate level. However, the fact that it
falls under the Planning Directorate the potential of its reports to be used is great subject to the
quality produced.
OFFICE of the PREMIER
Chief Dirfectorate: Monitoring &
Evaluation
Directorate: Monitoring
Sub-directorate: Monitoring
Sub-directorate:
Performance Management
Directorate: Evaluation
Sub-directorate: Evaluation
Sub-directorate:
Training Coordination
Directorate: Business Unit
& GIS
Sub-directorate: Information
Management
Business Intelligence & Development
GIS Management
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 14
Figure 2: Department of Agriculture & Environmental Affairs, M& unit.
2.1.3. Arts & Culture
The so called Performance Management (M&E unit) was established in 2007 and no person was
assigned to it so far. The unit falls under the Directorate of Corporate Strategy and is temporarily
run by the Directorate of the Corporate Strategy. The fact that Policy and Planning fall under one
directorate it can make it easier for data to flow for planning purposes. There is also an M&E
committee that meets annually in the Department that oversees M&E functions. The M&E unit is
low in the hierarchy as a sub-directorate. However, the horizontal relationship with the planning
unit under one directorate is a suitable position for information sharing and usage depending on
the relationships.
HEAD OF DEPARTMENT:
AGRICULTURE & ENVIRONMENTAL AFFAIRS
Deputy Director General (DDG)
Chief Directorate: Strategic Support Services
Directorate: Policy and Planning
Sub-directorate:
Monitoring & Evaluation
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 15
Figure 3: Department of Arts & Culture, M& unit.
2.1.4. Community Safety and Liaison
There is no internal M&E in this Department. Its M&E unit is externally monitoring the provincial
police service. However, there is a need and there is plan to establish an internal M&E in April
2010. The M&E unit revises and refines its external monitoring tool every year. Accordingly it has
widened its scope to monitor procurement and budget expenditure, specialized units (like dog
units), and service excellence award to encourage police. To operationalise the excellence award
the Unit involves community structures and do survey once a year and rewards
stations/individuals/community service centers based on merit. This is a good incentive to
inculcate the culture of compliance and has to be continued on a wider scale. The M&E unit is
strategically positioned as a chief directorate which maximizes information usage, though it is
monitoring an external organization.
HEAD of DEPARTMENT: ARTS & CULTURE
Chief directorate: Corporate Governance
Directorate Corporate Strategy
Sub-directorate Policy and
Planning Sub-directorate Performance
Management (M&E ) Sub-directorate Intergovernmental Relations
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 16
Figure 4: Department of Community Safety & Liaison, M& unit.
2.1.5. Economic Development & Tourism
The M&E unit was established in 2006 but it is still staffed by one person. The unit is organized
under the Strategic Planning and Monitoring & Evaluation Management which ensures usage of its
reports and involvement in the planning process. The Unit has been under continuous process of
review and improvement and has produced a number of evaluation reports that is lacking in most
of the M&E units in the province. Its previous M&E system did not have clear outcome indicators.
The new system called MBP (managing by projects) integrates finance and human resources and is
able to track finances, and staff performance. In 2007 the Unit developed an M&E framework and
was approved by management. The Directorate is planning to establish MIS to be positioned in
the Knowledge Management unit which is under the same directorate. Though the Unit is still
monitoring project level activities, it seems to be under way to institutionalize M&E functions. It
has gained inroads to political buy-in and has positioned itself to create demands for its outputs.
The M&E and planning units are managed together under one Chief Directorate. This positioning
enables usage of the reports of the M&E unit for planning purposes.
The Unit has produced several evaluation reports.
HEAD of DEPARTMENT: COMMUNITY SAFETY &
LIAISON
Chief Directorate: M&E
Directorate: Service Delivery Evaluation
Directorate: Complaints Management
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 17
Figure 5: Department Economic Development & Tourism, M& unit.
2.1.6. Education
The M&E unit of this department was ‘established’ in 2007 but did not feature in the
organizational structure until 2008. It is staffed by 2 persons but need four more. The M&E and
Planning units fall under separate directorates which may not be conducive for coordination.
There are directorates of Quality assurance, M&E, Strategic Planning, EMIS that report to different
Chief Directorates. However, these directorates are closely interlinked and would be better if they
come under one chief directorate for ease of information sharing and usage. The M&E unit does
not track the school level performance, suggesting that it does not have any information about
learner and teacher attendance, for example. This means important information that has impact
on learning is missing from the M&E system. Such information is sent to the Education
Management Information Systems (EMIS) that is managed by a different directorate where the
M&E unit does not have access. The EMIS collects information twice a year and the M&E unit does
general annual survey. There is another web-based system called SASAMS (South African Schools
and Administration Management System) developed by the National Department of Education
that tracks students’ progress from entering to leaving the school system. The EMIS is making use
of it now. This system was rolled out in 2008 and 2500 out of 6000 schools in KZN are linked to the
system. It has modules that can track such as school finances, assets, and attendances. The
challenges of not rolling out throughout the province are electricity and connectivity problems.
HEAD of DEPARTMENT: ECONOMIC DEVELOPMENT & TOURISM
Chief Directorate: Economic Planning Program
Directorate M&E Policy Planning R&D Knowledge Management
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 18
This Unit has a plan to computerize the logical framework as a move to automate its M&E
functions.
Figure 6: Department Education, M& unit.
2.1.7. Health
The M&E unit in the Department of Health was created in 2007 and has got 3 members. M&E is
strategically positioned and directly linked to planning; they are both under one Chief Directorate
Figure 7: Department of Health, M& unit.
HEAD of DEPARTMENT: EDUCATION
Branch:
Planning and Support Services
Chief Directorate Planning
Branch:
Human Resources and Administrative Services
Chief Directorate Administrative Services
Directorate Monitoring & Evaluation
HEAD of DEPARTMENT: HEALTH
Chief Diretorate: Health Service Planning and Monitoring & Evaluation
Diretorate: Health system Monitoring & Evaluation
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 19
2.1.8. Human Settlements
The M&E unit for Department of Human Settlements was established in 2002 but does not have
any consistent M&E functions as the staff that was assigned to it was constantly relocated. The
M&E is a directorate position and no director has been appointed to date. A permanent staff in
position of deputy manager was appointed at the beginning of 2009 but the manager has other
tasks not related to M& functions. This makes it difficult for the Department’s monitoring unit to
assess the quality, quantity, cost and delivery time of new homes. The M&E unit is organized
within the office of the Head of Department and all the housing information system is linked to GIS.
Figure 8: Department of Human Settlements, M& unit.
2.1.9. Local Government & Traditional Affairs
The M&E unit came into existence in 2007 and is staffed by two persons that have clear job
descriptions. Their electronic system is Excel-based and all of their business units report in one
format. The system was piloted for two years and functions as an early warning system that
shows status of progress against targets. The Department is happy with the current system and it
is not looking for other sophisticated systems for the near future. To push down the full operation
of the system, instability of upper management is also a challenge, though the middle
management is relatively stable. There is also problem of staff retention especially for the younger
generation, the moment they start to gain experience they leave the department The M&E unit is
strategically positioned under the chief directorate, and it is headed by a director. However, the
HEAD of DEPARTMENT: HUMAN SETTLEMENTS
Directorate: Monitoring & Evaluation
Directorate: Planning
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 20
planning unit falls under a different chief directorate though it reports to the same deputy director
general.
Figure 9: Department of Local government & Traditional Affairs, M& unit.
2.1.10. Public Works
The M&E unit in the Department of Works in only in name and has no staff. It has been created
recently and placed under the Directorate of Strategic Analysis (Planning) The Director of Strategic
Analysis (Planning) is temporarily in charge of M&E activities. There is a departmental policy of
performance management information that was approved in 2009. There is a data capturing tool
called Works Information System (WIMS) that is only used in projects.
Figure 10: Department of Public Works, M& unit.
HEAD OF DEPARTMENT: LOCAL GOVERNMENT & TRADITIONAL AFFAIRS
Deputy Director General: Chief of Operations
Chief Directorate: Monitoring & Evaluation
Directorate: Monitoring Directorate: Evaluation Directorate: Systems and Reporting
Chief Directorate: Corporate Services
Directorate: Strategic Planning
HEAD of DEPARTEMNT: PUBLIC WORKS
Directorate: Strategic Analysis (Planning)
Monitoring & Evaluation (Vacant)
Chief Directorate: CFO
Chief Directorate: Corporate Services
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 21
2.1.11. Social Development
Although established in 2002, the M&E unit only became functional during the 2008-2009
financial year when it started receiving funding. A draft policy on Management of Performance
Information (financial and nonfinancial data) was developed and the document provides details on
management of source documents, collation of information, verification of information, and
submission dates. The unit is staffed by one person instead of the planned 27 at full operation. The
M&E unit is structured as a directorate reporting to the Chief Financial Officer (CFO).
Figure 11: Department of Social Development, M& unit.
2.1.12. Sports & Recreation
The M&E unit has only one member. M&E is still in its infant stage and the general awareness of
its functions is low. There is general misconception and no clear purpose of M&E. The current
M&E system is manual but it is developing an electronic system. The M&E unit’s position is low in
the hierarchy as a sub-directorate under the directorate of strategic management support.
HEAD of DEPARTMENT:
SOCIAL DEVELOPMENT
Deputy Director General: Chief Financial Officer
Directorate: Monitoring & Evaluation Chief Directorate: Financial Management
Directorate: Planning
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 22
Figure 12: Sports & Recreation, M& unit.
2.1.13. Transport
The M&E unit in the Department of Transport was the first of its kind in the province. It was
created in August 2000 and is a relatively well a organized unit, to the extent that all department
planning cannot be done without the unit’s input. The M&E unit has gone beyond the QPR’s
requirement and developed its own template of data collection and reporting called Annexure 6.
The Annexure is a basic Excel system that captures comprehensive and detailed information on
plans, activities, expenditure and outputs. The tool is able to generate routine monthly and
quarterly expenditures and output variance reports. The M&E unit does site visits to monitoring of
projects, and data quality audits. However, due to staff limitation such supervision is not optimum
at all. To partially address the shortcoming the M&E unit endeavors to train contractors on how to
complete Annexure 6.
HEAD of DEPARTEMNT: SPORTS & RECREATION
Rogramme 2
Chief Direcotrate: Corporate Governance
Directorate: Strategic Management Support
Sub-directorate: Monitoring & Evaluation
Planning & Development
Transformation & Service Delivery
Research & Innovation
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 23
Figure 13: Transport, M& unit.
2.1.14. Treasury
Treasury does not have an M&E unit for its internal functions. It does perform a monitoring
function over the provincial departments from an expenditure perspective through its Resource
Management Unit. Treasury’s M&E function over the provincial departments is not coordinated as
it is done separately through its various units like Budgeting and Supply Chain. For reasons of
security, there is no information sharing among these units and reports are submitted separately
to National Treasury and Provincial Government. The provincial Treasury’s monitoring mandate
over the province is unclear or limited. For example, the supply chain in the municipalities is not
monitored by the provincial Treasury as it is not its mandate. The QPR in municipalities also goes
directly to the National Treasury.
HEAD of DEPARTMENT: TRANSPORT
Chief Directorate: Strategic Planning and Monitoring
Directorate: Monitoring & Evaluation
Monitoring Evaluation Service delivery Improvement
Programme
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 24
2.1.15. Sisonke District Municipality
Although the municipality is aware of the urgent need to set up a functional M&E unit, none exits
due to budget constraints. Currently ‘M&E’ functions are performed through the Integrated
Development Plan and Performance Management System (IDP & PMS) unit that is staffed by one
person and uses a reporting format provided by the Department of Local Government.
Despite the requirement managers are reluctant to submit quarterly performance reports, and
apparently submit them on demand. The absence of M&E policy contributes to the irregularity of
reporting obligations and lack of information sharing among the departments in the district. The
Performance Management unit (that has functions similar to M&E) is not strategically positioned
to monitor all the activities of the District Municipality.
Figure 14: Sisonke, District Municipality, M& unit.
2.1.16. Msunduzi Local Municipality
This municipality also does not have M&E unit and the M&E function is more or less done by the
Performance Management unit. This unit was established in December 2003 and has 3 members.
The unit has been reorganized and now falls under Organizational Development and Research &
Analysis. However, the members of the Performance Management unit are still administered by
their previous head that has nothing to do with performance management. A major challenge is
the failure of managers to submit non-financial data routinely.
MUNICIPAL MANAGER
Executive Director
Manager: Operations
IDP & PMS (Performance Management
System)
Internal Audit Special programs IGR Communications
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 25
Figure 15: Msunduz, Local municipality, M& unit.
2.1.17. Public Service Training Academy
In order to mitigate the challenges of growth and development the provincial government of KZN
has instituted a Public Services Training. The academy is mandated to embark on a mission of
public service renewal by producing competent public servants that deal effectively with the
challenges facing the province including backlogs in basic services, poverty, protection of
vulnerable groups, human development, unemployment, economic performance, the natural
environment, and HIV/AIDS.
As an institutional response mechanism to the capacity challenges, the academy aims at ensuring
a coordinated and structured approach to human resource development in the province. It is a
dedicated unit in the Office of the Premier, whose exclusive function is to provide specialized
training and organizational development service to the officials of the KwaZulu-Natal Provincial
Administration. Its main function include to facilitate, develop and implement skills development
policy and program; to facilitate generic, functional and executive management development
programs; to facilitate and coordinate basic literacy and numeracy within the provincial
administration; to provide curriculum development and quality assurance, and to provide
leadership, guidance and support on skills development interventions.
As provincial trainings are organized by the Office of the Premier, PALAMA, etc. the Academy acts
as a coordinating unit in the province (fixing dates, venue, nomination, etc). As PALAMA’s trainings
have the syndrome of being generic the Academy has the responsibility of contextualizing courses.
MUNICIPAL MANAGER
Research & Analysis
Performance Management
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 26
The Public Service Training Academy has an M&E unit called Compliance, Monitoring and Impact
Assessment. Its functions include to undertake skills planning, compile and maintain a provincial
skills plan/strategy, monitor implementation of learnership and internship, facilitate and
coordinate reporting, assess the impact of human resources development (HRD) interventions.
It is a very new unit created in 2008 and is staffed by one person against the required 3. The unit
has to still develop policy guidelines for its operations.
Like most departments the M&E unit of the Academy has challenges of verifying data quality; for
example it cannot validate the training needs done by departments. The main challenges
identified in the academy for the provision of M&E training include: funding, lack of cost recovery
mechanisms, curriculum and materials development, lack of in-house M&E experts, and electronic
systems.
Figure 16: Public Service training Academy, M& unit.
For more information on intervention to capacitate the Academy see Section 5.
SENIOR GENERAL MANAGER (HEAD of the TRAINING ACADEMY)
Directorate: Skills Planning, Materials Development & Quality Assurance
Sub-directorate: Skills Planning
Monitoring & Evaluation
Mateirals Development
Training and developemnt
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 27
3. OVERALL ASSESSMENT FINDINGS
The assessment was adapted from the UNDPs framework of capacity assessment and focused on
four dimensions: Institutional arrangements and environmental situations, M&E advocacy, M&E
knowledge and skills, and Reporting mechanisms.
3.1. Institutional arrangements for monitoring and evaluation
3.1.1. Legal framework, policies and procedures to implement an M&E systems
This component of the assessment involved asking interviewees about the current monitoring and
evaluation public sector environment in which provincial departments and municipalities operate
and whether such an environment promotes M&E. The most significant aspect of the assessment
that was reviewed included adequacy, availability and implementation of current government and
or provincial policies and reforms that influence monitoring and evaluation implementation in the
province. This component further looked at adequacy of roles and responsibilities of individuals to
manage the M&E system, level of interdepartmental coordination, availability of assistance in
capacity development, and mechanisms to engage stakeholders on M&E.
In the whole, most interviewees agreed that the national legal frameworks and policies are
adequate to support M&E functions in the province. Many of them cited the Constitution (1996),
Public Finance Management Amendment Act (No 29 of 1999); the Municipal, Finance
Management Act (2003); Public Service Act (1994 as amended); Cabinet’s 2004 decision to initiate
M&E plans; Batho Pele Principle; Framework for Managing Program Performance Information14
(2007). Treasury Guidelines15 (Preparation of Expenditure Estimates for the 2009 Medium Term
Expenditure Framework), and the presidency’s development of the Government-wide Monitoring
and Evaluation System. Departments and municipalities expressed awareness of the current Green
Paper on National Strategic Planning (2009) and the discussion paper on Presidency’s approach to
Performance Management Monitoring and Evaluation (2009), and they believe these documents
can further consolidate and further contribute to strengthening their M&E activities. Interviewees
in the whole agreed that most of the new M&E policies and reforms were synergistic and had
huge potential to institutionalize M&E functions in the public sector.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 28
In terms of institutionalizing the reforms, some of the provincial departments have used the
mandate given to provincial departments and municipalities to develop M&E systems and
frameworks and created M&E units. The typical example is the Office of the Premier (OTP) that
created the provincial Nerve centre, an automated and integrated information management
system that monitors and evaluates the performance of KZN provincial government against set
goals, targets, equity in resource allocation, effectiveness and efficiency in service delivery across
all relevant levels of spheres of government. At the time of the review 4 of the 14 departments
and municipalities interviewed had an M&E framework and 12 had an M&E unit.
However, 60% of the departments perceive the current M&E public sectors policies as
inadequately implemented mainly due to the poor appreciation of the concept of performance
monitoring management and low staff levels as well as lack of adequate integrated M&E systems
to track program inputs, outputs, outcomes, finances and data quality. Interviewees expressed a
failure of departments to operationalise the policy frameworks and there is a general lack of
readiness to implement M&E functions. A number of the departments perceived the national
guidelines as too broad and difficult to be directly implemented and that provincial and
municipality structures must adapt them to develop appropriate M&E frameworks and
implementation plans.
Another challenge identified was the general lack of understanding and acceptance of a results-
based management as interviewees indicated that many provincial staff including at management
level view performance monitoring only as a “policing” process. This leads to the general behavior
that results-based or performance monitoring means reporting for the sake of compliance and not
for continuous improvement or using data for decision making. A typical example of this situation
was the observation by the Treasury department of inadequate (10% completion) or late
completion by departments of non-financial indicators in the Quarterly Performance Reports (QPR)
from departments. This can also be attributed to the perceived greater importance of financial
data and a general lack of appreciation of outcome and quality measures by departments.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 29
Furthermore, there are some organizational impediments characterized by absence of an
inadequately positioned M&E unit in provincial department organograms or inadequate
recognition of the role for M&E even when the unit is available. In three cases however, provincial
departments clearly recognized the role of M&E and have or are in the process of integrating M&E
unit into the planning unit to create one directorate.
In terms of institutionalization of evaluation very few departments have performed outcome or
impact evaluations in part fulfillment of the mandate to determine extent to which department or
municipality program and service objectives have been fulfilled. In addition none of the
departments interviewed had adopted the SASQAF to monitor quality of information generated at
the different service levels.
3.1.2. Overall system used to perform M&E tasks
Of the municipalities and departments interviewed 80% indicated that the current overall system
to support M&E functions was not adequate or not adequate at all. Interviewees indicated that
the current department systems did not have adequate systems to track inputs (finances and
human resources), outputs, outcomes, impacts and data quality simultaneously. A key gap in the
M&E systems is the lack of baseline data for most departments and it therefore makes it difficult
to determine progress towards goals and objectives. Two departments with relatively good
practices are Departments of Transport, and Local Government & Traditional Affairs. In both cases,
departments are able to simultaneously track inputs, outputs and progress (alerts) towards
targets. Even then, in both departments the M&E process does not include institutionalized data
quality audit system and as such no reports on data quality are routinely generated.
3.1.3. Roles and responsibilities to manage and implement M&E system
Only 43% of the departments and municipalities interviewed indicated that M&E roles and
responsibilities were adequate. The rest (57%) indicated that inadequacy in roles and
responsibilities was mainly due to failure to recognize role of M&E, failure to recruit quality staff
and failure to formalize job descriptions and roles of M&E personnel. In most cases this has led to
overlap and exhaustion of personnel and failure to focus on stewardship of M&E units by the few
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 30
personnel. M&E personnel are therefore underutilized leading to missed opportunity for input
into key provincial and municipal management areas like planning and resource allocation.
3.1.4. Level of coordination on management and implementation of M&E system
Only two departments indicated adequacy in inter-sectoral, interdepartmental and stakeholder
coordination on M&E. The majority agreed that departments were working in silos and the
opportunity to share and use inter-related information has not been adequately utilized. For
example both the Departments of Arts & Culture, and Economic Development & Tourism train
craftsmen, artists, and musicians but do not share a common database on such trainees; no
sufficient information flows from the Department of Education, and Department of Health to the
Department of Arts and Culture about the libraries in schools and hospitals; and the Department
of Education does not have real time information on private schools.
The presence of the Provincial Nerve Centre (NC) is an opportunity for inter-departmental and
inter-governmental collaboration and utilization of information. However access to the NC is
limited to MECs and Heads of Departments. Although a provincial M&E Forum is present for
networking and training of departments about the opportunity of sharing and utilization of M&E
data it has not been adequately supported.
3.1.5. Donors and other external stakeholder involvement in M&E capacity building
The majority of interviewees (86%) indicated that donor and other external support for
development of M&E systems has been inadequate. External support has been focused on
compliance and M&E primer trainings with limited technical support that leads to generation of
actual systems and process for M&E. For example, recent trainings by the World Bank and
Regenesys for the Departments of Social Development, and Agriculture & Environmental Affairs
were generic M&E primer workshops; two days training by Statistics South Africa to the
Department of Education about research methodology and statistics was not sufficient in
providing sustainable skills on evaluation; memorandum of understanding between the
Department of Local Government & Traditional Affairs for technical support by National Treasury
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 31
for M&E capacity development seems to emphasize compliance to reporting and not using data
for quality improvement.
3.1.6. Mechanisms to engage civil society, public and the private sector in M&E
Departmental and municipal respondents were asked to assess adequacy of the scope and quality
of mechanisms to engage civil society, the public and private sector, and the majority (73%)
indicated that mechanisms were inadequate. For example, respondents indicated that use of
community satisfaction surveys was minimal and that involvement of community in the
monitoring of provincial and municipality performance was inadequate or inexistent in most cases.
In the latter case this was attributed to inadequate knowledge about performance monitoring by
civil society and the community. Examples where such coordination and involvement have been
successful include collaboration between the Department of Community Safety & Liaison and
Business Against Prime, Community Police Forum, Portfolio Committee, and the Police Service in
the design of its M&E activities; Department of Agriculture & Environmental Affairs in the District
Agri-Forum for community participation; Department of Transport in the Rural Road Transport
Forum (RRTF) composed of the youth, women, church, commerce, etc to supervise commissioned
projects and ensure community voice is heard; the Public Service Training Academy also evaluates
its own services at the end of training session and report back to the concerned departments,
Office of the Premier, and to the Governance and Administration Cluster.
3.2. Monitoring and evaluation advocacy
This section assessed leadership influence (demand) on the development of an M&E system and
the extent to which the information generated is used in planning and budgeting. The assessment
would provide an inference into the provincial budget systems and how they link to the
performance monitoring systems. The ideal is that a public sector resource allocation system to be
effective requires full information on the problems and needs of the community, the best
interventions (policies) and their alternatives to solve or alleviate the problems and previous
experiences on implementation of interventions. In the case of South Africa, relevant provincial
and municipal civil servants would then use such information to provide policy, planning and
budgeting advice. Leadership and management structures would need to have the capacity to use
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 32
and promote or advocate for use of performance monitoring tools to generate data for decision
making and also have the capacity to query the quality of performance data. The overall finding
was that leadership advocacy for M&E was not adequate (71%) even when it was present.
3.2.1. Leadership influence on development of an M&E system
Until recently, most respondents indicate there was little advocacy for M&E systems in
departments and municipalities. Most recently, advocacy (pressure from MECs and Heads of
departments) for M&E across departments has been due to the recent emphasis on M&E by the
presidency. In some departments, department heads and managers have gone ahead to elevate
the role of M&E and have restructured departments to strengthen the role of M&E. For example,
the Department of Social Development has or is in the process of consolidating the M&E unit into
the Strategic Planning directorate. Support for M&E can also be seen through lobbying for training
of staff and allocation of a budget for M&E. Even with these developments, 64% (9) of the
departments and municipalities interviewed indicated that leadership influence on development
and use of M&E systems was inadequate. Interviewees indicated that signs of inadequate support
from leadership include failure to enforce timely submission of activity reports by project
managers and auditing of reports. Interviewees indicated that budget support for M&E was still
quite inadequate especially when you consider that the general recommendation for M&E budget
support is 5-10% of overall budget. For example, although the budgets for Departments of Social
Development and Health M&E units were the largest in absolute terms (R 13 million and R 19
million respectively), the budgets were actually 1% of the overall departmental budgets.
3.2.2. Use of M&E information for planning, resource allocation and policy making
Although there is general increase in M&E advocacy by leadership, 73% of interviewees indicated
that there is inadequate demand and usage of M&E reports. Interviewees suggested the main
reason is the lack of trust by both civil servants and political leaders of information generated by
the M&E systems as data is viewed as incomplete (no outcome indicators), inaccurate and un-
verifiable.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 33
3.2.3. Management feedback on quality of M&E system
Interviewees (79%) indicated that management feedback on quality of M&E reports was
inadequate even when some feedback is given. Interviewees attributed this to inadequate
readiness by management to objectively audit data quality. In fact none of the departments had
institutionalized internal data quality monitoring as part of their M&E systems they used.
3.2.4. Management capacity to perform and use inferential analyses on M&E data
Up to 67% of interviewees perceived management capacity to perform and use of inferential
analyses as inexistent, inadequate even when present or not adequate at all. To quote one of the
interviewees “…during statistical presentations 90% of the audience gets lost…” A typical example
of this situation was described by the department of Community Safety & Liaison where the
interviewee explained that the department changed from mere submission of report to include
presentation to the police management because the latter seemed not to understand them.
Interviewees observed that there is a tendency for management to rely on narrative reports and
ignore statistical data that would otherwise be used to make generalizations of data on service
delivery.
3.3. Monitoring and evaluation knowledge and skills
In this section the assessment reviewed in more detail the supply of M&E in the form of capacity
of infrastructure available to generate M&E data. The capability to supply M&E depends on skills
of personnel, structures and processes used to generate targeted type of good quality M&E data.
In general 72% of respondent departments and municipalities perceive the current information
infrastructure as inexistent, inadequate even when present or not adequate at all.
3.3.1. Capacity of the M&E system to generate and disseminate good quality data
87% of interviewed departments and municipalities indicated that available M&E systems were
inadequate even when present or not adequate at all. Respondents maintained that current
systems do have clear documented processes for data collection, collation, analysis, reporting and
use. However, most departments did not indicate that lower level service points have standard
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 34
data collection tools commissioned from the top. In addition there are limited personnel with skills
to manage the data management processes. The majority of interviewees, for example, indicated
that they had limited skills in statistical analyses, program evaluation, cost-benefit analysis and
data quality audits. This situation increases the likelihood of generating inadequate performance
quantity and quality of data for use by provincial departments and thus compromising quality of
policy and planning decisions and yet the province has no capacity to monitor quality of data.
3.3.2. Electronic system to collate data for the M&E system
This component of the assessment was performed to determine the availability and quality of data
collation tools (databases) used by departments and municipalities including interoperability and
harmonization of tools in the event of need to share data across stakeholders. In addition
interviewees commented on capacity of staff to use such collation tools. In the whole most
departments use simple Microsoft Excel collation tools. 73% of the departments and
municipalities indicated that good quality relational databases were inexistent, inadequate even
when present or not adequate at all. A key weakness of available databases is that they are non-
relational Excel based databases with minimal or no option to perform more sophisticated
analyses including linking all performance data from program goals, objectives, inputs (finances),
outputs and outcomes. Despite Excel’s capability for sophisticated operations most of the
departments are using it as a ‘manual’ system, not more than storing data. Although most
departments indicated knowledge about usefulness of GIS, there is very minimal use of this tool by
departments as value addition to their M&E systems. Another challenge is that collation tools are
available at the provincial and municipal level yet none are available or utilized at service point
levels even when available. For example, the Department of Health has the District Health
Information System (DHIS) which is adaptable to health facilities but few of them actually use it to
collate their data. Some examples of databases in place or planned by departments include:
The Departments of Local Government &Traditional Affairs has designed a web-based
electronic system (Excel) that is approved by SITA and is ready for development. It is based
on Logical Framework and is user-friendly.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 35
Department of Transport is comprehensively using Excel to develop reporting templates. It
also uses GIS to extract information from maps.
The Department of Agriculture & Environmental Affairs has a system called Knowledge
Bank that is used by the national Department of Agriculture Forestry & Fisheries.
The Department of Community Safety & Liaison has an internet based “Electronic
Complaints System” developed in 2006 for lodging community complaints and is linked to
cell phones. After a complaint is lodged the system generates reference number, and
acknowledgement letters, and faxes back automatically to the complainant and keeps
reminding both sides on status of follow up.
The Department of Education has a Management Information System (EMIS) that collects
information to support planning, monitoring and decision-making in the department. The
EMIS collects aggregate survey data twice a year from each school.
The provincial departments are still using PERSAL (Personnel and Salary Administration
System) and BAS (Basic Accounting System) both which have limitations and are outdated16
17 18.
3.3.3. Availability and quality of recent Monitoring and evaluation reports
Although most departments indicated that they generate great “glossy” reports most interviewees
indicated that these reports may not stand rigorous data quality audits. In addition, most
departments have no culture of performing program evaluations or cost benefit analyses. It is in
this light that the majority of interviewees (80%) indicated that such recent M&E reports are
either not adequate or do not exist at all.
3.3.4. Availability of M&E system to generate data on the MDG indicators
Although most interviewees indicated awareness about MDGs and their integration into provincial
development plan, 80% of them indicated that the M&E system was not adequately aligned to
accurately collect such data. Interviewees argued that this was mainly because MDGs were not
adequately adapted to the South African provincial setting and the indicator definitions were
unclear and no specific guidelines have been provided on how stakeholders can measure their
contribution to the MDGs. For example, one interviewee complained that “…we cannot measure
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 36
global partnership for development, which is completely out of our domain...” A key requirement
for using MDG data is a baseline and most interviewees indicated that departments and
municipalities do not have such information. Even when such information is available, it is not
readily available to all stakeholders. For example, MDG information in the Department of
Education is kept in the Education Management Information System (EMIS) where the M&E unit
does not have real-time access.
3.4. Reporting mechanisms
This aspect of the assessment reviewed adequacy of provincial M&E systems to generate reports
that are usable the extent that they generate information on targeted beneficiaries and more
importantly that ability of the systems to generate organizational and staff performance reports.
In general, the majority of interviewee (72%) perceived the current system as present but not
adequate, not adequate at all or inexistent.
3.4.1. Existence of an effective reporting mechanisms
Although 40% of the departments and municipalities indicated that reporting mechanisms were
effective in the sense that reporting templates and guidelines at respective sub-reporting levels on
what, how, whom, when to report were generally followed, written guidelines were not
necessarily available. In addition all interviewees indicated a lack of written guidelines to track and
feedback quality of reports received from different sub-reporting levels including what actions to
take in the event of data quality queries. In fact, almost all interviewees indicated a lack of
capacity to perform such data quality checks on submitted reports. None of the departments had
written data retention policies on source documents. Though all departments have standardized
reporting format (Treasury’s QPR) the Provincial Treasury does not have the power on addressing
reporting issues and enforcing compliance.
3.4.2. Adequacy of monitoring system to generate disaggregated data reports
Departments were assessed on the level of disaggregation of their data (gender, age-group,
location, socio-economic status, education, etc.). Most of the departments (67%) felt they do not
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 37
have an adequate level of disaggregated data in their reports and they attributed this to poor
design of source documents.
3.4.3. Availability of mechanisms to disseminate M&E reports to stakeholders
Interviewees were assessed for perceived adequacy of mechanisms to engage stakeholders on
M&E issues and the majority (80%) indicated such mechanisms, if present, were inadequate or not
adequate at all. Interviewees argued that high level reports generated at departmental and
municipal levels are often not simplified for the community and that the lack of a community M&E
feedback process of engagement and participation on provincial performance was needed. For
example, the Department of Community Safety & Liaison uses community-based volunteers
(Trained by the Manchester police) to collect information that is fed into the department’s M&E
system. The Department also assesses police performance through community forums.
3.4.4. Availability of mechanism to link M&E data and performance
The majority of interviewees (80%) indicated that M&E system was currently inadequate in linking
programme data to staff performance. The main reason given for this not being done is that staff
performance contracts are not linked to program activities and deliverables. In addition there is
inadequate understanding of the relationship between programme targets and individual staff
performance and lack of willingness to collaborate between human resource and M&E units. In
one department, interviewees indicated that the M&E system is able to generate performance
reports based on deliverable achieved by staff but there is no incentive to disseminate such
reports to human resource units or departmental managers.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 38
4. RECOMMENDATIONS ON M&E CAPACITY BUILDING
There is broad agreement among experts that to successfully strengthen public sector M&E
systems a number of factors have to be in place. Below are recommendations that the province
may implement and these could be the backbone of an action plan to improve M&E systems in the
province.
4.1. Substantially increase demand for M&E by provincial governing structures
Any efforts to develop an M&E system must be intentionally created and should be based on real
demand and powerful incentives not just political statements and M&E policies. The biggest
barrier to demand is lack of knowledge about the role of M&E at the key stakeholder level where
buy-in is crucial. As this rapid assessment has shown, a number of departments expressed lack of
knowledge at both political and management levels. Even with government push for
institutionalizing M&E through the GWM&F, a number of departments are yet to develop M&E
frameworks mainly due to lack of experience and incentives to actually develop M&E systems.
To increase demand a key strategy would therefore entail increasing knowledge on usefulness of
M&E at the key stakeholder level. Such awareness should involve hands on exposure to the tools
and methods of M&E and how they are used including practical examples of public sector
institutions that have developed and used such systems to great value. In the case of KZN
departments, department stakeholders should be exposed to peer-departments with good M&E
systems within the province as well across the country. Such encounters would go a long way in
persuading department and municipality stakeholders about the value of a good M&E system. By
securing buy-in of the key stakeholders such as key departments (e.g. Treasury, Local
Government) the latter would then offer the substantive effort to create, strengthen and fund
M&E systems.
4.2. Elevate awareness of incentives for M&E across the province
Governments build M&E systems not for the sake of it but rather to use the information
generated to support budgeting, to develop plans and policies, and to promote accountability.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 39
This is supposed to be the inherent merit for institutionalizing an M&E system and it is these
incentives that may drive demand for institutionalizing M&E. Often (as in the case of a number of
departments interviewed) the perception is that M&E is conducted primarily for accountability
purposes thus narrowing its focus. The government of South Africa has developed the GWM&F
and the core PPIF as tools to guide provinces and municipalities in developing performance based
budgeting processes. For example, in one of the departments, because of the lack of trust in its
M&E system the department continues to use an incremental budgeting system as data is
considered unreliable to force significant resource reallocation. Staff only feel obliged to report
whatever quality of data is available because they believe that “after all the budget will not
change.” There are several strategies that can be implemented by the province to increase
incentives for having good M&E systems. The strategies may take the form of:
Encouragement and rewards for conducting M&E and utilizing the findings e.g. provide
additional funding for M&E, rewards and high level recognition for departments with
best M&E practices, use of M&E experience for staff recruitment, promotion and
certification, provide free training for managers and staff on M&E to improve service
delivery and performance, provide budget-related incentives for departments to
improve performance, develop and enforce performance contracts for ALL staff.
Prompts or penalties for failing to conducting M&E and utilizing the findings e.g.
withholding part of funding or pay for departments or staff with no M&E systems or
poor performance respectively, create formal requirement for planning and budgeting
to include M&E, create formal process to report poor quality M&E systems and
adverse M&E data; develop formal system to involve civil society in review of
provincial and municipal performance including publication of poor performance,
provision of regular feedback to all stakeholders including data management teams at
all levels.
Advocacy and awareness about importance of M&E e.g. awareness raising seminars to
demystify M&E including examples of good M&E systems and their utility; pilot M&E
systems to demonstrate usefulness; high level statements within the province,
departments and municipalities endorsing merits for M&E systems; lobby for support
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 40
from donors for M&E support, eestablishment of M&E forums and networks to share
experiences, best practices and discuss strategies to resolve M&E challenges.
4.3. Perform detailed diagnosis of department and municipality M&E systems
The purpose of this diagnosis is to identify in detail how key information is collected, collated,
analysed, reported and used. In the later case, the extent of actual utilisation of data from the
M&E system would be reported. In the whole this process would help to identify specific
functional areas of the M&E system across departments and municipalities that require
strengthening or need new systems. The rapid assessment performed was able to identify some
weaknesses of departmental M&E systems. For example, almost all departments have no
documented or formal data quality control procedures to ensure that information generated from
service delivery points is measured for quality.
Good quality data is in general an intrinsic character of data and most departments will have to
improve on this component of their M&E systems. A key activity for departments and
municipalities would therefore involve performing baseline data quality audits on key
performance indicators. The audit would then be used to diagnose overall strengths and
weaknesses of departmental M&E systems and decisions can be made on developing new systems
or strengthening new ones and M&E human resource needs (including training needs) to
implement M&E systems from the service delivery point to the highest level of management.
4.4. Re-arrange organizational structures to strengthen the role of M&E
As stewards of performance monitoring department and municipal M&E units are mandated to
generate objective and accurate information to be used by the province and municipalities to
inform policy and planning. However, as highlighted earlier, demand for using M&E data is poor
and in fact M&E units in a number of departments do not have a prominent role in the
departmental organograms related to planning, financing and human resource. This is a paradox
as the objective of having an M&E system (M&E unit) is to generate information that is utilised by
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 41
the core provincial and municipal government functions i.e. policy development, planning,
budgeting and resource allocation.
In order to increase utilisation of data and to increase relevance of M&E, reorganisation of the
departmental hierarchy to bring the M&E units closer to the decision making units in the
departments would be useful. The province could consider some options of re-positioning the
M&E unit. For example, one option would be to locate the M&E, planning, finance, and human
resources units within the Office of the Head of Department at the level of chief directorate (See
Figure 17). Because of their dynamic interactions this strategic position enables them to oversee
the entire activities of all units and streamline data flow and strengthen horizontal relationships
between departments.
Another option would be that M&E and planning units must not go lower than the chief
directorate level and directly report to the Branch or Head of Department (while being outside the
Office of the Head of Department), depending on presence/(absence) of branches (See Figures 18
and 19). The twinning of human resources with M&E helps to easily integrate individual
performance with the overall organizational performance. The status quo in all departments is
that individual performance information is not flowing to M&E units.
Figure 17: M&E unit restructuring – Option 1
HEAD of DEPARTMENT
Programme 1 Programme 2 Programme 2
Chief Directorate: Montitoring & Evaluation
Chief Directorate: Planning
Chief Directorate: Finance Chief Directorate Human Resources
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 42
Figure 18: M&E unit restructuring – Option 2a.
Figure 19: M&E unit restructuring – Option 2b.
4.5. Increase quantity and quality of M&E supply
In order to perform well an M&E system needs to have highly skilled M&E personnel. Therefore
most M&E capacity-building needs to include provision of training in a range of M&E tools,
methods, approaches, and concepts. However, this should not be the emphasis of an M&E
capacity improvement response. Based on the findings of the assessment, it was clear that most
departments were understaffed and key departmental and municipal M&E activities like data
quality monitoring and evaluations were hardly done. Even the available M&E unit personnel are
not well equipped to perform such tasks. Although interviewees indicated that they had received
HEAD of DEPARTMENT
Chief Directorate: Monitoring & Evaluation
Chief Directorate: Planning Chief Directorate: Finance Human Resources
HEAD of DEPARTMENT
Branch 1
Chief Directorate:
Monitoring & Evaluation
Chief Directorate: Planning
Chief Directorate: Finance
Chief Direcdtorate: Huamn Resources
Branch 2
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 43
some training, most of such training was quite basic and most current skills needs were beyond
just M&E primer training. The preference would be to have results based training on performing
data quality audits, statistical analysis, evaluations and more specifically how to use data to advise
finance, planning and policy making departments. All these trainings should include mandatory
follow up to track significant changes in staff performance and most importantly quality of data
and data utilisation by key stakeholders. Key responses to improve M&E supply should therefore
include:
Perform M&E skills gap analysis from service delivery point to highest level
Increase resource allocation to recruit staff with evaluation and research skills
Develop appropriate job descriptions for M&E
Identify specific courses for staff to train on but not limited to program planning,
developing M&E plans, statistical analysis, program evaluation, performance
budgeting and resource allocation, data quality auditing, economic evaluations and
operational research.
The province has a Training Academy that can be used to strengthen supply of M&E through
customized departmental M&E training. However, to ensure quality training by the Academy, the
following aspects have to be considered.
The academy has to develop or outsource expertise to train on skills as deemed
necessary by departments and municipalities (See above).
All M&E training curricula have to be reviewed and appropriately certified /
accredited.
All trainings must be based on needs assessments and must involve human
resource units within departments and municipalities.
The academy should include pre-post training evaluations to assess outcome and
impact of M&E trainings and provide feedback to departments and municipalities.
The academy should consider applying cost recovery models to complement the
budget from the province.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 44
Ensuring that the training academy’s M&E unit strategically positioned in the hierarchy
of the academy to inform decisions in the institution.
4.6. Strengthen structures and process to support M&E systems
4.6.1. Increase financial resources
In order to have a functional M&E system, adequate financial resources have to be made
available. Funds are used to recruit staff, train staff and more importantly to perform M&E
activities like field trips to collect data, perform on-site technical support, perform data quality
audits, evaluations and any other research that may be deemed necessary to generate
information for planning, policy and resource allocation decisions. The rapid appraisal revealed
that financial support for M&E units was way below national and international standards. It is
generally recommended that a budget of between 5-10% of the global budget be put aside for
M&E. None of the departments met this criterion and in fact the majority of departments
allocated less than 1% of the global budgets to M&E. Departments must be assisted to develop
budgets for M&E plans that are linked to their activities.
4.6.2. Develop reliable data systems
In order to have a reliable data system the following aspects should be considered.
4.6.2.1. Good quality performance indicators:
The challenge has always been for sector departments to collect data on a large number of
performance indicators that end up not being used and actually the data is of poor quality. The
target is to avoid an over engineered system that generates too many indicators. Departments
should have a minimum of clear useful core indicators that best inform planning and budgeting.
In developing performance indicators we have to consider the levels of relevance. For example,
these can be small number of high-level strategic indicators focusing on outputs and outcomes, for
use by senior officials; greater number of operational indicators, focusing on inputs, processes,
and outputs, for mid-level officials; and much larger number of operational indicators focusing on
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 45
processes and services, for use by line managers and their staff at facility levels relevant to that
work unit (See Figure 20).
Figure 20: Performance indicators
4.6.2.2. Reliable interoperable information systems
There should be a single reliable and interoperable information system that is coordinated within
and across departments and municipalities. The province should perform an audit of available
systems within and across departments to determine capacity of such systems to provide raw
data. The audit would involve a review of the capacity of the system to store and use data within
and across the departments. For example, the audit would specifically look at the capacity of the
system to link program objectives, activities, outputs and financial expenditure. A few
departments appeared to have developed or are in the process of developing a system with an
integrated approach to M&E data. In making a decision about what data systems to use,
departments have to remember that simple electronic data systems can still do the job of storing
and processing raw data. The Departments of Transport, and Local Government & Traditional
Affairs have Microsoft Excel-based systems that have been successfully used in storing
departmental data and skills transfer is easy to provincial staff. Whatever M&E system is
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 46
developed there should be documented standard operating procedures. Such procedures would
clearly define the following:
Definition of indicators (and level of disaggregation) including Millennium
Development Goals (MDG) indicators and how these are localized at service
delivery points
Source of primary data and how it is collected, who collects it and when it is
collected
Description of how primary source of data is collected and submitted to the next
level
Description of how data is aggregated, analyzed and reported to the next level
including how data is captured by an electronic system
A list of the potential data quality challenges and how these are dealt with,
recorded and reported
A sample of standard data collection tools and reports as used at different levels
including instructions on how they are completed
Data retention policy and procedures
A minimum of evaluation questions that should be answered to determine program
success including terms of reference on how to perform such evaluations
As a complement, intra- and inter-departmental/ sectoral coordination should be improved
through sharing of raw data generated from M&E systems. Widely accessible provincial
information sharing is relevant for synergy leading to better planning and implementation of
projects and programs. Information could be shared at planning and M&E forums and would, for
example, involve review of interrelationship and correlations between raw data items across and
departments and sectors. In the interim this level of coordination is a mandate of the Office of the
Premier (OTP) through the M&E Directorate. The M&E directorate has built and implemented a
fully functional electronic M&E system called the Nerve Centre (NC) that provides for a platform
for integration of already existing M&E in provincial departments, district and local municipalities.
The NC has been developed to be compatible with data from all provincial M&E systems. However,
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 47
one of the challenges has been, but not limited to, alignment of different sector M&E performance
indicators to the NC requirements.
4.6.2.3. Coordination with civil society and community
The rapid appraisal revealed that there is inadequate coordination and actual involvement of civil
society in M&E. Interviewees indicated that probably the best available mode of communication
with civil society is through published annual plans and reports. However, the perception is that
the detail in such documents is too much and there is very little room for community to engage on
implications of plans and reports. Civil society would play a role in actually monitoring
performance of the province in service delivery and would also help guide province in refocusing
service priorities. A number of reasons put forward included lack of knowledge and skills by civil
society and community to engage on M&E issues and lack of adequate environment to engage
provincial structure on actual performance feedback including review of challenges and reasons
why performance may be poor. In this regard therefore the province could do the following:
Develop framework to involve community and civil society in M&E through
participatory performance assessments.
Hold routine community and civil society sectoral feedback forums in which
province shares performance achievements and short comings and reasons for such
short comings.
Develop simplified versions of provincial plans and reports that are more accessible
to communities and civil society.
Sensitize communities on M&E using participatory M&E techniques.
Routinely perform and publish results on satisfaction surveys; consideration could
be made to avail funds for community driven satisfaction or performance
monitoring surveys.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 48
5. CONCLUSION
5.1. Way forward
Experts agree that there is no best model of what a public sector M&E system should look like.
Similarly, any capacity building cannot be expected to be successful in over a short term. In fact
the most successful public sector M&E systems have taken years to get to where they are. This
does not mean that any M&E capacity building should be slow. Rather the approach should be
purposeful and focused with clear intent to build and institutionalise key components of the M&E
system as quickly as possible while champions are available.
The findings of the assessment generally indicate that the provincial demand and supply of the
M&E is relatively weak and therefore any strategies to institutionalise M&E would be two
pronged. However, the key driver of institutionalising M&E would be strengthening provincial
demand for M&E first rather than the supply. By strengthening demand at political and higher
management level, priority M&E capacity needs would be identified. Political and provincial
management leadership need to have intrinsic demand for M&E and have a clear reason for such
demand. M&E data is typically required to inform resource allocation, accountability, effectiveness
and efficiency and guide line management. Increasing M&E demand at this level would show
explicitly that the province is committed to a culture of M&E. Such commitment would therefore
include improving the quality of available M&E supply and increase utilisation of such resources.
Key strategies to improve quality of M&E supply would for example include promoting
recruitment of highly skilled M&E staff within department with clear mandates and job description
to inform provincial performance-based planning and budgeting, avail funds to implement M&E
activities as well as improve skills of available M&E staff and provide continuous feedback to M&E
service providers about the quality of services provided and current priorities.
A logical framework for a provincial M&E capacity building response is shown in Figure 21 and
provides a simple view of how any response would be expected to lead to improvements in the
province. The logic is that specific outputs (e.g. individuals trained on M&E) would lead to
intermediate outcomes (e.g. increased demand for M&E), and to final outcomes (e.g. utilisation of
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 49
M&E information by province). Such outcomes would help lead to final impacts like improved
government performance and service provision. Table 2 provides a results framework of M&E
capacity building framework in Figure 21.
Figure 21: M&E capacity building framework (Adapted from Keith Mackay, How to Build M&E Systems to Support Better Government, 2007 (IEG, World Bank)
19
Concerted activities to strengthen provincial M&E functions are undertaken by the province and donors, through provincial and local seminars on M&E systems; Detailed diagnoses of provincial/sector M&E functions; Audits of data systems; provision of M&E
training—including trainer training, officials, Conduction of evaluation are conducted on pilot/demonstration basis.
The action plan leads to the production of a range of outputs, such as number of officials trained in M&E; harmonized data systems; improved quality and focus of available monitoring indicators; improved quality and range of evaluations completed;
strengthened incentives for departments and municipalities to conduct and use M&E
Formal M&E framework or system is established by the province, leading to the systematic planning, conduct, reporting, and use of monitoring information and evaluation findings.
Strengthened demand for M&E at provincial
administrative & political levels
Strengthened supply of M&E and M&E skills at
provincial administrative & political levels
Strengthened M&E systems at provincial
administrative & political levels
Strengthened demand and supply of M&E & skills in civil society
M&E used for: provincial and local government decision making on policies, plans, and budget resource allocation; implementation and management of provincial and local government activities; monitoring of activities,
accounting of expenditures, evaluation of programs and projects; provincial and local government analysis and policy review; provincial and local government accountability.
M&E information directly supports provincial & LG
budget balancing, planning, and policy
formulation.
M&E information directly supports provincial & LG
budget balancing, planning, and policy
formulation.
Political and administrative levels in
province & LG assess and debate
performance.
Civil society assesses provincial and LG performance and inputs freely to
planning and policy debates
Improvements in provincial and departmental performance and service delivery
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 50
Table 2: M&E capacity building results framework
Objectives Activities Outputs Outcomes Impact
1. Perform detailed diagnosis of M&E systems
1. Perform provincial data management system and data quality audit
2. Develop data quality improvement action plans
1. M&E system audit reports 2. Department data management improvement
plans
1. Improved quality of M&E data
2. Increased utilization of M&E data in planning and budgeting
3. Improved
community satisfaction
1. Improved service delivery
2. Improved socio-economic wellbeing
2. Increase provincial demand for M&E
1. Develop M&E advocacy plan 2. Train political and management heads on
usefulness of M&E 3. Peer workshops on best practice M&E systems 4. Develop / review sector program performance
information frameworks
1. M&E advocacy strategic plan 2. M&E use workshops 3. M&E peer review workshops 4. M&E frameworks
3. Increase quality/quantity of supply for M&E
1. Perform M&E skills gap analysis 2. Recruit appropriate M&E personnel 3. Train M&E staff on appropriate skills 4. M&E learning tours and develop networks to
share experiences and best practices 5. Perform internal & external data quality audits 6. Increase funding for Training Academy to
support M&E capacity building (including new funding models)
1. M&E skills gap analysis reports 2. Recruit M&E staff 3. Focused M&E curricular developed 4. Train staff on M&E 5. Routine data quality audit reports 6. Increased funding for Training Academy
4. Strengthen structures and processes to support M&E systems
1. Increase financial resources for M&E 2. Develop M&E plans, policies, and frameworks 3. Select minimum performance indicators 4. Perform audit of electronic databases 5. Develop data management SOP 6. Develop or strengthen new electronic data
bases
1. Increased M&E budgets 2. M&E plans and frameworks 3. Minimum set performance indicators 4. M&E database audits
5. Re-arrange organisational structures to strengthen the role of M&E
1. Conduct M&E organizational restructuring to meet departmental needs
2. Develop new provincial M&E organograms
1. Organizational restructuring met 2. M&E units strategically positioned
6. Strengthen M&E collaboration and coordination within province and civil society
1. Conduct M&E system alignment workshops 2. Develop community M&E engagement strategy 3. Train communities on participatory M&E 4. Conduct community lead satisfaction surveys 5. Hold civil society performance forums
1. Department M&E alignment workshops 2. Community M&E strategic plan 3. Participatory M&E workshops 4. Satisfaction surveys
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 51
5.2. Action plan
An action plan is an overview of tasks the province needs to undertake to improve overall M&E
capacity. Capacity building development may be at organizational, individual (human) and systems
level within and across the province. In addition the capacity development action plan should be
based on achieving short, medium and long term outcomes as an ideal M&E system cannot be
achieved immediately. Provinces should therefore perform an options appraisal to prioritize
remedial actions. The options appraisal should be based on identifying most efficient and cost
effective actions to implement in the short term to long term period. Table 3 is a provincial action
plan showing timing and responsibility of capacity building tasks.
Table 3: Provincial action plan
Task Duration
(Term) Responsibility
Human
Recruit appropriate M&E personnel Short-Medium
Departments and Municipalities
Develop M&E advocacy plan Short OTP, and Training Academy
Train politicians and management on M&E Short-Medium OTP, and Training Academy
Conduct peer workshops on M&E best practices Short-Medium
(OTP and Department of Performance M&E
Perform M&E skills gap analysis and develop M&E training curriculum
Short OTP, Departments and Municipalities
Conduct M&E skills training Short-Medium Training Academy
M&E learning tours and develop networks Medium-Long
(OTP and Department of Performance M&E
Train communities on participatory M&E Medium-Long
Municipalities and Training Academy
Organizational
Perform detailed diagnosis of M&E systems Short OTP, and UNDP
Conduct baseline and routine data quality audits Short-Long
OTP, Departments, Municipalities & UNDP
Restructure and strategically position the M&E unit to facilitate informed decisions
Short Departments and Municipalities
Localize MDGs and incorporate in the MTSF and IDPs Short
OTP, Departments and Municipalities
Increase M&E usage Medium
OTP, Departments and Municipalities
Hold civil society performance forums Medium-Long OTP
Involve communities in evaluations Long
OTP, Departments and Municipalities
Systems
Develop M&E plan and policies frameworks Short
Departments and Municipalities
Develop relevant data management systems Short-Medium OTP and UNDP
Perform audit of electronic databases and develop interoperable information systems
Short OTP, Departments, Municipalities & UNDP
Establish performance indicators for the whole results-chain Short
OTP, Departments and Municipalities
Increase M&E budget up to 5-10% of budget Short-Medium Departments and
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 52
Municipalities
The training academy is expected to be engaged in extensive training exercises to improve M&E
skills among provincial officials that will be responsible for supply and demand of data from
generated by the M&E system. The OTP should lead the drive to increase allocation of extra
funding to support the training academy as an M&E capacity building partner in the province.
Table 4 is a summary of key tasks to which extra funding would be used. As a capacity building
partner with the OTP, UNDP would also be involved supporting the training academy through
improvement of information systems (as identified by UNDP ICT consultant)
Table4. Public Service Training Academy action plan
Tasks Duration (Term)
Responsibility
Human
Outsource high level skills training as deemed necessary by departments and municipalities
Short The Academy
Develop a training program in Knowledge Management
Short The academy
Train/employ to have own experts Medium The Academy and PALAMA
Organizational
Consider applying cost recovery models to complement the budget from the province
Short The Academy
Restructure and strategically position the M&E unit to facilitate informed decisions
Short The Academy
Training needs assessment that involves department and municipal human resource units
Medium (The Academy, departments, municipalities)
Systems
Review and accredit M&E curricula Short The Academy and OTP
Develop Web-services for electronic resources and publications
10
Short OTP and UNDP
Develop communication network and video streaming for distance learning
11
Short OTP and UNDP
Install computer centers in the Library to help research and access to digital libraries
12
Short OTP and UNDP
Provide UN publications and documents to create a UN section at the library (OTP, UNDP)
13
Short OTP and UNDP
Conduct pre-post training evaluations to assess outcome and impact of M&E trainings and provide feedback to departments and municipalities
Medium-Long The academy
10
Identified by Gonzalo Aramayo-Careaga (ICT/e-governance consultant at the UNDP) during recent visit to the Training Academy 11
Identified by Gonzalo Aramayo-Careaga (ICT/e-governance consultant at the UNDP) during recent visit to the Training Academy 12
Identified by Gonzalo Aramayo-Careaga (ICT/e-governance consultant at the UNDP) during recent visit to the Training Academy 13
Identified by Gonzalo Aramayo-Careaga (ICT/e-governance consultant at the UNDP) during recent visit to the Training Academy
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 53
Appendix 1: M&E assessment questionnaire
Kwa-Zulu Natal Monitoring and Evaluation Capacity Assessment Questionnaire/Discussion Guide
Background
A. The main objective of this exercise is to undertake an monitoring and evaluation (M&E) capacity needs assessment study for the Province
with a view to identifying essential gaps (human and institutional), suggesting how the capacity gap could be bridged and developing
requisite program of intervention. The information will therefore be treated confidential and will be used mainly for the foregoing purpose.
B. Name of the unit/Department...............................................................................................
C. Year the Unit was established....................................................................................
D. Number of staff in the unit..............................................................................................
E. INSTRUCTIONS: Rate the perceived or actual status of dimensions of capacity using the rating key below. When selecting the rating please consider the issues/factors suggested as critical to adequacy and or quality of the Monitoring, Evaluation and Reporting System for your unit. Write a comment on the rationale of your rating after this consideration.
5 4 3 2 1
Highly adequate
Adequate Present but not adequate
Not adequate at all Does not exist at all
Very high High Medium Low Very low Excellent Very good Average Fair Poor
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 54
A. Institutional Arrangements and environmental situations
Discussion Guide/Question Rating (1-5)
1 Adequacy and availability of legal framework, policies and procedures to manage and implement an M&E system (department or provincial level).
Please consider the following factors when rating: Availability and currency of framework/policies; Clarity of framework/policies; Suitability framework/policies; Staff and leadership Awareness about framework/policies; Extent of application at different levels; responsibility to monitor application; incentives to apply legal framework and policies; Conflict or synergy of framework/policy.
2 Adequacy of available overall system used by the organization to perform M&E tasks
Please consider the following factors when rating: Currency (changes) of system; origin and development of system; scope of system (i.e. tracks inputs, outputs, outcomes, data quality, finances) suitability of scope at different levels; auditing of M&E system.
3 Adequacy of roles and responsibilities to manage and implement M&E system
Please consider the following factors when rating: Clarity of roles to manage and implement; suitability of roles; discrepancy between formal and actual roles; barriers.
4 Level of available inter-sectoral, inter-departmental, intergovernmental and stakeholder coordination on management and implementation of M&E system(s)
Please consider the following factors when rating: Processes to support coordination; scope of coordination; suitability of scope of coordination; harmonization of data collection and reporting; integration, synergy & interoperability of ICTs used; challenges of systems; barriers.
5 Adequacy of available donors and other stakeholders in capacity building of M&E system
Please consider the following factors when rating: Scope of capacity building; quality; sustainability; capacity to meet needs; barriers.
6 Availability and adequacy of mechanisms to engage civil society, public and the private sector in the design, reporting and use of information from M&E system
Please consider the following factors when rating: current scope and quality of engagement e.g. satisfaction surveys; promotion of engagement; barriers.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 55
B. M&E Advocacy
Discussion Guide/Questions Rating (1-5)
7 Leadership influence on development of an M&E system to generate data across key levels
Please consider the following factors when rating: Existence of influential and outspoken champions for M&E System; Advocacy within and across departments on M&E system development; Budget allocation to develop and operationalise M&E units and M&E unit staff development (e.g., what percent of the total departmental budget is allocated to the M&E unit?
8 Use of M&E information for program planning, resource allocation and policy making
Please consider the following factors when rating: Frequency of demand for M&E reports; opportunity given to M&E Unit to report; Usage of M&E reports; Readiness of management to objectively review M&E reports; Use of all data categories in M&E reports (e.g. inputs, outputs, outcomes); Barriers to M&E information use
9 Management feedback on quality of M&E system.
Please consider the following factors when rating: Frequency of feedback on M&E reports; internal and/ or external M&E system audits commissioned by management; barriers to feedback
10 Management capacity to perform and use inferential analyses on M&E data
Please consider the following factors when rating: Frequency of management’s request for clarification on statistical data
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 56
C. M&E knowledge and skills
Discussion Guide/Questions Rating (1-5)
11 Capacity of the M&E system to generate and disseminate good quality data for evidence-based decision making
Please consider the following factors when rating: Quality of data collection, collation, analysis and reporting processes; use of information for organizational learning and decision making; Contribution of M&E system to setting, targets and indicators.
12 Extent of assistance given to management for establishing indicators and target setting of departmental goals
Please consider the following factors when rating: Extent of opportunities to contribute to setting departmental goals and targets; the M&E system’s report submitted to management that contributed to setting indicators and targets (including quality of report (management reactions and follow up activities).
13 Adequacy of electronic system to collate data for the M&E system in your department
Please consider the following factors when rating: Type (operating system) of software used, scope of software, ability of staff to use software, interdepartmental /governmental harmonization of software; Harmonization and use of software with complimentary software e.g. GIS.
14 Availability and quality of recent M&E reports/evaluations/policy reviews/resource allocation/ costing data/program output data
Provide sample of most recent reports From M&E unit during interview]
15 Level of staff skill to understand and use M&E concepts to develop results (logical) frameworks, data collection and collation tools, data analyses
Please consider the following factors when rating: current number of staff relative to burden of work, current skills relative to technical inputs needed in M&E unit e.g. skills to perform data quality audits, program/ project evaluations, policy analysis, cost effectiveness/ benefit analyses, design databases.
16 Availability of M&E system to generate data on the MDG indicators
Please consider the following factors when rating: Awareness of MDG indicators, Domestication of MDG indicators, awareness of sources of data for MDG indicators; Challenges to reporting on MDG indicators.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 57
D. Reporting mechanisms
Discussion Guide/Question Rating (1-5)
17 Existence of an effective reporting mechanisms
Please consider the following factors when rating: Availability and use of standardized reporting formats; presence of written guidelines to each sub-reporting level on what, how, whom, when to report; the feedback mechanisms to all sub-reporting levels on the quality of their reporting (i.e., accuracy, completeness, timeliness, etc); the consistency of reporting tools; Policy on retention of source documents for reports; availability of written procedure to address late, incomplete, inaccurate, and missing reports including follow-up with sub-reporting levels on data quality issue.
18 Adequacy of current M&E system to generate reports with targeted disaggregation of data showing status of reach on outputs and outcomes of priority target beneficiaries
Please consider the following factors when rating: Presence of disaggregation of data by priority e.g. Gender, age-group, location, socio-economic status, e.t.c
19 Availability of mechanisms to make M&E reports available to relevant stakeholders
Please consider the following factors when rating: Mechanisms being used to engage stakeholders on M&E issues e.g. Forums; the availability of community-based M&E system that your department is using; the regularity of dissemination of progress reports about projects/programs to relevant stakeholders; the availability of data dissemination plans.
20 Availability of a mechanism to link M&E data and overall organizational and staff performance
Please consider the following factors when rating: Existence to link program/ project outputs and outcomes to staff performance, Use of M&E data and staff performance for skills and knowledge improvement, Use of M&E data and staff performance for promotion/demotion or change in roles and responsibilities.
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 58
Appendix 2a: Assessment summary findings
KZN M&E CAPACITY ASSESSMENT: SUMMARY FINDINGS OF ADEQUACY OF M&E SYSTEM CAPACITY
CAPACITY CATEGORIES AND QUESTIONS
Loca
l G
ove
rnm
en
t &
Trad
itio
nal
Aff
airs
Spo
rts
& R
ecr
eat
ion
Agr
icu
ltu
re
&
Envi
ron
me
nta
l Aff
airs
He
alth
Soci
al D
eve
lop
me
nt
Co
mm
un
ity
Safe
ty
Art
s &
Cu
ltu
re
Eco
no
mic
De
velo
pm
en
t &
Tou
rism
Tran
spo
rt
Trai
nin
g A
cad
em
y
Siso
nke
DM
Edu
cati
on
Hu
man
Se
ttle
me
nts
Pu
blic
Wo
rks
Msu
nd
uzi
mu
nic
ipal
ity
COUNTS OF ADEQUACY SCORE
HIG
HLY
AD
EQU
ATE
AD
EQU
ATE
PR
ESEN
T B
UT
NO
T A
DEQ
UA
TE
NO
T A
DEQ
UA
TE
AT
ALL
NO
T P
RES
ENT
1. Institutional arrangements & environmental situation
QN1 3 3 3 4 3 5 4 3 4 3 1 4 3 4 3 1 5 8 0 1
QN2 4 3 3 3 2 4 4 3 3 2 2 3 3 3 2 0 3 8 4 0
QN3 4 2 3 4 3 4 4 3 4 3 2 4 2 ? 2 0 6 4 4 0
QN4 1 1 3 3 2 1 2 3 4 3 3 4 1 2 2 0 2 5 4 4
QN5 5 1 2 2 2 1 2 4 N/A 2 1 2 2 1 2 1 1 0 8 4
QN6 3 1 4 3 2 3 3 4 4 4 1 2 3 1 2 0 4 5 3 3
2. M&E Advocacy
QN7 5 2 3 4 4 3 4 3 4 3 1 2 3 N/A 2 1 4 5 3 1
QN8 2 2 2 5 2 3 4 3 4 3 3 4 3 3 1 1 3 6 4 1
QN9 3 1 3 3 3 3 4 4 4 3 1 3 2 N/A 1 0 3 7 1 3
QN10 3 1 2 4 2 4 4 3 4 4 1 2 3 3 1 0 5 4 3 3
3. M&E knowledge and skills
QN11 2 2 3 3 2 3 4 4 3 2 2 3 3 2 2 0 2 6 7 0
QN12 4 2 3 4 2 3 4 4 4 4 3 4 3 3 1 0 7 5 2 1
QN13 4 3 4 3 1 4 3 2 4 1 1 1 3 3 2 0 4 5 2 4
QN14 4 2 3 4 3 3 3 3 4 2 1 3 2 2 1 0 3 6 4 2
QN15 4 1 2 4 2 3 3 4 4 2 4 4 2 N/A 2 0 6 2 5 1
QN16 2 1 3 4 2 1 2 2 4 2 3 2 3 4 2 0 3 3 7 2
4. Reporting mechanisms
QN17 5 3 4 4 2 3 4 3 3 3 3 4 3 4 3 1 5 8 1 0
QN18 1 4 2 4 3 4 2 3 4 2 1 3 4 3 2 0 5 4 4 2
QN19 2 2 4 4 3 3 2 3 4 3 3 2 3 3 3 0 3 8 4 0
QN20 1 3 4 3 2 3 4 3 3 2 2 2 4 2 3 0 3 6 5 1
5 77 105 75 33
1.70% 26.10% 35.60% 25.40% 11.20%
27.80% 35.60% 36.60%
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 59
Appendix 2b: Assessment summary findings
KZN M&E CAPACITY ASSESSMENT: SUMMARY FINDINGS OF ADEQUACY OF M&E SYSTEM CAPACITY
CAPACITY CATEGORIES AND QUESTIONS
COUNTS OF ADEQUACY SCORE
HIG
HLY
AD
EQU
ATE
AD
EQU
ATE
PR
ESEN
T B
UT
NO
T A
DEQ
UA
TE
NO
T A
DEQ
UA
TE
AT
ALL
NO
T P
RES
ENT
HIGHLY ADEQUATE AND ADEQUATE (%)
PRESENT BUT NOT ADEQUATE (%)
NOT ADEQUATE AT ALL AND NOT PRESENT (%)
1. Institutional arrangements & environmental situation
QN1 1 5 8 0 1
25.3% 31.0% 43.7%
QN2 0 3 8 4 0
QN3 0 6 4 4 0
QN4 0 2 5 4 4
QN5 1 1 0 8 4
QN6 0 4 5 3 3
2. M&E Advocacy
QN7 1 4 5 3 1
QN8 1 3 6 4 1
23.7% 39.0% 37.3% QN9 0 3 7 1 3
QN10 0 5 4 3 3
3. M&E knowledge and skills
QN11 0 2 6 7 0
QN12 0 7 5 2 1
32.6% 32.6% 34.8%
QN13 0 4 5 2 4
QN14 0 3 6 4 2
QN15 0 6 2 5 1
QN16 0 3 3 7 2
4. Reporting mechanisms
QN17 1 5 8 1 0
QN18 0 5 4 4 2
27.4% 36.2% 36.5% QN19 0 3 8 4 0
QN20 0 3 6 5 1
5 77 105 75 33
1.70% 26.10% 35.60% 25.40% 11.20%
27.80% 35.60% 36.60%
Capacity Needs Assessment of Monitoring and Evaluation Systems in KwaZulu-Natal 60
References 1 http://www.mg.co.za/article/2009-07-22-sa-hit-servicedelivery-protests
2 http://www.municipaliq.co.za/press/press_release_service_delivery_protests.doc
3 Chabane, C. 2009. National planning, performance monitoring green papers to go before Parliament.
http://www.polity.org.za/article/national-planning-performance-monitoring-green-papers-to-go-before-parliament-2009-09-04 4 Auditor-General. 2008. “PFMA roadshows 2008.” [Online]. Available: http://www.agsa.co.za/
5 Cloete, F. 2009. Evidence-based policy analysis in South Africa: Critical assessment of the emerging Government-
Wide Monitoring and Evaluation System. Journal of Public Administration 44(2). 6 Presidency. 2007. Policy Framework for the Government-wide Monitoring and Evaluation System. The Presidency,
November 2007, Pretoria. http://www.thepresidency.gov.za/mainasp?include=learning/reference/policy/index.html. 7 Cloete, F. 2009. Evidence-based policy analysis in South Africa: Critical assessment of the emerging Government-
Wide Monitoring and Evaluation System. Journal of Public Administration 44(2). 8 Presidency. 2009. Green paper: National strategic planning. www.thepresidency.gov.za
9 Presidency. 2009. Improving government performance: Our approach.
10 The Framework for Managing Programme Performance Information. Department of National Treasury. Government
of South Africa. May 200
11 South African Statistical Quality Assessment Framework (SASQAF). 1
st Ed. Statistics South Africa, Pretoria, 2008
12 http://www.southafrica.info/about/geography/kwazulu-natal.htm
13 KwaZulu-Natal Provincial Government. 2007. SA: Ndebele: Public Service Training Academy.
http://www.polity.org.za/article/sa-ndebele-public-service-training-academy-31082007-2007-08-31
14 National Treasury. 2007. Framework for Managing Programme Performance Information, National
Treasury: Pretoria, www.treasury.gov.za/publications/guidelines/FMPI.pdf 15
National Treasury. 2008. Treasury Guidelines Preparation of Expenditure Estimates for the 2009 Medium Term Expenditure Framework. www.treasury.gov.za/publications/guidelines 16
Western Cape Provincial Treasury. 2003. “Western Cape Fiscal Policy 2003-2006: Financial Management Issues.” 17
National Treasury 2000. “In year management, monitoring and reporting.” 18
Auditor-General. 2007. “Information systems audit report on the system development life cycle followed in the implementation of the FINEST system at the Limpopo Provincial Government: Provincial Treasury.” 19
Keith Mackay, How to Build M&E Systems to Support Better Government, 2007 (IEG, World Bank)