Embed Size (px)
Citation preview
Dr Ian Goldman,Head: Evaluation and Research
PSPPD PSC26 August 2014
The Presidency Department of Performance Monitoring and Evaluation
National Evaluation System
The Presidency: Department of Performance Monitoring and Evaluation
Overall situation with evaluations39 evaluations completed, or underway, of
which 37 are NEP evaluations.
22
Improvement plans being implemented
Served at Cabinet
Approved reports
Research underway
TORs approved
Preparation stage
Stuck
8 2 11 23 0 2 2
Cost benefit - R50 billion of programmes evaluated (over MTEF). 10% improvement = R5 billion, annual cost of evaluation system R50 million, so very high return on investment.
The Presidency: Department of Performance Monitoring and Evaluation
Nos of departments involved
15/46 national departments now involved in evaluations
Nos of national + provincial departments involved in evaluations risen from 13% to 23%
33
The Presidency: Department of Performance Monitoring and Evaluation
NEP 2012/13Dept Title of evaluation Status as 5 Maydti Implementation/ design evaluation of the
Business Process Services ProgrammeFinal report approved. Report on way to Cabinet
DBE Impact Evaluation of Grade R Final report approved. Report been to Cabinet
Health Implementation Evaluation of Nutrition Programmes addressing Children Under 5
Final report approved and been to Cab Committee
DRDLR Implementation Evaluation of the Land Reform Recapitalisation and Development Programme
Final report approved and been to Cab Committee
DRDLR Implementation Evaluation of CRDP Final report approved and been to Cab Committee
DHS Implementation Evaluation of the Integrated Residential Dev. Programme
Evaluation underway. DHS procurement has delayed
DHS Implementation Evaluation of the Urban Settlements Development Grant (USDG)
Evaluation underway. DHS procurement has delayed
DBE Impact Evaluation of the National School Nutrition Programme (NSNP)
DBE requested to drop. Reallocated to 2014/15.
44
Completing Underway About to start Delays!
The Presidency: Department of Performance Monitoring and Evaluation
NEP 2013/14Dept Title of evaluationPresidency Implementation Evaluation of Government’s Coordination Systemsdti Evaluation of Export Marketing Investment Assistance Incentive programme
(EMIAI)dti Evaluation of Support Programme for Industrial Innovation (SPII)dti Impact Evaluation of Technology and Human Resources for Industry
Programme (THRIP) Military Veterans
Evaluation of Military Veterans Economic Empowerment and Skills Transferability and Recognition Programme.
DST Evaluation of National Advanced Manufacturing Technology StrategySARS Impact Evaluation on Tax Compliance Cost of small businessesCOGTA Impact evaluation of the Community Works Programme (CWP)DRDLR Evaluation of the Land Restitution Programme
DAFF Impact Evaluation of CASPDAFF Implementation Evaluation of MAFISA
DHS Baseline for informal settlements targeted for upgrading
DHS Evaluating interventions by DHS to facilitate access to the city.
DHS Diagnostic of whether the provision of state-subsidised housing has addressed asset poverty for households and local municipalities
DPME Impact Evaluation of the Outcomes Approach
55
Completing by Sept Underway About to start Delays!
The Presidency: Department of Performance Monitoring and Evaluation
2014/15
66
Dept Evaluation
DEA Evaluation of the Effectiveness of Environmental Governance in the Mining Sector (EEGM)
DHET Design Evaluation of the Policy on Community Education and Training Colleges (PCETC)
DHS Impact Evaluation of the Social Housing Programme (SHP)DST Evaluation of the Indigenous Knowledge Systems Policy (IKSP)DSD Diagnostic Evaluation/Programme Audit for Violence Against Women and
Children (AVAWC)DSD Diagnostic Review of Coordination of the Social Sector Expanded Public
Works ProgrammeSAPS Economic Evaluation of the incremental investment into the SAPS Forensic
ServicesDAFF/DRDLR Impact Evaluation of the Ilima Letsema Programme and Irrigation SchemesDAFF Impact evaluation of MAFISA (quantitative) – through 3ie
DAFF/DRDLR Policy Evaluation of Small Farmer SupportDBE Evaluation of the Funza-Lushaka Bursary SchemeDBE Impact evaluation of National School Nutrition Programme
DRDLR Impact evaluation of Land Restitution Programme – through 3ie
DPME Impact/implementation evaluation of the MPAT systemDPME Implementation evaluation of the dept strategic planning and APP system
Underway TORs developed
No TORs yet
Delays!
The Presidency: Department of Performance Monitoring and Evaluation
Evaluations selected for 2015/16
Impact/implementation evaluation of the evaluation system (DPME). Implementation evaluation of citizen-based monitoring programme
(DPME). Agricultural Extension Recovery Plan (DAFF). Evaluation of the New School Curriculum (DBE). Evaluation of the Asset Forfeiture Unit Sub-programme (NPA). Diagnostic evaluation of the Non-Profit Organisations Regulatory
Framework and Legislation (DSD). Implementation Evaluation of the National Drug Master Plan in
addressing all forms of Substance abuse (DSD). Evaluation of the effectiveness of the South African Post-School
Education and Training (PSET) quality assurance regime (DHET). Evaluation of Early Grade Reading in SA (DBE). Evaluation of the Mining Charter Evaluation of the service delivery improvement planning system
77
The Presidency: Department of Performance Monitoring and Evaluation 88
MTSF Outcome All evaluations since 2009
Nos of evaluations
in NEPs
1. Improved quality of basic education. 15 5 2. A long and healthy life for all South Africans. 3 1 3. All people in South Africa are and feel safe. 4 3 4. Decent employment through inclusive economic
growth. 14 9
5. A skilled and capable workforce to support an inclusive growth path.
9 3
6. An efficient, competitive and responsive economic infrastructure network.
1 0
7. Vibrant, equitable and sustainable rural communities with food security for all.
12 8
8. Sustainable human settlements and improved quality of household life.
7 6
9. A responsive, accountable, effective and efficient local government system.
3 2
10. Environmental assets and natural resources that are well protected and continually enhanced.
5 1-2
11. Create a better South Africa and contribute to a better and safer Africa and World.
0 0
12. An efficient, effective and development oriented public service and an empowered, fair and inclusive citizenship
10 5
13. Social Protection 8 6 14. Social Cohesion 3 1
The Presidency: Department of Performance Monitoring and Evaluation 99
Programme being evaluated Outcome needed from evaluation results
Close Major changes needed
Minor changes needed
No changes needed
Early Childhood Development (ECD) X Business Process Services Programme X Grade R (reception year of schooling) X Nutrition Programmes addressing under 5s X Land Recapit. and Dev Prog (RECAP) X Comprehensive Rural Dev Prog (CRDP) X Export Marketing Investment Assistance X Support Prog for Industrial Innovation (SPII)
X
Land Restitution Programme X Government Coordination System (clusters/MinMECs/Implementation Forums)
X
Micro Agric Financial Institution (MAFISA) X Presidential Intervention in KSD X
Many programm
es need major
changes
The Presidency: Department of Performance Monitoring and Evaluation
Progress with National Evaluation System
3 National and Provincial Evaluation plans to date, last week selected evaluations for 4th
39 evaluations completed, underway (11 completed, 2 public) 18 guidelines, evaluation standards, competences, trained >600
government staff Building demand – emphasising learning, reports to Cabinet, working
with Parliament, publicising reports, training for DGs/DDGs in use of evidence
Evaluation Repository – audit of evaluations since 2006, plus new - 103 evaluations on the website
2 provincial evaluation plans (WC and GP), working with 4 more provinces, and 5 departments have departmental evaluation plans
First Annual Report on the National Evaluation System (for 2013/14) now available
1010
The Presidency: Department of Performance Monitoring and Evaluation 1111
The Presidency: Department of Performance Monitoring and Evaluation 1212
The Presidency: Department of Performance Monitoring and Evaluation 1313
The Presidency: Department of Performance Monitoring and Evaluation
Early indications of policy impact Role of improvement plans - produced for ECD, Land Restitution,
Grade R, CRDP, RECAP and Business Process Services. First 6 month report for ECD, 2nd due (late). Apart from the ECD evaluation, the next evaluation was only
commissioned in October 2012 – ie 18 months prior to this report. It is therefore early days to see concrete impacts of the evaluations.
Early days, some examples of policy influence can already be seen: New ECD policy drafted responding to the ECD Diagnostic Review, including
the need to target children from conception; Renewed focus on quality of Grade R rather than just rollout as a result of
the Grade R evaluation – included in MTSF. Renewed focus on nutrition in children resulting from the evaluation of
nutrition interventions for children under 5, and a stunting target in the Medium-Term Strategic Framework. The Improvement Plan will take this further.
Revised Business Process Outsourcing Scheme launched by Minister Davies in UK
1414
The Presidency: Department of Performance Monitoring and Evaluation
Grade R – short-term targets
1515
The Presidency: Department of Performance Monitoring and Evaluation
Key challenges Inadequate supply of skilled evaluators - range of skills
needed from proposal writing, evaluation methodology, writing reports DPME working with universities and World Bank,
promote training of SPs Insufficient PDI participants
Changing panel criteria to promote access, but keep quality, have 30% PDI requirement in CFPs
Some evaluations are taking a long time to be brought by departments to cluster (often 6+ months) and thence Cabinet. DPME will bring if takes longer than 4 months.
Departments taking long (6 months+ delay) to finalise improvement plans arising from evaluations, and to report on progress. Include in MPAT standards and discuss with AG.
1616
The Presidency: Department of Performance Monitoring and Evaluation
Challenges (2) Weak programme planning
Cabinet agreed need for support programme using government/donor resources. Also develop MPAT standard to drive change.
Insufficient funding for complex evaluations Programmes should reserve between 0.1% and 5% of
programme budgets for evaluation. Some departments who are funding NEP evaluations
themselves are taking a very long time to procure (notably DHS).
These should be in departmental evaluation plans and DPME procures for all evaluations selected for NEP. Need to meet Minister DHS to discuss delays on 5 evaluations.
Some sectors/outcomes not proposing evaluations – notably Health, Economic Infrastructure, Environment, Public Service, Local Government.
FOSAD suggested propose and 6 evaluations being proposed to Cabinet on Crime, Health, Public Service
1717
The Presidency: Department of Performance Monitoring and Evaluation
Taking the evaluation system forward Undertaking audit of implementation
programmes in gov. – may well be 1000! If each were to be evaluated every 5 years = 200/year – big challenge
To do that extending national evaluation system to provinces in 2014-15/2015-16 and to departments from 2015-16. Is an issue of capacity to support 46 national depts and 9 provinces.
Focus on programme planning (again needs resources)
1818
The Presidency: Department of Performance Monitoring and Evaluation
ResearchWork starting on a diagnostic which will inform a
research strategy – balance between work we do on actual research, vs support to influence the research system so it provides research relevant to the outcomes Some work being undertaken internally in DPME PSPPD supporting a research exercise in a number of
provinces and departments to see what is happening and what role we should play
Working to develop immediately a repository of key research around each outcome
Assignment on DG workload – starting
1919
The Presidency: Department of Performance Monitoring and Evaluation
Thank you
Dr Ian GoldmanHead: Evaluation and [email protected]
2020
