Embed Size (px)
Citation preview
Supreme Audit Institution Performance Measurement
Framework (SAI PMF)
Presented by: Yngvild H. Arnesen, World Bank
The Exchange, Abu Dhabi, 13-15 May 2013
1
Presentation Outline
1. What is the SAI PMF and why is it developed?2. Why do a SAI PMF, and how?3. Practical example: Audit Service Sierra Leone 4. What is next on the SAI PMF?
4
What is the SAI PMF?
• A Performance Measurement Framework tailored to SAIs – developed by INTOSAI
• Intended to give a holistic, high level assessment of SAI performance against established INTOSAI points of reference:– The International Standards of Supreme Audit Institutions (ISSAI) framework;– The Framework on the Value and Benefits of SAIs;– Other INTOSAI guidance material and international good practices.
• Combination of objective measurement and qualitative assessment
• Assesses SAI performance in the country context and contributes to identifying the value and benefits of the SAI to citizens
Why is the SAI PMF developed?
Full set of International Standards of
Supreme Audit Institutions (ISSAIs) adopted by INTOSAI
in 2010
INTOSAI Framework on the Value and Benefits
of SAIs adopted 2010
Increased focus on use of country public financial management
systems within the international donor
community
How is the SAI PMF being developed?
Mandated by Johannesburg Accords, INTOSAI Congress, 2010
Responsibility of Working Group on the Value and Benefits of SAIs
SAI PMF Task Team• INTOSAI Regional Organizations: African Organization of English Speaking
SAIs(AFROSAI-E) and African Organization of Francophone SAIs (CREFIAF)
• SAIs: Bahrain, Brazil, Chile, Germany, India, Mexico, Russia, United Kingdom
• Development Agencies: Inter American Development Bank
• Coordinator:INTOSAI-Donor Secretariat @ IDI
SAI PMF Reference Group• INTOSAI Regional Organizations:
Pacific Association of SAIs (PASAI)
• SAIs: Canada, Denmark, Estonia, European Court of Auditors, Japan, Netherlands, Norway, Peru, Sweden
• Development Agencies:African Development Bank, OECD, European Commission, World Bank
• Civil Society Organizations:International Budget Partnership, Global Reporting Initiative (GRI), PEFA Secretariat
77
Qualitative Assessment
(Performance Report)
Measurable indicators
SAI PMF: Evidence Based Assessment of Performance
Objective measurement to inform qualitative assessment and track progress over time
Holistic, judgmental assessment of the SAI and its environment – including factors not covered by the indicators
B. Independence and Legal Framework
A. SAI Reporting
Value and Benefits of Supreme Audit Institutions
SAI Environment SAI Capability
Country Context, Governance and Public Financial Management
D. Audit Standards and Methodology
SAI SUPPORT SERVICES
E. Management & Support Structures
F. Human Resources and Leadership
G. Communication and Stakeholder Management
C. Strategy for Organizational Development
SAI STRATEGY
SAI CORE BUSINESS
SAI Performance Measurement Framework
SAI Reporting SAI Impact
Why do a SAI PMF assessment?
• Manage own performance: – Where are we? – Monitor progress over time
• Implementation of International Standards for Supreme Audit Institutions (ISSAIs)
• Demonstrate accountability • Provide evidence to donors on SAI
performance
11
Voluntary use
Who decides to initiate a SAI PMF?
Who decides when to do a SAI PMF?
Who decides how to do a SAI PMF?
Who decides if, when and how to publish a SAI PMF?
Head of SAI
Head of SAI
Head of SAI
Head of SAI
12
How will the SAI PMF Assessment be conducted?
SAI Involvement
1. SAI Self Assessment
2. Peer Review
3. External Assessment
• The approach will not necessarily affect how the assessment is carried out…
• … but the SAI may use the results for different purposes depending on the approach
13
Issues in measuring SAI performance
• SAI PMF is not designed to judge and rank SAI against each other, but aims to identify strengths and weaknesses of the individual SAI
• SAI’s should not be held accountable for matters over which they have no controlSolution: Clear distinction between factors within and outside the SAI’s control
• Some aspects of SAI performance are difficult to measure, especially ‘quality’Solution: Qualitative analysis in performance report, focus on processes and systems that should result in high quality audits
14
Issues in measuring SAI performance (2)
• The mandate of a SAIs differs across countries and may not be aligned with the ISSAIs. Solution: Assess the SAI against its own mandate (but assess its mandate against ISSAIs)
• The financial resources allocated to the SAI may not be sufficient to carry out its mandate; not appropriate to punish SAI for low performance on ‘coverage’, professional training etc.Solution: Assess SAI against its legal mandate, assess its reporting to parliament on coverage, include pertinent issues in performance report
16
• First round of pilots of SAI PMF: – Second half of 2012– Three SAIs: Djibouti, Norway, Sierra Leone– Carried out by SAI PMF Task Team
• Pilot of Audit Service Sierra Leone:– Field work in October 2012 in cooperation with
AFROSAI-E– Document analysis, review of audit files, interviews– Entire process: about six months
Piloting the SAI PMF
17
Independence & Legal Framework
SAI Reporting
Audit Standards & Methodology
Management & Support Structures
Strategy for Organizational Development
Human Resources & Leadership
Communication & Stakeholder Management
• Risk-based financial & compliance audit manual• Training based on manual• Quality assurance system• Performance audit manual• Documentation & quality control
• Supervision & review• Performance audit capacity• Planning & monitoring delivery & cost of audits
• Constitutional independence of SAI & AG• Mandate for public sector• Financial, compliance & performance audit• Freedom to publish reports• Financial independence: Budget process & receipt of funds
• Strategy guides organizational development & delivery of mandate• Monitoring weaknesses limits continual improvement of audit methods & public reporting of SAI performance
• Code of ethics, job rotation, structures• Monitoring staff utilization• Infrastructure & IT
• External training support• Link staff plan to mandate• Staff accountability & reward for performance• Attract & retain staff• Managerial vacancies• Training resources
• Reporting to PAC• Working with PAC• Radio debates• Internal comms.• Comms. with audited entities
• Financial audit coverage
• Timely submission & publication of audit reports
• Selection of performance audit topics
• Performance audit coverage
• Follow-up of audits
• Reporting performance
Results – Sierra Leone
18
Internal Factors Impacting on SAI Performance
Quality reports.
Coverage.Timeliness.
Poor documentation
Quality control not
implemented
Weak supervision &
reviewQuality
assurance system
Audit manuals
Staff
acco
unta
bility
for
perfo
rman
ceManagerial vacancies
Planning &
monitoring use of
(staff) resources
Staff motivation &
reward
Internal
comm
unications
19
External Factors Impacting on SAI Peformance – Sierra Leone
• Strong donor support• Participation in INTOSAI & AFROSAI-E programs• Weak accountability undermines cooperation with SAI
Context
• Weak record keeping, internal control & internal audit• Incomplete or missing financial statements• Unclear performance objectives & no measurement
PFM
• Weak financial independence• Otherwise strong legal framework
SAI Legal Framework
• Inadequate budget to hire & train sufficient, appropriate staff• Inadequate premises, vehicles, IT networks to function
efficientlySAI Resources
20
Value & Benefits of the Audit Service to Citizens
Fiscal Discipline
• Identification of:• Major revenue
discrepancies• Failings in collection
of fisheries revenues
• Audit of public debt management
Efficient Resource Allocation
• Ensures funds spent for purposes intended
Effective Service Delivery
• Legality of spending• Identification of
failings in:• Anti-malarial drug
delivery systems• Access to clean
water program• Distribution of
agricultural inputs
Promoting fiscal transparency and accountabilityDeterring, preventing & detecting corruptionCommunicating with & engaging citizens: demonstrating relevance to stakeholders
21
Added Value for Audit Service Sierra Leone
• Identification of strengths and weaknesses – knows what to focus on in future
• Knowledge about weaknesses caused by internal factors which are under the SAI’s control…
• … and weaknesses which are caused by external factors that the SAI may try to influence
• Foundation for communicating value and benefits of the Audit Service
• Used as baseline for tracking performance over time – linked to external funding
• Used to inform design of capacity building interventions
4. What is next in the development of SAI PMF?
SAI PMF Roll-out
• INTOSAI Congress 2013: Official exposure draft• Training: For assessors and future trainers – in
all INTOSAI regions during 2013/2014• Piloting: Around 20 volunteer SAIs with
different characteristics • Guidance material: Prepared by IDI• Support function @ IDI
Thank you!
