PRESENTED BY: RAVEEN SARJU
(SENIOR AUDIT MANAGER : TRANSVERSAL AUDITS )
ETHEKWINI MUNICIPALITY AUDIT AND RISK ASSURANCE SERVICES (EMARAS)
INTERNAL AUDIT
THE AUDITING OF PERFORMANCE INFORMATION (PREDETERMINED OBJECTIVES) PRESENTATION
01 NOVEMBER 2018
AGENDA
NO DESCRIPTION
A. DEFINITION AND PURPOSE OF INTERNAL AUDIT
B. THE DIFFERENT ROLES OF KPI OWNERS , PME UNIT AND INTERNAL AUDIT
C. THE ROLE OF INTERNAL AUDIT
D. AUDIT PROCESS AND QUALITIES OF EVIDENCE
E. THE ROLE OF OVERSIGHT COMMITTEE– AUDIT COMMITTEE
F. AUDIT CHALLENGES
G. TYPES OF AUDIT FINDINGS
H. PERFORMANCE FUNCTIONALITY AUDIT
I. PREPARATION FOR AGSA AUDIT REVIEW
DEFINITION AND PURPOSE OF INTERNAL AUDIT
The MFMA and King report strongly emphasize the need for an Audit Committee
and an Internal Audit function:
OUR PRIMARY CONSIDERATION
Institute of Internal Auditors (IIA)
Internal Audit is an independent, objective, assurance and consulting activity that
adds value to and improves an organization's operations. It helps an organisation
accomplish its objectives by bringing a systematic, disciplined approach to evaluate
and improve the effectiveness of governance, risk management, and control processes.
What is Internal Auditing?
AttributeStandards
PerformanceStandards
Institute of Internal Auditors (IIA)Standards
Implementation Standards
Our values…… our office ambiance…….
Integrity
Accountability
Honesty
Equality
Fairness
Objectivity
Confidentiality
Competency
Values
THE ROLE OF MANAGEMENT / KPI OWNERS
Purpose of Performance Management
Performance Management defined as “a strategic approach to management, which equips leaders,
managers, employees and stakeholders at different levels with a set of tools and techniques to regularly
plan, continuously monitor, periodically measure and review performance of the organization in terms
of indicators and targets for efficiency, effectiveness and impact”
ROLE OF MANAGEMENT & KPI
OWNERS
Developing & Defining KPI,
Project Plans & Setting of KPI
targets,
Capturing, reporting of KPI scores (monthly
& quarterly) on PMS system, Uploading of POE, Review of KPI
scores captured.
Compiling of POE and making available for
audit purposes including source
documents
Attend timeously to audit queries and findings , provide
management comment and action plans
Regular Liaisons with PME Unit regarding KPI performance ,
SMART, definitions, mid terms
amendments, POE issues,
Monthly Business review sessions to
continuously evaluate KPI
performance, relevance of target,
POE etc.,
Role of Management / Plan & KPI Owners
1st Line of Defence
THE ROLE OF PERFORMANCE MANAGEMENT & EVALUATION (PME)
Role of PME in Performance Management
2nd Line of Defence
Develop and Implement Performance Management framework and systems to ensure the reporting of performance information for the Municipality
Ensuring and Advising
KPI owners in :
Aligning of National Strategic Focus areas (SFA’s) to IDP 8 point plan
Aligning of KPI’s on the scorecard to SDBIP and Budget
Developing ‘SMART’ KPI’s
KPI definitions
Developing Project plans and milestones for KPI’S ,
Compiling Portfolio of Evidence
Responsible for the Performance Management systems to facilitate monthly and quarterly reporting –Organisational scorecard
Quality Review of reported information by management
Report monthly and quarterly to City Manager , Council and oversight structures
(Audit Committee , MPAC )
THE ROLE OF INTERNAL AUDIT
Legislative - Audit Mandate
Municipal Systems Act, Act no.32 of 2000, Section 45 states that the results of performance measurements in terms of section 41 (1) (c) must be audited:
As part of the Municipality’s Internal Auditing processes; and
Annually by the Auditor General.
Section 165(2)(b) of the Municipal Finance Management Act, 2003 (Act 56 of 2003) prescribes that the Internal Audit Unit of the Municipality or Municipal Entity
must advise the Accounting Officer and report to the Audit Committee on the implementation of the Internal Audit plan and matters relating to:
Internal Audit;
Internal controls;
Accounting procedures and practices;
Risk and risk management;
Performance management;
Loss control; and
Compliance with the MFMA, the annual Division of Revenue Act and any
other applicable legislation.
Legislative - Audit Mandate
Municipal Planning And Performance Management Regulations, 2001
Section 14. (1) (a) A municipality must develop and implement mechanisms,
systems and processes for auditing the results of performance measurements as
part of its internal auditing processes.
(b) Any auditing in terms of paragraph (a) must include assessments of the
following:
(i) The functionality of the municipality’s performance management system;
(ii) whether the municipality’s performance management system complies with the
Act; and
(iii) the extent to which the municipality’s Performance measurements are reliable
in measuring
performance of municipalities on indicators referred to in regulation 9 and 10
(c) A municipality’s internal auditors must -
(i) on a continuous basis audit the performance measurements of the municipality;
and
(ii) submit quarterly reports on their audits to the municipal manager and the
performance audit
committee
Assess the functionality of the Performance Management &
Evaluation Unit.
Evaluate whether the system complies with applicable Acts.
Determine whether the performance measurements are
reliable in measuring performance.
Continuously audit the performance measurements of the
municipality.
Submit quarterly reports on the audits of performance
information to the Municipal Manager and the performance
Audit Committee.
Internal Audit role in Performance Management System (PMS)
3rd Line of Defence
Risk Based Approach
Quantitative consideration – relate to the Total budget both Capital and Operating budget size) of thedevelopment priority and strategic objectives as reflected in the Integrated Development Plan (IDP).
Qualitative consideration - take into account the information needs of the stakeholders and users of theperformance information, the qualitative significance of the development priorities and strategic objectivesagainst qualitative criteria (basic service delivery, financial sustainability).
Legislative reporting requirements of KPIs – KPIs that require monthly reporting to National Treasury andother external parties
KPIs financed by external or grant funding - KPIs that require compliance with conditions of finance or grantagreement.
Service Delivery Backlogs – KPIs directly relate to the level of service in addressing backlogs, performance ofbusiness against objectives
Frequency of the KPIs in the relevant Performance Management reporting cycle. (quarterly, biannually andannually per plan)
AOPI Audit Objectives
Review performance information as reported in the scorecard and the evidence in support of the performance scores
Determine the accuracy, completeness and validity of performance information reported
Evaluate the targets in terms of the “SMART” criteria
Evaluate the risks and control measures of the systems used to manage and report performance information
THE AUDIT PROCESS AND QUALITIES OF EVIDENCE
AOPI Audit Process
Evaluate portfolio of evidence in terms of Accuracy, Validity and Completeness for those KPI’s reported as achieved.
Confirm the achievement or non- achievement of the
performance target
Conclude on the KPI’s in terms of the ‘’SMART” criteria
Report audit findings to respective plan owners, KPI
owners for response and action plans to address poor
Performance
Determine trend analysis of KPI’s that are continuously
not achieved throughout the performance reporting cycles
Quarterly reporting to the Audit Committee on the results and outcome of
Performance information
Qualities of Audit Evidence
Evaluation of Portfolio of Evidence
• Information is the best attainable information through the use of appropriate methods. Reliable information is competent, valid which accurately represents the status of the KPI being reported.
• Information supports engagement observations and recommendations and is consistent with the objectives for the engagement.
• Information is factual, adequate, and convincing so that the prudent person would reach the same conclusion. The information should have a degree of persuasiveness .
• Information helps the Municipality to meet its goals
Useful Sufficient
ReliableRelevant
Qualities of Audit Evidence
Accuracy
Completeness
Accuracy is free from errors and distortions and are faithful to the underlying facts. The re-performance of a calculation or formulae with the same information should yield the same result as the actual score
presented by the KPI owner.
Completeness include all significant and relevant information that the auditor will be able to verify and validate from source documentation to final result of the KPI status reported and vice versa (i.e. from final
result back to source documentation)
Validity
The information supports the actual score reported based on source documents and actual work completed to fairly represent that status of
the KPI achievement for the reporting period in question.
Specific
Measurable
Achievable
Relevant
Time - Bound
SMART Principles
“SMART” Acronomy
A strategic outcomes oriented goal should ideally be written as a statement of intent that is specific,measurable, achievable and time bound. A useful set of criteria for selecting performance targets is the"SMART" criteria:
• The nature and the required level of performance can be clearly identifiedSpecific
• The required performance can be measured.Measurable
• The target is realistic given existing capacity.Achievable
• The required performance is linked to the achievement of a goalRelevant
• The time period or deadline for delivery is specifiedTime-bound
What Internal Audit Don’t Do?
Do not own the process for reporting or quarterly performance management
Do not develop , design KPIs , project plans , KPI definitions for plan owners
Do not manage the electronic systems used in the reporting of performance information.
Do not produce , maintain a portfolio of evidence for KPIs or sub projects.
Do not make decisions when disputes between KPI owners and PME arise affecting KPI achievement or portfolio of evidence, project plans etc.
THE ROLE OF OVERSIGHT COMMITTEES
Council
Mayor / EXCO
Portfolio/ EXCO
Committee’s
MPAC
Audit and Risk
Committee
Oversight Role
Review and comment on compliance with statutory requirements and performance management best practices and standards.
Review and comment on the alignment of the Integrated Development Plan, the Budget, Service Delivery and Budget Implementation Plan and performance agreements.
Review and comment on relevance of indicators to ensure they are measureable and relate to services performed by the municipality and its entities.
Reviews compliance with in-year reporting requirements.
Review the quarterly performance reports submitted by internal audit.
Review and comment on the municipality’s and entities annual reports within the stipulated timeframes. Review and comment on the municipality's performance management system and make recommendations for its improvement.
Meet Quarterly during the financial year. Review the municipality’s Performance Monitoring and Evaluation system and make recommendations to EXCO & Council.
Role of the Audit Committee in
Performance Management
The Audit Committee will be responsible for
monitoring the assessment by Internal Audit of
the performance management system.
Includes reviewing the strategy and plan to
monitor
Evaluation of the implementation and
effectiveness of performance management
processes and framework; the results of the audit
of performance management and the quarterly
internal audit reports on performance
management and performance information.
Submit audit report to EXCO & Council at least
twice during the financial year.
Audit Committee role in implementation
of Performance Management System
Review Annual report in terms of matters raised by AG
including Performance management and seek accountability
and responsibility in developing their report to Council.
Role of Municipal Public Accounts
Committee (MPAC)
AUDIT CHALLENGES
Audit Plan vs. Reality
Tight reporting timelines to Audit Committee on performance information.
Non Availability and lack of timeous response from KPI owners to audit queries and issues
Weak management responses and action plans to address audit findings.
Poor attendance by Plan owners at Audit committee meetings
Oversight of changes/deletions of KPIs during Midterm.
Challenges
Increased number of KPIs and sub-projects to audit
Extent of audit testing and audit sample size
Decentralisation of POE and source documentation
Culture of Performance management not embedded and integrated into business processes
Client relationships and expectations of KPI owners impacting on future audits
Challenges
34
Resource Planning
TYPES OF AUDIT FINDINGS
Types of Audit Findings Raised
• No Portfolio of evidence submitted for Audit purposes.
• Non Achievement due to Portfolio of evidence with insufficient, irrelevant, or inaccurate information.
• KPIs not meeting the ‘SMART’ principle.
• Pure non-achievement as reflected on the scorecard.
• Lack of Project plans / milestones / KPI definitions.
• KPI Scores not reported on scorecard.
Types of Audit Findings
Raised
• Incorrect weighting of sub projects failure of one sub project leads to total non achievement of the KPI.
• Evidence not uploaded on the Performance Management system.
• Consistent Non achievement of KPI’s in the past 3 years.
• No Targets defined for KPI’s to enable quarterly reporting.
• Absence of KPI definition with No defined linkage to sub-projects and milestones negatively contributing to overall achievement of KPI’s.
• Oversight of changes/deletions of KPIs during Midterm.
• Impact on non achievement on individual performance plans and consequence management.
INTERVENTIONS REQUIRED FROM KPI OWNERS
Identify KPIs performing poorly and
hold relevant Plan & KPI Owners to account
Zoom into areas of pure non-achievement with specific emphasis on
consistently non –achieved KPIs and
recommend actions plans to address
Engage with relevant standing committees
on their areas of oversight that are performing poorly
Performance Bonus cant be paid out when performance Targets
are not Achieved
39
Areas of Improvement & Learnings
Engagement of plan owners and relevant KPI owners on audit outcomes
Monthly action tracking as part of business unit meetings; and quarterly Self Assessment outcome prior to audit
Quarterly business Performance presentations by business to A BUSINESS PERFORMANCE meeting and reporting to Accounting Officer
Engagement with Performance Management Evaluation Unit throughout audit process
AG liaisons on AOPI issues, scope as well as reliance work
40
Impact on Risk Profile
Non-achievement of predetermined objectives indicates a risk to organizational strategic objectives
The reasons associated with not meeting the targets, may also be indicative of other emerging risks
Risk treatment plans to be put in place to expedite delivery
This must inform the quarterly risk reviews
Consideration must be given in the development of the following year scorecard and organizational risk profile
PERFORMANCE AUDIT FUNCTIONALITY
42
Regulatory Audit
This is a mandatory AGSA Regulatory audit for all municipalities in terms of Municipal Planning And Performance Management Regulations, 2001
Section 14. (1)(a) A Municipality must develop and implement mechanisms, systems and processes for auditing the results of performance measurements as part of its internal auditing processes.
(b) Any auditing in terms of paragraph (a) must include assessments of the following:
(i) The functionality of the municipality’s performance management system
(ii) whether the municipality’s performance management system complies with the Act; and
(iii) the extent to which the Municipality’s Performance measurements are reliable in measuring performance of
Municipalities on indicators.
43
Regulatory Audit
A compliance audit of the performance regulations relating to the: • IDP• Budget • Performance Management
Audited using a comprehensive compliance checklist of the relevant regulations and supporting evidence of the process/events/activities that occurred in the previous, current and future financial reporting periods of the Municipality.
44
Regulatory Audit
Audit focus areas:
1. Existing of policies and procedures • Performance Management Framework• Performance Management System • Strategic Policies
2. Compliance with laws and regulations• Municipal Finance Management Act No. 56 ,2003• Municipal Systems Act 32 Of 2000.• Municipal Planning and Performance Management Regulations • Municipal Performance Regulations for Municipal Managers• Municipal Budget and Reporting Regulations
45
Regulatory Audit
3. Monitoring and Reporting process • IDP• Budget • Organizational Scorecard and SDBIP
2. Employees Performance management (Executive Management)• Performance Agreements• Performance Contracts• Performance evaluation/assessments
46
Regulatory Audit Guidelines And Approach
Objective Procedures
Obtain understanding of internal control Environment Is there an official responsible for function? Are roles and responsibilities formalized and implemented? Is there monitoring – senior management and performance/audit committee? Are guidelines/ framework/ templates/formats implemented? Is there adequate segregation of duties?
Information Has policy for development of performance indicators and the recording and reporting of information been
formalized and implemented? Are staff fully aware of policies and been trained?
Monitoring Does management prepare quarterly reports? Does management monitor controls in place to ensure performance information is accurate and complete.
Obtain copy of previous reports to note concerns and follow-up with management to ascertain if any action taken to rectify.
Understanding of system Document system implemented to Establish measurable objectives, performance measures, indicators and targets during strategic and annual
planning process. Collect and record data (frequency/electronic or manual) on quarterly basis and annual reporting Monitor and report on performance information on quarterly and annual basis.
By recording process, designation of person(s) responsible, inputs, outputs, systems, documents, policies and procedures and laws and regulations.
Approved policy and procedures Establish the existence of approved policies and procedures for performance measurement.
Procedures and polices are aligned to laws and regulations
Review legislation, policy and procedures and process implemented to evaluate any deficiency in the policy and or non-compliance with legislation.
Process implemented in accordance with polices and procedures
Perform a walk through on system documented and note any deviations from documented system and procedures and polices.
47
Regulatory Audit Guidelines and Approach
Objective Procedures
Adequate monitoring of system
Review the process documented and results of above to determine if is being adequately monitored by management (review, comparisons, independent checks, exceptions reports, follow-up of corrective action etc)
Consistency and alignment
Compare objectives and KPI’s from IDP to budget to objectives reported in the annual report to ensure consistency
Compare current year measurable objectives to previous year and from quarter to quarter in the current year
Compare a sample of objectives/SFA and KPI’s from organisational scorecard to the CM and individual performance plans of Direct Reportees
Review IDP and scorecard to ensure that SFA/objectives has KPI’s and targets. Inspect measurements to determine if clearly linked to mandate of municipality and Is aligned to priorities and strategies
Quality and presentation
Inspect reported information and determine whether it is easy to follow from objective to KPI and target (straightforward and meaningful and not fragmented)
Inspect to ensure that both financial and non-financial indicators Inspect indicators to ensure meet the SMART principle. Inspect to ensure that actual performance is reported. Inspect annual report to ensure that reasons for deviation from planned is included.
Compliance How does the municipality ensure compliance to laws and regulations. Test for compliance - Refer to listing of legislation.
Other – Has policies, procedures and systems implemented taken into account guidelines as per framework.
PREPARATION FOR AGSA AUDIT
49
Guidelines to Prepare for AGSA Audit Review
Attending and resolving previously reported AG findings and queries and incorporating learnings and lessons to improve efficiencies in the AOPI processes.
Increasing AG reliance on the work of Internal Audit through:• Comprehensive and detailed audit methodology on AOPI • Adequate audit sample size, working papers, evidence etc.• Competent and skilled staff resources
Preferably an automated performance management systems that is reliable and accurate performance information
Regular communication and engagement with AGSA prior and post audit on:• Understanding AGSA approach • Technical matters and updated legislation
50
Guidelines to Prepare for AGSA Audit Review
Clearly defined roles and responsibilities in the Performance management policy & framework between KPI owners,PME and Internal audit including understanding of the AOPI process.
AGSA predominately focusses on service delivery and financial plans / KPIs requires management systems , processand information to be adequate and effective.