Compliance System Validation- An Audit Based Approach

December 2012

Uday Gulvadi, CPA, CIA, CISA, CAMSDirector - Internal Audit, Risk and Compliance

Mahesh Viswanathan, CAMSSr. Vice President

2

• Wide range of service providers and skills• Inconsistent quality of the assessment and

deliverables• Often independent contractors are used

resulting in lost continuity year to year• Lacking consistent standards of performance• Findings frequently not tied to risk and

potential impact• Level of independence is not always clear

Current Challenges

System Validatio

n

Independent

Assessment

System Review

System Verificatio

nSystem Audit

Independent Review

Terminology

3

• Boards and management are recognizing both o Need to perform independent validations of

systems and o Lack of consistent high quality “audit based”

assessments in the past• Critical role of technology in BSA/AML Compliance

program • Increased scrutiny by regulators• Mitigate the probability and impact of critical risk

events • Avoid severe regulatory penalties and reputational

risk

Need for an Audit Based Approach

4

• Required by FFIEC BSA Examination Manual:o “A periodic review of the effectiveness of the suspicious

activity monitoring systems (manual, automated, or a combination) used for BSA/AML compliance.”

o Evaluate the system’s methodology for establishing and applying expected activity or filtering criteria

o Evaluate the system’s ability to generate monitoring reports (Cases/alerts)

o Determines whether the system filtering criteria are risk based & reasonable.

o Validate the auditor’s reports and work papers to determine whether the bank’s independent testing is comprehensive, accurate, adequate, and timely.

Need for Audit Based Approach

5

Independent &

Objective

Systematic,

Disciplined

approach

Assess conforma

nce to regulatio

ns, policies & procedur

es

Assess the

culture of complian

ce

Identify control

weaknesses and

remedial measures

Follow up on action

taken

6

What is an Audit based approach?

Knowledge of regulatory expectationsRisk Based approachUnderstanding of the “red flags” unique to the business

Distinguish regulatory violations and best practices.

Internal or Third-Party Credentials and Experience

Appropriate, robust report, work papers

7

Essential Requirements for Audit Based approach

Audit Compliance Technology

Planning and

Scoping

Assessment

Validation Report

Follow up Review

8

Audit based approach phases

• Should be performed by qualified individuals within the FI or by a qualified third party

• Should be performed annually or should match the frequency of Risk Assessment

• Should consider the alignment of BSA AML System with Risk Assessment includingo Customerso Geographieso Lines of Businesso Products and Services

Independent Validation - Components

9

10

Independent Validations - Coverage

Typical Coverage• Data Mapping, Interfaces and

Reconciliations• Risk Model• Customer Due Diligence and

EDD• Profile configurations• AML Monitoring rules –

Thresholds, Effectiveness & Efficiency

• Audit Trails• Case Management

• Match Level Management• Sanctions Filtering Rules –

Thresholds, Effectiveness & Efficiency

• Batch, Real Time and Incremental Filtering

• Business and Functional Requirements

• User Acceptance Testing• Application Security and

administration

11

• Assessing the functionality of rules and that the data supports rule processing  

o Logic is not always transparento Flaws in logic processingo Too many false positives

• Validating all required SWIFT Messages are being scanned

• Inconsistent thresholds on rules/scenarios leading to incorrect or no alerts

• Absence of data or poor data quality providing incorrect customer risk classification

Technical Challenges

Staff and ManagementImplements

BSA/AML Compliance Monitors

Independent Audit

Assesses independent

ly

12

Organization’s Roles & Responsibilities

1st Line of Defense

2nd Line of Defense

3rd Line of Defense

Identify high risk services, products

and clients

Consider results

of recent audit

and regulatory

examinations

Resolutio

n of past

remediati

on items

Well-organized work papers

evidencing assessment

Document clear linkages between

risk and assessment program

13

Keys to an Effective Validation

14

Audit based Performance Standards

• Consistent with professional practice standards

• Audit procedures and testing commensurate with risk

• Quality Assurance reviews • Build on knowledge of best practices• Continuous improvements methodology• Confidentiality and Security protocols• Specialized analytical tools

15

• Assessment Reporto Key observationso Associated risks and potential impacto Recommendations for risk remediation

• Significant Items Management Action Plano Living document with significant findingso Management responseso Remedial action plan with “Ownership” and due dates

• Test Work Papers and Supporting Documentation

Deliverables

16

• Should integrate three essential skillsets:o Audit expertiseo Compliance & regulatory knowledgeo Strong technology and in-depth product knowledge

• Well defined structured process/framework that is adaptive

• Completely independent• Continuity of permanent staff• Professional Certifications – CPA, CIA, CAMS CCRP

etc.• Good customer references

How to select a Third Party Vendor?

Internal Staff or Third-Party Credentials and Experience

Knowledge of Regulatory

Requirements

Understands Your

Institution

Establishing Expectations

17

Essential qualifications

Audit Compliance

Technology

18

Questions