Embed Size (px)
Citation preview
Preparing for $10 Billion and Beyond
Mike Guglielmo
Managing Director
David Ballas
Sr. Sales Executive
The Pathway to SuccessThursday, April 27 10:30-11:20
Page 2
Preparing for $10 Billion and Beyond
Page 3
A
B
C
D
Introduction
Key areas of focus, lessons learned (fact vs. fiction)
Dodd-Frank Act Stress Test (DFAST)
Regulatory context, goals, issues and challenges, major modeling decisions
Model Risk Management
MRM framework, the “three line of defense”, effective challenge
Concluding Thoughts
Getting strategic value from DFAST and MRM, Q&A
Survival PlanningPREPARING FOR $10 BILLION AND BEYOND
Page 4
A
B
C
D
Introduction
Key areas of focus, lessons learned (fact vs. fiction)
Dodd-Frank Act Stress Test (DFAST)
Regulatory context, goals, issues and challenges, major modeling decisions
Model Risk Management
MRM framework, the “three line of defense”, effective challenge
Concluding Thoughts
Getting strategic value from DFAST and MRM, Q&A
Survival PlanningPREPARING FOR $10 BILLION AND BEYOND
Page 5
Three Notable Areas of Focus
• Consumer Financial Protection Bureau (CFPB)
• Dodd-Frank Act Stress Testing (DFAST)
• Model Risk Management (MRM)
Page 6
Fact or Fiction: Lessons LearnedREALITIES AND MISCONCEPTIONS
• Regulators will give you time once you cross $10B in assets
• Stress testing is just for regulatory compliance
• Average DFAST development costs exceed $1m
• DFAST can be fully outsourced
• It is challenging to find/retain quality resources
• Good documentation is crucial to success
Page 7
A
B
C
D
Introduction
Key areas of focus, lessons learned (fact vs. fiction)
Dodd-Frank Act Stress Test (DFAST)
Regulatory context, goals, issues and challenges, major modeling decisions
Model Risk Management
MRM framework, the “three line of defense”, effective challenge
Concluding Thoughts
Getting strategic value from DFAST and MRM, Q&A
Survival PlanningPREPARING FOR $10 BILLION AND BEYOND
Page 8
DFAST
Page 9
Mandatory Stress Testing:
A Response to Taxpayer “Bailout”GOAL: PREVENT BAILOUTS; TOOLS: GOVERNANCE & IMPROVED RISK MANAGEMENT
• Initial focus upon top 5-6
Recovery Planning & Resolution Plans
Stress Testing
• Then CCAR top 18-19
Comprehensive Capital Analysis & Review
Consistent Stress Testing
• Then CapPR over $50B
Capital Plan Review
• Then $10B-$50B DFAST institutions
• Community Banks?
Page 10
Two Approaches to Regulation
Prescriptive
“Tell me what to do”
For example, 5% Tier
1 leverage is required
to be well-capitalized
Principles-based
“Tell me the goal”
For example, 1996
Interest Rate Risk
policy statement (FIL-
52-96)
Page 11
Goal ClarificationWHY ARE U.S. BANKS DOING DFAST OR CCAR CREDIT STRESS TESTING?
“Only doing for regulatory compliance” is the wrong answer!
Regulatory
Compliance?
Strategic
Benefit?
Wrong answer for regulators
Wrong answer for competent risk managers
What credit losses should we anticipate in future
economic environments?
Page 12
The “board” is “regulatory compliance”
“beyond” is “strategic benefit”
Focus Beyond the Board
Banks who have focused on the
board have gotten hurt…
Page 13
Traditional DFAST Process Maturity
The Framework
• Governance
• Modeling
• Experience building
The Numbers
• Credit model refinement
• Increased emphasis on PPNR
• Improved data/assumption development and support
Strategic Value
• Performance
• Agility
• War gaming
Phase 1
Phase 2
Phase 3
Develop a long-term plan!
Page 14
Top Five DFAST Issues & Challenges
1. Lack of awareness and leadership (“we don’t know what we don’t know”)
2. Partner / technology selection
3. Internal resourcing/management
4. Documentation
5. Data
Page 15
Strengthening Awareness
• Education
– Conferences/seminars
– In-house executive and Board table top discussions
– Regulatory forums
– Direct interactions with examiners
• Gap analysis
• Peer outreach
Leadership tone drives level of success
Page 16
Partner and Technology Selection
• Readiness assessments (gap analyses)
• Governance framework
• Development partner(s)
– Credit loss modeling
– PPNR (Pre-Provision Net Revenue) modeling
– ALM model integration & reporting
• Documentation
• Validation
Due diligence is critical
Page 17
Major DFAST Modeling Decisions
1. Development Options
2. Aggregator Model Options
3. Peer Group Selection
Page 18
• Develop and retain in-house
expertise
• Pathway for ongoing
development/ongoing strategic
use
• Leverage resources to support
other quantitative initiatives
• In line with ultimate regulatory
desires
1. Development Options (In-House)
• Finding quantitative resources
• Find the “right” athletes
• Management and
communication
• Retention/key-person risk
• Can be limited by an
individual’s personal
experience/expertise
• Regulatory reaction
• Cost
Advantages Challenges and Risks
COMPLETE DEVELOPMENT OF THE DFAST MODELS, MODELING PROCESSES, AND DOCUMENTATION
Page 19
Staffing Challenges
• Underestimated budget
• Thinning talent pool
• Skillset gaps
• Communication
• Training
• Retention
Page 20
• Expertise with broader
horizontal experience
• Shorter implementation
timeframe
• More likely to meet technical
regulatory expectations
• Least impactful on current
resources
• Can come with executive and
Board education
1. Development Options (Outsourced)
• Ongoing dependency
• Vendor management
• Does not meet ideal state from
regulatory viewpoint
• Can entice management into
offloading too much
responsibility
• Selection is challenging
• Cost
Advantages Challenges and Risks
COMPLETE OUTSOURCING OF THE DFAST MODEL DEVELOPMENT AND DOCUMENTATION
Page 21
1. Growing List of “DFAST” Vendors
Credit Modeling
• Accounting and consultancy firms
• Data/analytics firms
• Niche firms
PPNR Modeling
• Accounting and consultancy firms
• Data/analytics firms
• Niche firms
ALM Platforms
• Empyrean
• FIS/Sungard
• Fiserv
• QRM
• ZM Financial Systems
Do your due diligence…
Specifically assigned resources matter!
Page 22
• Develop and retain in-house
expertise that is informed by
experienced external experts
• Knowledge transfer
• Can use experience to develop
resources
• Management/Board education
• More probable regulatory
desires and validation
requirements will be met
1. Development Options (Hybrid)
• Same in-house resourcing
issues
• Same vendor selection and
management issues
• Cost
Advantages Challenges and Risks
DEVELOPING PROCESS IN-HOUSE WITH EXTERNAL CONSULTING ASSISTANCE
Page 23
Major DFAST Modeling Decisions
1. Development Options
2. Aggregator Model Options
3. Peer Group Selection
Page 24
2. Aggregating DFAST Model Results
Call Report
Data (Loss/
NAC Rates)
Macro
Factors
Budget
Loan-level
Data
Econometric
Credit
Models
Other
Econometric
Models
(PPNR, etc.)
ALM Model
ALLL Model
Other
Assumptions
(Capital Plan,
Municipal
Bonds,
Operational
Risks)
Output
Consolidation
External Data
Sources
Reporting
Templates
Capital
Ratios
Data Sources Models Reporting
Page 25
2. ALM Model as Aggregator ModelALM AGGREGATES AND GENERATES RESULTS
Macroeconomic
Forecast
DataWarehouse
Summary-level Balances
DATA
Credit Loss Models
Calculates
Projected Balances,
Non-II, Non-IE, Other
PPNR
Model
Allowance for Loan and Lease
Losses
ALLL
Model
ALM Engine
Transaction-levelBalance Sheet,
Income Statementand Cash Flow
ALM
Page 26
2. Capital Aggregator Model (often Excel)ALM PLAYS A CONTRIBUTING ROLE
Data Warehouse
FRB-Released Variables and Other Economic Variables
MacroeconomicDrivers
DataWarehouse
Summary-levelAccount Balances
and Other Financials
EDW
Statistical Software
Calculate ProjectedLoan Loss Rates
DFAST Loss Models
CalculatesProjected Balances,
Non-II, Non-IE, Other
PPNRModel
Excel File
Federal Reserve Reporting Tool
CapitalModelAllowance for
Loan and Lease Losses
ALLLModel
ALM Engine
Transaction-levelBalance Sheet,
Income Statementand Cash Flow
ALM System
Page 27
Major DFAST Modeling Decisions
1. Development Options
2. Aggregator Model Options
3. Peer Group Selection
Page 28
3. DFAST Models Require Data
•No complete application solutions
•Use of statistical tools (SAS, R)
•Bottom up/down approachesCredit Modeling
•No real application solutions
•Use of statistical tools (SAS, R, Excel)
•Volume, non-interest income/expense modeling
Pre-Provision Net Revenue Modeling
(PPNR)
•Most often performed with ALM model
•Can incorporate inputs from multiple models (Fed scenarios, credit, PPNR, other)
•Interest income/expense (NII)Balance Sheet Modeling
•Existing ALLL model (if sophisticated enough)
•Often developed in Excel
•Growing use of vendor models (CECL)
ALLL/Provision Modeling
•ALM model or commonly developed in Excel
•Brings results together from all models
•Organizes results for regulatory submission and management reporting“Aggregator” Model
Good data is critical!
Page 29
3. Lack of Data Causes Need for Peer Data
• Lack of actual losses
• Poor tracking of loss migrations – Most spend little time on collecting information
on loans expecting to charge off
• Idiosyncratic nature of individual bank losses may tie less to economic factors– Makes building robust statistical models more
challenging
Page 30
3. Ask About Peer Group Selection Process
• Peer group selection process
varies greatly
• Don’t let desired results drive
peer selection process
• “We have a unique business
model” doesn’t fly
Again - do your due diligence
Credit Modeling
•Accounting and consultancy firms
•Data/analytics firms
•Niche firms
PPNR Modeling
•Accounting and consultancy firms
•Data/analytics firms
•Niche firms
ALM Platforms
•Empyrean
•FIS/Sungard
•Fiserv
•QRM
•ZM Financial Systems
Page 31
Internal Resourcing
• Challenging for most
– Availability
– Cost
– Retention
– Communication
• Ideally need good project management skills
• Quantitative experience a plus
Page 32
Common Sources
• Former consultants or advisors
• Former DFAST managers or developers
• Former examiners
• Internal staff with appropriate skills
Page 33
Good Documentation FrameworkTHE KEY TO SUCCESSFUL MODEL MANAGEMENT
Model Theory & Design
Data
Executive
Summary
Assumptions Output
Ongoing
Performance
MonitoringProcedures Governance
Appendices
Page 34
DataEVERY INSTITUTION’S CHALLENGE
• We have not been good data stewards
• Most organizations deficient in data quality,
reliability, and availability
• Inconsistent capture, retention, reconciliation, use
• Organizations large and small affected
Page 35
Data Solutions
• Formalized data development and management initiatives
• Centralized accountability framework
• Leadership (Chief Data Officer/Scientist)
• Operational and technological changes
• Goal: “Single Source of Truth”
Page 36
Validation
• Validation should ideally occur through the development and
implementation process
– Conceptual review
– Full end-to-end of dry run model
– Full end-to-end of submission model
• Not a once and done activity
– Model Risk Management framework will dictate scope and ongoing frequency
– DFAST model/model group most often rated high-risk and subject to annual
validation
• Timing is challenging
– Pre-submission vs. submission model
– Time for remediation
• Governance, documentation, processes and controls is as important as
the math
– Poor documentation will get an instant “needs improvement”
Page 37
A
B
C
D
Introduction
Key areas of focus, lessons learned (fact vs. fiction)
Dodd-Frank Act Stress Test (DFAST)
Regulatory context, goals, issues and challenges, major modeling decisions
Model Risk Management
MRM framework, the “three line of defense”, effective challenge
Concluding Thoughts
Getting strategic value from DFAST and MRM, Q&A
Survival PlanningPREPARING FOR $10 BILLION AND BEYOND
Page 38
MRM
Page 39
Model Risk ManagementA CRITICAL DFAST COMPANION
• Organizations preparing to cross $10B threshold also need to
develop robust MRM framework
• Not something you can implement afterwards
• Can take a few years to effectively develop
• Will continue to evolve
Page 40
MRM Framework
• Governance– MRM Policy
– Processes and controls
– Committee composition and structure
– “Three lines of defense” roles and responsibilities
• Model inventory management
• Model risk rating methodology
• Model documentation standards
• Model validation standards– Documentation
– Sourcing
– Review frequency
– Issue tracking
– Annual touch
• Model lifecycle management
Page 41
Roles and ResponsibilitiesTHE THREE LINES OF DEFENSE
3rd Line of Defense
2nd Line of Defense
1st Line of Defense
Model Owner
Management
Controls / Internal
Control Measures
Internal Audit
Model Risk Mgmt
Se
nio
r Ma
na
ge
me
nt
Bo
ard
/ Au
dit C
om
mitte
e
External Audit
Regulator
Page 42
Validation as a Process
• Principles-based approach contributes to mean reversion
– Best practices change over time
– Regulators inform those falling behind
– Ratings don’t matter as much as might think
• Focus upon constant and continuous improvement
– Never arrive at “perfect” models
– Validation feedback is part of a wider business conversation
Can improve Can deteriorate
Page 43
Model Volume & Complexity
• Number of “Models" by Institution Size
1,000+
25-100
5-25
CCAR (>$50B)
DFAST ($10B-$50B)
Community Banks (<$10B)
& Credit Unions
Page 44
10-100+
1-5
0-1
CCAR (>$50B)
DFAST ($10B-$50B)
Community Banks and
Credit Unions (<$10B)
Staffing Needs
Page 45
Effective Challenge
GUIDANCE ON MODEL RISK MANAGEMENT (OCC 2011-12/ FED SR 11-7)
Effective Challenge:
“critical analysis by objective, informed parties that can identify model limitations and produce appropriate changes. Effective challenge depends on a combination of incentives, competence, and influence.”
• Incentives to provide effective challenge to models are stronger when: – Greater separation of that challenge from the model development process
– Well-designed compensation practices
– Corporate culture
• Competence is a key to effectiveness since technical knowledge and modeling skills are necessary to conduct appropriate analysis and critique
• Challenge may fail to be effective without the influence to ensure that actions are taken to address model issues
– Explicit authority
– Stature within the organization
– Commitment and support from higher levels of management
Page 46
A
B
C
D
Introduction
Key areas of focus, lessons learned (fact vs. fiction)
Dodd-Frank Act Stress Test (DFAST)
Regulatory context, goals, issues and challenges, major modeling decisions
Model Risk Management
MRM framework, the “three line of defense”, effective challenge
Concluding Thoughts
Getting strategic value from DFAST and MRM, Q&A
Survival PlanningPREPARING FOR $10 BILLION AND BEYOND
Page 47
Recipe for Success
• Education/communication
• Short- and long-term planning
• Reach beyond compliance – envision
strategic value
• Address data as early as you can
• Choose your partners wisely
Page 48
Due Diligence TipsPARTNER/VENDOR SELECTION
• Seek specialized experience – DFAST – each aspect of modeling
– MRM – right-sized model management process
– CFPB
• Horizontal perspective with DFAST – DFAST experience over CCAR
• Seek A-teams– Solid due diligence on the specific resources (not just the firm)
Page 49
Getting Strategic ValueHOW INSTITUTIONS ARE USING STRATEGICALLY
• Merger and acquisition modeling
• Business line growth/diversification evaluation
• More holistic capital planning / stress testing
– Integrated risk assessments (credit, IRR, liquidity, ops,
capital/earnings)
• Expanded vulnerability risk assessments
• Better informed risk monitoring
• Risk Appetite Statements
Page 50
Questions?
Page 51
Preparing for $10 Billion and Beyond
Thank
You!
