Embed Size (px)
Citation preview
No Victims: How to Measure & Communicate Risk
Jared Pfost
@JaredPfost
Hello
• InfoSec 17 years • Consulting• Practitioner
– Microsoft– Washington Mutual
• Software Development– Microsoft– Startups– Third Defense
• Process Nut
2
Human Motivation
3
Straight Forward Tasks
Ambiguous Tasks
Autonomy Mastery Purpose
Just My Opinion
4
Secu
rity
$pen
ding
2012+20082005200290’s
Risk
Avoid unacceptable risks in the most efficient manner?
Just good enough to meet a standard of “due care?” Be compliant 2 months per year?
Q: What does success look like?
Infosec Evolution
6
Proactive Prevention
Assume Breach
Accept Risk
Nothing Miraculous Here!
Seeking Acceptance
8
Risk Assessment Process
Bus
ine
ss
Ow
ner
IT S
erv
ice
O
wne
rB
usin
ess
O
wne
rIn
form
atio
n
Sec
urity
Scope Assessment
Identify RisksPrioritize Initial
Risks (collect evidence)
Identify Mitigation Candidates
Finalize Risks & Mitigation Candidates
Finalize Treatment
Options
Facilitate Risk & Mitigation Prioritization
Appropriate Owner
Acceptance
Engage Exec. Mngt.
Schedule followup if needed
Yes
No
Ref:
IT Risk Management Process Flow
Prioritize Risk
Risk Assessment Process
Bus
ine
ss
Ow
ner
IT S
erv
ice
O
wne
rB
usin
ess
O
wne
rIn
form
atio
n
Sec
urity
Scope Assessment
Identify RisksPrioritize Initial
Risks (collect evidence)
Identify Mitigation Candidates
Finalize Risks & Mitigation Candidates
Finalize Treatment
Options
Facilitate Risk & Mitigation Prioritization
Appropriate Owner
Acceptance
Engage Exec. Mngt.
Schedule followup if needed
Yes
No
Ref:
IT Risk Management Process Flow
Mitigation Cost-Benefit
Risk Assessment Process
Bus
ine
ss
Ow
ner
IT S
erv
ice
O
wne
rB
usin
ess
O
wne
rIn
form
atio
n
Sec
urity
Scope Assessment
Identify RisksPrioritize Initial
Risks (collect evidence)
Identify Mitigation Candidates
Finalize Risks & Mitigation Candidates
Finalize Treatment
Options
Facilitate Risk & Mitigation Prioritization
Appropriate Owner
Acceptance
Engage Exec. Mngt.
Schedule followup if needed
Yes
No
Ref:
IT Risk Management Process Flow
Manage Risk Register
Treatment Decisions Control Performance
Scope Measurements
Define Target Values
Optimize Targets
Risk Prioritization: Kiss The Ring Of Process
9
10
Task Program Manager Dev Test Security
SMEProduct
Manager Exec
Define vuln/bug exploitability R AR R R I -
Define temporal, agents, attack frequency, etc. R R R AR - -
Define vuln/bug impact R R C R AR -Finalize bug priority AR R R R C -Determine fix options and recommendation. R AR R R I -
Define fix complexity & time R AR R C I -
Define test time & regression risk R R AR C I -
Finalize fix priority (sometimes called severity)
AR R R C R -
Escalated bug and fix priority decision C C I C C AR
The Exercise Is Important
11
Risk
Impact
Direct
Regulatory
Recovery
Revenue
Indirect
Goodwill
Scrutiny
Competitive
Corrective
Capability
Frequency
Vuln. Attributes
Complexity
Vector
Access
Availability
Occurrence
Control Effectiveness
Roles
Awareness
Tools
Policy & Process
Detect/DeterEvidence
Evidence Drives Treatment
• Don’t prioritize risk without it...
12
Opinion
Peer Incidents
Intelligence Services
Metrics
Attack & Pen Assessments
Internal Incidents
Compliance Requirements (floor)
It finds you
You find it
Risk Narrative
• Grabber– Agent– Action: CIA– Asset– Impact
• Details– Vulns– Controls– Occurrence
• Evidence
13
Criminals copying payment card data through Internet facing
web app. We have 50K records, business owner and IT expect X
direct and Y indirect costs.
Development practices failed to validate malicious input
leading to...
We found 3 vulns per assessment. Peers lost 100K
records last year.
14
Use Culture to Select Model
http://beechplane.wordpress.com/2011/08/17/the-simple-power-of-openpert-ale-2-0/
User Defined Ordinal Values Expert Opinion DistributionsFor ARO and SLE
• Evidence In -> Treatment Decision Out
And/Or
Minimize Ordinal Flaws
• Non-linear scales– Reserve Highest Values to reference risk details
• Edge Cases– Document edges or create a new risk
• Understand previous Treatment Decisions against “Color Bands”
• Combine quantitative and qualitative values– Include risk narrative elements– Align to other department models e.g. ERM
15
Narrative Scale Definition
16
Value Direct Costs Indirect Costs Examples
10...
Revenue: Missed Targets of $xxx,xxxRegulatory: Fines & Audits of...
Competitive: Differentiator of...Goodwill: Customer departure of...
Focus: Mitigate Risk e.g. material loss estimated above $xx,xxx,xxx.
6 Revenue: Limited to department...Regs: Increased scrutiny...
Goodwill: Customer churn of 5-10%...
Focus : Owner Judgement e.g. business considerations.
Value Description ARO Guide Examples
10...
Strong evidence of imminent realization, precedent exists, reliable intelligence.
> 1 annually, see risk details for estimates
Known control weaknesses of..., confirmed agent...
6 Difficult to exploit without internal...
Realized once in 4 years...
Private system, agent unconfirm
Impact
Frequency
Single Event Risks
17
Criminals copying payment card data
through Internet facing web app. We have 50K
records, business owner and IT expect X direct and Y indirect costs.
Evidence...
Accept Evaluate Act
2
3
4
5
6
7
8
9
10
2 3 4 5 6 7 8 9 10
ECom: App. Vulns
ECom. Device Vulns
Incident Response
Access Certification
Segregation of Duties
DDoS
Vendor Security Controls
Details...
Tell Me A Story
• Vulnerability Attributes
18
Evidence: We found 3 injection vulns per assessment. Vulns are easily identified and exploitable from the Internet. Only basic knowledge and a motivated Agent are needed. Peer Company was breached last month by a Criminal Group.
Tell Me A Story (cont.)
• Control Effectiveness
19
Evidence: Development practices failed to validate malicious input. Training is mandatory but ineffective. Quarterly Assessments occur but site updates occur
monthly.
Tell Me A Story (cont.)
• Impact
20
Evidence: Last year’s breach estimated at $xx,xxx direct and $xxx,xxx indirect costs. Peer Companies breach estimated at $xxx,xxx. However minimal customer
departures.
Multiple Hop Risks
• Advanced Adversary copying intellectual property through “Aurora” style attack
21
Social Engineering
•Targeted Phishing
Device Root
•Malicious Payload
Escalate Access
Test
Test
Test
Test
Multi-Hop
22
• Keep it simple• Add a “roll-up” risk
to represent chain of events
Accept Evaluate Act
2
3
4
5
6
7
8
9
10
2 3 4 5 6 7 8 9 10
Adv. Adversary: IP Theft
Social Engineering: Employee
Device Compromise
Privilege Escalation
Data Exfiltration
AAA
Don’t Forget The Agents
23
Controls: Spending & Process Maturity
Chaotic ActorsFor LOLz
CriminalsFor Cash
AdvancedAdversaryFor IP
Basic SDL
Vuln Scans
Fraud Detection
Advanced SDL
Full Packet Capture
Rock Star Response & Forensics
DoS
• Prioritize by “Business Value” Construct– Risk(s) Priority– Team Capability– Business Support– Political Reality– Cost
24
Efficiency Gain Save
$110K
Spend Or Accept
25
Business Drivers
Service Maturity
Regulatory Requirements
Mandatory
Discretionary
Discretionary
“Legally Defensible” Security
• Risk-Based Decisions, Budgets
• Internal Consulting• Process Improvement
Spending: No Room For Victims
Risk Register - Skeletons
• Authoritative Source• Defined Process• Treatment Status
– Mitigating– Mitigated– Accepted– “Watching”
26Active Mitigated Other Watching Accepted
2
3
4
5
6
7
8
9
10
2 3 4 5 6 7 8 9 10
Unencrypted Tapes
Proliferation of PII
Unencrypted PII in Email
Application Vulns
Device Patching
Rogue Devices
Network Segmentation
Rogue Wireless Access
DDoS
Log Retention
Segregation of Duties
Access CertificationEmployee Terminations
Break Glass Access
Vendor Security Controls
Incident Response
SaaS Security Transparency
SaaS Storage
Paper Statements
Measure EvolutionDon’t Measure
Measure what’s easy
Set Targets
Justify More
Optimize Cost vs. Target
Real Metrics Have Outcomes
• Metrics have Winners|Losers– Measure actual performance against target– Benefits
• Drives “acceptable risk” conversation with Management• Simplifies reporting e.g. are we above|below?
Start With “Easy”– Incidents - # of High, Moderate, Annoying
– Application- # of Post-production security bugs
– Scanned Vulnerabilities- # Patch & config vulns not mitigated per policy timeframe
- e.g. Critical, Ecommerce Vulns mitigated within 30 days
Severity 1
Severity 2
Severity 3
Severity 4
Critical
0
50
100
150
200
250
300
350
WorkstationECommerce
Servers
Severity 1
Severity 2
Severity 3
Severity 4
Critical
0
50
100
150
200
250
300
350
400
450
> 90 90 60 30 30 60 90 > 90
31
Expand Measurement
– Access Management- % Employee termination within policy- % Role/Access verification
– Network- % critical systems monitored- Moving to % of full packet capture
– Vendors- % assessed per policy- # overdue findings
– Employee- # of duplicate incidents
– Change Management- # emergency or unplanned changes- % of changes with a regression
Every Metric Must Have A
Target
Optimize Cost - Target• Is target optimal?
67
75
84
92
100
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb
Proposed Target
Current Target
Integrate Metrics Into Root Cause Analysis
Find Leading Indicators
Zen
34
Accept
•Process•Evidence•Communication•Measurement
QUESTIONS?
35
[email protected] thirddefense.wordpress.com@JaredPfost
Feedback Survey!https://www.surveymonkey.com/sourceboston12
APPENDIX
36
Cost - Benefit - Accountability
Rate Hrs Per Test/Deploy
# Personnel Cost Per Server Update
$100/HR 40 10 $40,000
Evidence: Incidents, response performance, attack attempts
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10
DoS PostMalware Post
Worm Post
Or
http://code.google.com/p/openpert/
Current Target
Proposed Target
Embrace Maturity Deltas
• Target Maturity used in Spending Decisions
• Hire a Benchmarking Service
38
IT Risk Assessment Deliverables
39
Type Output Purpose Duration
Ad Hoc Statement Email, Meeting Clarify Policy 1-2 Hours
Position Paper 1-2 Pages Official Team Statement 1 Week
Project Support Varies Identify Security Requirements Varies
Detailed Assessment Multiple Active Evidence
Collection, Testing 2-3 Weeks
Portfolio Summary & Presentation Prioritize Budget Quarterly -
Annually
Task Security Control Owner Business Owner
Define Metric A,R R C
Define Target R R A,R
Report Metric A,R R I
Review Target A,R R R
R – ResponsibleA – AccountableC – ContributeI - Informed(There can be only one “A”)
RACI in action
Cheap & Easy
Spend to Comply
Fix Gaps Now!
Ok, how much do we really need...?
Are You Ready For The Answer?
41
Motivating Event
