Upload
douglas-gray-cissp-ciso
View
193
Download
0
Embed Size (px)
Citation preview
© 2015 Carnegie Mellon University
Software Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213
Distribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity GovernanceDouglas Gray
2Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Copyright 2015 Carnegie Mellon University
This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center.
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense.
NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
[Distribution Statement A] This material has been approved for public release and unlimited distribution. Please see Copyright notice for non-US Government use and distribution.
This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].
CERT® and OCTAVE® are registered marks of Carnegie Mellon University.
DM-0003094
3Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Introduction
4Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
• The Software Engineering Institute (SEI) is a U.S.-owned not-for-profit federally funded research and development center (FFRDC) operated by Carnegie Mellon University to focus on software and cybersecurity.
• The CERT Division of the SEI is a trusted provider of operationally relevant cybersecurity research and innovative and timely solutions to our nation's cybersecurity challenges.
• The CERT Division developed and maintains the CERT Resilience Management Model (CERT-RMM) and OCTAVE Allegro Methodology.
Who We AreIntroduction
We work with the following organizations:• Carnegie Mellon University• Discover Financial• Highlands Union Bank• Lockheed Martin Corporation• Marshall & Ilsley Corporation• PNC Corporation• Pacific Gas and Electric• University of Pittsburgh
Medical Center• U.S. Dept. of Defense• U.S. Dept. of Energy• U.S. Dept. of Homeland
Security• U.S. Dept. of Health & Human
Services• U.S. Environmental Protection
Agency• U.S. National Security Agency• U.S. Postal Inspection Service• USBank
5Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
To discuss a process to integrate data analytics into operational cybersecurity governance decision making and execution in a way that• frames the problem quickly and accurately and that enables a
fast, effective Observe, Orient, Decide, Act Loop• facilitates better data collection and synthesis, quantitative and
qualitative analysis, and visualization• enables practical and repeatable analytical battle drills and
TTPs for leaders and enablers at all echelons
PurposeIntroduction
6Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
W. Edwards Deming’s thoughts What it means to us
“If you do not know how to ask the right question, you discover nothing.” We must have a reason to analyze data
“If you don't understand how to run an efficient operation, new machinery will just give you new problems of operation
and maintenance. The sure way to increase productivity is to better administrate man
and machine.”
We can’t “tool” our way out of cybersecurity challenges
“People with targets and jobs dependent upon meeting them will probably meet the targets - even if they have to destroy the
enterprise to do it.”Compliance is the beginning, not the end
“Whenever there is fear, you will get wrong figures.”
The use of data analytics must be productive in the aggregate, punitive as
the exception
Improving People and ProcessIntroduction
Technology is useless without effective processes and trained people
7Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Governance vs. OperationsIntroduction
Operations GovernanceScope Individual
networks, systems, users, organizations
Multiple networks, systems, user bases, organizations
Timescale Immediate to 6 months
3 to 36 months*
Level of Abstraction
Transactional Trends, aggregations
Management Impact
Direct interaction Context setting
*Although maximum technology-related decision making is limited to approximately three years due to rate of technological change, military organizations must program
their expected budget needs five years in advance.
8Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Governance vs. OperationsIntroduction
8
OperationsWeather – “It will snow.”
Tactical Cyber – “CVE 2015-xx-xxxx is prevalent and is being
compromised.”
GovernanceClimate – “Drought in the
southwest limits irrigation.”Strategic/Operational Cyber – “FedRAMP usage improves
asset management.”
9Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Why Focus on GovernanceIntroduction
Know Prevent Detect
Respond
Recover
Reconnaissance Weaponization Delivery Exploitation Installation Command and Control
Actions on the Objective
Threat Actor Actions1
Friendly Actions2
Harden People, Information, Information, Technology,
FacilitiesCreate Faster, More Accurate
TTPs, Battle Drills
Source:1Lockheed Martin Kill Chain2NIST Cybersecurity Framework
Effective preparation creates the context for effective response
10Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Leveraging Situational Awareness to Enable Cyber Mission Command
Introduction
Observe
Orient
Decide
Act
mutual
trust
shared
understanding
clear
leadership
intent
disciplined
initiative
mission-
oriented
directives
prudent risk
management
11Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Observe
12Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Facets of Cybersecurity GovernanceObserve
13Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data FusionObserve
Data Fusion Activities
Automated vulnerability sensor information• H
ardware & Software
• Behavioral Observables (Insider Threat)
Threat Information• T
hreat Actor Analysis
• Prevailing Attack Patterns
Management Information• B
udget Information
• Demographic Information
• Legal & Administrative Investigation Statuses
• Mission Impact Analysis
Qualitative Assessment• I
nspections/Assessments
• Professional Sentiments Analysis
Orient
Unstructured DataMachine Learning
Text AnalysisTrend Analysis
Correlation
14Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Orient
15Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Decision Science vs. DashboardOrient
15
Dashboard“It’s going to snow.”
Decision Science“It’s going to snow.
Wear galoshes, gloves,scarf, winter coat.”
16Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Level 1 Perception of the elements
in the environment
Level 2 Comprehensi
on of the current
situation
Level 3 Projection of future status
Developing Situational AwarenessOrient
Source: Endsley, M. & Jones, D. Designing for Situation Awareness: An Approach to User-Centered Design (2nd ed.). CRC Press. 2012.
17Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Decomposing the Situation to Develop Situational Awareness
Orient
Situational Awareness
Voice of the Environment
Socio-Political
Legal and Policy
Technological
Business
Physical
Voice of the Organization
Voice of the Mission
Voice of the Service
Strategic Objectives and
Supporting Services
Organizational Culture
Organizational Assets
External Dependencies
Voice of the Threat Actor
Describe Threat Actor
Develop Threat Actor Use Cases
IndicesProbabilistic Models
Game TheoryExpert Opinion
18Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Build and Update Targeted MetricsOrient
RequirementsIdentify requirements from mandates, doctrine, strategy
Group requirements into categories
GoalsDevelop one or more goals for each category
QuestionDevelop one or more questions that, if answered, help determine if the goal is met
IndicatorsIdentify the information requirements to answer the question
MetricsIdentify the metrics that will measure the indicator to answer the question
Use new metrics to mature current metrics
What do we want to know? Why do we want to know it? What will we do once we know it? Build and add to a metrics library.
19Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Authoritative vs. Non-Authoritative DataOrient
Authoritative Data•Based on their ability to stand alone as a source for one or more facets of cybersecurity governance•Population•Comprehensiveness•Poor data quality does not make a source not authoritative; it means the quality problems should be fixed
Non-Authoritative Data•Source does not cover enough of the population or not comprehensive enough to be authoritative•Can speak to confidence level of an authoritative data source•Examples: reviews, assessments, inspections, surveys.
20Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Using Behavioral Models to Target Stakeholder Need
Orient
Executives:• Elected leaders, appointees,
GOs, FOs, SESs• Target data with eye toward
organizational mission and constituents
Middle Management:• Staff officers, analysts• Target data with eye toward
routines, procedures
information
Source: Allison, G. T., & Zelikow, P. (1999). Essence of Decision: Explaining the Cuban Missile Crisis (2nd ed.) (Kindle Edition). New York: Longman.
Results of data analysis must be impactful to the recipient.Frame products according to organizational behavioral models.
21Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Decide
22Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
• Determine confidence level in assessed data• Low – analyze through subsequent OODA loop• Medium to High – develop action plan to effect change
• Identify and prioritize governance-level risks; identify metric-supported thresholds of acceptability and unacceptability
• Support solutions. Go beyond “name and shame.” Use metrics to identify key trends and corrective governance-level actions
• Tie metrics to a resulting set of possible risk management outcomes• Identify enablers such as SMEs, funding, contract vehicles • Identify organizations that exceed expectations in certain areas and
their lessons learned• Identify what expected changes in metric values should be and how to
avoid bias/gaming• Prioritize and identify metric thresholds where costs will exceed benefits
Key Planning and Decision-Making Factors Decide
23Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Act
24Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Leveraging Enablers to Achieve Desired Effects
Act
25Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
• Leverage enablers at the proper organizational level; avoid the “3,000-mile screwdriver”
• Governance sets the direction through governance facets. Operations executes through disciplined project management
• Avoid numerous, rapid changes that cause enterprise turbulence
• Tie actions to expected outcomes and expected timeframes; socialize and communicate expectations
• Set decision points to check progress against expectations• Build knowledge base to make for faster and more effective
OODA loop
Success at the Point of ExecutionAct
26Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Implementation
27Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Identify success stories• Lessons Learned• Tie to data analysis
Identify cautionary tales• Lessons Learned• Tie to data analysis
Track event-driven events• Identify trends that respond to events• Resourcing, technology, incidents
Building a Cybersecurity Knowledge BaseImplementation
27
28Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
• Inventory on-hand data• Inventory metrics• Develop data fusion capabilitiesObserve• Refine metrics based on constraints and mandates• Define stakeholders based on behavioral models• Develop quantitative and qualitative analysis engines• Develop visualization capabilities
Orient• Inventory enablers and their capabilities• Identify desired outcomes for metrics (i.e., thresholds)• Develop decision support TTPs• Develop decision-support systems
Decide• Develop knowledge base• Simulate and practice new decision-making TTPs• Develop and refine process control mechanisms• Develop, refine and leverage communications channels
Act
How to ImplementImplementation
29Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Outcomes of Data Driven GovernanceImplementation
• Faster, more accurate decision making
• Better use of resources• Better enterprise cohesion
and synchronization• Data-driven outcomes• Improved information
sharing• Adaptable to change
Observe
Orient
Decide
Act
30Data Driven Cybersecurity GovernanceDec. 18 2015© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
© 2015 Carnegie Mellon UniversityDistribution Statement A: Approved for Public Release; Distribution is Unlimited
Data-Driven Cybersecurity Governance
Questions