Upload
cydney-davis
View
989
Download
2
Embed Size (px)
DESCRIPTION
For those who are truly interested in developing a Security Metrics program with VALUE
Citation preview
SECURITY METRICS
A presentation developed by Cydney Davis, Senior Technical Writer
What are Metrics?
A method which facilitates decision-making and improved performance and accountability through collection, analysis and reporting of performance-related data.
Information Security metrics must be:• based on Information Security performance goals
and objectives• useful for reduction and management of risks• readily obtainable and replicable• useful for tracking performance and directing
resources• able to yield quantifiable information
2
It is critical that we use metrics that are relevant to our organization and to the mission we are measuring.
But first, we have to determine:• Where we are (Baseline)• Where we are going (End Goals)• Who/what relies on us?
(Users/Management)• What do they need/expect?
(Reports/Assurance)• What are we trying to prove?• What are we trying to solve?• What are we trying to improve?
What is our Mission/Goal?3
How can we use Metrics?
Communicate PerformanceDrive Performance ImprovementMeasure Effectiveness of Security
ControlsHelp Diagnose ProblemsProvide Effective Decision-making
SupportIncrease AccountabilityGuide Resource AllocationDemonstrate the state of complianceFacilitate Benchmark Comparisons
4
Metrics can help determine:
• the number of resources it takes to accomplish security goals
• justifiability for financing new security measures• If the company is getting its money’s worth• If the company is managing risk appropriately*• what Information Security needs to do to
improve Security ˉ administration/processes/procedures/
policies/personnel/enhancements/technology/etc.)
• where we are with comparisons to peers regarding to standards, best practices, execution and results of security measures
*The residual risk that a company is willing to take based on; business needs, budget limits, industry regulations/requirements and other criteria.
5
Building the Security Metrics Program
6
“The heart of it is that if a business process cannot be measured in one way or another, we likely ought to cast it off as wasted effort.”
Comment from a CEO to an anonymous Information Security Professional
Translation: Why do it if we can’t prove/justify its value?
(time, money, effort, results and actions)
7
Executive Focus
Good Metrics Guidelines
•Consistently Measured •apples to apples/same time same place
•Cheap to Produce (Time-wise)•Yield Quantifiable Information•Contextually Specific – who •Expressed using at least 2 units of measure or data points
8
Identify incident trends important to key senior managers, stakeholders and to the InfoSec Mission from a management perspective.*
Provide consistent information that adds value and is actionable by:
• Tracking changes on a consistent basis.
• Focusing on what's important in our business
• Developing a few value indicators that we can track with a high degree of reliability
• Doing some service benchmarking with our peers.
Metrics Program Success Criteria
9
*This is the first and most important decision
Basic Information Security Measures
Anti-malware Firewalls Asset Management
Intrusion Detectionand Prevention
Anti-SPAM Patch Management
Vulnerability Management
Unified Threat Management
Application Security Scanners
Databases Website Statistics
Network Access Control
System Integrity Checking
Operating Systems
Data Leakage Protection
Configuration Hardening
Secure Web Gateways
Web Application Firewalls
Mobile Data Protection
Media Sanitation
Storage Encryption
10
Formula for Deriving True Meaning
Determine how the information will be analyzed, interpreted and used!
•C-Level•Board of Directors•Marketing Releases•Industry Report•General Staff
•Financial•Governance•Legal•Regulatory•Directive
DATADATADATADATADATA
11
WHY we need to measure it
WHAT we need to measure
WHO we aremeasuring it for
“Good metrics facilitate discussion, insight and analysis...”
12
Metrics Program - Components
Program Component
Define the metrics program goal(s) and objectives
Decide which metrics to generate
Develop strategies for generating the metrics
Establish benchmarks and targets
Determine how the metrics will be reported
Create an action plan and act on it
Establish a formal program review/refinement cycle
13
Obtain management input, agreement and support for the
implementation of a strong metrics program.
Review our organization’s mission statements, policies, plans, procedures, goals
and objectives, and assess them against legislative and regulatory requirements,
as well as against effectiveness goals.
Describe how we will achieve company and department goals
List milestones, dates and quantifiable objectives against which to map
progress.
Select appropriate, quantifiable effectiveness metrics to indicate
baseline, interim and final success.
Gather the metrics.
Analyze and present the results to management and key stakeholders.
Recommend that management make decisions based on the metrics, and
plan the execution of these decisions. * Metrics are often referred to as
“decision support.”
Evaluate the outcome of decisions against goals. This should be done from a
perspective of effectiveness.
High Level Process Steps
*The real value of a metrics program
14
Project Plan Overview15
Metrics Versus Numbers16
Good metrics are those that are SMART;
•Specific•Measurable•Attainable•Repeatable,• Time-dependent
Truly useful metrics indicate the degree to which security goals are being met – and they drive actions that need to be taken to improve our overall security goals.
17
Exhibit A - This set of numbers can give us a sense of the overall health of anti-virus defenses and can show trends over time; but the information is not actionable in any way and will not serve as a meaningful diagnostic tool. SO WHAT??? = False sense of security without more knowledge
Metrics? Or Just Numbers?18
Exhibit B displays the same measurements as Exhibit A. By drilling down into the data we can begin to understand which locations are struggling with this activity. This in turn will help us choose where to focus in order to improve the performance of our organization. This kind of actionable intelligence is valuable and it can really drive performance improvement and provide information that is actionable to a productive end.
Example Metrics showing RELEVANCE
Good Metrics = Numbers with Relevance
19
City A
City B
City C
99.4 %
94.7 %
89.8 %
Percentage of computers with current anti-virus definitions
75 % 80 % 85 % 90 % 95 % 100 %70 %65 %60 %55 %50 %
20
CITY A
City B
City C
99.4 %
94.7 %
89.8 %
Percentage of computers with current anti-virus definitions
75 % 80 % 85 % 90 % 95 % 100 %70 %65 %60 %55 %50 %
Example Question: Why is one location so much farther behind in implementation? Possible Reasons: Understaffed
Limited Bandwidth More staff traveling that previous years
Possible Actions: Hire additional staff Share resources if the implementation MUST be done by xxx date Set different schedules for each location for future projects
Good Metrics = Actionable
Presenting and Interpreting Data Reports
_______% improved
_______% improved
from _______ and that means _________ .
What we need is ______ based on
requirements for __________ . Going
forward we should consider doing
___________ .
Visually Appealing Visually AppealingInterpreted and Actionable
21
Measuring for value not numbers
Defining, refining and Interpreting data/results for the intended audienceExamples to work with
22
Measurement of how well we are protecting our enterprise against the most basic information security threats.
Just Numbers: ________ %
What would an additional relevant value be that we can use to have SMART data?
Metrics: ________ % Increase since (prior month/inception/year over year/etc.) Device Type Location Length of time it took to detect
EXAMPLE Metric : Baseline Defenses Coverage (Antivirus, Antispyware, Firewall, etc)
23
Legitimate e-mail traffic analysis is a family of metrics including incoming and outgoing traffic volume, incoming and outgoing traffic size, and traffic flow between our company and others.
By monitoring legitimate e-mail flow over time, we can learn where to set alarm points.
Numbers: Compare the amount of good and junk e-mail that we are receiving ____ percent good____ percent junk
What would an additional relevant value be that we can use to have SMART data?
Metrics____ percent good____ percent junk Quarterly/Annually/Since inception/Current MonthSince adding the _________ criteria Received from _________ types/organizationsSent During ____________ (AM/PM – Holidays , etc.)Junk Detected Quicker _______ (first time/second time)
EXAMPLE Metric : Legitimate E-Mail Traffic Analysis
24
By presenting information in a sufficiently granular way we can inject business relevance into the exhibits. Producing a benchmark is also a powerful approach to performance improvement.
Conclusion
Frequently this level of visibility will spark a competitive fire in those being measured. Professional pride will drive most people to make sure they are found among the high performers on your report.
25
City A
City B
City C
99.4 %
94.7 %
89.8 %
Percentage of computers with current anti-virus definitions
75 % 80 % 85 % 90 % 95 % 100 %70 %65 %60 %55 %50 %