Chaitanya Kunthe - Security Metrics - Interop Mumbai 2009

Information Security

Metrics

MUMBAI � PUNE � AHMEDABAD � BANGALORE � HYDERABAD � CHENNAI � LONDON � UAE � USA Consulting - Products Solutions - R&D – Education

© 2009 MIEL eSecurity Pvt Ltd

Confidential

Chaitanya KuntheHead – Consultancy [email protected]

Unfortunately, no one can be told what the Metrics is…You have to see it for yourself…

2© 2009 MIEL eSecurity Pvt Ltd

Confidential

MUMBAI � PUNE � AHMEDABAD � CHENNAI � LONDON � UAE Consulting - Products Solutions - R&D - Education

You take the red pill,You stay in wonderland andI show you how deep the Rabbit hole goes…

You take the blue pill,The story ends, you wake upIn your bed and believeWhatever you want to believe

Remember, all I am offering is the truth, nothing more…-Morpheus, (The Matrix)

Objectives

• Understanding Security Metrics

• What

• Why


Confidential


• A few common views on metrics

• Practical Implementation

Understanding Security MetricsWhat IS “The Metrics”? – Definitions

• “Information Security measures are used to facilitate decision making and improve performance and accountability through the collection, analysis and reporting of relevant performance related data”– NIST 800 – 55 Rev 1


Confidential


• Measurement - the action or set of actions that make it possible to obtain the value of a measurement for the attribute of an entity using a form of measurement – ISO/IEC 3rd WD 27004

Understanding Security MetricsWhat IS “The Metrics”? – Simplification

• Consistently gathered, quantifiable data, analyzed to provide an organization a view of the efficiency and effectiveness of the information security practices implemented within.

A defined method to collect data at pre-defined

Is the process being followed correctly and


Confidential


data at pre-defined intervals

Identify what data to collectand what it will signify

What does this mean to the organization?

followed correctly and regularly?Is the process effective in meeting information security?

Understanding Security MetricsWhy do we need Metrics?

• “It is easy to lie with statistics, but it is easier to lie without them”– Fred Mosteller

• Increase accountability• Improve Information Security Effectiveness• Demonstrate compliance


Confidential


• Demonstrate compliance• Provide quantifiable inputs for resource allocation decisions

A few common views on MetricsThe list

• NIST SP 800-55• ISO 27004• SSE CMM• CoBIT 4.1


Confidential


A few common views on MetricsNIST 800 – 55 Rev. 1

Results-Oriented Measures Analysis


Confidential


Strong Upper Management Support

Practical Information Security Policies and

Procedures

Quantifiable Performance Measures


Goals and Objectives

IS Policies and procedures

IS program implementation

StakeholdersInterests


Confidential


Level of implementation

Program Results

Business/ Mission impact


• Roles and Responsibilities– Agency Head– Chief Information Officer– Senior Agency Information Security Officer


Confidential


– Senior Agency Information Security Officer– Program Manager/ Information System Owner– Information System Security Officer– Other Related Roles

A few common views on MetricsNIST 800 – 55 (Types of Measures)

Implementation Measures

Effectiveness/

Percentage of servers patched by the latest application patches

Percentage of security management team members

who attended the last meetings

Percentage of servers where the


Confidential


Effectiveness/ Efficiency

Impact Measures

Percentage of servers where the patch was not applied after a

week of release

Number of security decisions approved by the management

Reduction in laptop theft due to implementation of biometric

access control

Metric/ Measure Implementation/ Efficiency/Effectiveness/ Impact

“Percentage of end user systems where anti-virus is deployed”

Implementation

“Percentage of systems where anti-virus is up-to-date”

ImplementationEfficiency

“Number of virus caught by the anti-virus Efficiency? Effectiveness?

A few common views on MetricsNIST 800 – 55 (How deep does the rabbit hole go?)


Confidential


“Number of virus caught by the anti-virus gateway”

Efficiency? Effectiveness?

“Ratio of number of virus attacks to number of virus caught”

Effectiveness

“Ratio of number of virus attacks to number of virus attacks in industry segment”

Effectiveness

“Rupee value of savings to the virus attacks prevented”

Can we even measure this?

A few common views on MetricsISO/IEC 3rd WD 27004

• “Do not try to bend the spoon. That is impossible. Instead, only try to realize the truth. There is no

Develop Metrics and prepare for

data collection

Collect and Implement


Confidential


truth. There is no spoon”

Collect and analyze data

Identify improvements

Implement improvements

A few common views on MetricsISO/IEC 3rd WD 27004

• Model Definition– From Information needs to the entity attribute

to be measured

• Identify the method


Confidential


• Identify the method– Subjective and Objective methods

• Identify the frequency– Intervals – daily, weekly, monthly, quarterly…

A few common views on MetricsCoBIT and SSE CMM

• Both use measures derived from the SEI –CMM model


Confidential


Try and relax…This will feel a little weird

I am trying to free your mind, Neo, but I can only show you the door. You are the one that has to walk through it.

Welcome to the real world, Neo…


Confidential


Practical Implementation of MetricsWhere do we begin?

Question Answer

Does my organization link its business goals to IT goals and Information Security goals?

Ask yourself these basic questions before entering “The Metrics”...


Confidential


Information Security goals?

Has my organization developed and implemented an information security programme?

Has the information security programme been in existence for at least two years?

Practical Implementation of MetricsDevelop the model for measurement

• Identify key policy statements that you would like to have measures for

• “All users will be trained on information security do’s and don'ts every quarter”


Confidential


• Identify what the measures are going to be

• # of users who attended the training courses

• Classify all the measures into the different categories– Implementation Metrics– Efficiency Metrics

Practical Implementation of MetricsDevelop the model for measurement


Confidential


– Efficiency Metrics– Effectiveness Metrics– Impact Metrics

• Identify methods to collect data

Practical Implementation of MetricsImplementation of identified metrics

Metric Data Collection Method

# of employees to whom the information security policies are communicated

•Review of HR records to see signed security policy documents•Review of end user training


Confidential


communicated •Review of end user training attendance records

# of network scan attacks blocked at the firewall level

•Firewall Logs

# of times files with rating ‘ confidential’ were tried to be sent outside the network

•DLP software logs

# of users who entered the data centre for maintenance of equipment

•Biometric access control logs review

• Identify effort required to collect the identified metrics

• Do a cost – benefit analysis to understand

Practical Implementation of MetricsImplementation of identified metrics


Confidential


analysis to understand if the metrics need to be collected

• Implement procedures to collect metrics

• Identify frequency of data collection• Wherever possible use technology to collect

data– May not be possible to automate everything, but

automate where you feel the cost benefit is justified

Practical Implementation of MetricsCollection of data


Confidential


automate where you feel the cost benefit is justified

• Wherever possible, build the data collection process into the existing security practice– Use of workflows will aid the metrics to a large extent

• What the security measurement tools vendors will never tell you...

Practical Implementation of MetricsCollection of data

What they say What they actually mean

Our software provides you a ...only for devices where our agent is


Confidential


Our software provides you a complete view of information security within the organization

...only for devices where our agent is installed, not for the other devices or your manual processes

Our software captures 184 different measures for customized use in your organization

...yet, we cannot capture what missed us and went through to cause an incident

Antivirus reports – collected daily

Number of changes to key

systems –collected monthly Number of

exception approvals for use

of USB –collected monthly

Practical Implementation of MetricsAnalysis of metrics


Confidential


Output of

metrics

Management review meeting –

collected annually

collected monthly

Number of visitors with laptops –

collected weekly

• What the consultants will never say...– “We will give you as many metrics as the

money you have. Analyzing and making them useful is not our concern”

Practical Implementation of MetricsWhat is the output of metrics analysis?


Confidential


Metric What it can mean...

Management review meeting happened once in year as against 4 times

Lack of management commitment is evident. Information security may not take off unless there is

Antivirus reports show 3-4 virus attempts blocked daily at the gateway level

Antivirus is working.

Number of changes to key systems has Investigate – a new system being deployed,

Practical Implementation of MetricsWhat is the output of metrics analysis?


Confidential


Number of changes to key systems has doubled this month as compared to the average

Investigate – a new system being deployed, causing the change? A change in the older implementation causing problems?

7 new laptops allowed to connect USB drives. A total of 44 out of 51 laptops now allowed

Investigate – looks like a potential cause for policy change

Out of the 69 visitors this week, 38carried laptops

Fire the consultant who told you to collectthis data.

• Security Metrics are useful at different levels in the organization

Practical Implementation of MetricsReporting the Metrics

Level of the organization What should be reported?

Senior Management –Security Management

•Level of security in terms of Red – Amber – Green•Legal and Contractual Compliance levels


Confidential


Security Management Forum

•Legal and Contractual Compliance levels

Information Security officer

•All the metrics

Business Owners •Incident metrics•Exception metrics•Security Compliance levels

• Dashboards• Control Objective/Control wise score• Reports

Practical Implementation of MetricsReporting the Metrics


Confidential


Practical Implementation of MetricsPolicy Compliance Graph


Confidential


Practical Implementation of MetricsPolicy Compliance – Another View


Confidential


Practical Implementation of MetricsPolicy Implementation Levels


Confidential


Information Security Policies and procedures

Develop a model for measurement

Reporting of metrics

Practical Implementation of MetricsSummary


Confidential


for measurement

Identify methods for

implementation

Collection and Analysis of data

metrics

• Metrics can be very useful to measure and improve the security of the organization

• Best if linked to business goals• Identify and measure only those metrics that

will be useful to the organization

Information Security MetricsSummary


Confidential


will be useful to the organization• Use the metrics to identify and implement

improvements – use the feedback to identify and improve the metrics

• Best if incorporated into the daily processes of the organization

Who are we?MIEL is a pure-play pioneering, end-to-end, Information Security Solutions company, with strong values, an unique business model that has helped service more than 800 premium Indian and International clients with footprints in15 countries spread across the globe.

What solutions do we deliver?We :

• preach (education services),• practice what we preach (process and technical

About MIEL


Confidential


• practice what we preach (process and technical consulting) and

• implement what we practice (product and service implementations)

Who and what helps us deliver?Our management teams supported by quality processes and backed by highly trained resources, and a strong R&D environment, help deliver desired results and accolades.

MIEL IS A CERT-IN EMPANELLED AND ISO 27001 CERTIFIED

Thank You

MUMBAI � PUNE � AHMEDABAD � BANGALORE � HYDERABAD � CHENNAI � LONDON � UAE � USA Consulting - Products Solutions - R&D – Education

© 2009 MIEL eSecurity Pvt Ltd

Confidential

Technology

Chaitanya Kunthe - Security Metrics - Interop Mumbai 2009