CM Introduction 081414

Introduction to ControlMetricThe Science of Internal Control™

This information is confidential and was prepared by ControlMetric solely for the use of our client; it is not to be relied on by any 3rd party without ControlMetric’s prior written consent.2

• Managing operational risks adds layers of complexity and associated costs to business processes, yet many companies find it difficult to assess how much risk is mitigated by their choice of controls

- Business process and IT-related risks are not sufficiently quantified as part of a risk assessment

- Internal controls are usually not formally described using appropriate attributes, i.e. effectiveness, efficiency, coverage, level of risk mitigation and cost

- Selection of internal controls can provide a source of competitive advantage to organizations by mitigating the appropriate level of risk at the right cost

• The ControlMetric approach is to develop a rigorous, quantitative view on the operational risks facing the business, and the ability of a group of controls to mitigate risk in a business or IT process

- This allows for the development of an “efficient frontier” of controls versus cost to enable the choice of the most cost-effective set of controls

- Using a quantitative approach aids in the adoption of good internal control practices by introducing a standard, data-driven methodology

- This can provide an alternative view of risk and control that can augment more traditional, qualitative approaches

• Our clients benefit from the knowledge that selected internal controls will mitigate the appropriate level of risk based on their design

- Investment decisions to support internal control spending are more consistent across the organization and ensure the most efficient use of internal control resources

Summary of our thinking


Most companies are at an early stage of maturity in managing these risks…

Drivers • Compliance with basic standards and regulations

• Reduction of regular surprises

Initial transparency

stage

KeyTools

• Opportunistic approaches

• Checklists• Very limited

enterprise technology support

Source: Adapted from McKinsey Working Papers on Risk, What’s Different in the Corporate World

• Avoiding unexpected large loss events

• Stability to enable growth plan

• Professional risk management

Systematicrisk

reduction

• Risk heat map• Consensus

management• Basic risk

quantification• Fragmented

technology tools

• ROE improvement requirements

• Competitive pressure

• Navigating trade-offs

Risk-return

management

• At-risk measures (eg. VAR, CFAR)

• Systematic scenario analysis of profit and loss and risk impact

• BU/function level technology in use

• Top management focus on risk-adjusted performance

• Finding niche in competitive marketplace

Risk as competitiveadvantage

As left plus:• Strong risk culture• Unbundling of risks

through contracting and markets

• Active visibility into enterprise risks through pervasive technology

1 2 3 4

This information is confidential and was prepared by ControlMetric solely for the use of our client; it is not to be relied on by any 3rd party without ControlMetric’s prior written consent.

…but operational risk events have the greatest impact on investor confidence

4

The challenge is to manage these risks in an effective and transparent way while promoting an atmosphere of innovation

and risk-taking


We bring an approach that moves companies to higher maturity level of risk management

Risk

Cost of Control(proxy for # of controls)

Quantify total risk (uncontrolled)

Select relevantcontrol subsets

Analyze possible control subsets

1

3

2

= Possible control subset


Most organizations fail to adequately quantify business process and IT risks

6

1

Traditionally, higher-level risks are often scored, usually as “high”, “medium” and “low”. Risks at the tactical level, where controls are selected and implemented, are usually not assigned any quantitative or even qualitative measure.

Usual Risk “Scoring”

• At the tactical level, usually no measure of size of risk assigned

• Subjective, qualitative• Based on individual knowledge or

expertise• No ability to assess impact of

incremental investment in internal controls

ControlMetric Approach

• Each risk is sized as a dollar impact• Quantitative• Based on collective knowledge and

expertise, driven through an analytical methodology

• Framework provided to analyze best use of marginal investments (or de-investments) in internal controls

We believe risk MUST be quantified for all tactical risks to ensure organizational acceptance of the controls proposed for

risk mitigation


The ControlMetric model scores all possible control options…

• All possible subsets of controls

• Ordered on process risk mitigation

• Includes “mandated” controls (e.g.

regulatory requirement)

• Addition of cost information enables “efficient frontier”

Ranking of viable subsets

Subset

Subset

Subset

Subset

Control

Control

Control

Control

Control

Control

Control

Control

Includes all sources of control

ControlMetric model

Universe of Controls All possible subsets

Subset

Subset

Subset

Subset

Subset

2


Risk

Cost of Control(proxy for # of controls)

= Possible control subset

Steep inflection points identify biggest impact of incremental

additional controls

“Interesting” solutions lie close to the efficient frontier

Residual risk can be explicitly defined and agreed

These control choices should move down or left to optimize

cost or risk mitigation

3…making it possible to select the best set of controls to mitigate the risk


Internal Control Analytics

• Apply the risk model to specific business processes, IT systems, departments, business units, etc.

• Output is focused on building consensus for the “right” set of controls to mitigate quantified risk

• Allows for sensitivity testing on control effectiveness and overall investment in control

Enterprise Risk Assessment/ Management

• Broad-based, qualitative and quantitative assessment of most important risks across department, business unit, or enterprise

• Statistical calibration of participants prior to assessment to minimize bias• Ongoing monitoring and scoring of risk “opinions”• Large scale surveys, “crowdsourcing” and prediction markets to optimize

risk forecasts

Internal Audit

Services

• Apply quantitative tools to determine appropriate controls for entity to be audited

• Shift IA role to controls experts and advisors, in addition to usual assessment role

• Increase value proposition for IA• Suited for turnkey operations for internal audit

Project Risk Assessment/ Management

• Three step approach that includes detailed project planning, “real options” based project structuring, and dynamic risk assessment and tracking

• Development of risk map showing correlation of risks and dependencies on outside variables

• Allows for earlier recognition of potential project risk failures enabling a more rapid management response

We apply our quantitative risk approach across different services…


…with a particular (but not exclusive) focus on growing companies

Observations

• Growing companies run faster than their ability to mature operations & capabilities

• Business operations grow independently across the enterprise

• Other business priorities reduce the focus on risk management and internal controls

2

1

3

Implications

• Complexity increases as rules and controls are added to manage “chaos”

• High risk of duplication of business approaches, processes and controls

• Controls are relegated to an afterthought or considered solely as a response to an adverse event

The strategy and methods for risk management and internal control must align with organizational values and be grounded in

a defensible, quantitative approach


Our approach includes the following six steps

• Identify and normalize the risks related to the domain being assessed

• Measure these risks using both qualitative and quantitative data to determine the size (dollar value) of the risk

• Define the universe of possible controls to mitigate each of these risks – including those controls already in place or planned

• Develop effectiveness scoring and cost information for each of the identified controls

• Run the ControlMetric™ model to generate the optimal control combinations for each level of risk mitigation – the “efficient frontier” of controls

• Analyze the possible optimal control combinations and determine the gap between these and current practice and make prioritized recommendations on additional or changed controls for each identified risk

1

2

3

4

5

6


Important to conduct a comprehensive analysis of the risks in that domain 1

Data Security (example)

• Personally-identifiable health information is disclosed to unauthorized individuals

• New product specifications are made available to the competition

• Employee payroll information is made available to all employees

• Critical financial records for accounts receivable are incorrectly modified

• Patent application materials for a new product are lost

Sources of these risks include external references (e.g., COBIT), previous control work (e.g., SOX), SME interviews and

broad-based polling (“crowdsourcing”)

• Financial results are released to the press ahead of schedule

• Many employees are granted access to restricted “superuser” functions on an important application

• Cryptographic keys used to generate access codes are not protected

• The master password for system recovery is lost• Key test data for a new product are modified by an

unauthorized employee

Risks could include:


Key principles for effective risk identification and normalization 1

1. The impact of each risk identified must be measurable in dollars. • Health information is disclosed to unauthorized individuals• Sensitive company information is disclosed

2. The risk must be capable of being mitigated using specific controls• Cryptographic keys are not protected• Public key symmetric algorithm is globally compromised

3. Only risks with a material impact are worth considering• Key test data are modified by an unauthorized employee• Employees posting to internal blog are not identified

4. There shouldn’t be too many risks!

The identification of risks for the domain combines “art” and “science” to provide a normalized basis for further analysis

KeyPrinciples

Applying these principles focuses the analysis only on those risks which have a material impact on the company and which

therefore warrant the most effort around controls


Point or Range Estimates

• Can be generated either externally (e.g. industry benchmark) or internally (e.g. planning assumption)

• Often backed by historical experience or external analysis• For example, external benchmark for risk of “shadow payroll” fraud is

0.1% of total payroll

Several different approaches can be used to quantify risk 2

Easier

Harder

Response Cost Analysis

• Focus on responses to risk occurrence as an estimate of the risk impact

• Responses are categorized and cost estimates are generated for each response

• Can either be a point or range estimate

Crowdsourcing

• Uses the power of many opinions to generate a more reliable estimate of risk

• Can be generated either internally to the organization or, in some cases, can be extended to business partners

• Can be extended to include prediction markets

Input Modeling

• Decompose risk down to input variables impacting the likely outcome of risk

• Decide on statistical distribution for each input variable• Model range of input variables to generate distribution of likely risk

values, e.g. Monte-Carlo


2For example, analyzing response costs can be useful in sizing data security risks…

Actions

EstimatedTotal Cost

$500K $1.2M

• Conduct emergency security audit

• Apply security patches

• Recertify server and security software

• Pay fines for non-compliance

• Increased communication to customers and regulators of security activities

• Senior management time on response

$600K

• Notify and follow-up with impacted customers

• Provide and activate credit monitoring

• Offer discount coupons on future purchases

• Impacted customers reduce spending by 80% for six months

• Web traffic reduced by 5% for one week

$3.4M

Account data of 10,000 customers released on

Internet

Security incident

managementFuture revenue

lossReputation

managementCustomer

notification

Risk

Responses

Total cost of response is approximately $5.7M – this becomes the estimate of risk

Illustrative


…while decomposing risk to its components can work for business risks…

Overall Fraud Risk

Claims FraudAccounts Payable Fraud

PayrollFraud

• Historical experience of claims fraud is in the range of 3-4% of incurred losses 1

• Industry data suggests 10% of incurred losses represent claims fraud 2

• Industry data suggests 5% total revenue is lost to all fraud 3

• Average loss per incident related to disbursements in the range $20-125k 3

Claims fraud risk dominates; overall fraud risk estimated at $12M

• No historical experience of payroll related fraud

• Industry averages are in the range of 1% of total payroll expense 4

1. Client provided2. ISO survey; 20103. AFP Payments Report; 20104. Association of Certified Fraud Examiners, “Report to the Nations” 2010

Illustrative

2


2…and Monte-Carlo modeling of project inputs can help assess ROI risk

Illustrative

90% Confidence Interval of expected ROI is -5% to +18%.

This provides an estimate of the ROI risk.

• Time to complete system – 12 to 18 months• Cost of new system - $4M to $8M• Predicted agent adoption – 40% to 70%• Additional revenue per agent - $500K to $1.5M• Margins on additional revenue – 20% to 25%• New system operating costs - $140K to $300K• Internal productivity savings with new system - $400K to $700K

Model these inputs to generate estimate of project ROI

Input factors influencing project

ROI


3For each risk, identify the universe of possible controls to mitigate that risk

• There will likely be multiple sources of reference to list possible controls- IT controls frameworks, including COBIT, ITIL, IT-CMF, etc.- Industry, function and process best practices- Professional reference

• Controls included in the universe are likely to be of different types- Manual vs. automated (technology-based)- Detective, preventive, administrative controls

• Many higher-level controls will mitigate many risks- Policies and procedures, for example system access procedures- Management reviews, for example expenditure vs. budget analysis and approval

• Universe should include existing or planned controls- Provides a starting point for the analysis- Allows for comparison of existing controls to optimal solutions

• Specific compliance and regulatory requirements should be included- Controls can be included now or added after optimal control subsets have been selected for

specific risks


Effectiveness and cost of each control is estimated from data and past experience 4

• For a specific risk, how much of that risk is mitigated assuming the control is operational at all time

• This is expressed as a percentage of the total risk

Coverage

• This is an estimate of how often this control works over time

• Does the control work all the time (e.g. many automated controls) or are the times when the control is not reliable (e.g. operator fatigue)?

Operational

• This measures how well this individual control can deal with minor anomalies related to the risk being mitigated

• For example, can the control recognize an alternate approver for a purchase if that information has not been formally included in the control?

Flexibility

Combine to generate an

overall control effectiveness score for each

control

• What are the estimated costs associated with this control?

• This should include operational (on-going) costs as well as any initial design and implementation costs

Cost


Controls are scored based on the particular risk being mitigated 4

Ref. Control Cov. Oper. Flex. Cost

1 Information security standards and guidelines exist. These standards and guidelines serve as the basis for security administration, management, and monitoring. This policy also defines the responsibilities of our Information Security Officer, users and management.

0.4 0.5 0.85 $300K

2 An Information Security awareness program exists and is updated on an annual basis. 0.4 0.5 0.85 $500K

3 Generic user accounts (e.g., Temp01) are not used to access and perform transactions within business applications.

0.65 0.85 0.9 $50K

4 Each business user is assigned a unique account using a standard naming convention to ensure accountability for each user.

0.85 0.5 0.9 $25K

5 All requests for new user access to App/DB/OS/Network are submitted in writing by an individual authorized to approve access.

0.8 0.4 0.8 $25K

6 Employee terminations are communicated by HR or management, in a timely manner. Accounts are disabled/removed in a timely manner.

0.7 0.3 0.9 $35K

7 All user access additions and modifications made in the App/DB/OS/Network are documented and maintained.

0.5 0.3 0.8 $50K

8 Application sets defined spending limits for each user 0.9 0.9 0.3 $10K

9 Reports of current App/DB/OS/Network access privileges are periodically generated and distributed to process/data owners for review. Process/data owners validate propriety of access rights. Access privileges are modified as appropriate.

0.8 0.5 0.7 $75K

Control scores (Coverage, Operational, Flexibility)

Illustrative

“Critical financial records for accounts receivable are incorrectly modified”


Some observations on effectiveness scores and costs 4

Observations

• Scores are generated from many available sources of subjective and objective data including external benchmarks, our experience, client history and qualitative and quantitative analysis

Implications

• The availability of “good” data on controls impacts the quality of the analysis; additional data gathering through “crowdsourcing” and other polling methods can make a big difference

• Automated controls tend to have higher operational scores but lower flexibility scores

• An over-reliance on automated controls, while cost-effective, can limit adaptability in the internal control structure

• Supervisory-type controls (e.g. management review) can be provide broad coverage and increase flexibility while empowering process owners to manage risk

• In order to internalize effective, quantitative-driven risk management into the IT organization, some number of supervisory controls must always be in place

• People-based controls have higher ongoing costs but are relatively easy to design and implement; the operating costs of automated controls approach zero but there are non-trivial costs associated with the design and implementation of the controls

• Both on-going operational costs and one-time design/implementation costs should be understood to ensure that a true cost picture is presented


The ControlMetric model presents an analysis of all possible control subsets

Risk($)

Cost of Control ($)0 100 200 300 400 500 600

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

5

• Each “dot” represents a particular set of controls – an individual subset of the original universe of controls

• Usually, the “efficient frontier” of controls is obvious from the chart and represents the optimal control choices for a given level of risk mitigation

• At this point, we can also determine the level of residual risk remaining for any particular set of controls chosen

Questions to be asked

1. Does the chart appear reasonable given our knowledge of the control environment?2. Do the control subsets that are on or close to the efficient frontier appear reasonable?3. Do any control subsets suggest that we need to reassess the cost or effectiveness

data inputs?4. Do the levels of residual risks appear to be within acceptable ranges to the business?

Residual risk


Important first step is to establish a range of residual risk acceptable to the business 6

• The residual risk represents the amount of risk for this particular process that will not be mitigated away when a particular set of controls is selected

• In this example, it ranges from about $22M on the high end with very minimal controls in place to about $4M on the low end

• The acceptable level of residual risk is generally defined by the process owner

• The cost of additional controls is a key factor in determining the level of acceptable residual risk

In this example, the cost of additional controls appears small compared to the associated risk mitigation; it makes sense to

mitigate as much risk as possible


Analysis focuses on controls that provide the required level of risk mitigation

Risk($)

Cost of Control ($)0 100 200 300 400 500 600

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

6

= Current controls

D14

C72

A17A36

B71 C65

D22A32

C41

D13

Control Subsets

• Each of the identified control subsets (D14, C72, etc.) represents a unique combination of controls from the universe of identified controls

• The currently implemented set of controls is some distance from the “efficient frontier” indicating an opportunity to either reduce control cost (while maintaining the current level of risk mitigation) or increase risk mitigation (while maintaining the current cost)


Control subsets should be assessed for effectiveness and cost of implementation 6

• This group of control subsets is chosen for further analysis based on acceptable residual risk

Subset Controls Residual Risk

Cost Overlap to Existing Controls

Difficulty of Implementation

A36 3,4,7,11,15 $9,750,000 $425,000 High Medium

A17 1,5,6,8,9 $8,500,000 $420,000 High Low

D14 3,5,8,12,20,21 $4,750,000 $405,000 Medium Low

B71 1,5,8,11,21 $4,600,000 $460,000 Medium Medium

C65 1,3,6,8,9,11,13,14,21,22 $4,450,000 $505,000 Medium High

The final choice of controls is based on a subjective review of these criteria


Why Controlmetric?

•We bring a rigorous, data-driven approach to risk analysis- ControlMetric was founded by industry veterans to radically improve the

quality and rigor of professional services related to all aspects of internal control. Our mandate is to enable our clients to build the most cost-effective systems of internal control while providing transparency into risk mitigation.

•We bring in-depth expertise in risk management and internal control

- Our leadership team consists of accomplished professionals with significant experience in internal controls. We have also worked with some of the premier companies across multiple industries. Finally, as an early stage company, our principals will be the team that works with you.

•We are fact based with no pre-conceived bias- We believe we can frame the issues and know how to conduct the analysis to

determine the right answer. We conduct rigorous analysis to determine the strength and the rigor of our work. Facts and analysis will always drive our recommendations


Documents

CM Introduction 081414