[email protected] Towards Incentive-based Cyber Trust Russell Cameron Thomas Principal, Meritology [email protected] Patrick Amon

[email protected]

Towards Incentive-based Cyber Trust

Russell Cameron Thomas

Principal, Meritology

[email protected]

Patrick Amon

Senior Researcher

Center for Interdisciplinary Research for Information Security, Ecole Polytechnique Federale de Lausanne

[email protected]

i-Society `07, Oct. 10, 2007, Merrillville, IN

i-Society 07, Merrillville, IN - Oct. 10, 2007

Page 2

[email protected]

© 2007 Meritology. All Rights Reserved


Summary

• “Cyber trust” = the confluence of information security, privacy, digital rights, and intellectual property protection

• People and institutions are not properly motivated to maximize social welfare

– Incentives are often perverse, misaligned, or missing.

• Conjecture: An incentive-based approach will work better than other approaches alone:

– Technology, mandates, penalties, and/or political actions

• Essential elements

– Usability, risk information systems, risk communications, social

knowledge, markets, and incentive instruments, plus enabling

technology and a supporting legal/ regulatory/institutional framework

• Example: accounting for cyber risk


Page 3

[email protected]



Q: Why “Cyber Trust”? A: Confluence!

• Sony BMG Music Case (2005) –

– Distributed a copy-protection scheme with music CDs that secretly

installed a root kit on computers that played the CDs

– “Root Kit” was installed without user’s knowledge or consent.

• A “root kit” can allow someone else to gain and maintain access to your computer system without your knowledge.

• Just one example of the confluence of…

– Information security

– Privacy

– Digital Rights

– Intellectual Property protection


Page 4

[email protected]



Alternative Approaches

• Technological Approach – “Automate”– Focus: Information infrastructure, platforms, applications, interfaces– Actions: technological capabilities and fixes– Human and organization actors either absent, secondary, or mere users of

technology.

• Mandates-based Approach – “Do this…”– Focus: “Security is about control”– Actions: regulations, policies, procedures, rules, laws, codes of conduct,

contracts, etc.

• Penalty-based Approach – “… or else!”– Focus: deviant behaviour and lack of will power to resist temptations to

cheat or exploit– Actions: fines, lawsuits, contractual penalties, decertification, etc.

• Political Approach – “Change the power structure”– Focus: sub-optimal power relationships and unrealized collective interests – Actions: alliances, coalitions, power-shifting actions (e.g. anti-trust law

suits), countervailing actions or threats, reciprocal commitments, standardization efforts, and communications to influence public opinion.


Page 5

[email protected]



Limitations of Alternative Approaches

• Technology alone will not be sufficient– Cyber Trust = f( People, Process, Technology)

• An emergent, systemic feature.

• Even if sufficient technology were available, would people buy it and use it?

• Mandates + Penalties are too cumbersome and slow– Environment is complex, context-dependent, and fast changing

– Does not promote creativity and innovation

– Hard to make it work across boarders

– Too many unintended consequences, especially in crisis situations

• Political solutions are too blunt– Will the new regime be better than the current regime?

– Hard to make it work across boarders

– It is very hard to fine-tune or refine


Page 6

[email protected]



The Incentive-based Approach

• Essence: “Give key actors a share of the potential gains of cyber trust, and thereby draw on the power of self-interest to drive the right actions.”

• Prerequisite: Good understanding of… – What motivates individuals and institutions and what they value

– How they perceive cyber risks and rewards

– How to create incentives to shift those motivations in positive directions.

• Advantages = power of markets– Can start on a small, local scale

• All that is needed to get started is for two parties to have some measure of their relative cyber risk across decision alternatives and how relative cyber risk is driven by observable metrics.

• Current state– Outside of copyright, digital rights, and IP licensing, there has been little

success in monetizing the value of cyber trust.


Page 7

[email protected]



Essential Elements

Pervasive Information &

Communications Technology (ICT)

7. Enabling Technology8. Supporting Legal, Regulatory

& Institutional Framework

6. Risk Info. Systems

3. Incentive Instruments

5. Markets

2. Risk Communications

4. Social Knowledge

foundations

influencingstakeholdermotivations

socio-economictechnological

1. Usability

pooling andprocessing

risk information


Page 8

[email protected]



Incentive Instruments• Definition:

– any social or economic device, mechanism, process, or agreement that explicitly ties payoffs for actors to desirable future states of the world so that those actors are motivated to help bring about those states.

– A “payoff” could be monetary, near-monetary (e.g. a tradable good or service), or non-monetary-but-valuable (e.g. offer of mutual assistance).

• Puts the value proposition of cyber trust front-and-centre for each stakeholder.

• Opens the possibility of side payments, and other balancing transactions.

• Cyber Trust Examples:– cyber insurance, risk-sharing contracts, and “bug bounties”

• Other Examples:– risk sharing pools in developing countries, risk-based payments and

contracts in supply chain management, decision insurance (internal to an organization)


Page 9

[email protected]



Incentives at Work:Accounting for Cyber Risk

• Problem

– How to map cyber risk metrics to business performance.

• Within a firm

• Between information supply chain partners, esp. outsource vendors

• Solution

– Total Cost of Cyber (In)security

• Analogous to “Total Cost of Quality” in Total Quality Management

– Involves

• Risk Information System

• Risk Communication

– Enables

• Incentive instruments


Page 10

[email protected]



Total Cost of Cyber (In)securityA

nnua

l Pro

babi

lity

Total Annual Cost of Information Security (log scale)

mean

1x 10x 100x 1,000x

1 2 3 4 5 6 7

“Budgeted” “Self-insurance” “Catastrophic”

bankruptcy


Page 11

[email protected]



Self-insurance CostsA

nn

ua

l Pro

ba

bili

ty


1x 10x 100x 1,000x

1 2 3 4 5 6 7

“Self-insurance”

Definition:• Self-insurance* premium to cover difference between the Budgeted costs

and an exposure limit.

What’s included:• Low probability/high magnitude loss events big enough to “bust the budget”

• material to quarterly earnings• threaten the firm’s credit rating


Page 12

[email protected]



Estimating Self-Insurance Cost

99th Percentile threshold

1 2

Budget threshold

Self-insurance pool (“Value at Risk”)

3 Time period*

54 Fund solvency*Shape of the curve

Annual premium ≈ Pool ÷ (Time Period)Annual premium ≈ Pool ÷ (Time Period)

Estimation Parameters

* Policy decisions by top management

Modeling:

• Distribution curves from parameters

• Monte Carlo simulation of self-insurance pool with funding parameters, interest rates, etc. to calculate annual premium

• Dominated by largest losses 2

6 Interest rates

Cost distribution

curve

Magnitude of costs

(if time period is long enough)


Page 13

[email protected]



Calculating Self-Insurance Cost (2)

How: A Competitive Marketplace for Models

time

ConsensusEstimates

Prediction Markets

Delphi Technique

Qualitative Reasoning (e.g. Inference to the Best Explanation,

Reasoning about Uncertainty, etc.)

Bayesian Networks

Statistical analysis of

historical loss data

External data bases, benchmarks

parameter

Parameter values change with new information

Assessments,Scorecards

Simulations


Page 14

[email protected]



Guiding Risk Management Decisions

BudgetedCosts Self-insurance

Costs

CatastrophicCosts

Prudence Gambling


Page 15

[email protected]



Research Questions

• Is it theoretically possible to model cyber trust risks and incentives in a unified, forward-looking valuation framework?

– What are the fundamental limits?

• If analytic models are not feasible, is it possible to devise coarse-grained or qualitative models that are robust and usable in practice (e.g. rating or ranking schemes) as the basis for incentive instruments?

[email protected]

Questions?

Russell Cameron Thomas

Principal, Meritology

[email protected]

Patrick Amon

Senior Researcher

Center for Interdisciplinary Research for Information Security, Ecole Polytechnique Federale de Lausanne

[email protected]


Page 17

[email protected]



Definitions

• “Cyber trust risk” or “Cyber risk” – the socio-economic risks associated with cyber trust, from the viewpoint of

all relevant stakeholders.

• “Incentive” – “Any factor (financial or non-financial) that provides a motive for a particular

course of action, or counts as a reason for preferring one choice to the alternatives.” plus…

– Gain sharing or shared equity associated with desirable outcomes, including remunerative, moral, and personal incentives.

– Excludes negative incentives (penalties)

• “Risk management” – Managing uncertain and uncontrollable outcomes by estimate the likelihood

and severity of uncertain events and then use these estimates in a rational decision-making framework

– General spirit: balance the expected value of losses with the costs for mitigating those losses.

– The sociological aspect of risk management incorporates ideas such as risk tolerance/aversion, bias, risk perception, and motivational dynamics


Page 18

[email protected]



Literature Review

• Overviews

– National Academies (2007), “Toward a Safer and More Secure Cyber Space”

– Computer Research Association (2003), “Four Grand Challenges of

Trustworthy Computing

– Rueschlikon Conference (2005), “Ensuring (and insuring?) critical information

infrastructure protection”

• Seminal Papers

– Anderson (2001),“Why information security is hard - an economic perspective”

– Acquisti

– Camp


Page 19

[email protected]



Q: Include Penalties as Negative Incentives?A: No

• Negative incentives tend to promote avoidance behaviours

– shirking, blame shifting, and information hiding (both obscuring and

misrepresentation)

• Doesn’t ensure or encourage the most desirable outcomes (i.e. optimization)

– At best, you can hope to avoid the worst categories of outcomes.

• Doesn’t mobilize the power of market systems

– Much of the power of market systems comes from its capability to

spawn new and complementary markets that share gains and risks.

– But market systems almost never "trade" negative incentives.


Page 20

[email protected]



1. Usability

• Personal incentives are embedded in the design of information and communication systems, and specifically in the usability of their cyber trust features.

– Making it easy to do the right things,

– Making it hard to do the wrong things

– Making it clear what the risk consequences are of possible actions.

• Usability includes technology, people, and processes.

• Poor usability can undermine all the other incentive elements.


Page 21

[email protected]



2. Risk Information Systems

• Information systems to continuously collect and aggregate operational cyber trust information.

– Security and privacy metrics and assessments

– IP asset inventories, licenses, obligations, contingencies

– Digital rights, cost and revenue streams

• Analysis to discover cause-effect relationships

• Models are needed to help stakeholders make forward-looking, value-based decisions based on risk scenarios and trade-offs.

• Models will have to cope with many forms of ignorance and uncertainty


Page 22

[email protected]



3. Risk Communications

• “Risk” has different meanings at individual, organization, and society levels

• Very specific to context and systemic performance

• Best to give feedback in real-time

• Risk factors are very interdependent, making the cause-effect relationships very complicated

• Relevant knowledge is contingent, tentative, vague, ambiguous, and even contradictory

• Risk cannot always be measured by a simple numerical scales

• Prior perceptions and mental models are critical

• Most people find the technical details befuddling and taxing

• Many social and political obstacles to disclosing information about cyber trust and risks


Page 23

[email protected]



4. Social Knowledge

• Knowledge about cyber trust is widely distributed, not concentrated

– Vulnerabilities, exposures, incidents, losses, mitigation, cost, and

forward-looking estimates and perceptions.

– Cyber trust is very dependent on context. Only the people in that

specific context have the necessary information and perspectives to

make proper judgments.

• Cyber trust involves both perceptions and forward-looking estimations of risk. These are social processes.

• There may be some elements of incentive-based cyber trust that can only be produced by the “wisdom of the crowds”, including valuation of hard-to-estimate risks and best practices.


Page 24

[email protected]



5. Markets

• Problem: incomplete markets– “markets” = trading systems that allow buyers and sellers to

exchange goods and/or services, including information.

– Buyers do not know what improvements cyber trust they are getting when they buy each product or service.

– Economic actors can not make rational trade-off decisions, leading to inefficient allocation of resources and less-than-optimal results.

• Possible market innovations:– Synthetic and simulated markets that are created specifically to

discover prices

– Prediction markets to draw out the “wisdom of the crowds”

– “Cap and trade” markets for externalities

– Markets for private information (e.g. “Zero-day” vulnerability auctions)


Page 25

[email protected]



Application: Financial Services (FS)

• The FS sector is one of many sectors that face significant cyber trust challenges:

– Information security – insider abuse and fraud, external targeted

attacks and fraud (e.g. “phishing”, money laundering, etc.)

– Privacy – misuse of customer’s private data (including identity

theft), threats to confidential information (insider trading, improper

disclosure, etc.),

– Intellectual property protection – computer software, proprietary

financial data, and even patented business processes.

– Digital rights – not significant in FS now, but might be changing.

• FS leads in operational risk modeling and enterprise risk management


Page 26

[email protected]



Context: Two Viewpoints on Economic Risk

#1 “Rational Investor”(Capital Asset Pricing,Discounted Cash Flow)

#2 “Insurance Actuary”(Ruin Theory,“Iceberg Risk”)

“Ruin”

time

timechange in value

change in value

valu

eva

lue random walk with

“avalanches”

random walk

p(v)

p(v)

What matters:• Mean, variance• Fat part of the curve

When:• Quarterly EPS• Earnings volatility• Shorter time periods

99%

What matters:• Extreme events• Tail of the curve

When:• Credit rating• Solvency• Reserve funds• Longer time periods

Normaldistributions

“Fat Tailed”and skeweddistributions


Page 27

[email protected]



Budgeted CostsA

nnua

l Pro

babi

lity


mean

1x 10x 100x 1,000x

2 3 4 5 6 7

“Budgeted” “Self-insurance” “Catastrophic”

bankruptcy

1

Definition: Costs that in are in the budget somewhere

What’s included:• All direct spending on security, privacy, IP protection…• Plus indirect costs, plus the expected value of all high

frequency losses and some small mix of lower frequency losses.

• Also includes the opportunity costs – business activities that are prevented or inhibited by security.

How it’s estimated:• Accounting records, budgets, and business cases• Cost-driver models (i.e. linear relationships between

operational metrics and indirect or overhead costs, etc.)


Page 28

[email protected]



Ways to Make Self-Insurance Cost “Real”

• Link it to real cyber insurance policies

• Set up a real self-insurance fund via Finite Risk program or tradable subordinated debt

• Use it as the “glue” for multi-firm “risk sharing” pools

– Focused on information sharing and mutual assistance, with incentive instruments

• Link to performance management and incentive compensation

– Subdivide Self-Insurance Cost into a “Risk Budget” for each org. unit, or

– Use it as a “risk adjustment” factor for other performance metrics

• Create incentive instruments tied to self-insurance costs or cost drivers for…

– Security outsource vendors

– Supply chain partners

– Channel partners

– Customers

– Alliance partners

• Public disclosure

– SEC filings, other regulatory

filings

– Stakeholder reports

– Credit rating agencies

– “Cap and Trade” markets


Page 29

[email protected]



Catastrophic Costs

• Q: How much confidence should we have that the firm can survive InfoSec catastrophes?

• The rule: prioritized loss scenarios above a significance threshold that cover the space of possibilities.

– Use for business continuity preparation → agility and robustness

– Avoid failures of imagination and “fighting the last war”

– Root out unintended consequences

– Categorize and prioritize – don’t waste time on precision estimates

– Strategic scenario analysis, “war gaming”, etc.

– Focus on discovery, “out of the box”, and reframing

– Challenge conventional wisdom!

• “It’s not what we don’t know that will kill us. It’s what we know that ain’t so”.


Page 30

[email protected]



The Challenge

• Information Security performance is an unruly moving target– Fast-changing, rapidly evolving.

– Evolutionary strategic game between attackers and defenders

• Potential Impact– False sense of security, complacency

– Fighting the last war

– Falling behind in the “arms race”

– Failures of imagination

– Chasing ghosts (F.U.D)

– Unintended consequences (self-defeating behaviors, mal-adaptations)

• Why it’s hard– Shrouded in uncertainty, ignorance, ambiguity, and indeterminism

– Not just “puzzle solving”, but also “mystery solving” (“Unknown unknowns”)

– Must integrate with enterprise performance and incentive systems


Page 31

[email protected]



Meta-metrics for “Double Loop Learning”

Single loop learning:• Control loop with pre-

defined outcome

Double loop learning:• Control loop that adjusts

the defined outcome.

Source: http://www.learning-org.com/


Page 32

[email protected]



A Model of Risk Management

Source:

The point: Risk mitigation must be seen in the context of risk taking


Page 33

[email protected]



Learning – Do We Know What We Need to Know?

• Balanced Scorecard: "Learning & Growth” perspective– Typically includes skills, employee retention/satisfaction, research, etc.

– Not independent of other metrics. Should link to other metrics in causal and feedback loops.

– Single loop vs. double loop learning

• Learning in order to solve “puzzles”– Security Awareness Training

– Security Management Training (incl. use of metrics)

– Incident Management (preparation, planning)

• Learning in order to solve “mysteries”– Threat awareness

– Emerging (and emergent) vulnerabilities

– Systemic risk

– Interdependencies with other enterprise risk “silos”

Covered in typicalsecurity metrics scorecards

Requires meta-metrics


Page 34

[email protected]



Learning in Order to Solve “Mysteries”

Puzzles(closed ended)

Mysteries(open ended)

Documents

[email protected] Towards Incentive-based Cyber Trust Russell Cameron Thomas Principal, Meritology [email protected] Patrick Amon