1

The Value of Threat Modelling1

Authors Tim Williams MSc (Royal Holloway, 2014) Lorenzo Cavallaro, ISG, Royal Holloway

Summary

Threat Modelling is an umbrella term covering a variety of powerful techniques for understanding the underlying causes of risks, enabling more effective risk management solutions to be designed and implemented. Every possible risk is caused by one or more underlying threats, and identifying and understanding threats enables earlier and more complete risk mitigation. As yet there is no generally accepted approach to Threat Modelling. However the shared aim of most techniques is to facilitate rapid, cost-effective exploration of leading indicators of future risks, allowing appropriate risk mitigation resources to be assigned in a timely manner. In general Threat Modelling should be performed actively throughout the development lifecycle of systems and software. Since it is impossible to predict all threats, even the best threat models are subject to errors and omissions. A well-managed Threat Modelling process ensures that analysis of actual incidents and live threat intelligence feeds are used continuously to refine Threat Models. This article gives an overview of the value of Threat Modelling and describes some common modelling techniques.

What is Threat Modelling?

Threat modelling is a valuable component of enterprise risk management. It focuses on developing a shared understanding of risks in terms of their underlying causes, the nature of possible adverse events caused by identified threats, the most likely intentional attack vectors, the extent to which threat exposures and intentional motivations are controllable, and the ways in which the probability and impact of adverse events can be reduced.

What is a Threat? A threat is a risk factor – someone or something that can cause a risk or increase an existing risk. Most threats from people/organisations are intentional. Most threats from things/environments are unintentional. Threats may also arise from human negligence. Some intentional and/or negligent human threats agents can be influenced to cause fewer risks, so some very effective risk mitigations involve influencing the motivation and behaviour of human threat agents. Threats which could result in risks to life, health, property or the environment may be described as hazards. Hazards may have intentional, negligent or unintentional causes.

1 This article is to be published online by Computer Weekly as part of the 2015 Royal Holloway info security thesis series. The full MSc thesis is published on the ISG’s website.

2

Many organisations such as the Open Web Applications Security Project (OWASP), MITRE, Intel, Microsoft and the Web Application Security Consortium (WASC) all recommend Threat Modelling.

What is a Threat Model? A threat model is a representation of threats which aids identification and understanding of the causes of risks, their potential consequences (impacts) and the probable effectiveness of various interventions to mitigate risks.

Threat models are, in effect, abstract “design descriptions” of “risk factories” where risks are generated. It is easier to understand how risks arise if the root causes are clearly described using design artefacts such as clear diagrams, tables and supporting notes. This table identifies generic examples of threats which may need to be modelled:

Threat Origin

Internal External

Threat Cause

Intentional / Active

Employee Contractor Authoriser Privileged User Systems Administrator

Competitors Suppliers Customers Journalists Hackers Organised Criminals Terrorists Governments

Unintentional / Passive

Human Errors Incorrect requirements Process Design Errors Security Design Errors Implementation Errors Operational Weaknesses Unidentified Failure Modes

False positive results from positive security testing

False negative results from negative security testing

Unreliable components: - Hardware defects - Software defects Unreliable services: - Power failures - Network failures Environmental Disasters - Sunspots/Radiation - Fire - Flood/Tsunami - Hurricane/Tornado - Earthquake - Nuclear/Biological contamination

Why is Threat Modelling so important?

Developing and reviewing threat models makes it easier for stakeholders to understand the causes of risks and what should be done to mitigate them. Since every threat is potentially the cause of multiple risks, focusing risk mitigation resources on threats/causes is far more efficient than focusing on particular risk instances. Threat Modelling enables:

1. the root causes of risks to be more fully understood; and 2. risk mitigation resources to be applied to best effect.

3

How do Threats and Controls Interact? Each security control added to reduce identified risks also increases the potential “attack surface” for intentional threats and introduces potential new unintentional “failure modes”. Residual risks to assets are the end result of interactions between intentional threats, unintentional threats and controls. Without understanding threat/control interactions, risks cannot be assessed early, completely and accurately.

Preparing and reviewing threat models from the outset of any IT project maximises the chances of identifying and mitigating threats before any risks have occurred.

How do Threats relate to Risks? Intentional threats lead to risks if related controls are inadequate i.e. if there are exploitable vulnerabilities. Modelling intentional threats helps to distinguish exploitable vulnerabilities from vulnerabilities which are already adequately protected. Some vulnerabilities can lead to risks without any involvement of intentional threat actors. For example a system component might fail in normal use. The impact could be temporary loss of system availability, permanent loss of information or uncontrolled release of sensitive information. When performing Threat Modelling it is important to ensure that potential unintentional failure modes are analysed both separately and in terms of how they interact with identified intentional threats.

What Threat Modelling Techniques Exist?

Different techniques are needed to explore and illustrate the causes and impacts of threats according to context. To handle differences in level of detail, different models may be needed at organisation, division, business process, system and component level. Similarly according to lifecycle stages, early exploration of threats at the requirements analysis stage requires different Threat Modelling techniques from those needed to support detailed pre go-live testing. The required level of fidelity (degree of exactness) also varies: in order to represent and explore threats for large and complex systems, it may be necessary to instantiate dynamic threat models using software and hardware rather than simply using static analysis techniques. Other reasons why different techniques are needed are due to differences in Threat Origin (internal/external) and Threat Cause (intentional/negligent/unintentional).

The majority of Threat Modelling approaches include:

architecture analysis focusing on data assets, data owners and controls;

graphical presentation formats supported by text, to promote stakeholder involvement in construction and validation of threat models; and

suggested groupings for similar items (threats, assets, vulnerabilities, controls etc) to simplify analysis.

4

Typically it is necessary to prepare and review more than one threat model. Common information presentation formats used in Threat Modelling include:

Attack Chains / Kill Chains – summarising generic strategies/phases typically used for attacking and defending enterprise information assets

Use and Abuse Cases – identifying system functions which may be used/misused

Attack Trees – enumerating possible attack methods against a defined target

Data Flow Diagrams – highlighting where data exists, where it crosses boundaries between security zones and how it may be attacked

Fault Trees – enumerating for a system possible unintentional failure modes

Cyber Threat Laboratories – simulating realistic threats without risking live data

Threat Matrices / Tables – structured, standardised analysis of threats and mitigations, which may have originally been identified using another technique.

Attack Chains / Kill Chains Attack Chain and Kill Chain diagrams improve understanding of security management issues at a strategic level. Chain diagrams make it very clear that intentional attackers typically follow a systematic process. Senior decision makers are typically quick to understand the significance of processes and how they can be controlled. Chain diagrams also demonstrate the need for a "Defence in Depth" approach: a coordinated set of complementary independent security controls is needed to counter multi-stage attacks.

5

Use and Abuse Cases Use Case and Abuse Case diagrams (in combination) are simpler to understand and easier to review than many other formats. They allow very early identification of generic attack types which are likely to be relevant. Use and Abuse Case diagrams can and should be first used at the requirements gathering stage before any system components have actually been designed. They allow rapid exploration of interactions between the expected behaviour of system components and authorised users, and possible accidental behaviours of authorised users and unauthorised actions, by insiders or by external attackers.

6

Attack Trees Attack Tree diagrams highlight the existence of multiple possible attack targets and attack techniques. Representing many attack options on a single diagram aids understanding of the wide variety of options open to intentional attackers and facilitates analysis of which attacks would have the highest benefits for the lowest time/effort/money and are therefore likely. Attack Trees also enable analysis of the most cost-effective ways to detect and/or block multiple attack paths simultaneously. Attack Trees may also allow identification of ways in which attackers could be demotivated from performing certain attacks (e.g. using legal warnings or disinformation) or in which attackers could be caused to waste resources/effort (e.g. honeypots, sandboxes and bogus responses to footprinting attempts).

7

Data Flow Diagrams Data Flow diagrams highlight key points in business processes and systems where threats to confidentiality, integrity or availability could exist. Data Flow diagrams come into play both when business processes and systems are being designed and tested and later during operational reviews of live processes and systems. They are also useful for clarifying in detail how attackers could potentially combine multiple attacks on multiple vulnerabilities to achieve their overall objectives. Whereas Attack Trees and Kill Chains identify high level attack and defence strategies, Data Flow diagrams identify detailed concrete ways in which specific attacks could potentially be blocked.

The Importance of Security Zoning All organisations should include appropriate security zones in their overall security architecture. Security zones (also called security compartments) are groups of information assets protected by an effective common security boundary/barrier. Security barriers around information assets may be enforced using logical security controls (e.g. data classification labels, firewall rules, password authentication and encryption) and/or physical/procedural security controls (e.g. site access controls, locked cabinets and restrictions on mobile phones). The important benefit of security zones is that, just like watertight compartments on a ship protect against catastrophic flooding, they prevent the spread of threats. Security zones are needed to prevent both external (generally intentional) and internal (generally unintentional) threats from spreading.

8

Data flow diagrams are a good way of showing where security zones exist, what security controls are being used to enforce barriers around zones and where zone restrictions are inadequate. In the example data flow diagram, it should be obvious that the administrator password needs to be better protected outside the Corporate Security zone. Two factor authentication, typically using one-time codes generated by a small hardware device, helps to limit risks related to password sniffing attacks.

Fault Trees Fault Tree diagrams highlight single points of failure and support identification of potential unintentional threats of failures at business process, system, subsystem and component levels. They can also be used to identify faults which might affect other Threat Models.

Cyber Test Laboratories The main advantage of using Cyber Test Laboratories (also called "Cyber Ranges" or "War Gaming" environments) is that synthetic test environments deliver a greater degree of fidelity and granularity than other threat modelling techniques. Greater realism in threat models, albeit more expensive and time-consuming to achieve, helps to accelerate technical learning about attack techniques and mitigations. Further advantages of Cyber Test Laboratories are that they:

◦ enable empirical exploration of threats, impacts and the effectiveness (or not) of various risk mitigation techniques;

9

◦ allow part or all of a live environment to be tested without risks to real data;

◦ have ongoing value in supporting production security incident response, incident recovery and investigation processes after systems have gone live.

How does Threat Modelling relate to Testing? Most Threat Modelling can be considered to be a form of testing known as “static analysis” which quickly exposes actionable results. The cheapest and most effective testing is to learn from the mistakes which have already occurred elsewhere. Empirically reproducing known errors, except for teaching purposes, is not a good use of limited test resources. However a Cyber Test Laboratory that allows realistic dynamic analysis of the security of complete systems before they are exposed to threats is an invaluable organisational asset.

Threat Matrices / Tables Threat Matrices / Tables enable known relationships between threats, motivations, capabilities, compromise methods, impacts and controls (which may have been identified using other Threat Modelling techniques) to be recorded, summarised and tracked in a common format. They also:

promote a structured approach to threat analysis;

can be developed and reviewed easily;

integrate well with other tabular project management and financial control techniques;

do not require any specialised tools or infrastructure.

10

Whichever Threat Modelling technique (or combination of techniques) is adopted, what is important and valuable about them is that they:

can deliver high return on invested effort/time before any expenditure on design/implementation;

aid identification of threat sources, threat targets and potential impacts;

support understanding of root causes of risks;

enable timely and accurate selection and implementation of appropriate mitigations;

generate sensitive information which needs to be adequately protected. It is important that only trustworthy people are allowed to participate in threat modelling activities and that unauthorised attempts to access and/or change threat model information are detectable. Otherwise the loss of threat model information or unauthorised alteration of threat model information could have adverse long-term impacts.

Protecting Threat Model Inputs and Outputs against Threats All Threat Modelling activities should ideally be performed within a security “enclave” i.e. a location which is both physically and logically well protected.

11

What is Threat Intelligence?

Threat Intelligence is information about new and changing threats, including common attack patterns. Threat Intelligence may be provided in semi-structured (human readable) and/or highly structured (machine processable) formats. Threat intelligence may come from external sources including trusted government organisations such as MITRE who publish useful Threat Modelling resources such as the Common Attack Pattern Enumeration and Classification (CAPEC™) taxonomy without charge. A number of companies also specialise in providing threat intelligence information feeds as a commercial service. However in many situations, the best sources of threat intelligence are experienced and well-motivated insiders. People who really understand the organisation's processes and systems know what security controls exist and are often aware of ways in which security controls can be defeated or bypassed. Such internal stakeholders often prove the most valuable sources for actionable threat intelligence.

Where to begin with Threat Modelling? A good starting point when developing threat models is to assemble a diverse group of trusted insiders in a workshop format meeting with experienced security consultants acting as facilitators. Key members of the organisation's leadership should be represented to make it clear that the activity of threat modelling is important and officially supported. Before attempting to model any actual threats, it can be helpful to perform a number of “ice-breaker” threat modelling practice activities. For example, workshop participants can try to develop a threat model for a particular subject (such as an email account or personal medical records) using a particular presentation format. Scenarios may also be modified, e.g. assuming that the subject of the threat modelling activity belongs to a family member rather than to a stranger. The aim of initial informal threat modelling practice activities should be to stimulate confidence in the workshop participants that:

the activity of threat modelling is not difficult;

threat modelling can quickly identify novel attacks and mitigation opportunities.

How to develop a Threat Modelling culture?

To maximise the effectiveness of Threat Modelling, it needs to be integrated into the organisation's culture. The value of Threat Modelling is maximised when it takes full account of the enterprise context and is conducted with an appropriate balance of people, process and technology. There is no single approach to Threat Modelling which can fully deal with the vast range of different organisational, process and technical requirements. However threat modelling activities are most likely to be successful when they are well integrated with closely related activities such as: Strategic Planning; Business Process Design/Engineering; Business Continuity and Disaster Recovery planning; Operational Risk management; Information Technology Service Management; and Information Security management.

The thesis on which this article is based contains a table summarising over 30 different Threat Modelling approaches in terms of their focus on:

12

Attackers – including the identity, motivation and capability of attackers and analysis of common attack methods;

Requirements – formal derivation of security protection needs from threat models;

Design - including developing models which analyse technical vulnerabilities;

Testing – including using threat models to improve the relevance and accuracy of positive and negative security tests;

Operations – threat models which emphasise business information risks over technology risks;

Impacts – including analysis of both negative impacts of threats and the effectiveness of defences against threats.

This can provide a starting point for analysing an organisation’s needs for Threat Modelling.

Conclusions

Threat modelling enables earlier identification of risks than is possible without the use of threat models. If organisations choose not to use Threat Modelling techniques, risks will still arise - but without any advance notice. Understanding the causes of risks well before they impact is a prerequisite for risk avoidance. Considering threats before risks occur also helps to identify adverse events which it would be difficult or time-consuming to recover from. Only by understanding the causes of risks is it possible to mitigate risks in a timely and cost-effective manner. More details about threat modelling in general and about how threat models can be used to guide security testing of enterprise database systems and services in particular can be found in the full thesis itself on https://www.ma.rhul.ac.uk/tech. Biographies Tim Williams is an independent security consultant and part-time security researcher specialising in security architecture, software security and security testing. Over 25 years of varied work experience has exposed Tim to business and security issues in a range of industry sectors. Tim has been a member of the CESG Listed Advisers Scheme (CLAS) since 2009 and holds a number of other professional memberships and certifications. Since 2011, Tim has been an active volunteer for CESG as a member of CLAS Policy and Tools Working Group and for (ISC)2 as a member of the examination boards for their CISSP®, ISSEP®, ISSAP®, CCSP® and HCISPP® professional certifications. Tim is also the Events Coordinator for the recently-formed Thames Valley Chapter of (ISC)2. Dr Lorenzo Cavallaro is a Senior Lecturer of Information Security in the Information Security Group (ISG) at Royal Holloway University of London. His research focuses largely on systems security. He has founded and is leading the recently-established Systems Security Research Lab (S2Lab) within the ISG, which focuses on devising novel techniques to protect systems from a broad range of threats, including those perpetrated by malicious software. In particular, Lorenzo's lab aims ultimately at building practical tools and provide security services to the community at large. In the past, Lorenzo was lucky enough to have the chance to work with a number of well-established groups (e.g., G. Vigna and C. Kruegel at UCSB, A. S. Tanenbaum and H. Bos at Vrije Universiteit, R. Sekar at Stony Brook University) during his PostDocs and visiting PhD periods. He is currently PI and co-I on a number of research projects funded by EPSRC and EU FP7, publishing in

13

top and well-known venues and serving as program committee member for well-known conferences and workshops.