22a Scenario Analysis - sandia.gov · •Develops details of realistic adversary attack plan –Specific, coordinated tasks and timeline for all attackers ... •Limited to threats

22 - Scenario Analysis

The Twenty-Seventh International Training CoursePage 1

22. Scenar io Ana lys i s

April 29 – May 18, 2018Albuquerque, New Mexico, USA

SAND2015-1984 TR

Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions ofSandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S. Department of Energy’s National Nuclear SecurityAdministration under contract DE-NA0003525.

Scenario Analysis

Learn ing Object ives

After completing this module, you should be able to:• State the purpose of scenario analysis in the context of

evaluating physical protection system (PPS) performance• Identify the four phases of the Scenario Analysis Process• Create adversary attack scenarios• Describe a process for selecting final attack scenarios

2



Scenario Analysis

IAEA Nuclear Secur i ty Ser ies 13 (NSS-13)• 5.9 Using the DBT, the operator should define credible scenarios by

which adversaries could carry out sabotage of nuclear facilities and nuclear materials

• 5.10 When defining scenarios, the operator should consider the location of the nuclear facility and all nuclear materials

• 5.11 Sabotage scenarios should consider external and/or insider adversaries who attempt to disperse nuclear material or to damage or interfere with equipment, systems, structure components or devices, including possible stand-off attack, consistent with the State’s threat assessment or DBT

• 5.12 The operator should design a PPS that is effective against the defined sabotage scenarios and complies with the required level of protection for the nuclear facility and nuclear material

3

Scenario Analysis

Rev iew: Two Methodo log ies Address D i f fe rent Aspects o f PPS Ef fec t iveness• Path Analysis

Does the PPS design adequately provide:• Timely detection?• Defense in depth?• Balanced protection?

• Scenario Analysis Does the PPS design provide the required level of protection

against an adversary attack (scenario) consistent with the design basis threat?

• This module focuses on scenario analysis

4



Scenario Analysis

What Is Scenar io Analys is?

Scenario Analysis: A methodology for analyzing PPS effectiveness (PE) by considering several possible adversary scenarios

Allows more detailed analysis of the attack, defense, and results of path analysis

• Path analysis can be used to help determine the scenarios to be analyzed

Focuses on identifying vulnerabilities Contributes to

• Overall PPS design• Contingency plans• Policies and procedures• Interagency coordination

5

Scenario Analysis

How is Scenar io Analys is Used?• Provides basis for level of confidence about PPS

performance• Helps to create robust security plans that match and fully

use the capabilities of the PPS design How?

• Develops details of realistic adversary attack plan– Specific, coordinated tasks and timeline for all attackers

• Develops detailed characterization of how PPS and response should behave, based on performance testing and site plans

• Simulates how PPS and response behave during attempted adversary attack scenario

6

IMPORTANT: Overall PPS effectiveness is represented by effectiveness against a few specific scenarios – in scenario analysis, there is no attempt to determine a worst‐case scenario



Scenario Analysis

Scope of Scenar io Analys is

• Scope of scenarios is based on specific site objectives The scope should cover key questions identified by site

stakeholders, e.g.,• General analysis

– How effective is the current PPS?– How effective is the existing response force strategy?

• Specific analysis– How effective is a specific procedure?– How effective is a potential upgrade?

The scope should reflect major site considerations • What PPS configuration should be tested?• What are the threat numbers and capabilities?

7

Scenario Analysis

How To Work with Scenar ios?• Red Team generally used as experts• First, design and develop

Design based on objectives by stakeholders, under scoping agreement After determining attack scenario characteristics, team develops

scenarios, reviews them, and selects scenarios to be evaluated• Then, implement and evaluate

After preparation, team simulates attack and records results

• Various simulation methods Team analyzes results,

determines vulnerabilities, and recommends upgrades

• This module focuses on design and development phases Implementation and evaluation phases will be covered in next module

8



Scenario Analysis

Des ign Phase: Ident i fy Stakeholders

• Identify people who are responsible for the design, implementation, evaluation, and risk acceptance of the PPS Competent authority Response force management Vulnerability analysis team Adversary planning subject-matter experts (SMEs) Security management Facility operations Offsite response Other required people

9

Scenario Analysis

Develop Scoping AgreementScoping Agreement: A contract among appropriate stakeholders that identifies parameters of scenario analysis

Defines requirements Includes design basis threat (DBT) statement Characterizes facility Identifies

• Targets (type of targets); scenario assumptions• Simulation tools and the process for using them; credible SMEs for

attack planning Determines

• Types of attacks (sabotage/theft) and numbers of scenarios • Type of insider (passive/active, etc.)• Picture-in-time

These criteria are used in design and during scenario selection10



Scenario Analysis

Oversee Scenar io Development

• Stakeholder(s) familiar with the design and evaluation of the PPS should be included in scenario development

• All participants should: Agree to confidentiality of all site / adversary information Remain unbiased to site or adversaries Ensure the adversary scenarios are within the parameters

of the scoping agreement Ensure accuracy of the PPS and target information

• Thickness of vault walls• Assessment capability• Response capability

11

Scenario Analysis

Deve lopment Phase: Determine Scenar io Character i s t i csAttack Scenario: A time ordered, detailed description of an adversary attack used in analyzing PE

For scenario analysis to be of maximum value, scenarios should be:

• Detailed• Credible• Limited to threats within the DBT• Well documented

Consider scenarios from Path Analysis • Add scenario details to these paths• Add supporting team plans to assist the attackers• IMPORTANT: The most vulnerable PI path from Path Analysis may

be a poor basis for a scenario

12



Scenario Analysis

Adversary Strategy

Adversary Strategy: Short description of the scenario used to achieve the adversary's objective• Two classes of adversary strategies

Direct: Adversary follows a direct path to target• Adversary goal: Minimize PI by defeating system detection or delay

elements Indirect: Adversary attacks PPS infrastructure before attacking

target• Adversary goal: Minimize PI or PN to:

– Increase response time– Decrease response numbers– Disable critical systems

13

Scenario Analysis

Defeat Strategies and Methods

Defeat Strategy: General approach used to defeat a path element or a PPS functionDefeat Method: Way to prevent a component within a path element from accomplishing its purpose or function• Three basic adversary defeat strategies / methods

1. Avoid, degrade, or disable detection systems• Include entry control and contraband detection systems

2. Degrade, disable, or circumvent delay systems

3. Degrade or eliminate response• Identify

– Weak links– Single points of failure

14



Scenario Analysis

Scenar io P lanning and Complex i ty Factors

• The best attack scenario for the adversary does not always use all of the equipment allowed within the design basis threat Not all of the equipment will provide an advantage to the

adversary Adding equipment may increase the complexity of the attack

scenario • Coordinating actions and synchronizing time between

groups increases difficulty

15

Scenario Analysis

Adversary ’s Perspect ive for Ma in and Suppor t ing Teams

Detection Time

AdversaryBegins Task Adversary Completes

Task

Time

Adversary Task Time

CT

FirstSensing

T0

Ad

vers

ary

Det

ecte

d

DT

Response Force Time A

dve

rsar

yIn

terr

up

ted

TI

PPS Response Time

Sensing Opportunities

Time Remaining

After Interruption

CumulativeP(Detection)

Adversary wants to control PD

Adversary also wants to control the time of engagement, TC < TI

16



Scenario Analysis

C reate Range o f Scenar ios by means o f S t ruc tured Approach

• Identify site vulnerabilities• Build scenarios to exploit the identified site vulnerabilities • Review and select final scenarios based on criteria

17

Scenario Analysis

Ident i fy S i te Vulnerabi l i t ies • Collect site-specific PPS data

Passive insider information Site surveillance Outside sources (Internet, libraries, etc.)

• Identify site vulnerabilities across various operational conditions and states Operational conditions (operational versus non-operational) Target material configurations (reactor refueling versus operations) Response force alert levels

• Identify sources of vulnerabilities Experts (site personnel, police) Path analysis results Previous vulnerability studies and performance tests

18



Scenario Analysis

Example: Ident i fy S i te Vulnerabi l i t ies• Vulnerability: At times the guard force is divided• Scenario conditions:

2 guards at entry portal 3 guards at guard house

• Results from Path Analysis Adversaries = 5 (DBT) Guard Force = 7

• Expected results from vulnerability-based scenario Task Plan A:

• 3 Adversaries v. 2 Guards• Surprise advantage to Adversaries

Task Plan B:• 3 Adversaries v. 3 Guards• Surprise advantage to adversaries 19

Guardhouse

Entry Portal

Scenario Analysis

Exp lo i t Ident i f i ed S i te Vu lnerab i l i t i es : Task P lans• Determine how an adversary could exploit identified

vulnerabilities at site• Create a list of essential tasks that must be accomplished

for the attack to succeed• Create task plans describing how an adversary team can

perform each task within resource constraints Who is involved? What are they doing as a function of time? How are they performing each step? What equipment are they using? How are they transporting the equipment?

20



Scenario Analysis

Example: Deve lop Task P lan to Exp lo i t S i te Vu lnerab i l i t i es

• Example Vulnerability: At times the response

force is divided List of tasks to exploit vulnerability

A. Ambush Guard PostB. Attack remaining Response ForceC. Enter Material Storage Building

and remove materialD. Escape to safe house

Guardhouse

Entry Portal

21

Scenario Analysis

Task P lan AStart Time Activity End

Time

00:00 A1, A2, and A3 drive vehicle up to the gate 00: 40

00:40 A1 waits until P1 and P2 arrive at vehicle 00:50

00:50 A1 engages P1 while A2 and A3 exit vehicle 00:55

00:55 A2 and A3 engage P2 01:00

01:00 A2 and A3 breach gate 01:30

01:30 A1 drives vehicle through gate and picks up A2 and A3 01:45

22



Scenario Analysis

Task P lan BStart Time Activity End

Time

01:45 A1 drives to guardhouse and A1, A2, and A3 dismount 03:00

03:00 A1, A2 and A3 surround guardhouse and wait for guards to exit 03:30

03:30 Adversary team engages guards in guardhouse 04:00

23

Scenario Analysis

Exp lo i t Ident i f i ed S i te Vu lnerab i l i t i es : Bu i ld Scenar io(s)• Combine task plans into a master attack plan / scenario

description, adjusting task activities to: Meet DBT and other constraints Determine how adversary team moves from offsite to target Achieve synchronization between teams Coordinate progress at key steps (e.g., the point of detection) Refine task time estimates Identify key locations for chance encounters with security or site

personnel Consider ambushes and diversions as ways of delaying /

defeating the guards and response force Identify:

• Target selection, minimum delay path, and breaching techniques24



Scenario Analysis

Rev iew and Select F ina l Scenar ios

• Include stakeholders in the review and selection process• Review and select final scenarios based on scoping

agreement criteria Are all analysis objectives covered?

• Are conditions and states covered adequately?• Do the scenarios address several means of adversary approach (on

foot, in land vehicles, on water, or by air), when applicable, based on the DBT?

Are scenarios credible, limited by threats within the DBT, etc.?

25

Scenario Analysis

Rev iew and Select F ina l Scenar ios (cont ’d)

• Consider impact of colluding insider Modify appropriate detection, delay, response force time, or

response force numbers to reflect what insider can accomplish Examples of collusion scenarios

• Detection: Insider tampers with alarm communication lines• Delay: Insider opens vault door at time of attack• Response:

– Insider activates an emergency alarm in a different location to divert response force

– Insider detonates explosive at armory

26



Scenario Analysis

Scenar io Ana lys i s : Implementat ion and Eva luat ion Phases

• Teams simulate the attack and record events, using one or more methods Tabletop exercise Computer combat simulation Force-on-Force (FoF) exercises

• This course uses the Tabletop method How to implement and evaluate scenarios using Tabletops is

presented in the following module

27

Scenario Analysis

Key Takeaways• Scenario analysis is an evaluation tool

Can be used to evaluate whether PPS design provides required level of protection against an attack consistent with DBT

Allows more detailed evaluation than a path analysis in terms of attack methods and response

Focuses on identifying vulnerabilities• Design scenarios based on needs of stakeholders• Develop scenarios to address site-specific objectives• Use structured approach to create range of scenarios

Identify site vulnerabilities Build scenarios to exploit the identified site vulnerabilities Review and select final scenarios based on criteria

28

Documents

22a Scenario Analysis - sandia.gov · •Develops details of realistic adversary attack plan –Specific, coordinated tasks and timeline for all attackers ... •Limited to threats