System Assurance and the Internet of Things › news › meetings › tc › nj-13 › special... · System Assurance and the Internet of Things Dr. Ben Calloni, P.E. CISSP, CEH,

System Assurance and the Internet of Things

Dr. Ben Calloni, P.E. CISSP, CEH, OCRES Lockheed Martin Fellow, Software Security • Lockheed Martin Representative to OMG

• OMG Board of Directors • Co-chair OMG System Assurance Task Force

• Member of DHS-OSD Software Assurance Forum / WG’s • Member of Future Airborne Capabilities Environment (FACE) Consortia

• Conformance Business WG • Safety and Security Technical WG

• Lockheed Martin Representative to The Open Group, • Member Trusted Technology Forum

• SMU Adjunct Professor, System Security Engineering

©KDM Analytics slides are used with permission ©Lockheed Martin slides used with permission

Overview

• Introduction • Defining Assurance • Semi Formal Methodology and Process • Graphical Claims Evidence Arguments • Addressing Automated Threat Risk

Assessment • Concluding Remarks

2 September 25, 2013

© 2013 LOCKHEED MARTIN CORPORATION

SEAD / DEAD (Vietnam) F-4C Wild Weasel (Hunter) F-4D Strike (Killer)

I’ve flown at Mach 2 @ 50,000 ft and Supersonic at 180 feet below Sea Level!

My First Job! Rigorous Aviation Safety

Squadron Safety Officer (No True Accidents!)

Functional Test Pilot


Presenter

Presentation Notes

Photo of pilot was taken in 1974 at George AFB by Base Photographer for documentation of F-4 graduation. It is public domain. An accident is a specific, unpredictable, unusual and unintended external action which occurs in a particular time and place, with no apparent and deliberate cause but with marked effects. It implies a generally negative outcome which may have been avoided or prevented had circumstances leading up to the accident been recognized, and acted upon, prior to its occurrence.


So Called “Accidents”

• An accident is a specific, unpredictable, unusual and unintended external action which occurs in a particular time and place, with no apparent and deliberate cause but with marked effects. It implies a generally negative outcome which may have been avoided or prevented had circumstances leading up to the accident been recognized, and acted upon, prior to its occurrence.

Once mankind got beyond a belief in unalterable FATE, we started investigating and understanding

Causal Links!



Cause and effect aka. Causal Links

• England Butter Production and Old Maids − Uptake in farmers married, resulted in reduction of Beef

Production and butter − Assumed farmers spending less time in the fields at work due to

marital bliss!

Old Maids Cats Mice Honey

Bees Clover Cattle Keep Kill Pollinate Feeds Disturb

Safety / Security Engineering requires identifying all the potential Causal Links and applying mitigating

strategies to prevent undesirable outcomes! 5 September 25, 2013


The Truth!

6

Every engineering design will have tradeoff’s and residual risk!

Reducing that risk depends on assurance processes.

September 25, 2013


The Nature of Assurance

• Assurance provides the members of society a basis for believing certain assertions

• Assurance Processes provide the foundation for a belief system − They provide the reasoning and evidentiary artifacts that underpin

the belief system − Can be static or dynamic or both…

• Complexity increases the difficulty of constructing the belief

system − Assurance boundary gets very large − Facts are harder to determine and verify − Reasoning gets more convoluted − Opportunity for ambiguity increases



There are risks in using all “machines”

• In commercial products, some risks are only discovered due to User Stupidity: aka “Darwin Award Nominees”

• Actual Warning Labels − On a hair dryer - “Do not use in shower or while sleeping.” − On a hand-held massaging device - “Do not use while sleeping

or unconscious.” − On a cardboard sunshield that keeps the sun off the dashboard.

- “Do not drive with sunshield in place.” − On a can of self-defense pepper spray. - “May irritate eyes.” − On a hair curling iron. - “For external use only!”

8

However, even when used correctly, all consumers, users, customers assume an implied acceptance of the

residual risk! September 25, 2013

Security Definition

Cyber Vulnerability (CISSP BoK) 1. A flaw* (aka weakness) exists in the system 2. Attacker has access to the flaw, and 3. Attacker has capability to exploit the flaw

• Examples – Lack of security patches – Lack of current virus definitions – Software Bug – Lax physical security

Basic definition of Vulnerability • refers to the inability to withstand the effects of a hostile environment • open to attack or damage

Defenders can only control these!

*e.g. Buffer Overflow is still on SANS Top 25 (#3). Industry has known and discussed since 1988!


Current INFOSEC Software Assurance activities focus on Reducing flaws, weaknesses, or vulnerabilities by:

• Running Static / Dynamic Code Analysis tools (flaw remediation)

• Performing Penetration Testing (pathway discovery)

BUT……


Presented by NSA at OMG Meeting in DC, Mar 2012



Bugs, Flaws and Defects (from the “safety domain”)

• Bugs (sometimes called “flaws or weaknesses” in the security domain) – Occur at Implementation (lower) level – Only exist in code – Can often be fixed in a single line of code

• Flaws – Can exist at all levels (code / design / requirements) – Subtle problems that are instantiated in code, but are also present

(or missing) in the design – Design (or requirement) flaws may require a redesign that can

affect multiple areas in the system • Defects encompass both implementation (bugs) and design (flaws)

problems – May lie dormant for years and surface later in a fielded system – Give way to major consequences

Airbus Auto-land Incident (N-version programming) https://www.atsb.gov.au/media/24388/aair200705576_Prelim.pdf



Quantifying Risk is about Statistics

•Everyday Statistics −Your favorite baseball player is batting 300 −70% Chance of Rain vs. 10% chance

Should you take the Umbrella? Depends:

•Optimist (risk tolerant) •Pessimist (risk averse)


Presenter

Presentation Notes

Optimist: Glass is half full Pessimist: Glass is half empty Engineer: Solution that is over designed by a factor of 2!


15

ITS TIME TO PLAY…

September 25, 2013


A Game of Chance!


Presenter

Presentation Notes

Picture from MS Office provided graphics. Assume the jar was full of 6000 jellybeans: 1 black and 5999 white. You were blind folded and asked to pick one out. How many would be confident you’d pick out a white one? What if the stakes were for everything in your wallet? Everything you owned? Your life?


Event Chance this Year Car stolen 1 in 100 House catch fire 1 in 200 Die from Heart Disease 1 in 280 Die of Cancer 1 in 500

Die by Homicide 1 in 10,000 Die of Tuberculosis 1 in 200,000

Win a state lottery (not Megamillions) 1 in 1 million

Killed by lightning 1 in 1.4 million Killed by flood or tornado 1 in 2 million Killed in Hurricane 1 in 6 million Die in commercial plane crash 1 in 10 million

Statistical Probabilities of Occurrence

Die in Auto Collision 1 in 6000


Presenter

Presentation Notes

In a Car Alternatively 5999 in 6000 of living.

What is Assurance? • Assurance is the measure of confidence that the security features,

practices, procedures, and architecture of an information system accurately mediates and enforces the security policy. - CNSS 4009 IA Glossary

• Information Assurance (IA) are measures that protect and defend information and information systems by ensuring their availability, integrity, authentication, confidentiality, and non-repudiation. These measures include providing for restoration of information systems by incorporating protection, detection, and reaction capabilities - CNSS 4009 IA Glossary

• Safety Assurance (SfA) is providing confidence that acceptable risk for the safety of personnel, equipment, facilities, and the public during and from the performance of operations is being achieved. – FAA/NASA

• Software Assurance (SwA) is the justified confidence that the system functions as intended and is free of exploitable vulnerabilities, either intentionally or unintentionally designed or inserted as part of the system at any time during the life cycle. - CNSS 4009 IA Glossary


Presenter

Presentation Notes

Measure of confidence == trustiworthiness

What is Assurance? (2) • Mission Assurance (MA) is the ability of operators to achieve their

mission, continue critical processes, and protect people and assets in the face of internal and external attack (both physical and cyber), unforeseen environmental or operational changes, and system malfunctions. (See notes page for further description.) – MITRE Systems Engineering Guide

• System Assurance (SysA) is the planned and systematic set of engineering activities necessary to assure that products conform with all applicable system requirements for safety, security, reliability, availability, maintainability, standards, procedures, and regulations, to provide the user with acceptable confidence that the system behaves as intended in the expected operational context. – OMG SysA Task Force


Presenter

Presentation Notes

Systems Engineering for Mission Assurance is the art of engineering into systems: the capabilities for operators to be aware of different and changing adversarial strategies as well as environmental and system conditions, options and alternatives to accomplish a mission under different circumstances, tools to assess and balance advantages and risks of available response options and alternatives, and the ability to transition to a selected option while simultaneously continuing the mission. Systems engineering for mission assurance extends throughout the entire traditional acquisition life cycle, from concept development through deployment and beyond, to include supply chain considerations and field operations

Interrelationships of Assurance

Mission Assurance

Systems Assurance

(*The “-ilities”)

Safety Assurance


Software Assurance


Information Assurance

*The “-ilities” Reliability, Schedulability, Maintainability, Dependability, etc.


Delivering System Assurance in any Domain: Delivering System Predictability and Reducing Uncertainty

• Software Assurance (SwA) is 3 step process 1. Specify Assurance Case

• Supplier must make unambiguous bounded assurance claims about safety, security dependability, etc. of systems, product or services

2. Obtain Evidence for Assurance Case • Perform system assurance assessment to justify claims of meeting a set of

requirements through a structure of sub-claims, arguments, and supporting evidence

• Collecting Evidence and verifying claims’ compliance is complex and costly process

3. Use Assurance Case to calculate and mitigate risk • Examine non compliant claims and their evidence to calculate risk (measure

of confidence / trustworthiness) and identify course of actions to mitigate it • Each stakeholder will have own risk assessment metrics – e.g. security,

safety, liability, performance, compliance

Currently, SwA 3 step process is informal, subjective & manual


Presenter

Presentation Notes

The security-related goals for a software system are twofold. The establishment and preservation across the post-release lifespan and possible situations (e.g. stolen laptop) of emergent systems security properties related to Integrity Confidentiality Availability Authenticity Accountability and Non-repudiation The establishment and maintenance of an adequate assurance case

http://www-users.cs.york.ac.uk/~tpk/04AE-149.pdf

For hazards associated with warnings, the assumptions of [7] Section 3.4 associated with the requirement to present a warning when no equipment failure has occurred are carried forward. In particular, with respect to hazard 17 in section 5.7 [4] that for test operation, operating limits will need to be introduced to protect against the hazard, whilst further data is gathered to determine the extent of the problem.


My thanks to SysA TF colleague Prof. Tim Kelly









Summary of Challenges

• Key Challenges − Systematic coverage of the system weakness space

• A key step that feeds into the rest of the process – if not properly done, rest of the process is considered add-hock

− Reduce ambiguity associated with system weakness space • Often due to requirements and design gaps that includes coverage, definitions and impact

− Objective and cost-effective assurance process • Current assurance assessment approaches resist automation due to lack of traceability and

transparency between high level security policy/requirement and system artifacts that implements them

− Effective and systematic measurement of the risk • Today, the risk management process often does not consider assurance issues in an

integrated way, resulting in project stakeholders unknowingly accepting assurance risks that can have unintended and severe security issues

− Actionable tasks to achieve high confidence in system trustworthiness

Overcoming these challenges will enable automation, a key requirement to a cost-effective, comprehensive, and objective assurance process and effective

measure of trustworthiness


Assured Software

“Mitigating Supply Chain Risks requires an understanding and management of Suppliers’ Capabilities, Products and Services”

More comprehensive diagnostic capabilities and standards are needed to support processes and provide transparency for more informed decision-making for mitigating risks to the enterprise

- Joe Jarzombek, PMP, CSSLP Director for Software Assurance National Cyber Security Division, Homeland Security

• In 2005 Mitch Komaroff, (NII/DoD-CIO) and Ken Hong Fong on behalf of DoD approached OMG to address this via standards.

• In 2007 Mr. Jarzombek also engaged OMG on behalf of DHS


OMG Organization

Architecture Board

Domain TC Vertical

Platform TC Horizontal

Liaison SC Object & Reference Model SC Spec Mgt SC MDA Users’ SIG Process Metamodels SIG SOA SIG IPR SC Sustainability SIG Architecture Ecosystems SIG Business Architecture SIG

A & D PTF ADM PTF MARS PTF SysA PTF Agent PSIG Data Distribution PSIG Japan PSIG Korea PSIG Ontology PSIG Telecoms PSIG

BMI DTF C4I DTF Finance DTF Government DTF Healthcare DTF Life Sciences DTF Mfg Tech & Ind. Systems DTF Robotics DTF S/W Based Comm DTF Space DTF Crisis Mgmt DSIG Regulatory Compl. DSIG SDO DSIG Sys Eng DSIG

*Software Assurance SIG charted Feb 2006 through Dec 2008 *System Assurance Task Force chartered and began work in March 2009


OMG System Assurance Task Force (SysA TF)

• Strategy – Establish a common framework for analysis and exchange of

information related to system assurance and trustworthiness. This trustworthiness will assist in facilitating systems that better support Security, Safety, Software and Information Assurance

• Immediate focus of SysA TF is to complete work related to

– SwA Ecosystem - common framework for capturing, graphically presenting, and analyzing properties of system trustworthiness

• leverages and connects existing OMG / ISO specifications and identifies new specifications that need to be developed to complete framework

• provides integrated tooling environment for different tool types • architected to improve software system analysis and achieve

higher automation of risk analysis


Establishing Assurance and Trust C&A Fed by

Assurance Documents

Operational Environment

CONOPS DoDAF OV’s etc.

Software Fault Patterns (Formalized CWE’s and TOIF results)

Implementation

System Artifacts •Req. •Use Cases •Design •Data Flow Diagrams •STIGS •etc.

Architecture

UPDM*

*UML Profile for DODAF/MODAF

Threat Risk Assessment /

Hazard Analysis

Attack Patterns

NVDB (through SCAP)

Common Fact Model

(Evidence)

KDM Engine

Word PPT Excel

C&A Fed by

Assurance Models



Addressing the Challenges

• Addressing challenges through set of integrated standards − Define a semi-formal methodology to address weakness space coverage − Graphically capture claims and evidence (common facts) about a system − Graphically capture threat-risk assessment information about a system − Automate vulnerability path assessments − Specifications for a suite of integrated tools providing end-to-end solution

• No one tool or one vendor can provide solution to address identified challenges − Tools integration possible only through standards

• Set of standards are needed requiring tight integration between standards • Integration of standards require that they are based of the same technology and

they follow the rules of technical development

The only standard organization producing such interoperability standards is the OMG!



FORMALIZATION OF WEAKNESSES AND STANDARDIZATION

Semi Formal Methodology and Process


Confidence Analysis through Assurance Process: Reducing Uncertainty

September 25, 2013 30

• Assurance does not provide additional security services or safeguards. It serves to reduce the uncertainty associated with systematic identification of weak links and ultimately with vulnerabilities

– (e.g. Common Criteria – Security Assurance

Requirements vs. – Security Functional

Requirements)

• Product of System Assurance is justified confidence delivered in the form of an Assurance Case. The Assurance Case is formed by a clear chain of evidence to argument to assurance claims.

TYPES OF EVIDENCES FOR ASSURANCE CASE

Assurance claim

© 2013 KDM Analytics

Tiered approach to Security Assurance: FORSA


• FORSA1 (Fact Oriented Repeatable Security Assessment) methodology

• FORSA is a four tiered approach to

security assurance that provides a progressively higher level of assurance through fidelity and policy orientation

• Repeatable and systematic way to perform risk assessment and system analysis

• Provides guidance to validation and verification activities as they are performed against the system, as well as the process of risk assessment

1 System Assurance – Beyond Detecting Vulnerabilities – Nikolai Mansurov, Djenana Campara, Elsevier Inc, 2011, ISBN 978-0-12-381414-2

Code Vulnerability

analysis

Security Architecture

Analysis

Safeguard Efficiency and

Threat Identification

Security Controls

Traceability and Analysis


Presenter

Presentation Notes

The security assurance is the way to communicate the findings of risk analysis, system analysis, validation, and verification in a systematic, rational, clear, and convincing way for consumption of the stakeholders. The key to security assurance is management of knowledge about the system, its mission, and its environment. Security assurance, through the use of its components, presents a powerful way of modeling and assessing trustworthiness in a more formal way (degree of formalism varies depending on use of narrative arguments versus well-structured, formalized sub-claims as arguments).

FORSA: A Methodology supported by Standards


• FORSA is aligned with the OMG System Assurance Eco-system – Standards-based

• OMG ISO/IEC 19506 Knowledge Discovery Metamodel • OMG Structured Assurance Case Metamodel • Goal Structured Notation (GSN) • UPDM • SysML • OMG Semantics of Business Vocabularies and Rules • OMG Structured Metrics Metamodel • OMG Risk Metamodel (standardization in progress) • Common Fact Model (standardization in progress) • RDF • XML • MOF

– Influences standards – Part of the growing community built around standards

• FORSA facilitated protocols for knowledge interchange between producers and consumers in cybersecurity space

– Making it objective and repeatable


Assurance as Part of Systems and Software Engineering - System Lifecycle (ISO 15288:2008)


Stakeholder requirements

definition

Requirements analysis

Architectural design

Implementation

Integration

Transition

Operation and Maintenance

Valid

atio

n Disposal

Verif

icat

ion

*Risk management process Enterprise Processes

Agreement Processes

Project Management

Processes

Technical Processes

Initial Assurance* Agreement

Assurance* Plan

Assurance* Needs Assurance

Case

CONOPS & Security Policy

Threat & Risk

Analysis

Security Monitoring &

Incident Response

Preliminary Assessment

Full Assessment

*Assurance interpreted as: Safety, Security, FDA, etc.

*risk to cost / schedule



Graphical Claims Evidence Arguments (Common Facts)


Goal

G0 Top Claim

J0001 Justification

Justification in context of

is solved by

Strategy

S001 Strategy

Goal

G1 Claim

Goal

G2 Claim

Goal

G3 Claim

is solved by is solved by is solved by

Goal

G1.1 Claim

is solved by

is solved by

A0001 Assumptions

Assumption

in context of

C0001 Context

Context in context of

Cr0001 Assurance Criteria

Context

is solved by

Goal

G1.2 Claim

ER1.1 Evidence Reference

Solution

ER1.2 Evidence Reference

Solution

is solved by

in context of

Model

M001 Model

Reference

in context of

OMG’s Structured Assurance Case Metamodel


G1 System is acceptably

secure Goal

Establishing the Security Assurance Case


DIACAP? JAFAN? CC? ….

Example CC Assurance Levels

TCB ….

•UML •SysML •DoDAF

CG1.4 Concept of operations Context

CG1.5 Subject to declared

assumptions and limitations Context

CG1.1 Security criteria are defined

Context

CG1.2 Assessment scope is defined Context

CG1.3 Assessment rigor is defined Context

G2 All threats are identified and

adequately Mitigated Goal

… G3

Trusted Computing Base Components

Identified Goal

… S1 Argument based on end-to-end risk

mitigation analysis Strategy

M1 Integrated

system model

G4 All threats to the

system are identified

Goal

G5 Identified threats are adequately mitigated

Goal

G6 Residual risk is

acceptable Goal



Addressing Automated Threat Risk Assessment

(Common Facts)


Identifying the Threats G4

All threats to the system are identified

Goal

G4.1.1 All operational

activities of the system are identified

Goal

G4.1.2 All assets of the

system are identified Goal

G4.1.3 All undesired

events are identified

Goal

G4.1.4 All threat scenarios

are identified Goal

G4.1 All known risk factors related

to similar systems are identified

Goal

G4.2 All risk factors for the system are

systematically identified Goal

G4.3 Risk analysis team is

adequately experienced Goal

G4.1.5 All threat agents

are identified Goal


S2 Argument based on various confidence factors

affecting threat identification Strategy

Presenter

Presentation Notes

Added since PIRA review Example comes from documents of the Object Management Group’s public specifications on Claims Evidence Argument Metamodel.


Trusted Systems flow from Unified Requirements

• Product Functional Requirements − Specify the thing(s) the system is being built to do! − These Requirements define the capabilities to be implemented

• Security / Safety Functional Requirements − The functions which enforce the security policy or safety

critical requirements • Security / Safety Assurance Requirements.

− Define and document how well ALL Functional Requirements are implemented (a level of confidence)

39

All applicable requirements must be considered as part of the overall System Attributes!

September 25, 2013

© 2013 LOCKHEED MARTIN CORPORATION 40

Delivering tested code and systems that meet system functional requirements without assurance evidence is

a NON-DELIVERY!

September 25, 2013


Thank You!

• Dr. Ben Calloni, P.E., CISSP, CEH, OCRES − LM Fellow − Lockheed Martin Aeronautics Company − Fort Worth, TX − [email protected]

− AIAA Associate Fellow − Object Management Group Board of Directors

• OMG System Assurance Task Force Co-Chair − Member of DHS-OSD Software Assurance Forum / WG’s − Future Airborne Capabilities Environment consortium

• Member of Safety / Security Technical WG • Member of FACE Conformance Business WG

− Texas Tech University Distinguished Engineer − SMU Adjunct Professor, System Security Engineering


Backup


Process, People, documentation Evidence

Formalized Specifications

Executable Specifications

Software system Technical Evidence

Technical Controls

Security Policy Protection Profiles

CWE / SFP

System/ Software Assurance Ecosystem: The Formal Framework for System Assessments with Focus on Automation

Reports, Threat Risk Analysis, etc

Software System Artifacts Data Structures

Hardware Environment

Assurance Case Repository

- Formalized in SBVR vocabulary - Automated verification of claims against

evidence - Highly automated and sophisticated risk

assessments using transitive inter-evidence point relationships

Supported by the following standards: - ISO/IEC 15026-4:2012 Sys & SwE Assurance - ISO/TC 37 / OMG SBVR - OMG Structured Assurance Case Metamodel - Software Fault Patterns (Target 2013) - UML Security Policy Extensions (2014)

Tools Interoperability and Unified Reporting Environment

Software System / Architecture Evaluation Many integrated & highly automated tools to assist evaluators Claims and Evidence in Formal vocabulary Combination of tools and ISO/OMG standards Standardized SW System Representation In KDM* Iterative extraction and analysis for rules

*ISO/IEC 19506:2012

Process, People & Documentation Evaluation Environment Some point tools to assist evaluators but mainly manual work Claims in Formal SBVR vocabulary Evidence in Formal SBVR vocabulary Large scope requires large effort Large scale system of systems (DODAF) Supported by The Open Group’s UDEF*

https:/ / buildsecurityin.us-cert.gov/ swa/ ecosystem.html

Requirements / Design / DODAF

Artifacts

Process Docs & Artifacts

Physical / Admin

Controls
