www.thales-esecurity.comThales Open

Cyber Security of Complex Systems that Matter: a view from the IT and Defence Industry:

Peter DaviesDirector Security Concepts

2Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Who am I, Where do I Come from (why should I Listen)?

Thales is a leading global provider of data protection and cyber solutions with more than 40 years’ experience

securing the world’s most sensitive information. Our customers — businesses, governments, and technology vendors

with a broad range of challenges — use Thales products and services to improve the security of applications that rely

on encryption and digital signatures. By protecting the confidentiality, integrity, and availability of sensitive information

that flows through today’s traditional, virtualized, and cloud-based infrastructures, Thales is helping organizations

reduce risk, demonstrate compliance, enhance agility, and pursue strategic goals with greater confidence

◼ I am

◼ A Security Expert

◼ Specialised in the convergence of Safety and Security

◼ Leading Expert on

◼ Countering Cyber Attacks targeted Supply Chain Infiltration

◼ Cyber Physical Attacks

◼ Lead 2 Cyber Security aspects of C-CAV research activities

◼ Chair the AESIN Security Workgroup

◼ 30+ years of verifying security systems in hardware and software

◼ I do security where it can’t afford to fail

◼ I advise organisations on their legal positionhttps://www.riscs.org.uk/2018/02/15/peter-davies-forward-security-for-emerging-problems/

3Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

What I would like us to talk about today

1. Cyber Security as an emergent property

2. What does a legally sustainable, through life, cyber argument look like in

the context of a safety case and what are the tools that we need in making

that argument?

3. What does type approval mean in the future?

4. How do we know that we have done enough and if we know that what

effect will that have on incubators / funding for cyber components?

5. How might the quality of your approach to Cyber affect access to capital.

6. How do we know that we have done enough and if we know that then what

effect will that have on access to funding for the things that we need to

improve cyber resilience?

4Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

What is an Automotive System?

Who’s the defendant, liable, the plaintiff and what court and where?

5Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

What is an Automotive System?

6Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

▪ Connected and Autonomous Cars are part of a Complex, Hyper-connected, bottom up system with emergent properties for which there is no guiding mind.

▪ A system yielding its benefits at scale▪ It is a price sensitive, worldwide and mobile system with vast amounts of data.▪ Owned by no one but in it both strict and contract liability apply and must coexist.▪ Multi-vendor with legal obligations not to exclude suppliers from the supply chain.▪ It is increasingly integrated with global information and management networks

▪ Intertwined and interdependent components which interact▪ Adaptive behaviour according to history or feedback▪ Self-organisation▪ Emergence which is not always predictable, centrally controlled or engineered▪ Constantly changes, appearing dispositional and lacking causality▪ Extreme, ‘cascading’ behaviour, power-laws can be observed – minor input

changes can result in major output changes

What is an Automotive System?

7Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

▪ Connected and Autonomous Cars are part of a Complex, Hyper-connected, bottom up system with emergent properties for which there is no guiding mind.

▪ Cyber attacks systematically downgrade the strength of mechanisms whilst at the same time changing the probability of exposure and controllability of harm.

▪ No option to control the global attack surface and many traditional security techniques will in fact worsen the ability of the system to defend itself.

▪ Cyber attacks against safety systems that must operate at scale will often seek merely to trigger a safety reaction knowing that given the complexity of the system these secondary functions are likely to have been far less well analysed than the primary functions.

▪ These attacks in triggering our own defences create situations where, in its weakened and far less well analysed state, the system will be unable to resist and will often be very vulnerable to second order infections.

Cyber Attacks Against The Automotive System.

8Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Cardiac Pacemakers found

vulnerable to cyber attack

• Pacing at dangerously high rate• Battery drain attack• Randomly directed within 50ft

radius• Can be executed at very large

scale

Resulted in investor ‘shorting’ the manufacturer

Manufacturer is no longer

trading

Cyber Resilience …

A lack of Cyber Resilience now threatens company integrity and survival

What has Changed ?

Example 2

Vehicles now target of

White Hat attacks

US Securities & Exchange

Commission Prosecution

• Company issued bonds achieved attractive rates, when

• Executives did not disclose known product compliance issues

Company and executives chargedwith• Making false statements• Defrauding investors

Will Cyber Resilience equate to

Product Compliance ?

9Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

The Problem: Breaking the Brakes …

This was true for Fluid based Electromechanical systems

• The design rationale associated with Braking continues to call up ASIL-D;

• Implying simplicity, replication and zero to small numbers of lines of code

Braking used to be ‘Simple’

ASIL D, the highest classification of initial hazard (injury risk)

defined in ISO 26262 (Road vehicles—Functional safety)

represents likely potential for severely life-threatening or

fatal injury in the event of malfunction

10Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

The Problem: Breaking the Brakes …

Without direct connection between controls and function, our assumption of ASIL-D becomes questionable – even before malevolent attacks are considered

The evolving Functional Braking System :

• ABS at City & Highway Speeds - Individual wheel braking, acceleration & steering

• Multiple sensors, often augmented by machine learning

• Data Fusion & algorithmic arbitration to optimise system

• Connected over a shared network infrastructure

• 10→23 Sensors; 1.5→3.5 million Lines-of-Code; Training Data sets …

Braking has become Digital and ‘Complicated’

11Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

The Problem: Breaking the Brakes …

Without direct connection between controls and function, our assumption of ASIL-D becomes questionable – even before malevolent attacks are considered

Cyber Attacks Against Braking Systems :

• Contradictions that may arise in the data stream

• Cycles (DoS)

• Non Determinism – Arbitration between Complex Algorithms

• Transition Analog / Digital

• Error Correction as input to ML

• Attack Detection & Attack Management (Function

and Control)

Braking has become Digital and ‘Complicated’

12Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

“A System is Resilient if, and only if, there is justifiable and enduring

confidence that it will function as expected, when expected”

Cyber Resilience …

➢ It is Secure if it displays this property in the face of an Adversary;

➢ It is Cyber Secure if it displays this property in the face of an Adversary that is not co-located.

13Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Conflicting Objectives and Responsibility …

Cyber resilience is a board level responsibility with company integrity at stakeMost C-level executives share in the consequences of a breach

SAFETYGoal: Meet regulatory,

public & media expectation

Accountability: CSO & COO

Authority: Criminal Courts BUSINESS

VALUEGoal : Profitability

Accountability: CEO & CFO

Authority : Criminal Courts

PRIVACY

Goal: Data Protection

Accountability: CIO & CDO

Authority: Information

Commissioner

INTEGRITY

14Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Agreeing the Design Limit for Safe Operation, and the Mitigation when Unsafe,are the new Sign-Off and Certification judgements

Certification requires a ‘Sustainable Argument’ that gives ‘Justifiable Confidence’of a ‘Good Outcome’ in the face of an Emerging System Failure

Design Limit

covers a

‘Reasonable’

operating area

governed by

‘Well-Founded’

theories

Known Capability in Safe Operating

Area

Plan A

Plan B

Mitigation Plans

with

‘Reasonable’

probability of a

‘Good Outcome’

are deployable

at point of need

Cyber Resilience …

15Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Could This Work … ?

16Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Monitoring Simulation

Deployment

17Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Three Principles

Cyber Resilience

Cyber Resilience = function ( PD, PU, rA , tC,, n, f )

Six Certification Arguments

1) Increase the probability of detection, understanding and acting

2) Increase the number of ‘Engineered Differences’

3) Invoke a continuum of ‘Proactive Updates’

1) Probability of detecting threats2) Probability of understanding threats 3) Rate of deploying mitigating

actions4) Time for a threat to propagate5) Quantity of ‘Engineered Differences’6) Frequency of ‘Proactive Updates’

f

n

18Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

▪ ACE-CSR Conference 2019

▪ Workshops▪ 6th Nov - Cardiff▪ 27th Nov - Glasgow▪ 4th Dec - Belfast▪ 15th Jan - London

Canvassing for Relevant Academic Research …

19Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

In Summary …

Establishing regulation, standards and best practices is in everyone's interests.Collaboration within companies, between companies and across sectors is essential.

20Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

• Cyber attacks are emergent properties triggered by an adversary for impact

• Nobody knows how the connected infrastructure will evolve or what new feature that we haven’t thought of yet will be valued by tomorrows consumer

• By changing our design methods to the operational space we are creating the room to innovate and exploit emergent properties

• It is for this reason that the organisations and the industries that master cyber resilience will be at the forefront of the emergence of value chains in this new world.

In Summary …

21Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

• We can numerically describe and defend a complex digital system – including emergent and non-deterministic behaviour (cyber attacks) – in a legal setting

• Forming the basis for a new definition for type approval

• Enabling investments in technologies that can bring quantifiable benefits

• Identified areas where actions and improvements are required

In Summary …

We have fundamentally reorganised engineering knowhow and methodsto be fit for Complex Systems that are Connected & Autonomous

We have a unique opportunity to invest and re-imagine the future of resilient systems across multiple sectors to the economic advantage of the UK

22Thales OpenThis document may not be reproduced, modified, adapted, published,

translated, in any way, in whole or in part or disclosed to a third party

without the prior written consent of Thales - © Thales 2015 All rights reserved.

Thank you