Workshop on Foundations of Dependable and Secure Cyber-Physical Systems … · The Workshop on the Foundations of Dependable and Secure Cyber‐Physical Systems (FDSCPS) will focus

Workshop on Foundations of Dependable and Secure

Cyber-Physical Systems (FDSCPS)

CPSWeek 2011

April 11, 2011 Chicago, Illinois

TABLE OF CONTENTS

TABLE OF CONTENTS ........................................................................................... 1

WELCOME MESSAGE ............................................................................................ 3

WORKSHOP OVERVIEW ....................................................................................... 4

WORKSHOP ORGANIZATION .............................................................................. 5

TRUST CENTER OVERVIEW ................................................................................ 6

KEYNOTE TALKS ................................................................................................... 7

WORKSHOP PAPERS ............................................................................................. 9

Correlated Failures of Power Systems: The Analysis of the Nordic Grid Martin Andreasson (Royal Institute of Technology, Sweden (KTH)), Saurabh Amin (University of California, Berkeley), Galina Schwartz (University of California, Berkeley), Karl Henrik Johansson (Royal Institute of Technology, Sweden (KTH)), Henrik Sandberg (Royal Institute of Technology, Sweden (KTH)), S. Shankar Sastry (University of California, Berkeley) .................................................................................................................... 9

Resource Allocation Problems in Networked Cyber Physical Systems in Adversarial Scenarios Sourabh Bhattacharya (University of Illinois at Urbana-Champaign), Ali Khanafer (University of Illinois at Urbana-Champaign), Tamer Basar (University of Illinois at Urbana-Champaign) ............................................................................................................................................. 18

Enhancing Cyber-Physical Security through Data Patterns Richard Chow (Palo Alto Research Center), Ersin Uzun (Palo Alto Research Center), Alvaro A. Cardenas (Fujitsu Laboratories of America), Zhexuan Song (Fujitsu Laboratories of America), Sung Lee (Fujitsu Laboratories of America) ....................................... 25

Novel Reconfigurable Silicon Physical Unclonable Functions Yingjie Lao (University of Minnesota, Twin Cities), Keshab K. Parhi (University of Minnesota, Twin Cities) ...................................................................................................................................... 30

Verification of Information Flow Properties in Cyber-Physical Systems Bruce McMillin (Missouri University of Science and Technology), Ravi Akella (Missouri University of Science and Technology) .......................................................................................... 37

Secure Data Transmission against False Data Injection Attacks in Wireless Sensor Networks Yilin Mo (Carnegie Mellon University), Bruno Sinopoli (Carnegie Mellon University) ............................................................................................................................................................. 41

Towards a Unifying Security Framework for Cyber-Physical Systems Quanyan Zhu (University of Illinois at Urbana-Champaign), Tamer Basar (University of Illinois at Urbana-Champaign) ................................................................................................. 47

Workshop on Foundations of Dependable and Secure Cyber-Physical Systems (FDSCPS)

CPSWeek 2011 | April 11, 2011 | Chicago, Illinois 1

TABLE OF CONTENTS (cont.)

Management of Control System Information Security: Control System Patch Management Quanyan Zhu (University of Illinois at Urbana-Champaign), Miles McQueen (Idaho National Laboratory), Craig Rieger (Idaho National Laboratory), Tamer Basar (University of Illinois at Urbana-Champaign) ................................................................................................. 51

Secure Routing in Smart Grids Quanyan Zhu (University of Illinois at Urbana-Champaign), Dong Wei (Siemens Corporation), Tamer Basar (University of Illinois at Urbana-Champaign) ................................................. 55


2 CPSWeek 2011 | April 11, 2011 | Chicago, Illinois

WELCOME MESSAGE On behalf of the Workshop Program Committee, it is our pleasure to welcome you to the Workshop on Foundations of Dependable and Secure Cyber-Physical Systems (FDSCPS) in conjunction with CPSWeek 2011in Chicago, Illinois.

This workshop will bring together novel concepts and theories that can help in the development of the science of dependable and secure cyber-physical systems (CPS). These systems govern the operation of critical infrastructures such as power transmission, water distribution, transportation, healthcare, building automation, and process control. The use of internet-connected devices and commodity IT solutions and the malicious intents of hackers and cybercriminals have made these systems vulnerable. Despite attempts to develop guidelines for the design and operation of security policies, much remains to be done to achieve a principled, science-based approach to enhance security, trustworthiness, and dependability of control systems.

The workshop will focus on system theoretic approaches to address fundamental challenges to make CPS secure, dependable, and trustworthy, with a particular emphasis given to control and verification challenges arising as a result of complex interdependencies between these networked systems. In doing so, the workshop will serve as a first step toward the development of a principled approach to high-confidence CPS.

We greatly appreciate your participation and hope you find the workshop informative.

Sincerely, The Workshop Steering Committee

Helen Gill, Ph.D. Brad Martin National Science Foundation National Security Agency S. Shankar Sastry, Ph.D. Janos Sztipanovits, Ph.D. University of California, Berkeley Vanderbilt University



WORKSHOP OVERVIEW The Workshop on the Foundations of Dependable and Secure Cyber‐Physical Systems (FDSCPS) will focus on system theoretic approaches to address fundamental challenges to make cyber‐physical systems (CPS) secure, dependable, and trustworthy. A particular emphasis will be given on the control and verification challenges arising as a result of complex interdependencies between these networked systems. In doing so, the workshop will serve as a first step toward the development of a principled approach to high confidence CPS. The main aim of this workshop is to bring together novel concepts and theories that can help in the development of the science of dependable and secure cyber‐physical systems. This workshop also aims to foster collaborations between researchers from the fields of control and systems theory, embedded systems, game theory, software verification and formal methods, and computer security. The scope of this workshop is to discuss theories and methodologies that encompass ideas from:

Fault-tolerant and networked control systems Game theory for multi-agent dynamics in uncertain environments, and Learning and verification theory for secure and trustworthy systems.

Topics of interest will include, but are not restricted to, the following:

Taxonomy of attacks and attack models for control systems Novel security challenges in control systems Testbeds for security of critical infrastructure systems Decision and game theoretic approaches to security analysis Design architectures for prevention and resilience against attacks Risk assessment and verification of security properties Detectability and diagnosis of attacks Economics based studies of security and reliability Resilience and robustness against attacks Response and reconfiguration methods Cyber awareness of human-centric systems Complexity and resilience in control systems

Approaches that can be applied to particular critical infrastructure systems in Transportation (surface and aviation), Energy (smart grid and building energy management), and Healthcare (medical systems and associated embedded devices) are particularly welcomed. Also welcomed is foundational work that cuts across multiple application areas or advances the scientific understanding of underlying principles for the development of secure and trustworthy systems, including ways to measure the security properties of a system and methods to conduct robust and repeatable experimentation.



WORKSHOP ORGANIZATION STEERING COMMITTEE

Helen Gill (National Science Foundation) Brad Martin (National Security Agency) Shankar Sastry (University of California, Berkeley) Janos Sztipanovits (Vanderbilt University)

PROGRAM COMMITTEE

Tamer Basar (University of Illinois at Urbana‐Champaign) Michael Chertkov (Los Alamos National Laboratory) Richard Chow (PARC) John Chuang (University of California, Berkeley) George Cybenko (Dartmouth College) Anupam Datta (Carnegie Mellon University) Karl Henrik Johansson (Royal Institute of Technology, Sweden) Gabor Karsai (Vanderbilt University) Rupak Majumdar (University of California, Los Angeles) Radha Poovendran (University of Washington) Craig Rieger (Idaho National Laboratory) Galina Schwartz (University of California, Berkeley)

WORKSHOP ORGANIZERS

Saurabh Amin (University of California, Berkeley) Katie Dey (PRO‐telligent) Frankie King (PRO‐telligent) Larry Rohrbough (University of California, Berkeley)



TRUST CENTER OVERVIEW The Team for Research in Ubiquitous Secure Technology (TRUST) is focused on the development of cyber security science and technology that will radically transform the ability of organizations to design, build, and operate trustworthy information systems for the nation's critical infrastructure. Established as a National Science Foundation Science and Technology Center (STC), TRUST is addressing technical, operational, legal, policy, and economic issues affecting security, privacy, and data protection as well as the challenges of developing, deploying, and using trustworthy systems. TRUST activities are advancing a leading-edge research agenda to improve the state-of-the art in cyber security; developing a robust education plan to teach the next generation of computer scientists, engineers, and social scientists; and pursuing knowledge transfer opportunities to transition TRUST results to end users within industry and the government. TRUST is addressing technical, operational, privacy, and policy challenges via interdisciplinary projects that combine fundamental science and applied research to deliver breakthrough advances in trustworthy systems in three “grand challenge” areas:

Financial Infrastructures – Creation of a trustworthy environment that links and supports commercial transactions among financial institutions, online retailers, and customers. Health Infrastructures – Technology that advances “Healthcare Informatics” to enable engaged patients, personalized medicine, providers as coach-consultants, and agile evidence-based care. Physical Infrastructures – Advances that support Next Generation Supervisory Control and Data Acquisition (SCADA) and control systems, including power, water, and telecommunications.

TRUST is led by the University of California, Berkeley with partner institutions Carnegie Mellon University, Cornell University, San Jose State University, Stanford University, and Vanderbilt University. TRUST projects have a holistic view that addresses computer security, software technology, analysis of complex interacting systems, and economic, legal, and public policy issues. As such, TRUST draws on researchers is such diverse fields as Computer Engineering, Computer Science, Economics, Electrical Engineering, Law, Public Policy, and the Social Sciences. More information on TRUST is available at http://www.truststc.org.



KEYNOTE TALKS Cyber-Security Analysis and Resilient Control Design for Infrastructure Systems S. Shankar Sastry (University of California, Berkeley) ABSTRACT: The extensive use of information and communication technologies raises concerns about the vulnerabilities of critical infrastructure systems to both random failures and security attacks. Cyber-security of Supervisory Control and Data Acquisition (SCADA) systems is especially important, because they are employed for sensing and control of large physical infrastructures. This work focuses on the design of novel control methods and security mechanisms for infrastructure systems, and applies them to water distribution, process control, and energy management systems. The talk will discuss the progress made in following research areas: (1) security threat assessment, (2) model-based attack diagnosis, and (3) resilient control design. First, cyber-security assessment for SCADA systems will be performed based on well-defined attacker and defender objectives. The mathematical model of SCADA systems considered in this work has two control levels; i.e., regulatory control using distributed proportional-integral controllers, and supervisory fault diagnosis based on approximate dynamical system models. Second, design of attack diagnosis schemes that incorporate the knowledge of physical dynamics of the system will be presented. The performance of these schemes on a class of deception attacks will be described. Third, some fundamental limits on the closed-loop performance of networked control systems, and their implications to security of SCADA systems will be discussed. Finally, the necessity of security mechanisms which account for these limits and the presence of interdependent risks will be justified using a game-theoretic model. BIOGRAPHY: S. Shankar Sastry is the Dean of Engineering and Roy W. Carlson Professor of Engineering at the University of California, Berkeley. He helped establish and is Director of TRUST (Team for Research in Ubiquitous Secure Technology), an NSF Science & Technology Center and is Faculty Director of the Blum Center for Developing Economies. He was Director of CITRIS, Chair, Department of Electrical Engineering and Computer Sciences, UC Berkeley, Director, DARPA Information Technology Office, and Director, UC Berkeley Electronics Research Laboratory. Dr. Sastry received his Ph.D. degree in 1981 from the University of California, Berkeley. He was on the faculty of MIT as Assistant Professor from 1980-82 and Harvard University as a chaired Gordon Mc Kay professor in 1994. His areas of research are embedded control especially for wireless systems, cybersecurity for embedded systems, critical infrastructure protection, autonomous software for unmanned systems (especially aerial vehicles), computer vision, nonlinear and adaptive control, control of hybrid and embedded systems, and network embedded systems and software. He has coauthored over 450 technical papers and 9 books and served as Associate Editor for numerous publications, including: IEEE Transactions on Automatic Control; IEEE Control Magazine; IEEE Transactions on Circuits and Systems; the Journal of Mathematical Systems, Estimation and Control; IMA Journal of Control and Information; the International Journal of Adaptive Control and Signal Processing; Journal of Biomimetic Systems and Materials. He is currently an Associate Editor of the IEEE Proceedings. Dr. Sastry was elected into the National Academy of Engineering in 2001 and the American Academy of Arts and Sciences (AAAS) in 2004 and made a Fellow of the IEEE in 1994. He received the President of India Gold Medal in 1977, the IBM Faculty Development award for 1983–1985, the NSF Presidential Young Investigator Award in 1985, the American Automatic Control Council Eckman Award in 1990, the Distinguished Alumnus Award of the Indian Institute of Technology in 1999, the David Marr prize for the best paper at the International Conference in Computer Vision in 1999, the Ragazzini Award for Distinguished Accomplishments in teaching in 2005, and the C.L. Tien Award for Academic Leadership in 2010. He was awarded an M. A. (honoris causa) from Harvard in 1994 and an honorary doctorate from the Royal Institute of Technology, Sweden in 2007. Dr. Sastry has been a member of the Air Force Scientific Advisory Board from 2002-2005 and the Defense Science Board in 2008, he is on the corporate boards of Crossbow, Inc. and C3-Carbon, LLC, and has supervised over 60 doctoral students and over 50 M.S. students to completion. Dependable and Secure Automotive Cyber-Physical Systems William P. Milam (Ford Motor Company) ABSTRACT: Recently a workshop was help in Michigan to discuss Developing Dependable and Secure Automotive Cyber-Physical Systems from Components. The goal of this workshop was to address emerging



challenges relative to reliability, availability, safety, and security attributes of software-intensive electronic automotive control systems and road infrastructure systems. An example of such a system would be a self-driving vehicle that must adapt in order to navigate safely and efficiently through traffic in the presence of intersections, pedestrians and other traffic. Another example would be an emergency vehicle with advanced engine and transmission controls integrated with stability control that is able to instantly respond to driver input and road conditions and keep the vehicle in the lane while traversing a curve in icy conditions. We will provide an early report out of the proceedings from that workshop as well as proposed actions and timetables. BIOGRAPHY: William Milam is a Technical Expert at the Ford Research and Innovation Center, Ford Motor Company. His research addresses modeling and implementation of advanced technology automotive powertrains for improved fuel economy and emissions, through improvements in systems engineering processes for the design of automotive embedded systems. He is a senior member of the IEEE and a member of the SAE. Mr. Milam is a member of the SAE Electronic Design Automation Standards Committee, the SAE Architecture Analysis and Design Language Standards Committee, and chairs the SAE Model Based Embedded System Engineering Task Force. He is also the chair of the USCAR Cyber-Physical Systems Task Force. Trustworthy Computing and Cyber-Physical Systems Sam Weber (National Science Foundation) ABSTRACT: This talk will discuss research challenges in the intersection of Trustworthy Computing and Cyber-Physical Systems. Both research communities have had long, mostly independent, histories despite their obvious common goal of building systems that humans can depend upon. As a result, collaboration between communities results in discovering interesting challenges, possible misunderstandings, and sometimes unexpected benefits. BIOGRAPHY: Sam Weber is a Program Director for the National Science Foundation’s Trustworthy Computing Program. He received his M.Sc. from the University of Toronto, and his Ph.D. from Cornell University. Gaps Between Traditional and CPS Security Janos Sztipanovits (Institute for Software Integrated Systems, Vanderbilt University) ABSTRACT: Security requirements are traditionally enumerated in terms of confidentiality, integrity, and availability. In CPS, security is integral part of the resilience and dependability requirements that intend to ensure critical functions at all times, despite damage caused by accidental faults, errors, and degradations in the physical plant or malicious intrusions in the computer control system and network. Tradeoff among autonomy, performance, resilience and dependability is complex and need to be approached holistically. If increased complexity brought about by advanced CPS automation leads to brittleness, instability, malfunctions, or vulnerability, the benefits will not be realized and the approach may fail. Achieving resilience to physical faults and cyber attacks in CPS requires an integrated suite of technologies. Current approaches are compartmentalized, not prepared for operating in a complex, highly coupled cyber-physical environment, and cannot take advantage of commonalities in solutions. Fault-tolerant control systems typically assume well-understood models and bounds on the nature of faults in system operation. Intrusion detection systems are rarely used in CPS, and cyber security considerations are largely neglected. Integration of functional, fault management and security architectures are missing even in safety critical industrial automation systems and military systems. In computing systems, fault-tolerance focuses on faults in the hardware or software infrastructure, not on the system-wide impact of these faults. Introducing privacy considerations in system design is a largely unsolved problem. This talk will explore these challenges and discuss research directions that may yield solution. BIOGRAPHY: Dr. Janos Sztipanovits is the E. Bronson Ingram Distinguished Professor of Engineering at Vanderbilt University. He is founding director of the Institute for Software Integrated Systems (ISIS). His research areas are at the intersection of systems and computer science and engineering. His current research interest includes the foundation and applications of model-based methods for the design of Cyber Physical Systems.



Correlated Failures of Power Systems:Analysis of the Nordic Grid

M. Andreasson∗, S. Amin†, G. Schwartz †, K. H. Johansson∗, H. Sandberg∗ and S. S. Sastry†∗ACCESS Linnaeus Center, Royal Institute of Technology, Stockholm, Sweden. mandreas, kallej, hsan @kth.se

†TRUST Center, University of California, Berkeley. saurabh, schwartz, sastry @eecs.berkeley.edu

Abstract—In this work we have analyzed the effectsof correlated failures of power lines on the total systemload shed. The total system load shed is determined bysolving the optimal load shedding problem, which is thesystem operator’s best response to a system failure. We haveintroduced a Monte Carlo based simulation framework forestimating the statistics of the system load shed as a func-tion of stochastic network parameters, and provide explicitguarantees on the sampling accuracy. This framework hasbeen applied to a 470 bus model of the Nordic power systemand a correlated Bernoulli failure model. It has been foundthat increased correlations between Bernoulli failures ofpower lines can dramatically increase the expected valueas well as the variance of the system load shed.

I. INTRODUCTION

Power systems are among the largest and most com-plex systems created by mankind. The complexity ofpower grids keeps increasing as power grids are ex-panded and new functionalities are added, as with thecurrent developments of the SmartGrid [1]. Since manyvital parts of today’s society require reliable supply ofelectricity, the reliable and secure operation of powersystems is indisputably essential [2], [3]. We give a briefoverview of the research in the two distinctive areasof security against adversarial attacks and reliability ofpower systems against random failures. We concludethat correlated failures in power systems represent a gapbetween these two research areas.

In the area of security, research on characterizing opti-mal attack and defense strategies has gained momentumover the past years. In [4] the optimization problem ofmaximizing the power outage, for a given number ofpower transmission lines that an adversary is capableof disconnecting, is considered. The system operator isassumed to take the best action to minimize the damagein form of compulsory load shedding. The problem isgame theoretic by nature and gives rise to a maximinoptimization problem, where the outer maximizationseeks the most disruptive attack for a given budget ofthe adversary, and the inner minimization solves the

This work was supported in part by the European Commissionthrough the VIKING project, the Swedish Research Council, theSwedish Foundation for Strategic Research, and the Knut and AliceWallenberg Foundation

optimal load shedding problem which minimizes theconsequences of an attack. Because of the non-convexityand the existence of integer variables, the problem is in-herently hard to solve for large systems. [5] approximatesthe nonlinear mixed integer bi-level program by a mixedinteger linear program, and derives an upper bound onthe severity of adversarial attacks.

Traditionally, the reliability of power systems hasoften been characterized by deterministic means, suchas the widely uses N − k criterion [6], and in almostall cases k = 1 is used [7]. A power system satisfyingthe N −k criterion is able to withstand any contingencyconsisting of k outages. The main drawback of the N−kcriterion and other deterministic reliability criteria is thatthey do not take into account the probabilities of thecontingencies. Furthermore, the number of events whichhave to be considered when evaluating the N−k criteriongrows exponentially in k, making the reliability evalu-ation computationally intractable. More recent researchon the reliability of power systems has emphasized thatmany events governing the reliability of power systemsare by nature stochastic, e.g. demands and generation ca-pacities. Various statistical and sampling based methodsfor evaluating the reliability of power systems have beendeveloped to analyze stochastic phenomena in powersystems [8], [9]. In [10] a two state Markov model for thefailure of various power system elements is considered,and the statistics of the power system are calculatedusing Monte Carlo techniques. However, the model doesnot take correlated failures into account. Other thanfailure correlations due to cascading failures [11], wefound no model which attempts to model correlatedfailures in power systems.

This work aims at studying the effects of correla-tions between failures of power system components,and in particular power lines. In contrast to previouspapers on the subject, we introduce a failure modelexplicitly taking into account correlated system failures.We evaluate the impact of correlations of failures bythe covariances between the failures. Our research ismotivated by the increased deployment of of-the-shelfhardware and software in SCADA systems governingthe power systems. When similar or even identical



software is deployed in several system components,software failures are likely to be correlated betweenthose components. Indeed, current literature suggeststhat bugs present in one system component are likelyto also be present in a similar or identical components[12]. From a cyber security point of view, this work ismotivated by the possibility of malicious code exploitingidentical software bugs and security flaws. A computervirus spreading in the SCADA network is likely to affectmultiple of its target components, and thus causing cor-relations between failures. The increased interconnectionof control and communication systems as well as theirconnection to the Internet, facilitates the exploitation ofsoftware bugs. There are indications that software bugscaused failures leading to a blackout in the northeastblackout of 2003 [13]. We believe that correlated failuresprovides a good mean of understanding the affects offailures caused by malicious software affecting multiplesystem components. Correlated failures may also occurdue to natural disasters such as earthquakes or hurricanes[14], [15]. Examples of major power system outagescaused by natural disasters include the New York cityblackout in 1977, where lightning struck a substationand a power line almost simultaneously [16].

We measure the impact of a system failure by theminimal system load shed required to restore the systemto a safe state. This formulation gives rise to an optimiza-tion problem, which under simplified conditions can bemade linear. For different values of the correlations of thefailure distribution, we compute the sampled statistics ofthe total system load shed by Monte Carlo techniques,and provide guarantees on the convergence rate of thesampled statistics. In particular, we use a weighted sumof the mean and variance of the total system loadshed as a risk measure of the failure statistics. Toobtain statistical data from a realistic power system,we apply our techniques to a 470 bus model of theNordic power system, acquired from publicly availablesources. We have found that increasing correlations be-tween Bernoulli failures of power lines lead to increasedexpected value and variance of the system load shedunder various topological structures of the correlations.

The rest of the paper is organized as follows. InSection II the optimal load shedding problem is pre-sented and the total system load shed is defined. Insection III a novel model of the Nordic power systemis presented and evaluated . In Section IV a MonteCarlo sampling technique is presented and the effectsof correlated failures on the Nordic power system arestudied, followed by concluding remarks in Section V.

II. OPTIMAL LOAD SHEDDING

Optimal power flow (OPF) problems are in manyways analogous to transportation problems, with the onlydifference that the power flows obey the Kirchoff voltagelaw, and are proportional to the relative phase angle

difference between the buses [17]. The optimal loadshedding problem is a special case of OPF problems,where the load shed is minimized subject to physicalconstraints [18], [19]. OPF problems have been solvedusing a variety of optimization techniques, ranging fromlinear programming [20], Newton methods [21] to in-terior point methods [22]. While many techniques relyon the linearized power flow equations, there are OPFproblems using the nonlinear power flow equations [23],[24]. While showing promising results, these methodssuffer from general limitations of non-convex optimiza-tion such as convergence and computational complexity.

We will only consider the linearized power flow equa-tions in our work, thereby ensuring robust and fast con-vergence. Since we will apply Monte Carlo techniquessolving the optimal load shedding problem, computationspeed is certainly of importance. By assuming that theadmittance in the shunt branch of the power lines isnegligible, and that the resistance-to-reactance ratio issufficiently small, reactive power flows can be neglected,and the real power real power flows are described by theDC-model [25]:

P line = V lineB sin (Aθ) (1)

where P line is a column vector of active power flowsin the transmission lines, V line = diag

(ViVj

)where

Vi is the voltage of bus i, B = diag(bij) where bij isthe admittance of the power line connecting bus i withbus j, θ is the vector of bus phase angles and A is thevertex-edge incidence matrix of the graph of the powersystem, defined as Aij = 1 iff ei = (vj , u) ∈ E, Aij =−1 iff ei = (u, vj) ∈ E and Aij = 0 otherwise. Heresin(x) =

[sin(xi), . . . , sin(xn)

]Tfor a vector x. By only

considering sufficiently small phase angle differences,i.e. ∆θmax = ‖Aθ‖∞ being sufficiently small, we maylinearize (1) around Aθ = 0;

P line = V lineBAθ (2)

By summing the power flows to each bus, we get thelinearized equation for the net power flows into thebuses;

P = ATV lineBAθ =: LBθ

where P is a vector of real power injections to the buses.LB can be interpreted as a weighted Laplacian matrixof the graph associated with the power system, withweights corresponding to the line admittances times thebus voltages. We may assume, wlog, that the buses of

the power system are partitioned as P =[P gT , P l

T]T

,where P g > 0 are generator buses and P l ≤ 0are load buses. We consider the optimization problemof minimizing the total load shed of the system. Theoptimal load shedding problem can, for the linearizedpower flow equations, be formulated as a linear Program



(LP);

minθ

cT θ (3)

s.t. Cθ d (4)

where

C =

V lineBA−V lineBA

LB−LBA−A

d =

P linemax

P linemax

P gmax0nl×1

0ng×1

−P ld∆θmax · 1np×1

∆θmax · 1np×1

(5)

andc =

[01×ng 11×nl

]LB (6)

and where ng , nl and np denotes the number of generatorbuses, load buses and power lines respectively, and0 < ∆θmax ≤ π

2 is a sufficiently small real numberfor which the linearized power flow equation (2) is avalid approximation. The matrix inequality in (4) com-bines line capacity constraints

(|P line| P linemax

), power

generation constraints (0 P g P gmax), power loadconstraints

(P ld P l 0

)and phase angle constraints(

‖Aθ‖∞ ≤ θmax). The objective function cT θ is the sum

of the power injections to the demand buses, which isa linearly affine function of the total system load shed.It can be easily seen that the minimum total load shedS∗(C, d), which is the difference between the total powerdemand and total delivered power, is given by

S∗(C, d) = minθ

cT θ|Cθ d

− 11×nl · P ld (7)

III. MODEL OF THE NORDIC POWER SYSTEM

In this section, we construct a model of the Nordicpower system to demonstrate the previously discussedsampling techniques. While IEEE standard power sys-tems offer great transparency for research and canfunction as benchmark systems, we believe that it isimportant to demonstrate our concepts on real powersystems. We have built a model of the Nordic powersystem, using only publicly available data. While therehave been similar efforts to model other interconnectedpower systems, such as the main European power grid[26], there are no known complete models of the Nordicpower system that are publicly available.

A. Obtaining the network topology

The topology of the power system. i.e. the geograph-ical positions of the main 400, 300, 220 and 132 kVpower lines and HVDC links in the Nordic countrieswere obtained from the respective TSOs websites. Thedata obtained included the coordinates of the powerbuses, the connectivity of the buses through power lines,the voltage of the power lines and the number of parallel

power lines, if applicable. The complete model has atotal of 470 buses and 717 power lines. The power linesand buses of the Nordic power grid are illustrated infigure 1.

Production node

Demand node

400 kV power line

300 kV power line

220 kV power line

132 kV power line

Figure 1. Model of the Nordel high voltage power transmissionnetwork. The circle area of the production and demand buses areproportional to the production and demand respectively.

B. Estimating power generation capacities

Information about all power generation facilities withcapacities of at least 100 MW were also acquired frompublic sources [27]. To compensate for the lack ofavailable data of power plants with generation capacityless than 100 MW, we have made assumptions about theremaining power plants. The remaining thermal powerplants are assumed to be located in populated areas, andhence the thermal generation capacity is proportional tothe demands. As for the remaining wind power capacity,we have assumed that the wind power generation isuniformly distributed over the land surface, and henceover the buses. These assumptions may appear crude, butconsidering that these assumptions only apply to powerplants whose generation capacity is below 100 MW, thenet effect of possible errors on the whole model is ofminor importance.

C. Estimating power demand data

There is no available electricity demand data, otherthan cumulative data for the Nordic countries. This datais to rough to be useful for our 470 bus model. Following[26] we have used population census data to estimate the



power demand. This methodology relies on the assump-tions that household power demand is proportional to thepopulation connected to a substation, as well as industrypower demand, since the workforce will settle relativelyclose to industries. This may however not necessarily bethe case for certain energy-intensive industries which areusually co-located with energy sources, nor for certainlocation-specific industries such as forestry or the oilindustry. Population statistics were collected from theBureau of Statistics of the respective countries. We havecollected cumulated population statistics for the majoradministrative regions of each country, and assumed thatthe population (and hence the demand) is distributeduniformly over the load buses within each region. Thenumber of administrative regions in each country wasbetween 12 and 21. Using smaller regions would in-troduce difficulties in assigning the right population toeach substation. To estimate the power demands, boththe yearly average and yearly maximum of the dailymaximum power consumption were used to create twodifferent load situations.

D. Estimating power line parameters

The only known parameters of the power lines ob-tained from public sources are the line voltages. Tosolve the optimal load shedding problem, also the lineadmittances as well as the maximum transmission ca-pacities of each line need to be known. The admittanceof power lines can be estimated by the length of thepower line. Typically the reactance of high voltage powertransmission lines is approximately 0.20 Ω/km [28]. Thelengths of a power line from a bus with coordinatesx to a bus with coordinates y is estimated by theeuclidean 2-norm as l = dist(x, y) = ‖x− y‖2 =√

(x1 − y1)2 + (x2 − y2)2 which is always an under-estimate of the actual line length. As for estimatingtransmission capacities, only cross-border transmissionline capacity constraints are available from the NordicTSOs. The transmission capacity of each power line ofequal voltage is assumed to be the average transmissioncapacity of the cross-border lines, which are shown inTable I.

Table IESTIMATED TRANSMISSION LINE CAPACITIES.

Voltage Capacity400 kV 1030 MW300 kV 650 MW220 kV 415 MW132 kV 143 MW

E. Evaluating the model

The optimal load shedding problem was applied tothe previously derived model of the Nordic power trans-mission grid. By using the YALMIP [29] interface with

the GLPK [30] LP solver in MATLAB [31], the optimalload shedding problem was solved. When solving thelinear optimal load shedding problem with the yearlymaximum loads, the total system load shed was foundto be 2 % of the total power demand. When solving theoptimal load shedding problem with the yearly averageof the daily maximum loads, no system load shed wasnecessary. This demonstrates that our model is indeedusable for our study.

IV. STATISTICS OF POWER SYSTEM FAILURES

A. Monte Carlo methods

In this paper we will consider stochastic failures ofthe power system, as in e.g. [8], [9]. To demonstrate thegenerality of our methods, we will not yet make anyassumptions about these failures. Consider the matricesC and d, associated with an arbitrary power system, asrandom variables endowed with a probability measureµC × µd. Since both the topology and load parametersof the power system are determined by C and d, such aprobability measure can represent any type of failuresof the power system. It can be shown that for anyprobability measure µC × µd, the minimum total loadshed S∗(C, d) is also a random variable. The total loadshed is a commonly used measure of the severeness ofa power system outage [32], [33].

We will consider the sampled probability distributionof the minimum total system load shed S∗(C, d), andin particular we will consider the mean and the vari-ance of the sampled probability distribution. Because0 ≤ S∗(C, d) ≤ −

∑P ld, as seen from (4), the mean

S∗ and variance σ2S∗ of S∗(C, d) always exist and are

finite. We will use S∗+α·σS∗ , α ∈ R+ as a risk measurefor the distribution µC ×µd. We show that S∗+α ·σS∗

is closely related to the commonly used risk measurevalue at risk (VaR), which for a random variable S∗ isdefined as follows [34]:

VaRα(S∗) = infl ∈ R : Pr(S∗ > l) ≤ 1− α

The intuition of the expression VaRα(S∗) is that themaximum loss, in our case system load shed, is boundedby VaRα(S∗) with probability 1−α. One serious compu-tational drawback with using VaR on sampled probabil-ity distributions, is that it requires knowledge of the fullprobability distribution of the random variable S∗. Whendealing with samples of random variables, estimatingVaR becomes hard since it requires estimation of the tailof the distribution S∗. The following proposition allowsus to obtain an upper bound VaRα(S∗) using a linearcombination of the mean and the variance of S∗, whichcan be estimated more robustly:

Proposition 1. The risk measure value at risk (VaRα)satisfies

VaRα(S∗) ≤ S∗ +1√α· σS∗



The proof is given in the appendix. Since obtaininganalytical expressions for S∗ and σ2

S∗ is in general notpossible, we will use Monte Carlo techniques [35] toestimate the mean and the variance of the load shed. Bydrawing N samples from the distribution µC × µd, weobtain the following approximations of S∗ and σ2

S∗ ;

S∗ ≈ S∗ =1

N

N∑i=1

S∗(Ci, di) (8)

σ2S∗ ≈ σ2

S∗ =1

N − 1

N∑i=1

(S∗(Ci, di)− S∗

)2

(9)

Due to S∗(C, d) being bounded, S∗ and σS∗ are guar-anteed to converge to S∗ and σS∗ respectively.

Proposition 2. Given ε > 0, δ > 0, the number ofsamples N1 and N2 which assure that

Pr

[∣∣∣S∗,N1 − S∗∣∣∣ ≥ ε] ≤ δ

Pr[|σS∗,N2 − σS∗ | ≥ ε

]≤ δ

are

N1 ≥

⌈S2

4δε2

⌉N2 ≥

⌈S4

8δε4

⌉Proof: Follows by lemma 3 and lemma 5 in the

appendix, and the fact that 0 ≤ S∗(C, d) ≤ −∑P ld.

With proposition 2 we have guaranteed bounds of theestimation error of both the sampled expected value andthe sampled variance of the load shed. The propositioncan of course be used in the reverse direction. For givenN1 and N2, we can obtain bounds on δ and ε. With theseexplicit bounds on the error of the estimated mean andvariance of the load shed, the number of samples can bechosen according to given accuracy requirements, andtrade-offs between accuracy and the number of samplescan be made a priory.

B. Sampling of correlated system failuresIn this section we examine the effects of correlated

system faults on the statistics of the minimum totalsystem load shed. As discussed in the introduction, thestudy of correlated faults in power systems is motivatedby the increased deployment of of-the-shelf hardwareand software in SCADA systems governing the powersystems. When identical software is deployed in severalsystem components, software faults between these iden-tical components are likely to experience correlation. Inthe following empirical study we will consider failureson the form of power line disconnections. We model thedisconnection of power line i as a binary random variableXi ∈ 0, 1 where Xi = 0 corresponds to line i beingfully functional with all parameters set to default, andXi = 1 corresponds to line i being disconnected. Thus,the failure statistics of the power system are given by

P (X1 = Y1, . . . , Xnp = Ynp) ∀ Yi ∈ 0, 1

Since parameterizing the full joint Bernoulli distributionwould require nnp ≈ 10216 variables, we will considerthe Bernoulli distribution with the first two central mo-ments given explicitly, i.e.

Xi = E[Xi] ∀i ∈ 1, . . . , npσij = E

[(Xi − Xi)(Xj − Xj)

]∀ (i, j) ∈ 1, . . . , np2

To consider the effects of increasing correlations Xi =0.02 is kept constant, while the covariances σij areincreased. We consider two different scenarios wherethe correlation between a subset of the power lines isincreased.

1) Correlations between incident power lines: Wefirst consider the case where σij is increased equallyfor all incident power lines, from 0 to 0.016 in stepsof 0.004. For each step, 1000 Monte Carlo simulationsare performed with the Bernoulli sampling algorithmdescribed in [36] to acquire sampled statistics S∗. Byproposition 2 the relative error of S∗ is guaranteed tobe less than 7 % with certainty 95 %. In figure 2,the histograms of the sampling processes are shown fordifferent σij , together with fitted Weibull distributions.The mean and the variance of the load shed for differentcorrelations are shown in figure 4. Clearly both S∗ andσX∗ are strongly increasing in σij .

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0

Load shed (MW)

Rela

tive p

robabili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.004

Load shed (MW)

Rela

tive p

robabili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.008

Load shed (MW)

Rela

tive p

robabili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.012

Load shed (MW)

Rela

tive p

robabili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.016

Load shed (MW)

Rela

tive p

robabili

ty

Figure 2. Histograms of the sampled load shed distributions fordifferent σij between 0 to 0.016 when the covariance betweenneighboring lines is increased, with fitted Weibull distributions.



0 500 1000 1500 2000 2500 30000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1x 10

−3

Load shed (MW)

Pro

babili

ty

σ=0

σ=0.004

σ=0.008

σ=0.012

σ=0.016

Figure 3. Fitted Weibull distributions of the sampled load sheddistribution for σij from 0 to 0.016, when the covariance betweenneighboring lines is increased.

0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016500

600

700

800

900

1000

1100

1200

1300

1400

σij

X∗ ,σX

∗

X*

σX

*

Figure 4. Expected value and standard deviation (MW) of the totalload shed for different covariances for power lines connected to thesame bus.

2) Correlations between power lines incident toPMUs: We here consider the scenario of failures beingcorrelated only between lines incident to nodes withphasor measurement units (PMUs). This scenario is mo-tivated by the use of identical software and hardware inPMUs, which could cause correlations between failuresof these nodes, and hence the incident lines. In figure 5,the histograms of the sampling processes are shown fordifferent σij , together with fitted Weibull distributions.The mean and the variance of the load shed for differentcovariances are shown in figure 7. While σX∗ is notincreasing in σij , S∗ is increasing by a factor 15 withincreasing σij .

3) Remarks: Although simulations indicate that cor-relations increase the expected value of the system loadshed, it is easy to find counterexamples where increased

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0

Load shed (MW)

Re

lative

pro

ba

bili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.004

Load shed (MW)

Re

lative

pro

ba

bili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.008

Load shed (MW)

Re

lative

pro

ba

bili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.012

Load shed (MW)

Re

lative

pro

ba

bili

ty

0 5000 100000

0.2

0.4

0.6

0.8

1x 10

−3 σ=0.016

Load shed (MW)R

ela

tive

pro

ba

bili

ty

Figure 5. Histograms of the sampled load shed distributions fordifferent σij between 0 to 0.016 when the covariance betweenpower lines incident to PMU nodes is increased, with fitted Weibulldistributions.

0 500 1000 1500 2000 2500 30000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1x 10

−3

Load shed (MW)

Pro

babili

ty

σ=0

σ=0.004

σ=0.008

σ=0.012

σ=0.016

Figure 6. Fitted Weibull distributions of the sampled load sheddistribution for σij from 0 to 0.016, when the covariance betweenpower lines incident to PMU nodes is increased.

correlation between power line failures decreases theexpected load shed. The simplest possible counterex-ample is a 3-bus and 2-line power network shown infigure 8. Let the demand bus have demand −1, andthe generation bus a capacity g ≥ 1, and the lineparameters be such that the demand is satisfied under



0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.0160

1000

2000

3000

4000

5000

6000

7000

8000

σij

X∗ ,σX

∗

X*

σX

*

Figure 7. Expected value and standard deviation (MW) of the totalload shed for σij from 0 to 0.016, when correlations are increasedfor power lines incident to PMU nodes.

normal operation. It can be shown that the expected loadshed of the system is E[X1] + E[X2]− σ12, where σ12

is the covariance between the failures of power linesl1 and l2. The intuition behind this counterexample isthat while the probability of both lines failing increases,the probability of each failing individually decreases,with the result that the total probability of any linefailing decreases. We here state sufficient conditions

g n dl1 l2

Figure 8. Topology of a 3-bus, 2-line power network where increasedcorrelations between power line failures result in decreased system loadshed. g is a generation bus, d a demand bus and n a demand with powerdemand 0.

under which increased correlations of power line failuresimply increased expected load shed.

Proposition 3. Let the power system satisfy the n − kcriterion, i.e. the disconnection of any k power lines doesnot induce any necessary load shedding. Furthermore,assume that all contingencies with at least k + 1 linefailures induce a total system load shed c. Assume, wlog,that the moment φ1,...,k+1 = E[X1 · . . . ·Xk+1] increasesby ∆φ, but all other moments E[Xi1 · . . . · Xil ] areconstant. Then

1) The central moment σ1,...,k+1 =E[(X1 − X1) · . . . · (Xk+1 − Xk+1)

]also

increases by ∆φ.2) All other central moments of order less than or

equal to k + 1 remain constant.3) The expected load shed S∗ increases by c ·∆φ.4) If E[Xi] < 1/2 ∀φ, the variance of the load shed,

σS∗ , increases by c2 ·∆φ, where 0 < c ≤ c.

A proof is given in the appendix. The following

corollary follows directly from proposition 3.

Corollary 1. Assume that all conditions of proposition3 still hold, except that all contingencies with at leastk+ 1 line failures induce a system load shed of at leastc. In this case the results of proposition 3 hold insteadfor the lower bound S∗ ≤ S∗ of the system load shed,which is the total system load shed assuming all linefailures result in the same system load shed c.

V. CONCLUSIONS AND FUTURE RESEARCH

In this work we have demonstrated that increasedcorrelations between power line failures can dramaticallyincrease the expected costs in terms of system load shed,although the expected value of the failure probabilitiesis kept constant. Furthermore we have demonstrated thatincreased correlations between power line failures canalso increase the variance of the system load shed, thusincreasing the risk of large system load sheds. We havedemonstrated our results by the sampling of correlatedpower line failures, using a model of the Nordic powergrid. We have furthermore provided sufficient conditionsunder which the mean and the variance of the totalsystem load shed increase with increasing correlationbetween line failures.

The framework presented in this paper should beseen as a general framework for reliability evaluation ofpower systems where correlations are known a priory,either by empirical data or by improved failure modelsof power systems.

It should be clarified that the failures we consider donot correspond to cascading failures, but that they couldrepresent potential causes of cascading failures. In manysituations, cascading failures further aggravate the stateof a partially failing power system, leading to even largerlosses in terms of system load sheds. In future work, itwould be of interest to consider the impact of correlatedfailures under power system on cascading failures. Also,it would be interesting to study if the conditions underwhich the expected value and the variance of the loadshed are increasing in the correlations, can be relaxed.

REFERENCES

[1] Massoud and B. F. Wollenberg. Toward a smart grid: powerdelivery for the 21st century. IEEE Power and Energy Magazine,3(5):34–41, September 2005.

[2] M.T.O. Amanullah, A. Kalam, and A. Zayegh. Network se-curity vulnerabilities in scada and ems. In Transmission andDistribution Conference and Exhibition: Asia and Pacific, 2005IEEE/PES, pages 1 –6, 2005.

[3] M. Amin. Security challenges for the electricity infrastructure.Computer, 35(4):8 –10, April 2002.

[4] J. Salmeron, K. Wood, and R. Baldick. Analysis of electric gridsecurity under terrorist threat. Power Systems, IEEE Transactionson, 19(2):905 – 912, May 2004.

[5] Ali Pinar, Juan Meza, Vaibhav Donde, and Bernard Lesieutre.Optimization strategies for the vulnerability analysis of theelectric power grid. SIAM Journal on Optimization, 20(4):1786–1810, 2010.

[6] Roy Billinton and Ronald N Allan. Reliability evaluation ofpower systems. 1995.



[7] Xifan Wang. Modern Power System Planning. Mcgraw-Hill,1994.

[8] W Li Roy Billinton, Billinton. Reliability assessment of electricalpower systems using monte carlo methods. 1994.

[9] R. Allan and R. Billinton. Probabilistic assessment of powersystems. Proceedings of the IEEE, 88(2):140 –162, February2000.

[10] R. Billinton and Peng Wang. Teaching distribution system reli-ability evaluation using monte carlo simulation. Power Systems,IEEE Transactions on, 14(2):397 –403, May 1999.

[11] R. Kinney, P. Crucitti, R. Albert, and V. Latora. Modelingcascading failures in the north american power grid. TheEuropean Physical Journal B - Condensed Matter and ComplexSystems, 46:101–107, 2005. 10.1140/epjb/e2005-00237-9.

[12] Michael Grottke and Kishor S. Trivedi. Fighting bugs: Remove,retry, replicate, and rejuvenate. Computer, 40(2):107 –109, 2007.

[13] Kevin Poulsen. Software bug contributed to blackout. Security-Focus, 2004.

[14] Dorothy A. Reed. Electric utility distribution analysis for extremewinds. Journal of Wind Engineering and Industrial Aerodynam-ics, 96(1):123 – 140, 2008.

[15] RA Davidson, H Liu, IK Sarpong, and P Sparks. Electric powerdistribution system performance in carolina hurricanes. NaturalHazards Review, 4(1):36–45, 2003.

[16] A.G. Phadke and J.S. Thorp. Expose hidden failures to preventcascading outages [in power systems]. Computer Applications inPower, IEEE, 9(3):20 –23, July 1996.

[17] J. Zhu. Optimization of Power System Operation(). 2009.[18] M. Huneault and F.D. Galiana. A survey of the optimal power

flow literature. Power Systems, IEEE Transactions on, 6(2):762–770, 1991.

[19] M.F. Carvalho, S. Soares, and T. Ohishi. Optimal active powerdispatch by network flow approach. Power Systems, IEEETransactions on, 3(4):1640 –1647, November 1988.

[20] B.. Stott and J.L. Marinho. Linear programming for power-system network security applications. Power Apparatus andSystems, IEEE Transactions on, PAS-98(3):837 –848, May 1979.

[21] D.I. Sun, B. Ashley, B. Brewer, A. Hughes, and W.F. Tinney.Optimal power flow by newton approach. Power Apparatus andSystems, IEEE Transactions on, PAS-103(10):2864 –2880, 1984.

[22] Hua Wei, H. Sasaki, J. Kubokawa, and R. Yokoyama. An interiorpoint nonlinear programming for optimal power flow problemswith a novel data structure. Power Systems, IEEE Transactionson, 13(3):870 –877, August 1998.

[23] R.C. Burchett, H.H. Happ, and D.R. Vierath. Quadraticallyconvergent optimal power flow. Power Apparatus and Systems,IEEE Transactions on, PAS-103(11):3267 –3275, 1984.

[24] X.-P. Zhang and E.J. Handschin. Advanced implementation ofupfc in a nonlinear interior-point opf. Generation, Transmissionand Distribution, IEE Proceedings-, 148(5):489 –496, September2001.

[25] Ali Abur and Antonio Gomez Exposito. Power System State Esti-mation: Theory and Implementation; electronic version. Dekker,Abingdon, 2004.

[26] Qiong Zhou and J.W. Bialek. Approximate model of europeaninterconnected system as a benchmark system to study effectsof cross-border trades. Power Systems, IEEE Transactions on,20(2):782 – 788, May 2005.

[27] Data and detailed sources are available from the first author uponrequest.

[28] Brugg. High voltage xlpe cable systems, technical user guide.2006.

[29] J. Löfberg. Yalmip : A toolbox for modeling and optimizationin MATLAB. In Proceedings of the CACSD Conference, Taipei,Taiwan, 2004.

[30] Andrew Makhorin. GNU Linear ProgrammingKit, Version 4.9. GNU Software Foundation,http://www.gnu.org/software/glpk/glpk.html, 2006.

[31] MATLAB. version 7.11.0 (R2010b). The MathWorks Inc.,Natick, Massachusetts, 2010.

[32] R. Billinton and P. Wang. Distribution system reliabilitycost/worth analysis using analytical and sequential simulationtechniques. Power Systems, IEEE Transactions on, 13(4):1245–1250, November 1998.

[33] R. Billinton and E. Khan. A security based approach tocomposite power system reliability evaluation. Power Systems,IEEE Transactions on, 7(1):65 –72, February 1992.

[34] Darrell Duffie and Jun Pan. An overview of value at risk. TheJournal of Derivatives, 4(3):7 – 49, 1997.

[35] R.Y. Rubinstein. Simulation and the monte carlo methods. 1981.[36] Jakob H. Macke, Philipp Berens, Alexander S. Ecker, Andreas S.

Tolias, and Matthias Bethge. Generating spike trains with speci-fied correlation coefficients. Neural Computation, 21(2):397–423,2009. PMID: 19196233.

[37] Yaakov Bar-Shalom, Thiagalingam Kirubarajan, and X.-Rong Li.Estimation with Applications to Tracking and Navigation. JohnWiley & Sons, Inc., New York, NY, USA, 2002.

APPENDIX

Proof: (of proposition 1) Note that

VaRα(X) ≤ X +1√ασ ⇔

Pr

(X < X +

1√α· σ)≥ 1− α

which is easily shown using Chebyshev’s inequality

Pr

(X < X +

1√α· σ)≥

Pr

(∣∣X − X∣∣ < 1√α· σ)

=

1− Pr

(∣∣X − X∣∣ ≥ 1√α· σ)≥ 1− α

Lemma 2. The variance of a random variable X withcompact support [a, b] is bounded by:

Var[X] ≤ (b− a)2

4Proof: Consider the random variable defined by

Y = X − a+b2 . By basic probability theory we have

Var[X] = Var[Y ] = E[Y 2]−(E[Y ]

)2≤(b− a

2

)2

=(b− a)2

4

Lemma 3. Let XN = 1N

∑Ni=1Xi be the sampled mean

of the random variable X with N samples. Let X havecompact support on [a, b], then for any given ε > 0, δ >0

Pr

[∣∣∣XN − X∣∣∣ ≥ ε] ≤ δ

for

N ≥

⌈(b− a)2

4δε2

⌉Proof: Since the samples are iid, we have

Var[XN ] = Var

1

N

N∑i=1

Xi

=1

N2Var

N∑i=1

Xi

=Nσ2(X)

N2≤ (b− a)2

4N



By Chebyshev’s inequality we have

Pr

∣∣∣XN − X∣∣∣ ≥ ε ≤ Var[XN ]

ε2≤ (b− a)2

4Nε2≤ δ

Lemma 4. Let σXN= 1

N

∑Ni=1

(Xi − XN

)2

be thesampled variance of the random variable X with Nsamples. Let X have compact support on [a, b], then forany given ε > 0, δ > 0

Pr

[∣∣∣σ2XN− σ2

X

∣∣∣ ≥ ε] ≤ δfor

N ≥

⌈(b− a)4

8δε2

⌉Proof: By [37], the variance of the sampled variance

is given by

Var[σ2XN

]=

2σ4X

N

By lemma 2, the variance Var[X] = σ2X is bounded by

σ2X ≤

(b− a)2

4

Thus, by Chebyshev’s inequality

Pr

[∣∣∣σ2XN− σ2

X

∣∣∣ ≥ ε] ≤ Var[σ2XN

]

ε2≤ (b− a)4

8Nε2≤ δ

Lemma 5. Given ε > 0, δ > 0, we have for a randomvariable X with compact support on [a, b]

Pr[∣∣σXN

− σX∣∣ ≥ ε] ≤ δ

for

N ≥

⌈(b− a)4

8δε4

⌉Proof: By concavity of

√·, Chebyshev’s inequality

and lemma 4

Pr[∣∣σXN

− σX∣∣ ≥ ε] ≤ Pr

[∣∣∣σ2XN− σ2

X

∣∣∣ ≥ ε2]≤

Var[σ2XN

]

ε4=

2σ4XN

Nε4≤ (b− a)4

8Nε4≤ δ

Proof: (of proposition 3) The first part followsdirectly, since:

σ1,...,k+1 = E[(X1 − X1) · . . . · (Xk+1 − Xk+1)

]=

E [X1 · . . . ·Xk+1]− X1 · E [X2 · . . . ·Xk+1] +

. . .+ (−1)k+1 · X1 · . . . · Xk+1

The second part also follows directly by calculation.Wlog, consider

σ1,...,l = E[(X1 − X1) · . . . · (Xl − Xl)

]=

E [X1 · . . . ·Xl]− X1 · E [X2 · . . . ·Xl] +

. . .+ (−1)l · X1 · . . . · Xl

which is constant for all l ≤ k by the assumption thatall other moments E[Xi1 · . . . ·Xil ] are constant.

As for the third part, let k = 1 and consider a networkwith 3 power lines. The expected value of the systemload shed is:

c · (Pr[X1 = 1, X2 = 1] + Pr[X1 = 1, X3 = 1] +

Pr[X2 = 1, X3 = 1]− Pr[X1 = 1, X2 = 1, X3 = 1]) =

c · (E[X1X2] + E[X1X3] + Pr[X2X3]− Pr[X1X2X3])

which increases by c ·∆φ as E[X1X2] increases by ∆φ.One can generalize this and show that for arbitrary k andnetwork size, the expected system load shed will still beproportional to c ·E[X1 · . . . ·Xk+1]. By the assumptionthat all other moments E[Xi1 · . . . ·Xil ] are constant.

To prove the last claim, we denote ps = Pr[S∗ = c],and pn = 1− ps = Pr[S∗ = 0]. By assumption ps < 1

2 .For this case, the variance of the load shed is simply:

σS∗ = pn(0− S∗)2 + ps(c− S∗)2 = psc2(1− ps)

The latter expression is increasing since

∂

∂pspsc

2(1− ps) = c2(1− 2ps) > 0

for ps < 12 . Since 0 < ∂σS∗

∂ps≤ c2, S∗ = cps, S∗ =

constant+ c∆φ and

∆σS∗ =

∫ φ0+∆Φ

φ0

dσS∗

dφ1,...,k+1dφ1,...,k+1

=

∫ φ0+∆Φ

φ0

∂σS∗

∂ps

∂ps∂φ1,...,k+1

dφ1,...,k+1

=

∫ φ0+∆Φ

φ0

∂σS∗

∂psdφ1,...,k+1

it holds that

∆σS∗ =

∫ φ0+∆Φ

φ0

∂σS∗

∂psdφ1,...,k+1 > 0

∆σS∗ =

∫ φ0+∆Φ

φ0

∂σS∗

∂psdφ1,...,k+1

≤∫ φ0+∆Φ

φ0

c2 dφ1,...,k+1 = c2∆φ



Resource Allocation Problems in Networked Cyber Physical Systemsin Adversarial Scenarios

Sourabh Bhattacharya, Ali Khanafer and Tamer BasarCoordinated Science Laboratory,

University of Illinois at Urbana-Champaignsbhattac, khanafe2, [email protected]

Abstract—In this paper, we address the issue of maliciousintrusion in a network of a team of autonomous vehicles.We present our recent results on ‘smart’ strategies for teamsof autonomous mobile platforms that mutually engage in ajamming attack against each other. Contrary to the existingwork in jamming, we formulate the problem as a zero-sumdifferential game, and provide optimal motion, power allocationand communication strategies for the agents.In our current scenario, we consider the special case of two

teams with each team consisting of two mobile agents. Agentsbelonging to the same team communicate over wireless ad hocnetworks, and they try to split their available power betweenthe tasks of internal communication and jamming the nodesof the other team. The agents have constraints on their totalenergy and instantaneous power usage. The cost function adoptedis the difference between the rates of erroneously transmittedbits of each team. We model the adaptive modulation problemas a zero-sum matrix game which in turn gives rise to a acontinuous kernel game to handle power control. Based on thecommunications model, we present sufficient conditions on thephysical parameters of the agents for the existence of a purestrategy saddle-point equilibrium (PSSPE).

I. INTRODUCTIONThe decentralized nature of wireless ad hoc networks makes

them vulnerable to security threats. A prominent example ofsuch threats is jamming: a malicious attack whose objectiveis to disrupt the communication of the victim network inten-tionally, causing interference or collision at the receiver side.Jamming attack is a well-studied and active area of researchin wireless networks. Unauthorized intrusion of such kind hasstarted a race between the engineers and the hackers; therefore,we have been witnessing a surge of new smart systems aimingto secure modern instrumentation and software from unwantedexogenous attacks.The problem under consideration in this paper is inspired by

real life situations. In the past year, there have been reports ofpredator drones being hacked [11], [15], resulting in intrudersgaining access to classified data being transmitted from anaircraft. Motivated by such incidents, we have analyzed vari-ous scenarios of evading jamming attacks among autonomousmobile agents in [5], [7], [4], [6]. Although, recent resultsin [12], [1] shed light on the issue of security in networkedcontrol systems, the subtle interplay between the limitationson the mobility, power, and communication capability amongagents in adversarial scenarios remains unaddressed. Our

Research supported in part by grants from AFOSR, ARMY MURI, andAFOSR MURI.

current work is a step towards bridging this gap. Finally,there have been recent discoveries of jamming instances inbiological species. It has been reported that females in residentpairs of Peruvian warbling antbirds respond to unpaired sexualrivals by jamming the signals of their own mates, who in turnadjust their signals to avoid the interference [21]. In the future,we hope that our research yields clues to help understand suchinstances of complex behavior among social animals as well.The rest of this paper is organized as follows. In Section

II, we provide our problem formulation. In Section III, wepresent the key results associated with the optimal controlproblem. The saddle-point equilibrium properties of the teampower control problem are studied in Section IV for thespecific example of systems employing uncoded M-quadratureamplitude modulations (QAM). In Section V, we present ourresults on the adaptive modulation schemes. We concludethe paper and provide future directions in Section VI. AnAppendix at the end includes explicit expressions for someof the variables introduced in the paper. In addition to varioussimulation scenarios, a thorough derivation of the results inthis paper is presented in [8] and [14].

II. PROBLEM FORMULATIONConsider two teams of mobile agents. Each agent is com-

municating with members of the team it belongs to, and at thesame time- jamming the communication between members ofthe other team. We consider a scenario where each team hastwo members, though at a conceptual level our developmentapplies to higher number of team members as well. TeamA is comprised of the two players 1a, 2a and Team B iscomprised of the two players 1b, 2b. The agents move on aplane and therefore, have two degrees of freedom (x, y). Thedynamics of the players are given by the following equations:

• Team A:xa

i = faxi

(xa

i,ua

i, t)

yai = fa

yi(xa

i,ua

i, t)

i ∈ 1, 2 (1)

• Team B:xb

i = f bxi

(xb

i,ub

i, t)

ybi = f b

yi(xb

i,ub

i, t)

i ∈ 1, 2 (2)

In the above equations, xi and ui denote vectors representingthe state and control input of agent i, with the superscript (aor b) identifying the corresponding team. The state space ofthe entire system is represented by X R

2 × R2 × R

2 × R2.



Moreover, ui ∈ Ui φ : [0, t] → Ai | φ(·) is measurable,where Ai ⊂ R

pi . f : R2 × Ai × R → R is uniformly

continuous, bounded and Lipschitz continuous in xi for fixedui. Consequently, given a fixed ui(·) and an initial point, thereexists a unique trajectory solving (1) and (2) [2].Now, we describe the physical layer communications model

in the presence of a jammer which is motivated by [20] and hasalso been discussed in [14]. For each transmitter and receiverpair, we adopt the following communications model. Giventhat the transmitter and the receiver are separated by a distanced, and the transmitter transmits with constant power PT , thereceived signal power PR is given by

PR = ρPT d−α, (3)

where ρ depends on the antennas’ gains and the wavelengthof the transmitted signal.The signal-to-interference ratio (SINR) s is given by

s =PR

I + σ, (4)

where σ is the ambient noise level and I is the total receivedinterference power due to jamming. The Bit Error Rate (BER)is given by the following expression:

p(t) = g(s), (5)

where g(·) is a decreasing function of s. The instantaneousBER depends on the SINR, the modulation scheme, andthe error control coding scheme utilized. Communicationsliterature contains closed-form expressions and tight boundsthat can be used to calculate the BER when the noise andinterference are assumed to be Gaussian [10]. For uncodedM-QAM, where Gray encoding is used to map the bits intothe symbols of the constellation, the BER can be approximatedby [18]

p(t) = g(s) ≈ ζ

rQ

(√

βs)

, (6)

where r = log(M), ζ = 4(1 − 1/√

M), β = 3/(M − 1),and Q(.) is the tail probability of the standard Gaussiandistribution.We also assume that players in each team have access to

different M-QAM modulation schemes. We denote the set ofavailable modulation sizes to the players in Team A by Ma

and that available to players of Team B by Mb. The sizes ofthe employed QAM modulation by the teams are Ma ∈ Ma

and Mb ∈ Mb. We assume that Team A can choose amongn different modulation schemes, and Team B chooses from aset of m different schemes, i.e., |Ma| = n, |Mb| = m.To ensure a non-zero communication rate between the

agents of each team, we impose a minimum rate constraintfor each agent: Ra

i (t), Rbi (t) ≥ R, where R > 0 is a threshold

design rate, which we assume is the same for all agents. Theresults can be readily extended to networks of players havinga different value of the minimum design rate.

For an initial position x0 ∈ X, the outcome of the game π,is given by the following expression:

π(x0, ua1 , u

a2 , ub

1, ub2) = N

∫ T

0

[pa1(t) + pa

2(t) − pb1(t) − pb

2(t)]︸︷︷︸

L

dt,

where pai (t) and pb

j(t) are the BERs of agent i in Team Aand and agent j in Team B, respectively; ua

i and ubj are

likewise the control inputs of agents i and j in teams Aand B, respectively; N is the total number of transmitted bitswhich remains constant throughout the game; and T is the timeof termination of the game. pi depends in si, i.e, the SINRperceived by agent i. From (3), si depends on the mutualdistances between the players. Therefore, we can see that theoutcome functional, π, depends on the state of the players andhence, their control inputs. The outcome functional models thedifference in the erroneous communication packets exchangedbetween the members of the same team during the entirecourse of the game. The objective of team A is to minimizeπ and the objective of team B is to maximize it.Let P a

i (t) and P bj (t) denote the instantaneous power levels

for communication used by player i in Team A and player jin Team B, respectively. Since the agents are mobile, there arelimitations on the amount of energy available to each agentthat is dictated by the capacity of the power source carried byeach agent. We model this restriction as the following integralconstraint for each agent

∫ T

0

Pi(t)dt ≤ E. (7)

The game is said to terminate when any one agent runs outof power, that is (6) is violated.In addition to the energy constraints, there are limitations on

the maximum power level of the devices that are used onboardeach agent for the purpose of communication. For each player,this constraint is modeled by the following set of inequalities:

0 ≤ P ai (t), P b

i (t) ≤ Pmax. (8)

At every instant, each agent has to decide on the fractionof the power that needs to be allocated for communicationand jamming. Each player uses its power for the followingpurposes: (1) Communicating with the team-mate, and (2)Jamming the communication of the other team. We assumethat fa and fb are the frequencies at which Team A and TeamB communicate, respectively, and fa = fb.Table I provides a list of the decision variables for the

players that models this allocation. Each decision variable isa non-negative real number and lies in the interval [0, 1]. Thedecision variables belonging to each row add up to one. Thefraction of the total power allocated by the player in row i tothe player in column j is given by the first entry in the cell(i, j). This allocated power is used for jamming if the playerin column j belongs to the other team; otherwise, it is used tocommunicate with the agent in the same team. Similarly, thedistance between the agent in row i and the agent in columnj is given by the second entry in cell (i, j). Since distance



P

P

P

P

P

A

B

a

a aP P

aP

b

b

b

b

1

1

Pa1

1

2

2

1

1

2

2

2

2

2

δ

δ

δ

γ

γ

γ

γ1

aP12γ γ 12

δ P1bδ21

12

12

δ21

11

11

P2

b21

22

22

21

Fig. 1. Power allocation among the agents for communication as well asjamming.

is a symmetric quantity, dij = dji and dij = dji. Figure 1summarizes the power allocation between the members of thesame team as well as between the members of different teams.

TABLE IDECISION VARIABLES AND DISTANCES AMONG AGENTS.

1b

2b

1a

2a

1a γ1

1, d1

1γ1

2, d1

2— γ12, d12

2a γ2

1, d2

1γ2

2, d2

2γ21, d21 —

1b — δ

12, d

12δ1

1, d1

1δ2

1, d2

1

2b δ

21, d

21— δ1

2, d1

2δ2

2, d2

2

In the above game, each agent has to compute the followingvariables at every instant:1) The instantaneous control, ui(t).2) The instantaneous power level, Pi(t).3) All the decision variables present in the row correspond-ing to the agent in Table I.

4) the size of the QAM schemes, Ma or Mb

In the next section, we analyze the problem of computingthe optimal controls for each agent.

III. OPTIMAL CONTROL PROBLEM

From the problem formulation presented in the previoussection, we can conclude that the objective functions of thetwo teams are in conflict. The tuple (ua∗

1 , ua∗2 , ub∗

1 , ub∗2 ) is said

to be optimal (or, in pair-wise saddle-point equilibrium) for theplayers if it satisfies the following conditions:

π[x0, ua∗1 , ua∗

2 , ub1, u

b2] ≤ π[x0, ua∗

1 , ua∗2 , ub∗

1 , ub∗2 ] (9)

π[x0, ua∗1 , ua∗

2 , ub∗1 , ub∗

2 ] ≤ π[x0, ua1 , u

a2 , ub∗

1 , ub∗2 ] (10)

In simple terms, the above equations imply that agents inTeam A are solving a joint optimization problem of mini-mizing the outcome. Similarly, agents in Team B are solvinga joint optimization problem of maximizing the outcome.Moreover, the two teams are playing a zero-sum game againstone another. In this case, the value of the game, denoted bythe function J : X → R, can be defined as follows:

J(x) = π[x0, ua∗1 , ua∗

2 , ub∗1 , ub∗

2 ] (11)

The value of the game is unique at a point X in the state-space.An important property satisfied by the value of the game isthe Nash equilibrium property. The tuple (ua∗

1 , ua∗2 , ub∗

1 , ub∗2 )

is said to be in Nash equilibrium if no unilateral deviationin strategy by a player can lead to a better outcome forthat player. Hence, there is no motivation for the playersto deviate from their equilibrium strategies. In terms of theoutcome of the game, the strategies (ua∗

1 , ua∗2 , ub∗

1 , ub∗2 ) are in

Nash equilibrium (for the 4-player game) if they satisfy thefollowing property:

π[x0, ua∗1 , ua∗

2 , ub∗1 , ub

2]π[x0, ua∗

1 , ua∗2 , ub

1, ub∗2 ]

≤ π[x0, ua∗1 , ua∗

2 , ub∗1 , ub∗

2 ]

π[x0, ua∗1 , ua∗

2 , ub∗1 , ub∗

2 ] ≤

π[x0, ua1 , ua∗

2 , ub∗1 , ub∗

2 ]π[x0, ua∗

1 , ua2 , ub∗

1 , ub∗2 ]

(12)

In general, there may be multiple sets of strategies for theplayers that are in Nash equilibrium. Assuming the existenceof a value, as defined in (11), we can conclude that the Nashequilibrium concept of person-by-person optimality given in(12) is a necessary condition to be satisfied for the value of thegame. Further, obtaining the set of strategies that are in Nashequilibrium yields the optimal strategies for the players. In thefollowing analysis, we assume the aforementioned conditionsin order to compute the optimal strategies.From Isaacs’ conditions [13], the optimal control ui is

obtained by the following expression:

ua∗i = minua

i

∂J∂xa

i

· fai (xa

i , uai , t)

ub∗i = maxub

i

∂J∂xb

i

· f bi (xb

i , ubi , t)

i = 1, 2

Additionally, the gradient of the value function satisfies theretrogressive path equations (RPE) given by the followingpartial differential equation for the players.



Team A

Jxai

= ∇J · ∂faxi

(x)∂xa

i

+ α∑

k=1,2

[sakg′(sa

k)(xaj − xa

i )

(d12)2+

sbkg′(sb

k)Pmaxγik(xa

i − xbk)

(dik)−(α+2)

]

Jyai

= ∇J ·∂fa

yi(x)

∂yai

− α∑

k=1,2

[sakg′(sa

k)(yaj − ya

i )

(d12)2+

sbkg′(sb

k)Pmaxγik(ya

i − ybk)

(dik)−(α+2)

]

Team B

Jxbi

= ∇J · ∂f bxi

(x)∂xb

i

− α∑

k=1,2

[sbkg′(sb

k)(xbj − xb

i )

(d12)2+

sakg′(sa

k)Pmaxδki (xb

i − xak)

(dki )−(α+2)

]

Jybi

= ∇J ·∂f b

yi(x)

∂ybi

+ α∑

k=1,2

[sbkg′(sb

k)(ybj − yb

i )

(d12)2+

sakg′(sa

k)Pmaxδki (yb

i − yak)

(dki )−(α+2)

]

Here, (.) denotes derivative with respect to retrograde time.Since termination is only a function of the power of eachplayer, J is independent of the position of the players on theterminal manifold. Therefore, ∇J = 0 at termination. Thisforms the boundary condition for the RPE.In the next section, we address the problem of power

allocation.

IV. POWER ALLOCATION

Since the players do not communicate, they possess infor-mation only about their own decision variables. This makesthe power allocation problem a continuous kernel zero-sumgame between the two teams:Team A: The objective of each agent is to minimize L.

minP a

i,γi

1,γi

2,γij

L(Ma, M b) ⇒ minP a

i,γi

1,γi

2,γ12

(paj − pb

1 − pb2

︸︷︷︸

Lai(Ma,Mb)

) (13)

subject to: 0 ≤ P ai (t) ≤ Pmax, Ra

i ≥ R

γi1 + γi

2 + γij = 1, γi1, γ

i2, γ

ij ≥ 0

Team B: The objective of each agent is to maximize L.

maxP b

i,δ1

i,δ2

i,δij

L(Ma, M b) ⇒ maxP b

i,δ1

i,δ2

i,δij

(pa1 + pa

2 − pbj

︸︷︷︸

Lbi(Ma,Mb)

) (14)

subject to: 0 ≤ P ib (t) ≤ Pmax, Rb

i ≥ R

δ1i + δ2

i + δij = 1, δ1i , δ2

i , δij ≥ 0

Note that the power allocation vector for 1a denoted byγ = (γ12, γ1

1 , γ12) belongs to the intersection between the

three-dimensional simplex Δ3 and the plane r0, where r0 =γ|γ12 = 1

a1

(2R − 1). The power allocation vectors of otherplayers belong to similar sets.Now we consider the problem of computing the optimal

value of the decision variables for the players. In order to doso, we use preexisting results from continuous kernel gamesfor the existence of a pure strategy saddle-point equilibrium(PSSPE).Theorem 1: [3] Let U be a closed, bounded and convex

subset of Rm, and for each i ∈ N the cost functional J i : U →

R be continuous on U and convex in ui for every uj /∈ U j ,j ∈ N, j /∈ i. Then, the associated N-person nonzero-sumgame admits a PSNE.In [8], we showed that the optimal value of the power

consumption for each player is Pmax. We also showed that theentire game terminates in a fixed time T = E

Pmax

irrespectiveof the initial position of the agents. Moreover we provided asufficient condition for the existence of a PSSPE for the powerallocation game when uncoded M-QAM schemes are used byall agents using Theorem 1.Theorem 2: The power allocation team game has a unique

Nash equilibrium in pure strategies if the following conditionshold for Team A:

g′′(sai ) > 0, (15)

g′′(sb1) +

2

bi

(ci + γi1)g

′(sb1) < 0, (16)

g′′(sb2) +

2

di

(ei + γi2)g

′(sb2) < 0, (17)

and equivalent conditions hold for Team B:

g′′(sbi ) > 0, (18)

g′′(sa1) +

2

li(mi + δ1

i )g′(sa1) < 0, (19)

g′′(sa2) +

2

ni

(oi + δ2i )g′(sa

2) < 0, (20)

where i ∈ 1, 2.The constants b2, c2, d2, ei, li, mi, ni, and oi are obtained

by re-writing the SINR expressions as done above; theirexpressions can be found in the Appendix.Theorem 3: When all players employ uncoded M -QAM

modulation schemes, the power allocation team game formu-lated above has a unique PSSPE solution if the followingcondition is satisfied:

Pmax · max

ρa(d12)−α

Ma − 1,ρb(d12)

−α

M b − 1

< σ2. (21)

For the special case of Ma = M b = M and ρb ≈ ρa = ρ, thecondition becomes

βρPmax

(mind12, d12

)−α< 3σ2. (22)



V. ADAPTIVE MODULATIONThe time-varying nature of the channels due to mobility

emphasizes the need for robust communications. Adaptivemodulation is a widely used technique as it allows for choosingthe design parameters of a communications system to bettermatch the physical characteristics of the channels in orderto optimize a given metric such as: minimizing BER ormaximizing spectral efficiency. In this work, we model theadaptive modulation as a matrix zero-sum game between thetwo teams. We therefore look for an equilibrium solutionwhich would dictate what modulations should be adoptedby the teams at each time instant. The competitive natureof the jamming teams makes our approach to the problemmost practical as any other non-equilibrium solution cannotproduce an improved outcome, relative to that yielded by theequilibrium, for any of the teams.The matrix game is given in (23). The rows are all the

possible actions for players of Team A, and the columnsare the different options available for Team B. The (i, j)-thelement of the matrix is the value of the objective function Lwhen Team A employs Ma = Ma(i), and Team B employsM b = Mb(j).A PSSPE does not always exist for the power alloca-

tion game. The condition for the existence of a PSSPE ismini maxj Aij = maxj mini Aij [3]. In case a PSSPE doesnot exist, we need to look for a solution in the larger class ofmixed-strategies. A pair of strategies Ma∗, M b∗ is said tobe a a mixed-strategy saddle point equilibrium (MSSPE) forthe matrix game if [3]

(Ma∗)TAM b ≤ (Ma∗)T

AM b∗ ≤ (Ma)TAM b∗

For an n × m matrix game, the following theorem from [3],which we state without proof, establishes the existence of anMSSPE for the adaptive modulation game.Theorem 4: The adaptive modulation game admits an

MSSPE.In case multiple MSSPEs exist, the following corollary

becomes essential [3].Corollary 1: If Ma(i1),Mb(j1) and

Ma(i2),Mb(j2) are two MSSPEs of the adaptivemodulation game, then Ma(i1),Mb(j2) andMa(i2),Mb(j1) are also MSSPEs.This is termed the ordered interchangeability property and itsimportance lies in that it removes any ambiguity associatedwith the existence of multiple equilibrium solutions as theteams do not need to communicate to each other whichequilibrium solution they will be adopting. Literature containsdifferent efficient low-complexity algorithms that computesMSSPEs for matrix games, such as Gambit [16]. We referthe interested reader to [3] for a discussion of some of theseapproaches.

VI. FUTURE EFFORTSThis paper has studied the power allocation problem for

jamming mobile teams. The motion of the teams was modelledusing the framework of pursuit-evasion games and the optimal

strategies were derived. An underlying static game was used toobtain the optimal power allocation, where the power budgetof each user is split between communication and jammingpowers. Further, a matrix game was formulated to solvethe adaptive modulation problem. This work focused on theanalysis of teams consisting of two players only. Potentialfuture directions include:

• Computation of Singular Surfaces: In this work, we havecomputed the trajectories based on the necessary condi-tions of optimality imposed by the Isaacs’ conditions.In order to complete the construction of the optimaltrajectories of the agents, we have to identify the singularsurfaces in the state space [3]. This is an interesting futureresearch direction since the construction and nature ofthe singular surfaces would depend on the value of thedecision variables obtained from the power allocationgame.

• Scheduling Schemes: An interesting direction would beexploring scheduling algorithms, similar to the one pro-posed in [9], in which players take turns in communicat-ing or jamming. For example, the users of a given teamthat are closest in distance to the the other team couldallocate all their resources to jamming, while the otherusers allocate all their resources to communicating witheach other.

• Power Control: When multiple users are present, and dueto the broadcast nature of wireless systems, networksbecome interference-limited. The transmission power ofone user can impede the links between other nodes dueto the interference; hence, it is important to regulate thetransmission power of the users in order to, for example,maximize the total capacity of the network.

• Routing: Multihop routing improves the total throughputand power efficiency of a network through relayingpackets via intermediate nodes to their final destination.Because a portion of the energy of each node has to beallocated to jam the other team, determining the optimalroute for transmission becomes a challenge, especiallyin the presence of mobility. An investigation of routingprotocols in the context of games is therefore essential forstudying the overall performance of the networks [19].

• Eavesdropping: When fa = fb, another security issuearises as the ADMs of a given team can receive anddecode messages intended for internal communications ofother teams. To ensure secure communications, each teamwould need to allocate power to jam the eavesdroppers.In fact, a more general scenario is when adversarial teamsconsist of active eavesdroppers: malicious nodes that canact as jammers and eavesdroppers [17].

REFERENCES

[1] S. Amin, A. Cardenas, and S. Sastry. Safe and Secure Networked ControlSystems under Denial-of-Service Attacks. Hybrid Systems: Computationand Control, pages 31–45, 2009.

[2] V. I. Arnold. Geometric Method in the Theory of Ordinary DifferentialEquations. Springer-Verlag, New York, 1983.



A =

Mb(1) Mb(2) ... Mb(m − 1) Mb(m)Ma(1) L(Ma(1),Mb(1)) L(Ma(1),Mb(2)) ... L(Ma(1),Mb(m − 1)) L(Ma(1),Mb(m))Ma(2) L(Ma(2),Mb(1)) L(Ma(2),Mb(2)) ... L(Ma(2),Mb(m − 1)) L(Ma(2),Mb(m))

......

......

...... Team A

Ma(n) L(Ma(n),Mb(1)) L(Ma(n),Mb(2)) ... L(Ma(n),Mb(m − 1)) L(Ma(n),Mb(m))Team B

(23)

[3] T. Basar and G. J. Olsder. Dynamic Noncooperative Game Theory, 2ndEd. SIAM Series in Classics in Applied Mathematics, Philadelphia,1999.

[4] S. Bhattacharya and T. Basar. Differential game-theoretic approach forspatial jamming attack in a UAV communication network. In 14thInternational Symposium on Dynamic Games and Applications, Banff,CA, June 2010.

[5] S. Bhattacharya and T. Basar. Game-theoretic analysis of an aerialjamming attack on a UAV communication network. In Proceedingsof the American Control Conference, pages 818–823, Baltimore, MD,June 2010.

[6] S. Bhattacharya and T. Basar. Graph-theoretic approach to connectivitymaintenance in mobile networks in the presence of a jammer. InProceedings of the IEEE Conference on Decision and Control, Atlanta,GA, Dec. 2010.

[7] S. Bhattacharya and T. Basar. Optimal strategies to evade jamming inheterogeneous mobile networks. In Proceedings of the Workshop onSearch and Pursuit-Evasion, Anchorage, AK, 2010.

[8] S. Bhattacharya, A. Khanafer, and T. Basar. Power alloca-tion in team jamming games in wireless ad hoc networks. In4th International ICST Workshop on Game Theory in Commu-nication Networks (Gamecomm), 2011. Submitted. Available athttp://arxiv4.library.cornell.edu/abs/1101.6030v1.

[9] L. Fu, S. C. Liew, and J. Huang. Fast algorithms for joint power controland scheduling in wireless networks. IEEE Transactions on WirelessCommunications, 9(3):1186 –1197, 2010.

[10] A. Goldsmith. Wireless Communications. Cambridge University Press,Cambridge, U.K., 2005.

[11] S. Gorman, Y. J. Dreazen, and A. Cole. In-surgents hack U.S. drones, December 2009.http://online.wsj.com/article/SB126102247889095011.html.

[12] A. Gupta, C. Langbort, and T. Basar. Optimal Control in the Presenceof an Intelligent Jammer with Limited Actions. In IEEE Conference onDecision and Control (to appear), 2010.

[13] R. Isaacs. Differential Games. Wiley, New York, 1965.[14] A. Khanafer, S. Bhattacharya, and T. Basar. Adaptive resource

allocation in jamming teams using game theory*. In 7th In-ternational workshop on Resource Allocation and Cooperation inWireless Networks(RAWNET), 2011. Submitted. Availabale athttp://arxiv4.library.cornell.edu/abs/1102.1115v1.

[15] D. McCullagh. Predator drones hacked inIraq operations, December 2009. Available athttp://news.cnet.com/301-1009_3-10417247-83.html.

[16] R. D. McKelvey, A. M. McLennan, and T. L. Turocy. Gambit: Softwaretools for game theory. Version 0.2010.09.01, 2010. http://www.gambit-project.org.

[17] A. Mukherjee and A. L. Swindlehurst. Equilibrium outcomes of dynamicgames in mimo channels with active eavesdroppers. In Communications(ICC), 2010 IEEE International Conference on, pages 1 –5, May 2010.

[18] D. P. Palomar, M. Bengtsson, and B. Ottersten. Minimum ber lineartransceivers for mimo channels via primal decomposition. IEEE Trans-actions on Signal Processing, 53(8):2866–2882, 2005.

[19] V. Srivastava, J. Neel, A. Mackenzie, R. Menon, L. Dasilva, J. Hicks,J. Reed, and R. Gilles. Using game theory to analyze wireless ad hocnetworks. Communications Surveys Tutorials, IEEE, 7(4):46 – 56, 2005.

[20] P. Tague, D. Slater, G. Noubir, and R. Poovendran. Linear programmingmodels for jamming attacks on network traffic fows. In Proceedings of6th International Symposium on Modeling and Optimization in Mobile,Ad Hoc, and Wireless Networks (WiOpt’08), Berlin, Germany, Apr 2009.

[21] J. Tobias and N. Seddon. Randomized pursuit-evasion with localvisibility. Current Biology, 19(7):577–582, 2009.



APPENDIX

The following are the expressions of the quantities appear-ing in Theorem 2:

a1 =1

σPmaxρ(d12)−α + δ2

1

(d2

1

d12

)−α

+ δ22

(d2

2

d12

)−α ,

b1 = δ21

(d12

d11

)−α

,

c1 =σ

Pmaxρ(d11)

−α+ γ2

1

(d21

d11

)−α

,

d1 = δ12

(d12

d12

)−α

,

e1 =σ

Pmaxρ(d12)

−α+ γ2

2

(d22

d12

)−α

a2 =1

σPmaxρ(d12)−α + δ1

1

(d1

1

d12

)−α

+ δ12

(d1

2

d12

)−α

b2 = δ21

(d12

d21

)−α

c2 =σ

Pmaxρ(d21)

−α+ γ1

1

(d11

d21

)−α

d2 = δ12

(d12

d22

)−α

e2 =σ

Pmaxρ(d22)

−α+ γ1

2

(d12

d22

)−α

k1 =1

σPmaxρ(d12)−α + γ1

2

(d1

2

d12

)−α

+ γ22

(d2

2

d12

)−α

l1 = γ21

(d12

d11

)−α

m1 =σ

Pmaxρ(d11)

−α+ δ1

2

(d12

d11

)−α

n1 = γ12

(d12

d21

)−α

o1 =σ

Pmaxρ(d21)

−α+ δ2

2

(d22

d21

)−α

k2 =1

σPmaxρ(d12)−α + γ1

1

(d1

1

d12

)−α

+ γ21

(d2

1

d12

)−α

l2 = γ21

(d12

d12

)−α

m2 =σ

Pmaxρ(d12)

−α+ δ1

1

(d11

d12

)−α

n2 = γ12

(d12

d22

)−α

o2 =σ

Pmaxρ(d22)

−α+ δ2

1

(d21

d22

)−α



Enhancing Cyber-Physical Security through Data Patterns

Richard Chow, Ersin UzunPalo Alto Research Center

Palo Alto, CA USArchow, [email protected]

Alvaro A. Cardenas, Zhexuan Song, Sung LeeFujitsu Laboratories of America

Sunnyvale, CA USalvaro.cardenas-mora, zhexuan.song, [email protected]

Abstract—In this position paper, we propose a data-drivenapproach for security management in a network that interactsor receives inputs from physical systems – including humanbehavior. Our goal is to leverage the unique features of cyber-physical systems. In particular we propose: (1) the use ofhistorical data from physical systems and human behaviors toenable anomaly detection, (2) the use of contextual data frommultiple and diverse sensor readings to obtain a higher-levelcollective vision of the network for better event correlation anddecision analysis, and (3) the use of physical sensor data andhuman behavior to enable fine-grained, dynamic access controland implicit authentication. We outline use cases describinghow our ideas can be applied in the Home Area Network(HAN).

Keywords-home area networking; authentication; security;privacy; anomaly detection

I. INTRODUCTION

As cyber-physical systems become more pervasive andimportant they will increasingly become the focus of attacks.One prominent recent example is Stuxnet. The Stuxnet wormtargets industrial controllers and is believed to target theuranium enrichment infrastructure in Iran [6], [2], [3]. Inaddition, the rise of ubiquitous computing and the SmartGrid imply the deployment of billions of smart sensorsand actuators embedded in our social infrastructures. Byrelying on information technology and networks, these newdeployments will expose our social infrastructure to regularcomputer vulnerabilities available to an ever-growing set ofmotivated and highly-skilled attackers.

While many existing security mechanisms can be appliedto cyber-physical systems, in this paper we explore someunique ways to enhance the security of these systems byleveraging the diverse physical and human behavior in-formation collected by these systems. Examples include:(1) the use of historical data from physical systems andhuman behaviors to enable anomaly detection, (2) the use ofcontextual data from multiple and diverse sensors to obtaina higher-level collective vision of the network for betterevent correlation and decision analysis, and (3) the use ofphysical sensor data and human behavior to enable implicitauthentication and fine-grained, dynamic access control.

For point (1), we propose the use of device and humanprofiles generated from a data driven approach. For example,a historical profile of the alarm system of a house that leaves

the alarm on every night might generate an incident reportif one day the alarm is shut off at 3 AM in the morning.In general, previous work has shown that cyber-physicalsystems might benefit from anomaly detection techniques,as physical processes give off large amounts of data thatare clearly non-deterministic in nature, and yet somewhatpredictable [7].

While the use of physical process data has already beenproposed as a way to detect computer attacks on cyber-physical systems [9], we believe that in order to obtain abetter and more complete view of the system, the data ofmultiple and diverse sensors needs to be aggregated andcombined to generate usable models with low false alarmrates. Therefore, we propose for point (2), the use of amaster controller that collects multiple data sources andintegrates them into the proper context. For example, whileshutting off the home alarm system at 3 AM in the morningmight be anomalous in itself, it might not be anomalousif the electric car associated with the home has also justpulled into the garage and was plugged into its charger. Weargue in this position paper, especially in the case of theHome Area Network (HAN), that higher level patterns thatinvolve collective behavior of devices and users should alsobe analyzed for anomalies. User behavioral patterns are alsonon-deterministic and yet somewhat predictable.

Finally, for point (3), we believe that the user and deviceprofiles obtained by (1), and the contextual informationobtained by (2) can be used to enhance user authentication,dynamic access control, revocation, and fine-grained accesscontrol policies. For example, a typical home area networkcurrently provides full security or no security at all. It is verydifficult for users at home to set the appropriate fine-grainedaccess control to devices at home, and the proper revocationmechanisms. For example, if a neighbor brings their laptopto a home and is given the keys to access the wirelessnetwork, the neighbor will remain a valid (authenticated)user of the network even in his own home. We propose theuse of device profiles for implicit authentication, allowingthe revocation of devices that do not match our experience.

In the remainder of this paper we explore in more detailour arguments by providing additional use cases, a generalsystem description, and a discussion of the security, usabil-ity, and performance of the system.



II. ADDITIONAL USE CASES

Collective Device BehaviorHigher-level components of cyber-physical systems may

also exhibit anomalous behavior, and this behavior is bestanalyzed given appropriate context. A simple example isthat intensive operation of an air-conditioning system is notanomalous given a heat wave, but may be under normalweather conditions. A more complex example is that shuttingoff the home alarm system at 3 AM might be very normal ifthe electric car has just pulled into the garage and is pluggedinto its charger.

Finding these sorts of rules may require sophisticatedmachine learning apparatus, but the advantage is moreintelligent security decisions and fewer false positives. Inlarge control systems this feature is usually called situationalawareness.

Consider for example a home network that controls ap-pliances, heating and cooling, and lighting. In most net-works available to consumers, there is typically a lack offine-grained access control: access to the central controllerimplies total control of the system. However, adding suchfine-grained access control runs the danger of making thesystem unusable.

Instead, we propose adding a behavioral analysis sys-tem on top of the controller security system. We amassa collective baseline of historical settings, readings, andnetwork traffic. The baselines can be used to detect intrusion,both cyber and physical (as well as broken or miscon-figured equipment). Detected anomalies in a device mayresult in revocation from the network, or a request for re-authentication or two-factor authentication. Revocation inhome area networks is not an easy task to accomplish incurrent mass market deployments.

Collective contextual information can also be used to de-sign more flexible access control mechanisms. For example,with the proliferation of surveillance cameras in everydaylife and webcams at home computers, the number of un-secured cameras on the Internet has become an increasingcause of concern. Bloggers have reported the ability to tapinto thousands of raw webcam feeds with a few simpleGoogle searches, and the Spanish police arrested a suspecton charges of developing a computer virus that can activate awebcam without the owner’s permission [5]. We propose thatthe context for camera networks might be used in the accesscontrol policy. For instance, the context contains propertiessuch as the content of the video stream (e.g., people orthe type of events happening), when the access request ismade, and location of the subject. These properties add theflexibility of describing rules such as “only people in a roomshould be able to control the camera located in that room.”Camera networks might also be part of complex networksincluding other sensors such as audio, temperature, humidityetc. The readings of these sensors might give additional

context relevant to the access control policy. For example,emergency response teams (firefighters or paramedics) maybe allowed to access any web camera if the fire alarm is on.

User behaviorThe cyber-physical systems of tomorrow are all envisionedto be connected and remotely controllable. With the adventof the smart grid, smart appliances within a consumer’s HANcan be controlled remotely by consumers and potentially byutility companies. See, for example, [1] for the latest iPhoneapplications for remotely controlling the home and [4] forthe PG&E SmartAC program. Given the ubiquity of Internet-connected smartphones, it is clear that cyberattacks againstsuch devices is a way to attack the HAN. In fact, as arguedby Neuman [10], creating a smartphone botnet may be arelatively easy way to generate traffic affecting the largescale power grid.

Given the possibility of direct intrusion into the home, theproper authentication of users becomes even more critical.One approach is to protect each remote control instance,for example requiring traditional authentication mechanisms(such as a password, one-time-password, or user certifi-cate) for each remote control directive. This strategy mayprovide a reasonable level of security when used properly(for example, unpredictable passwords, secure storage ofcertificates, etc.), but does not scale well when the numberof devices increases. Imagine, for a moment, the follow-ing common scenario: you have many personal Internet-connected devices such as a laptop, desktop, smartphone andeven an Internet-connected TV that you use to control all theappliances in your HAN, such as the air-conditioning unit,alarm, electric car, lights and other smart appliances. Havinga separate password for all these devices and authenticatingto each every time you need access is very unusable whilehaving the same password for all of them would significantlyreduce the security.

Another approach is to authenticate the user once andprovide general remote control capabilities after this au-thentication, at least for some limited period of time. Thesetechnologies reduce the burden on the user by requiring, say,only one password. However, these technologies not onlyease access for legitimate users but ease it for attackers aswell. A natural answer is to add an additional authentica-tion factor, but traditional second factors such as hardwaretokens or biometrics reduce usability, especially on a mobileplatform.

We propose that patterns of user behavior can be usedas a second factor, in line with the theme of this paper ofusing behavioral patterns to increase security. For mobilephones in particular, the behavioral patterns can be basedfrom rich data, such as location, calls, SMS, and web sitevisits. See [8] and [13] for some work in this direction, calledimplicit authentication. This approach does not directlyaddress malware on the phone, but reduces the risk of a



lost phone providing a gateway into the home network. Oneof the themes in this paper is to generalize the techniquesof implicit authentication beyond humans to a collection ofdevices.

We remark also that improving authentication on mobilephones is critical because smartphones might become the defacto remote control device. We believe the phone will alsobe involved in various enrollment protocols in the HAN.For instance, in [12] the phone is the party that mediatesthe pairing between two devices.

III. SYSTEM DESCRIPTION

We outline here one possible system architecture basedon the above ideas. See Figure 1 for a representation inthe home network. Appliances are enrolled into a star-shaped network centered at a Master Controller. Commandsto each appliance go through and are vetted by the MasterController. The Master Controller contains an Authentica-tion Service to authenticate commands, and also a PatternAnalyzer to build data models and evaluate recent data basedon the models.

Appliances periodically upload data to an authenticationservice. These include settings, sensor readings, status, etc.In the case of a device with a user interface, user-relateddata may be recorded as well. For instance, data such aslocation, call logs, SMS logs, and web sites visited (allsuitably anonymized and obfuscated) might be collected fora cell phone. This data is stored in the Database and used bythe Pattern Analyzer to generate a model for the data. Forinstance, a simple model might say that the air conditioneris usually on during the afternoon but off at night.

Note that the smart meter may also upload data to theMaster Controller. In this way, patterns of household electricdata may be incorporated into the Pattern Analyzer. Asdetailed in [11], these patterns can identify the use of evennon-smart appliances, as well as reconstruct daily routines,including sleep habits of inhabitants.

When the user wishes to change a setting on a device,say, through an application on his phone, the applicationconnects with the Master Controller with the usual password.Depending on the policy enforced for the device, the MasterController may measure how well recent behavior on thephone matches historical behavior. If the behavior is verydifferent, another credential may be requested. The policymay also request another credential if a change is requestedthat is unusual for that device, in conjunction with all otheravailable data. For example, turning on the air conditionerin the middle of the night may be unusual, but especially ifnobody is in the house.

We allow the possibility of a device directly interactingwith another device, not going through the Master Con-troller. For instance, in Figure 1, a cell phone might be usedto control a car without going through the Master Controller.In this case, the user has chosen to drop back to unaided

traditional measures, without the added assurance providedby the Master Controller. The user may not want to rely onthe Master Controller being operational, for instance.

A novel aspect of the system is that the data uploaded tothe Master Controller must be treated as sensitive. Knowl-edge of the data would be equivalent to knowing a securitykey in a traditional system, since the algorithms used bythe Master Controller must be considered public. Hence,communication channels to the Master Controller should besecured, and we expect that data will not reside in the longterm on individual devices, limiting the risk of compromisesof individual devices. We recognize, however, that for somedevices historical data facilitates daily operation of thedevice (for example, the recent calls on a cell phone).

A. Anomaly Detection

One key question is how well anomaly detection worksfor this kind of data. Experiments described in [13] indicatethat user modeling on cell phones is promising. With trainingdata of around one or two weeks and four types of datacollected (web sites, call logs, SMS, location), one can setthresholds such that an adversary would be detected withinabout 10 phone usages, while the legitimate user is identifiedas illegitimate about every 130 usages. It is reasonable tobelieve (but have no hard evidence) that most devices wouldhave less variance and less richness in their data and sowould be more predictable.

In [13], the approach is to treat each type of data collectedseparately. For each type of data, the software builds a modelbased on training data and then evaluates recent data againstthe model, deriving a score. See Figure 2. The scores foreach data type are combined, assuming independence ofeach data type. To enable the discovery of the complexrules and interactions between HAN appliances, it wouldbe necessary to detect and model the actual correlationsbetween data types, going beyond the work in [13].

LearningAlgorithm

RecentBehavior

Score

PastBehavior

UserModel

Figure 2. Scoring Data

IV. SUMMARY AND DISCUSSION

In this position paper, we propose data-driven securitymanagement for cyber-physical systems. We argue that inthe cyber-physical systems of the near future, such as HANs,better and more usable anomaly/attack detection can onlybe built if the collective information from all devices as



Policy Engine

AuthenticationService

Master Controller

Pattern Analyzer

Database

Figure 1. System Architecture

well as the behavior of their user and the inferred con-text are integrated into the decision making process. Fromthe examples given in previous sections, we indicate howthis collective high-level vision can detect anomalies fordevices that operate within usual limits when consideredindividually, or how false alarms can be prevented with acollaborative and contextual view.

On the other hand, there are some obvious limitationsto such a system. For example, one real advantage of acollaborative view over a range of smart devices is the abilityto capture the behavioral patterns of a human user and makecontextual decisions based on these patterns. However, hu-mans do not always follow established behavioral patterns,and in the case of multiple users interacting with the samesystem, the system may be slow or ineffective in detectingattacks/anomalies due to the lack of one clear behavioralpattern. Having one master controller is another weaknessof the system as it may be a single point of failure, butthis weakness can be easily overcome by employing fault-tolerance practices such as replication.

Note that decision making in the proposed system is basedon automated modeling which requires sufficient initialtraining data and also dynamically evolves with time to bemore effective. However, the ongoing training process makesthe system vulnerable at the beginning and can possibly beexploited by smart attackers to slowly evolve the system toan insecure state if it is not designed carefully. In the caseof user authentication with behavioral data, one important

limitation is the reaction time of the system to a userchange. In other words, it is not possible for the system todetect changes in the user behavior before certain numberinteractions between the system and an attacker. Thus, weonly recommend this kind of implicit authentication as asecond factor or as a low-security authentication mechanism.

One main motivation is usability. Since security is not themain objective when users interact with a system, they getirritated when they are bothered with security mechanisms(for example, requests for an additional credential) whiletrying to achieve their goal (for example, turning on the air-conditioner). With this system, the overall authentication canbe made more usable by reducing the requests for additionalcredentials. False alarms by security systems are one of thebiggest issues of current security systems, affecting bothusability and security. Too many false alarms not only makethe system less usable, but also seriously damages security(for example, nobody pays attention to car alarms). Hence,usability of our system is intimately tied to the rates of bothfalse positives and false negatives of our anomaly detectionalgorithms; acceptable rates would need to be determinedwith user studies.

REFERENCES

[1] Apps for Remote Control. On the Web at http://www.apple.com/iphone/apps-for-everything/remote.html.

[2] Israeli Test on Worm Called Crucial in Iran Nuclear Delay.On the Web at http://www.nytimes.com/2011/01/16/world/middleeast/16stuxnet.html.



[3] Malware Aimed at Iran Hit Five Sites, Report Says.On the Web at http://www.nytimes.com/2011/02/13/science/13stuxnet.html.

[4] PG&E SmartAC Program. On the Web athttp://www.pge.com/myhome/saveenergymoney/energysavingprograms/smartac/.

[5] Spanish police nab suspected creator of webcam Trojan. Onthe Web at http://www.computerworld.com/s/article/99034/Spanish police nab suspected creator of webcam Trojan.

[6] Stuxnet. On the Web at http://en.wikipedia.org/wiki/Stuxnet.

[7] A. A. Cardenas, S. Amin, Z.-S. Lin, Y.-L. Huang, C.-Y.Huang, and S. Sastry. Attacks against process control sys-tems: Risk assessment, detection, and response. In ASIACCS,2011.

[8] M. Jakobsson, E. Shi, P. Golle, and R. Chow. Implicit Au-thentication for Mobile Devices. In HotSec ’09: Proceedingsof the 4th USENIX Workshop on Hot Topics in Security, 2009.

[9] Y. Mo and B. Sinopoli. Secure control against replay attacks.In Proceedings of the 47th annual Allerton conference onCommunication, control, and computing, Allerton’09, pages911–918, Piscataway, NJ, USA, 2009. IEEE Press.

[10] C. Neuman. Challenges in Security for Cyber-Physical Sys-tems. On the Web at http://cimic.rutgers.edu/positionPapers/CPS-Neuman.pdf.

[11] E. Quinn. Privacy and the New Energy Infrastructure. Onthe Web at http://www.townsend.com/Templates/media/files/media%20coverage/Elias%20Quinn%20-%20SmartGridPriv.pdf.

[12] N. Saxena, M. B. Uddin, and J. Voris. Universal devicepairing using an auxiliary device. In Proceedings of the4th symposium on Usable privacy and security, SOUPS ’08,pages 56–67, New York, NY, USA, 2008. ACM.

[13] E. Shi, Y. Niu, M. Jakobsson, and R. Chow. Implicitauthentication through learning user behavior. In ISC, pages99–113, 2010.



Novel Reconfigurable Silicon Physical UnclonableFunctions

Yingjie Lao and Keshab K. ParhiDepartment of Electrical and Computer Engineering,

University of Minnesota, Twin Citieslaoxx025, [email protected]

ABSTRACTPhysical Unclonable Functions (PUFs) are novel circuit primi-

tives which store secret keys in silicon circuits by exploiting un-controllable randomness due to manufacturing process variations.Previous work has mainly focused on static challenge-response be-haviors. However, it has already been shown that a reconfigurablearchitecture of PUF will not only enable PUFs to meet practical ap-plication needs, but also can improve the reliability and security ofPUF-based authentication or identification systems. In this paper,we propose several novel structures for non-FPGA reconfigurablesilicon PUFs, which do not need any special fabrication methodsand can overcome the limitations and drawbacks of FPGA-basedtechniques. Their performances are quantified by the inter-chipvariation, intra-chip variation and reconfigurability tests.

Keywords: Physical Unclonable Function, Reconfigurable Ar-chitecture, Hardware Security, Counterfeit IC Chip Prevention

1. INTRODUCTION

1.1 Physical Unclonable FunctionIn today’s world, as electronic devices become increasingly in-

terconnected and pervasive in people’s lives, security, trustwor-thy computing, and privacy protection have emerged over the pastdecade as hardware design objectives of great significance. Tradi-tionally, secret keys, which are used as unique identifiers, are em-bedded into integrated circuits (ICs) in a ROM immediately aftermanufacturing. Unfortunately, digital keys stored in a non-volatilememory are vulnerable to physical attacks. Several invasive andsemi-invasive physical tampering methods have been developed,these include techniques such as micro-probing (access to the sili-con to manipulate the internals of system), power analysis (predictthe secret keys from power consumption analysis) and so forth.These approaches have made it possible to learn the ROM-basedkeys through attacks and compromise systems by using counterfeitcopies of the secret information.

The described problem has become more intense recently, andthis motivated the idea of using intrinsic random features of phys-ical objects for identification and authentication. The concept ofphysical unclonable function (PUF) proposed in [1–3] has suc-cessfully addressed the problems faced by traditional techniques.PUF has been defined as a function that exploits the unique in-trinsic uncontrollable physical features by process variations dur-ing manufacturing. Physical Unclonable Functions enable signifi-cantly higher secure authentication by extracting secrets from com-plex properties of a physical material rather than storing them innon-volatile memory. Due to the uncontrollable random compo-nents, PUFs are easy to measure but almost impossible to clone,predict, or reproduce. Furthermore, it is infeasible for an adversaryto mount an attack to counterfeit the secret information withoutchanging the physical randomness. Taking coating PUF as an ex-ample, which is a function built in the top layer of an IC by fillingthe space between and above the comb structure with an opaque

material and randomly doping with dielectric particles, any physi-cal attack on a coating PUF would damage the protective coatingand destroy the cryptographic key.

1.2 Related Work and Our ContributionThe first PUF in the literature is the optical PUF [1], which uti-

lizes the randomness in the placement of the light scattering parti-cles and the complexity of the interaction between the laser and theparticles. After that, several PUF hardware structures have beenproposed [2–6]. Most PUFs use conventional silicon techniquesso that they do not require any special fabrication and can be eas-ily integrated into IC chips, except a few types such as coatingPUF and magnetic PUF. Among these PUFs, silicon PUFs are ofgreat interest, as these exploit manufacturing variability of wire de-lay to generate a unique challenge-response mapping for each IC.These unique properties of each IC are easy to measure through thecircuits but hard to copy without changing the challenge-responsepairs (CRPs).

The delay-based silicon PUFs in previous work have always con-sidered a static challenge-response behavior. In those protocols, thePUF should always generate the same or error tolerated response.Unfortunately, recent analysis has demonstrated that those PUFstructures are vulnerable to several attack methods including em-ulation, replay (man-in-the-middle attack), and reverse engineer-ing [7]. Moreover, updatable cryptographic keys are very attractivein some applications [8]. Therefore, a dynamic PUF that can al-ter the CRPs every time the data is modified to prevent the hiddeninformation leaked out is very desirable.

Our work builds on the prior work of the PUF community. Inthis paper, we mainly focus on the design of reconfigurable siliconPUFs. We propose several novel reconfigurable PUFs and analyzetheir performance. We also examine the security of different PUFstructures. The key idea in our approach is that we try to makeCRPs updatable. By doing this, the challenge-response behavior ofa PUF can be altered to generate highly secure hardware system.Furthermore, we discuss the techniques to improve the reliabilityof silicon PUFs.

1.3 Paper OrganizationThe rest of the paper is organized as follows. In Section 2, we

introduce the background of silicon PUFs, and then present a briefoverview of previous works on reconfigurable PUFs and discusstheir disadvantages and limitations. In Section 3, we describe ourmanufacturing process variation model for silicon PUFs and theexperimental methods. Section 4 discusses several novel reconfig-urable PUF designs and analyzes their efficiency. Section 5 demon-strates the performance of the discussed reconfigurable structuresby providing experimental results on SPICE simulation. Finally,Section 6 concludes the paper.

2. BACKGROUND

2.1 Silicon Physical Unclonable Function



Silicon PUFs exploit the delay variations of CMOS logic compo-nents to generate a unique response for each IC. There are two maintypes of delay-based silicon PUFs: Ring Oscillator (RO) PUF [2]and Multiplexor (MUX) PUF [9]. However, the MUX PUF is moresecure than the RO PUF, as the frequencies of the ring oscillatorscan be relatively easily evaluated by attackers; moreover, a MUXPUF is more suitable for resource-constrained applications. Insteadof duplicating the hardware N times as in a RO PUF, we can useN different challenges to obtain a N-bit long response in a MUXPUF, as illustrated in Figure 1. This kind of silicon PUF con-sists of N stages MUXs and one arbiter which connects the finalstage of the two paths. MUXs in each stage act as a switch to ei-ther cross or straight propagate the rising edge signals, based onthe corresponding challenge bit. Each MUX should be designedequivalently, while the variations will be introduced only duringmanufacturing process. Finally, the arbiter (always simply a D flip-flop) translates the analog timing difference into a digital value.For transistors, manufacturing randomness exists due to variationsin transistor length, width, gate oxide thickness, doping concentra-tion density, metal width, metal thickness, and ILD (inter-level di-electric) thickness, etc [10]. These manufacturing variations showa significant amount of variability, which are sufficient to gener-ate unique challenge-response pairs for each IC by comparing thedelays of two paths.

Figure 1: Silicon MUX Physical Unclonable Function.

2.2 Feed-Forward StructureA feed-forward structure of silicon PUF has been proposed in [11]

to prevent attacks by linear modeling. Figure 2 shows one basicstructure of feed-forward MUX PUF, which uses the racing resultof an intermediate stage as the select signal for a block of MUXsin a later stage. This structure provides nonlinearity to the origi-nal PUF, which increases the complexity for numerical modelingattacks. However, the reliability of the PUF has been degraded inthis feed-forward structure since an error in the output of an in-ternal feed-forward arbiter caused by environmental variation canincrease the noise probability in the final response.

Figure 2: Feed-Forward Silicon MUX PUF Structure.

2.3 Reconfigurable PUFA reconfigurable PUF is a PUF with a mechanism to update its

challenge-response pairs. It should preserve the properties of orig-inal PUF but have unpredictably different challenge-response be-haviors after every reconfiguration. The first reconfigurable PUF [9]

was presented in 2004, which is an arbiter-based PUF built withfloating gate transistors. It was also shown that the reconfigurabil-ity for a PUF is a very desirable characteristic.

Recently, two types of reconfigurable PUFs have been proposedin [12]: reconfigurable optical PUF and phase change memorybased reconfigurable PUF. These two types of PUF can update theirCRP mappings while inheriting the properties of the original PUF.However, these PUFs have certain limitations and constraints forpractical use; for instance, the reconfigurable optical PUF needs aspecial material, which is not widely used in industry.

Several implementations of reconfigurable silicon PUFs have alsobeen published [8, 13–15]. However, all of these efforts are basedon the reconfigurability of FPGA, and most of them are built onthe Ring Oscillator PUF. In these existing FPGA-based solutions,many assumptions are required as significant amount of informa-tion regarding the underlying VLSI layout of the reconfigurablefabric is lost. Moreover, many PUF designs require a symmetricalrouting that is difficult to implement on an FPGA platform as a de-signer can only manipulate the higher level design blocks such asthe LUTs, the memory blocks, and the connection matrices. Ad-ditionally, it is possible to reverse the reconfigurations of contem-porary FPGAs, which makes the FPGA-based reconfigurable PUFmore vulnerable to attacks.

Therefore, non-FPGA based architectures for reconfigurable sil-icon PUF are very desirable. Our major contribution is to presentseveral novel structures of silicon PUFs which can be implementedon non-reconfigurable hardware, that is which can be designed attransistor level and fabricated and integrated into chips. Moreover,we also target on the security and the reliability perspectives of thereconfigurable PUFs.

3. METHODOLOGY

3.1 PUF ModelAs shown in Figure 1, a MUX PUF consists of a sequence of

N-stage MUXs and an arbiter. The rising edge signal will excitethe two parallel paths simultaneously. The actual propagated pathswill be determined by the external applied challenge bits. After thelast stage, the arbiter will generate the output bit by comparing thearrival time of the two different paths. It has become standard tomodel the MUX PUF via an additive linear delay model. Accord-ing to the efforts in the field of Statistical Static Timing Analysis(SSTA) [10], the manufacturing process parameter variations fortransistors can be modeled by a Gaussian distribution. As a result,the variations of delay will also be approximately Gaussian.

Manufacturing process variations can be classified as the fol-lowing two categories: inter-die variations and intra-die variations.Inter-die variations refer to parameter variations that affect all de-vices equivalently across a single die, while intra-die variationshave different effects on the devices within the same chip. It is alsoimperative to consider the correlation of these variations to increasethe accuracy of the model. A very widely used model for delay spa-tial correlation of process variations is the Grid model [10], whichassume high correlations among the devices in nearby grids andlow correlations in faraway grids, as manufacturing process varia-tions are more likely to have similar effects on closer devices. Ad-ditionally, experimental results have already shown that the inter-chip variation across the wafer is similar to that within a singlewafer [9]. Moreover, the output of the arbiter in silicon PUF isonly based on the difference of two selected paths. Therefore, theinter-chip variations are the primary factors that contribute to therandomness of response for each IC, while these die-to-die andwafer-to-wafer manufacturing variations will have minimum effecton the output response.

For simplicity, as every MUX is designed equivalently in a MUXPUF, we can model the delay of each single MUX as i.i.d. randomvariable Di, which follows N(μ, σ2); therefore, the total delay of



the N stages will be N(Nμ,Nσ2). Since the output of arbiterwill only depend on the delay difference between the two paths,the time difference will also follow a Gaussian distribution Δ ∼N(0, 2Nσ2).

We denote the delay in the top path of the i-th stage as Dti, thedelay in the bottom path of the i-th stage as Dbi, and the challengebit for each stage as Ci. Thus the delay difference of the i-th stagewill be:

Dti −Dbi ∼ N(0, 2σ2)

Then if the challenge is 0, then the delay difference added into thewhole paths will be Dti −Dbi; otherwise, if the challenge bit forthe i-th stage is 1, the additive delay difference will be Dbi −Dti.It can be expressed as:

Δi = (−1)Ci(Dti −Dbi) ∼ N(0, 2σ2)

As a result, the arrival time difference between the two inputs ofthe arbiter is:

Δz =N∑

i=1

(−1)Ci(Dti −Dbi) ∼ N(0, 2Nσ2)

Thus, the final response is:

r = sign(Δz)

where we use the convention that r = sign(a) = 0 when a < 0,and r = sign(a) = 1 when a ≥ 0.

3.2 Simulation ModelIn our experiment, we use simulation method to test and analyze

the performances of PUFs instead of fabrication method. There areseveral advantages of using simulation method: First of all, fabrica-tion is relatively expensive. Second, a good simulation method canbe used as a pre-fabrication test, which can predict the efficiency ofa new PUF design before fabrication. Moreover, we can analyze allthe possible properties and characteristics of the PUFs under differ-ent environmental conditions. Additionally, it is also convenient tofollow the shrinking of technology scale.

In our simulation, we apply the Gaussian model which has al-ready been described in Section 3.1 for manufacturing process vari-ations. We set up the process parameters and their max percentagesof deviations based on the predictions from [16, 17]. For spatialcorrelation, we assume perfect spatial correlations within one sin-gle MUX. The process variations will have the same effect on thePMOS and NMOS devices in each MUX, while the parameter vari-ations among different MUXs have no correlation.

In our simulation result, the total delay deviation of 100 stages is≤ ±0.4%. Since

σz/μz =√

(1/N)(σ/μ)

and μz increases linearly with N, we can conclude that our resultconforms with other published results of 65nm technology, basedon the experimental results in [8] that 3σ/μ ≈ 5% for a singlestage of MUXs. Furthermore, our simulation result of inter-chipvariation leads to a Hamming distance range from 22 to 59 bits fora total of 100 stages, while the intra-chip variation is 5.8 bits onaverage, with a maximum value 13 bits. These results are also inagreement with published results for fabricated chips. Thus, webelieve that our simulation delay model is consistent with the in-dustrial manufacturing process variations.

4. NOVEL RECONFIGURABLE PUFSIn order to add reconfigurable property into general MUX based

silicon PUFs, we must make the challenge-response pairs (CRPs)reconfigurable, which can be used to update the database for anauthentication system. The methods can be classified into two cat-egories:

(a) Make the challenge-response pairs reconfigurable directly,by adding some extra circuits into the structure, but withoutconfiguring the main PUF circuit. This can be achieved byutilizing some techniques to pre-process the challenge beforeapplying to PUF or pre-process the response before using itfor authentication.

(b) Make the PUF circuit reconfigurable, therefore the challenge-response pairs will be reconfigurable as well.

We propose several novel non-FPGA reconfigurable PUFs imple-mentations for the above two categories, which would be more suit-able for practical use than FPGA-based techniques. Furthermore,we address the reliability and the security of the PUF performance,as some information of the hidden secrets that an adversary cantake advantage of may leak out during reconfigurations.

4.1 Reconfigurable Challenge and/or ResponseStructures

The reconfigurable structures of PUF are built on the prior workin Physical Unclonable Function, which can also be applied to vari-ous types of silicon PUFs as well as other challenge-response basedPUFs. Our goal is to develop reconfigurable PUF which is a PUFwith a mechanism to transform it into a new PUF with an un-predictable and uncontrollable challenge-response behavior, evenif the challenge-response behavior of the original PUF is alreadyknown. Additionally, the new PUF inherits all the security proper-ties of the original one.

An early reconfigurable design PUF [9] in the literature treatedsome challenge bits as the configure data. As an example, the last10 bits of a 100-bit challenge can be fixed as the configure data,leaving only 90 bits for actual challenge. A user can update theCRPs by applying another 10-bit stream to the last 10 stages ofthe PUF. However, it is very clear that the reconfigured PUF willhave high correlation between different configurations and will bevulnerable to attacks, as this method is similar to adding a certaintime difference between the two paths or introducing an interval be-tween the two rising edge signals. Even worse, the performance ofthe PUF will be greatly degraded, if the cumulative variations in thelast 10 stages are relatively large. Due to these disadvantages, thisarchitecture of reconfigurable PUFs cannot generate unpredictablechallenge-response behaviors.

Intuitively, adding reconfigurable elements before the challengesapplied to the PUF can definitely make the PUF reconfigurable. Atthe same time, the performance of the original PUF will be pre-served. The main structure of this type of reconfigurable PUF isshown in Figure 3.

Figure 3: Reconfigurable Challenge and Response PUF Structure.

4.1.1 PUF with LFSR

We can adopt the linear feedback shift register (LFSR) as thereconfigurable component. Such a structure is shown in Figure 4.LFSR is an important part of sequence cipher and can be used togenerate pseudo-random key stream. We can apply different seedsto the IC to generate various random patterns. Furthermore, we canalso alter the characteristic polynomial by utilizing the properties



of reconfigurable linear feedback shift register [18, 19]. Such ca-pability makes it extremely difficult for adversaries to obtain PUFsignature. It is important to point out that we can improve the se-curity of the PUF system, by benefiting from the property of theLFSR in cryptography.

Figure 4: PUF Structure of Using LFSR to Configure the Challenge.

4.1.2 PUF with Hash Function

Hash function is a kind of "one-way" function, which means itis easy to compute the hash value for a given message, but hard tofind a message with a given hash. Due to the random property ofhash function, we can employ a hash function as the reconfigurableelement to generate a reconfigurable PUF. This structure can be re-configured very easily, such as by adding several different lengthsof 0’s at the end of every challenge. Additionally, the security ofPUF can be increased, due to the "one-way" property of hash func-tion. Many hash algorithms have been investigated and developedin the last years. Currently, the SHA-1 algorithm is the Nationalinstitute of Standards and technology (NIST) secure hash standard.Several reconfigurable hash function unit architectures have beenpublished in past years [20].

In fact, this structure has already been named as Controlled Phys-ical Unclonable Function in [21], which was described as addingcontrol logic to a PUF structure to prevent an adversary from ac-cessing the PUF directly. Instead of doing a simple hash beforethe challenges applied to the PUF, we can consider adding anothercontrol logic, which would make the CRPs updatable. We proposeseveral reconfiguration methods:

(a) Adding different bit streams into the challenges, e.g., addingdifferent numbers of 0’s at the end of the challenges.

(b) Reordering the challenge stream by certain rules.

(c) Reconfiguring the hash function, by using the reconfigurabil-ity of these reconfigurable hash function implementations.

Due to the property of hash function, it is extremely hard foran adversary to model the PUF, even after we configure it severaltimes, since the output of hash function is unpredictable.

4.1.3 PUF with Output Recombination

Another idea is to add an extra reconfigurable component to pre-process the output of the arbiter before using it as an authentica-tion key. One simple example is to use two parallel MUX PUFsto update the CRPs, as shown in Figure 5. In this case, the signal(rising edge) will propagate through 4 paths which are selected bychallenges. Then we can select two of the four paths using the con-figure data and forward to the arbiter to generate the response. Wewill have a total of 12 possible combinations if we use a 2 paral-lel MUX PUFs. Therefore, we can reconfigure this architecture 12times. However, there will be very high correlations among these12 different combinations. For example, if we know that path 1 isfaster than path 2, and path 2 is faster than path 3, then we can con-clude that path 1 will be faster than path 3. Therefore, there should

be some constraints for the pre-processing, which will decrease thetotal number of reconfigurations. In fact, there are N ! possiblecases for ordering N paths based on their arrival time. Therefore,log2(N !) independent bits can be produced by N paths. We canincrease the number of parallel PUFs to obtain more possible com-binations to meet the practical application needs. If we want toachieve the entropy limit as log2(N !), we need to choose the out-put comparison pairs adaptively, which would increase the designcomplexity and fabrication area significantly.

Figure 5: Two Parallel MUX PUF Structure.

However, there will be a problem by employing this structure,since the pre-processing component after the last stage also hasvariations, which will affect the performance of the PUF. To solvethis problem, we can add pre-processing components after the ar-biters, as in structure of Figure 6. If we use N parallel MUX-basedPUF, we will need 2N-1 arbiters, where we only compare the neigh-bor paths. This is a concept borrowed from ring oscillator PUFwhich could ensure there will be no correlation between the outputbits of the arbiters, as the comparison pairs are non-cyclic. There-fore, this structure can update its challenge-response behavior in anunpredictable manner.

Figure 6: MUX PUF Structure of Output Recombination.

4.2 Reconfigurable Circuit StructuresInstead of only making the CRPs reconfigurable by processing

the challenge and response directly, we can alter the main circuitto update the challenge-response behavior. This kind of reconfig-urable PUFs will have better performance from the security per-spective, since it leads to a different PUF circuit after reconfigura-tion, while the previous method only changes the CRP mapping.

The most important thing in these structures is to ensure the ex-tra circuit will not affect the PUF performance, or more generally,



the extra circuit will have identical effect on the delay of differentpaths statistically. Otherwise, the behavior of the PUF can be easilypredicted which leads to an insecure system.

4.2.1 Reconfigurable Feed-Forward PUF

It has been shown that the security of the MUX PUF in Figure 2can be improved by adding feed-forward arbiters to it. However, inprevious literature, how to choose the feed-forward stages and howmany stages are chosen for feed-forward purpose have not beenclearly presented. One constraint is trivial: the signal producedby the feed-forward arbiter should arrive earlier than the two sig-nals propagating through the MUX paths. Therefore, we shouldensure that there are at least 5-8 stages between stages connectedto the input and the output of a feed-forward arbiter. We denote thestages from the input of a feed-forward arbiter to the output of thefeed-forward arbiter as a feed-forward component. We consider thefollowing three feed-forward structures:

(a) Feed-forward overlap: This structure has at least one stageoverlap between two feed-forward components.

(b) Feed-forward cascade: In this structure, the last stage of afeed-forward component will be the first stage of anotherfeed-forward component.

(c) Feed-forward separate: Here the different feed-forward com-ponents will be separated. Thus, there is no stage overlapbetween any two feed-forward components.

Figure 7: Feed-Forward Silicon MUX PUF Overlap Structure.

Figure 8: Feed-Forward Silicon MUX PUF Cascade Structure.

Figure 9: Feed-Forward Silicon MUX PUF Separate Structure.

In our experimental results, the intra-chip variations are increasedby adding non-linearity to the circuits. Among the 3 different struc-tures, the feed-forward cascade structure has the largest intra-chipvariation, with 10.7 bit Hamming distance on average with responselength of 100 bits, compared to 5.8 bits for non-feed-forward struc-ture.

We can also examine the nature of these different structures the-oretically. In the feed-forward structure, some of the challenge bits

will be the intermediate stage arbiter outputs instead of the externalbits. For example, if there is only one feed-forward component in aMUX PUF, which is from the a-th stage to the b-th stage, the timedifference of the b-th stage could be expressed as:

Δb = (−1)sign(∑a

i=1(−1)Ci (Dti−Dbi))(Dtb −Dbb)

An error occurred in the output of the feed-forward arbiter willalso affect the time difference in the b-th stage. Therefore, the errorprobability of the final response is increased by adding nonlinearity.

In the feed-forward cascade structure, the noise from an earlierfeed-forward component will directly affect the output of the nextfeed-forward arbiter. Therefore, this structure will have the leastreliability. From above, we know the time difference of the laststage of the feed-forward arbiter is Δb. We assume the c-th stage isthe first stage of the next feed-forward component. Then the outputof the second arbiter is:

Cc = sign(

c−1∑

i=1

(−1)Ci(Dti −Dbi) + Δb)

= sign(

c−1∑

i=1

(−1)Ci(Dti −Dbi)

+ (−1)sign(∑a

i=1(−1)Ci (Dti−Dbi))(Dtb −Dbb))

.It can be seen that the intra-chip variations are increased in this

cascade structure, since the noise in earlier stages is more likely tocause the outputs of the later arbiters to flip.

For structure (a), the second feed-forward arbiter output of thisoverlap structure is:

Cc = sign(c∑

i=1

(−1)Ci(Dti −Dbi))

As each output of feed-forward structures will not be affected bythe noise from feed-forward arbiters, we expect this structure willhave the best reliability.

For structure (c), since there are several stages between two feed-forward components, the noise effect from feed-forward bit willcombine with the path difference before the beginning point of thenext feed-forward component. If the process variations of thesestages are more significant than the noise effect from the feed-forward arbiter, the output of next feed-forward arbiter will alsonot be affected. The second feed-forward arbiter output of this sep-arate structure is:

Cc = sign(

b−1∑

i=1

(−1)Ci(Dti −Dbi) + Δb

+c∑

i=b+1

(−1)Ci(Dti −Dbi))

From above, it can be seen that the mathematical models for the3 feed-forward structures are different. We find that feed-forwardarbiters also enable us to reconfigure the circuit as well as to im-prove the performance from a security perspective. A basic recon-figurable feed-forward structure that combines overlap and separateapproaches is shown in Figure 10.

The structure can be configured among the 3 different structures((a) overlap, (b) cascade, (c) separate), which will increase the com-plexity of PUF model. By configuring the PUF, the mathematicalmodel for the PUF will be altered. This makes it infeasible for at-tackers to break the PUF by only using one single uniform linearmodel. The delay of MUXs connected after the feed-forward struc-ture (normally just an arbiter) may also affect the delay differenceof the two paths. However, this time difference could add into the



Figure 10: Proposed Highly Secure Reconfigurable Feed-Forward MUXPUF Structure.

total path delay difference both positively and negatively, depend-ing on the select signal. Therefore, the effect of these MUXs wouldbe statistically equivalent to the two paths of original MUX PUF,even if the delay of the added two MUXs vary quite significantly.From above, we conclude that the MUX based PUF will be moresecure when feed-forward arbiters are reconfigurable.

4.2.2 MUX and DeMUX PUF

The function of MUX is multiplexing; it selects one of manyinput signals and forwards the selected signal into a single line.The DeMUX is a device that takes a single input signal that carriesmany channels and distributes them over multiple output signals.Using DeMUX enables us to select the direction of the propagatingsignal, and makes the PUF reconfigurable. A basic reconfigurablestructure is shown in Figure 11. Instead of propagating the ris-ing edge signal successively, we can choose to skip some stagesby adding DeMUX components, which could make the challenge-response behavior reconfigurable and hard to predict. This struc-ture will be harder for attackers to model than the silicon PUF onlybased on MUX.

Figure 11: MUX and DeMUX PUF Structure.

5. EXPERIMENTAL RESULTSAll of our experiments have been carried out using SPICE sim-

ulations on a 65-nm technology process. We use Monte Carlomethod to simulate the effect of process variations and environmen-tal variations. In our simulation, we set up the transistor param-eters and process variations based on a major industrial standardmodel. Each proposed structure has been simulated over at least20 Monte Carlo runs in SPICE. We simulated 100 MUXs stagesfor each structure of these silicon PUFs. Accordingly, we need toapply a 100-bit challenge to the PUF to produce a 1-bit response;as a result, 100 different challenges were required to generate thefinal 100-bit digital signature for each IC.

Simulated PUF Structures: In our experiment, we added 10feed-forward arbiters into each feed-forward structure of MUX PUF.For instance, the feed-forward arbiters were from stage 1 to stage11, from stage 11 to stage 21 ... from stage 91 to stage 100 ina feed-forward cascade structure. The feed-forward arbiters werefrom stage 1 to stage 7, from stage 11 to stage 17 ... from stage91 to stage 97 in feed-forward separate structure. In a feed-forwardoverlap structure, the feed-forward arbiters were from stage 1 tostage 51, from stage 6 to stage 56 ... from stage 46 to stage 96. For

the reconfigurable feed-forward structure, we also added 10 sucharbiters and MUXs structures (as discussed in Section 4.2.1) intothe original PUF circuit, which can switch among the 3 differentfeed-forward structures. Moreover, we also simulated 10 DeMUXcomponents in the MUX and DeMUX PUF. The inputs and the out-puts of the DeMUXs were from stage 3 to stage 8, from stage 13 tostage 18 ... and from stage 93 to stage 98. Finally, we simulated anoutput recombination structure with 20 parallel MUX PUFs. Wederived the digital signature by comparing adjacent paths amongthe total 40 paths. Therefore, except for the first and the last paths,each path was compared to two other paths.

Inter-chip Variations: The inter-chip variations were evaluatedby the Hamming distance between two digital signatures whichwere generated by a same challenge and configure data from differ-ent chips. Since we simulated 20 chip instances, we had 20*19/2,i.e., 190 possible digital signature comparisons. We provide themaximum and the minimum of these numbers for the inter-chipvariations.

Intra-chip Variations: The intra-chip variations were determinedby comparing the digital signatures of the same IC under differentenvironmental conditions; in our case, we consider the temperatureas the primary environmental factor. It has been shown in our ex-periment that the intra-chip variations introduced by different tem-peratures from 0oC to 100oC were more significant then the intra-chip variations caused by voltage varying from 1V to 1.2V . Thedigital signatures of the PUF at 0oC, 20oC, 40oC, 80oC, 100oCwere obtained; however, we only present the comparisons between0oC and 100oC, as those exhibited the largest variations. We ap-plied 10 different challenges for each IC, and simulated 20 differentIC instances. Therefore we had 200 comparisons in total. We pro-vide the maximum and the average of these values of Hammingdistance for the intra-chip variations.

Reconfigurability: The reconfigurability was determined by thevariations of digital signatures generated by different configure datain a same IC. We fixed the challenge for a reconfigurability test,while we fixed the configure data of the different structures whenexamining the inter-chip variations and the intra-chip variations.In fact, the challenge-bit lengths were decreased by adding recon-figurable components in the feed-forward structures; therefore, weneed to adjust the challenge bits when simulating these reconfig-urable structures. We also applied 10 different configure data foreach IC, and simulated total 20 different IC instances, which aresimilar to intra-chip variation test. All the simulations were doneunder the environmental condition of 25oC and 1.1V .

Table 1 presents the inter-chip variations and intra-chip varia-tions for different MUX Physical Unclonable Function structures.First, it can be observed that the minimum inter-chip variationsare larger than the maximum intra-chip variations for all of thesimulated structures. Thus, we can conclude that the variationscaused by the randomness in manufacturing process are more sig-nificant than the variations under different environmental condi-tions. Therefore, these PUFs can be used as reliable secret keyswith some error correcting techniques. Second, it can also be ob-served that by adding feed-forward arbiters into the MUX PUF cir-cuit, the inter-chip variations and intra-chip variations are both in-creased, since the noise can have influence on the select signals ofsome intermediate stages. By comparing the inter-chip variationsand the intra-chip variations, we can say the feed-forward separatestructure is the most reliable structure while the feed-forward cas-cade is the least reliable one among the 3 feed-forward structures.The reconfigurable feed-forward structure has very close perfor-mance to the 3 types of feed-forward structures, since its function-ality is switching among the 3. Moreover, the reconfigurable MUXand DeMUX PUF has similar inter-chip variation as the non-feed-forward structure, but the intra-chip variation is increased, as thenumber of stages may be reduced by configurations. Therefore, thereliability of this structure is decreased.



Table 1: Simulation Results: Variations

Inter-chip Variation Intra-chip VariationStructures Max Min Max Avg

Non-feed-forward 59 22 13 5.8Feed-forward Overlap 66 27 15 8.7Feed-forward Cascade 64 25 20 10.7Feed-forward Separate 65 26 17 9.9

Reconfigurable Feed-forward 65 25 19 10.3MUX and DeMUX 57 23 16 7.1

Table 2 shows the reconfigurability of each reconfigurable struc-ture. It can be seen that the output recombination structure hasthe best reconfigurability, i.e., by fixing the challenge bits and onlychanging the configure data, the digital signature of this structurehas the most significant variations. In our simulation results, theaverage variation is 38.7 bits. The MUX and DeMUX PUF ex-hibits the least reconfigurability. This is because the function of theDeMUX is only to determine whether to skip some stages or not.When the process variations of other stages are relatively large, thedifference of digital signatures with two different configure datamay only vary a little bit. For the output recombination structure,it is similar to comparing different paths with different configura-tions. Therefore, its performance is close to the inter-chip varia-tion of the non-feed-forward MUX PUF. It also can be observedthat although the challenge hash structure and the challenge LFSRstructure are both pre-processing the challenge before applying tothe circuit, their reconfigurability still has some difference. As thechallenge LFSR appears to have better reconfigurability, we canconclude that the number generated by the LFSR in our case mayhave better randomness than that of the hash function. Finally, theproposed reconfigurable feed-forward MUX PUF has the averageHamming distance 32.4 bits by different configurations, which willbe sufficient to be used as a secure and reliable secret key storagemethod, considering its complex and nonlinear functionality.

Table 2: Simulation Results: Reconfigurability

VariationStructures Max Avg Min

Challenge LFSR 44 34.6 28Challenge Hash 42 28.3 19

Output Recombination 57 38.9 25Reconfigurable Feed-forward 47 32.4 22

MUX and DeMUX 33 24.7 13

Overall, all the proposed reconfigurable structures have consid-erable reconfigurability, and can be used for reliable authenticationand identification within certain error tolerance, as the minimum ofthe inter-chip variations is larger than the maximum of intra-chipvariations. The output recombination structure has the best recon-figurability; however, the reconfigurable feed-forward MUX PUFhas the best performance due to its security, as it is extremely hardto be modeled by linear modeling methods.

6. CONCLUSIONWe have presented several reconfigurable silicon MUX Physical

Unclonable Functions based on two major approaches and demon-strated their effectiveness by experimental results via inter-chip vari-ation and intra-chip variation. We also have discussed the reliabil-ity perspective of PUFs and proposed several methods to increasethe security. Ongoing work includes novel highly secure and reli-able reconfigurable PUF designs and their mathematical analysis.Furthermore, we are also interested in developing an authentica-tion scheme for reconfigurable PUFs, which will use several pairsof CRPs as a set for authentication by utilizing the reconfigurableproperty of reconfigurable PUFs.

7. REFERENCES[1] R. Pappu, B. Recht, J. Taylor, and N. Gershenfeld, “Physical one-way

functions.” Science, vol. 297(5589), p. 2026, 2002.[2] B. Gassend, D. Clarke, M. V. Dijk, and S. Devadas, “Silicon physical

unclonable functions,” the 9th ACM Conference on Computer andCommunications Security, p. 160, 2002.

[3] ——, “Controlled physical unclonable functions,” in Computer SecurityApplication Conference, 2002, pp. 149–160.

[4] S. Kumar, J. Guajardo, R. Maesyz, G. Schrijen, and P. Tuyls, “Extendedabstract: The butterfly PUF protecting IP on every FPGA,” Hardware-OrientedSecurity and Trust (HOST 2008), pp. 67–70, 2008.

[5] R. Maes, P. Tuyls, and I. Verbauwhede, “Intrinsic PUFs from flip-flops onreconfigurable devices,” in Benelux Workshop Information and System Security(WISSec 08), 2008.

[6] D. E. Holcomb, W. P. Burleson, and K. Fue, “Initial SRAM state as a fingerprintand source of true random numbers,” in Conference on RFID Security, 2007.

[7] U. Ruhrmair, F. Sehnke, J. Solter, G. Dror, S. Devadas, and J. Schmidhuber,“Modeling attacks on physical unclonable functions,” in Conference on RFIDSecurity, 2010.

[8] M. Majzoobi, F. Koushanfar, and M. Potkonjak, “Techniques for design andimplementation of secure reconfigurable PUFs,” ACM Transactions onReconfigurable Technology and Systems, vol. 2, no. 1, pp. 1–33, 2009.

[9] D. Lim, J. W. Lee, B. Gassend, G. E. Suh, M. V. Dijk, and S. Devadas,“Extracting secret keys from integrated circuits,” IEEE Transaction on VeryLarge Scale Integration Systems, vol. 13, no. 10, p. 1200, 2005.

[10] H. Chang and S. Sapatnekar, “Statistical timing analysis considering spatialcorrelation in a pert-like traversal,” in IEEE International ConferenceComputer-Aided Design Integrated Circuits and Systems, 2003, pp. 621–625.

[11] J.-W. Lee, D. Lim, B. Gassend, G. E. Suh, M. van Dijk, and S. Devadas, “Atechnique to build a secret key in integrated circuits with identification andauthentication applications,” in IEEE International ConferenceComputer-Aided Design Integrated Circuits and Systems, 2003, pp. 621–625.

[12] K. Kursawe, A. Sadeghi, D. S. B. Skoric, and P. Tuyls, “Reconfigurablephysical unclonable functions – enabling technology for tamper-resistantstorage,” in 2nd IEEE International Workshop on Hardware-Oriented Securityand Trust(HOST), 2009, pp. 22–29.

[13] A. M. S. Morozov and P. Schaumont, “An analysis of delay based PUFimplementations on FPGA,” Springer, pp. 382–387, 2010.

[14] D. Merli, F. Stumpf, and C. Eckert, “Improving the quality of ring oscillatorPUFs on FPGAs,” in WESS ’10 Proceedings of the 5th Workshop on EmbeddedSystems Security, 2010.

[15] J. Guajardo, S. S. Kumar, G.-J. Schrijen, and P. Tuyls, “FPGA intrinsic PUFsand their use for IP protection,” Cryptographic Hardware and EmbeddedSystems, 2007.

[16] J. Cong, “Challenges and opportunities for design innovations in nanometertechnologies,” SRC Design Science Concept Paper, 1997.

[17] S. Nassif, “Delay variability: Sources, impact and trends,” in Solid-StateCircuits Conference, 2000, pp. 368–369.

[18] L. Alaus, D. Noguet, and J. Palicot, “A reconfigurable linear feedback shiftregister operator for software defined radio terminal,” IEEE InternationalSymposium on Wireless Pervasive Computing, 2008.

[19] P. Kitsos, N. Sklavos, N. Zervas, and O. Koufopavlou, “A reconfigurable linearfeedback shift register (LFSR) for the bluetooth system,” in IEEE InternationalConference on Electronics, Circuits and Systems (ICECS), 2001.

[20] M. Zeghida, B. Bouallegue, A. Baganne, and M. Machhout, “A reconfigurableimplementation of the new secure hash algorithm,” Second InternationalConference on Availability, Reliability and Security (ARES), pp. 281–285, 2007.

[21] B. Gassend, D. Clarke, M. V. Dijk, and S. Devadas, “Silicon physical randomfunctions,” in ACM Conference on Computer and Communications Security,2002, pp. 148–160.



Verification of Information Flow Properties inCyber-Physical Systems

Bruce McMillin and Ravi AkellaDepartment of Computer Science

Intelligent Systems CenterRolla, MO, United States 65409-0350

Email: [email protected], [email protected]

Abstract—Information flow typically refers to the explicit aswell as implicit information resulting from the interaction ofcyber processes constituting a system. Information flow alsooccurs in cyber-physical systems (CPSs). Information flow isdifficult to detect in CPSs, due to their physical nature andcomplex interactions among various computational and physicalcomponents. In this work, formal methods of security specifica-tion and verification are extended to describe confidentiality inCPSs. This paper presents a general approach to specify andverify information flow properties, such as non-deducibility, ina CPS using bisimulation techniques.

I. INTRODUCTION

Information flow refers to the explicit as well as implicitinformation that could be the result of communication amongvarious processes constituting the system. In multi-levelsystems, determining information flow from a high-leveldomain to the low-level domain violates confidentialityin most cases. Information flow properties like Non-interference, Non-deducibility [1] and extensions [2] arestudied in literature that deal with preventing informationfrom being downgraded through covert channels and othersuch potential causes. This work extends the applicationof these information flow properties to more complexcyber-physical systems (CPSs) which are integrations ofphysical and computational processes.

In CPSs, due to physically observable cyber and physicalevents, information flow is complex resulting in a greaterrisk of confidentiality and privacy violations. For example,“smart metering" in smart grids involves the collection ofprivate data of consumers to manage the resources efficiently.Failure to monitor the usage of such information could posean increased threat to the privacy of the consumers [3].

Cyber-physical interactions result from the coupling ofthe information and physical flows in a CPS. In a CPS,cyber processes interact with physical components foractuating control and monitoring. The key challenge isto represent the physical nature of the system. Invarianceof flow, such as power flow that satisfies Kirchoff’s laws,forces execution of any event within the system to satisfythe invariant. Information flow vulnerabilities are a natural

This work was supported in part by the Future Renewable Electric EnergyDistribution Management Center; a National Science Foundation supportedEngineering Research Center, under grant NSF EEC-0812121 and in partby the Missouri S&T Intelligent Systems Center.

consequence of this interdependence of actions in the cyberand physical domains. In other words, an observation aboutphysical flow could permit an observer to infer possiblysensitive cyber actions. The physical components that areexposed to any observer outside the CPS divulge someinformation regarding the system (physically observablebehavior). For example, the operation of a wind turbine ina smart grid depends on its physical size, velocity of wind,etc., that are observable. Definitions of High and Low leveldomains change according to the physical location of theobserver on the CPS; for example, an observer controllinga physical component knows more about the system thanan external observer (with only physical observability). Tocompletely analyze the information flow, various cases ofsuch observers should be accounted for, that reveal the extentof confidentiality violation within the system [4] as in Fig. 1.

Fig. 1. Different levels of confidentiality violation possible in a CPS

Security considerations for a CPS, therefore, depend oncyber information flow, physically observable behavior, andthe interactions among the cyber and physical components ofthe system. Due to infrastructure interdependencies [5] [6],a compromise in the security of one system may threatenanother system. For example, the information flow resultingfrom the composition of a power electric flow controller,with a distributed control, affects the security of thepower grid. Timing, security [7] and frequency [8] are keyproperties that have an impact on the confidentiality of asystem. The complexity of these interactions exceeds theability of informal information flow analysis. Informationflow theory [9] provides an attractive formal method to



quantify information flow, but it is not automated. Processalgebras [10] provide both a rigorous system specificationand associated verification (model checking) procedures, butare rarely used because of their complexity.

In this work, a methodology has been adopted that ad-dresses the aforementioned issues for the information flowverification of CPSs. Security process algebra (SPA) [2] isused to specify the CPS as a communicating set of concurrentprocesses and bisimulation techniques from literature arepresented to be useful to verify non-deducibility in thecomposed system. Section II introduces some aspects ofSPA and bisimulation-based non-deducibility on composition(BNDC) [2], which are vital to the approach presented.In section III, the proposed approach for the analysis ofinformation flow is discussed. Finally, conclusions and futurework are presented in section IV.

II. BACKGROUND

A. SPA

Security Process Algebra (SPA, for short) [2] is an exten-sion of the Calculus of Communicating Systems (CCS) [10].Any system can be defined as: E ::= 0|µ.E|E1 +E2|E1|E2|E\L|E\IL|E/L|E[f ]|Z, where 0 is the emptyprocess, which cannot do any action; µ.E can do actionµ and then behaves like E; E1 + E2 can alternativelychoose to behave like E1 or E2; E1|E2 is the parallelcomposition of E1 and E2, where the executions of the twosystems are interleaved, E\L can execute all the actions Eis able to do, provided that they do not belong to L ∪ L(L refers to the output); The operation E1 ||

︸︷︷︸

A

E2 represents

the synchronized parallel composition of E1 and E2 uponthe events from set A.

B. Bisimulation-based Non-Deducibility on CompositionModel

Bisimulation [10] is an equivalence technique based onthe idea that two states of the system can be distinguishedif the distinction can be detected by a process interactingwith the system. A system is considered to have the BNDCproperty if it can preserve its security after composition [2].A system ES is BNDC if for every high-level process Π,a low-level user cannot distinguish ES from ES|Π (EScomposed with process Π). In other words, a system ESis BNDC if what a low-level user sees in the system isnot modified by composing any high-level process Π withES. BNDC(ES) ≡ ∀Π ∈ EH , ES\H ≈B (ES|Π)\Hwhere ES\H changes all the H events in ES into internalevents represented as a sequence of τ actions. A systemis BNDC-preserving if the above property holds for allpossible behaviors of the system.

III. ADVANCED ANALYSIS

A formal methodology to automate the process of verifyingconfidentiality of information flow within a CPS involvesaddressing three main issues.

A. Representation of cyber and physical processes and theirinteractions into computational framework

A process algebra (as in section II-A) can be used tomodel the CPS as a composition of cyber and physicalprocesses that communicate concurrently, if possible, in asynchronized manner. Each process is defined as a sequenceof events within the system which determine different statesthe system could transition into. In [7], SPA has beenused to represent the physical actions interacting with thecomputational elements in a gas pipeline network. A similarapproach has been taken in [4], to analyze informationflow within a more complex smart grid that uses advanceddistributed algorithms to manage the underlying physicalresources like renewable energy sources, house loads, powerstorage etc.

The physical network in a CPS forms a continuous timesubsystem and the computational part forms a discrete eventsubsystem. SPA is not equipped to model the continuous-time nature of physical processes in a CPS. To capture theinformation flow of the combined system thereby forcesthe modeling of the continuous physical subsystem to beevent-based, so that the physical events can be capturedusing process algebra. Physical events include 1) a localstate change of the physical subsystem resulting froma cyber component controlling it (for example, a powerflow controller increases/decreases voltage on the powerline causing a flow change) or 2) a local physical statechange brought by the dynamics of the physical network(for example, load on a power line increases/decreases asa stochastic process to which the power electronics reactby making a setting). Invariance on physical flow can bemodeled such that events that change the flow at variousphysical components are reflected in an aggregate flow thatsatisfies the invariant. The impact of physically observablebehavior cannot be ignored to study information flow inCPSs. This forces the observable actions to be consideredas events that are used as building blocks of processspecification.

Cyber events within a CPS involve in distributed computa-tion based on 1) communication with other cyber componentsor 2) communication with the physical component that itcontrols. Composition of cyber processes result in the trans-formation of complementary actions of the processes intointernal silent actions in the composed process, defined bythe SPA ‘|’ operator as below:

E1

a−→ E′

1E2

a−→ E′

2

E1|E2

τ−→ E′

1|E′

2

(1)

The communication between physical processes is differ-ent from that between cyber processes; in the former case, thepair of processes make a synchronized physical change on theshared network (such as power transfer on the shared powerbus from a component with high potential to a componentwith lower potential) and in the latter case, the pair of pro-cesses synchronize on complementary request/response type



messages. Methods to counter interception of messages onthe communication channel exist, validating the assumptionthat cyber processes can be securely composable as long asthey perform complementary actions. By contrast, physicalchange between two physical processes is observable by anintermediary in the path of their ‘direct communication’,making it difficult to securely model such composition. Usingthe proposed approach, CPSs can be modeled in terms ofSPA followed by bisimulation-based equivalence testing ofthe processes as outlined in section III-B.

B. Adequacy of Bisimulation-based Non-deducibility proper-ties for CPS Models

According to the definition of BNDC (II-B), the SPAspecification of the system has to be composed with all thehigh-level processes (Π) that can be modeled using the high-level actions of the system. However, this would significantlyincrease the complexity of verification of BNDC propertysince it should be verified that E\H ≈B (E|Π)\H , for allΠ (Π could be defined as some combination of events fromH ∪ H reaching possibly any state E ′ ∈ ξ, where ξ is theset of all the states of the system).

To avoid such universal quantification on Π, a strongform of BNDC called strong BNDC (or SBNDC) has beenproposed in [11]. SBNDC states that iff for all E ′ reachablefrom E, if E′ h∈H

−−−→ E′′, then E′\H ≈B E′′\H . Thisdefinition of SBNDC implies that the system before andafter executing a high-level action remains indistinguishable.

Such a definition avoids the fact that the propertyshould hold for all possible high-level processes withinthe system, transforming the bisimilarity relation betweenE and E|Π into a bisimulation up to H relation on Eand E\H i.e., it will be verified if E ≈\H E\H with thehigh actions replaced with silent internal action, τ . Thebisimilarity up to H relation for SBNDC (≈0

\H) transforms

the high-level events in E ′ into a sequence of (τ−→)0 or zero

actions. An exhaustive study of BNDC and its variants ispresented in [11]. The impact of silent internal actions onweak bisimulation relation is equivalent to that defined inCCS [10].

1) SBNDC verification on a test CPS model: In thissection, the approach outlined above is generalized for aninformally specified CPS. Assuming a homogeneous CPSwith identical cyber and physical components on the system,we can define the events as below:

Events at a given cyber process (represented asCyber):RS: Read local physical state, CO: Compute, IC: Issuecommand to a local physical controller, Send: Send messageto peer cyber components, Recv: Receive message frompeer cyber components.

Events at a given physical process (represented asPhysical):

IN : Change due to invariant maintenance, PO: PhysicalObservability, SC: Local state change, EC: Executecommand from cyber component.

The rest of the system can be built up from these eventsin a bottom-up fashion as shown in Equations 2 through 6.

Cyber ∼=RS.CO.IC.Cyber + RS.CO.Send.Cyber+

Recv.CO.IC.Cyber(2)

Physical ∼= PO.SC.IN.Physical + PO.EC.Physical(3)

Node ∼= Cyber|Physical (4)

Invariant ∼=(IN1.Invariant + IN2.Invariant...)

+ τ.Invariant(5)

System, E ∼= (Node1|Node2...)|Invariant (6)

(IC, EC are complementary Input/Output pair. Similarly,SC and RS form a complementary I/O pair.) For anobserver internal to the system the events pertaining to restof the system can be classified as, High = RS, CO, IC,EC, Send and Low = PO, IN , Recv.

To verify whether this system satisfies SBNDC, we pro-ceed as follows: Unwinding E in terms of the events thatdefine it and using the definition of composition (as inEquation 1), a possible state E ′ (such that E → E′) canbe defined as:

E′ ∼= PO.τ.IC.Node + τ.CO.IC.Node+

EC.τ.τ.PO.CO.IC.Node + PO.τ.CO.Send.Node

+ τ.CO.Send.Node + EC.τ.CO.Send.τ.PO.Node

+ Recv.τ.SC.τ.PO.Node(7)

Now,

E′\H ∼=PO.τ.Node + Recv.Node (8)

The bisimulation up to H on E ′ yields

E′ ≈0

\HPO.τ.Node + Recv.τ.SC.PO.τ.Node (9)

From Equations 8 and 9, E ′ 6≈0

\HE′\H . This proves

that the generic CPS defined above does not satisfy SB-NDC. The trace that resulted in the failure of SBNDC isRecv.τ.SC.τ.PO.Node implying that on receiving a mes-sage, the Node performs an internal action, a State Changeevent, followed by an internal action to maintain theInvariant which is physically observable (for example, dueto the voltage drop at a node on the power line, a smallchange is observed by a neighboring node sharing the phys-ical line). In practice, such a physical change can be hiddenthrough coordination with other nodes in which the followingtakes place: if one node makes a change, the other node(s)perform(s) a compensating event [12] [13]. This processE′ can now be made SBNDC by adding a complementary



PO event such that the effect of the PO event is nulli-fied. Therefore, the system trace Recv.τ.SC.τ.PO.Node inEquation 7 can be modified as Recv.τ.SC.τ.PO.PO.Node∼= Recv.τ.SC.τ.τ.Node. The modified E ′ will have thefollowing characteristics.

E′Modified\H

∼=PO.τ.Node + Recv.τ.Node (10)

E′Modified ≈0

\HPO.τ.Node + Recv.τ.Node (11)

Thus, E′Modified satisfies the SBNDC property. Similarly,

it is verified for every E ′ reachable from E whether E ′

and E′\H belong to the bisimilarity up to H relation. Thepossibility of multiple observers coordinating in an effortto deduce more information pertaining to the system is achallenging task that needs to be well addressed. Thus,the SBNDC property and its weak forms are sufficient forCPS verification (in many cases). This process reduces theproblem of verifying the BNDC property on the SPA modelof the CPS into verifying a bisimulation relation betweenall the reachable states of the system. The above approachwas performed on the SPA model of a two node smart grid(such as in [14]) that includes renewable energy sources,power electronics and battery to define the Physical processthat satisfies a global invariant function of physical flowand optimization algorithms that define the Cyber process.Such a complex interaction of cyber and physical processesresulted in over 3600 states (using an automated modelchecker) the system could transition into. The problem ofverifying the bisimilarity equivalence between E and E\H isthe next logical step to perform complete system evaluation.

C. Testing for Bisimulation Equivalence of Processes

Two processes are bisimilarly equivalent if there existsa bisimulation relation including both the processes as apair. This problem of equivalence testing has been wellstudied in literature [15] [16] as a relational coarsest partitionproblem: given a relation, R (Bisimilarity) and an initialstate, E over a global set of states ξ, find the coarsest stablerefinement which is such that either E ⊆ R−1(E\H) orE ∩ R−1(E\H) = φ. This verification is performed forall the states E′ ∈ ξ such that E → E′. The efficiencyof equivalence testing can be improved by the developmentof algorithms that minimize the state space using advancedtechniques of bisimulation and partial order reduction, as in[17].

IV. CONCLUSIONS AND FUTURE WORK

In this work, information flow analysis, with its originsin computational systems, has been extended to the realmof cyber-physical systems to verify their security, therebyexposing potential confidentiality violations. In [4], suchan analysis has been shown to uncover various levels ofconfidentiality violation within a smart grid architecture(called FREEDM [14]). The approach presented providesa preliminary but powerful method of verifying the confi-dentiality within a CPS. Future work involves developmentof generalized bisimulation-based security properties and

development of algorithmic techniques to reduce the statespace for equivalence testing in cyber-physical systems.

REFERENCES

[1] J. McLean, “Security models and information flow,” in Procs. of the1990 IEEE Computer Society Press. IEEE Computer Society Press,1990, pp. 180–187.

[2] R. Focardi and R. Gorrieri, “A classification of security propertiesfor process algebras,” Computer Security, vol. 3, no. 1, pp. 5–33,1994/1995.

[3] http://www.law.berkeley.edu/7966.htm, “Privacy protection needed assmart grid arrives,” accessed on: February 14, 2011.

[4] R. Akella and B. M. McMillin, “Information flow analysis of energymanagement in a smart grid,” in SAFECOMP, ser. Lecture Notes inComputer Science, E. Schoitsch, Ed., vol. 6351. Springer, 2010, pp.263–276.

[5] P. Pederson, D. Dudenhoeffer, S. Hartley, and M. Permann, “Criti-cal Infrastructure Interdependency Modeling: A Survey of U.S. andInternational Research,” no. INL/EXT-06-11464. Idaho NationalLaboratory.

[6] J. Min, W. Beyler, T. Brown, Y. J. Son, and A. Jones, “Towardmodeling and simulation of critical national infrastructure interde-pendencies,” in Institute of Industrial Engineers (IIE) Transactions,vol. 39. Special issue on Industrial Engineering and OperationsResearch in Homeland Security, 2007, pp. 57–71.

[7] R. Akella, H. Tang, and B. M. McMillin, “Analysis of information flowsecurity in cyber-physical systems,” International Journal of CriticalInfrastructure Protection, vol. 3, no. 3-4, pp. 157 – 173, 2010.

[8] Y. Sun, B. McMillin, X. F. Liu, and D. Cape, “Verifying Noninterfer-ence in a Cyber-Physical System: The Advanced Electric Power Grid,”in Proceedings of the Seventh International Conference on QualitySoftware (QSIC), Portland, OR, October 2007, pp. 363–369.

[9] J. McLean, “Security models,” in Encyclopedia of Software Engineer-ing, J. Marciniak, Ed. John Wiley & Sons, 1994.

[10] R. Milner, Communication and Concurrency. Prentice Hall, 1989.[11] A. Bossi, R. Focardi, C. Piazza, and S. Rossi, “Verifying persistent se-

curity properties,” Computer Languages, Systems & Structures, vol. 30,no. 3-4, pp. 231–258, 2004.

[12] R. Akella and B. M. McMillin, “Model-Checking BNDC Propertiesin Cyber-Physical Systems,” Computer Software and ApplicationsConference, Annual International, vol. 1, pp. 660–663, 2009.

[13] T. T. Gamage, B. M. McMillin, and T. P. Roth, “Enforcing informationflow security properties in cyber-physical systems: A generalizedframework based on compensation,” Computer Software and Appli-cations Conference Workshops, vol. 0, pp. 158–163, 2010.

[14] R. Akella, F. Meng, D. Ditch, B. McMillin, and M. Crow, “DistributedPower Balancing for the FREEDM System,” in The 1st IEEE Interna-tional Conference on Smart Grid Communications, October 2010, pp.7–12.

[15] R. Paige and R. E. Tarjan, “Three partition refinement algorithms,”Society for Industrial and Applied Mathematics Journal of Computing,pp. 973–989, 1987.

[16] P. C. Kanellakis and S. L. Smolka, “CCS expressions, finite stateprocesses, and three problems of equivalence,” Information and Com-putation, vol. 86, no. 1, pp. 43–68, 1990.

[17] A. Dovier, C. Piazza, and A. Policriti, “A fast bisimulation algorithm,”Univ. di Verona, Univ. di Udine, Tech. Rep. UDMI/14/00/RR, 2000.



Secure DataTransmission Protocol in Multi-Hop Sensor NetworksYilin Mo, Bruno Sinopoli

Abstract—In this paper, we discuss how to design secure datatransmission protocol in wireless sensor networks. The networkis assumed to havep sensors andn− p routers which relay theobservations from the sensors to the fusion center. A maximum off routers can be compromised by an adversary. The fusion centerwants to detect and identify malicious nodes in the system andreconstruct the observations. Due to computational constraints,we could not use advance information security technique, suchas digital signature to protect the integrity of the data. Instead,we model the whole network as a linear system. By leveragingfault detection and identification theory and generic propertiesof structured system, we can provide a necessary and sufficientcondition for generic detectability of the malicious nodes andreconstructability of the observations, which is related to thetopology of the network. We also show that resilience can befurther improved by using trustworthy nodes.

I. INTRODUCTIONDesign and analysis of systems based on sensor networks

involve cross disciplinary research spanning domains withincomputer science, communication systems and control theory.A sensor network is composed of low-power devices thatintegrate computing with heterogeneous sensing and commu-nication. Sensor network based systems are usually embeddedin the physical world, with which they interact by collecting,processing and transmitting relevant data.Sensor networks span a wide range of applications, includ-

ing environmental monitoring and control, health care, homeand off ce automation and traff c control [1]. Many of theseapplications are safety critical. Any successful attack maysignif cantly hamper the functionality of the sensor networksand lead to economic losses. In addition, sensors in large dis-tributed sensor networks may be physically exposed, making itdiff cult to ensure security and availability for each and everysingle sensor. The research community has acknowledged theimportance of addressing the challenge of designing secureestimation and control systems [2] [3].The impact of attacks on control systems is discussed in

[4]. The authors consider two possible classes of attacks onCPS: Denial of Service (DoS) and deception (or false datainjection) attacks. The DoS attack prevents the exchange ofinformation, usually either sensor readings or control inputsbetween subsystems, while a false data injection attack affectsthe data integrity of packets by modifying their payloads.A robust feedback control design against DoS attacks is

Department of Electrical and Computer Engineering, Carnegie Mellon Uni-versity, Pittsburgh, PA. Email: [email protected], [email protected] research is supported in part by CyLab at Carnegie Mellon under

grant DAAD19-02-1-0389 from the Army Research Off ce Foundation andgrant NGIT2009100109 from Northrop Grumman Information TechnologyInc Cybersecurity Consortium. The views and conclusions contained here arethose of the authors and should not be interpreted as necessarily representingthe off cial policies or endorsements, either express or implied, of ARO, CMU,or the U.S. Government or any of its agencies.

discussed in [5]. We feel that false data injection attacks can bea subtler attack than DoS as they are in principle more diff cultto detect. In this paper we want to analyze the resilience ofdata transmission protocol against such types of attacks.Secure data transmission protocols have been extensively

studied by the information security community. Many tools,such as digital signature, have been developed to protect dataintegrity. However, these cryptographic methods are usuallycomputationally expensive and may not be applicable in sensornetworks, in which sensors usually have limited computationalpower.Another approach is to model the data transmission protocol

as a linear system and malicious behavior in the network asan external input. Therefore the whole problem of detectionand identif cation of malicious nodes can be reformulated asa fault detection and identif cation (FDI) problem. Over thepast few decades, a signif cant amount of research effort hasbeen spent on FDI problems [6], [7]. Recently, Pasqualetti etal. [8] and Sundaram et al. [9] show how to use FDI to detectand identify malicious behavior in consensus algorithm andwireless control networks. In this paper, we discuss how to useFDI to design a secure data transmission protocol in sensornetworks. The network is assumed to have p sensors and n−p

routers which relay observations from the sensors to the fusioncenter. A maximum of f routers can be compromised by anadversary. The fusion center wants to detect and identify mali-cious nodes in the system and reconstruct the observations. Weshow that the achievability of such goals is determined by thetopology of the network. We also provide a way to increasethe resilience of the network by using trustworthy nodes.The rest of the paper is organized as follows: in Section II,

we introduce the sensor network model and the attack models.We also introduce the concepts of weak and strong resilience.In Section III, we prove a necessary and suff cient conditionfor the network to be weakly or strongly resilient to the attackbased on network topology. In Section IV, we propose a wayto enhance the resilience of the network by using trustworthyrouters. Finally Section V concludes the paper.

II. PROBLEM FORMULATION

In this section we want to introduce the system which willbe discussed in the rest of the paper.

A. System Description

Consider a set of p sensors S = 1, . . . , p monitoring theenvironment. At each time step k, each sensor makes a scalarobservation denoted as wk,i. To simplify notation, let us def nethe measurement vector wk = [wk,1, . . . , wk,p]

′ ∈ Rp.

The sensors need transmit their measurements back tothe fusion center. However, due to certain constraints, such



as energy or bandwidth, they cannot directly communicatewith the fusion center. As a result, a multi-hop network isused to relay the observations from the sensors to the fusioncenter. We assume that the network consists of n− p routersp+1, . . . , n, where n ≥ p and the fusion center is listeningto the last m (m ≤ n) nodes n−m + 1, . . . , n, the set ofwhich is denoted as T .We model the whole network (both the routers and sensors)

as a directed graph G = (V, E). V , s1, . . . , sn is the setof vertices1. E ⊆ V × V is the set of edges. (i, j) ∈ E if andonly if node i can transmit to node j directly. We denote theset of incoming neighbors of node i as

N Ii = j ∈ V : (j, i) ∈ E, (1)

and the set of outgoing neighbors as

NOi = j ∈ V : (i, j) ∈ E, (2)

The topology of the network is illustrated in Fig 1.Let W and W ′ be two subset of V . A path from W to

W ′ is a sequence of vertices i1, . . . , ik, where (ij , ij−1) ∈ E,i1 ∈ W and ik ∈ W ′. A path is called simple if there is norepetition of vertices. Two paths are disjoint if they do notshare common vertex. We call l paths from W to W ′ disjointif they are mutually disjoint, i.e. any two of them are disjoint.A set of l disjoint and simple paths from W to W ′ is calleda linking of size l or an l linking from W to W ′.We assume that at each step, a node can only broadcast

one scalar to its outgoing neighbors due to the bandwidth andenergy constraints. One way for the fusion center to recoverthe observations wk is to construct a p linking from S to T .Each node in the p linking simply forwards the messages fromthe incoming neighbor to the its unique outgoing neighbor inthe linking.In this paper we want to study a more general transmission

scheme. Instead of simply relaying the messages, each sensorcomputes a linear function of the messages from its incomingneighbors and its own state and broadcasts the result to theoutgoing neighbors. To be specif c, suppose each node keeps astate xk,i, where k is the time index and i is the node index. Ateach time step, each node i broadcasts its state to its outgoingneighbors,

zk,i,j = xk,i, j ∈ NO(i), (3)

where zk,i,j is the data transmitted from node i to node j attime k. Once node i receives the data from all its incomingneighbors, it performs an update as follows:

xk+1,i = ai,ixk,i +∑

j∈NIi

ai,jzk,j,i +∑

j∈S

δi,jwk,j , (4)

with initial condition x0,i = 0, and δi,j is 1 if i = j and is 0otherwise.To simplify notation, let us def ne xk = [xk,1, . . . , xk,n]

′ ∈R

N , A , [ai,j ] ∈ Rn×n and B , [e1, . . . , ep] ∈ R

n×p, whereei , [δ1,i, . . . , δn,i]

′ ∈ Rn. (4) can be written in the matrix

form asxk+1 = Axk +Bwk, x0 = 0. (5)

1We use vertex or node to denote both sensors and routers.

Def ne C = [en−m+1, . . . , en]′ ∈ R

m×p. At each time thefusion center receives

yk = [xk,n−m+1, . . . , xk,n]′ = Cxk. (6)

ST

Fig. 1. Multi-Hop Sensor Networks

B. Attack Model

In this section we describe the attack model. We assume thatseveral routers are compromised2. We model the maliciousbehavior of a compromised router as an external input. Tobe specif c, we assume the compromised router follows thefollowing update rule

xk+1,i = ai,ixk,i +∑

j∈NIi

ai,jzk,j,i + uk,i. (7)

Remark 1. If a compromised router choosesuk,i = 0 forall k, then there is no chance for the fusion center to detectit since it behaves exactly as a benign node. Moreover thecompromised router does no damage to the network. Therefore,we assume that each compromised router uses a non-zerouk,i

for at least once.

The set of compromised routers is denoted as F , which isassumed to be unknown to the fusion center in the beginning.However, we assume that |F | ≤ f , where f is known to fusioncenter. Let us denote F as the set of all possible F , which isdenoted as3

F = F ⊆ V : |F | ≤ f, F⋂

S = φ. (8)

It can be seen that the capability of the attacker is completelycharacterized by F . As a result, we will call such attack anF -attack.Def ne BF = [ei1 , . . . , eij ], where F = i1, i2, . . . , ij.

Then the update equation can be written as

xk+1 = Axk +Bwk +BFuFk . (9)

Def ne w, y, uF as inf nite sequences(w0, w1, . . .), (y0, y1, . . .), (u

F0 , u

F1 , . . .) respectively. It

is easy to see that y is a function of w, uF and F . As aresult, we will denote y as

y = Φ(w, uF , F ). (10)

2We assume the sensors are always benign in the whole paper.3Note that the empty set φ also belongs to F .



Next we def ne the concept of resilient to F -attacks:

Definition 1. The system is called weakly resilient to theF -attacks if and only if

Φ(w, uF , F ) = Φ(0, 0, φ) (11)

implies thatF = φ andw = 0.

Definition 2. The system is called strongly resilient to theF -attacks if and only if

Φ(w, uF , F ) = Φ(w′, vF′

, F ′) (12)

implies thatw = w′, uF = vF′

andF = F ′.

The following theorems relate the weak and strong re-silience to the detectability of the attack and reconstructabilityof the sensor measurements.

Theorem 1. If the system is weakly resilient to theF -attacks,then the system can successfully detect anyF -attack.

Proof: If there is no attack, then the trajectory of theoutput will be in the following set

Yφ = y = Φ(w, 0, φ).

When an attack is present, the trajectory will be in thefollowing set4

YF = y = Φ(w, uF , F ).

The system can detect the attack if and only if Yφ

⋂YF = φ.

Now suppose the opposite, i.e. Yφ

⋂YF 6= φ. Then there

exists w, w′, uF , F 6= φ, such that

Φ(w, 0, φ) = Φ(w′, uF , F ).

By linearity of the system, we have

Φ(0, 0, φ) = Φ(w′ − w, uF , F ),

which further implies that F = φ due to the weak resilience.However, that contradicts the assumption that F 6= φ. ThusYφ

⋂YF = φ and the system is able to detect the attack.

Theorem 2. If the system is strongly resilient to theF ,then giveny, the system can successfully identify the set ofmalicious nodesF and reconstruct the inputw.

Proof: Suppose F is the set of compromised nodes.Def ne

YF = y = Φ(w, uF , F ).

The set of malicious nodes can be identif ed if and only ifYF s, for all F ∈ F are mutually disjoint. Now consider theopposite, there exists F1 6= F2 and YF1

⋂YF2

6= φ, whichimplies that

Φ(w, uF1 , F1) = Φ(w′, vF2 , F2),

for some w, w′, uF1 , vF2 . However, due to strong resilience,F1 = F2, which contradicts the assumption that F1 6= F2.Hence, the system can identify the set of malicious nodes.

4Note that we assume that for each compromised node, uk,i will be nonzero for at least once.

The reconstructability follows directly from the fact that wis uniquely determined by y = Φ(w, uF , F ).In the next section we will relate the concepts of weak and

strong resilience to the topology of the network.

III. MAIN RESULT

In this section we will characterize the resilience of thenetwork by studying its topology.

A. Algebraic Conditions

First, we wish to relate the concepts of strong and weakresilience via the following theorem.

Theorem 3. A system is strongly resilient to theF -attacks ifand only if it is weakly resilient to theG-attacks, where thesetG is defined as

G = P ⊂ V : P = F1

⋃

F2, F1, F2 ∈ F.

Proof: First let us prove that F strong resilience impliesweak resilience to G-attacks. We prove that by contradiction.Suppose not, then there exists G ∈ G and G 6= φ, such that

Φ(w, uG, G) = Φ(0, 0, φ).

From def nition, we know that G = F1

⋃F2, where F1, F2 ∈

F and F1 6= F2. It can be shown that

uG = ua + ub,

where ua (ub) is non-zero only for the nodes that belong toF1 (F2). By linearity, it can be proved that

Φ(w, ua, G) = Φ(0, −ub, G).

Moreover, we can f nd uF1 and uF2 such that

Φ(w, uF1 , F1) = Φ(w, ua, G),Φ(0, uF2 , F2) = Φ(0, −ub, G),

which contradicts the fact that the system is strongly resilientto F -attacks.The proof of the converse is similar and hence is omitted.

As a result, we only need to consider weak resilience. Thefollowing def nition is needed:

Definition 3. Consider a linear system(A, B, C) of thefollowing form:

xk+1 = Axk +Buk, x0 = 0,

yk = Cxk.

The system is called left-invertible ifyk = 0 for all k impliesthat uk = 0 for all k.

From the def nition, it is easy to see that the followingtheorem holds.

Theorem 4. The network is weakly resilience toF -attacks ifand only if for all possibleF ∈ F , the system(A, [B BF ], C)is left invertible.

The following theorem characterize the relationship betweenthe left invertibility and (A, [B BF ], C).



Theorem 5 ( [10]). Thefollowing statements are equivalent:

1) The system(A, B, C) is left invertible.2) The transfer functionK(z) = C(zI − A)−1B has

normal rank5 m.

Once the matrix A is specif ed, one can use the abovetheorem to check if the system is weakly or strongly resilientto F -attacks. However, this could be cumbersome since oneneeds to check for all the possible F ∈ F . Moreover, theabove theorem does not provide any insight on the relationshipbetween network topology and resilience. In the next subsec-tion, we characterize such relationship, based on the genericproperties of structured systems.

B. Topological Conditions

In this subsection, we want to relate weak and strongresilience with the topology of the network. We f rst introducethe concept of structured system. Consider the linear system

xk+1 = Axk +Buk,

yk = Cxk.

The system is structured if the elements of matrices A, B, C

are either f xed zeros or free parameters. Suppose that thereare l elements in A, B, C matrices that are not f xed zeros.We can parameterize the system by a vector λ ∈ R

l. As aresult, we will write the structured system as

xk+1 = Aλxk +Bλuk,

yk = Cλxk.

A structured linear system can be represented by a di-rected graph H = VH , EH, where VH = U

⋃X

⋃Y .

U = u1, . . . , up, X = x1, . . . , xn and Y = y1, . . . , ymrepresent the inputs, states and outputs respectively. p, n, mare the dimensions of inputs, states and outputs. The edge setis def ned as

EH = (uj, xi) : Bij is not a f xed zero.⋃

(xj , xi) : Aij is not a f xed zero.⋃

(xj , yi) : Cij is not a f xed zero..

For a structured system, we have the following theorem:

Theorem 6 ( [11]). Let the transfer function beKλ =Cλ(zI−Aλ)

−1Bλ and letnrank(Kλ) denote its normal rank.Define

r = maxλ∈Rk

nrank(Kλ), R = λ ∈ Rk : nrank(Kλ) 6= r.

ThenR has Lebesgue measure 0.

r is call the generic normal rank of the transfer functionKλ. The above theorem indicates that for almost all possiblerealizations of the structured system, the normal rank of thetransfer function will be r. We can further relate the genericnormal rank with the graph associated to the structured system.

5If a rational matrix M(z) has normal rank r, it means that the rank ofM(z) is r for all but f nitely many z.

Theorem 7 ( [11]). The generic normal rank of the transferfunctionKλ equals the maximal size of linking inH from U

to Y .

For our system, the A matrix is structured. The matricesB, BF , C have f xed zeros and ones. However, the f xedones can be changed to free parameters by proper scaling.Therefore, the system is structured. We are now ready to provethe main theorem

Theorem 8. If for all F ∈ F , there exists ap+|F | linking fromS

⋃F to T , then for almost all possible realizations ofA, the

system is weakly resilient againstF -attacks. If there exists anF ∈ F , such that there does not exists ap+ |F | linking fromS

⋃F to T , then the system is not weakly resilient against

F -attacks regardless ofA.

Proof: Consider the graph H = VH , EH associatedwith the structured system (A, [B BF ], C). It can be easilyseen that H is an augmented graph of the original topologyG = V, E. To be specif c, VH = V

⋃U

⋃Y , where U

contains p + |F | vertices and Y contains m vertices. Eachvertex in U is connected to a vertex in S

⋃F and each vertex

in Y is connected by a vertex in T . As a result, for everylinking in H from U to Y , there exists a linking from S

⋃F

to T and vice-versa. Therefore, by Theorem 5, 6 and 7 wecan complete the proof.Combining Theorem 8 and 3, we have the following corol-

lary:

Corollary 1. If for all F1, F2 ∈ F , there exists ap+|F1

⋃F2|

linking from S⋃F1

⋃F2 to T , then for almost all possible

realizations ofA, the system is strongly resilient againstF -attacks. If there existF1, F2 ∈ F , such that there does notexists ap+ |F1

⋃F2| linking fromS

⋃F1

⋃F2 to T , then the

system is not strongly resilient againstF -attacks regardless ofA.

Since the generic resilience of the system is only related tothe topology of the system, as shown in Theorem 8, we havethe following def nition:

Definition 4. A topologyG = (V,E) is said to be weakly(strongly) resilient toF -attacks, if and only if for anyF ∈ F(F1, F2 ∈ F), there exists anp+ |F | (p+ |F1 + F2|) linkingfrom S

⋃F (S

⋃F1

⋃F2) to T .

To illustrate Theorem 8, let us consider the network shownin Fig 2. Suppose at most two nodes can be compromised.There are 4 possible scenarios due to symmetry shown inFig 2(a)(b)(c)(d) respectively. The compromised nodes aredenoted as the red nodes and the linking is marked with redlines. It can be seen that the topology is weakly resilientto two compromised nodes and is strongly resilient to onecompromised node.In the next section we propose a technique to improve the

resilience of the system using trustworthy nodes.

IV. ENHANCING THE RESILIENCE WITH TRUSTWORTHYNODES

It is clear from the Section III that the resilience of networkis directly related to the connectivity of the topology. As a



S

T

(a)

S

T

(b)

S

T

(c)

S

T

(d)

Fig. 2. Linking from S⋃

F to T

result, one trivial way to improve the resilience is to addmore communication links to the network. However, such asolution is not always feasible due to various constraints suchas bandwidth and energy. Another approach is to augmentsome routers with more states without modifying the topology,which will be discussed in this section.To be specif c, let us assume that a router i has nO

i outgoingneighbors. We assume that router i updates a vector stateXk,i ∈ R

nOi using the following update equation:

Xk+1,i = Ai,iXk,i +∑

j∈NI i

Ai,jzk,j,i, (13)

where Ai,i ∈ RnOi ×nO

i and Ai,j ∈ RnOi . The router sends

its outgoing neighbors a scalar message, which is a linearcombination of all its states:

zk,i,j = Di,jXk,i, j ∈ NOi , (14)

where Di,j ∈ R1×nO

i .

Remark 2. An augmented node needs more computationalpower to update its states. Also more bandwidth is neededsince the node needs to singlecast instead of broadcasting.

Since each state variable is a vertex in graph H associatedwith the structured linear system, an augmented node i will berepresented by a collection of nO

i nodes in H , all of which areconnected from the incoming neighbors and to the outgoingneighbors of i and interconnected with each other. As a result,augmented nodes increase the connectivity of the graph H .However, it is harder to detect an attack when the attackercompromises augmented nodes. As a result, it is important thatsuch nodes are well protected, which leads to the followingdef nition:

Definition 5. An augmented node is called trustworthy if itdoes not belong toT or S and is guaranteed to follow theupdate equation(13) and the transmission equation(14).

Remark 3. Tamper-resistant microprocessors can be used foraugmented nodes to prevent the adversary from compromisingthem.

In the rest of the paper we will assume that all theaugmented nodes are trustworthy.Given a graph G and the set of trustworthy node Q, we can

def ne a new graph G′ = V ′, E′ = M(G,Q) by removingtrustworthy nodes. To be specif c, f rst suppose there is onlyone trustworthy node in the network. As a result, Q = q,where p < q < n−m+ 1. Then

V ′ = V \q.

Def ne the set of incoming edges of q as EI(q) = (i, q) ∈ E :i ∈ N I

q and set of outgoing edges as EOq = (q, i) ∈ E : i ∈

NOq . Also def ne the set EIO

q = (i, j) : i ∈ N Iq , j ∈ NO

q ,which connects all the incoming neighbors of q to its outgoingneighbors. Note that EIO

q may not be a subset of E, whichmeans some of the links may not exist in E. Then E′ can bedef ned as

E′ = (E⋃

EIOq )\(EI

q

⋃

EOq ).

In other words, E′ is def ned as removing all the edgescontaining q and then directly connecting all the incomingneighbors of q to its outgoing neighbors. Suppose Q containsmore than one trustworthy node, then the function M can bedef ned recursively as

M(G, Q) = M(M(G, Q\q), q),

where q ∈ Q. We call G′ = M(G, Q) the reduced topology.Now we have the following theorem to characterize the effectof trustworthy node on the network.

Theorem 9. The sensor network with a set of trustworthynodesQ is generically weakly(strongly) resilient toF -attacksif the reduced topologyG′ = M(G,Q) is weakly(strongly)resilient toF -attacks, whereF is given by

F = F ⊂ V : |F | ≤ f, F⋂

S = φ, F⋂

Q = φ.

Proof: The proof can be carried out by constructing a oneto one correspondence of a linking in H(from U to Y ) to a



linking in G′ (from S⋃F to T ). The technical details of the

proof is omitted.The effect of trustworthy node can be illustrated by the

following example. Consider the network shown in Fig 3.Suppose that each node updates the state according to thefollowing equation:

xk+1,i =∑

j∈N I (i)

xk,j + δ1,iwk,1.

In other words, each router updates its state to the sum ofstates of its incoming neighbors. Suppose that the red node2 is compromised. The fusion center cannot detect that (seeTheorem 8). In fact, if the attacker injects a sequence u, it canbe proved that

Φ(w, u, 2) = Φ(w + u, 0, φ).

ST

12

3

45

6

Fig. 3. Wireless Sensor Network with Trustworthy Node

Such vulnerability is caused by the blue node 4 which actsas a bottle-neck for the network. If we instead replace node 4with a trustworthy node, and change its update equation to

Xk+1,4 = [xk,2, xk,3]′,

and transmission equations to

zk,4,5 = [1, 0]Xk,4, zk,4,6 = [0, 1]Xk,4,

then it can be proved that the system is weakly resilient toone malicious node by Theorem 9. The reduced topology isillustrated in Fig 4.

ST

12

3

5

6

Fig. 4. Reduced Topology of the Sensor Network after Removing ofTrustworthy Node

V. CONCLUSION AND FUTURE WORK

In this paper we propose a secure data transmission protocolfor sensor networks. We def ne the resilience of the protocolagainst malicious attacks and relate it with the topology ofthe network. We further discuss how to improve the resilienceof the network by using trustworthy sensors. Future workincludes the development of eff cient algorithms to identify theset of malicious nodes and reconstruct the input. We wouldalso like to generalize the results to distributed control systems.

REFERENCES

[1] N. P. M. (Ed.), Sensor Networks and Configuration. Springer, 2007.[2] E. Byres and J. Lowe, “The myths and facts behind cyber security risks

for industrial control systems,” in Proceedings of the VDE Kongress.VDE Congress, 2004.

[3] A. A. Cardenas, S. Amin, and S. Sastry, “Research challenges for thesecurity of control systems,” in HOTSEC’08: Proceedings of the 3rdconference on Hot topics in security. Berkeley, CA, USA: USENIXAssociation, 2008, pp. 1–6.

[4] ——, “Secure control: Towards survivable cyber-physical systems,” inDistributed Computing Systems Workshops, 2008. ICDCS ’08. 28thInternational Conference on, June 2008, pp. 495–500.

[5] S. Amin, A. Cardenas, and S. S. Sastry, “Safe and secure networkedcontrol systems under denial-of-service attacks.” in Hybrid Systems:Computation and Control. Lecture Notes in Computer Science.Springer Berlin / Heidelberg, April 2009, pp. 31–45. [Online].Available: http://chess.eecs.berkeley.edu/pubs/597.html

[6] A. Willsky, “A survey of design methods for failure detection in dynamicsystems,” Automatica, vol. 12, pp. 601–611, Nov 1976.

[7] A. S. Willsky and H. L. Jones, “A generalized likelihood ratio approachto the detection and estimation of jumps in linear systems,” IEEETransactions on Automatic Control, vol. 21, pp. 108–112, Feburary1976.

[8] F. Pasqualetti, A. Bicchi, and F. Bullo, “Consensus computation inunreliable networks: A system theoretic approach,” IEEE Transactionson Automatic Control, Feb. 2010, to appear.

[9] S. Sundaram, M. Pajic, C. Hadjicostis, R. Mangharam, and G. J. Pappas,“The wireless control network: monitoring for malicious behavior,” inIEEE Conference on Decision and Contro, Atlanta, GA, Dec 2010.

[10] H. L. Trentelman, A. Stoorvogel, and M. Hautu, Control Theory forLinear System. Springer, 2001.

[11] J. W. van der Woude, “On the structure at inf nity of a structured system,”Linear Algebra and its Applications, vol. 148, pp. 145–169, April 1991.



Towards A Unifying Security Framework forCyber-Physical Systems

Quanyan Zhu and Tamer Basar

Abstract—The critical infrastructures that support major in-dustries are increasingly dependent on information systems fortheir command and control. The migration of legacy controlsystems to new communication technologies enables more ro-bust data, quicker time to market and interoperability butalso introduces new cyber-related vulnerabilities and risks. Thesecurity solution to control systems needs to be built based onthe understanding of the trade-off between system security andaccessibility. We propose a unifying theoretical framework thataddresses the cross-layer design of security architecture and itsconfigurations from its interactions with the physical layer plantand the security mitigation strategies from the perspective ofcontroller design as well as cyber security defense strategies.Our holistic viewpoint on the cyber-physical systems allows usto understand the fundamental limitations and design principlesfor secure control systems.

I. INTRODUCTION

The integration of IT infrastructure with industrial controlsystems has created a closed network of systems embeddedin the publicly accessible network. The integration bringsmany cost and performance benefits to the industry as wellas arduous challenges of protecting the automation systemsfrom security threats [3]. IT networks and automation systemsoften differ in their security objectives, security architectureand quality-of-service requirements. Hence, the conventionalIT solutions to security cannot be directly applied to controlsystems. It is also imperative to take into account many controlsystem specific properties when developing new network se-curity solutions. In this paper, we study the influence of cybersecurity policies on various control system performances anddevelop an optimal security policy for a networked controlsystems.

Due to the total isolation of control systems from theexternal networks, control system security has historicallybeen defined as the level of reliability of the system, withdesigns aimed at increasing the reliability and the robustnessof the system, rather than considering the network security.Hence, merging a modern IT architecture with an isolatednetwork that does not have built-in security countermeasureis a challenging task. From a mitigation perspective, simplydeploying IT security technologies into a control system maynot be a viable solution. Although modern control systems usethe same underlying protocols that are used in IT and businessnetworks, the very nature of control system functionality

Q. Zhu and T. Basar are with Dept. ECE and CSL, Univer-sity of Illinois, 1308 West Main St., Urbana, IL, 61801, USA. E-mail:zhu31,[email protected]. This research was supported in part byBoeing Company through the Information Trust Institute, University ofIllinois.

Fig. 1. Security of cyber-physical systems: the physical and the cyber layersof control system interact with each other.

may make even proven security technologies inappropriate.Sectors such as energy, transportation and chemical, havetime-sensitive requirements. Hence, the latency and throughputissues with security strategies may introduce unacceptabledelays and degrade acceptable system performance. The re-quirement of accessibility of control systems is a distinct factorthat distinguishes it from its IT counterpart. Understandingthe system tradeoff between security and system accessibilitywould enable us to learn the fundamental limitations anddesign principles for secure control systems.

Security solutions at the physical and the cyber layer ofan integrated control system need to take into account theinteraction between these two layers. Figure 1 illustrates theconcept of security interdependence between the cyber systemand the physical plant. In this paper, we adopt a holistic cross-layer viewpoint towards a hierarchical structure of control sys-tems. The physical layer is comprised of devices, controllersand the plant whereas the cyber layer consists of routers,protocols, and security agents and manager. The physical layercontrollers are often designed to be robust, adaptive, andreliable for physical disturbances or faults. With the possibilityof malicious behavior from the network, it is also essential forus to design controllers that take into account the disturbancesand delay resulting from routing and network traffic as wellas the unexpected failure of network devices due to cyberattacks. On the other hand, the cyber security policies are oftendesigned without consideration of control performances. Toensure the continuous operability of the control system, it isequally important for us to design security policies that providemaximum level of security enhancement but minimum level ofsystem overhead on the networked system. The physical andcyber aspects of control systems should be viewed holisticallyfor analysis and design.



Fig. 2. Defense in-depth strategies for control systems: a multitude of securitydevices and agents are cascaded to offer countermeasures against potentialthreats.

II. A UNIFYING SECURITY MODEL

The cyber infrastructure serves as an interface betweenthe controller and the physical plant. Depicted in Figure 3,the control signals are sent through a security enhanced ITinfrastructure such as wireless networks, the Internet and localarea networks (LANs). The security architecture of the ITinfrastructure is designed to enable the security practice ofdefense in depth for control systems [3]. Figure 2 illustratesthe concept of in-depth defensive strategies. The cascadingcountermeasures using a multitude of security devices andagents, ranging from physical protections to firewalls andaccess control, can offer the administrators more opportunitiesfor information and resources control with the advent ofpotential threats. However, it also creates possible issues on thelatency and the packet drop rate of communications betweenthe controller and the plant. We propose a unifying securitymodel for this cyber-physical scenario and investigate themitigating strategies from the perspectives of control systemsas well as of cyber defenses. At the physical layer, we oftenaim to design robust or adaptive controllers that take intoaccount the uncertainties and disturbances in the system toenhance robustness and reliability of the system. At the cyberlevel, we often employ IT security solutions by deployingsecurity devices and agents in the network. The securitydesigns at the physical and the cyber layers usually followdifferent goals without a unifying framework. In this section,we describe a cross-layer security model that connects thecyber security with the physical layer control system.

We model the physical plant with the following continuous-time dynamics:

x(t) = A(t)x(t) +B(t)u(t) +D(t)w(t), x(0) = x0, t > 0.(1)

where x0 ∈ Rn is the initial condition; x(t) ∈ Rn is thesystem state; u(t) ∈ Rm is the control variable; w ∈ Rl is the

Fig. 3. Security architecture of control systems serves as an interface betweenthe physical plant and the controller.

disturbance variable; A(t) ∈ Rn×n, B(t)×Rn×m, D(t)×Rn×lare system parameters defined for t > 0. In order to enhancethe system’s response to uncertainties, faults, and disturbances,we adopt an H∞ optimal control approach to design robustcontrols for the system in (1) [6]. The goal of the controldesign is to find an optimal control u∗(t) that optimizes theperformance index in the worst-case of w∗(t)

Lγ(u,w) = |x(tf )|2Qf+

∫ tf

0

|x(t)|2Qf (t) + |u(t)|2 − γ2|w(t)|2

,

(2)where Qf > 0, Q(t) > 0, t ∈ [0, tf ], γ > 0, tf is the terminaltime, all matrices have piecewise-continuous entries, and sodo u and w. The cost criterion (2) is the performance indexof the soft-constrained differential game associated with theperformance index (3) of the disturbance attenuation problem[6], [7]:

L(u,w) = |Q(tf )|2Qf+

∫ tf

0

|x(t)|2Q(t) + |u(t)|2

. (3)

The associated differential game has two players. One player(P1) is the controller who minimizes (2) and the other player(P2) is the disturbance who maximizes it.

The control signals of the system (1) are sent over asecurity-enhanced communication network where security de-vices such as intrusion detection systems (IDSs), intrusionprevention systems (IPS), firewalls, authentication servers, etc.are deployed. Figure 4 describes a simple security archi-tecture for the access of the control system LAN from theInternet. A control signal packet has to pass many securityzones to reach the local network. We model the securityprocess in a given architecture by a queueing network. Eachqueue represents a device or a checkpoint where controlsignal traffic has to go through. Hence, the entire securityarchitecture can be represented by a queueing network. LetN = n1, n2, · · · , nN be the set of N security devices thatform an acyclic network of queues. The connection of the



Fig. 4. An illustration of cyber defense strategy against potential threat overthe Internet.

queues is represented by a graph G := (N , E), where E isthe set of directed edges that connect between two queues.Denote by Ni ⊂ N the set of devices that ni connects toafter its service, i.e. Ni := j ∈ N : (i, j) ∈ E. Weassume that the external packet arrival to a device ni ∈ Nfollows a Possion process of rate λi and the service time of thedevice follows an exponential distribution with rate µi. Packetsare examined on a first-come, first served basis. A packetcompleting the scan or inspection at ni will move to somenew device nj with probability pij or will be dropped withprobability pi := 1−

∑j∈Ni

pij . Under these assumptions, theset of security agents form a Jackson network in which eachdevice operates as an M/M/1 queue [4], [5].

At a finer level of granularity, within a security device,for example, an IDS/IPS, the process of comparing networktraffic against a pre-defined rule set or a set of known attacksignatures can also be viewed as a queueing system. In thefollowing, we study the configurations of IDS/IPS systemsand their effects on the design of the control system. LetL = l1, l2, · · · , lL be the set of L rules in an IDS/IPSsystem. A rule has its predefined signatures against whichan IDS/IPS makes a comparison. An IDS raises an alert toa system administrator once it recognizes an attack pattern orany deviation from what has been defined as normal/allowabletraffic. As an extension to IDS, an IPS can drop packets toprevent malicious activities in addition to the identificationand information logging of the activities. The process ofexamination and recognition of a pattern in a rule can also beviewed as a queue. Let L ⊂ L be a subset of L libraries loadedor configured to examine the network traffic. We assume thatthese libraries in the set L form a tandem queue where theinspections of the rules are in a series connection. Figure 5illustrates the tandem queue model of the IDS/IPS system.Each queue represents a rule in L and is assumed to be M/M/1.The probability pij captures the rate of a legitimate packetpassing from li to lj . pij = 1 if the rule li is an IDS rulewhereas pij 6 1 if the rule belongs to an IPS. The probability1 − pij , li, lj ∈ L, is the false negative rate of an IPS rule,i.e., packet dropping rate of legitimate traffic. Figure 6 depictsthe block diagram that characterizes a single IDS/IPS systemby its arrival rate λ, service rate µ and package drop ratep. Henceforth we drop the j index of the probability pij forsimplicity, because of the series connection of the queues.

We consider two major effects from IDS/IPS: delay and

Fig. 5. Queueing network model for cyber defense: L M/M/1 queues in aseries connection to model IDS/IPS.

Fig. 6. Block diagram of IDS/IPS model with arrival rate λ, service rate µ,and packet loss rate p.

packet loss. Previous research on packet loss in networkcontrol systems include [1], [2]. Let τ be the time delay forthe state information to reach the controller. The admissiblecontrol policies for P1 will hence be of the form

u(t) = µ(t, x[0,t−τ ]), t > τ

= µ(t, x0), 0 6 t < τ, (4)

and the control policies for P2 will be of the form

w(t) = ν(t, x[0,t−τ ]), t > τ

= ν(t, x0), 0 6 t < τ. (5)

We denote the class of such controllers for P1 and P2 byMi

τD, i = 1, 2. The delay from IDS/IPS is given by the meansojourn time

τ =1

µ1+

L∑j=2

pj−1

µj. (6)

Let Θ(t) = diag(θ1(t), θ2(t), · · · , θm(t)) ∈ Rm×m be thepacket drop rate as a result of the IDS/IPS system. A diagonalentry θi(t), i = 1, 2, · · · ,m, is a Bernoulli random variable thatdenotes the packet drop rate of i-th control input. θi(t) = 1with probability 1 − p and θi = 0 with probability p. Notethat different control inputs may go through different routesor different IDSs/IPSs to reach the control system and henceθi can take different forms for each control input i. The controlsystem dynamics (7) can be modified into

x(t) = A(t)x(t) +B(t)Θ(t)u(t) +D(t)w(t), t > 0, (7)

and the performance index becomes

Lγ(u,w) = EΘ[Lγ(u,w)]. (8)

For simplicity, we can assume all control inputs to have thesame packet drop rate θ, i.e., Θ(t) = θI, which is given by

θ = 1−L∏j=1

pj (9)



It is easy to deduce from (6) and (9) that if we enhancethe security by loading a larger set of IDS rules, it will resultin longer delays; and if we secure the system with more IPSrules, it will be more likely to incur a higher level of packetloss. Given the IDS/IPS configuration, an optimal control toachieve disturbance attenuation for a given γ is

u = uγ(x− τ, t) = −E[Θ(t)]BT (t)Zγ(t)η(t), 0 6 t 6 tf ,(10)

where Zγ is a solution to the following generalized Riccatiequation (GRDE):

Z +ATZ + ZA+Q− Z(E[Θ2]BBT − γ−2DDT )Z = 0; (11)Z(tf ) = Qf ,

and η(·) is generated by the (infinite-dimensional) compen-sator

η(t) = Φ(t, t− τ)x(t− τ)−∫ t

t−τΦ(t, s)

·(E[Θ2(s)]B(s)BT (s)− γ−2D(s)DT (s)

)·Z(s)η(s)ds, η(s) = 0, 0 6 s < τ, (12)

with Φ(·, ·) being the state transition matrix associated withA(·).

For every t′ > τ , we have an open-loop type of Riccatiequation defined on [t′ − τ, τ ]:

St′ +ATSt′ + St′A+Q+ γ−2St′DDTSt′ = 0; (13)

St′(t′) = Z(t′); t′ − τ 6 t 6 t′, t′ ∈ [τ, tf ].

Each time point t′ > τ corresponds to one Riccati differentialequation. Let γCL be the optimal attenuation level underclosed-loop information pattern. We define

γτD := infγ > γCL : For every t′ > τ , (13) does nothave a conjugate point on the corresponding

interval. (14)

Theorem 1 ([6]): Consider the linear-quadratic zero-sumdifferential game associated with (7) and (9), with delayed-state information (4) and (5). Let γτD be defined by (14).Then:(1) If γ > γτD, the game admits a unique noise-insensitive

saddle-point solution, which is given by the policies (10).The saddle-point value is given by

L∗γ := Lγ(u∗, w∗) = xT0 Z(0)x0. (15)

(2) If γ < γτD, the differential game does not admit a saddlepoint and has unbounded upper value.

The preceeding theorem reveals the following separationprinciple regarding the effect of cyber security on controlsystems:(1) For a given γ > γτD, the optimal saddle-point value is

only dependent on the packet loss rate Θ(t) as in (11).(2) For a given packet loss rate Θ(t) and its corresponding

Z(t), the optimal attenuation level is only dependent onthe delay as in (13).

We can make use of the above observation to find an optimalcyber security policy by taking into account the performanceindex of control systems. Let αi ∈ R+, li ∈ L be the utilityassociated with each IDS/IPS rule. The total rule set L canbe partitioned into a set of IDS rules LIDS and a set of IPSrules LIPS. According to the first principle, we can find anoptimal configuration L

∗IPS ⊂ LIPS on the set of IPS rules

that maximizes the total security utility subject to the controlsystem constraints for a sufficiently large γ, i.e.,

(IPS-OPT) maxL∗IPS∑li∈L

∗IPS

αi (16)

s. t. xT0 Z(0)x0 6 ρL, (17)

where ρL is the maximum acceptable cost on the performanceindex so that the feasible set of L

∗IPS is not empty. The con-

straint in (IPS-OPT) essentially can be turned into a constrainton the packet loss rate. The next level of optimization is tofind a configuration of IDS rules such that γ is satisfied giventhe packet loss rate as a result of the IPS configuration, i.e.,

(IDS-OPT) maxL∗IDS∑li∈L

∗IDS

αi (18)

s. t. γτD(Θ(t)) 6 γ. (19)

Every delay τ corresponds to a γτD. The constraint in (IDS-OPT) gives bounds on the range of feasible τ to choose from.

III. CONCLUSION

In this paper, we have proposed a unifying framework toaddress security issues in cyber-physical systems. We havemodeled the cyber security architecture by Jackson networksand the security devices by tandem queueing networks. Wehave studied two effects of the cyber architecture on controlsystems: delay and packet drop rate. We have incorporatedthese two factors into the H∞ controller design and proposeda cross-layer optimization framework for the design and con-figuration of cyber security.

REFERENCES

[1] B. Azimi-Sadjadi, Stability of networked control systems in the presenceof packet loss. Proc. 42nd IEEE Conference on Decision and Control(CDC’03, Dec. 2003; Maui, Hawaii).

[2] P. Bommannavar and T. Basar, Optimal control with limited control ac-tions and lossy transmissions. Proc. 47th IEEE Conference on Decisionand Control (CDC’08, Dec 9-11, 2008; Cancun, Mexico).

[3] M. Fabro and T. Nelson, Control systems cyber security: defense in-depth strategies, 2007 ISA Expo, INL Technical Report, INL/CON-07-12804.

[4] G. Bloch, S. Greiner, H. de Meer and K. S. Trivedi, Queueing Net-works and Markov Chains: Modeling and Performance Evaluation withComputer Science Applications, Wiley-Interscience; 2 edition (April 14,2006).

[5] H. Chen and D. D. Yao, Fundamentals of Queueing Networks, Springer;1 edition (June 15, 2001).

[6] T. Basar and P. Bernhard, H-infinity Optimal Control and RelatedMinimax Design Problems: A Dynamic Game Approach. Birkhauser,1995.

[7] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory.SIAM Series in Classics in Applied Mathematics, Philadelphia, January1999.



Management of Control System Information Security: Control

System Patch Management

Quanyan Zhu, Miles McQueen, Craig Rieger and Tamer Basar

Abstract—The use of information technologies in controlsystems poses additional potential threats due to the frequentdisclosure of software vulnerabilities. The management of infor-mation security involves a series of policy-making decisions onthe vulnerability discovery, disclosure, patch development andpatching. In this paper, we use a system approach to devisea model to understand the interdependencies of these decisionprocesses. Specifically, we establish a theoretical framework formaking patching decision for control systems, taking into accountthe requirement of their functionability. We illustrate our resultswith numerical simulations and show that the optimal operationperiod of control systems given the currently estimated attackrate is roughly around a half a month.

I. INTRODUCTION

The integration of new information technologies into con-trol systems in critical infrastructures enables more efficientmethods of communication as well as more robust data andinteroperability [6]. However, the usage of technologies withknown vulnerabilities exposes control systems to potentialexploits. Information security management becomes a crucialissue for control systems. The timing between the discoveryof new vulnerabilities and their patch availabilities is crucialfor the assessment of the security risk exposure of softwareusers [1], [3], [4], [5]. The security focus in control systemsis different from the one in computer or communicationnetworks. The application of patches for control systems needsto take into account the system functionability, avoiding theloss of service due to unexpected interruptions. The disclo-sure of software vulnerabilities for control systems is alsoa critical responsibility. Disclosure policy indirectly affectsthe speed and quality of the patch development. Governmentagencies such as CERT/CC (Computer Emergency ResponseTeam/Coordination Center) currently act as a third party in thepublic interest to set an optimal disclosure policy to influencebehavior of vendors [7].

The decisions involving vulnerability disclosure, patch de-velopment and patching are intricately interdependent. In thispaper, we introduce a model for information security manage-ment of control systems. We investigate the decision processesof vulnerability discovery, disclosure, patch development andcontrol system patching. In Figure 1, we illustrate the rela-tionship between these decision processes. A control systemvulnerability starts with its discovery. It can be discovered

Q. Zhu and T. Basar are with the Department of Electrical and ComputerEngineering and Coordinated Science Laboratory, University of Illinois, 1308West Main St., Urbana, IL, 61801, USA. E-mail:zhu31,[email protected];C. Rieger and M. McQueen are with Idaho National Laboratory, 2525Fremont Ave., Idaho Falls, ID 83402, USA. Email: Craig.Rieger,[email protected].

by multiple parties, for example, individual users, governmentagency, software vendors or attackers and hence can incurdifferent responses. The discoverer may choose to not discloseto anyone, may choose to fully disclose through a forum suchas bugtraq [2], may report to the vendor, or may provide to anattacker. Vulnerability disclosure is a decision process that canbe initiated by those who have discovered the vulnerability. Apatch development starts when the disclosure process reachesthe vendor and finally a control system user decides on theapplication of the patches once they become available. Anattacker can launch a successful attack once it acquires theknowledge of vulnerability before a control system patches itscorresponding vulnerabilities. The entire process illustrated inFigure 1 involves many agents or players, for example, systemusers, software vendors, government agencies, attackers. Theirstate of knowledge has a direct impact on the state of thevulnerability management.

We propose to compartmentalize the task of vulnerabilitymanagement into different submodules: discovery, disclosure,development and patching. The last two submodules are rela-tively convenient to deal with since the agents who involvedin the decision making are very specific to the process. Themodels for discovery and disclosure can be more intricate inthat these processes can be performed by many agents andhence specific models should be used for different agents tocapture their incentives, utility, resources and budgets. In thesubsequent sections, we establish models for control systempatching to assist users to make optimal patching decisions.

II. USER PATCHING

In this section, we establish a model for users to determinethe optimal time to patch for their control systems. In controlsystems, it is known that the attack rate is low and the patchingrate is low as well. It often occurs that users do not patch untilthere is a security alert, an available patch announcement or anexperienced security breach. The operation of control systemis separated into several operating periods. A control systemcannot halt its operation until the end of an operating period.Let Bk, k = 1, 2, · · · denote the k-th operating period sincethe last patching and Tk = Tk−1 + Bk =

∑ki=1Bi, where

T0 is the beginning of the first operation period. Let τ be thetime length between the start of the first operation period anda security alert or an attack. Let fτ (t), t > 0, be its probabilitydensity function, which is taken to be hyper-exponential withn phases and parameters λ = (λ1, · · · , λn) and normalized



Fig. 1. A holistic viewpoint towards vulnerability discovery, disclosure, development and patching. An attacker can discover a vulnerability or learn it froma disclosure process, eventually influencing the speed of patch application. A discoverer can choose to fully disclose through a forum or report to the vendoror may provide to an attacker. A vulnerability can be disclosed to a vendor for patch development or leaked to the attacker.

Fig. 2. An illustration of control system patching model.

weighting factor q = (q1, · · · , qn), i.e.,

fτ (t) =

n∑i=1

qigi(t) =

n∑i=1

qiλi exp(−λit),n∑i=1

qi = 1. (1)

Each phase i can be interpreted differently. For example, letg1(t) be the distribution of the arrival rate of alerts; g2(t) bethe arrival rate of an attack; g3(t) be the arrival rate of anannouncement of an available patch. We illustrate this modelin Figure 2. At every Tk−1, k > 1, a control system starts tooperate for a period of Bk and then stops for monitoring andpatching.

The decision of an administrator is to determine Bk so thatthe risk of an unpatched control system subject to potentialattacks is minimized. The decision-making process can beviewed as a black box as in Figure 3, where the decisioninput is the knowledge of the arrival rates of an attack; theintrinsic system parameters are the monitoring cost cm and theproduction cost parameters c0 and c1; and the decision outputis the operation period Bk. The input and output characteristicsof the decision process assist us in integrating it into the systemmodel in Figure 1.

A. A General Model

In this section, we introduce a generalized model for controlsystem patching. Let cm ∈ R+ be the monitoring cost at theend of each operation period and cp(tk, bk+1) : R2

+ → R+

be the operation cost of running the plant for bk+1 startingfrom tk. We have the following dynamic programming (DP)

Fig. 3. A system viewpoint towards control system patching decision.

equation to find the optimal decision policy to be taken at eachperiod, taking into account the whole lifetime of the system:

V ∗k (tk) = min

bk+1>0E[c(tk, bk+1)+P(τtk > bk+1)V

∗k+1(tk+1)],

(2)where bk is the decision variable of operating time; V ∗

k (tk)represents the optimal cost at time tk. The term P(τtk > bk+1)represents the transition probability given by the conditionalprobability

P(τtk > bk+1) := P (τ > tk + bk+1|τ > tk). (3)

The term E[c(tk, bk+1)] is the stage cost at tk given by

E[c(tk, bk+1)] = γE[(bk+1−τtk)1(τtk6bk+1)]+cm+cp(tk, bk+1),(4)

where γ denotes the unit cost of untimely patching; τtk is theconditional residual time counting from tk given that τtk > tk.By solving the DP equation, we can find a dynamic policy forthe operation of the plants at each starting time tk.

B. A Simplified Model

In this subsection, we simplify the general model by invok-ing the following assumptions.

Assumption 1: (A1) The arrival of security alerts orbreaches form a single Poisson process with rate λ andthe arrival time τ and the conditional residual time τt areexponentially distributed with parameter λ.

Let C0k = 1

2c0b2k be the cost for operating a plant non-stop

for a period of time bk. Denote by C1k = c1bk the linear gain



or profit from running the plant. Hence, we can impose thefollowing assumption:

Assumption 2: (A2) The cost cp is given by

cp(tk, bk+1) := C0k − C1

k =1

2c0b

2k − c1bk (5)

Under assumptions (A1) and (A2), due to the memorylessproperty of exponential distribution, the DP equation in (2)can be simplified to

V ∗(λ) = minb>0

γE[(b− τ(λ))1(τ(λ)6b)] +

1

2c0b

2 − c1b

+P (τ(λ) > b)V ∗(λ) . (6)

For each fixed b > 0, V (λ) can be solved from (6) (withoutthe minimum) to yield

V (λ, b) =γE[(b− τ(λ))1(τ(λ)6b)] +

12c0b

2 − c1b1− P (τ(λ) > b)

. (7)

Note that in the above,

E[(b− τ(λ))1(τ(λ)6b)] =λb− 1 + exp(−λb)

λ; (8)

and the term in the denominator of (7) is as follows:

P (τ(λ) > b) = exp(−λb). (9)

Hence, to solve the DP is equivalent to finding a solution tothe optimization problem (R-OPT) that takes into account therisk factor of potential threats and attacks1:

(R-OPT) V ∗(λ) = minb>0

V (λ, b) (10)

Note that a simple solution for operation time b withoutsecurity risk consideration, i.e., ignoring the potential coststhat can be incurred by attacks in (R-OPT), is based on a costand benefit analysis solving the risk-free optimization problem(NR-OPT)2:

(NR-OPT) min1

2c0b

2k − c1bk + cm, (11)

which yields a benchmark solution b? = c1/c0.Theorem 1: Optimization problem (10) is a strictly convex

program and admits a unique solution.Proof: We can show this property by taking first and

second order derivatives of the objective function.The problem formulation can be further generalized to

include additional features such as law of small numbersand noncooperative behavior between the attacker and thedefender.

Remark 1: In the DP formulation, we can add a discountfactor δ ∈ (0, 1) in (2) to capture the short-sightedness of thecontrol system users. In addition, we can have another similarDP to model the attacker’s response to user’s decision making.The attacker may reduce the attack rate when he becomes lesssuccessful and foresees increasing efforts in attacks. Hence, we

1“R-OPT” stands for risk-based optimization.2“NR-OPT” stands for no-risk-based optimization.

Fig. 4. Operation period vs. monitoring cost.

Fig. 5. Operation period vs. attack rate.

will have a game-theoretical situation between the attacker andthe defender [8].

V ∗(b) = minλ>0

−βE[(b− τ(λ))1(τ(λ)6b)] + ca(τ(λ), b)

+P (τ(λ) > b)V ∗(b) , (12)

where β ∈ (0, 1) is the discount factor of the attacker;ca(τ(λ), b) is the cost of an attacker when he attacks withrate λ.

Remark 2: We can employ the same technique to study thevendor’s decision-making on patch development and also thegovernment’s disclosure process. The government may waitfor a period of Bk to review its decision on whether to disclosea certain vulnerability and λ can be viewed as the rate ofreports of cases to the agency.

III. SIMULATION

In this section, we numerically solve the DP equations fora given scenario of control systems. Set the parameters c1 =10, c0 = 0.1, γ = 10, cm = 10, λ = 0.001 as the nominalcase. In Figure 4, we show the optimal operation period versusthe monitoring cost. It can be seen that when the cost getshigher, the control system cannot afford a frequent checkingand monitoring and hence the operation period increases. InFigure 5, we show the optimal operation period versus theattack rate. We observe that when the attack rate is high, thecontrol system need to decrease its operation period to monitorand update its system more often.



Fig. 6. Operation period vs. attack rate.

Fig. 7. Update of attack rate in response to the patching strategy of thecontrol system at each iteration.

In the next numerical simulation, we examine the cost of theattacker and its impact on its attack rate. We let ca(τ(λ), b) =c1,aλ+cm,a, where c1,a is the unit cost on an attempted attackand cm,a is a fixed cost for an attacker to be able to launchan attack. Set c1,a = 200000, ka = 1, b = 40, cm,a = 10. InFigure 6, we show the attacker’s response to the patching rateof a control system. It can be seen that the higher the patchingrate, the less frequent an attack will happen.

Combining the response of an attacker and a control system,we examine the best response dynamics of the two playersand find the Nash equilibrium strategies of the attacker anddefender. We start with the patching rate of a control systemset to be b0 = 1. Within two iterations, we observe that b1 =b2 = 14 which gives the equilibrium patching strategy b∗ = 14for a control system. On the other hand, we start with the attackrate λ0 = 1. By iterations, we observe λ1 = 0.00536, λ2 =0.00269, λ3 = 0.00267 which gives an equilibrium strategyfor an attacker to choose an attack rate of λ∗ = 0.00267. InFigures 7 and 8, we illustrate the attacker’s response to thecontrol system patching strategy and the defender’s responseto attack rate at each iteration respectively. We can see thatafter two or three iterations, both players reach their Nashequilibrium.

IV. CONCLUSION

In this paper, we have proposed a systematic approach totackle the management of control system information secu-rity by compartmentalizing the processes into multiple input-

Fig. 8. Update of operation period in response to the attack rate at eachiteration.

output blocks. We have investigated in detail the decisionprocess of control system patch management and establisheda theoretical model to find an optimal operation period of acontrol system taking into account the risks of potential threatsand the operation costs. A similar framework can be employedto study the disclosure process. From the simulation results,we notice that an optimal operation period of control systemsgiven the currently estimated attack rate is roughly around halfa month.

It is important for us to point out that the model describedin the paper is idealized in that we have assumed that anattack can be prevented by a timely patching decision. How-ever, in reality, for zero-day vulnerabilities, an attack can besuccessfully made regardless of patch rate. Hence, it is alsoessential to study the resilient control design in the event ofunanticipated or rare attacks. As future work, we intend toextend the theoretical framework here to study the decisionsinvolving vulnerability disclosure and the R&D developmentof control patches.

REFERENCES

[1] S. Frei, B, Tellenbach and B. Plattner, “0-Day Patch: ExposingVendors (In)security Performance”, BlackHat, 2008, Europe

[2] Bugtraq, http://www.securityfocus.com/[3] M. A. McQueen, W. F. Boyer, T. A. McQueen and S. McBride,

“Empirical Estimates of 0Day Vulnerabilities in Control Systems”,SCADA Security Scientific Symposium, January 21-22, 2009

[4] S. Ragan, “The new era of vulnerability disclosure -a brief chat with HD Moore”, Aug. 16, 2010, URL:http://www.thetechherald.com/article.php/201033/6025/The-new-era-of-vulnerability-disclosure-a-brief-chat-with-HD-Moore

[5] The H Security, “Pressure mounts for a swifter response to vulnera-bilities”, URL: http://www.h-online.com/security/news/item/Pressure-mounts-for-a-swifter-response-to-vulnerabilities-1050624.html

[6] M. Fabro and T. Nelson, “Control systems cyber security: defense in-depth strategies”, 2007 ISA Expo, INL Technical Report, INL/CON-07-12804.

[7] Symantec Inc, Symantec Internet Security Threat Report.http://www.symantec.com, 2003

[8] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory,2nd edition, Classics in Applied Mathematics, SIAM, Philadelphia,1999.



Secure Routing in Smart GridsQuanyan Zhu, Dong Wei and Tamer Basar

Abstract—The migration of power grids from an isolatednetwork to a public communication network has created manychallenging issues in the security of smart grids. One of them isthe assurance of secure routing of PMU and smart meter datain the open network, which is enabled by the adoption of IP-based network technologies. The quality-of-service in routing ismeasured by the data integrity, latency and packet loss rate. Weleverage the hierarchical structure of power grids and propose arouting protocol that maximizes the quality-of-service along therouting path to the control room. In addition, we optimize thedata communication rates between the super data concentratorat the penultimate level with the control center. We propose ahybrid structure of routing architecture to enable the resilience,robustness and efficiency of the smart grid.

I. INTRODUCTION

To meet the challenge of climate change, Smart Grid (SG)is evolving rapidly from a relatively closed communicationnetwork to an open network. Wide use of Phasor MeasurementUnits (PMUs) and smart meters (SMs) will help electricitypower transmission and distribution system operators under-stand better power usage pattern and delivery of electricityenergy to end users in a cost-effective way. It is forecastedthat 276 million smart grid communication nodes will beshipped worldwide during the period from 2010 to 2016,with annual shipments increasing dramatically from 15 millionin 2009 to 55 million by 2016 [1]. The current dedicatednetwork or leased line communication methods are not cost-effective to connect large number of PMUs and SMs. Thus,it is foreseen that IP-based network technologies are widelyadopted since they enable data to be exchanged in a routablefashion over an open network, such as the Internet [2]–[6].This will bring benefits such as efficiency and reliability, andrisks of cyber attacks as well. Without doubt, SG applicationsbased on PMUs and SMs will change the current fundamentalarchitecture of communication network of power grid, andbring new requirements for communication security. Delay,incompleteness and loss of PMU and SM data will adverselyimpact SG operation in terms of efficiency and reliability.Therefore, it is important to guarantee integrity and availabilityof those PMUs and SMs data. To meet the Quality-of-Servicerequirements in terms of delay, bandwidth and packet loss rate,QoS-based routing technologies have been studied by bothacademia and the telecommunications industry [7]–[10]. Un-like video and voice, data communications of PMUs and SMshave different meanings of real-time and security, especially

Q. Zhu and T. Basar are with the Department of Electrical and ComputerEngineering and Coordinated Science Laboratory, University of Illinois, 1308West Main St., Urbana, IL, 61801, USA. E-mail:zhu31,[email protected];D. Wei is with Siemens Corporation, Corporate Research, Princeton, NJ, USA.E-mail: [email protected]. The research of Q. Zhu and T. Basar wassupported in part by a grant from DOE.

in terms of timely availability [2], [11]–[15]. Therefore, QoS-based and security-based routing schemes for smart grid com-munications should be studied and developed to meet SmartGrid application requirements in terms of delay, bandwidth,packet loss and data integrity.

In this paper, we leverage the hierarchical structure of SGand propose a secure routing protocol for the emerging appli-cations in SG. In Section II, we establish a hierarchical routingframework for local household data to reach a super dataconcentrator. In Section III, we propose an optimal mechanismfor the communication between super data concentrators andthe control room. We conclude in Section V with future workand challenges.

II. SECURE HIERARCHICAL ROUTING

SG has a hierarchical structure that is built upon the currenthierarchical power grid architecture. The end-users, such ashouseholds, communicate their power usage and pricing datawith a local area substation which collects and processes datasfrom SMs and PMUs. In SGs, the path for the measurementdata may not be predetermined. The data can be relayedfrom smaller scale data concentrators (DCs) to some superdata concentrators (SDCs) and then to the control room.With the widely adopted IP-based network technologies, thecommunications between households and DCs can be in amulti-hop fashion through routers and relay devices. The goalof each household is to find a path with minimum delay andmaximum security to reach DCs and then substations. Thisoptimal decision can be enabled by the automated energymanagement systems built in SMs. Figure 1 illustrates thephysical structure of the SG communication network. ThePMUs and SMs send data to DCs through a public network.DCs process the collected data and send the processed data toSDCs through (possibly) another public network.

The physical communication structure from local metersto the control room can be logically divided into severallevels. Figure 2 illustrates a snap shot of routing paths in asimplified example of the hierarchical structure of the physicalcommunication architecture, which is divided logically into 5levels. For simplicity, we only depict only level of routers inFigure 2. In practice, there can be multiple levels of routersand they can be found in the public network between DCsand SDCs as well as SDCs and the control room. In thedepicted SG, the data from a PMU or a SM has to makefour hops to reach the control room. The decision for ameter to choose a router depends on the communication delay,security enhancement level and packet loss rate. In addition,the decisions for a DC to choose a SDC also depends onthe same criteria. The communication security at a node is



Fig. 1. An example of the physical structure of the hierarchical SGcommunication network.

Fig. 2. A snapshot of routing paths in hierarchical SGs.

measured by the number of security devices such as firewalls,intrusion detection systems (IDSs) and intrusion preventionsystems (IPSs) deployed to reinforce the security level at thatnode. We assign higher utility to network routers and DCsthat are protected by a larger number of firewalls, IDSs/IPSsand dedicated private networks in contrast to public networks.This relatively simple metric only considers one dimension ofthe control system cyber security. It can be further extended toinclude more security aspects by considering the authorizationmechanisms, the number of exploitable vulnerabilities, poten-tial damages as well as recovery time after successful attacks.The readers can refer to [16]–[19] for more comprehensivemetrics.

A trade-off with higher security is the latency and packetloss rate incurred in data transmission. A secure networkinevitably incurs delays in terms of processing (encryt-ing/decrypting) and examining data packets. We can model theprocess of security inspection by a tandem queueing network.Each security device corresponds to an M/M/1 queue whoseexternal arrival rate follows a Poisson process and the service

time follows an exponential distribution. The latency caused bythe security devices such as IDSs/IPSs, are due to the numberof pre-defined attack signatures and patterns to be examined.In addition, devices such as IPSs can also lead to a high packetloss due to their false negative rates in the detection.

Furthermore, a node with higher level of security maybe preferred by many meters or routers, eventually leadingto a high volume of received data and hence higher levelof congestion delay. This fact motivates us to employ agame-theoretical approach to analyze the distributed routingdecisions in SGs. We are interested in mixed Nash equilibriumas a solution outcome for two reasons. Firstly, mixed Nashequilibrium always exists for a finite matrix game [20] andmany learning algorithms such as fictitious play and replicatordynamics find mixed Nash equilibrium [22]. Secondly, therandomness in the choice of routes makes it harder for anattacker to map out the routes in SGs.

A. Game-Theoretic Model

Let M = m1,m2, · · · ,mM be the set of M meters ina local area SG. Let N = n1, n2, · · · , nM be the set ofrouters, DCs and SDCs in SG. Let CS denote the controlstation. The entire communication infrastructure is representedby I := 〈,N ,M, CS〉. Let L = 1, 2, · · · , L be the set of Lhierarchies in the infrastructure I. Denote by Nl, l ∈ L, theset of nodes at level l. By default, the terminal level L consistsof only the control station CS and the initial level 1 consistsof M meters. The devices in N form layers between levels1 and L. Denote by li ∈ L the level where a node ni ∈ Nresides. A meter mi chooses a node a(mi,2) ∈ N2 at level2. The chosen node a(mi,2) picks the next hop a(mi,3) ∈ N3

and a similar decision is made by node a(mi,3) at level 4.The process ends at penultimate level L−1 where every nodechooses to send to CS . The path Pi,mi ∈ M formed bymi → a(mi,2) → a(mi,3) · · · a(mi,L−1) → CS is the routetaken by the meter mi to reach CS . Denote by default thata(mi,1) := mi. Let ua(mi,l)

(a(mi,l+1), a(m−i,l+1)),mi ∈ M,be the stage utility of node a(mi,l) choosing a route betweena(mi,l) and a(mi,l+1), where a(m−i,l) denotes the set of choicesmade of the other meters at the same level, more precisely,

a(m−i,l) := a(mj ,l) ∈ Nl, j 6= i, mi,mj ∈M.

The utility along the path Pi for mi is given by the sum ofthe stage utilities

Ui(Pi) :=L∑

l=2

ua(mi,l). (1)

The goal of a meter mi is to maximize its total utility Ui.Note that the utility depends on not only its own choice butthe paths of other meters as well. Since the routing decisions atthe levels beyond the second one are made by the relay nodesin the set N , the total utility maximization can be decomposedas follows:

maxPi

Ui(Pi) = maxa(mj,2)

∈N2

ua(mj,1)+ maxPa(mj,2)

Ui(Pa(mj,2)) (2)



Such a decomposition allows us to find the optimal path usinga backward induction from the last stage L. The mixed strate-gies yielded by this method belong to a class of behavioralstrategies and carry the property of strong consistency [20].

We choose the stage utility as a ratio between the securityperformance index and the delay.

ua(mi,l)(a(mi,l+1), a(m−i,l)) =

pa(mi,l+1)

ra(mi,l+1)

τa(mi,l)(a(mi,l+1), a(m−i,l+1))

, mi ∈M, (3)

where ra(mi,l+1)is the security index at the node a(mi,l+1)

chosen by user a(mi,l), which is measured by the number ofbuilt-in security devices and features. An alternative measureis by security investment in dollars at the node. τ is the sojourntime experienced by node a(mi,l) in the queue of a(mi,l+1).pa(mi,l+1)

is the packet loss rate at a(mi,l+1).

III. FLOW CONTROL IN SMART GRID

In Section II, we have proposed a mechanism for meterdata to reach the SDCs and then the control station. Thedecision at the penultimate level is trivial as all the data shouldroute to the control station. In reality, a local control stationpulls data from its communicating SDCs. SDCs cannot senddata to the control station at an arbitrary rate. The bandwidthand communication resources of a control station are oftenconstrained. We next formulate a game-theoretical problemwhere SDCs choose data rates to communicate with the controlroom similar to the framework described [21].

Let Ns = NL−1 := n1, n2, · · · , nNs be the set of Ns

SDC substations. A SDC substation ni ∈ Ns serves a load ofLi(t) at time t and communicates with the control center CS .Let di(t) be the data rate at which SDC ni decides to send andthe service rate at the control room is sr(t). We assume that allthe data come into one server and the control center allocatesits resources to monitor the data from the substations. DifferentSDCs have different levels of significance wi ∈ (0, 1) to thecontrol center. A substation with higher values of wi are oftenthe ones that serve a large area. The control center processesthe data by differentiating their sources. The data from eachsubstation is served at a service rate si(t) = aisr(t), whereai ∈ [0, 1] denotes the fraction of service capacity dedicatedto CPS substation ni, satisfying

∑ni∈Ns

ai = 1. Let q(t) bethe queue length of the server and ui(t) = di(t)− si(t). Thequeue evolves according to the following dynamics

q(t) =∑

ni∈Ns

di(t)− si(t) =N∑i=1

ui. (4)

The objective of the control problem is to ensure queuelength to stay around some desired level Q. The choice of Qhas a direct impact on loss probabilities and throughput thecommunications between SDCs and the control room. Denoteby x(t) = q(t)− Q the shifted queue length with origin at Q.Then, the dynamics in (4) become

x =N∑i=1

ui(t). (5)

We consider two problems. The first one is a centralizedproblem where the control center makes data rate decision foreach SDCs to achieve the optimal “social welfare”. The secondone is a decentralized problem where each SDC decides onits own data rate sent to the control station.

In the centralized problem, the control room makes a globalplanning on the sending rates di(t) and hence ui(t). The goalof each SDC is to minimize its cost functional

Ji =

∫ ∞0

ci|x(t)|2 + ri|ui(t)|2dt, (6)

where ri is the cost on the control action ui. The higherthe sending rates will consume a high bandwidth at cost riinversely proportional to the load Li and wi, i.e., it is relativelyless expensive for more important SDC to send information.The cost ri can be seen as a variable controlled by the controlcenter that provides incentive for SDCs to respond in anefficient way. ci is the cost on the queue length such that∑

ni∈Nsci = 1. Each substation with higher data rate di(t)

or Li(t) also shares a higher responsibility of keeping thequeue away from overflow.

A centralized planning seeks uI = u1, u2, · · · , uNs ∈

UI that maximizes the social welfare J := 1N

∑Ni=1 Ji,

where U is a set of admissible control under informationstructure I . Denote by JI the optimal welfare achievedunder the optimal control u. On the other hand, the de-centralized problem constitutes a differential game Ξ :=〈Ns, Jini∈Ns

, Uini ∈ Ns〉, where Ns is the set of theplayers; UI

i is the set of admissible controls of a playerni, which depends on the information structure I of thegame; Ji is the cost functional that player ni attempts tominimize independently of the other players. We denote theset of equilibrium solution to Ξ by U∗I and the social welfareachieved under an equilibrium u∗I ∈ U∗I by J∗I (u∗I). Themeasure of efficiency loss can thus be described by the priceof anarchy

ρI := minu∗

I∈U∗I

J∗I (u∗I)

JI,

which is always upper bounded by 1.

IV. SECURE ROUTING ARCHITECTURE

A centralized routing architecture ensures the global effi-ciency and it is robust to small disturbances from SMs andindividual DCs or SDCs. However, it is costly to implement acentralized planning on a daily basis for a large-scale SG. Inaddition, global solutions can be less resilient to unexpectedfailures and attacks as it is less nimble for changes in routesand it takes time for the centralized planning to respond in atimely manner.

On the other hand, decentralized decision-making can bemore computationally friendly based on local informationand hence the response time to emergency is relative fast.The entire system becomes more resilient to local faults andfailures, thanks to the independence of the players and thereduced overhead on the response to unanticipated uncertain-ties. However, the decentralized solution can suffer from high



Fig. 3. An illustration of the architecture of the routing protocol in SGs.

loss of inefficiency. Hence, we need to assess the tradeoffbetween efficiency, reliability and resilience for designing thecommunication protocol between the control stations and theSDCs.

In this work, we propose a hybrid architecture of thecommunication infrastructure I as illustrated in Figure 3. Weadopt a centralized planning at the top levels L − 1 and Lwhile building a decentralized routing protocol between levels1 and L − 1. Such an architecture can enable the robustnessat the critical data centers and the resilience at the lower userlevel.

From the problem formulation in Section III, the final stageutility u(ai,L−1) of node (ai, L − 1) ∈ NL−1 can be writtenas

u(ai,L−1) = −J(ai,L−1)(uI). (7)

Hence, the last stage utility has been determined in a cen-tralized manner by the control room based on their priorityand load. The resource allocation at the last level is robust tosmall parametric disturbances and is independent of routingdecisions between level 1 and level L − 1. The routingdecisions of the meters are resilient to router failures in apublic network. The proposed learning algorithm can respondto a dysfunctional router by selecting a new router once theone in use is compromised.

A. Reliability Analysis

We can analyze the reliability of the hierarchical routingarchitecture in Figure 3. We assume that each device ni ∈ Nhas a failure rate of λi, which is highly dependent on thesecurity index ri. We let λi = gi(ri), where gi(·) : R→ [0, 1]is a monotonically decreasing and continuous function. Denoteby Γ the unreliability of the hierarchical routing architecture.It can be estimated by its unreliability lower bound ΓL, i.e.,

Γ > ΓL :=

L−1∑l=2

∏ni∈Nl

λit. (8)

Assuming that components at each level have the same failurerate λl for ni ∈ Nl, then we arrive at

ΓL 6 Γ 6 ΓU , (9)

where

ΓL :=

L−1∑l=2

(λlt)Nl , (10)

and

ΓU :=L−1∑l=2

Nl(λlt)Nl−1. (11)

V. CONCLUSION

Secure routing is a pivotal problem in smart grids asthe power infrastructure is migrating to new communicationtechnologies. In this paper, we propose a hybrid routingprotocol that enables the resilience, robustness and efficiencyof routing in smart grids. The smart meter data are routed tothe control room in a hierarchical manner, where decisions aremade in a decentralized manner between the first level and thepenultimate one. We show that the mixed Nash equilibrium canbe computed using backward induction and it can be achievedthrough learning algorithms such as fictitious play and bestresponse dynamics. The data rates between the penultimatelevel and the control center are determined in a centralizedway to achieve Pareto efficiency as well as the robustness ofsmart grids. Future work will be aimed at the implementationof the routing algorithm on a small testbed as well as the studyof a more refined and more comprehensive security metric thatincorporates potential threats and attacks.

REFERENCES

[1] Pike Research’s report, “Smart Grid Networking and Communications”,2010.

[2] K. Tomsovic, D. Bakken, V. Venkatasubramanian and A. Bose, “Design-ing the Next Generation of Real-Time Control, Communication, andComputations for Large Power Systems” IEEE Proceedings In IEEEProceedings, Vol. 93, No. 5, May 2005, pp. 965-979.

[3] G. N. Ericsson, “On requirements specifications for a power systemcommunications system”, IEEE Transactions on Power Delivery In IEEETransactions on Power Delivery, Vol. 20, No. 2, Apr. 2005, pp. 1357-1362.

[4] M. S. Amin and B. F. Wollenberg, “Toward a smart grid: power deliveryfor the 21st century”, IEEE Power and Energy Magazine, Vol. 3, No.5, Oct. 2005, pp. 34-41.

[5] E. Santacana, G. Rackliffe, L. Tang and X. Feng, “Getting Smart”, IEEEPower and Energy Magazine, Vol. 8, No. 2, Mar. 2010, pp. 41-48.

[6] “Grid 2030 – A National Vision for Electricity’s Second 100 Years”,Technical Report, Department of Energy, July 2003.

[7] D. H. Lorenz and A. Orda, “QoS routing in networks with uncertainparameters”, IEEE/ACM Transactions on Networking, Vol. 6, No. 6,1998, pp. 768 - 778.

[8] P. Van Mieghem and F. A. Kuipers, “Concepts of exact QoS routingalgorithms”, IEEE/ACM Transactions on Networking, Vol.12 , No. 5,2004 , pp. 851 - 864.

[9] G. Xue and S. K. Makki, “Multi-constrained QoS Routing: A NormApproach”, IEEE Transactions on Computers, Vol. 56 , No. 6, 2007 ,pp. 859 - 863.

[10] F. Kuipers, P. Van Mieghem, T. Korkmaz and M. Krunz, “An overviewof constraint-based path selection algorithms for QoS routing”, IEEECommunications Magazine, Vol. 40 , No.12, pp. 50 - 55.

[11] Department of Energy, “Communications Requirement of Smart GridTechnologies”, Oct. 5, 2010.



[12] G. N. Ericsson, ”On requirements specifications for a power systemcommunications system”, IEEE Transactions on Power Delivery In IEEETransactions on Power Delivery, Vol. 20, No. 2., Apr. 2005, pp. 1357-1362.

[13] P. McDaniel and S. McLaughlin, “Security and Privacy Challenges inthe Smart Grid”, IEEE Security & Privacy, Vol. 7, No. 3, June 2009,pp. 75-77.

[14] “NIST Framework and Roadmap for Smart Grid Interoperability Stan-dards”, Release 1.0, January 2010, by National Institute of Standardsand Technology.

[15] C. H. Hauser, D. E. Bakken and A. Bose, “A failure to communicate:next generation communication requirements, technologies, and archi-tecture for the electric power grid” IEEE Power and Energy Magazine,Vol. 3, No. 214, Mar. 2005, pp. 47-55.

[16] “Primer Control Systems Cyber Security Framework and Technical Met-rics”, June 2009, Technical Report, Department of Homeland Security.

[17] W. Boyer and M. McQueen, “Ideal Based Cyber Security TechnicalMetrics for Control Systems”, 2nd International Workshop on CriticalInformation Infrastructures Security, October 2007.

[18] A. McIntyre, B. Becker and R Halbgewachs, “Security Metrics for Pro-cess Control Systems”, SANDIA REPORT, SAND2007-2070P, Septem-ber 2007.

[19] D. Wei and K. Ji, “Resilient industrial control system (RICS): Concepts,formulation, metrics, and insights”, 3rd International Symposium onResilient Control Systems (ISRCS), 2010.

[20] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory,SIAM Series in Classics in Applied Mathematics, Philadelphia, January1999.

[21] E. Altman and T. Basar. “Multi-user rate-based flow control, IEEETransactions on Communications”, Vol. 46, No. 7, pp. 940-949, July1999.

[22] D. Fudenberg and D. K. Levine, The Theory of Learning in Games,MIT Press, May 1998.



Documents

Workshop on Foundations of Dependable and Secure Cyber-Physical Systems … · The Workshop on the Foundations of Dependable and Secure Cyber‐Physical Systems (FDSCPS) will focus