Applied Computational Intelligence Lab & Trustworthy Systems Lab

University of Missouri - Rolla

Systems of Systems: Cybersecurity Vulnerabilities and Opportunities

Applied Mathematics for Deregulated Electric Power Systems:

Optimization, Control, and Computational IntelligenceCrystal City, November 2003

Donald Wunsch, ACIL DirectorAnn Miller, TSL Director

Acknowledgements

Funding– NSF– Sandia– Boeing– MK Finley Professorship– Cindy Tang Professorship

Senior Personnel– Ganesh Kumar

Venayagamoorthy– Ron Harley– Daryl Beetner– Danil Prokhorov– Raonak Uz-Zaman– Frank Harary

Personnel – Narayan Vishwanathan– Amit Agarwahl– Sam Mulder– Wenxin Liu– Nian Zhang– Alexander Novokhodko– Xindi Cai– Rohit Dua– Hu Xiao– Rui Xu– Brian Blaha– Paul Pigg– Arvind Rapka Nath– Qiang Yao– Kevin Bollum– Anjaya Shrestra– Karthik Balasubramanian– Pinar Demircan– Daniel Treat– Ian Downard– Eyad Salah Tagiedin– Ganesh Sridharan– Jason White– Krishnaprasad Balasubramanian– Dayle Majors– Nartaj Lakshminarasimhan– Siddarth Panchal– Robert Wayne Denier– Tongquan Wei– Jimish Doshi– Ravikiran Sharda

“system of systems”– Grown/evolved by adding components not

initially designed to be part of the system– Interdependencies not easily identified

Potential for cascading failures Potential for hidden robustness

Systems of Systems: Interdependencies

Trustworthiness Testing Market Demands Complexity Safety Life-Cycle Model Integration

Issues in Systems ofSystems

19841984 8686 9090 9292 9494 9696 199819988888

11

1010

100100

Re

co

mm

end

ed

dis

k s

pa

ce,

MB

Math package 1Math package 2Math package 3

Moore

’s L

aw

Source: IEEE Spectrum, January 1998

Complexity: Software Size Growth

Complexity: Software Size Growth

Complexity: Interdependencies

A graph representing almost 6 million lines of computer code. The graph contains approximately 33 thousand nodes and 34 thousand relations.

Source: NATO Report on Visualization, 1999.

Memory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environment

Normalised Failure Rate, %0 5 10 15 20 25 30

LINUX

NT

Win-2000

Win-CE

Failure Rates – System Calls

(Source: Carnegie Mellon, CS Dept.)

Cascading failures Opportunities for errors Control, Communication, IT

– Pres. Commission on Critical Infrastructure Protection

– Particularly EMS & SCADA

Voltage Collapse

Effects of Complexity and Growth

High-Consequence

Even brief – expensive– Circuit fab: 20 min = ($30 M)

Recent large disruption caused deaths

Backups no guarantee– Well-known in software safety circles

Therac 25 classic example

At 0903 CST on 18 December 1997, at the Olathe (Kansas City) Air Route Traffic Control Center, a technician routed power through half of the redundant uninterruptible power system, preparatory to performing the annual preventive maintenance on the other half. Apparently the wrong board was pulled.

Complexity: Ripple Effect Example

Results: – Power only out for 4 minutes– Radar and communications working within 17

minutes– However, at least 300 planes were in the Olathe-

controlled airspace; domino effect: hundreds of flights canceled, diverted, or delayed with problems well into the evening.

Complexity: Ripple Effect Example

Not only did the Air Route Traffic Control Center have redundant systems, there were also standby generators and emergency batteries.

Yet, that December morning, these back-up systems were bypassed.

Why?

Back-up Systems Are Not a Guarantee

The back-up systems were bypassed because the system was in a maintenance state.

This particular combination of inputs was not anticipated to occur when the system was in maintenance mode.

Complex Interactions: States and Inputs

Tempting Target

Dramatic growth in number of knowledgeable experts

Potential to insert incorrect data or Denial of Service attacks

High leverage / low risk

Computational Intelligence Tools Can Help

Neural Net Intrusion Detection ADP Robust Controls Combinatorial Optimization for

reconfigurability

Intrusion Detection with Neural Nets

RBFNs can be used for misuse and anomaly detection using sequences of system calls

Data are obtained from 1998 DARPA Intrusion Detection Evaluation program

Also collaboration with Sandia Red Team

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False positives

Tru

e p

osi

tive

s

accuracy=0.74

RBFNN Generalization on unknown test data

PNN + ADABOOST

Multi-Machine Power System Control

G1 G2

G3

1 2

3

4 5

900 KmExciter

AVR Vref2

Exciter

AVR

Vref1

Ve1 Ve2Vt1 Vt2

900 Km

900 Km

Pref1

1Governor Governor

Turbine Turbine

2

Pref2

Multi-Machine Power System with Conventional Controllers

Multi-Machine Power System with DHP Neurocontrollers

G1 G2

G3

1 2

3

4 5

900 KmExciter

Vref2

Exciter

Vref1

Vf1 Vf2Vt1

Vt2

900 Km

900 Km

Pref1

1

Turbine Turbine

2

Pref2

Neurocontoller

TDL TDL

P1

Neurocontoller

TDL TDL

P2

Vref1 Vref2

GovernorGovernor

DHP Critic Network Adaptation

PLANTTDL

ACTIONNeural

NetworkMODELNeural

Network

CRITICNeural

Network

CRITICNeural

Network

TDL

MODELNeural

Network

YrefY(t)

A(t))(

)(

tA

tU

(t+1)

)(^

1tY

)(^

tY

)(^

1tY

TDL

TDL

)(^

tY

)(^

1tY TDL

TDL )(^

2tY

++

)(

)(

)(

^1tY

1tJ

1t

)(

)(

)(

^1tY

1tJ

1t

)(

)(

tY

tU

+---

EC2(t)

Terminal Voltage of Generator G2 for a 5% Step Change in its Desired Terminal

Voltage & Operating Point Changed

1 2 3 4 5 6 7 80.98

0.99

1

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

Time in seconds

Ter

min

al v

olta

ge in

pu

DHP

AVR

Speed Deviation of Generator G2 - Operating Point Changed

0 1 2 3 4 5 6 7 8-1.5

-1

-0.5

0

0.5

1

1.5x 10

-3

Time in seconds

Sp

ee

d d

evi

atio

n o

f G

1 in

pu

Conventional

Neurocontroller

Traveling Salesman Problem

Great benchmark NP – complete

– Maps to other NP – complete problems Public databases

Big need – get learning capability of NN without brittleness of other techniques.

Paper Method Largest

Instance

Quality

(percent

excess over

optimal )

Test bed

[11] 1st 100 14.6% NS

[13] 1st 100 14% NS

[10] 1st 400 NR NS

[5] 2nd 532 6.8% TSPLIB

[12] 1st 1000 NR NS

[16] 2nd 1000 NR NS

[15] 1st 2392 5% TSPLIB

[17] 2nd 2392 9% TSPLIB

[2] 1st 10000 NR NS

[4] 1st 11849 17.4% TSPLIB

Previous contributions -- disappointing

Clustered Traveling Salesman

Divide problem into clusters using ART in O(n)

Use Lin-Kernighan algorithm for global tour

Use Lin-Kernighan algorithm for local tours

Merge local tours in O(n) time Global operations limited to O(n) time

Algorithm Overview

Read problem from file O(n)

ART O(n lg n)

cluster

cluster

cluster

LK O(k2.2)

LK O(k2.2)

LK O(k2.2)

Merge Clusters O(n)

Result

Implemented in C++ thread-safe code Uses Windows threads for parallelism Operating System-specific code isolated to one

file Should be easy to port to other parallel

systems

Implementation

#cities Tour Length 1P Time 2P Time Vig factor % off Speedup 1000 2.58E+07 0.422 0.281 0.7 10.40% 1.50

2000 3.61E+07 1.031 0.672 0.7 10.64% 1.53

8000 7.14E+07 8.328 4.281 0.72 10.97% 1.95

10000 7.97E+07 11.359 7.297 0.75 10.57% 1.56

20000 1.12E+08 24.641 14.406 0.8 10.53% 1.71

250000 4.00E+08 315.078 209.687 0.92 11.64% 1.50

1000000 7.94E+08 1468.165 986.48 0.97 11.03% 1.49

10000000 2.52E+09 10528.7 0.98 1.27%

CONCORDE

1000 2.34E+07 1.670

2000 3.26E+07 3.500

8000 6.43E+07 26.570

10000 7.20E+07 37.620

20000 1.01E+08 84.830

250000 3.58E+08 1379.540

1000000 7.15E+08 9013.53 10000000 2.495E+09 43630.7

1k

4k

8k

10k

20k

50k

85k

150k

250k

1 M

Even better news…

Continued Scaling Results Parallelizability Memory Management

BUT – To Move Beyond

Clear Need for more advanced architectures– Especially to Learn from

Experience Cellular Structures necessary Same with SRNs Therefore, combine them and

ACDs

Recurrent Nets

Obviously achieve dynamic behaviors Possible similarity to adaptive systems

but with fixed parameters Simultaneous recurrent nets particularly

challenging, esp. architectures

            

            

 

•Graph Theoretic Representation

•SRN Necessary (Werbos & Pang, ’96 & ’98)

•Cellular structure – scaling

•Closed form now

•Convergence time now

•Importance of design principles

Generalized Maze Problem

Require for the output node: x16 = (x2 / x1)[min{x6, x5, x4, x3} + 1].

This is a known SRN!

Design from output backward

Output J = (x2/x1) * sum = x16(a,b)

CurrentNode inputs Neighbor

node inputs

Feedback inputs

             

 

(Occurs at each node (a,b) in maze.)

Product Nodes

+1

/ *

Cellular SRN Structure Complete

Analyze worst-case convergence

       

 

WCT = N2 - 2N + N - 3 = N2 - N - 3.

Also true for N x N maze by simple induction proof.

Note that this is convergence in J steps.

Conclusions

Power networks inherit the full range of “systems of systems” issues.

These are amenable to computational intelligence solutions:– Detection– Robust Control– Reconfigurability

Combinatorial Optimization