A Secure Decentralized Data Centric

INTRODUCTIONThe power industry is at the dawn of a new eraof transformation, triggered by the increase inworldwide energy consumption and the subse-quent increase in pollution. Recent months havewitnessed an increasing focus on the next gener-ation power grid from both industry andresearch. The goals of the next generation powergrid is to address pressing problems such asenabling clean energy, energy storage, andresilience of the grid in the event of power flowdisruptions.

ROLE OF SMART GRID IN THE GREEN ECONOMYTo address these challenges, innovations areoccurring at all phases of the power supply-cycle:generation, transmission, distribution, and con-trol. However, one of the greatest impedimentsto the deployment of these innovations is theaging information infrastructure of the powergrid. The primary purpose of any informationgrid that operates and manages a power systemis to ensure that generation, transmission, anddistribution of power are reliably and securelyprovided, while the installed infrastructure canbe effectively utilized up to threshold limits.

Current infrastructure often fails these require-ments, as witnessed by the increased occurrenceof blackouts [1], costly over provisioning for reli-ability, and difficulty to deploy new services tobalance various power demands from the mar-ket. This situation is further exacerbated by mar-ket conditions that are expected to break manyof the traditional assumptions about the powergrid, such as: Power flows are unidirectional and synchro-

nized from power plants to consumers A wide-area authority can control the entire

power gridTodays centralized information infrastructure

is not resistant (to faults or cyber-attacks), exten-sible or scalable to accommodate the emergingpower grid requirements. Therefore, the powerindustry is in the process of a defining a newinformation infrastructure Smart Grid toaddress the challenges posed by the future cleanand flexible power grid.

NEW NETWORK REQUIREMENTSIntegration of renewable energy sources on tothe main power grid as well as active user partic-ipation in making power demand decisions canbring high variability into the grid by creating amismatch between the demand and supply ofpower. It has been shown that there may bealmost 81 percent drop in output power over thespan of just five minutes in a solar photo voltaicsystem, due to the high variability in the cloudpatterns [2]. Therefore, fast reaction times areessential to ensure the reliable operation of thegrid. Furthermore, in addition to the variabilityof the power sources, the changing physical envi-ronment and the unintended consequences ofdisturbances require that the power entities aremonitored at very fine time scales (60720 sam-ples per second) from thousands of sensors thatare part of the grid. This fine-grain monitoringgenerates a large volume of measurement data,which must be efficiently managed by the nextgeneration grid. The combination of fast reac-tion times and distributed data sources pose ascalability challenge for the conventional gridthat uses a centralized information infra-structure.

IEEE Communications Magazine November 201058 0163-6804/10/$25.00 2010 IEEE

ABSTRACTIn recent years the power grid has been under-

going transformative changes due to the greaterpenetration of renewable energy sources andincreased focus on power demand shaping. Theseinnovative transformations on the grid require aflexible IP-based communication grid that is reli-able and secure. In this article we describe an IP-based decentralized and data-centric informationinfrastructure that can reliably, securely, and cost-effectively support the operation and innovativeapplications of the next generation grid. The pro-posed infrastructure differs from a typical dis-tributed system since it addresses the specificrequirements of power applications such as secu-rity, distributed data sources, latency sensitivedata transactions and real time event updates.The work presented here paves the way for afuture data-centric power network infrastructure.

ENERGY EFFICIENCY IN COMMUNICATIONS

Young-Jin Kim, Marina Thottan, Vladimir Kolesnikov, and Wonsuck Lee, Alcatel-Lucent

A Secure Decentralized Data-Centric Information Infrastructure for Smart Grid

KIM LAYOUT 10/20/10 3:53 PM Page 58

IEEE Communications Magazine November 2010 59

OUR CONTRIBUTION

To address the network challenges describedabove we propose a secure decentralized data-centric information infrastructure (middleware)that specifically addresses the requirements ofthe power grid. The decentralization helps elimi-nate single bottleneck failures, while ensuringscalability. Furthermore, as shown in the sectiondescribing our approach, the architecture can beeasily extended to support reliability require-ments such as self-healing the ability toautonomously recover from any disturbances,and self-configurability the ability toautonomously use or recommend contingencyoptions in the event of disturbances to the grid.

Our main contributions in this work are: An outline for the decentralized data-cen-

tric information grid along with illustrationof the applicability of the grid for powersystem control applications

An outline of an overlay control plane ser-vice

A survey of commonly used data transportprotocols based on reliability and latencyrequirements for power applications

A security architecture for the proposedinformation infrastructure

TERMINOLOGY AND DEFINITIONSBefore an in depth discussion of our decentral-ized data-centric information infrastructure, wediscuss some preliminaries. A power grid com-prises power infrastructure and informationinfrastructure. The power infrastructure is aninterconnected network of power equipmentthat delivers electricity from the power plants toconsumers such as residential areas, businesssector, or public sector. It consists of variouspower devices such as generators, towers, lines,transformers, circuit breakers, voltage regulators,feeders, capacity banks, meters, and so on. (Fig.

1). The information infrastructure is a communi-cation network that controls the power infra-structure and enables the reliable and safeoperation of the power grid. It is responsible tomeasure the status of devices in a grid and usesmeasured information to compute local or wide-area state updates (residential, micro grid, sub-stations, or entire grid). These status updates areused by the power grid to control servers or helphuman operators to take appropriate actions toprotect disturbance propagation, implement con-tingency plans, and provide information for thirdparties, such as electricity wholesale markets.The information infrastructure has two types ofelements: control entities, and sensing entities,as shown in Fig. 1. A sensing entity is an entitythat has a measuring unit, but holds no actuatingfunctionality. Control entities, on the otherhand, have an actuating entity. Both types ofentities are assumed to have communicationcapabilities.

DECENTRALIZED DATA-CENTRICINFORMATION INFRASTRUCTURE

The important differentiator for the next genera-tion power grid is the massive amounts of mea-surement data that will be made available atdistributed locations that can and must be lever-aged to optimally operate the power grid. Toaddress this challenge, our decentralized data-centric information infrastructure enables thescalable handling of large quantities of data,while still enabling the necessary two-way com-munication of control and management informa-tion. Our information infrastructure supportsboth data transport and control traffic with thenecessary reliability and latency requirements, toensure that data accuracy is maintained and con-trol operations are implemented in a timely fash-ion.

Figure 1. P2P associated information infrastructure for power generation, transmission, distribution, andconsumption.

Transformer

Voltage regulator

Circuit breaker

Substation

EVs Solar farm Wind turbines

Electricity flow Power infrastructure

Information infrastructure

Nuclear plant

Np

:Nsp

Yp

Is

Ne

Ys

Ip

Ns

___=

___=

___

Ip Is

Ep

Es Vs

PS: Protection scheme

Network for smart metering

IP-enabled network

PS Control entity - wide area

(regional control)

PS Control entity -

local

Information/control flow

PS Control entity -

local Sensing entity -

(towers, feeders)

Control entity - wide area

Control entity in other utility

Vp

A sensing entity is an

entity that has a

measuring unit, but

holds no actuating

functionality. Control

entities, on the other

hand, have an

actuating entity.

Both types of entities

are assumed to have

communication

capabilities.


IEEE Communications Magazine November 201060

THE NEED FOR A DATA-CENTRIC ANDDECENTRALIZED INFRASTRUCTURE

Todays power grid is deployed with a largelycentralized information infrastructure, with theEnergy Management System (EMS) acting as themain control center. All control entities within autility are directly connected to the EMS; thecontrol entities have no mechanism to communi-cate with each other; i.e., the present communi-cation graph is a star centered at the EMS. TheEMS queries in a round-robin manner the differ-ent substations for reporting the current status ofthe power devices. The Remote Terminal Unit(RTU) in each substation interfaces with theEMS, collects status information of the powerdevices, and reports it to the EMS. The stateestimator at the EMS estimates the entire gridstate using all of the information acquired fromthe power grid. Most applications in the EMS,including contingency analysis, depend on theestimated state of the grid. In short, wide-areamonitoring and control is based on SupervisoryControl and Data Acquisition, and Energy Man-agement System (SCADA/EMS), and has beenbuilt in a centralized manner. However, the wide-area control via SCADA is unable to rapidly andprecisely respond to emergency situations such asdisturbances, since the state estimation is an off-line computation and its precision relies on pre-defined models. Thus, special protection schemeshave been installed in a hard-wired and localizedmanner as complimentary to the EMS for preciseresponse to pre-defined critical events (detectingdisturbances, protecting propagation, and restor-ing power grid) within certain time constraints.

Table 1 summarizes the data/events in thepresent power grid and this information can beclassified into four types: Data measured by Intelligent Electronic

Devices (IEDs), such as circuit breakersand digital fault recorders, PMUs (PhasorMeasurement Units) or sensors attached topower devices

Data computed by entities using measured data

Time-sensitive events for protectionschemes

Metering data read by human collectors (itis not used in wide-area control)Under the current centralized SCADA/EMS

structure, locally measured data, used in a sub-station, can neither be sent nor stored, as com-munication bandwidth is limited, and a centralrepository does not have enough space to storeall the data. Instead, a subset of data summariesis computed by each local entity and this sum-mary sketch is sent to the EMS via serial inter-faces or microwave links, and stored at a centralrepository. Either IEEE 60870-5 or DNP3 (Dis-tributed Network Protocol Version 3.3) proto-cols, specifically designed to reliably transmitrelatively small messages for SCADA applica-tions, are used for data transfer. All wide-areacontrol and monitoring rely on the summarydata acquired from local entities; therefore theirflexibility to adapt to new control applications isseverely restricted.

In the next generation power grid, accordingto J. Giri et al. [3] a PMU in a power grid gener-ates either 50 or 60 phasor measurements persecond. Even a small number of PMUs can resultin bandwidth bottlenecks in a super-PDC orPDCs, since just a single PMU can generate alarge amount of data in the course of several ofminutes. Under the centralized structure, trans-mitting all phasor data generated from PMUs tothe EMS will inevitably result in a collapse ofthe information infrastructure because of thesingle point of failure or bottlenecks in the EMS.One approach to alleviate this problem is toover provision the communication network forbandwidth. However, a peak-rate model forbandwidth provisioning is not necessarily a goodoption due to cost efficiency requirements.Another option is to implement control applica-tions locally such as in the approach taken bythe Special Protection Scheme (SPS). However,the implementation of SPS is limited, since therequired control actions must be pre-configured,which leaves the system unable to react to all sit-

Table 1. Data in todays information infrastructure from the viewpoint of wide-area control and monitoring.

Measured dataComputed data Protection events Metered datafrom consumers

IED PMU

Sample rate In circuit breaker,beyond 1 KHzIn one device,50~60 Hz

In one substation,0.1~0.5 Hz Close to zero

Once per month(off-line reading)

Time-critical Medium Medium Low High None

Information consumer Substation PDC, Super-PDC Substation, EMS SPS Billing center

Notification frequency Periodic Periodic Polling-based Event-based Periodic

Storage location None PDC EMS EMS Billing center

Communication

Medium None Serial, Ethernet Serial, Microwave Serial, Microwave None

Protocol None IEEE C37.118 IEC 60870-5, DNP3 IEC 60870-5, DNP3 None

Topology None Two-level Tree Star Star None



uations that may arise in the new power grid.For example, unknown and undocumentedpower disturbances that develop from anoma-lous events cannot be fully recovered by usingexisting SPS systems. SPS systems are typicallycapable of handling disaster conditions that arewell known. On the other hand, with an informa-tion infrastructure which supports fine-grainmetering data collection, real-time pricing, andconsumer participation in voluntary load shed-ding, a relatively small number of peak-load gen-erators can balance power demand and supplyso that carbon emission and operating costs areboth significantly reduced [4].

OUR SOLUTION: A DECENTRALIZEDDATA CENTRIC INFORMATION

INFRASTRUCTURE

Given the challenges presented above, it is notsurprising that the information infrastructure forthe next generation grid must be optimized fordata handling. With regard to data handling, asreviewed in the last section, the availability,timeliness and location of the data are critical.According to the IEEE 1646 standard, the timescale of the wide-area control can range fromminutes to hours; thus data delivered and storedmay not be immediately used by the EMS or thesuper-PDC. On the other hand, data corre-sponding to notification of protection schemeswill be exploited immediately and is required tohave an end-to-end latency of 812 ms. There-fore from the data-centric perspective, mostpower grid applications can be characterized asoperations where a subset of data (spatial locali-ty) in either EMS or super-PDC is consumedwithin a certain specified time (temporal locali-ty). This spatial and temporal locality regardingdata usage on the next generation grid can beleveraged as an opportunity to optimize theusage of communication bandwidth, storagespace, and CPU cycles, while significantlyimproving the reliability of the power grid. Thisobservation is in fact the intuition behind ourdecentralized data-centric information infra-

structure. The basic goal of our informationinfrastructure is to deliver the right amount ofmeasured or computed data at the right time ina cost-efficient, secure, and reliable manner.

INFORMATION INFRASTRUCTUREMIDDLEWARE

Our decentralized data-centric information infra-structure (middleware) comprises of five majorcomponents shown in Fig. 2: publisher-subscriberdata delivery, networked cache/storage, reliable,a secure grid overlay network comprising gridhub nodes, low-latency transport protocols, andthe Application Programming Interface (API).We will now describe each of these main compo-nents in detail. In a subsequent section we out-line some implementation-specific issues.

PUBLISHER-SUBSCRIBER DATA DELIVERYIn our information infrastructure a publisher-sub-scriber system is used to deliver time-sensitivedata to appropriate entities immediately as andwhen it is created. The pub-sub system replacesthe master-slave communication model thatexists today between the EMS and substations,thus enabling efficient scaling to support peer-to-peer associations as shown in Fig. 1. The basicoperation of the publisher-subscriber model isdescribed in [5]. Publishers announce the avail-ability of certain types of data, and subscribersannounce their interest in certain types of data.The matching of the publisher and the sub-scriber for delivering data of particular interestis made within the information middleware andwill be discussed in the grid overlay section.

As compared with the master-slave communi-cation, the following properties hold in generalfor the publisher-subscriber model: It enables the decoupling of information in

terms of space, time, and synchronization. It is by nature distributed, peer-to-peer, and

enables multicasting. It is highly scalable. Improved security due to the decoupling,

which in turn effectively prevents Distribut-ed Denial of Service (DDoS) attacks.

Figure 2. Data-centric middleware architecture.

Secured grid overlay network (unicast, multicast, and broadcast)

Network cache/storage(pull-based data access)

Pub/sub dissemination(push-based data access)

Middleware API

Power applications

IP networks

Non-time criticaldata/event

Time criticaldata/event

Reliable, low-latency, and lightweight transport

Controlcommands

TCP/UDP/SCTP

Our decentralized

data-centric informa-

tion infrastructure

(middleware) com-

prises of five major

components: pub-

lisher-subscriber data

delivery, networked

cache/storage, reli-

able, a secure grid

overlay network

comprising grid hub

nodes, low-latency

transport protocols,

and the Application

Programming

Interface


There is no single point of failure or bottle-neck.However, some specific issues must be taken

into account when applying the pub-sub modelto the power grid. For example, in the powergrid we have both LAN and WAN environmentsand therefore the matching of publishers to sub-scribers must handle the additional delaysincurred over the WAN.

NETWORKED CACHE/STORAGEIn the smart grid of the future it is expected thatmeasurement data with critical latency require-ments will dominate the traffic. This highlightsthe need for storage efficiency. Todays central-ized storage system is inefficient primarily due tothe lack of scalability in the face of the deluge ofdata that needs to be stored. For example, if thenumber of PMUs in power grid exceeds 1000,PMU data is written to the central disk at therate of at least 100 Mb/s, since a PMU can gen-erate a few hundred bytes of packets at 50~60Hz. In the current power grid, the amount andtypes of data stored in the infrastructure is limit-ed by the disk capacity on the central repository.This storage limitation is due to the fact that thecentralized system used today requires all thenecessary data be stored in a central repositoryclose to the EMS. Faced by this limitation, only asubset of data generated from the power grid isstored. Even if the disk capacity of this central-ized system was increased to accommodate thestorage of all generated data, its performance,cost-effectiveness, and the single point of failureissues are unavoidable in this centralized system.

Our information middleware provides a scal-able storage system that addresses the vulnerabil-ities of the centralized system by building virtuallydistributed storage systems made up of manyphysical disks, whose individual capacities aresmaller than that of a centralized disk. This stor-age system is cost-efficient and, when deployedwithin the information infrastructure, can allevi-ate the problems of the single bottleneck. Manyentities in the grid, such as substations and evensensors in transmission towers or line feeders,have or can have their own storage devices, suchas flash memories or hard disks. We note that allstorage-equipped entities in the grid constitute asingle distributed storage network, with different

lifetimes and security requirements for the storeddata. For example, for historical data we can use,todays distributed data storage mechanism,which employs high-volume disks located withina secured perimeter. For more time critical datawe use a distributed network storage system. Atthe core of the distributed storage is a hash func-tion shared by all entities [6]. Given a certaindata type, a uniform hash function determines acertain entitys identifier within the informationinfrastructure as illustrated in Fig. 3. Using thehash function, any power application can accessthe necessary data from the distributed storageunits. Since the data is accessed just in time, wecan avoid unnecessary high bandwidth transac-tions between the different entities on the grid.

RELIABLE, LOW-LATENCYCOMMUNICATION PROTOCOLS

Compared with the existing communicationinfrastructure (dedicated channel, point-to-pointconnection, and star topology) of the presentpower grid, the gains obtained by using an IP-based communication network is expected to sig-nificantly reduce cost, configuration complexity,and maintenance cost for the peer-to-peer asso-ciations that are expected in the future grid.However, the move to an IP-based network isnot without concern. The reliability and deliverylatency in an IP network is a significant issue.Reliable delivery of information has to be con-sidered a basic requirement especially for theimplementation of wide-area control applica-tions where even a small data loss can have direconsequences to the power grid. These issues donot arise in an environment of point-to-pointconnections relying on dedicated channels,which is the case in some substation-level appli-cations such as Automated Protection Systems.

The requirement of reliable data delivery withlow latency is a primary concern whether the IP-enabled network is the public Internet or a sepa-rate private network for the grid. The Tablebelow categorizes the different transport proto-cols based on latency sensitivity and reliability.

For data-centric systems, data transactionsrequire varying levels of reliability and latency.The middleware uses the conventional TCP pro-tocol (or SCTP [7]), since it incorporates reliabledata transfer mechanisms. Application layer pro-tocols over UDP, such as Reliable DatagramSocket (RDS), designed to efficiently and reli-ably deliver data in large data-centers, are alsogood options due to their lightweight propertiesin terms of computation and communication.However, the best solution for data-centric sys-tems remains an open problem. Real-time Trans-port Protocol (RTP) over UDP is a well-knownprotocol to address latency constraints over anIP network. However, this protocol is not suit-able for the power grid due to the inheritedunreliability from UDP. Even though RTP canalso be defined over TCP, to achieve reliability,it comes at the cost of increasing latency. Thereis no well-known protocol that meets reliabilityand low-latency requirements; additionally, mostwell-known protocols are not light-weight. Con-sider the case of asynchronous events, which arerare that need to be delivered or stored in data-


Figure 3. Data storing and retrieval using hash function.

Nodes having their own storage, IP-addressable, computing power

write(data_type,data)

Hash(data_type) -> D

Producer

Information network

read data request

Consumer

write data

D

read(data_type)

Hash(data_type) -> D



centric systems. How can we minimize the totallatency from connection setup to complete deliv-ery of the event data, while minimizing the useof network resources? A naive approach is toproactively establish a full-mesh of connectionsamong all entities. This clearly raises scalabilityconcerns. In our information infrastructure weachieve reliability, low latency, and lightweightrequirements by using the grid overlay network.

SECURED GRID OVERLAY NETWORKThe control of the middleware is implementedin an overlay network of trusted grid hub nodes(Fig. 4), which are deployed in physically securelocations and run hardened code. The grid over-lay is akin to a structured P2P network. Theoverlay substrate is used to determine the laten-cy optimal path. It also ensures reliable andsecure data delivery.

Among the control services, hub nodes pro-vide multicast and access control. As such, theycreate and maintain the multicast trees (span-ning over the potentially insecure IP cloud whichcarries the bulk of the data). Further, thesenodes facilitate the security of multicast (andother channels) by authenticating and authoriz-ing joins, and provisioning users with group keysthey are authorized to have. This, in particular,allows fine-grain and scalable control over theencryption of topics and ensures that the topicsinjected by publishers are securely and efficientlydelivered only to the right group of subscribers.Using encryption allows multicast optimizations:trees can be now safely merged, since subscriberswould not possess the keys for the content theyare not authorized to view. No traffic is sentunencrypted and, for modularity, traffic wouldoften be encrypted twice, once with the channelkey, and once with the multicast group key.)

MIDDLEWAREIMPLEMENTATION ISSUES

The implementation of the proposed data-cen-tric middleware involves some key advances indifferent basic functions implemented on the IPnetwork.

NAMING, ROUTING, AND FORWARDINGNaming and routing (i.e., path finding) are basicfunctions of any network on which the middle-ware is built. In the case of an IP network eachcontrol entity either keeps all IP addressesassigned to all entities within the informationinfrastructure or uses a DNS to resolve a logicalname into an IP address. This naming approachclearly faces scalability issues and it moves awayfrom our decentralized design concept. More-over, in terms of enabling self-configuration, it ishard to seamlessly respond to topology updates,such as the addition of new Network InterfaceCards (NICs) to existing IP-enabled entities orwhen the NIC configurations are changed. Incontrast, the use of an overlay network, as shownin [8], addresses the scalability and self-configu-ration issues due to its decentralized nature.Further, the overlay approach does not requireany significant modifications to current routersor switches.

Due to performance concerns in data for-warding, in the network assisted overlay-basedapproach [9], overlay nodes can exploit the sta-tus information of network paths and identifycandidate peers as advised by their accessrouters. With this approach, the latency indexcan be significantly improved in the overlay net-work.

STANDARDIZED DATA FORMATIn a decentralized information infrastructure,data formats need to be standardized. Com-mon Information Model (CIM) for EMS appli-cation program interface has recently beenstandardized to seamlessly share data betweenEMS and Distributed Management System(DMS) servers via a service bus. Automationof substations, as well as the information mod-els for substations, have been standardized inIEC 61850 specifications. More recently, underthe leadership of Electric Power ResearchInstitute (EPRI), there have been some discus-sions to combine CIM with the IEC 61850information model. Hence, our middlewareuses the combined standard for data exchangeamong the different entities in the informationinfrastructure.

Figure 4. Secure grid overlay network.

MicroGrid gateway

IP enabled network

Smart meter Smart

meter Frequency

sensor

Voltage sensor

PMU aggregator

Smart meter

Wide-area control

Market price estimator Utility

center

IP interface Middleware

Group#c Securely managed by

Group#b Trusted network

Secure control server

Group#a

Naming and routing

are basic functions

of any network on

which the middle-

ware is built. In the

case of an IP net-

work each control

entity either keeps all

IP addresses assigned

to all entities within

the information

infrastructure or uses

a DNS to resolve a

logical name into an

IP address.



SECURITY ASPECTSIn this section we overview our goals with respectto security, sketch design options, attract atten-tion to several prominent issues and trade-offs,and suggest solutions and directions. In particu-lar, we show that natural security architecturessupport our middleware design features, such asmulticast. For lack of space, we limit the amountof detail in this exposition.

Our goal is to ensure protection and reliabili-ty of data and communications. That is, datamust be available with high probability, both forstorage and transmission, even in the presenceof failures, and adversary must not be able toread the data or interfere with it. We start withdiscussing data protection.

Clearly, since data is traveling through publicnetworks, it must be secured (i.e., encrypted andauthenticated). This is efficiently achieved byestablishing and using secure channels. Eachchannel is secured based on a session key secure-ly derived by the players based on their creden-tials, via a key exchange (KE) procedure. Eachsmart grid channel will be secured.

We assume that each grid device is capable ofsymmetric-key operations (e.g., AES), and thuscan use secure channels. For greater flexibility inestablishing secure channels, public-key-basedcredentials should be used. We note that public-key operations (e.g., RSA) are much more costly.While some end devices (e.g., sensors) may notsupport them, the core components in decentral-ized information infrastructure do. We envisionthe use of PKI and the corresponding simplifica-tion of key management. While there are severalInternet standards specifying KE based on PKI(e.g., TLS and derivatives such as TTLS andEAP-TLS), we believe currently there is no per-fect candidate for smart grid deployment. Themain reason is that smart grid is a mission-criticalsystem, and protocol simplicity and analysis pos-sibility prevails over its feature list. One approachto the solution could be stripping down TLS toits core (formally proven secure in [10]) and rein-stating only the necessary functionality.

We now shift focus away from how to estab-lish secure channels to how to use them. Encryp-tion can be performed End-to-End (E2E) orHop-by-Hop (HBH). E2E provides strongersecurity guarantees; namely, no intermediatenode can read or influence the messaging. Inmost multicast scenarios, straightforward E2Emay cause significant bottlenecks, for example,with broadcast and multicast messaging. Indeed,a publishers message, encrypted with two differ-ent subscribers keys, requires carrying bothencryptions along the entire transmission path.One solution to this problem is group keyexchange. In our architecture group KE is per-formed by the trusted overlay control networkproviding group keys to authorized sets of pub-lishers and subscribers. Further, we preventadversarial tracing of message flow, connectivityand other meta-information (which can be done,e.g., by observing packet headers and encrypteddata blocks leaving publishers and delivered tosubscribers). We do this by additionally requir-ing HBH encryption for all channels.

Recall, individual hub nodes must have

stronger security properties. This approachtaken by many current system designs (e.g., SIP),allows for simpler solutions, and is justified bysmart grid envisioned deployment scenarios.Security can be further hardened, e.g., as fol-lows. First, hub nodes should be diligent aboutdeleting group keys as soon as they are no longerneeded. Further, several hub nodes can servekey distribution, and players set the group key tobe the XOR of several keys. These techniquesmitigate the consequences of hub node compro-mise. Middleware and publishers may also storedata for future use. We envision that this datashould be encrypted, and the keys stored inhardened physical locations on site, or distribut-ed. For data availability, we envision it being dis-tributed, e.g., via coding among several nodes.

It can be seen that our approach allows forsecure and efficient E2E data delivery. Indeed,since the data is encrypted and authenticated, itcan be handled in a number of ways without com-promise of security. In particular, its use in dis-tributed storage, group communications, etc. doesnot introduce new vulnerabilities. (Note, theproblem of data retention and deletion, exacer-bated by distributed storage, is easier to handlewhen data is encrypted deletion of secret keyseffectively deletes the data.) At the same time,performance impact of our schemes is negligibledue to the use of symmetric-key encryption, e.g.,AES. As reported in [11], 4.27 Gb/s AES through-put was achieved using a 0.13 m CMOS technol-ogy with a 333 MHz clock rate, 86.2 K gates, and40.9 mW power. Even with dual-layer encryption,this will not constrain bandwidth or affect latencyor costs in a noticeable way. The more expensivestep of setting up secure channels occurs veryinfrequently, e.g., at node initialization time, andis not likely to have a performance impact.

Finally, we mention a critical security researchquestion of allowing emergency overrides forgaining access and controls. For this, we see spe-cial key escrow schemes developed or borrowedfrom literature. The final solution would dependon the actual deployment scenarios and require-ments.

RELATED WORKRecent work on smart grid communication sys-tems can be categorized as research-driven andstandards-driven.

Data-centrism, as opposed to the traditionalhost-centric networking, has been first addressedin wireless-sensor networks [6]. It has beenrecently advocated as content-centrism by Kopo-nen et al. [12] and Van Jacobson et al. [13].

In GridStat [14] D. E. Bakken et al. proposeda general QoS network architecture for support-ing peer-to-peer communication among entities.This framework includes status routers for dataplane and QoS brokers for the managementplane. However, the specific methods used tomeet specific QoS requirements are notaddressed. Moreover, according to their evalua-tion results, latency in GridStat linearly increaseswith the number of status routers. This impliesthat in communication paths with large numberof hops (status routers), it is hard to delivertime-sensitive data.

Our goal is to ensure

protection and

reliability of data and

communications.

That is, data must be

available with high

probability, both for

storage and

transmission, even in

the presence of

failures, and

adversary must not

be able to read the

data or interfere

with it.



The Electric Power Research Institute andNational Institute of Standards and Technology(NIST) have been building standardized informa-tion models: IEC 61850 for substation automa-tion and CIM for DMS applications. Theirapproach [15] is to evolve the current informa-tion infrastructure for the power grid. The net-work architecture uses a star-topology whereeach substation is allowed to communicate withonly the EMS, and communication among sub-stations is not allowed. Moreover, all necessarydata is sent and stored at the EMS to provide acomplete view of the power grid at any giventime. This centralized network architecture is vul-nerable to single points of failure and bottle-necks. In a slightly more advanced informationinfrastructure described in [3], phasor data fromPMUs is sent to PMU Data Concentrator (PDC)and then the aggregated data in the PDC is sentto the super-PDC. Using this hierarchical struc-ture, wide-area control using measured phasordata can be made more rapidly and precisely.However, since the network graph is a two-leveltree of PMU, PDC, and super-PDC, the scalabili-ty issue with respect to the large number ofPMUs still remains. Also, the single point of fail-ure still exists in PDCs due to the tree structure.

CONCLUSIONIn this article, we demonstrate the benefits ofdecentralized and data-centric information infra-structure for the next-generation power grid. Wepropose a secure middleware architecture thatleverages these features and can support theoperation of the power grid reliably, efficiently,and scalably by eliminating bottleneck failurepoints. The information infrastructure presentedhere differs from a typical distributed system dueto traits that are characteristics of the power gridapplications such as the coexistence of both ofLAN and WAN system, strict requirements ofboth latency and reliability and a combination ofboth data and event transactions. As an addi-tional contribution, we discuss our choices andimplementation details; we also take note of theimportant challenges that we will face as we planthe development and deployment of the nextgeneration grid.

REFERENCES[1] M. Amin and P. F. Schewe, Preventing Blackouts, Sci-

entific American, May 2007.[2] S. Chu, Investing in our Energy Future, GridWeek

Talk, Sept. 2009;http://www.energy.gov/news2009/documents2009/Sec-retary_Chu_Grid_Week.pdf.

[3] J. Giri, D. Sun, and R. Avila-Rosales, A More IntelligentGrid, IEEE Power & Energy Mag., Mar./Apr. 2009.

[4] UK DFB, Dynamic Demand: Government Response toClause 18 of the Climate Change and Sustainable Ener-gy Act, ACM CoNEXT, Aug. 2007;http://webarchive.nationalarchives.gov.uk/+/http://www.berr.gov.uk/files/file41011.pdf.

[5] P. T. Eugster et al., The Many Faces of Publish/Sub-scribe, ACM Comp. Surveys, vol. 35, 2003.

[6] S. Shenker et al., Data-Centric Storage in Sensornets,ACM Comp. Commun. Rev., vol. 33, 2003.

[7] R. Stewart, Stream Control Transmission Protocol,IETF RFC 4960, Sept. 2007.

[8] A. Rowstron, and P. Druschel, Pastry: Scalable, Decen-tralized Object Location and Routing for Large-ScalePeer-to-Peer Systems, IFIP/ACM Intl. Conf. DistributedSys. Platforms (Middleware), Nov. 2001.

[9] H. Xie et al., P4P: Provider Portal for Applications,ACM SIGCOMM, Aug. 2008.

[10] P. Morrissey, N. P. Smart, and B. Warinschi, A Modu-lar Security Analysis of the TLS Handshake Protocol,ASIACRYPT 08, 2008, pp. 5573.

[11] S.-Y. Lin and C.-T. Huang, A High-Throughput Low-Power AES Cipher for Network Applications, ADP-DAC,2007, pp. 595600.

[12] T. Koponen et al., A Data-Oriented (and Beyond) Net-work Architecture, ACM SIGCOMM, Aug. 2007.

[13] V. Jacobson et al., Networking Named Content,ACM CoNEXT, Dec. 2009.

[14] H. Gjermundrod et al., GridStat: A Flexible QoS-Man-aged Data Dissemination Framework for the PowerGrid, IEEE Trans. Power Delivery, vol. 24, Jan. 2009.

[15] EPRI, Intell iGrid Program 161;http://mydocs.epri.com/docs/Portfolio/PDF/2010_P161.pdf

ADDITIONAL READING[1] A. Bose, Smart Transmission Grid Applications and

Their Supporting Infrastructure, Report in Consortiumof Electric Reliability Technology Solutions (CERT), Oct.2008.

BIOGRAPHIESYOUNG-JIN KIM ([email protected]) holds aPh.D. degree in Computer Science from University ofSouthern California. He is a member of technical staff inBell Labs Network Technologies domain in Murray Hill, NJ.His research interests include distributed algorithms, proto-cols, and systems in large-scaled networks such as peer-to-peer networks and wireless ad-hoc/mesh networks. He isnow contributing to the design of scalable, secure anddata-centric network middleware through the Bell Labs Gachon Energy Research Institute (GERI) Smart Grid R&Dprogram. Before joining Bell Labs in 2010, he worked forSamsung Electronics Telecommunication R&D on wirelessnetworks domain. His research has been published in net-work conference proceedings and journals, and has beendistributed as publicly available software.

MARINA THOTTAN [M] ([email protected])is Director of the Mission-Critical Communications andNetworking Group at Bell Labs. She has contributed to awide variety of projects in diverse subject areas, includingonline gaming, content distribution, VoIP, routing proto-cols, data over optical networks, high-speed router design,network management, and anomaly detection. Mostrecently she has been leading work on smart grid commu-nication networks. She holds a Ph.D. degree in Electricaland Computer Engineering from Rensselaer PolytechnicInstitute (RPI). She has published over 40 papers in scien-tific journals, book chapters, and refereed conferences.She is a member of ACM.

VLADIMIR KOLESNIKOV ([email protected]) is a Member of Technical Staff in Bell LabsEnabling Computing Technologies domain in Murray Hill,NJ. He received his Ph.D. in Computer Science from theUniversity of Toronto in 2006. His research interests includekey exchange, secure multiparty computation, foundationsof cryptography, and network security. His work is con-nected to the practice of cryptography. He has worked onsecuring channels in Smart Grid and WiMAX, biometricauthentication, digital rights management, and a variety ofother subjects. Vladimir published his work in top crypto-graphic and security conferences and journals. He hasserved on program committees of several internationalcryptography conferences and supervised several summerinterns at Bell Labs. He is an editor of the WiMAX ServerCertificate Profile and Device Certificate Profile stan-dards documents.

WONSUCK LEE ([email protected]) was trained asa computational applied mathematician and received hisPh.D. degree from State University of New York at StonyBrook. He worked for The University of Texas at Austin,IBM Research, before he joined Bell Labs in 2001. His pro-fessional interests span across multiple fields of industrialapplied mathematics and their applications includingnumerical analysis, optical networks, fluid mechanics, andcomplex systems. He is the project leader of the multi-yearjoint Smart Grid R&D program between Bell Labs of Alca-tel-Lucent and GERI of Kyungwon University of Korea.

The information

infrastructure pre-

sented here differs

from a typical dis-

tributed system due

to traits that are

characteristics of the

power grid applica-

tions such as the

coexistence of both

of LAN and WAN

system, strict require-

ments of both laten-

cy and reliability and

a combination of

both data and event

transactions.


Documents

A Secure Decentralized Data Centric