Cyberthreat Analysis and Detection for Energy Theft …theft manipulates the output of a smart meter. Due to the nature of social system of a smart community, energy theft could potentially

148 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 2, NO. 4, DECEMBER 2015

Cyberthreat Analysis and Detection for EnergyTheft in Social Networking of Smart Homes

Yang Liu and Shiyan Hu, Senior Member, IEEE

Abstract—The advanced metering infrastructure (AMI) hasbecome indispensable in a smart grid to support the real timeand reliable information exchange. Such an infrastructure facil-itates the deployment of smart meters and enables the automaticmeasurement of electricity energy usage. Inside a community ofnetworked smart homes, the total electricity bill is computed basedon the community-wide energy consumption. Thus, the coordi-nated energy scheduling among smart homes is important sincethe energy consumptions from some customers can potentiallyimpact bills of others. Given a community of networked smarthomes, this paper analyzes the energy theft cyberattack, whichmanipulates the energy usage metering for bill reduction anddevelops a detection technique based on Bollinger bands and par-tially observable Markov decision process (POMDP). Due to thehigh complexity of the POMDP-solving process, a probabilisticbelief-state-reduction-based adaptive dynamic programming tech-nique is also designed to improve the detection efficiency. Oursimulation results demonstrate that the proposed technique cansuccessfully detect 92.55% energy thefts on an average while effec-tively mitigating the impact to the community. In addition, ourprobabilistic belief-state-reduction-based adaptive dynamic pro-gramming technique can reduce the runtime by up to 55.86%compared to that without state reduction.

Index Terms—Adaptive dynamic programming, energy theft,game theory, partially observable Markov decision process(POMDP), smart community.

I. INTRODUCTION

I N the modern smart grid, the massive deployment ofadvanced metering infrastructure (AMI) facilitates the effi-

cient and reliable information exchange [1]. The AMI can bedivided into several parts based on the locations, in whichthe home area network (HAN) is crucial for customers. InAMI, a smart meter is installed at the home of each customer,which in real time measures the energy usage of the customerand receives the electricity price from the utility (Fig. 1). Thesmart meter enables the automatic transmission of meteringdata, which saves the labor cost significantly. It also facilitatesthe smart home scheduling technology, which potentially con-tributes to the reduction of the electricity bills and balance ofthe grid-energy usage [2], [3].

The smart homes connecting to the same distribution net-work form a smart community. Such networking promotes thecooperation for various purposes. In particular, the communitybill is computed based on the total energy consumption and

Manuscript received June 14, 2015; revised November 30, 2015; acceptedJanuary 03, 2016. Date of current version February 25, 2016. (Correspondingauthor: Shiyan Hu.)

The authors are with the Department of Electrical and ComputerEngineering, Michigan Technological University, Houghton, MI 49931 USA(e-mail: [email protected]; [email protected]).

Digital Object Identifier 10.1109/TCSS.2016.2519506

proportionally distributed to the customers based on the indi-vidual energy usage, and more importantly, the energy usageof a customer can potentially impact the electricity bills ofothers [4]. Thus, the smart home scheduling problem is usu-ally tackled using game theory [5]–[7]. In the game theoreticsmart home scheduling technique, each customer plans his/herenergy consumption according to the energy usage of others.This helps balance the energy load in a power grid and reducethe electricity bill of a smart community.

Despite these advantages, the AMI is vulnerable to themalicious behavior. The modern smart meter typically has amicroprocessor with an embedded operating system. Such asmart meter can support functionalities such as remote updateof firmware. However, this creates the backdoor for malicioushackers. For example, a hacker can gain the root access and thenexecute the malicious code [8]. At least two types of cyberat-tack can be implemented utilizing this hacking. In our previouswork [9], the pricing cyberattack is studied, which manipulatesthe pricing information as the input to a smart meter. On theother hand, a hacker can tamper his/her own smart meter andmanipulate the measurement of energy usage to reduce his/herelectricity bill. This is known as energy theft. While the pricingcyberattack manipulates the input of a smart meter, the energytheft manipulates the output of a smart meter.

Due to the nature of social system of a smart community,energy theft could potentially impact electricity bills of allcustomers. In a smart community, the electricity bill is com-puted based on the total energy consumption in a past timewindow, and this total energy consumption is measured at thecommunity substation level. Since a community substation isregarded as being secure, energy theft typically happens at thelower level. Thus, this paper assumes that the substation energy-consumption measurement cannot be manipulated by hackers,whereas energy theft aims at manipulating the measurementof the home-level smart meters. Although energy theft cannotimpact the total electricity bill (which is computed as a functionof the total energy consumption), it has profound implication onbills of individual customers. After the total bill is computed,it will be distributed to customers proportional to their energyusage. The customers who consume more energy will be billedmore, while the sum among all customers is equal to the totalbill. Consequently, if a hacker reduces his/her smart meter mea-surement, his/her own bill will be reduced, while the bills ofothers will be increased, due to the fact that the total bill keepsthe same (Fig. 2).

This paper aims to explore the above social behav-ior among networked smart home customers for analyzingsmart community cybersecurity. To detect energy theft, there

2329-924X © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

LIU AND HU: CYBERTHREAT ANALYSIS AND DETECTION FOR ENERGY THEFT IN SOCIAL NETWORKING OF SMART HOMES 149

Fig. 1. AMI and the social networking of smart homes.

Fig. 2. Suppose that there are two customers in the community. The energyusage of customer 1 and 2 are 2 kWh and 1 kWh, respectively, and the totalelectricity bill is $3. The individual electricity bills are $2 and $1, respectively,according to the proportions of the individual energy usages. If customer 1hacks his/her own smart meter and manipulates the metering from 2 kWh to1 kWh, both customers will pay $1.5.

are actually some previous works [10], [11]. However, thetechniques there rely on the usage of FRTUs, which are expen-sive and they cannot handle the collusive energy theft attacks.In contrast, in this paper, energy theft is detected based on thehistorical meter readings of energy usage at each customer. Astraightforward idea is to use an statistical data-analysis tech-nique such as Bollinger bands, but it suffers from transientfluctuation on the customer energy usage, and it focuses onlyon the short-term energy theft effect. Thus, this paper integratesBollinger-bands-based detection with the partially observableMarkov-decision process (POMDP), which specially excels inhandling long-term detection. Note that POMDP-based detec-tion technique has been developed in our previous work [9].However, the work [9] studies the pricing cyberattack, whichmanipulates the guideline electricity price as the input of smart

Fig. 3. Model of smart community considered in this paper.

meters. In contrast, this paper studies the detection techniquefor energy theft, which manipulates the metering measurementas the output of smart meters. In addition, the probabilisticbelief-state reduction method is proposed to solve the POMDPproblem in this paper, which is more efficient than the conven-tional recursive method used in [9]. The contribution of thispaper can be summarized as follows.

1) The social behavior among networked smart-home cus-tomers is explored for the study on smart communitycybersecurity. In particular, the impact of the energy theftcyberattack which manipulates metering measurements isanalyzed.

2) A detection technique for the energy theft cyberattack isproposed through leveraging the statistical data analysisand control theoretic techniques such as Bollinger bandsand POMDP.

3) Due to the high time complexity of POMDP, a prob-abilistic belief-state-reduction-based adaptive dynamic-programming method is proposed to improve theefficiency.


4) As is demonstrated by the simulation results, energy theftcan reduce the electricity bill of the hackers at the costof increasing the electricity bills of other customers byup to 208.1%, and the energy usage measurement mis-match between substation level and home level can be upto 77.08%.

5) The proposed detection technique has a detection accu-racy of 92.55% on average. Compared to the scenarioswithout detection and with only Bollinger-bands-baseddetection, both of the average electricity bill increase andenergy usage measurement mismatch are significantlyreduced.

This paper is organized as follows. In Section II, the sys-tem model studied in this paper is presented. In Section III,the energy theft cyberattack is analyzed. In Section IV, thedetection technique for energy-theft cyberattack is proposed. InSection V, the simulation results are presented and analyzed. Asummary of the paper is given in Section VI.

II. SYSTEM MODEL

Consider a community consisting of N customers, which isdenoted by the set N = {1, 2, . . . , N} as shown in Fig. 3. Thescheduling horizon (e.g., a day) is divided into H time slots,which is denoted by H = {1, 2, . . . , H}. Each customer in thecommunity is equipped with a smart meter, which takes mea-surements of the energy usage and receives the electricity priceat each time slot. Furthermore, the total energy consumptionof the community is measured at the substation. The real timepricing model is deployed such that the community is billedbased on the total energy usage in a past time window, and sucha bill is split to each customer based on the individual energyusage. Without loss of generality, the popular quadratic modelis used for real time pricing [7]. At time slot h, the substation-level metering is denoted by Lh, which is the measurementof total energy consumption of the community. Thus, the totalelectricity bill of the community is [7]

Ch = ahL2h (1)

where ah is the pricing parameter. The home-level metering ofcustomer n at time slot h is denoted by ln,h, such that

Lh =∑n∈N

ln,h + θh (2)

where θh accounts for the energy usage measurement mismatchbetween substation-level metering and home-level meteringdue to the energy-theft cyberattack. Thus, the electricity bill ofcustomer n is calculated by

Cn,h =Chln,h∑i∈N li,h

=ahL

2hln,h∑

i∈N li,h. (3)

For the ease of readability, the variables used throughout thispaper are summarized in Table I.

TABLE ILIST OF NOTATIONS

III. ENERGY THEFT

The implementation of real time pricing needs the supportfrom AMI. While the utility calculates the total electricity billof the community based on the substation-level metering mea-surement, the community needs the home-level metering mea-surement to compute each individual electricity bill. However,this infrastructure is vulnerable to malicious behaviors of hack-ers [9]. The modern smart meters are based on system-on-chipdesigns that might have security backdoors. For example, thesmart meter produced by Texas Instrument supports the remoteupdate of firmware [12], and the hacker can potentially lever-age such backdoors to manipulate the measurement of the smartmeter [9].

A hacker which could also be a customer can launch theenergy theft cyberattack to reduce the metering measurementfor the reduction of his/her own electricity bill (Fig. 4). Supposethat customer n attacks his/her own smart meter at time sloth and reduces the measurement to l′n,h, such that l′n,h < ln,h.Thus, the electricity bill of the customer is reduced by

ahL2hln,h∑

i∈N li,h− ahL

2hl

′n,h∑

i∈N ,i�=n li,h + l′n,h

=ahL

2h(ln,h − l′n,h)

∑i∈N ,i�=n li,h∑

i∈N li,h(∑

i∈N ,i�=n li,h + l′n,h). (4)

The total electricity bill of the community is computed basedon the substation-level metering measurement, which is typi-cally assumed to be secure. This means that the total electricitybill also cannot be hacked. However, energy theft reduces the


Fig. 4. Energy theft can reduce the electricity bill of the hacker at the cost ofincreasing those of other customers.

metering measurement of the hacker, which further reduceshis/her electricity bill. Since the total bill of the communityremains the same, the reduction of electricity bill of the hackeris actually paid by other customers. For example, given a com-munity of two customers, the energy usages are 1 kWh forcustomer 1 and 2 kWh for customer 2, respectively. Supposethat the total electricity bill is $3. Thus, they will pay $1 and $2,respectively, for energy usage. However, if customer 2 launchesenergy theft to reduce the metering measurement from 2 to1 kWh, each customer will pay $1.5. In this scenario, customer2 reduces the electricity bill by $0.5 at the cost of increasingthat of customer 1 by $0.5.

Denoted by Np, the set of customers conducting energytheft, and Nq , the other customers that are also called normalcustomers. The electricity bill of each normal customer n iscalculated as

C ′n,h =

ln,h∑i∈Np

l′i,h +∑

i∈Nqli,h

. (5)

Since l′n,h < ln,h, one can derive C ′n,h > Cn,h according

to (5). Thus, the electricity bill of each normal customer isincreased by the energy-theft behavior.

On the other hand, energy theft can increase the energy-usagemeasurement mismatch between the substation-level meteringand home-level metering. When the customers in Np launchenergy theft, the mismatch between substation-level meteringand home-level metering can be computed as

Lmh = Lh −

∑n∈Np

l′n,h −∑n∈Nq

ln,h. (6)

Note that when energy-usage measurement mismatchexceeds some threshold, labor cost could be induced for per-forming checking.

IV. DETECTION TECHNIQUE FOR ENERGY THEFT

In this section, a detection technique for energy-theft cyber-attack is proposed based on POMDP, which is an advancedcontrol theoretic technique. Note that, POMDP-based detec-tion technique has been developed in our previous work [9].

Fig. 5. General flow of the proposed detection method.

However, this paper has some critical differences comparingto it.

1) The work [9] studies the pricing cyberattack, whichmanipulates the guideline electricity price as the input ofsmart meters. In contrast, this paper studies the detec-tion technique for energy theft, which manipulates themetering measurement as the output of the smart meters.

2) The work [9] uses Bollinger bands method in the way thatit only provides the short-term view of anomaly detection.In contrast, this paper fully explores the statistical ruleof the energy usage to construct Bollinger bands, whichis further integrated with POMDP to develop a highlyeffective long-term detection methodology.

3) The probabilistic belief state reduction method is used tosolve the POMDP problem in this paper, which is moreefficient than the conventional recursive method usedin [9].

A simple example of the detection framework is shown inFig. 5. The Bollinger bands technique is employed to detectif the energy-usage metering of each customer is normal.Subsequently, POMDP with probabilistic state-reduction tech-nique is used to compute the optimal action. According to theoptimal action, the community makes a decision on ignoringthe energy theft reports or checking and fixing the hacked smartmeters.

The POMDP technique consists of three key componentsincluding state, observation, and action [13]. The state impliesthe information obtained from the real world, which cannot beknown perfectly. Thus, observation is introduced to model theenergy usage measurement with uncertainty. In this paper, theBollinger-band technique is used as the method to obtain obser-vations. At each time slot, the community obtains observationand estimate the real state, according to which the communitytakes the action.

The POMDP model in our detection technique is formulatedas follows. The finite set S = {s0, s1, s2, . . . , sN} is the set ofstates corresponding to the energy theft in the community. Inthe set S , sn ∈ S means that there are n smart meters hacked.The finite set O = {o0, o1, o2, . . . , oN} is the set of observa-tion. on means that n smart meters are observed to be hackedby energy theft. Thus, sn is the real-world state while on is theobservation with uncertainty. The finite set X = {x0, x1} is theset of actions. In our formulation, there are two actions, wherex0 means that the energy theft cyberattack has no or negligi-ble impact to the community, and the community just needs


to keep monitoring the smart meters, while x1 means that thecommunity needs to check and fix the hacked smart meters.

POMDP models the mapping and interactions among state,observation, and action. Once an action x is taken, the state ischanged, which is modeled by the state transition probability.Suppose that the system state is s before taking x and s′ aftertaking x. The transition probability is T (s′, x, s). If x = x0,the community ignores the energy theft. Thus, the state tran-sition only depends on the behavior of the hackers. If x = x1,the community checks the energy theft and fixes all the smartmeters with problems. Thus, the state is reset to s0, i.e.,

T (s′, x1, s) =

{1, ifs′ = s0

0, otherwise.(7)

The observation is updated when action x is taken. The map-ping between state and observation is denoted by Ω(o, x, s).It refers to the probability that the observation is o when thestate is s. If x = x0, the mapping between state and observationdepends on the accuracy of the Bollinger-bands-based detectiontechnique. If x = x1, all the smart meters are checked and fixed.Thus, the observation can only be o0, i.e.,

Ω(o, x1, s) =

{1, if o = o0

0, otherwise.(8)

In the context of our problem, ignoring the energy theft canintroduce system loss. On the other hand, checking and repair-ing the smart meters also induces labor cost due to the necessaryhuman interactions. These costs are modeled by the reward inPOMDP problem. Suppose that the state transits from si to sjafter taking action x. If x = x0, the system loss is relevant tothe number of hacked smart meters. In our formulation, a linearmodel is deployed to model such a system loss that each hackedsmart meter induces a system loss Ca. If x = x1, the commu-nity needs to check the smart meters and fixes the hacked ones.Suppose that it costs Cl

1 to check the smart meters and Cl2 to fix

a hacked smart meter. The reward R(sj , x, si) is

R(sj , x, si) =

{−j · Ca, if x = x0

−Cl1 − (i− j)Cl

2, if x = x1.(9)

The community aims to take an action at the current timeslot, which can maximize the long-term reward. Suppose thatthe current time slot is time slot 1 and the reward received bythe community at time slot t is rt. Since the future event hasmore uncertainties than the current moment, POMDP typicallyuses the discount factor γ to reduce the impact of the future.Thus, the expected long-term reward is [13]

E

[ ∞∑t=1

rt · γ]. (10)

The community obtains observation at each time slot andchooses the action that maximizes (10).

A. Observation Technique

The detection technique can be divided into substation-leveldetection and home-level detection. Note that the substation-level metering should match the summation of home-level

metering. However, they cannot match perfectly due to theenergy losses in the distribution system and noise of measure-ment. Thus, the substation-level detection technique reportsenergy theft if the mismatch between substation-level measure-ment and home-level measurement is significant, i.e.,

Lh −∑

n∈N ln,h

Lh> η (11)

where η is the threshold.The home-level detection is based on the comparison with

historical energy usage of each customer. From the historicaldata of the energy usage, one can derive the range of energyusage for each customer. If the measurement of smart meterexceeds that range, energy theft is reported. In this paper,Bollinger bands method is used as the range of the energyusage, which is a popular statistical data analysis methodwidely used in financial analysis [14]. Based on the historicaldata, the Bollinger bands method constructs an upper band andan lower band, which limit the normal range of the future data.In our problem context, the Bollinger-bands-based technique isimplemented as follows.

1) Denoted by [l1n, l2n, . . . , l

Tn ], the historical energy usage of

customer n in the past T days, where ltn is the energyusage of customer n in the day t from time slot 1 to timeslot H , such that

ltn =[ltn,1, l

tn,2, . . . , l

tn,H

]. (12)

2) Based on the historical energy usage [l1n, l2n, . . . , l

Tn ],

compute the moving average lAn = [lAn,1, lAn,2, . . . , l

An,H ]

for each customer. For completeness, the details formoving-average computation is included as follows.

Suppose that the average electricity bill In =∑T

t=1 ItnT ,

where In = {I1n, I2

n, . . . , IH

n }. Given a window size mparameter, the moving-average is computed by IAn,t =∑t+m

h= t−m Ih

n, where h = h−H if h > H and h = H −h if h < 1.

3) Based on the average historical energy usage{Itn|h−m ≤ t ≤ h+m} and the moving averagelAn = [lAn,1, l

An,2, . . . , l

An,H ], compute the standard

deviation σn,h.4) The upper band lun,h and lower band lln,h of the Bollinger

bands are computed as

lun,h = lAn,h +Kσn,h (13)

and

lln,h = lAn,h −Kσn,h (14)

where K is a positive integer.

If the energy usage at time slot h exceeds the lower band, i.e.,

ln,h < lln,h. (15)

Energy theft is reported at the home level. The energy theft alertis sent to the utility only if both conditions (11) and (15) aresatisfied. If there are i smart meters reporting energy theft, theobservation is obtained as oi.


Fig. 6. Algorithmic flow of the proposed energy theft detection technique.

B. Solution of POMDP Problem

Since the real-world state cannot be perfectly known, thecommunity needs to estimate it from the observation. The esti-mation of the real-world state is defined as belief state b ={b(s0), b(s1), b(s2), . . . , b(sN )}, where b(sn) = P (s = sn) isthe probability that the current state is sn. For example, b(s1) =0.5 means that the probability that one smart meter is hacked byenergy theft is 0.5. After an observation o is obtained, the beliefstate is updated according to [13]

b(s′) = P (s′|o, x, s) = Ω(o, x, s′)∑

s∈S T (s′, x, s)b(s)P (o|x, b)

(16)where P (o|x, b) is a normalizing factor, such that∑

s′∈S b(s′) = 1. Since the belief state is the estimationof the state, it can represent the best knowledge of the com-munity to the energy theft. Thus, the previous model can besimplified based on the belief state to facilitate the solution ofthe POMDP problem. Based on the belief state, the POMDPproblem can be represented by 〈b, x, τ, ρ〉, where τ is thebelief-state transition probability and ρ is the correspondingreward function.

The belief state transition can be computed as [13]

τ(b′, x, b) = P (b′|x, b) =∑o∈O

P (b′|b, x, o)P (o|x, b) (17)

where the definition of P (o|x, b) is the same with that in(16). P (b′|b, x, o) represents the transition probability of beliefstate given the observation o. Note that the new belief state b′

can be directly computed from b, x, and o according to (16).

Following [9], one has

P (b′|b, x, o) ={1, if(b, x, o)⇒ b′

0, otherwise.(18)

When the community takes action x while the belief state isb, it can achieve a reward ρ(x, b), which is similar to the rewardR(s′, x, s). The reward ρ(x, b) is defined based on belief stateas follows [13]:

ρ(x, b) =∑s∈S

∑s′∈S

b(s)R(s′, x, s)T (s′, x, s). (19)

Given the complete POMDP model problem, the communityaims to maximize the long-term expected reward (10). Denotedby V ∗(b), the optimal value of the long-term expected rewardwhen the current belief state is b. Based on the belief state b,the belief state transition τ(b′, x, b), and the belief state rewardρ(x, b), the optimization procedure can be cast to solving theBellman equation as follows [13]:

V ∗(b) = maxx∈X

{ρ(x, b) + γ

∑b′∈B

τ(b′, x, b)V ∗(b′)

}. (20)

C. Probabilistic Belief-State Reduction Method

There are many techniques to solve Bellman equation, whichinclude Q-learning [15], dynamic programming, and adaptivedynamic programming [16], [17]. In the standard dynamic-programming-based solving process, one needs to search thewhole belief state space and the time complexity increases


exponentially [18]. In contrast, adaptive dynamic programmingestimates the maximum value of Bellman equation based on theprevious knowledge or partial information of the future, whichsignificantly reduces the time complexity.

In this paper, through exploring the nature of our formu-lated POMDP problem, an innovative probabilistic belief-state-reduction-based adaptive dynamic programming technique isproposed. The basic idea is to give the preference to thestates with higher probabilities. Based on this philosophy, thebelief-state space can be reduced before solving the Bellmanequation (20). Given a belief state space, only the states whoseprobabilities are large enough are kept for further computingprocedure.

In our implementation, two sets are defined which are Bzand Bc, respectively. Among them, Bz is the set of belief stateswith nonzero probabilities, and Bc is the set of candidate beliefstates used for solving the Bellman equation. Denoted by P (b),the average probability of the belief states in Bz , such that

P (b) =

∑s∈Bz

b(s)

|Bz| (21)

where |Bz| is the size of Bz . The state s is included in Bc ifb(s) ≥ P (b). Denoted by br, the reduced belief state space, inwhich the probability of each belief state is computed by

br(s) =b(s)∑

s∈Bcb(s)

∀s ∈ Bc. (22)

Thus, the POMDP problem is formulated with the reducedbelief states and the Bellman equation is reduced to

V ∗(br) = maxx∈X

{ρ(x, br) + γ

∑b′∈B

τ(b′, x, br)V ∗(b′r)

}(23)

where b′r is the reduced belief state of b′. Similar to solving(20) using standard dynamic programming, (23) is solved usingadaptive dynamic programming. The complete procedure ofthe probabilistic belief-state-reduction-based adaptive dynamicprogramming is presented in Algorithm 1.

The complete algorithmic flow of the proposed energy theftdetection technique is shown in Fig. 6. As is shown, the com-munity receives the real-time measurements of the smart metersand compares them with the Bollinger bands, which are con-structed from the energy usage record. The result is obtainedas the observation of the POMDP problem. Subsequently,the community estimates the belief state according to (16)and reduces it according to (21) and (22). Subsequently, theBellman equation (23) is solved.

A simple example illustrating our detection procedure isshown in Fig. 7. Given a mini community consisting of fivecustomers, each smart meter compares the real time meteringwith the lower Bollinger band. Suppose that a hacker manip-ulates the metering from 1.5 to 0.5. The smart meter reportsenergy theft since 0.5 is below 1.2, the lower Bollinger band atthe corresponding time slot. Meanwhile, suppose that two othersmart meters also report energy theft, and thus the observationis o = o3. The community uses POMDP technique to computethe optimal action. It updates the belief state b according to

Algorithm 1. Probabilistic Belief-State-Reduction-BasedAdaptive Dynamic-Programming Technique (SR-ADP)

1: g = 12: Obtain b3: Begin SR-ADP(b, g, γ, ε)4: if g < ε then5: Return 06: else7: V = ∅8: Compute P (b) using P (b) =

∑s∈Bz

b(s)

|Bz| .

9: Compute br(s) using br(s) =b(s)∑

s∈Bcb(s) , ∀s ∈ Bc.

10: for Each action xi in X do11: Vi = g × ρ(xi, br)12: b′ ← τ(b′, x, b) =

∑o∈O P (b′|b, x, o)P (o|x, b)

13: Vi = Vi + SR-ADP(b′, g × γ, γ, ε)14: V⋃Vi

15: end for16: Return max(V ∈ V)17: end if18: End SR-ADP

Fig. 7. Simple example illustrating the detection procedure.

the observation and computes the expected rewards associatedwith x0 and x1, denoted by Vx0

(b) and Vx1(b), respectively.

If Vx1(b) > Vx0

(b), the optimal action is x1, and the identifiedsmart meters are checked and fixed.

V. SIMULATION RESULTS

In this section, three sets of simulations are conducted toanalyze the impact of energy theft and the performance ofdetection techniques. In the first set of simulations, differ-ent attacking levels of the hackers and the electricity bills,increases of electricity bills, and mismatch between home-leveland substation-level measurements are presented. In the secondset of simulations, the performance of detection technique isanalyzed under different labor cost, which can impact the opti-mal action computation, and different sets of historical energyusage, which can impact the computation of Bollinger bands. In


Fig. 8. Average bill increase ratio of normal customers when the number ofhackers increases.

Fig. 9. Mismatch between substation-level metering and home-level meteringwhen the number of hackers increases.

the third set of simulations, the runtime of the detection tech-niques is compared with and without state-reduction techniqueunder different sizes of communities. The energy usage profileof each customer is designed similar to our previous work [5].The simulations are conducted on a 64 bit computer with dualIntel(R) Core(TM) i5-2410M 2.30 GHz CPU and 4G RAMusing MATLAB.

In our setup, each customer in the community can become ahacker based on the probabilistic transition. When a customerbecomes a hacker, he/she has two choices, either being inactiveor active. If the hacker stays inactive, he/she will not manipulatethe measurement of the smart meter. At this moment, this cus-tomer will still be treated as a normal customer. If the hackeris active, he/she will tune down the measurement. Basically,the hacker will randomly decide whether he/she will becomeactive and how much measurement he/she will manipulate.We use state-transition probability T (sj , x0, si) to computethe probability that the number of hackers becomes j from i.Subsequently, we uniformly randomly choose j customers fromthe community as hackers. The probability for a hacker to beinactive is set as 1/3.

Fig. 10. Energy usages of the customer with and without energy theft and theBollinger bands.

Fig. 11. Detection accuracies of the POMDP-based and Bollinger-bands-basedmethod.

TABLE IIPARAMETERS USED IN SIMULATIONS

A. Impact of Energy Theft

In this simulation, four different attacking categories areconsidered. For each attacking category, the upper bound ofmeasurement reduction is 100%. The lower bounds are 0%,30%, 60% and 90% below the upper bound for attacking cat-egory 1, 2, 3 and 4, respectively, which represent differentenergy theft levels from high to low. In order to analyze theimpact of energy theft to the community, the bill increase ratiois used, which is the increase of electricity bill compared tothat without energy theft for a customer. The electricity bills ofthe normal customers, bill increase ratios, and the energy usagemeasurement mismatch between substation-level metering andhome-level metering are compared, when the number of hack-ers increases from 1 to 100. The results are shown in Figs. 8 and9, respectively. The following observations can be obtained.


TABLE IIICOMPARISONS BETWEEN DIFFERENT DETECTION TECHNIQUES

1) Shown in Fig. 8 is the average bill increase ratios ofthe normal customers. The average bill increase ratiobecomes larger with more hackers and it can reach208.1%, 167.3%, 139.9%, and 115.1%, respectively,under different attacking levels.

2) Shown in Fig. 9 is the energy usage measurement mis-match between substation-level metering and home-levelmetering. The mismatch increases while the number ofhackers increases, and the largest mismatch is 77.08%,64.82%, 53.87%, and 48.05%, respectively, under differ-ent attacking levels.

B. Detection

The detection technique is evaluated in this simulation. ThePOMDP-based long-term detection technique is compared to anatural heuristic, which only uses Bollinger-bands-based detec-tion. The comparison on the detection accuracy, electricity billincrease, mismatch, and labor cost are made. Detection accu-racy refers to the ratio of successful detected events to the totalnumber of events under detection is presented. Unsuccessfullydetected events include the detection results containing falsepositive and false negative. The detection accuracy is definedas 1− |j−i|

i , where i is the real number of hacked smart meters(with energy theft) and j is the detected number of hacked smartmeters (with energy theft). For the POMDP-based detectiontechnique with state reduction, different setups for labor costare considered. In addition, different historical energy usage isalso considered, which can potentially impact the computationof Bollinger bands.

For each customer, the Bollinger bands of energy usage areconstructed based on the historical energy usage in the past 7days. The Bollinger bands are shown in Fig. 10. Furthermore,the energy usage curves with and without energy theft arealso presented there. As is shown, when there is no energytheft, the energy usages locate within the range of Bollingerbands. In contrast, when there is energy theft, the energyusage is below the lower Bollinger band from 11:00 to 16:00.Thus, the energy theft can be detected by the Bollinger bands.However, such a technique has a very local view of energy theftcyberattack and cannot account for the energy usage behav-ior change of customers. The accuracies of the POMDP-baseddetection technique and Bollinger-bands-based technique arecompared in Fig. 11. As one can see, the average accuracy ofthe POMDP-based detection technique is 92.55%, while that ofthe Bollinger-bands-based detection technique is only 65.30%.

The energy usage measurement mismatch, average billincrease, and labor cost of the POMDP-based detectiontechnique and Bollinger-bands-based detection technique areshown in Table III. In order to demonstrate the advantagesof the proposed detection technique, the performance of the

Fig. 12. Detection accuracies of the POMDP-based detection technique withdifferent assumptions of historical energy usages.

detection techniques is also compared to the scenario with-out any detection technique. The following observations can beobtained.

With different setup of labor cost, the detection accuracyof Bollinger-bands-based detection technique and POMDP-based detection technique with and without state reductiontechnique is shown in Fig. 11. While the detection accu-racy of Bollinger-bands-based detection technique is 65.30%,the detection accuracy of POMDP-based detection techniquewithout state reduction technique is 92.77%. The detectionaccuracies of POMDP-based detection technique with statereduction are 91.65%, 91.92%, 92.15%, and 92.55%, respec-tively. Using these techniques for cyberattack detection, themismatch between substation level and home-level detection,average bill increase of normal customers and normalized laborcost are presented in Table III. Without detection technique, themaximum mismatch is 25.21% and the average bill increase is53.45%. Using Bollinger-bands-based detection technique, themaximum mismatch and the average bill increase are 8.64%and 18.31%, respectively. Using the POMDP-based detec-tion technique without state reduction, the mismatch betweenhome-level and substation-level metering is 1.49%, the aver-age bill increase is 3.87% and the normalized labor cost is1.092. Using the POMDP-based detection technique with statereduction, the mismatch between home-level and substation-level metering varies from 1.87% to 5.14%, the average billincrease varies from 3.98% to 8.43%, and the normalizedlabor cost varies from 1.042 to 1.214, which are shown inTable III.

With different historical energy usage profiles, the detectionaccuracies are shown in Fig. 12. Basically, the accuracy variesfrom 92.14% to 93.56%. The corresponding mismatch between


TABLE IVCOMPARISONS OF THE POMDP-BASED DETECTION TECHNIQUE UNDER DIFFERENT ASSUMPTIONS ON HISTORICAL ENERGY USAGES

TABLE VRUNTIME COMPARISON BETWEEN THE POMDP-BASED DETECTION TECHNIQUES WITH AND WITHOUT STATE REDUCTION

substation-level metering and home-level metering, average billincrease, and normalized labor cost are shown in Table IV.The mismatch between substation-level metering varies from2.03% to 1.71%. The average bill increase varies from 4.16%to 3.24%, and the normalized labor cost varies from 1.054 to1.143.

In the third set of simulation, the runtime of the POMDPmethod using state reduction is compared to that based on thestandard dynamic programming. The community consisting of100, 200, 300, 400, and 500 customers is considered, respec-tively, for a simulation period of 48 time slots. The results areshown in Table V. According to Table V, the runtime is reducedsignificantly by the state reduction technique. In particular, over50% runtime reduction has been observed for the large testcaseswith 300, 400, and 500 customers.

VI. CONCLUSION

In this paper, the social system nature of networked smarthomes is explored to study the energy theft cyberattack ina smart community. A technique based on Bollinger bandsand POMDP is developed to detect the energy theft cyber-attacks. Due to the high complexity of POMDP solving, aninnovative probabilistic belief-state-reduction-based adaptivedynamic-programming technique is also proposed to improvethe detection efficiency. Our simulation results demonstrate thatenergy theft can increase the average electricity bill of nor-mal customers by up to 208.1% and increase the energy-usagemeasurement mismatch between substation-level metering andhome-level metering by 77.08%. In addition, the proposedPOMDP-based energy theft detection technique can success-fully detect 92.55% energy theft on average with significant billreduction and little labor overhead.

REFERENCES

[1] W. Wang and Z. Lu, “Cyber security in the smart grid: Survey andchallenges,” Comput. Netw., vol. 57, no. 5, pp. 1344–1371, 2013.

[2] X. Chen, T. Wei, and S. Hu, “Uncertainty-aware household appliancescheduling considering dynamic electricity pricing in smart home,” IEEETrans. Smart Grid, vol. 4, no. 2, pp. 932–941, Jun. 2013.

[3] L. Liu, Y. Liu, L. Wang, A. Zomaya, and S. Hu, “Economical and bal-anced energy usage in the smart home infrastructure: A tutorial andnew results,” IEEE Trans. Emerging Topics Comput., vol. 3, no. 4,pp. 556–570, Dec. 2015.

[4] Y. Liu, S. Hu, H. Huang, R. Ranjan, A. Zomaya, and L. Wang, “Gametheoretic market driven smart home scheduling considering energy bal-ancing,” IEEE Syst. J., to be published.

[5] L. Liu, Y. Zhou, Y. Liu, and S. Hu, “Dynamic programming basedgame theoretic algorithm for economical multi-user smart home schedul-ing,” in Proc. IEEE Midwest Symp. Circuits Syst. (MWSCAS), 2014,pp. 362–365.

[6] S. Caron and G. Kesidis, “Incentive-based energy consumption schedul-ing algorithms for the smart grid,” in Proc. IEEE Int. Conf. Smart GridCommun. (SmartGridComm), Oct. 2010, pp. 391–396.

[7] A.-H. Mohsenian-Rad, V. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia, “Autonomous demand-side management based on game-theoreticenergy consumption scheduling for the future smart grid,” IEEE Trans.Smart Grid, vol. 1, no. 3, pp. 320–331, Dec. 2010.

[8] G. Hernandez, O. Arias, D. Buentello, and Y. Jin, “Smart nest thermostat:A smart spy in your home,” Black Hat USA, 2014.

[9] Y. Liu, S. Hu, and T.-Y. Ho, “Leveraging strategic detection technique forsmart home cyberattacks,” IEEE Trans. Depend. Secure Comput., to bepublished.

[10] C. Liao, C.-W. Ten, and S. Hu, “Strategic FRTU deployment consider-ing cybersecurity in secondary distribution network,” IEEE Trans. SmartGrid, vol. 4, no. 3, pp. 1264–1274, Sep. 2013.

[11] Y. Zhou, X. Chen, A. Zomaya, L. Wang, and S. Hu, “A dynamic pro-gramming algorithm for leveraging probabilistic detection of energy theftin smart home,” IEEE Trans. Emerging Topics Comput., vol. 3, no. 4,pp. 502–513, Dec. 2015.

[12] Texas Instruments, Smart Meter. [Online]. Available: http://www.ti.com/solution/docs/appsolution.tsp?appId=407, accessed on May 2015.

[13] L. P. Kaelbling, M. L. Littman, and A. Cassandra, “Planning and actingin partially observable stochastic domains,” Artif. Intell., vol. 101, no. 1,pp. 99–134, 1998.

[14] C. Lento, N. Gradojevic, and C. Wright, “Investment information contentin bollinger bands?” Appl. Financ. Econ. Lett., vol. 3, no. 4, pp. 263–267,2007.

[15] S. Singh, A. Barto, R. Grupen, and C. Connolly, “Robust reinforcementlearning in motion planning,” in Proc. Adv. Neural Inf. Process. Syst.,1994, pp. 655–662.

[16] Z. Ni, H. He, D. Zhao, X. Xu, and D. Prokhorov, “GrDHP: A generalutility function representation for dual heuristic dynamic programming,”IEEE Trans. Netw. Learn. Syst., vol. 26, no. 3, pp. 614–627, Mar. 2015.

[17] Z. Ni, H. He, J. Wen, and X. Xu, “Goal representation heuristic dynamicprogramming on maze navigation,” IEEE Trans. Netw. Learn. Syst.,vol. 24, no. 12, pp. 2038–2050, Dec. 2013.

[18] P. Poupart and C. Boutilier, “VDCBPI: an approximate scalable algo-rithm for large POMDPS,” in Proc. Adv. Neural Inf. Process. Syst., 2004,pp. 1081–1088.

Yang Liu received the B.S. degree in telecommu-nications engineering from Huazhong University ofScience and Technology, Wuhan, China, in 2011.Currently, he is pursuing the Ph.D. degree in electricalengineering at Michigan Technological University,Houghton, MI, USA.

He was a Visiting Student at Carnegie MellonUniversity, Pittsburgh, PA, USA, in 2015. Hisresearch interests include smart home system, cyber-physical systems, and big data analytics.


Shiyan Hu (SM’10) received the Ph.D. degree incomputer engineering from Texas A&M University,College Station, TX, USA, in 2008.

He was a Visiting Professor at IBM Research(Austin) during Summer 2010. He has been a VisitingAssociate Professor at Stanford University, Stanford,CA, USA, and an Associate Professor with theDepartment of Electrical and Computer Engineering,Michigan Tech., Houghton, MI, USA, where heis Director of the Michigan Tech Cyber-PhysicalSystem Research Group and Director of the Michigan

Tech VLSI CAD Research Laboratory. He has authored more than 100 refereedpapers. His research interests include cyber-physical systems, cybersecurity,computer-aided design of VLSI circuits, and embedded systems.

Dr. Hu is the Founding Chair for IEEE Technical Committee on Cyberneticsfor Cyber-Physical Systems, an ACM Distinguished Speaker, and an IEEEComputer Society Distinguished Visitor. He is an Associate Editor/GuestEditor for the 7 IEEE/ACM Transactions such as IEEE TRANSACTIONS ON

COMPUTERS and the IEEE TRANSACTIONS ON CAD. He was the GeneralChair, the Technical Program Committee (TPC) Chair, a TPC SubcommitteeChair, a Session Chair, and a TPC Member for various conferences formore than 70 times, which include the TPC Subcommittee Chair for DACand ICCAD. He was the recipient of the National Science Foundation(NSF) CAREER Award, the ACM SIGDA Richard Newton DAC Scholarship(as the faculty advisor), and the Faculty Invitation Fellowship from JapanSociety for the Promotion of Science (JSPS).

Documents

Cyberthreat Analysis and Detection for Energy Theft …theft manipulates the output of a smart meter. Due to the nature of social system of a smart community, energy theft could potentially