A Fuzzy Markov Model for Scalable ReliabilityAnalysis of Advanced Metering Infrastructure

Saman A. Zonouz, Robin Berthier, and Parisa HaghaniUniersity of Illionois at Urbana Champaign{saliari2, rgb, haghani1}@illinois.edu

Abstract—The capabilities of smart meters and the potentialvulnerabilities they introduce make the Advanced MeteringInfrastructure (AMI) a critical components of the Smart Grid.A virus propagating among meters and massively issuing remotedisconnect commands could have catastrophic consequences. Tounderstand and assess the risk posed by a given set of meterdevices getting compromised, we introduce a novel modelingapproach based on fuzzy Markov chain. Markov modeling isone of the main approaches used to analyze the reliabilityof critical systems. Markov models are usually analyzed usingmathematical approaches or Monte-Carlo Simulation. For largeand complex systems, traditional Markov models suffer from thestate explosion problem and quickly become intractable. The keyidea of our approach to solve this issue in the context of the smartgrid is to leverage the hierarchical topology of AMI to builda high-level model where components represent NeighborhoodArea Networks (NAN) instead of individual meters. The fuzzyMarkov formalism enables our solution to take into accountgradual component degradation. We propose a Fuzzy MarkovModel (FMM) for reliability analysis of a Smart Grid. Thereliability of each NAN is represented by a number in [0, 1]which denotes the state of that component in the FMM.We definethe transition probabilities between states of the fuzzy Markovmodel based on the fuzzy time-to-failure value for each NAN.We utilize the Fuzzy Monte Carlo Simulation (FMCS) approachto calculate the fuzzy time-to-failure value of each NAN. Ourproposed method enables us to assess the reliability of the entireAMI at the meter level while keeping the model manageable.

I. INTRODUCTION

Advanced Metering Infrastructure (AMI) are being de-ployed at a fast rate worldwide. They often represent largeengineering systems for which studying the reliability overtime is mandatory. The failure of smart meters in the fieldcan lead to expensive recovery operations and can have asignificant impact on the energy delivery mission of utilities.Moreover, it is critical to understand the failure mode of AMIin order to correctly plan and deploy protective solutions. Keyquestions to address include: How would a given AMI beimpacted if X meters fail?, or Which section of a given AMIshould be protected in priority to better mitigate the impactof repeated failures?To answer these questions and to gain a deep understanding

of the impact of failures on AMI functionalities, we suggestthat a relevant approach is to develop a model-based solution.Generally, there are two approaches for the analysis of critical

systems: inductive and deductive [?], [?]. Markov modelis one of the deductive approaches employed in reliability,availability, maintainability and safety domain [?], [?], [?],[?]. In fact, the Markov modeling is an approach of deter-mining system behavior by using information about certainprobabilities of events within the system [?]. Markov modelsare usually analyzed using a mathematical approach, e.g.,Laplace transforms, or a simulation approach, e.g., Monte-Carlo Simulation (MCS) [?]. However, Markov models havetwo main drawbacks. First, they rely only on the probabilityparadigm. Exploiting only probabilities is often inappropriatedue to insufficient probabilistic information during the designand prototyping phases [?], [?]. Moreover, because in theclassical Markov models, components are modeled to beonly in two types of states: (1) fault free and (2) failed;therefore, components that fail gradually cannot precisely bemodeled by the classical Markov models. To overcome thefirst limitation, M. L. Leuschen et al. proposed fuzzy Markovmodels in [?], [?]. They employed fuzzy logic in conjunctionwith probability paradigm in reliability engineering of roboticscontext, providing a solution that is much more appropriate toanalyze real systems under considerable uncertainty [?], [?].We believe that developing a fuzzy Markov model for AMI

offers a sound approach to assess the reliability of large AMIbecause we can leverage the hierarchical topology of AMIto build a scalable and still accurate model. The key ideato develop such model is to represent neighborhood areanetworks (NANs) at the component level so that the numberof components in the model remains tractable. The accuracyof the reliability analysis can be maintain at the meter levelthanks to the fuzzy logic that provides a granular failure modefor NANs.This paper presents a fuzzy Markov modeling approach and

shows how to apply it to study the reliability of AMI. Morespecifically, the contributions of this work are the followings.• We show how traditional reliability approaches are nottractable to assess the failure risk of large AMIs.

• We discuss two advantages of fuzzy Markov modelingover traditional Markov models: first, the capability toanalyze the systems with components that fail gradually(i.e. degradation); second, the possibility to model asystem in environments with both cognitive and non-

978-1-4577-2159-5/12/$31.00 ©2011 IEEE

cognitive uncertainties.• We present a Fuzzy Markov Model (FMM), illustrate itsintegration for AMI, and show how the Fuzzy MonteCarlo Simulation approach [?] is used to derive the fuzzytime-to-failure values necessary to calculate the statetransition probabilities in our FMM.

This paper is organized as follows. In Section II, we givea brief overview of the related work on reliability analysis ofSmart Grids and fuzzy logic applied for reliability. In SectionIII, we present the proposed model, i.e., Fuzzy Markov Model(FMM). In Section IV, we present an approach to measurethe time-to-failure for a specific NAN using the Fuzzy MonteCarlo Simulation (FMCS) approach [?]. Finally, we show howto apply our approach to assess the security of an AMI inSection V and we conclude the paper in Section VI .

II. RELATED WORK

Reliability analysis of Smart Grids has been a hot topicin the previous years. Moslehi and Kumar [?] review thereliability impacts of major smart grid resources such asrenewables, demand response, and storage. Green et al. [?]present an approach based on state pruning to improve thecomputational efficiency and convergence of Monte CarloSimulation used in analyzing the reliability of power sys-tems. Choi et al. [?] present a methodology for evaluatingthe probabilistic reliability of a grid constrained compositepower system. The introduction of fuzzy logic into reliabilitymodeling has attracted several research efforts. S. Aliari et al.presented a Fuzzy Monte Carlo Simulation Approach (FMCS)in [?]. FMCS is briefly explained in Section IV. Using fuzzyarithmetic, FMCS produces fuzzy time-to-failure for the com-ponents of fault-tolerant systems. As input, these fuzzy times-to-failure are given to a kind of fault tree, i.e. Fuzzy Time-To-Failure (FTTF) trees to estimate reliability of the fault-tolerantsystems. M. L. Leuschen et al. proposed fuzzy Markov modelsin [?], and [?]. They employed fuzzy logic in conjunctionwith probability paradigm in reliability engineering of roboticscontext. Therefore, their modeling approach is capable ofconsidering both cognitive and non-cognitive uncertainties.However, their approach is not able to model systems withcomponents that fail gradually, i.e. degradation. This meansthat the components of a system are thought to fail completelyin a single point of time before which they are thought towork completely well (i.e., be fault-free). P. S. Cugnasca et al.presented a generic technique based on the Markov models,using the fuzzy theory for the reliability and safety assessmentof fault-tolerant computational systems [?]. As the Markovmodel requires some parameters which may not be preciselyknown during the calculation, this method allows uncertainty-based parameters that are represented as fuzzy numbers. As aconsequence, the reliability and the safety time response arecomposed by several curves which have associated degrees ofconfidence. This result is then used to estimate the reliability

Fig. 1. Traditional Markov model for a two-component system

and safety value ranges at a given instant of time where eachvalue has an associated possibility degree in the system. Wealso note that A. Ejlali et al. [?] presented a new model calledtime-to-failure tree. Both static and dynamic fault trees can beeasily transformed into time-to-failure trees. In fact, each time-to-failure tree is a digital circuit, which can be synthesized toa field programmable gate array (FPGA). Therefore, MonteCarlo simulation can be significantly accelerated using FPGAs.

III. FUZZY MARKOV MODEL

We start by describing a traditional Markov model formodeling a network of smart meters. In this model, the systemconsists of a network of smart meters, and each smart meteris considered as an independent component of the system. Letus assume that there are a total of N smart meters in thissystem. The overall state of the system is described by thesequence s1, s2, · · · , sN , where si presents the state of the i-th component (smart meter) of the system. Each component ican be either fault-free, i.e., si = 0, or it can be failed, i.e.,si = 1. Therefore, there are 2N states for the overall system.Figure 1 illustrates the traditional Markov model for a networkof two meters. We consider the failure rate of each componentto be constant and we denote it by λ.The number of states for the overall network grows expo-

nentially with the number of meters. Therefore, the traditionalMarkov model soon becomes intractable. We are interestedin assessing the reliability of the overall network, therefore,such a precise model may not be needed. Smart meters in aNeighborhood Area Network (NAN) are likely to behave sim-ilarly. In analyzing the reliability of the whole network, we areinterested in the number of failed meters in a neighborhood,rather than the state of each individual smart meter. Therefore,instead of assuming the network to consist of individual smartmeters, we use a higher level of abstraction, and consider theoverall system to consist of NANs. Thus, each componentof the system in the FMM consists of a NAN. The state ofthe system is represented by the sequence sn1, sn2, · · · , snn,where sni denotes the state of the i-th NAN in the overallnetwork, and there are n NANs in the network.Let us assume each NAN consists of m smart meters and

the whole network includes n NANs. The reliability of a NANis specified by the number of fault-free smart meters in thatNAN. In most cases, it is not important to know which smartmeters have failed, but the number of such meters matters forreliability analysis of the NAN. Therefore, we consider m+1state values per NAN : 0, 1

m ,· · ·,m−1m ,1. In this case a state

value of sni = km , means k out of m smart meters of the i-th

NAN have failed. Since each NAN in our model has a degreeof being failed, we call this model a Fuzzy Markov Model(FMM).

Fig. 2. An FMM for a two-component system

Fig. 3. A graphically simplified FMM for a two-component system

To have an illustration of the above model, let us considera network of smart meters consisting of two NANs. Let usfurther assume that each NAN consists of two smart metersand it fails with a constant failure rate of λ. Constant failurerate is assumed for simplicity only. We later describe how thetransition probabilities are specified using fuzzy time-to-failurevalues per NAN. Each NAN can be in one of the followingstates : (1) 0, meaning it is fault-free (all its smart meters arefault-free), (2) 0.5, meaning it has partially failed, or half of itssmart meters have failed, and (3) 1, meaning it has completelyfailed (all its meters are faulty). Figure 2 illustrates the FMMfor this system. To have a simplified graphical model of thissystem, let each component (NAN) be represented by one axisof the coordinate system. In Figure 3, the first NAN of thesystem is represented by the horizontal axis, while the secondNAN is represented by the vertical axis. The origin of thecoordinate system, i.e., (0, 0) is the starting point where bothcomponents are fault free. The terminal state of the FMM is(1, 1). This FMM consists of nine states.Figure 4 illustrates the Markov model corresponding to a

system of two NANs, where each NAN consists of 20 smartmeters. In this case, the system has a total of 20× 20 = 400states. Note that with a traditional Markov model, the systemwould have 220×2 = 240 states.So far, we have described the states in our FMM and

have assumed that each component has a constant failurerate, denoted by λ which specifies the probability transitionsbetween states. Unlike traditional Markov models where acomponent is either fault-free or failed, i.e., s i = 0 orsi = 1, in our model, the state of each component (NAN)is represented by a number in [0, 1]. Therefore, our modelis capable of analyzing systems with components which failgradually (i.e., degradation). Assuming a constant failure ratefor a component failing gradually is too simplistic. Our goalis define the transition probabilities between states, based onthe values of the states, which capture different failure ratesat different points of time in the life time of a component. Forthat, we need to know the time-to-failure (TTF) value of eachcomponent (NAN). One way to represent gradual failure, isby having a fuzzy TTF value for a component. In fuzzy TTF,each time-to-failure value has a membership degree to the setfailure set. Figure 13 shows a sample fuzzy TTF value for acomponent. In Section IV we will describe how a fuzzy TTFvalue is calculated using the Fuzzy Monte Carlo Simulation(FMCS) approach [?] for each NAN.

Fig. 4. An FMM for a system of two NANs, each consisting of twenty smartmeters

Fig. 5. An example of the fuzzy TTF for a component

For the i-th NAN, let εi = 1mi, where mi is the num-

ber of smart meters in the i-th NAN. Conceptually, ε i isthe maximum failure incremental step for the i-th compo-nent of the system. If an n-component system is in state(sn1, sn2, · · · , snn), the next state will be in an axis paralleln-dimensional cube with the corners (sn1, sn2, · · · , snn) and(sn1 + ε1, sn2 + ε2, · · · , snn + εn). For now, let us assume wehave a fuzzy TTF value for each NAN in the system. We showhow the transition probabilities between states are calculatedusing the components’ fuzzy TTF values. The derivative of thefuzzy TTF value of a component represents its probability fail-ure. Let us assume the system is in state (sn1, sn2, · · · , snn).The probability of failure for the i-th NAN is computed asthe normalized value of the derivative of its fuzzy TTF atfailure degree sni. Let us denote this value by λsni . Then, thetransition probability from state (sn1, sn2, · · · , sni, · · · , snn)to state (sn1, sn2, · · · , sni + εi, · · · , snn) is λsni ×Δt.Now that we have completed describing the FMM for our

system, we can estimate the reliability of the system utilizingthis model. Similar to traditional Markov models, to estimatethe reliability of the system, reliable and unreliable states ofthe Markov model should be determined. Figure 6 illustratesa Markov model for a two-component (NAN) system withεi = 0.01 for all i where each dot can be considered as stateof the system. The coordinates of a dot show its component’sfailure degree. For instance, consider the point (0.45,0.83). Inthis case the first NAN has a failure degree of 0.45, while thesecond NAN has a failure degree of 0.83. The reliable statesin Figure 6 are shown in blue, while the pink area representsthe unreliable states. Figure 6 also shows a trajectory of thesystem during the failure process. A trajectory is defined asthe sequence of states the system passes from the startingstate (i.e., (0,0)) to the terminal state (i.e., (1,1)). We use thefollowing equation to estimate the reliability of the system:

∫time in reliable states dt

∫time in both reliable and unreliable states dt

(1)

The above equation conceptially captures the fraction oftime random trajectory stays in the reliable states, beforeentering in the unreliable states.We use Monte Carlo Simulation (MCS) to analyze the

reliability of the system based on its Markov model. As intraditional MCS, a large number of samples are required. Inan FMM, trajectories are the required samples. By using thetransition probabilities between states, as described above, alarge number of trajectories are produced. The reliability ofthe system is estimated using each trajectory. Finally, we givean estimate of the reliability of the whole system, by averagingover all these reliabilities.

Fig. 6. Fuzzy Markov Model (FMM) for a two-component system

Fig. 7. Sample trajectory for a system with one repairable component

Repairable NANs

So far we have only considered unrepairable NANs.However, we can assume that the NANs are repairable.This assumption introduces other transitions in theMarkov model from a state (sn1, sn2, · · · , sni, · · · , snn) to(sn1, sn2, · · · , 0, · · · , snn), which demonstrates the situationwhere the i-th NAN is repaired after sni percent of its smartmeters have failed. The probability of a repair is calledthe coverage factor and we assume it is a fixed value forevery component of the system. Figure 7 illustrates a sampletrajectory for a system consisting of two NANs, where thefirst NAN is repairable. As shown in this figure, the firstNAN is repaired when the system is in state (0.75, 0.61).The repair causes the system to remain in reliable states fora longer time before entering the unreliable states. Figure8 shows a sample trajectory for a system consisting oftwo repairable NANs. Clearly, a larger portion of trajectoryremains in reliable state.

System with multiple NANs

Our model can be extended to handle systems consisting ofmore than two NANs. The state space of a system of n NANsis an n-dimensional cube. Each axis of the n-dimensional cubecorresponds to one NAN in the system. Figure 9 illustrates asample trajectory for a system with three NANs, where noneof the NANs are repairable (i.e., the coverage factor of all ofthe NANs is zero). Figures 10 and 11 show sample trajectoriesfor the case where only the second component is repairable,and the case where all components are repairable, respectively.

IV. MEASURING THE FUZZY TIME-TO-FAILURE

In this section, we describe how a fuzzy time-to-failure(TTF) value can be calculated for a component (e.g., a NAN)with gradual failure using the Fuzzy Monte Carlo Simulation(FMCS) approach [?]. We show how this approach is appliedto the Weibull probability distribution function (PDF) to re-trieve the fuzzy TTF of a component. The Weibull distributionis widely used in the reliability, availability, maintainability,and safety (RAMS) domain to model failures caused by thedegradation process [?]. The Exponential PDF, another popularfailure probability distribution, is a kind of Weibull distributionwith a constant failure rate (λ). To have a better understanding

Fig. 8. Sample trajectory for a system with two repairable component

Fig. 9. Sample trajectory for a system consisting of three unrepairablecomponents

Fig. 10. Sample trajectory for a system consisting of three components withonly the second component being repairable

Fig. 11. Sample trajectory for a system consisting of three repairablecomponents

of the parameters in the Weibull distribution, we compare theExponential and Weibull distributions:

fe(t) = λe−λt (2)

fw(t) =β

η(t− γ

η)β−1e−( t−γ

η )β

(3)

Equations 2 and 3 present the Exponential and Weibull prob-ability distributions. The failure rate (e.g., λ in the ExponentialPDF) is obtained by special formulas [?] in which ambiguityis disseminated throughout different factors. In our case, acomponent is a NAN and its failure rate can be calculatedbased on factors such as the number of smart meters in thatNAN, their failure rate, and their critical locations. In theExponential distribution, λ corresponds to the failure rate ofa component/subsystem which is considered constant duringits life cycle [?] (useful life period [?]). In contrast to theExponential distribution, Weibull PDF can support simulationof other life periods of a system such as infant and wearout periods with increasing (1 < β) and decreasing failurerate (β < 1) in order. For instance, if β = 2, we willhave a linearly rising failure rate which exactly emulatesRayleigh distribution. In Weibull PDF used in RAMS domain,γ represents the minimum life, η symbolizes the characteristiclife, and β denotes the shape parameter. The failure rate in aWeibull distribution takes the following form:

λ(t) =β

η(t− γ

η)β−1 (4)

Given the Weibull distribution as the failure rate distribution,we can obtain TTF of a component using traditional MonteCarlo simulation. However, because the parameters in theWeibull distribution are usually obtained by experience, eachof them is considered as a fuzzy number to account forthe imprecision inherent in choosing these parameters. Theserandom fuzzy numbers are generated using fuzzy arithmeticand Inverse Transform method [?].Let R denote the reliability function of the component based

on time. At each instant of time t, R(t) is a number in [0, 1].Given the Weibull distribution to model failure, we can obtainR(t) as follows:

R(t) = 1− F (t) = 1− (1− e−( t−γη )β

) = e−( t−γη )β

(5)

where F (t) is the Weibull cumulative distribution function andpresents the quantitative value for cumulative failure at timet.

Fig. 12. A sample Reliability function of a system using the Weibulldistribution with fuzzy parameters

Fig. 13. A sample fuzzy TTF

Similar to traditional Monte Carlo simulation, FMCS relieson F (t) having a uniform distribution.

r = F (t) → r = 1− e−( t−γη )β → t = η[l ln(1− r)]frac1β + γ

(6)r−1 is also a uniformly distributed random number, thereforeit can be replaced to simplify the generator function:

t = η[l ln(R)]frac1β + γ (7)

Employing above equation, t is calculated using fuzzynumbers addition [?] and power function as explained in [?]:

ax̃α = ax̃α = [ax−

α , ax+α ] (8)

For instance, taking β, η, and γ as Gaussian fuzzy numbers[?] with their cores as βc = 3, ηc = 1000, and γc = 0, onesample of the Reliability function 5 of the system is shownin Figure 12. Figure 13 shows a sample fuzzy TTF value,as produced by taking a uniform random number r in [0, 1]and employing Equation 7. The fuzzy TTF of a component iscalculated by repeated application of Equation 7 and averagingover the sample fuzzy TTF values.

V. CASE STUDY WITH THREE-NANS

In this section, we apply our proposed Fuzzy Markov Model(FMM), described in Section III, to estimate the reliability ofa sample smart grid with three NANs. Each NAN consistsof hundred smart meters. We use the Weibull distribution tomodel the time-to-failure value of each NAN. We assumethe parameters in the Weibull distribution, β, η, and γ tobe Gaussian fuzzy numbers with their cores as βc = 3,ηc = 1000, and γc = 0.We have implemented the FMCS and FMM approaches in

JAVA. We use a Celeron(R) 3.2 GHz CPU for performing thesimulations. The final fuzzy TTF of the whole system after5,000 iterations of the FMCS approach with the mentionedWeibull distribution parameters is shown in Figure 14. Thisexperiment took 15,163 seconds. We use this fuzzy TTF tocalculate transition probabilities for the FMM of the systemas described in Section III. We assume the states in the axis-parallel cube with corners (0.9, 0.9, 0.9) and (1.0, 1.0, 1.0)to be the unreliable states of the system. The coverage factorfor each of the NANs was taken as 0.005. The simulationconsisted of 10,000 iterations of the Monte Carlo Simulationto produce several trajectories to estimate the reliability ofthe system. This experiment took 33,784 seconds. Figure 15illustrates a sample of trajectories generated by this simulation.The overall reliability of the whole system is estimated as0.993811.

Fig. 14. Fuzzy TTF for a NAN with the give Weibull distribution parameters

Fig. 15. Sample trajectories for a smart grid consisting of three repairableNANs with 100 smart meters each

VI. CONCLUSIONA fuzzy Markov modeling approach was presented in this

paper in order to study the reliability of AMI. Employing theFuzzy Markov Models (FMM) has two advantages. First, itconsiders both non-cognitive and cognitive uncertainties usingprobability and possibility distributions, respectively. Second,the FMM is capable of modeling the systems with componentsthat fail gradually resulting in performance degradation. In thecontext of an AMI, we presented an approach to leveragethe NAN topology and build a high level Fuzzy MarkovModel (FMM) where failure at the meter level can still beintegrated thank to gradual failure mode. This solution offersboth scalability and accuracy. We applied it to a sample AMIand showed how Fuzzy Monte Carlo Simulation can be usedto measure the time-to-failure (TTF) of a NAN.