Analytic Model and Simulation Study for Network Scalability in Smart Utility Networks

Gopalakrishnan Iyer, Prathima Agrawal Dept. of Electrical and Computer Engineering

Auburn University, Auburn, AL - USA gbi0001@tigermail.auburn.edu, pagrawal@eng.auburn.edu

Ruben Salazar Cardozo Landis+Gyr, North America

Alpharetta, GA - USA ruben.salazar@landisgyr.com

Abstract— The rapid deployment of smart meters across North America to measure consumption of utility resources such as water, gas and electricity has presented a problem of scale. The huge amount of data thus collected is being used to make demand-response applications smarter as a part of the Smart Grid initiative. The wireless mesh architecture is a popular deployment method for smart utility networks and Advanced Metering Infrastructure (AMI), due to the low deployment costs offered by this method. Severe limitations are imposed on the scale of deployment by wireless environment parameters such as fading and path loss, differing widely from home and outdoor to industrial and in-building scenarios. Reliability in such networks is strongly affected by deployment scale and vice-versa. In this paper, we propose an analytic model for smart utility network scale as a function of link reliability and aim to deduce a relationship between network size and average link packet success probability for scales of deployment. We verify this result by simulating a routing protocol in large scale deployments and comparing with results from analytic model.

I. INTRODUCTION Smart meters are measurement devices for utility resource

consumption over a period of time or in units specified by the utility provider. These devices are deployed in large scale to measure data at a higher granular level in order to closely analyze a particular consumption metric. The value in deploying such devices in a large scale collection of data is in smarter demand-response applications. Reliable collection of data in such networks is greatly impacted by the collection method [1]. Advanced Metering Infrastructure (AMI) comprising of smart meters, collectors, transmission and distribution devices and automated aggregation systems are now being widely deployed as a part of the Smart Grid initiative [2]. Over 36 million smart meter devices have been deployed as of 2012 in the United States alone. This is a 33% increase over 2011 deployments and is projected to be at 65 million by 2015 [3]. Fig. 1 shows the current electric smart meter deployment and projected scale for future deployments. In a vast majority of such deployments, devices in the AMI have wireless interfaces for communication to collector devices. The wireless mesh architecture used for deployment provides a plethora of advantages over other methods, primarily in deployment costs. In addition to this, smart utility

networks are characterized by harsh wireless environment conditions in which they are deployed. Industrial and outdoor

Figure 1. Smart Meters deployment scale [3].

environments suffer from greater path loss, shadow fading and noise from industrial machines. Wireless fading and path loss have spatial and temporal components which require complex models to study performance in such environments. The enormous scale of the network also makes scalable solutions an important part of achieving efficiency in such networks. A typical smart meter network can range from a few thousand metering nodes reporting to a single collector to millions of metering end points reporting to the head-end systems. These networks can be deployed using a plethora of evolved communication technologies that are available today. Physical layer technologies such as wireless cellular, power-line communications, fiber-optical networks and wireless mesh networks are under consideration for deploying AMI.

In addition, the usage of license free spectrum for deploying wireless mesh networks is highly lucrative in terms of deployment and operating costs. The goal of deploying smart utility networks is for the utility to monitor resource consumption at a highly granular level and detect and correct imbalances in the service proactively. In addition, utility pricing information can be sent to these connected devices to

IEEE ISGT Asia 2013 1569818161

1

allow consumers to modify their consumption pattern in synchronization with peak and off-peak demand based pricing. Google’s now retired PowerMeter initiative was one such device which provided the consumer with timely and decision making information about temporal fluctuations in consumption and time-of-day pricing [4]. In addition to demand response applications, utilities deploy such networks for continuous monitoring of the health of transmission and distribution lines using distribution automation devices such as Phasor Measurement Units (PMUs) to measure and analyze high-resolution data which is further processed by Supervisory Control and Data Acquisition (SCADA) systems to make critical decisions on re-routing resources to avoid cascading failures [5]. It is hence evident that an efficiently deployed smart utility network of scale is vital to several key applications at the utility hierarchy. An efficient data routing protocol, which is agnostic to lower layer technologies is imperative to the efficiency and reliability of the smart utility network in general. In addition, inter-operability requirement is inherent to all devices in the AMI and must be the same with a routing protocol used in such a deployment. One such IPv6 based routing protocols developed by the IETF (Internet Engineering Task Force) community, RPL (Routing Protocol for Low power and lossy networks) [6], [7] has been studied herein for scalability.

We develop an analytic model of a smart utility network deployed according to a Poisson spatial distribution model. In this model, we aim to determine a relationship between network size and maximum packet success expectation in the network. Furthermore, we develop the model according to various deployment densities and device transmission ranges.

In this paper, section II details the analytic model and analytic curves resulting from the model. In section III we present a brief validation of the simulation model and an overview of the routing protocol used for scalability analysis in large scales of deployment. Section IV presents the results obtained from the simulation experiment. Section V presents future work and concludes this paper.

II. ANALYTIC MODEL Scalability of routing protocols is an important problem in

wireless mesh networks [8], [9]. The large scale deployment of smart utility networks poses challenges in determining the maximum capacity and scale which each collector or root node can support [10]. Very little field deployment data is available on the maximum number of nodes supported by each collector. We statistically model the smart utility network in order to determine a theoretical limit on the maximum number of nodes supported by each collector. In addition, we verify these results with extensive simulations of large scale networks. The model is based on the idea that the network size is limited by the packet success probability of each link in the mesh network.

A. Poisson Spatial Distribution Model In this approach, we model the smart utility network as

spatially distributed according to a Poisson distribution. Fig. 2 shows the smart utility network comprised of a collector at the center of a geographical area of radius R. Each node in the network has coverage range of radius r. This radius r is

derived from eq. (1) using wireless channel and environment parameters. ( ) = − ( , + 5 ) − 10 log( ) (1)

Figure 2. Smart Utility Network spatial distribution model.

Here, , , are radio transmission power, receiver sensitivity and noise floor respectively. is the path loss exponent and varies according to prevalent wireless environment. ( ) is the path loss at reference distance of 1 meter and is set to 60 dB, derived empirically. The transmission radius is dependent on the radio transmission power and the wireless channel characteristics. Since the wireless channel model has been validated by comparing with real world experiments in [11], the r thus calculated provides realistic values for the transmission range of each node in the smart utility network. Let the total number of nodes in the network be N. Typically, the collector node has a higher transmission power than the participating nodes in real deployments. For the sake of simplicity in this analysis, we assume that all the nodes including the collector transmit at the same power, thus having the uniform transmission range r. In addition we define the network density parameter ρ calculated by eq. (2). We define parameter ρd as the deployment density and can vary for densely (urban) and sparsely (rural) deployed smart utility networks. = [ ] = . .. . = (2)

B. Connectivity Model

The nodes have connectivity to each other and to the collector node defined by connectivity degree k by a Poisson distribution whose parameter is nf. Here, nf is the number of forwarding neighbors available to each node at the kth hop. Thus the probability of finding a node at kth ring from the collector is given by eq. (3). Consequently, the probability and expected number of nodes that are detached from the network is at k=0, given by equations (4) and (5). Parameter α in (5) is a weight that determines number of forwarding neighbors in the given area. ; = = . ! (3)

2

= = 0; = (4) [ ] = = ; α ∈ (0,1] (5)

Let the packet success probability for each link be u. The probability of a successful transmission over n hops is give by eq. (6). The expectation of successful transmission of packets over k hops according to the Poisson spatial distribution and derived connectivity model in (2) is given by eq. (7). Here, the parameter ncell is the number of forwarding neighbors found in the ring of the kth hop. = (6)

= ∑ . ! ( ) (7)

Here, the expected number of successful packets over k hops, corresponding to a total of n connected neighbors for a given number of nodes in a cell ncell is given by (7). In order to maximize the expected number of successful transmissions, with average link packet success probability as u, we find the limit of (7) as n approaches infinity. Number of forwarding neighbors ncell is given as a function of deployment density and network size by eq. (8). This will provide us with an upper limit on the network size at which the expected number of successful transmissions will be maximized. Furthermore, we differentiate the limited expression with respect to N (total network size), to arrive at a relationship between network size N and average link packet success probability u. This is the value of N at which we can expect maximum number of successful transmissions in the network, given average link packet success probability u. = (8)

Limit of E[pf] as n → ∞ yields,

lim→ E[p ] = lim → . ! ( )

= ( − 1) (9)

To find maximum E[pf], we differentiate eq. (9) with respect to N given by eq. (10).

, = 0 (10)

This gives us a value for N at which E[pf] in (9) is maximum, in terms of the packet success probability defined by eq. (11).

= ; [ ] (11)

C. Results from Analytic Model Thus, we have a relationship between network size N and

average link packet success probability u for maximizing the expected number of successfully transmitted packets. Fig. 3 shows the relationship between average link packet success probability (u) and maximum expected successful transmissions according to eq. (12). Expression (12) is derived by substituting (11) in (9) to give the maximum E[pf] for a given u. = . − 1 (12)

Figure 3. Maximum E[pf] vs. u

Thus, Fig. 3 gives a general relationship, allowing for geographical size R and transmission range r as variable parameters to determine the value of u at which we can obtain a maximum expected successful packet transmission for network size N as a function of average link packet success probability (u). Further, we can determine the scalability in smart utility networks by simulating different network sizes and determine the average link packet success probability yielded by each network size N. Figures 4 and 5 show the various theoretical values of N as a function of the link packet success probability as calculated using (11). This is the value of N which will yield the maximum E[pf]. In Fig. 4, we vary the transmission range r from r=100m to r=450m in steps of 50m and obtain the corresponding values of N against the link packet success probability u. In practice, this can be done by varying the transmission power of the node. In Fig. 5, we vary the geographical area R, keeping the transmission range r constant at r=450m. This yields a new set of values of N against u and by utilizing this data, we can calculate N for maximum expected successful packet transmissions from Fig. 3. In order to verify these theoretical results, we ran simulation scenarios with transmission range set at r=100m and geographical area R=3536m. Average link success probabilities (u) were calculated for varying network size using the same radio and wireless environments using (1).

3

Figure 4. N vs. u for varying transmission range r

Figure 5. N vs. u for varying geographical size R

III. SIMULATION MODEL AND VALIDATION A. Simulator and Validation

The discrete event simulator OMNeT++ along with the wireless sensor network framework Castalia was used to develop the simulation model. The 802.15.4 MAC and PHY modules provided with the Castalia framework were used as lower layers under the routing module of RPL routing. The module was developed by us with the routing protocol draft and related documentation as guide [12]. In order to validate the analytic model, we tested RPL for scalability. Simulation runs were performed for varying network sizes starting from 25 nodes up to 7500 nodes. In each case there was only 1 collector node. The spatial distribution of the network was populated according the poisson model, as assumed in the analytic model. Fig. 6 shows the spatial distribution of one such network comprising of 7500 nodes and a single collector at the center of the grid. Although network size was varied, the density was kept constant in order to obtain consistent results.

Prior to the simulation runs, we validated the lower layer simulation models of 802.15.4 MAC and PHY by comparing

the RSSI (Received Signal Strength Index) of a 9 node simulated network versus a 9 node network comprised of real world Tmote Sky sensor motes for a duration of 24 hours. The Tmote sky wireless sensor devices use the TI CC2420 radio, which is in compliance with IEEE 802.15.4 MAC and PHY standards [13]. Fig. 7 shows the results from the validation. We concluded that the simulation model is in close agreement with the real world wireless sensor network in terms of performance in wireless channel conditions assumed in (1).

Figure 6. Spatial distribution of 7500 node network

Figure 7. RSSI: Real vs. Simulated for 24 hr period

B. Routing Protocol RPL In order for the smart utility network to function as

designed for its application, an efficient routing protocol is imperative. A single routing protocol is designed with target applications, according to which the routing algorithm optimizes routes and controls network functions. The requirements for routing in a smart utility network are diverse, as it has to be compatible for a wide range of devices, from home appliances to metering devices to collectors. Routing Protocols designed for the smart utility networks have to adhere to the requirements spelled out in [RFC5826], [RFC5673], [RFC5548] and [RFC5867] which are home automation routing, industrial routing, urban routing

4

environments and in-building routing requirements respectively. In [11] we have shown that the performance RPL in terms of reliability is better than other popular routing protocols deployed in smart utility networks. Here, we simulate RPL for large scale smart utility networks to analyze its scalability performance. We aim to study if the RPL routing follows the scalability trends predicted by the model in section II as a function of the average link packet success probability.

RPL is based on construction of a DODAG (Destination Oriented Directed Acyclic Graph) for the network which has the collector or data aggregating device as the root of the directed acyclic graph. It is a distance vector routing protocol for Low Power and Lossy Networks (LLNs). The route is constructed with a goal of optimizing an objective-function. The objective function is constructed using a combination of metrics and constraints to compute the best path for routing data packets to the destination. Thus, there can be multiple DODAGs for the same network depending on the objective-function. Applications such as outage reporting may require hard deadlines on delivery and can use an objective function to minimize latency as an example. Network state information such as connectivity and degree of connectivity of each node to its parent node up to the root is maintained and periodically updated using the RPL control plane. The root node initiates formation of the graph using the DODAG Information Object (DIO) control message. Consider node j receiving a DIO (DODAG Information Object) from node i. Upon receiving the DIO message the jth node will compute its own rank Rj according to eq. (13) and determine its position in the DODAG with respect to other nodes. Parameter δ defines the granularity of the network topology. It is defined as minimum rank increase and can be adjusted as required during deployment. δ varies depending on the network density. For a sparse network, β is small and for a dense network δ is large. Fig. 8 illustrates a RPL instance after formation of the DODAG. DIOs are used to form up-routes for MP2P (Multipoint to Point) traffic. = (13)

RPL also creates down-routes for P2MP (Point to Multipoint) traffic. DAO (Destination Advertisement Object) messages are periodically transmitted by the root requesting reverse route to a destination node in the tree. DAO messages are used to advertise prefix reach-ability towards the leaf nodes in support of the down traffic. RPL forms acyclic graphs and inherently does not allow nodes at the same hierarchy level to form an edge. A modification in RPL supports point-to-point (P2P) traffic instead of data packets traversing all the way up to the root and then down-route. RPL makes use of the trickle-timer [14] in order to control flooding at the control plane. Trickle timer governs the DIO conditions and DAO intervals. Flooding of control packets in smart utility networks would be particularly costly where node resources such as energy and computing power are scarce and highly constrained. The adaptive trickle-timer is crucial to the efficiency of RPL. As the routes form and the nodes are connected (a rank is established) the network stabilizes and control plane messages decrease. When an inconsistency

arises (such as a loop or a change in the DODAG parameters) as local repair is initiated by the node which detected the inconsistency, the trickle timer is reset and DIOs are exchanged to quickly restore connectivity. In the worst case, a global repair is initiated where the graph is built starting at the root. The metric used to make the rank change decision is Expected Transmission Count (ETX). It is calculated using the link packet success probability. Packet success probability of each link k, denoted pk is randomly assigned for each link to a parent. The metric ETX of link k, X (k), is computed as the inverse of the packet success probability of each link according to eq. (14).

( ) = (14)

The initial value of ETX of any link k is set to be 1.0. The ETX is updated and calculated using (14) from a randomized set of packet success probability values when a DIO is received by a node from a parent. Hence, the ETX value of each link keeps changing over time. This introduces the temporal nature of link characteristics observed in Low power and Lossy Networks (LLNs). The rank is computed based on the updated ETX count and is transmitted in the DIO. Figure 2.4 shows an instance of the Directed Acyclic Graph of RPL after DIO control packets exchange. Node 1 is the root of the graph with the least rank and node 5 is the leaf node with the maximum rank.

Figure 8. Instance of RPL after DODAG formation

IV. SCALABILITY RESULTS AND CHALLENGES RPL was simulated for large scale smart utility networks

of maximum size 7500 nodes. The transmission range was set at r=100 m and geographical area at R=3536 m. For each network size, the average link packet success probability was recorded. We plotted this data against the analytic results of section II-C. Fig. 9 shows the maximum E[pf] versus u for varying network sizes N for both analytically calculated and RPL simulated. The values are scaled to fit in a multiple y-axis grid. The scaling factors for maximum E[pf] (left y-axis) is the value 5 and for the Supported nodes N (right y-axis) is ρ, given by (2). The graph can be read and interpreted by choosing a point on the calculated or simulated curves, projecting it on the x-axis for a value of u. Further, we can project this value of u onto the maximum E[pf] curve to get a maximum expected successful packet transmission for the network size N. Using this graph, we can obtain the maximum expected successful transmissions in a given network of size N running RPL as the routing protocol for a prevalent average

5

link packet success probability. An example illustrated in Fig. 9 for u=0.7, the RPL simulated curve yields a network size of N=1930 nodes. The corresponding maximum expected successful packet transmissions E[pf] is 41.78%. The simulated results were obtained for a maximum network size of 7500 nodes before terminating.

Figure 9. Calculated and Simulated N and E[pf] vs. u (Scaled)

Simulating networks of larger scale presents significant challenges in terms of simulation time and computation power. Fig. 10 shows the time taken for simulating networks of various sizes. The simulation time increases exponentially with increase in network size and this presents a significant challenge in simulating networks of larger sizes.

Figure 10. Simulated Network size versus Simulation Execution Time

V. CONCLUSION AND FUTURE WORK We have developed an analytic model for smart utility

networks based on a Poisson distributed spatial and connectivity model. Further, we have developed a relationship between network size and maximum expected successful packet transmissions as a function of the average

link packet success probability. In order to verify the model, we simulated a large scale smart utility network using the well known routing protocol RPL to study its scalability. Through validation using real-world hardware, the obtained link packet success probabilities in the simulated network accurately reflects the effect of real-world wireless channel and wireless environment characteristics. The data from the RPL simulation results shows that RPL is a scalable routing protocol and maintains fair packet success probabilities and maximum expected successful transmissions in the smart utility network.

In future, we will simulate other popular routing protocols being deployed or are in consideration for smart utility networks such as LOAD (6LoWPAN Ad Hoc On-Demand Distance Vector Routing) [15], [16] and Geographic routing [17] protocols and compare them with RPL for scalability. In addition, we aim to tweak the simulator to allow simulations of larger scales. With further simulations the analytical model can be validated to arrive at an experimental upper bound on capacity of each collector node in large scale smart utility networks.

REFERENCES [1] Ian. F. Akyildiz and Xudong Wang, “A Survey on Wireless Mesh

Networks," IEEE Communications Magazine, vol. 43, Sept. 2005. [2] “NIST Framework and Roadmap for Smart Grid Interoperability

Standards,” US Dept. of Commerce, Release 1.0 [3] “Utility Scale Smart Meter Deployments, Plans and Proposals,”

Institute for Electric Efficiency, May 2012. [4] Google PowerMeter, http://www.google.com/powermeter/about/ [5] Carl H. Hauser; David E. Bakken; Anjan Bose, “A Failure to

Communicate,” Power and Energy Magazine, IEEE, pp. 47-55, Mar-Apr, 2005.

[6] T.Winter; P. Thubert, “RPL: IPv6 Routing Protocol for Low Power and Lossy Networks,” draft-ietf-roll-rpl-19

[7] T. Watteyne; K.Pister; D. Barthel; M. Dohler; I. Auge-Blum, “Implementation of Gradient Routing in Wireless Sensor Networks,” IEEE GLOBECOM, Hawaii, December 2009.

[8] J. Jun and M.L. Sichitiu, “The nominal capacity of wireless mesh networks,” Wireless Communications Magazine, Vol. 10, no. 5, pp. 8-14, Oct. 2003.

[9] P. Gupta and P.R. Kumar, “The capacity of wireless networks,” IEEE Trans. On Information Theory, vol. 46, no. 2, pp. 388-404, Mar’ 2000.

[10] J. Li, C. Blake, D.S.J. De Couto, H.I. Lee, R. Morris, “Capacity of Ad-hoc wireless networks,” Proceedings of ACM MobiCom, July, 2001.

[11] Iyer G.,Agrawal P., Monnerie E.,Cardozo R., “Performance Analysis of Wireless Mesh Routing Protocols for Smart Utility Networks,” IEEE SmartGridComm, Brussels, Oct 2011.

[12] F. Varga A., et.al. , “An overview of the OMNeT++ simulation environment,” ICST, Brussels, Belgium.

[13] Instruments, Texas. "CC2420 datasheet." 2007-05. http://focus. ti. com. cn/cn/lit/ds/symlink/cc24 20. pdf (2007).

[14] Levis P., et.al, “The Trickle Algorithm,” IETF RFC 6206. [15] Kim K., Daniel S., et. al., “6lowpan Ad-hoc On-demand Distance

Vector Routing Protocol (LOAD),” draft-daniel-6lowpan-load-adhoc-routing-03

[16] Ian. Perkins C., et.al, “Ad hoc On-Demand Distance Vector (AODV) Routing,” IETF RFC 3561

[17] B. Lichtensteiger; B. Bjelajac; C. Müller and C. Wietfeld, “RF Mesh Systems for Smart Metering: System Architecture and Performance,” IEEE SmartGridComm, Maryland, October 2010.

E[pf] =2.089/5 = 0.4178

N = 1.544/ρ = 1930

6