AVAILABILITY AND NETWORK CAPACITY

AVAILABILITY AND NETWORK CAPACITY

Jeffery JimesAdvisory Consultant Dell EMC [email protected]

Ricardo HerrmannAdvisory Consultant, Data Scientist Dell EMC [email protected]

Wei LinSr. Manager and Chief Data Scientist – Big DataDell [email protected]

Kiranmai SarabuSenior Forensic Technology AssociatePricewaterhouseCoopers [email protected]

2016 EMC Proven Professional Knowledge Sharing 2

Table of Contents Abstract ....................................................................................................................................................... 3

The Problem .................................................................................................................................................. 4

Proof the Problem Exists ........................................................................................................................... 5

Additional Problems .................................................................................................................................. 8

The Basic Solution ......................................................................................................................................... 8

The Solution ................................................................................................................................................ 13

Conclusion ................................................................................................................................................... 17

References .................................................................................................................................................. 18

Disclaimer: The views, processes or methodologies published in this article are those of the authors.

They do not necessarily reflect Dell EMC’s views, processes or methodologies.


Abstract

Applying analytics to the telecom network can result in significant company savings on facility cost. Our

Return of Investment analysis shows that, given an expected network growth of 18% per year (averaged

from financial reports) and estimated network costs from leased facilities of 8%, by using our routing

method we can expect 4% cost savings from network optimization and a calculated ROI of 101%.

Communication networks are in a state of constant change, not only because they have to grow to keep

up with consumer demand, but also due to constant acquisitions and network merges. Traditionally,

employed methods of engineering and analysis of these changing networks are limited to single

platforms of information, constrained by rules defined in legacy systems and overwhelmed by the

increasing volume of data to analyze.

To remain competitive and keep up with customer demand for bandwidth, telecom companies, or

providers, need to optimize communication routes to reduce operational costs. Under these

circumstances, routing is a challenging problem. The provider must transition to new methods of

determining the lowest cost routes for services.

In this article, we propose a methodology to identify available sections in a network that present cost

saving opportunities for the telecommunications carrier, in a way that is not overwhelmed by choices

that may not be the lowest cost option. The method we describe is based on solving a system of route

equations within a given network, where the total cost of the route is given by the total length of the

communications path multiplied by the rates of the route, avoiding overhead costs from leased wires

where maximum bandwidth is not reached. The main idea is to find routes with the shortest distance

and routes that have lower levels of traffic by looking at time-stamped data points that conform to

bandwidth requirements.

For demonstration purposes, we created a prototype of a network topology and used the proposed

route analytics to find alternate routes between network locations. Due to the challenges involved in

obtaining proprietary and confidential network data, we carried out our experiments on simulated data,

based on summarized frequencies observed in real data sets. Our simulation was generated for 24 hours

of data transmission along three sections in a Synchronous Optical Networki (SONET), assuming a circuit

comprising of three main routes. In our tests, transmission data are assumed to be normally distributed


and sampled at 1 minute intervals for the three routes. The transmission rates range from 0 Mbps to

48.93 Mbps, since we simulated an optical carrier OC-192ii, that can handle 192 Digital Signal 3s (DS3s),

or, equivalently, 129,024 voice calls. We generated 4,320 time stamped data points for three circuit

positions and assume that the three used a total of five leased sections by three different carriers, each

of which has its own cost per mile rate, sampled from realistic carrier rates.

The Problem

Network service providers are challenged with consumer demand for bandwidth and having to expand

their data network at a pace that has never been experienced in the industry. Most of the network

service providers initially were established as telecom providers with business processes based on

telephony models. As telecom providers changed from analog to digital service the demand for data

circuit bandwidth increased between offices. The change from analog to digital was a very linear process

and the business process around these changes accommodates for a linear scale. As Telco providers

establish network routes between offices they lease lines from various providers and must ensure they

manage the cost of these leased lines to maintain profitability. This remains true even with new

technology implementation of software-defined data centers (SDDC) and software-defined networking

(SDN).

Over the last 10 years the demand for data bandwidth has increased in the consumer market. The first

jump in demand was for consumer internet service for web content; the second jump was for cell phone

service, and most recently with the introduction of mobile smart devices with smart apps, causing data

network demand to explode. The network service providers have increased capacity at a rapid pace and

also have most likely acquired many other companies’ infrastructure in order to increase network

footprints. As data network technology has advanced over the years, different protocols, such as ISDN,

ADSL, ATM, SONET, DWDM, etc. have been established. As new technology is introduced, the older

technology remains in place to recover the initial cost of the install or until end of service life or support

ends. This adds to the complexity of managing the network as well as understanding how to optimize

the cost of the network for service providers.

Network providers must implement new methods of thinking to ensure they are expanding the network

most efficiently to ensure they remain competitive. Traditional methods of network route optimization


lack the ability to scale and most lack tools to analyze dimensions of the network such as bandwidth

utilization, cost effective routing, forecasted demand, and route diversity.

Proof the Problem Exists

Network traffic is increasing at an alarming rate as shown by Figure 1. The telecom backbone

infrastructure has to expand at a significant rate to keep up with customer demand for bandwidth. This

increase has a direct impact on the teams provisioning and defining these networks.

iiiFigure 1

The trend of network growth is expected to continue over the next few years. Another indication of

increase can be seen by looking at the data traffic growth in Figure 2. This prediction shows that data

traffic is expected to double in less than 5 years. The data traffic expansion aligns with other models

showing that network utilization is expected to increase.


ivFigure 2

Cisco’s forecast also predicts an increase in network data traffic in the next few years, as shown in Figure

3. The Cisco prediction shows that traffic growth is expected to triple in less than 5 years.

vFigure 3


Network providers will have to increase network capacity to meet the demand of the consumer market

as well as increase bandwidth between network locations. Capacity between network locations can be

increased by adding to the carrier’s own facilities or via adding more capacity by leasing facilities. The

most efficient cost is utilizing owned facilities vs. leasing facilities from other providers. It may be that

the only option available is through a leased option.

The number of network providers is constantly changing with acquisitions and companies turning up

fiber to interconnect cities and central offices. The volume of data can be overwhelming when

attempting to determine the lowest cost routing. Figure 4 represents a sample of backbone networks

available to service providers.

viFigure 4

As shown from multiple sources this data usage and bandwidth pressure will continue to increase from

the expected growth in consumer demand.


Additional Problems

For network service providers to remain competitive, they must constantly continue to change as new

technology is introduced to the market. As more providers implement newer technologies these may

result in price decreases to existing service providers resulting in lower leased cost. However, without

analysis of existing network and route optimization the provider will not realize the benefits of potential

cost savings. The service provider must shift to new methods of analyzing multiple dimensions of the

data network to remain competitive with other service providers.

The Basic Solution

In the context of Networks and Network Traffic Data, it is important to understand the major

networking protocols used by the industry. Among the main protocols and architectures, we can refer to

three of them:

1. OSI Seven Layer Model

2. SONET Synchronous Optical Networks

3. Optical Carrier Transmission Rates

Most vendors involved in telecommunications make an attempt to describe their products and services

in relation to the OSI modelvii. Although useful for guiding discussion and evaluation, OSI is rarely

actually implemented as-is, as few network products or standard tools keep all the related functions

together in the same kind of the model’s well-defined layers, sometimes combining some of them. The

main concept of OSI is that the process of communication between two endpoints in a

telecommunication network can be divided into seven distinct groups of related functions, or layers.

Each communicating user or program is at a computer that can provide those seven layers of function.

In a given message between users, there will be a flow of data going down through the layers in the

source computer, across the network and then up through the layers in the receiving computer. These

seven layers are provided by a combination of applications, operating systems, network card device

drivers and networking hardware that enable a system to put a signal on a network cable or out over

Wi-Fi (or similar wireless protocols). The combination of a multitude of technologies in each layer, which

may be incompatible in some cases, provides additional challenges to finding good paths in multi-layer

networksviii.


SONET defines optical signals and a synchronous frame structure for multiplexed digital traffic. It is a set

of standards that defines the rates and formats for optical networks which follow the OSI models.

SONETs are made up of sections, lines, and paths. A section relates to a single fiber cable that can be

terminated by a network element (Line or Path) or an optical regenerator. The main function of the

section layer is to properly format the SONET frames, and to convert the electrical signals to optical

signals. Section Terminating Equipment (STE) can originate access, modify, or terminate the section

header overhead. (A standard STS-1 frame is nine rows by 90 bytes. The first three bytes of each row

comprise the Section and Line header overhead). Line-Terminating Equipment (LTE) originates or

terminates one or more sections of a line signal. The LTE does the synchronization and multiplexing of

information on SONET frames. Path-Terminating Equipment (PTE) interfaces non-SONET equipment to

the SONET network. At this layer, the payload is mapped back and forth into the SONET frame. SONET is

very widely deployed in the telco space, and is frequently used in a ring configuration with link failover

provided by cyclesix.

An OC-192 is an example of an Optical Carrier (OC) within a SONET ring, that can handle 192 DS3s. A DS3

is a 45 Mbits/s signal that can carry internet data or it can carry 28 T1s, voice or data, and 28 T1s can

handle 672 phone calls. In other words, an OC192 is a fiber optic system that can carry 192 DS3's, which

has 5376 T1s or, equivalently, handles 129,024 voice calls. An OC-192 has 192 STS-1 channels. The

payload bandwidth for one channel (with available capacity to sell) is 50,113 kbits/s or 50,113/1024 =

48.93 Mbits/s per channel. Thus, its maximum capacity is 48.93 Mbits/s.

With these concepts in place, we can detail our approach to simulation of these kinds of networks. Our

simulation was generated for 24 hours of data transmission along three sections in a SONET. Figure 5

depicts portions of our simulated network.


Figure 5: Portion of the simulated SONET we used in our tests

The circled blue area in Figure 5: Portion of the simulated SONET we used in our tests represents one

circuit in the SONET. In our simulation, we assume the circuit is comprised of three main routes (shown

in red). Using a normal distribution, we generate 24 hours of transmission data at 1 minute intervals for

the three identified routes. The transmission rates range from 0 to 48.93Mbits/s, as we are simulating

an OC-192. We generate 4,320 time stamped data points for three circuit positions. Figure 6 depicts a

sample of the generated data.

Figure 6: Sample rows of the results produced by the simulation


The next set of numerical information we needed to simulate was the carrier cost per mile per month.

This cost represents the cost of sending data to the end nodes in a network in which the network route

utilizes leased sections from other carriers. Each carrier will have their own cost per mile rate,

depending on the investments they have made in that area. The FCC primarily regulates prices that

different carriers charge each other for intrastate and international communication. The FCC also closely

monitors and imposes fines on telecom companies employing any discriminatory pricing policies to

charge consumers and businesses. According to studies from US Telecomx, the telecommunication

companies in the US invested $1.1 trillion since 1996, where these costs come mainly from the network

infrastructure to provide fixed line and wireless services, thus making telecom very capital-intensive.

Due to these high entry costs, there are few entrants in the industry. The profitability in the telecom

industry is highly dependent on the number of customers, due to its high operating leverage, so the

focus of these companies is on customer acquisition and retention. In the context of wireless networks,

the cost is mostly based on network traffic, unlike in wired networks, where the cost is proportional to

the number of customers. This explains why wireless companies have to make more continuous

investments on upgrades to the network's capacity as data and voice traffic increases.

We assumed that the three routes we were analyzing used a total of five leased sections by three

different carriers. Using samples of realistic carrier rates we set the cost per mile data to be the

following table:

Carrier Cost per mile per month

CARRIER 1 $0.540

CARRIER 2 $1.422

CARRIER 3 $3.366

The last set of data that we needed to simulate was a conceptual sample of SONET topology. This data

set was used to help us understand the structures of routes available in a real SONET network. Figure 7

depicts a SONET topology.


Figure 7: a SONET topology

Using the three simulated data sources, we propose a methodology to identify other available sections

in a network for cost saving opportunities for a telecommunications carrier.


The Solution

In this section, we propose a new routing algorithm that considers additional costs imposed by available

leased communication lines. The implementation of the Telecommunications Act of 1996 entails

measurement of the costs of supplying telecommunications services for at least two distinct purposes:

1. Pricing unbundled network elements and interconnection

2. Quantifying the subsidy contained in current prices for local exchange services

The Federal Communications Commission (FCC, 1997) has specified that these costs are to represent

forward-looking efficient costs – that is, the costs that would be incurred by a carrier that provided

these elements and services using least-cost methods at today’s prices and technology.

For the purpose of finding cheapest routes, we need to be able to compute shortest paths in the cost-

based communication graph. Using shortest-path queries based on Dijkstra’s shortest paths algorithm

we can identify the shortest path between two end points in this graph.

Given a source node s and a destination node t in a network, the objective is to find the path P for all

feasible paths P between points s and t. This algorithm has two main features: keep track of the

adaptations along the route and multiple paths may be stored at a single node (similar to a k-shortest

paths algorithm). Instead of working with a queue of nodes, as in the breadth-first-search algorithm and

Dijkstra’s algorithm, in our proposed algorithm we maintain a queue of paths. Basically, this algorithm

simply tries all possible paths P in order of their lengths, even those with loops or already visited nodes.

Each path object p in queue Q contains the following properties:

1. A sequence of edges

2. The last visited vertex

3. The technology stack

4. The list of used bandwidths for every edge

5. The distance between edges

The base of the algorithm is a queue Q of path objects. The algorithm begins with a path starting at node

s, and without an adaptation in the technology stack. The main loop takes the path with the shortest

length from the queue, and extends it by one hop in all directions. The next step is to check if the


shortest path ends at the destination, and if so, returns that path as the result of the algorithm. Not all

extensions of a path with one hop will result in a feasible path.

When a leased wire in a network reaches higher rates of traffic the original network carrier incurs a cost

for high use of that section. In an attempt to find routes that minimize costs to the carrier by avoiding

leased sections we propose the algorithm described below to identify alternate routes when leased

network sections reaches maximum transmit capacity:

1. Flagging leased wires within a network that reach max bandwidth capacity

Figure 8

In our sample data set we know the maximum capacity of the particular OC-192 carrier is 48.93

Mbits/s. Using transmission data, we identify the times and the wires that reach the maximum

capacity.

2. Calculate length of all wires in routes containing leased wires

Using the geo-coded data of the circuit position data, we calculate the distance between

endpoints to retrieve the length of the wire.

3. Sum lengths for all wires

Summing the lengths of each section between endpoints results in the total length of the route.


4. Multiply sum of lengths by cost per mile rate (given information); this information tells us the

total cost of the route

The total cost of the route is given by the total length multiplied by the rates of the route.

5. Solve a system of route equations within a given network

Cost per mile * (lengthseg1Route1netZ) + Cost per mile * (lengthseg2Route1netZ) = distance of

Route1netZ * cost per mile at non-leased rate.

In Figure 9 we identify two end points and three routes between them that have a traffic flow. Using a

shortest path algorithm as described above, it is clear that route 1 is the shortest route using two leased

lines. Route 3 is the longest and route 2 is not as long as route 3, but it’s longer than route 1 and uses

two leased lines.

We know the distance between the end points of the route is 1,500 miles (the length of the shortest

route).

Figure 9


We also know the cost per mile of the carrier lines is $1.42 and the cost of the leased lines is $3.46. The

minimum cost of a route between the two end points is $2,130. Solving the system of equations for wire

lengths we can identify the maximum distance a carrier should allocate to a leased wire to keep their

costs to a minimum.

In this example the leased wires of route 1 should have a length no more than 106 miles to stay at the

minimum cost. However, route 3 contains no leased wires and, although it is slightly longer, it can be

utilized as an alternate route. Originally. route 1 costs were $3,690 (with two leased lines). Finding an

alternate route resulted in the minimum cost of $2,130 with cost saving of 42%. The idea is to match the

optimal wire lengths with available wires in an alternate route.

$ per

mile

Wire 1

Length

$ per

mile

Wire 2

Length

$ per

mile

Wire 3

Length

$ per

mile

Wire 4

Length

$ per

mile

Wire 5

Length

$ per

mile

Wire 6

Length

Shortest

Route *

own rate Total Miles

Route 1

1.42

(O) 540

1.42

(O) 540

3.46

(L) 106

3.46

(L) 106 NA

NA

$2,130 1500

Route 2

1.42

(O) 250

1.42

(O) 250 1.42(O) 250

3.46

(L) 205

1.42

(O) 250

3.46

(L) 205 $2,130 2777

Route 3

1.42

(O) 375

1.42

(O) 375

1.42

(O) 375

1.42

(O) 375 NA

NA

$2,130 1700


Conclusion

In this article we considered the problem of optimizing the cost associated to network traffic in multi-

company networks while lines from other companies with available excess bandwidth can be leased.

While still remaining a challenging problem in large-scale settings, we provided an algorithm for finding

routes that minimize the cost of routes that consider the use of the available lines for leasing. Due to the

confidentiality of real network data and in order to protect clients’ privacy, we created simulated

models that reflect the real structure and behavior of communication networks. We have a high level of

confidence that similar savings can be achieved by employing the same technique to real-world

datasets. Moreover, due to the fact that these networks cannot have their physical route moved

without planning, designs, installation cost, and coordination between vendors. Our method can be

employed by calculation of batches of routes whenever the topology or an associated cost changes. The

result is daily recommendations for lowest cost routing and additional details immediately available to

the design team turning up new services. These insights allow companies to remain competitive in this

rapidly changing and competitive market.


References

ihttp://www.cisco.com/c/en/us/support/docs/optical/synchronous-optical-network-sonet/13567-sonet-

tech-tips.html

iihttp://www.incontact.com/blog/just-how-big-oc192/

iiiFigure 1 : http://www.slideshare.net/zahidtg/super-cell-sk-telecoms-network-evolution-vision

ivFigure 2: https://www.linkedin.com/pulse/technology-trends-telecommunications-industry-beyond-

2015-kurian

vFigure 3: http://dupress.com/articles/rising-tide-exploring-pathways-to-growth-in-the-mobile-

semiconductor-industry/

viFigure 4: http://www.clarendon.tv/_images/207-E1B9DD05.jpg

vii http://searchnetworking.techtarget.com/definition/OSI

viii https://www.nas.ewi.tudelft.nl/people/Fernando/papers/Kuipers_2009_Computer-

Communications.pdf

ix http://dimacs.rutgers.edu/~tcar/97-02.pdf

x http://marketrealist.com/2015/01/key-costs-wireless-wired-telecom/

http://www.cisco.com/c/en/us/support/docs/optical/synchronous-optical-network-sonet/13567-sonet-tech-tips.html

http://www.cisco.com/c/en/us/support/docs/optical/synchronous-optical-network-sonet/13567-sonet-tech-tips.html

http://www.incontact.com/blog/just-how-big-oc192/

http://www.slideshare.net/zahidtg/super-cell-sk-telecoms-network-evolution-vision

http://www.slideshare.net/zahidtg/super-cell-sk-telecoms-network-evolution-vision

https://www.linkedin.com/pulse/technology-trends-telecommunications-industry-beyond-2015-kurian

https://www.linkedin.com/pulse/technology-trends-telecommunications-industry-beyond-2015-kurian

http://dupress.com/articles/rising-tide-exploring-pathways-to-growth-in-the-mobile-semiconductor-industry/



http://www.clarendon.tv/_images/207-E1B9DD05.jpg

http://www.clarendon.tv/_images/207-E1B9DD05.jpg

http://searchnetworking.techtarget.com/definition/OSI

https://www.nas.ewi.tudelft.nl/people/Fernando/papers/Kuipers_2009_Computer-Communications.pdf

https://www.nas.ewi.tudelft.nl/people/Fernando/papers/Kuipers_2009_Computer-Communications.pdf

http://dimacs.rutgers.edu/~tcar/97-02.pdf

http://marketrealist.com/2015/01/key-costs-wireless-wired-telecom/


Dell EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL EMC MAKES NO

RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS

PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS

FOR A PARTICULAR PURPOSE.

Use, copying and distribution of any Dell EMC software described in this publication requires an

applicable software license.

Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries.

Documents

AVAILABILITY AND NETWORK CAPACITY