Internet Inter-Domain Traffic Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian Presented by: Kaushik Choudhary

Internet Inter-Domain TrafficCraig Labovitz, Scott Iekel-Johnson,

Danny McPherson, Jon Oberheide, Farnam Jahanian

Presented by: Kaushik Choudhary

Outline

• Introduction• Data Collection Methodology• ASN Traffic Analysis• Application Traffic Analysis• Internet Size Estimates• Conclusion

Introduction

• “The Internet has changed dramatically over the last five years” – Cliché.

• Internet traffic has gone over the roof due to:

The changes

• Content providers (like Google) build their own global backbones.

• Cable internet service providers (ISPs) offer wholesale national transit.

• Transit ISPs offer content distribution networks (CDNs).

The changes

Transition from:

Fig 1: Traditional Internet logical topology.

The changes

to:

Fig 2: Emerging new Internet logical topology.

What is new in this paper

• Most studies about traffic have focused on BGP route advertisements, DNS probing, industry surveys, private CDN statistics etc.

• In this paper, the authors studied over 3000 peering edge routers of 110 participating internet providers over two years.

Outline


Data Collection Methodology

• Data collected by probes of a commercial security and traffic monitoring platform instrumenting BGP edge routers.

• Each probe exports traffic flow statistics that includes traffic per BGP AS, ASPath, network and transport layer protocols, countries, etc. independently to form different datasets.

Data Collection Methodology

Fig 3: Netflow network probes.

Challenges in Data Collection

• Commercial privacy concerns.• Misconfiguration of probes (resulting in wild

fluctuations in daily traffic).• Decommissioning of edge routers!• Decommissioning of older probes and

addition of new ones!

Need for Data Aggregation

• Wild fluctuations in the data.

• Heterogeneity of the providers.

• Ratios and percentages were consistent despite the fluctuations.

Data Aggregation

• The probes calculated average traffic volume every five minutes for all members of all datasets throughout every 24 hour period.

• They also calculated the average volume of total inter-domain network traffic.

• Finally, the daily traffic volume per item and network total were used to calculate the daily percentage.

Data Aggregation

• Calculation of weights:

where, = Weight of participant i on day d. = Router count for participant i on day d.N = All study participants.

• Calculation of weighted average percentage:

where,= Weighted average percent share of traffic for A on day d.

( )𝐴 = Each provider’s measured average traffic volume for A on day d.= Total average inter-domain traffic for day d.

A = Traffic attribute like ASN, TCP port, country of origin, etc.

Data Aggregation

Data Defects and Validation

• The probes did not detect traffic over peering adjacencies between enterprise business partners or in cases of similar agreements.

• The data collected was validated through private discussions with large content providers, transit ISPs and regional networks.

Outline


ASN Traffic Analysis

Rank Provider Percentage

1 ISP A 5.77

2 ISP B 4.55

3 ISP C 3.35

4 ISP D 3.2

5 ISP E 2.6

6 ISP F 2.77

7 ISP G 2.24

8 ISP H 1.82

9 ISP I 1.35

10 ISP J 1.23

Table 1: Traffic contributors by weighted average percentage in July 2007.

ASN Traffic Analysis


1 ISP A 9.41

2 ISP B 5.7

3 Google 5.2

4 ISP F 5.0

5 ISP H 3.22

6 Comcast 3.12

7 ISP D 3.08

8 ISP E 2.32

9 ISP C 2.05

10 ISP G 1.89

Table 2: Traffic contributors by weighted average percentage in July 2009.

Trends

• Data from July 2007 is compliant with textbook diagrams of internet topology.

• Changes in commercial policy and traffic engineering have drastically impacted shares in Internet inter-domain traffic.

Trends


1 Google 4.04

2 ISP A 3.74

3 ISP F 2.86

4 Comcast 1.94

5 ISP K 1.60

6 ISP B 1.36

7 ISP H 1.21

8 ISP L 0.66

9 Microsoft 0.62

10 Akamai 0.06

Table 3: Providers with most significant inter-domain traffic share growth in 2007-2009.

Google Trends

• Google, a content provider, now rivals global transit networks and enjoyed the highest growth.

• Providers and the data collected suggest that Google’s huge growth may be ascribed to the acquisition of Youtube.

Google Trends

Fig 4: Google inter-domain traffic contribution.

Comcast TrendsConsumer Content

Fig 5: Comcast inter-domain traffic contribution.

Comcast Trends

• Majority of the traffic growth in Comcast came from transit traffic.

• What did Comcast do?– Consolidated regional backbones into a single

nationwide network.– Rolled out a consumer product called “triple play”

(voice, video, data).– Began offering wholesale transit, cellular backhaul

and IP video distribution!

Outline


Application Traffic Analysis

• Applications could be identified from TCP/UDP port numbers but:– Applications could use non-standard ports.– Port-based classification only accounts the control

traffic and not the often random port numbers associated with subsequent data transfer.

• The authors obtained validation data from five cooperating provider deployments in Asia, Europe and North America.

Largest ApplicationsRank Application 2007 2009 Change

1 Web 41.68 52 10.31

2 Video 1.58 2.64 1.05

3 VPN 1.04 1.41 0.38

4 Email 1.41 1.38 -0.03

5 News 1.75 0.97 -0.78

6 P2P 2.96 0.85 -2.11

7 Games 0.38 0.49 0.12

8 SSH 0.19 0.28 -0.08

9 DNS 0.2 0.17 -0.04

10 FTP 0.21 0.14 -0.07

Other 2.56 2.67 0.11

Unclassified 46.03 37 -9.03

Table 4: Top applications by weighted average percentage.

Largest ApplicationsAverage Percentage

Web 52.12

Video 0.98

Email 1.54

VPN 0.24

News 0.07

P2P 18.32

Games 0.52

SSH N/A

DNS N/A

FTP 0.16

Other 20.54

Unclassified 5.51

Table 5: Average application breakdown in July 2009 across five consumer providers.

Application Traffic Changes

Fig 6: Distribution of weighted average percentage of traffic from well known ports (July).


• TCP and UDP combined account for more than 95% of all inter-domain traffic.

• In July 2007, 52 ports contributed 60% of traffic.

• In July 2009, 25 ports contributed 60% of traffic.– What’s happening here?


• Internet application traffic is getting migrated to a smaller set of ports and protocols.

• Microsoft migrated all Xbox Live traffic to use port 80 on June 16, 2009.

• One reason for this is that majority of firewalls allow HTTP traffic.

• The other reason for this trend is the dominance of

Applications Exhibiting Growth

• Much of the growth in web data is due to video (Youtube uses progressive downloading).

• Growth in video traffic is due to the introduction of services like Hulu, Youtube, Veoh, etc.

Applications Exhibiting Growth

Fig 7: Change in weighted average percentage of video protocols.

Obama!

Applications Exhibiting Decline

• P2P dropped by 2.8% between 2007-2009.• Possible reasons– Migration to tunnelled overlays.– Use of P2P encryption.– Stealthier P2P clients and algorithms.

• Network operators however, suggest P2P traffic may have migrated to alternatives like direct download and streaming video.

Outline


Internet Size Estimates

Fig 8: Independent inter-domain traffic volumes vs calculated aggregate ASN share..

𝑆𝑙𝑜𝑝𝑒=2.51


• The authors solicited verification data from twelve large providers and content sites.

• The slope of the above curve means that a 2.51% of share of all traffic means 1 Tbps of traffic. Extrapolating this way,

• This was in July 2009 (before the release of iPad!).


• In other analyses, the authors report the annual growth rate of the internet to be about 44.5% and the monthly traffic volume to be about 9 exabytes ( = 9 million terabytes!)

Outline


Conclusion

• This paper is the first longitudinal study of Internet inter-domain traffic.

• It identifies a significant ongoing evolution of provider interconnection strategies from a hierarchical to a more densely connected model.

• Most internet content is migrating to a relatively small number of hosting, cloud and content providers.

Conclusion

• Google is the largest contributor to Internet traffic growth.

• Majority of inter-domain traffic has migrated to a relatively small number of ports and protocols.

• As of July 2009, the inter-domain traffic peaks exceeded 39 Tbps and were growing at 44.5% annually.

Questions?

Documents

Internet Inter-Domain Traffic Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian Presented by: Kaushik Choudhary