Internet peering, graphics only

Impact on Network Architecture and Economics

Internet Peering

Brough Turner Founder & CTO, netBlazr Inc.

Today’s Network of Networks

12 March 2014 2

Billions of Internet users

Tens of Millions of Networks

Access ISPs

IXP

IXP

Backbones

IXP Aggregation Regional

12 March 2014 3

1991

 NSF lifts restrictions on commercial use

 SURFnet, PSInet & AlterNET form CIX exchange

12 March 2014 4

1993 Plan – 1995 Reality

12 March 2014 5

12 March 2014 6

AGIS, ANS, BBN, CERFnet, MCI, PSInet, SprintLink

NAP NAP

12 March 2014 7

AGIS, ANS, BBN, CERFnet, MCI, PSInet, SprintLink

NAP NAP

De-peering   UUNET – April/May 1997   MAE-East FDDI shutdown – Feb 2001   Cable & Wireless – May 2001   Sprint – gradual 2001/2002

12 March 2014 8

Donut Peering

12 March 2014 9

Tier 1 ISPs

Access ISPs

Tier 2 ISPs

Cable Co’s

Large, savvy content providers

Content providers

$

$

$ $

$

Donut Peering

12 March 2014 10

Tier 1 ISPs

Access ISPs

Tier 2 ISPs

Cable Co’s


Content providers

P P

P

$

$

$ $

$

Donut Peering

12 March 2014 11

Tier 1 ISPs

Access ISPs

Tier 2 ISPs

Cable Co’s


Content providers

P P

P

$

$

$ $ $ $

$

CDNs emerge

12 March 2014 12

IXP IXP CDN NOC

CDNs emerge

12 March 2014 13

IXP IXP CDN NOC

Turbulence

12 March 2014 14

  UUNET (Alternet) MFS Worldcom Verizon   ANS AOL "

  SprintLink Sprint   BBN GTE Genuity Level 3   MCI Cable & Wireless   CERFnet AT&T   AGIS Telia   PSInet (US) Cogent

(Canada) Telus (EU) Interroute

ISP1

Peering vs Transit

12 March 2014 15

Backbone

ISP2

ISP

U U U

U ISP ISP

ISPx

Transit

ISP1

Peering vs Transit

12 March 2014 16

Backbone

ISP2

ISP

U U U

U ISP ISP

ISPx Peering

Transit

ISP1

Peering vs Transit

12 March 2014 17

Backbone

ISP2

ISP

U U U

U ISP ISP

ISPx Peering

Transit

Peering Technology   Internet Protocol –

  IPv4 == 98% of all traffic   IPv6 == ~2% of traffic

  Border Gateway Protocol –  Based on distance and path  Routing information exchanged between

Autonomous Systems

12 March 2014 18

Border Gateway Protocol

12 March 2014 19

ISP1

ISP2

A

B

C

My Edge Router


12 March 2014 20

ISP1

ISP2

A

B

C

I can handle traffic for A in 3 hops &

for B in 1 hop

My Edge Router


12 March 2014 21

ISP1

ISP2

A

B

C


for B in 1 hop

My Edge Router


12 March 2014 22

ISP1

ISP2

A

B

C


for B in 1 hop

1.  Do I believe your offer? 2.  Who has shortest path? 3.  Are there $ considerations? 4.  Add routes to routes table.

My Edge Router

Economics of a Growing ISP   Transit from one upstream

  Circuit (transport) to an open IXP  Transit from two or more upstreams  Peering for content (Google, Akamai, etc.)

  Transport to other IXPs  Peering for additional traffic

12 March 2014 23

Costs of YouTube-like growth

12 March 2014 24

Peering in Practice

12 March 2014 25

  2011 survey of 4,331 ISPs (86% of the backbone) analyzed 142K interconnections

  Settlement-free peering dominates  99.5% based on handshake agreements  99.7% symmetric

  Typical peering requirements  Operations (24 hr NOC; enough traffic)  Technical (shortest prefix; consistency)  General (NDAs, Suspend at will)

  Emergence of multi-lateral peering

Governing Law Preferences

12 March 2014 26

Source: Survey of Characteristics of Internet Carrier Interconnection Agreements by Woodcock & Adhikari, Packet Clearing House. May 2011

Probability of selection as a country of governing law

Multi-lateral peering emerges

12 March 2014 27


Number of networks (X axis) with each quantity of peering partners (Y axis)

Many BGP sessions for Bi-lateral traffic exchange

12 March 2014 28

ISP1

ISP3

ISP2

ISP4

ISP5

Multi-lateral Peering

12 March 2014 29

ISP1

ISP3

ISP2

ISP4

ISP5

Route Server

Tier 1s and everyone else

12 March 2014 30

Number of advertised IPv4 addresses (Y axis) vs number of interconnects/ISP


Tier 1 ISPs

Market concentration declining

12 March 2014 31

The Internet

12 March 2014 32

  A voluntary agreement among network operators to exchange traffic for their mutual benefit.

  Fred Goldstein, TMCnews, June 2009

  Essentially unregulated   Largely informal   Not well understood by outsiders

  Extremely successful

Brough Turner netBlazr Inc.

[email protected]

12 March 2014 33

Thank You