1

15-744: Computer Networking

L-14 Network Topology

Today’s Lecture

•  Structural generators •  Power laws •  HOT graphs •  Graph generators •  Assigned reading

•  On Power-Law Relationships of the Internet Topology

•  A First Principles Approach to Understanding the Internet’s Router-level Topology

2

Outline

•  Motivation/Background

•  Power Laws

•  Optimization Models

•  Graph Generation

3

Why study topology?

•  Correctness of network protocols typically independent of topology

•  Performance of networks critically dependent on topology •  e.g., convergence of route information

•  Internet impossible to replicate •  Modeling of topology needed to generate

test topologies

4

2

Internet topologies

AT&T

SPRINT MCI

AT&T

MCI SPRINT

Router level Autonomous System (AS) level

5

More on topologies.. •  Router level topologies reflect physical connectivity

between nodes •  Inferred from tools like traceroute or well known public

measurement projects like Mercator and Skitter

•  AS graph reflects a peering relationship between two providers/clients •  Inferred from inter-domain routers that run BGP and publlic

projects like Oregon Route Views

•  Inferring both is difficult, and often inaccurate

6

Hub-and-Spoke Topology

•  Single hub node •  Common in enterprise networks •  Main location and satellite sites •  Simple design and trivial routing

•  Problems •  Single point of failure •  Bandwidth limitations •  High delay between sites •  Costs to backhaul to hub

7

Simple Alternatives to Hub-and-Spoke •  Dual hub-and-spoke

•  Higher reliability •  Higher cost •  Good building block

•  Levels of hierarchy •  Reduce backhaul cost •  Aggregate the

bandwidth •  Shorter site-to-site

delay … 8

3

Abilene Internet2 Backbone

9

SOX

SFGP/ AMPATH

U. Florida

U. So. Florida

Miss State GigaPoP

WiscREN

SURFNet

Rutgers U.

MANLAN

Northern Crossroads

Mid-Atlantic Crossroads

Drexel U.

U. Delaware

PSC

NCNI/MCNC

MAGPI

UMD NGIX

DARPA BossNet

GEANT

Seattle

Sunnyvale

Los Angeles

Houston

Denver Kansas

City Indian- apolis

Atlanta

Wash D.C.

Chicago

New York

OARNET

Northern Lights Indiana GigaPoP

Merit U. Louisville

NYSERNet

U. Memphis

Great Plains

OneNet Arizona St.

U. Arizona

Qwest Labs

UNM

Oregon GigaPoP

Front Range GigaPoP

Texas Tech

Tulane U.

North Texas GigaPoP

Texas GigaPoP

LaNet

UT Austin

CENIC

UniNet

WIDE

AMES NGIX

Pacific Northwest GigaPoP

U. Hawaii

Pacific Wave

ESnet

TransPAC/APAN

Iowa St.

Florida A&M UT-SW Med Ctr.

NCSA

MREN

SINet

WPI

StarLight

Intermountain GigaPoP

Abilene Backbone Physical Connectivity (as of December 16, 2003)

0.1-0.5 Gbps 0.5-1.0 Gbps 1.0-5.0 Gbps 5.0-10.0 Gbps

10

Points-of-Presence (PoPs) •  Inter-PoP links

•  Long distances •  High bandwidth

•  Intra-PoP links •  Short cables between

racks or floors •  Aggregated bandwidth

•  Links to other networks •  Wide range of media

and bandwidth

Intra-PoP

Other networks

Inter-PoP

11

Deciding Where to Locate Nodes and Links

•  Placing Points-of-Presence (PoPs) •  Large population of potential customers •  Other providers or exchange points •  Cost and availability of real-estate •  Mostly in major metropolitan areas

•  Placing links between PoPs •  Already fiber in the ground •  Needed to limit propagation delay •  Needed to handle the traffic load

12

4

Trends in Topology Modeling Observation

•  Long-range links are expensive

•  Real networks are not random, but have obvious hierarchy

•  Internet topologies exhibit power law degree distributions (Faloutsos et al., 1999)

•  Physical networks have hard technological (and economic) constraints.

Modeling Approach •  Random graph (Waxman88)

•  Structural models (GT-ITM Calvert/Zegura, 1996)

•  Degree-based models replicate power-law degree sequences

•  Optimization-driven models topologies consistent with design tradeoffs of network engineers

13

Waxman model (Waxman 1988)

•  Router level model •  Nodes placed at random

in 2-d space with dimension L

•  Probability of edge (u,v): •  ae^{-d/(bL)}, where d is

Euclidean distance (u,v), a and b are constants

•  Models locality

14

v

u d(u,v)

Real world topologies

•  Real networks exhibit •  Hierarchical structure •  Specialized nodes (transit, stub..) •  Connectivity requirements •  Redundancy

•  Characteristics incorporated into the Georgia Tech Internetwork Topology Models (GT-ITM) simulator (E. Zegura, K.Calvert and M.J. Donahoo, 1995)

15

Transit-stub model (Zegura 1997) •  Router level model •  Transit domains

•  placed in 2-d space •  populated with routers •  connected to each other

•  Stub domains •  placed in 2-d space •  populated with routers •  connected to transit

domains

•  Models hierarchy

16

5

So…are we done?

•  No! •  In 1999, Faloutsos, Faloutsos and

Faloutsos published a paper, demonstrating power law relationships in Internet graphs

•  Specifically, the node degree distribution exhibited power laws

That Changed Everything…..

17

Outline

•  Motivation/Background

•  Power Laws

•  Optimization Models

•  Graph Generation

18

Power laws in AS level topology

19

A few nodes have lots of connections

Ran

k R(

d)

Degree d

Source: Faloutsos et al. (1999) Power Laws and Internet Topology

•  Router-level graph & Autonomous System (AS) graph •  Led to active research in degree-based network models

Most nodes have few connections

R(d)

= P

(D>

d) x

#no

des

6

GT-ITM abandoned..

•  GT-ITM did not give power law degree graphs

•  New topology generators and explanation for power law degrees were sought

•  Focus of generators to match degree distribution of observed graph

21

Inet (Jin 2000) •  Generate degree sequence •  Build spanning tree over nodes

with degree larger than 1, using preferential connectivity •  randomly select node u not in

tree •  join u to existing node v with

probability d(v)/Σd(w) •  Connect degree 1 nodes using

preferential connectivity •  Add remaining edges using

preferential connectivity 22

Power law random graph (PLRG) •  Operations

•  assign degrees to nodes drawn from power law distribution •  create kv copies of node v; kv degree of v. •  randomly match nodes in pool •  aggregate edges

may be disconnected, contain multiple edges, self-loops •  contains unique giant component for right choice of

parameters

23

2

1 1

Barabasi model: fixed exponent

•  incremental growth •  initially, m0 nodes •  step: add new node i with m edges

•  linear preferential attachment •  connect to node i with probability ki / ∑ kj

24

0.5

0.5 0.25

0.5 0.25

new node existing node

may contain multi-edges, self-loops

7

Features of Degree-Based Models

•  Degree sequence follows a power law (by construction)

•  High-degree nodes correspond to highly connected central “hubs”, which are crucial to the system

•  Achilles’ heel: robust to random failure, fragile to specific attack

25

Preferential Attachment Expected Degree Sequence

Does Internet graph have these properties?

•  No…(There is no Memphis!) •  Emphasis on degree distribution - structure

ignored •  Real Internet very structured •  Evolution of graph is highly constrained

26

Problem With Power Law

•  ... but they're descriptive models!

•  No correct physical explanation, need an understanding of: •  the driving force behind deployment •  the driving force behind growth

27

Outline

•  Motivation/Background

•  Power Laws

•  Optimization Models

•  Graph Generation

28

8

Li et al.

•  Consider the explicit design of the Internet •  Annotated network graphs (capacity,

bandwidth) •  Technological and economic limitations •  Network performance

•  Seek a theory for Internet topology that is explanatory and not merely descriptive. •  Explain high variability in network connectivity •  Ability to match large scale statistics (e.g.

power laws) is only secondary evidence 29

10"0"

10"1"

10"2"

Degree"10"

-1"

10"0"

10"1"

10"2"

10"3"

Band

wid

th (G

bps)"

15 x 10 GE"

15 x 3 x 1 GE"

15 x 4 x OC12"

15 x 8 FE"

Technology constraint"

Total Bandwidth "

Bandwidth per Degree "

Router Technology Constraint

30

Cisco 12416 GSR, circa 2002 high BW low degree

high degree low BW

approximate aggregate

feasible region

Aggregate Router Feasibility

core technologies

edge technologies

older/cheaper technologies

Source: Cisco Product Catalog, June 2002 31 Rank (number of users)

Con

nect

ion

Spee

d (M

bps)

1e-1

1e-2

1

1e1

1e2

1e3

1e4

1e2 1 1e4 1e6 1e8

Dial-up ~56Kbps

Broadband Cable/DSL ~500Kbps

Ethernet 10-100Mbps

Ethernet 1-10Gbps

most users have low speed

connections

a few users have very high speed connections

high performance computing academic

and corporate residential and small business

Variability in End-User Bandwidths

32

9

Heuristically Optimal Topology

Hosts

Edges

Cores

Mesh-like core of fast, low degree routers

High degree nodes are at the edges.

33

Comparison Metric: Network Performance

Given realistic technology constraints on routers, how well is the network able to carry traffic?

Step 1: Constrain to be feasible

Abstracted Technologically Feasible Region

1

10

100

1000 10000

100000

1000000

10 100 1000 degree

Ban

dwid

th (M

bps)

Step 3: Compute max flow

Bi

Bj

xij

Step 2: Compute traffic demand

34

Likelihood-Related Metric

•  Easily computed for any graph •  Depends on the structure of the graph, not the generation

mechanism •  Measures how “hub-like” the network core is

•  For graphs resulting from probabilistic construction (e.g. PLRG/GRG),

LogLikelihood (LLH) ∝ L(g)

•  Interpretation: How likely is a particular graph (having given node degree distribution) to be constructed?

Define the metric (di = degree of node i)

35

Lmax"l(g) = 1"P(g) = 1.08 x 1010"

P(g) "Perfomance (bps)

PA PLRG/GRG HOT Abilene-inspired Sub-optimal

0 0.2 0.4 0.6 0.8 1 10 10

10 11

10 12

l(g) = Relative Likelihood 36

10

100101102101100101102DegreeAchieved BW (Gbps) 100101102101100101102DegreeAchieved BW (Gbps)

100101102101100101102DegreeAchieved BW (Gbps)

PA PLRG/GRG HOT

Structure Determines Performance

P(g) = 1.19 x 1010 P(g) = 1.64 x 1010 P(g) = 1.13 x 1012

37

Summary Network Topology •  Faloutsos3 [SIGCOMM99] on Internet topology

•  Observed many “power laws” in the Internet structure •  Router level connections, AS-level connections, neighborhood sizes

•  Power law observation refuted later, Lakhina [INFOCOM00]

•  Inspired many degree-based topology generators •  Compared properties of generated graphs with those of measured

graphs to validate generator •  What is wrong with these topologies? Li et al [SIGCOMM04]

•  Many graphs with similar distribution have different properties •  Random graph generation models don’t have network-intrinsic

meaning •  Should look at fundamental trade-offs to understand topology

•  Technology constraints and economic trade-offs •  Graphs arising out of such generation better explain topology and its

properties, but are unlikely to be generated by random processes!

38

Outline

•  Motivation/Background

•  Power Laws

•  Optimization Models

•  Graph Generation

39

Graph Generation

•  Many important topology metrics •  Spectrum •  Distance distribution •  Degree distribution •  Clustering…

•  No way to reproduce most of the important metrics

•  No guarantee there will not be any other/new metric found important

40

11

dK-series approach

•  Look at inter-dependencies among topology characteristics

•  See if by reproducing most basic, simple, but not necessarily practically relevant characteristics, we can also reproduce (capture) all other characteristics, including practically important

•  Try to find the one(s) defining all others

0K

Average degree <k>

1K

Degree distribution P(k)

2K

Joint degree distribution P(k1,k2)

12

3K

“Joint edge degree” distribution P(k1,k2,k3)

3K, more exactly

4K Definition of dK-distributions

dK-distributions are degree correlations within simple connected graphs of size d

13

Nice properties of properties Pd

•  Constructability: we can construct graphs having properties Pd (dK-graphs)

•  Inclusion: if a graph has property Pd, then it also has all properties Pi, with i < d (dK-graphs are also iK-graphs)

•  Convergence: the set of graphs having property Pn consists only of one element, G itself (dK-graphs converge to G)

Rewiring

50

Graph Reproduction

51

The elephant in the room…

•  How good is the underlying data on which these studies are based?

•  Impact •  Sampling bias traceroute of shortest paths

on random graph can produce power-law distribution [Lakhina03]

•  Similar issues with AS-level views

52

14

Better Measurements?

•  Rocketfuel [sigcomm02] •  Better router alias resolution •  Detailed maps based on multiple viewpoints

•  Reverse traceroute [nsdi10] •  Improves ability to view peer-to-peer links

•  RouteViews and BGP collection efforts

53

Next Lecture

•  Overlay networks •  Challenges in deploying new protocols •  Required readings:

•  Active network vision and reality: lessons from a capsule-based system

•  Optional readings: •  Resilient Overlay Networks •  Future Internet Architecture: Clean-Slate

Versus Evolutionary Research

54

•  Faloutsos3 (Sigcomm’99) •  frequency vs. degree

Power Laws

topology from BGP tables of 18 routers 55

•  Faloutsos3 (Sigcomm’99) •  frequency vs. degree

Power Laws

topology from BGP tables of 18 routers 56

15

•  Faloutsos3 (Sigcomm’99) •  frequency vs. degree

Power Laws

topology from BGP tables of 18 routers 57

•  Faloutsos

•  frequency vs. degree

•  empirical ccdf P(d>x) ~ x-a

Power Laws

58

Power Laws

•  Faloutsos3 (Sigcomm’99)

•  frequency vs. degree

•  empirical ccdf P(d>x) ~ x-a

α ≈1.15

59