SkipNet: A Scalable Overlay Network with Practical Locality

PropertiesNick Harvey, Mike Jones, Nick Harvey, Mike Jones,

Stefan Saroiu, Marvin Theimer, Alec WolmanStefan Saroiu, Marvin Theimer, Alec Wolman

Presented by Qingqing YuanPresented by Qingqing Yuan

Overlay NetworksOverlay Networks

Overlays have achieved several goals:Overlays have achieved several goals: Scalable and decentralized infrastructureScalable and decentralized infrastructure Uniform and random load and data Uniform and random load and data

distributiondistribution

But, at the price of data controllabilityBut, at the price of data controllability Data may be stored far from its usersData may be stored far from its users Data may be stored outside its domainData may be stored outside its domain Local accesses leave local organizationLocal accesses leave local organization

Basic Basic trade-offtrade-off: data : data controllability vs. data uniformitycontrollability vs. data uniformity

SkipNet: SkipNet: Traditional overlay functionalityTraditional overlay functionality Provides an abstraction to control Provides an abstraction to control

this this trade-offtrade-off:: Constrained load balancing (CLB)Constrained load balancing (CLB)

OutlineOutline Basic SkipNet designBasic SkipNet design SkipNet locality SkipNet locality

propertiesproperties Performance evaluationPerformance evaluation ConclusionsConclusions

OutlineOutline Basic SkipNet designBasic SkipNet design SkipNet locality SkipNet locality

propertiesproperties Performance evaluationPerformance evaluation ConclusionsConclusions

Why not a DHT?Why not a DHT?

Controlling Data Location is not the Controlling Data Location is not the goal of a DHTgoal of a DHT

DHT’s provide load balancing at the DHT’s provide load balancing at the price of where data is storedprice of where data is stored May be stored far awayMay be stored far away May be stored out of the domainMay be stored out of the domain

Destroy LocalityDestroy Locality Discard useful application-specific Discard useful application-specific

informationinformation

Structure of SkipNet: Structure of SkipNet: Perfect Skip ListsPerfect Skip Lists

SkipNetSkipNet Key property: two address spacesKey property: two address spaces

1.1. Name ID space: nodes are sorted by their Name ID space: nodes are sorted by their names (e.g. DNS names)names (e.g. DNS names)

2.2. Numeric ID space: nodes are randomly Numeric ID space: nodes are randomly distributeddistributed

Combining both spaces achievesCombining both spaces achieves Content + Path localityContent + Path locality

Scalable peer-to-peer overlay networkScalable peer-to-peer overlay network O(log N) routing performance in both spacesO(log N) routing performance in both spaces O(log N) routing state per nodeO(log N) routing state per node

SkipNet RingSkipNet Ring

Pointers at level Pointers at level hh skip over 2 skip over 2hh nodes nodes Nodes are ordered by namesNodes are ordered by names

A

D M

V

T

XZ

O

SkipNet RingSkipNet Ring

Pointers at level Pointers at level hh skip over 2 skip over 2hh nodes nodes Nodes are ordered by namesNodes are ordered by names

A

D M

V

T

XZ

O

SkipNet RingSkipNet Ring

Pointers at level Pointers at level hh skip over 2 skip over 2hh nodes nodes Nodes are ordered by namesNodes are ordered by names

A

E F

M

H

SZ

G

SkipNet Global ViewSkipNet Global View

A Level: L = 0

L = 1

L = 3

L = 2

Root Ring

Ring 0 Ring 1

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

D M OT

VXZ

OZ

A TM

X

DV

A T

M

X

D

VZ

O

O

ZA T

M

X

D

V

Two Address SpacesTwo Address Spaces

SkipNet can route efficiently SkipNet can route efficiently in both address spaces:in both address spaces: Name ID space (e.g. DNS Name ID space (e.g. DNS

names)names) Numeric ID spaceNumeric ID space

Routing by Name IDRouting by Name ID

Level: L = 0

L = 1

L = 2

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

O

ZA T

M

X

DV

A TM

XD

V Z

OL = 3

Node A’sRoutingTable

Routing by Name IDRouting by Name ID

Level: L = 0

L = 1

L = 2

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

O

ZA T

M

X

DV

A TM

XD

V Z

OL = 3

Routing by Name IDRouting by Name ID

Level: L = 0

L = 1

L = 2

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

O

ZA T

M

X

DV

A TM

XD

V Z

OL = 3

Node T’sRoutingTable

Routing by Name IDRouting by Name ID

Level: L = 0

L = 1

L = 2

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

O

ZA T

M

X

DV

A TM

XD

V Z

OL = 3

Node T’sRoutingTable

Routing by Name IDRouting by Name ID

Level: L = 0

L = 1

L = 2

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

OZ

A TM

X

DV

A TM

XD

V Z

OL = 3

Node T’sRoutingTable

Routing by Name IDRouting by Name ID

Example: route from A to VExample: route from A to V Simple Rule: Forward the message to node that is closest to Simple Rule: Forward the message to node that is closest to

dest, without going too far.dest, without going too far.

Level: L = 0

L = 1

L = 2Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

OZ

A TM

X

DV

A TM

XD

V Z

OL = 3

Routing by Numeric IDRouting by Numeric ID Provides the basic DHT Provides the basic DHT

primitiveprimitive To store file “Foo.c”To store file “Foo.c”

Hash(“Foo.c”) Hash(“Foo.c”) a random numeric a random numeric IDID

Find highest ring matching that Find highest ring matching that numeric IDnumeric ID

Store file on node in that ringStore file on node in that ring

Log N routing efficiencyLog N routing efficiency

DHT ExampleDHT Example

Store file “Foo.c” from node AStore file “Foo.c” from node A Hash(“Foo.c”) = 101…Hash(“Foo.c”) = 101…

Route from A to V in Route from A to V in numeric numeric spacespace

Level: L = 0

L = 1

L = 2Ring 00 Ring 01 Ring 10 Ring 11

Ring000

Ring001

Ring010

Ring011

Ring100

Ring101

Ring110

Ring111

A Root RingD M O

TVXZ

Ring 0A

M

T

X

Ring 1D

Z V

O

O

ZA T

M

X

DV

A TM

XD

V Z

OL = 3

Foo.c

Talk OutlineTalk Outline

Basic SkipNet designBasic SkipNet design SkipNet locality SkipNet locality

propertiesproperties Performance evaluationPerformance evaluation ConclusionsConclusions

Key Locality PropertiesKey Locality Properties In practice, two properties are In practice, two properties are

important:important: Content LocalityContent Locality – ability to explicitly – ability to explicitly

place dataplace data Placement on a single node or on a set of Placement on a single node or on a set of

nodesnodes Path LocalityPath Locality – ability to – ability to guaranteeguarantee that that

local traffic remains locallocal traffic remains local

One abstraction is important – CLB:One abstraction is important – CLB:

Constrained Load Balancing Constrained Load Balancing (CLB)(CLB)

Multiple DHTs with differing scopes using a single Multiple DHTs with differing scopes using a single SkipNet structureSkipNet structure

A result of the ability to route in both address A result of the ability to route in both address spacesspaces

Divide data object names into 2 partsDivide data object names into 2 partsusing the ‘!’ special character using the ‘!’ special character CLB DomainCLB Domain CLB SuffixCLB Suffix

microsoft.commicrosoft.com!!skipnet.htmlskipnet.htmlNumeric RoutingName Routing

CLB ExampleCLB Example

To read file “com.microsoftTo read file “com.microsoft!!skipnet.html”skipnet.html” Route by name ID to “com.microsoft”Route by name ID to “com.microsoft” Route by numeric ID to Hash(“skipnet.html”)Route by numeric ID to Hash(“skipnet.html”)

within the “com.microsoft” constraintwithin the “com.microsoft” constraint

com.sun

edu.ucbgov.irs

com.microsoftskipnet.html

SkipNet Path LocalitySkipNet Path Locality

Organizations correspond to contiguous SkipNet Organizations correspond to contiguous SkipNet segmentssegments Internal routing by NameID remains internalInternal routing by NameID remains internal

Nodes have left / right pointersNodes have left / right pointers

com.sun

edu.ucbgov.irs

com.microsoft

com.microsoft.research

Fault ToleranceFault Tolerance Many failures occur along organizational Many failures occur along organizational

boundaries:boundaries: Gateway/firewall failure, BGP misconfig, physical Gateway/firewall failure, BGP misconfig, physical

network cut, …network cut, …

SkipNet handles organizational disconnect SkipNet handles organizational disconnect gracefullygracefully Results in two well-connected, partitioned SkipNetsResults in two well-connected, partitioned SkipNets Efficient remerging algorithmsEfficient remerging algorithms

Node independent failuresNode independent failures Same resiliency as systems such as Chord and PastrySame resiliency as systems such as Chord and Pastry

Primary Security Benefit & Primary Security Benefit & WeaknessWeakness+ SkipNet + name access control SkipNet + name access control

mechanism:mechanism: Content locality ensures that content stays Content locality ensures that content stays

within organizationwithin organization Path locality prevents: Path locality prevents:

malicious forwarders malicious forwarders analysis of internal trafficanalysis of internal traffic external tampering external tampering

- Easier to target organizations:Easier to target organizations: Someone creates one million nodes with Someone creates one million nodes with

name prefixes name prefixes microsofa.com microsofa.com and and microsort.commicrosort.com

Most traffic to/from Microsoft will go through Most traffic to/from Microsoft will go through a microsofa / microsort intermediate nodea microsofa / microsort intermediate node

Talk OutlineTalk Outline

Basic SkipNet designBasic SkipNet design SkipNet locality SkipNet locality

propertiesproperties Performance evaluationPerformance evaluation ConclusionsConclusions

Routing by Name ID PerformanceRouting by Name ID Performance

Benefits come at no extra costBenefits come at no extra cost

Surviving Organizational Surviving Organizational DisconnectDisconnect

Disconnected Org Size = 15% of all Disconnected Org Size = 15% of all nodesnodes

ConclusionsConclusions SkipNetSkipNet: :

Traditional overlay functionalityTraditional overlay functionality Explicit control of data placementExplicit control of data placement

Constrained load balancingConstrained load balancing Content + Path Locality are basic Content + Path Locality are basic

ingredients to:ingredients to: Data controllability Data controllability ManageabilityManageability SecuritySecurity Data availabilityData availability PerformancePerformance

Questions?Questions?

Thank Thank you!you!