University Computing Services EDUCAUSE Mid-Atlantic Regional Conference 16 January 2003 An Infrastructure and Accounting Response to Peer to Peer Traffic

University Computing ServicesUniversity Computing Services

EDUCAUSE Mid-Atlantic Regional Conference16 January 2003

An Infrastructure and An Infrastructure and Accounting Response to Accounting Response to

Peer to Peer Traffic VolumePeer to Peer Traffic Volume

Dr. Michael R MundraneDirector of Telecommunications

Rutgers University Computing Services


CopyrightCopyright

Copyright Michael R Mundrane 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.


AgendaAgenda

• Statement of Problem

• Objectives

• Approach

• Results

• Conclusions


Statement of ProblemStatement of Problem

Is he kidding? P2P is the problem!


Network EvolutionNetwork Evolution

• Sporadic

• Unequally funded

• Unstructured

• Immediacy

• Complex

• Point services

• Faculty centric


Application ModelsApplication Models

• Limited customer interface • Minimal administration• Centralized management• Centralized storage• hub and spoke infrastructure• Minimal bandwidth

Terminal Host



• Rich customer interface

• Medium administration

• Centralized management

• Hybrid storage (server and client)

• Tiered network infrastructure

• Bandwidth server/s dependant

Client Server



• Rich user interface

• High touch administration

• Distributed management (costly)

• Distributed storage (difficult to manage)

• Complex mesh infrastructure

• High bandwidth

Peer Peer


Cooperative?!?Cooperative?!?

A. Badges? We don’t see no stinking badges!

Q. Excuse me, would you please forward the business activity associated with your traffic so that we can adjust our records?


ObjectivesObjectives

More than near term survival!


Essential CharacteristicsEssential Characteristics

• Preserve behavior

• Ensure access

• Moderate impact

• Protect privacy

• Avoid value judgments

• Apply to new applications


AssumptionsAssumptions

• Large number of hosts

• Small number of problems

• Service consumers

• Many random light providers

• Few heavy providers

• Responsive community


Just Use Traffic ShapingJust Use Traffic Shaping


Just Use Traffic ShapingJust Use Traffic Shaping

• Cisco routers

• CAR – traffic class

• MicroCAR – identified flow

day

Gigabytes

bits

byte

M

G

K

M

day

onds

ond

32.1

8024,1024,1

sec400,86

sec

Kilobits128


Just Use QoSJust Use QoS


Just Use QoSJust Use QoS

• Classification

• Differentiation

• Admission control

• Provisioning

• Bandwidth

• Latency

• Jitter


QoS DifferentiationQoS Differentiation

P2P Other

10Mbit 90Mbit


QoS DifferentiationQoS Differentiation

10Mbit

Differentiation w/o admission control only

defers the problem!


Rutgers NetworkRutgers Network

• 40,000+ hosts

• 1200+ networks

• 200+ routers

• 17 zones

• 7 campuses

• 3 regions

• 1 autonomous system


ApproachApproach

No single solution!


Best Network PracticesBest Network Practices

• Modular

• Layered

• Aggregated

• Scalable

• Uniform

• Deterministic

• Comprehensible


DeviceDevice DeviceDevice

DeviceDevice DeviceDevice

Intra-building Backbone

Building

Intra-building BackboneIntra-building Backbone

RUNet ~ 1200


BuildingBuilding BuildingBuilding

BuildingBuilding BuildingBuilding

Inter-building Backbone

Zone

Inter-building BackboneInter-building Backbone

RUNet 17


ZoneZone ZoneZone

ZoneZone ZoneZone

Intra-campus Backbone

Campus

Intra-campus BackboneIntra-campus Backbone

RUNet 7


CampusCampus CampusCampus

CampusCampus CampusCampus

Inter-campus backbone

Region

Inter-campus BackboneInter-campus Backbone

RUNet 3


MANMAN MANMAN

MANMAN MANMAN

Inter-region Backbone

Autonomous System

Inter-region BackboneInter-region Backbone

RUNet 1


CharacteristicsCharacteristics

• Geographic independence

• Shallow topology

• Similar (not optimal) paths

• Low latency

• Uniform characteristics

• 1 autonomous system


Collect DataCollect Data

• Netflow

• Source/Destination address

• Source/Destination ports

• Protocol

• Packets/Octets/Flows

• Start/End time


Raw DataRaw Data

• 10 minute granularity

• Each source

• Each destination

• 1,000,000 addresses

• 10,000,000 records

• 1 Gigabytes, 1 day


Rollup DataRollup Data

• Rutgers sources/sinks

• Data >= 1024, 10 minutes

• Data >= 6*1024, 1 hour

• Data >= 24*6*1024, 1 day

• 20,000 unique hosts

• 20,000 records

• 1 Megabyte


Filtered DataFiltered Data

• Rutgers sources/sinks

• Data >= 512 Megabytes, 1 Day

• 125 unique hosts

• 125 records

• 50 Kilobytes


ReductionReduction

10,000,000 99.799%20,000 0.200%125 0.001%

10,020,125

Addresses

1,000,000 98.027%20,000 1.961%125 0.012%

1,020,125

Records1,073,741,824 99.898%

1,048,576 0.098%51,200 0.005%

1,074,841,600

Size

1,000 90.090%100 9.009%10 0.901%

1,110

Model


DistributionDistribution

• Reread entire data set

• Limit to filtered only

• Rollup based on external address

• Preserve individual distributions

• Useful to reduce contact


Questionable DistributionQuestionable Distribution


Good DistributionGood Distribution


Storage

Process ModelProcess Model

Rollup

Internet

NetflowFilterDistribution

Analyze


Residence AssumptionsResidence Assumptions

• RFC1918 address space

• Large number of hosts

• Small number of problems

• Service consumers

• No service providers

• Unresponsive community


Set LimitsSet Limits

• 2048 MB download

• 512 MB upload

• 7 day granularity

• Sliding window

• Enforcement


ReferenceReference

• 4 movies

• 400 songs

• 45,000 web pages

• 2048 Megabytes


Oracle

Process ModelProcess Model

Table

Rollup

Table

Enforce

Table

GatherInternet

Netflow

WWW

Custom ACL


Traffic ShapingTraffic Shaping

• 1 Day on

• 7 Days off

• Multiplexed

• 1:8 ratio

• Automatic

• Aggregated

• Not legalistic

Load

Impact


Differentiated ServiceDifferentiated Service

• Residence facilities

• Other locations

• Two traffic classes

• 1:2 host distribution

• 1:1 bandwidth allocation

• CAR enforced


ResultsResults

Some pains, some gains!


Extra EffortsExtra Efforts

• Registration

• Port Address Translation

• Split horizon DNS

• Help desk/Appeals

• Address hopping

• Proxy services

• Oracle


90% Data Sinks90% Data Sinks


99.99% Data Sinks99.99% Data Sinks


90% Data Sources90% Data Sources


99.99% Data Sources99.99% Data Sources


Internet TrafficInternet Traffic


ConclusionsConclusions

• Modest applications with broad demographics have profound impact.

• Students have free time.

• Network best practices never more important.

• Cooperative generic methods can be effective (w/ encouragement).

• No magic bullet.


Questions?

[email protected]

Documents

University Computing Services EDUCAUSE Mid-Atlantic Regional Conference 16 January 2003 An Infrastructure and Accounting Response to Peer to Peer Traffic