Internet2 QBone: Building a Testbed for IP Differentiated Services INET99 June 23 rd, 1999 San Jose, California Ben Teitelbaum

Internet2 QBone:Building a Testbed for IP Differentiated Services

INET99June 23rd, 1999

San Jose, California

Ben Teitelbaum<[email protected]>

ISOC INET99 Conference - San Jose, CA, June 23, 1999

Internet2 Dogma:There is a circularity between advanced networks and advanced apps

NetworkedApplications

NetworkEngineering

Enables

Motivate


QBone Dogma Article1:Inverse Apps Networking circularity has applied to QoS

QoS-needyApplications

NetworkQoS

Inhibited

Prevented


QBone Dogma Article2:Work with the neediest apps, build a testbed, and turn the arrows around!

QoS-needyApplications

NetworkQoS

Enables

Motivates


Internet2 QBone Initiative Build interdomain testbed infrastructure

– Balance networking research with providing a service– Experiment and improve understanding of DiffServ– Iterate and improve testbed design

Support intradomain & interdomain deployment Lead and follow IETF standards work

– Some parts of DiffServ architecture cooked; others far from it– Our experience will inform standards process

Openness of R&E community gives us an edge– We can live with somewhat flaky infrastructure– We are open to sharing implementation experiences and measurement

data


Internet2 QoS Working Group

– Osama Aboul-Magd (Nortel)– Andy Adamson (Michigan)– Grenville Armitage (Lucent) – Steve Blake (Torrent)– Scott Bradner (Harvard) – Scott Brim (Newbridge)– Larry Conrad (Florida State)– John Coulter (CA*net2) – Chuck Song (MCI/vBNS) – Fred Baker / Larry Dunn

(Cisco) – Rüdiger Geib (Deutsche

Telekom)

– Terry Gray (U Washington) – Jim Grisham (NYSERNet)– Roch Guerin (Penn)– Susan Hares (Merit) – Joseph Lappa (CMU)– Jay Kistler (FORE)– Klara Nahrstedt (UIC)– Kathleen Nichols (IETF coordination) – Ken Pierce (3com) – John Sikora (ATT Labs)– Ben Teitelbaum (chair)– John Wroclawski (MIT)– a liaison from each MOU partner


Internet2 Applications Qualitative and quantitative

improvements in how we conduct research, teaching, and learning

Require advanced networks Examples:

– Interactive research collaboration and instruction

– Real-time access to remote scientific instruments– Large-scale, multi-site computation

and database processing– Shared virtual reality


Big Problem #1: Understanding Application Requirements What services do tomorrow’s applications

need? Range of poorly-understood needs

– Both intolerant apps (e.g. tele-immersion) and tolerant apps (e.g. large FTPs, desktop video conferencing) important

– Many apps need absolute, per-flow QoS assurances

– Adaptive apps may require a minimum level of QoS, but can exploit additional network resources if available

– Some institutions/users want multiple classes of best-efforts service (CoS) with relative precedence levels

Good

Bad

Intolerant

Better

Tolerant

Adaptive

Need better understanding through experience

Different App Needs


Big Problem #2: Scalability

Convergence of flows on the core means:– Large numbers of flows through each router– High forwarding rate requirements

Need to support QoS end-to-end, but keep per-flow state & packet forwarding overhead out of the core

Lots of

flows

here!


Big Problem #3: Interoperability

CampusNetworks

GigaPoPs

GigaPoPs

CampusNetworks

… and between multipleimplementations of network elements ...

Backbone Networks(vBNS, Abilene, …)

... between separately administered and designed clouds ...

… is crucial if we are to provide end-to-end QoS.


DiffServ for Internet2 July 1997 - February 1998

– WG struggled to understand needs of advanced applications / realities of QoS engineering

– Frustrations with RSVP give birth to IETF DiffServ

May 1998– WG recommends EF/Premium DiffServ focus for I2 QoS– First Internet2 Joint Applications/Engineering Workshop,

Santa Clara, CA (report on web site)

October 1998– QBone initiative launched


DiffServ Overview Exploits edge/core distinction for scalability Applications contract for specific QoS profiles

– Policing at network periphery– A few simple, differentiated per-hop forwarding behaviors

(PHBs) Indicated in packet header Applied to PHB traffic aggregates

– PHBs + policing rules = range of services

Clouds contract for aggregate QoS traffic profiles– Policing at cloud-cloud boundary– Supports simple, bilateral business agreements


DiffServ Architecture

Leaf Router (police, mark flows)

Ingress Edge Router (classify, police, mark aggregates)

EgressEdge Router

(shape aggregates)

Corerouters

Corerouters

Bandwidth Brokers(perform admissions control, manage network resources,

configure leaf and edge devices)

BB BB

Source

Destination


Example Service #1: Premium Contract: leased line emulation at a

specified peak rate PHB = “forward me first” (EF) Policing rule = drop out-of-profile packets On egress, clouds must shape EF aggregates to

mask induced burstiness


Why Premium First? Simplest absolute service to understand Strongest flavor of DiffServ

– Could support our most demanding applications– Less demanding applications should work fine on

emerging high-performance BE infrastructure

Explore other PHBs (AF) later To understand this, consider “typical” Internet2 performance:

– I2 networks are largely uncongested– Jitter and loss still occur– Route flaps to the commodity Internet still occur

Prem

ium

Assured

Olym

pic (CoS

)


Typical 1999 Internet2 Performance

East Coast University to West Coast DOE Lab

• Minimum Delay• 50th Percentile Delay• 90th Percentile Delay


......

Initial QIG** 11 February 1999 (actual connectivity and participating networks may vary as deployment progresses)

OtherNGIXs

ESNet NREN

Abilene

UMass

UMN

vBNS

MAGPI NYSERNetTexas GP PSCNCNI

CA*Net2

APAN

SingAREN SURFNet

Merit

UMichUPennCMUTAMU Duke NCSU UNC

CTIT

IU

EVLiCAIRNWUUBC

ARDNOC

NTU

NUS

LBNL

AmesANL

KDD Labs

Korea

RISQ

CRC UNB

MREN /STAR TAP

Other DOE Labs

Other NASA Labs


QBone BB Group Open group chartered to recommend BB solutions for the

QBone Lead by Sue Hares - Merit Networks Six R&E proto-BBs:

– Merit– BCIT – UCLA

Extensive participation from corporate partners QBone BB requirements draft on web site Prototype inter-BB signaling protocol due soon

– Telia / Luleå University of Technology– ANL/Globus– LBNL Clipper


QBone Milestones 1998 - 1999 Sep 25th - Call for participation Oct 30th - WG recommends initial QIG participants Dec 1st - 1st QIG / QBone BB meeting (Evanston) Jan 1st - WG makes major push on architecture draft Jan 26th - 2nd QIG / QBone BB Meeting (RTP) Mar 7th - Measurement sub-group drafts QMA Mar 9th - 3rd QIG / QBone BB Meeting (Las Cruces) May 21st - WG opens QIG June 8th - Open QBone interop BOF (Pittsburgh)


QBone Architecture (10km view) IETF “Diff” (EF PHB) + QBone “Serv” (QPS) QBone Premium Service

– Idea: converge on Jacobson’s VLL “Premium” service– Well-defined SLS:

Peak rate R & “Service MTU” M implying a token bucket meter Near-zero loss Low jitter

– Delay variation due to queuing effects should be no greater than the packet transmission time of a service MTU sized packet

– All bets are off if the reserved interdomain route flaps

Plus important value-adds:– Integrated measurement/dissemination infrastructure– Experimentation with pre-standards inter-domain bandwidth

brokering and signaling


QBone Measurement Architecture1

Collection metrics, EF and BE...– Active metrics

One-way delay-variation One-way loss Traceroutes e.g. IPPM Surveyors

– Passive metrics Load Discards (suggested) Link bandwidths (suggested) EF reservation load e.g. OCxMon, RTFM, MIBs

Boundary Router

Intra-Domain Premium Path

Inter-Domain Premium Path

ActiveMeasurements

MIB-basedstatistics

PassiveMeasurements

PassiveMeasurements

PM node PM node

AM node

QBone Domain2

QBone Domain1

QBone Domain3


QBone Measurement Architecture2

Dissemination– HTTP, even for raw data– real-time + archived measurements– Canonical names for:

Metrics Domains

– Standard metric aggregations: Mostly 5-minute aggregations

– Standard URL name space for: MRTG-style plots Raw ASCII data http://<root_URL>/<source_domain>/<dest_domain>/<first_hop>/<date>/<type>.<aggregation>.{html | gif | txt}


Starting Simply Intradomain

– Van’s campus example – At least 10Mbps everywhere– “Count to ten” admissions control

with no topological knowledge

Interdomain– Could we do something

similar in the early QBone?– Problem: Worst case

down-stream provisioning starts to look pretty bad even with small initial participant set.

H

HH

H

GigaPoP

GigaPoP

... ...


Generic Internet2 Topology

NGIXs

CC

GigaPoP

C C

GigaPoP

C C

GigaPoP

C C

Abilene

CC

GigaPoP

C C

GigaPoP

C C

GigaPoP

C C

vBNS

L

C

ESNet, NREN, Int’l, ...

L

C


ESNet, NREN, STARTAP, ...

Abilene

GigaPoP GigaPoP GigaPoP

GigaPoP GigaPoP GigaPoP

vBNS

Phase 0 Demand Assessment

NGIXs

CC

C C C C C C

CC

C C C C C C

Int’l

L

C

L

C


Phase 0 Deployment Planning Converge on a consensus reservation matrix Reservations will be static for period of phase Reservation = {S, D, R, M, TR}

– S = source– D = dest– R = peak rate

S, D are on campus network demarks All bets are off if routing between S and D changes All SLSs still bi-lateral, but Internet2 engineering will

facilitate convergence

– M = service MTU– TR = inter-domain traceroute


Phase 0 Demand Matrix

C1 C2 C3 C4 C5 L1 L2

4 6 4 2 12 2

C1 - 2 2

C2 2 - 2

C3 - 10

C4 2 - 2

C5 -

L1 2 2 -

L2 2 2 -

Campus EF Ingress Load

R

D

Campus Policer Config

Implies

Max

imum

EF

load

to b

e of

fere

d fr

om h

ere

… to here


Coming Attractions... Jun 99: QBone Architecture in last call Jun 99: QBone BB Advisory Council will recommend

a prototype inter-BB protocol Jun/Jul 99: “Phase 0” rollout planning Aug/Sep 99: Interdisciplinary QBone workshop Fall 99: QBone Connect-a-thon (“QCon”) event Fall 99: “Phase 0”


For more information... QBone home page:

http://www.internet2.edu/qbone Internet2 QoS Working Group home page:

http://www.internet2.edu/qos/wg

Documents

Internet2 QBone: Building a Testbed for IP Differentiated Services INET99 June 23 rd, 1999 San Jose, California Ben Teitelbaum