1

CHEETAH Network

Xuan Zheng & Malathi Veeraraghavan{xz3y, mv5g}@virginia.edu

Sept. 11, 2006

2

Raleigh PoP (MCNC)

Controlcard

GbE/10GbEcard

ORNL PoP

Controlcard

GbE/10GbEcard

SN16000

Controlcard

OC192card

Via Nysernet/HOPI

OC-192

GbE/10GbEcard

End hosts

Atlanta PoP (SOX/SLR)

SN16000 SN16000

G. Tech

End hosts

End hosts

ORNL

OC192card

NCSU

UVa

CUNY

OC192card

OC-192 (via NLR/SLR/NCREN)

via NCREN

Via Vortex/HOPI

CHEETAH Network Overview

3

Orbitty Cluster

1GOC192 OC192 GbE

1-8-331-8-34

1-8-35

1-8-36

1-6-1

1-6-171-8-37

MCNCCatalyst

6500

1G

1-7-1

1G

1G1G

1G

Wukong 152.48.249.102

1-8-381-7-17

cheetah-nc

3x1G VLAN

OC192

1-6-1

1-6-17

10GbE

1-7-1

GbE

1-7-33

1-7-34

1-7-35

1-7-36

1-7-37

1-7-38

1-7-39

1GZelda1 10.0.0.11

H

H

H1G

1GZelda2 10.0.0.12

Zelda3 10.0.0.13

1G

1G

Zelda4 10.0.0.14

H

H

Zelda5 10.0.0.15

2x1G MPLS tunnels

1G1G

Cheetah-atl

OC-192 lamda

10GbEGbE

1-7-33

1-7-34

1-7-35

1-7-36

Cheetah-ornl

1-7-1 1-8-1

OC192

X1(E)UCNS1GFC1G

1G

JuniperT320

JuniperT320

1G

1G

Force10E300

switch

ORNL PoP

Atlanta PoP

NCSU

Direct fibers

VLANs

MPLS tunnels

Wuneng 152.48.249.1031-8-39

H

UVa Catalyst

4948WASHHOPI

Force10

WASHAbileneT640

NCSUM20

2x1G MPLS tunnels

CUNYFoundry

NYCHOPI

Force10

1G

1GMvstu6 152.48.249.106

H

H

1G

UVa

CUNY

CHEETAH Network (data-plane)

OC-192 lamda

HCentuarFastIron

FESX448

H

Raleigh PoP

cuny-h1 134.74.17.112

H

1G

cuny-h2 134.74.17.113

4

NC configuration – Data Plane

CentuarFastIron

FESX4481G

NCSU

MCNC

Orbitty Compute Nodes152.48.249.2-9

1GOC192 OC192 GbE

1-8-33

1-8-34

1-8-35

1-8-36

1-6-1

1-6-17

Direct fibers

1-8-37

OC-192 to1-6-1 oncheetah-atl

MCNCCatalyst

6500

1-7-1

1G

1G

Wukong 152.48.249.102

1-8-38

1-7-17

VLAN connections

cheetah-nc

3x1G VLANs

H

Wuneng 152.48.249.103

1-8-39

1G

H1G

To mvstu6

10G

To CUNY

HHH

1GViz Nodes152.48.249.30-33

HHH

5

CHEETAH NCSU VLAN Plan

Cheetah VLAN setup - MCNC to Centaur

VLAN IDCentaur FNNI-01

MCNC 6500MCNC

SN16000Machine

Name

4084 3/12 Gig1/9 1-8-33Orbitty/Viz

cluster/wukong

4080 3/13 Gig1/5

4081 3/14 Gig1/6

4082 3/15 Gig1/7 1-8-36 mvstu6

4083 3/16 Gig1/8 1-8-37 CUNY

6

NC configuration – Control Plane

NCSU

MCNC

Orbitty Compute Nodes 128.109.45.160-167

Orbitty compute nodes

OC192 OC192 GbE

1-6-1

1-6-17

1-7-1

Wukong 128.109.45.180

1-7-17

Ethernet port

128.109.34.18

192.168.4.2

Internet

Internet

Control

NS-5

1-8-33

1-8-34

1-8-35

1-8-36

1-8-37

1-8-38

128.109.34.19

Internet

Internet

cheetah-nc

OC-192 TE link to cheetah-atl

Router ID=switch IP =192.168.5.1

1-8-39

Wuneng 128.109.34.221Gbps

TE link

1Gbps TE link to CVLSR-1

1Gbps

H

1Gbps TE link to mvstu6

HHH

H

7

GbE

1-7-33

1-7-34

1-7-35

1-7-36

1-7-37

1-7-38

1-7-39

Atlanta configuration – Data Plane

OC192

1-6-1

1-6-17

10GbE

1-7-1

1GZelda1 10.0.0.11

H

H

H1G

1GZelda2 10.0.0.12

Zelda3 10.0.0.13

Atlanta

ORNL

1G MPLS tunnel 1

to 1-7-36 oncheetah-ornl

GaTech

1G

1G

Direct fibers

To GaTech

2x1G

MPLS tunnels

cheetah-atl

JuniperT320

1G MPLS tunnel 2

to 1-7-33 oncheetah-ornl

JuniperT320

OC-192 to 1-6-1 on cheetah-nc

OC-192 to 1-8-1 on cheetah-ornl

8

OC192

1-6-1

1-6-17

10GbE

1-7-1

GbE

1-7-33

1-7-34

1-7-35

1-7-36

1-7-371-7-38

1-7-39

Zelda1 130.207.252.131

H

H

H

Zelda2 130.207.252.132

Zelda3 130.207.252.133

Atlanta configuration – Control Plane

Control

NS-5

Ethernet port

130.207.252.138

130.207.252.136192.168.2.2

3x1Gbps

Internet

Internet

OC-192 TE link to cheetah-nc

cheetah-atlRouter ID=switch IP=192.168.3.1

1Gbps TE link to 1-7-36 on cheetah-ornl

1Gbps TE link to 1-7-33 on cheetah-ornl

TE link

OC-192 TE link to cheetah-ornl

9

ORNL configuration – Data Plane

OC192

1-8-1

10GbE

1-7-1

GbE

1-7-33

1-7-34

1-7-35

1-7-36

cheetah-ornl

Force10 E300

1GZelda4 10.0.0.14

H

H1G

Zelda5 10.0.0.15

JuniperT320

1GbE MPLS tunnel 1

to 1-7-36 oncheetah-atl

1GbE MPLS tunnel 2

to 1-7-37 oncheetah-atl

1G

1G

1G

UCNS1G X1(E)1G FC

1-7-1

Direct fibers

MPLS tunnels

0/50/20

VLAN connections

OC-192 to1-6-17 oncheetah-atl

10

OC192

1-8-1

10GbE

1-7-1

GbE

1-7-33

1-7-34

1-7-35

1-7-36

cheetah-ornl

1-7-1

Router ID=switch IP =10.1.0.1

Control

NS-510.1.1.2

Internet

Internet

ORNL configuration – Control Plane

Force10 E300

Zelda4 198.124.42.17

H

H

Zelda5 198.124.42.18

JuniperT320

1Gbps

1Gbps

1Gbps TE link to 1-7-36 on cheetah-atl

1Gbps TE link to 1-7-37 on cheetah-atl

198.124.42.3

TE link

198.124.42.19

OC-192 TE link to cheetah-atl

11

UVa/CUNY configuration – Data Plane

UVa Catalyst 4948

WASH HOPIForce10

47/48

1GH

UVaCatalyst

6509

McleanCatalyst

760010G 10G10G

2x1G MPLS tunnels

Direct fibers

MPLS tunnels

MCNCCatalyst

6500

1G

10G1G

VLAN connections

UVa 1G VLAN(4082)

NY HOPIForce10

CUNY FoundryCVLSR-1

1G

1G

CUNY 1G VLAN(4083)

WASH Abilene T640

15/16

17/18

19/20

21/22

23/24

mvstu6 152.48.249.106

…

OC192 OC192 GbE

1-8-331-8-34

1-8-35

1-8-36

1-6-1

1-6-17 1-8-37

OC-192 to1-6-1 oncheetah-atl

1-7-1

1-8-38

1-7-17

cheetah-nc1-8-39

NCSUM20

CentuarFastIron

FESX448

H

H

1G

cuny-h1 192.168.183.112

cuny-h2 192.168.183.1131G

10G

1G

10G

Check the next two pages for more details about UVa/CUNY-MCNC connections

12

WASH-MCNC Connections

13

CUNY-WASH Connection

Hunter695 Park Av

BMCC199 Chambers St

Grad Ctr.365 5th Ave

Baruch55 Lex

32 Ave of A

14.4Km

9.16Km

26.85Km

4.22Km

5.42Km 5.0Km

3.94Km

4.92Km

Physical Fiber Ring

RING 1

RING 2

CIS1 555 West 57th St

Internet 1

Internet 1

1.96Mi

1.62Mi

CISDual Homed Apps

Blackboard etc.

OC-3

L2/L3

1 GE (24 STSs) Backup for Data Center

2 RPR Rings for Internet 1 & 2

CCNYConvent Ave

& 138th St

RING 2 SONET OC-48

5 RPR Rings for the 5 Colleges

RING 1 DWDM

Blue - Physical Fiber Ring with SONET overlayOther Colors - Logical Resilient Packet Rings

CIS2 555 West 57th St

RSVPApps

RSVPApps

GE

GE

GE

GE

CUNY

1118th Ave

Internet 1

Internet 1

OC-3

GE Ckt Path:Ring 1: GE/SONET/ DWDMCross-connected betweenCIS1 & CIS2 at West 57th St.,at Layer 2Ring2: SONET/GE

CollegeNetwork

14

UVa-Dragon-MCNC VLAN PlanCheetah VLAN setup – UVa – Mclean

Cisco 4948 port(UVa Small Hall) VLAN ID

15/16 4080

17/18 4081

19/20 4082

21/22 4083

23/24 4084

25/26 3000

27/28 3001

29/30 3010

31/32 3011

33/34 3020

35/36 3021

37/38 4001

39/40 4002

41/42 4003

43/44 4004

45/46 4005

47/48 4006

Ports 1-14 are level-3interfaces and are connected to AbileneIPs: 128.143.215.11 - 128.143.215.30Gateway: 128.143.215.1 Netmask: 255.255.255.224

15

UVa/CUNY configuration – Control Plane

MCNC

OC192 OC192 GbE

1-6-1

1-6-17

1-7-1

1-7-17

Ethernet port

128.109.34.18

192.168.4.2

Internet

Internet

Control

NS-5

1-8-33

1-8-34

1-8-35

1-8-36

1-8-37

1-8-38

128.109.34.19

cheetah-ncRouter ID=switch IP =192.168.5.1

1-8-39

TE link

Hmvstu6 128.143.67.213 H

Hcuny-h1

134.74.17.112cuny-h2

134.74.17.113

Internet

Internet

CVLSR-1Router ID=switch IP =192.168.6.5

192.168.6.77

134.74.17.77

NS-5

134.74.17.30

Control GbE

1Gbps

1Gbps

1Gbps

1Gbps

To Dragon VLSR

UVa CUNY

16