Upload
duane
View
31
Download
1
Tags:
Embed Size (px)
DESCRIPTION
ATLAS Tier 2 Paths Within ESnet. Mike O’Connor ESnet Network Engineering Group Lawrence Berkeley National Lab [email protected]. ATLAS Tier 2 Connectivity. Typical ATLAS tier 2 site connectivity should be via Abilene and ESnet in order to reach Brookhaven Lab. - PowerPoint PPT Presentation
Citation preview
ATLAS Tier 2 Paths Within ESnet
Mike O’Connor
ESnet Network Engineering Group
Lawrence Berkeley National Lab
ATLAS Tier 2 Connectivity
• Typical ATLAS tier 2 site connectivity should be via Abilene and ESnet in order to reach Brookhaven Lab.
• “ESnet’s Goal is that DOE Lab ↔ Univ. connectivity should be as good as Lab ↔ Lab and Univ. ↔ Univ.” W. E. Johnston
ATLAS Tier 2 Support• The role of ESnet is to provide networking that supports and
anticipates the needs of the Office of Science Labs and their collaborators.
• ESnet support for ATLAS tier 2 site connectivity makes high bandwidth and fault tolerant paths between BNL and Abilene an absolute priority.
• Planning is underway to upgrade BNL’s ESnet OC48 circuit to a multi 10G lambda fault tolerant Metropolitan Area Network (MAN). (Targeted for Q1 2006)
• Today ESnet and Abilene peer in Atlanta, Chicago, New York and Sunnyvale with a combined bandwidth of 18.2Gbps.These four locations comprise a high bandwidth and fault tolerant interconnect between these two networks.
ES
ne
t S
cie
nc
e D
ata
Ne
two
rk (
SD
N)
co
re
LVMR-NOC
SNLL
YUCCA MT
BECHTEL-NV
PNNLLIGO
INEEL
LANL
SNLAAlliedSignal
PANTEX
ARM
KCP
NOAA
OSTI ORAU
SRS
JLAB
PPPLLab DCOffices
MIT
ANL
BNL
FNALAMES
LLNL
GA
DOE-ALB
OSC GTNNNSA
International (high speed)10 Gb/s SDN core10G/s IP core2.5 Gb/s IP coreMAN rings (≥ 10 G/s)OC12 ATM (622 Mb/s)OC12 / GigEthernetOC3 (155 Mb/s)45 Mb/s and less
Office Of Science Sponsored (22)NNSA Sponsored (12)Joint Sponsored (3)
Other Sponsored (NSF LIGO, NOAA)Laboratory Sponsored (6)
42 end user sites
ESnet IP core
SINet (Japan)Russia (BINP)CA*net4
FranceGLORIAD (Russia, China)Korea (Kreonet2
Japan (SINet)Australia (AARNet)Canada (CA*net4Taiwan (TANet2)Singaren
ESnet IP core: Packet over SONET Optical
Ring and Hubs ELP
ATL
DC
commercial and R&E peering points
MAE-E
PAIX-PAEquinix, etc.
PN
WG
Po
P/
PA
cifi
cWav
e
SEA
ESnet Provides Global High-Speed Internet Connectivity forDOE Facilities and Collaborators (Spring, 2006 Configuration)
ESnet core hubs IP
Abilenehigh-speed peering pointswith Internet2/Abilene
Abilene
Ab
ilen
e
CERN(USLHCnet
CERN+DOE funded)
GÉANT - France, Germany, Italy, UK, etc
NYC
StarlightChi NAP
CH
I-SL
SNV
Ab
ilene
SNV SDN
JGI
LBNL
SLACNERSC
SDSC
Equinix
MA
N L
AN
Ab
ilen
e
MAXGPoP
SoXGPoP
SDN
ALB
ORNL
CHI
MRENNetherlandsStarTapTaiwan (TANet2, ASCC)
NA
SA
Am
es
AU
South AmericaWest South America
East
international R&E peers
AU
NREL
ESnet AS 293
Abilene AS 11537
10 Gbps 2.4 Gbps 1 Gbps
Atlanta GASunnyvale CA Chicago IL New York NY
ESnet & Abilene Peering points
*paths do not include all router hops**Example site logos chosen at random.
ESnet AS 293
Abilene AS 11537
Atlanta GASunnyvale CA Chicago IL New York NY
Abilene Interior Gateway Protocol (IGP) mileage based metrics determine the best exit path toward the ESnet. This common method also known as “hot potato routing” it will always compute an exit path to the nearest peering point.
Abilene Routing Toward ESnet
ESnet AS 293
Abilene AS 11537
Atlanta GASunnyvale CA Chicago IL New York NY
ESnet Routing Toward Abilene
Abilene Multi Exit Discriminators (MEDs) are used by ESnet to choose the best exit path toward Abilene. These MEDs are based on the same IGP metrics used in the interior of their network. This creates a symmetric path back to Abilene.
32 Avenue Of the Americas NYC
ESnet PeeringRouter
MANLAN
ESnet CoreAOA (NYC)
Abilene
OC48DC
OC192CHI
1GE
10GE
10GE
2GE
Peerings@MANLANnysernet 3754abilene 11537sinet 2907geant 20965
BNLOC48
The MANLAN peering is the primary path from ESnet to Abilene at AOA. The directly connected 1Gbps peering is a secondary failover connection.
Chicago
ESnet Starlight
ESnet CoreChicago
Abilene
Peerings@Starlight ANL 683 ASCC 9264 CANET 6509 CERN 513 DREN 668 FNAL 3152 GLORIAD 5568 KREONET 17579 MREN 22335 NASA 297 NWU 103 OPENTRANSIT 5511 SINET 2907 TANET 7539 UIUC 38 ULTRALIGHT 32361 USLHCNET 1297 UWMADISON 59 WISCNET 2381
OC192NY
OC192CA
OC48
10GE
Abilene is the only external peer connected directly to the ESnet Core router in Chicago.
ESnet Starlight
ESnet CoreChicago
Abilene
OC192NY
OC192CA
OC48
10GE
1.9 Gbps peak19% Load
0.8 Gbps peak 33% Load
Current Load in Chicago
Sunnyvale & Atlanta
ESnet and Abilene have direct OC48 peerings in Atlanta GA and Sunnyvale CA.
ESnet CoreSunnyvale CA
Abilene
OC192CHI
OC48
ESnet CoreAtlanta GA
Abilene
OC48
OC48DC
OC48
El PasoTX
OC48
Future Directions
• Long Island MAN Ring
• ESnet Lambda Infrastructure
MANLANAbileneGEANT
Manhattan Long IslandIP Core to Chicago
IP Core to DC
111th & 8th
BNL
60 Hudson
32 AOA
λ2 SDN/circuit
λ1 production IP
10 Gb/soptical
channels
• 2 λs (10 G circuits) delivered as two 10 GigEther circuits
-8 X current BNL bandwidth
• Physically diverse fiber paths forming a ring topology.
• Will be used as a10 Gb/s production IP and a10 Gb/s path for SDN circuit services
• Services switch over to the other service path during a circuit failure.
ESnet LI MAN Ring
Circuits based on WDM rings
*SDN – Science Data Network
ESnet 2010 Lambda Infrastructure and LHC T0-T1 Networking
Denver
Seattle
Su
nn
yv
ale
LA
San Diego
Chicago
Pitts
Raleigh
Jacksonville
Atlanta
KC
Baton Rouge
El Paso - Las Cruces
Phoenix
Pensacola
Dallas
San Ant.Houston
Albuq. Tulsa
New YorkClev
Boise
CE
RN
-1G
ÉA
NT
-1G
ÉA
NT
-2
Wash DC
CE
RN
-2
Tier 1 Centers
ESnet IP core hubs
New hubs
ESnet SDN/NLR hubsESnet Production IP core (10-20 Gbps)ESnet Science Data Network core (10G/link)(incremental upgrades, 2007-2010)Other NLR linksCERN/DOE supplied (10G/link)International IP connections (10G/link)
Cross connects with Internet2/Abilene
NLR PoPs
TRIUMF
FNAL
BNL
CE
RN
-3
Vancouver
Toronto
CANARIE
Conclusions
• ESnet provides an infrastructure and services that are critical to ATLAS science
• ESnet is well positioned to provide essential network services to the LHC tier 2 community
• ESnet is working on providing the DOE mission science networking requirements with several new initiatives and a new architecture