Update on National Networks: Status, Application Targets, Challenges & Opportunities

Update on National Networks:Status, Application Targets, Challenges & Opportunities

Charlie Catlett, Senior Fellow, Computation Institute

University of Chicago and Argonne National Laboratory

Director, NSF TeraGrid Initiative

Charlie Catlett ([email protected])

Office of Science US Community

Universities supported by SC

Major User Facilities

DOE Multiprogram Laboratories

DOE Program-Dedicated Laboratories

DOE Specific-Mission Laboratories

SC program sites

Pacific NorthwestPacific NorthwestNational LaboratoryNational Laboratory

Ames LaboratoryAmes Laboratory

Argonne National Argonne National LaboratoryLaboratory

BrookhavenBrookhavenNationalNational

LaboratoryLaboratory

Oak RidgeOak RidgeNational National

LaboratoryLaboratoryLos AlamosLos Alamos

National National LaboratoryLaboratory

Lawrence Lawrence LivermoreLivermoreNational National


LawrenceLawrenceBerkeley Berkeley NationalNational


FermiFermiNationalNational

Accelerator Accelerator LaboratoryLaboratory

PrincetonPrincetonPlasmaPlasmaPhysicsPhysics


Thomas Jefferson Thomas Jefferson National National

Accelerator FacilityAccelerator Facility

NationalNationalRenewable Energy Renewable Energy


StanfordStanfordLinearLinear

Accelerator Accelerator CenterCenter

Idaho National Idaho National LaboratoryLaboratory

General General AtomicsAtomics

SandiaSandiaNational National

LaboratoriesLaboratories

Bill Johnston, ESNET


TeraGrid PI’s

Blue: 10 or more PI’sRed: 5-9 PI’sYellow: 2-4 PI’sGreen: 1 PI

TeraGrid PI’s By Institution as of May 2006


TeraGrid Science Gateways Initiative:Community Interface to Grids

• Common Web Portal or application interfaces (database access, computation, storage, workflow, etc).

• “Back-End” use of TeraGrid computation, information management, visualization, or other services.

• Standard approaches so that science gateways may readily access resources in any cooperating Grid without technical modification.


TG-SGW-Partners

21 Science Gateway Partners (and growing) - Over 100 partner Institutions

TeraGrid Science Gateway Partner Sites


Some Observations

While many challenges and concepts are not new…

– The need for huge pipes remains constant; the definition of huge continues to grow

– There is a large dynamic range of bandwidth required between average and peak - this too is not new, but the prefixes change (Mb->Gb->TB…)

– Switched circuits are older than the Internet, certainly not new, but the names and the bandwidth both change (POTS, ATM SVC, switched lambdas…)

The nature of applications really does seem to be changing

– Service oriented architectures

– Web services

– More direct control in the hands of application developers… for everything but the network…


Examples

Many Flows leading to huge loads: YouTube

– After one year, YouTube drives 20 Gb/s

– Traffic growth is at 20% per monthWeb Services enabling growth driven by embedding services within applications

– Amazon S3 - many clients, can write your own (it is a web service), and supports bittorrent

We will “big brother” ourselves

– More video…larger email attachments!


Driver for user controlled networks

Increasingly more and more organizations are acquiring their own fiber networks

– Universities, schools, hospitals, businessAcquiring fiber in the long haul is very expensive to light and obtain

– Alternative is to use “dim fiber” –point to point wavelengths

– But want flexibility to do configuration and change management as with dark fiber

Increasingly science needs dedicated networks for specific applications and disciplines for high data volume grids

– Want to be able to manipulate the network in the same way they can manipulate the application

Bill St. Arnaud (CANARIE)


Application Requirements Driving ESnet Architecture



Next Generation ESnet

Architecture and capacity is driven by the SC Program requirementsMain architectural elements and the rationale for each element

– 1) A High-reliability IP core (e.g. the current ESnet core) to address• General science requirements, Lab operational requirements, Backup for

the SDN core, Vehicle for science services, Full service IP routers– 2) Metropolitan Area Network (MAN) rings to provide• Dual site connectivity for reliability, Much higher site-to-core bandwidth,

Support for both production IP and circuit-based traffic, Multiply connecting the SDN and IP cores• 2a) Loops off of the backbone rings to provide for dual site connections

where MANs are not practical– 3) A Science Data Network (SDN) core for• Provisioned, guaranteed bandwidth circuits to support large, high-speed

science data flows, Very high total bandwidth, Multiply connecting MAN rings for protection against hub failure, Alternate path for production IP traffic, Less expensive router/switches, Initial configuration targeted at LHC, which is also the first step to the general configuration that will address all SC requirements, Can meet other unknown bandwidth requirements by adding lambdas


NLR owned fiber* NLR WaveNet, FrameNet & PacketNet PoPNLR WaveNet & FrameNet PoPNLR WaveNet PoP

PoP for primary connection point by a member (“MetaPoP”)PoP needed because of signal regeneration requirementsbut can also be used for secondary connection by a memberPoP established by NLR for members regional needsPoP established at exchange points

RATO

BATO

SAND

STAR

CHICSALT

HOUS

DALL

SYRA

TULS

PENS

ELPA

KANS

PHOE

LOSA

ALBU

PITTWASH

BOIS

CLEV

ATLA

PORT

RALE

NEWY

SANA

DENV

SUNN

SEAT

JACK

OGDE

* Fiber on the SAND-LOSA-SUNN path belongs to CENIC

% used/Free

5%/19

15%/17

60%/8

23%/31

23%/31

23%/31 30%/28 35%/26

56%/14

40%/12

25%/15

35%/13

25%/3020%/16

25%/15

10%/18

15%/34

15%/27

19%/26

19%/26

15%/2715%/2719%/2619%/26

19%/2622%/25

22%/25

25%/2425%/24

25%/24

25%/2425%/24

19%/26

NLR Allocated Waves

Tom West, NLR

Charlie Catlett ([email protected])Rick Summerhill

Layer 1 Topology with IP Network

PROVISIONAL TOPOLOGY – SUBJECT TO DISCUSSION

Charlie Catlett ([email protected])Bill Johnston, ESNET

Virtual Circuit Network Services

A top priority of the science communityToday

– Primarily to support bulk data transfer with deadlinesIn the near future

– Support for widely distributed Grid workflow engines

– Real-time instrument operation

– Coupled, distributed applications To get an idea of how circuit services might be used to support the current

trends, look at the one year history of the flows that are currently the top 20

– Estimate from the flow history what would be the characteristics of a circuit set up to manage the flow

Charlie Catlett ([email protected])Bill St. Arnaud (CANARIE)

CANARIE Approach: UCLP

User Controlled LightPaths – a configuration and provisioning tool built around grid technology using web services

Third party can concatenate cross connects together from various links, routers and switches to produce a wide are network that is under their control

– Articulated Private Network (APN)

– Next generation VPNUses Service Oriented Architecture (SOA) and so network can be integrated with

other web service applications


Extending Networks into Applications

Bill St. Arnaud (CANARIE)


“Life After Internet: Making Room for New Applications” (Larry Smarr and Charlie Catlett) in “Building Information Infrastructure,” ed. Brian Kahin, McGraw-Hill)

Hybrid Networks: A Blast from the Past ( 1991 ! )

Technical Issues– Rapid growth of Internet– New applications increasing bandwidth

requirements and network usage– End-to-end performance

Applications– Distributed computing– Digital libraries– Require peak bursts of bandwidth

several orders of magnitude greater than average traffic

Proposed Architecture• “The Internet architecture should,

wherever possible, combine the advantages of packet switched networks and of virtual circuit networks [to support] the ability to increase dynamically in capacity..."


Conclusions

National Networks will do on demand today, and better on demand next year

– Today in days or weeks - in 2007/8 will do it in minutes

– Today granularity is per site - multiple efforts trying to move to per application• CANARIE UCLP• ESnet OSCARS• I2 HOPI

Issues will remain

– Getting a switched lambda to a desktop is not a national backbone issue• Regional networks• Campus/Lab networks

Maybe we are doing OK!

Documents

Update on National Networks: Status, Application Targets, Challenges & Opportunities