Update on National Networks: Status, Application Targets, Challenges & Opportunities

Update on National Networks:Status, Application Targets, Challenges & Opportunities

Charlie Catlett, Senior Fellow, Computation Institute

University of Chicago and Argonne National Laboratory

Director, NSF TeraGrid Initiative

Charlie Catlett (catlett@mcs.anl.gov)

Office of Science US Community

Universities supported by SC

Major User Facilities

DOE Multiprogram Laboratories

DOE Program-Dedicated Laboratories

DOE Specific-Mission Laboratories

SC program sites

Pacific NorthwestPacific NorthwestNational LaboratoryNational Laboratory

Ames LaboratoryAmes Laboratory

Argonne National Argonne National LaboratoryLaboratory

BrookhavenBrookhavenNationalNational

LaboratoryLaboratory

Oak RidgeOak RidgeNational National

LaboratoryLaboratoryLos AlamosLos Alamos

National National LaboratoryLaboratory

Lawrence Lawrence LivermoreLivermoreNational National

LaboratoryLaboratory

LawrenceLawrenceBerkeley Berkeley NationalNational

LaboratoryLaboratory

FermiFermiNationalNational

Accelerator Accelerator LaboratoryLaboratory

PrincetonPrincetonPlasmaPlasmaPhysicsPhysics

LaboratoryLaboratory

Thomas Jefferson Thomas Jefferson National National

Accelerator FacilityAccelerator Facility

NationalNationalRenewable Energy Renewable Energy

LaboratoryLaboratory

StanfordStanfordLinearLinear

Accelerator Accelerator CenterCenter

Idaho National Idaho National LaboratoryLaboratory

General General AtomicsAtomics

SandiaSandiaNational National

LaboratoriesLaboratories

Bill Johnston, ESNET

Charlie Catlett (catlett@mcs.anl.gov)

TeraGrid PI’s

Blue: 10 or more PI’sRed: 5-9 PI’sYellow: 2-4 PI’sGreen: 1 PI

TeraGrid PI’s By Institution as of May 2006

Charlie Catlett (catlett@mcs.anl.gov)

TeraGrid Science Gateways Initiative:Community Interface to Grids

• Common Web Portal or application interfaces (database access, computation, storage, workflow, etc).

• “Back-End” use of TeraGrid computation, information management, visualization, or other services.

• Standard approaches so that science gateways may readily access resources in any cooperating Grid without technical modification.

Charlie Catlett (catlett@mcs.anl.gov)

TG-SGW-Partners

21 Science Gateway Partners (and growing) - Over 100 partner Institutions

TeraGrid Science Gateway Partner Sites

Charlie Catlett (catlett@mcs.anl.gov)

Some Observations

While many challenges and concepts are not new…

– The need for huge pipes remains constant; the definition of huge continues to grow

– There is a large dynamic range of bandwidth required between average and peak - this too is not new, but the prefixes change (Mb->Gb->TB…)

– Switched circuits are older than the Internet, certainly not new, but the names and the bandwidth both change (POTS, ATM SVC, switched lambdas…)

The nature of applications really does seem to be changing

– Service oriented architectures

– Web services

– More direct control in the hands of application developers… for everything but the network…

Charlie Catlett (catlett@mcs.anl.gov)

Examples

Many Flows leading to huge loads: YouTube

– After one year, YouTube drives 20 Gb/s

– Traffic growth is at 20% per monthWeb Services enabling growth driven by embedding services within applications

– Amazon S3 - many clients, can write your own (it is a web service), and supports bittorrent

We will “big brother” ourselves

– More video…larger email attachments!

Charlie Catlett (catlett@mcs.anl.gov)

Driver for user controlled networks

Increasingly more and more organizations are acquiring their own fiber networks

– Universities, schools, hospitals, businessAcquiring fiber in the long haul is very expensive to light and obtain

– Alternative is to use “dim fiber” –point to point wavelengths

– But want flexibility to do configuration and change management as with dark fiber

Increasingly science needs dedicated networks for specific applications and disciplines for high data volume grids

– Want to be able to manipulate the network in the same way they can manipulate the application

Bill St. Arnaud (CANARIE)

Charlie Catlett (catlett@mcs.anl.gov)

Application Requirements Driving ESnet Architecture

Bill Johnston, ESNET

Charlie Catlett (catlett@mcs.anl.gov)

Next Generation ESnet

Architecture and capacity is driven by the SC Program requirementsMain architectural elements and the rationale for each element

– 1) A High-reliability IP core (e.g. the current ESnet core) to address• General science requirements, Lab operational requirements, Backup for

the SDN core, Vehicle for science services, Full service IP routers– 2) Metropolitan Area Network (MAN) rings to provide• Dual site connectivity for reliability, Much higher site-to-core bandwidth,

Support for both production IP and circuit-based traffic, Multiply connecting the SDN and IP cores• 2a) Loops off of the backbone rings to provide for dual site connections

where MANs are not practical– 3) A Science Data Network (SDN) core for• Provisioned, guaranteed bandwidth circuits to support large, high-speed

science data flows, Very high total bandwidth, Multiply connecting MAN rings for protection against hub failure, Alternate path for production IP traffic, Less expensive router/switches, Initial configuration targeted at LHC, which is also the first step to the general configuration that will address all SC requirements, Can meet other unknown bandwidth requirements by adding lambdas

Bill Johnston, ESNET

NLR owned fiber* NLR WaveNet, FrameNet & PacketNet PoPNLR WaveNet & FrameNet PoPNLR WaveNet PoP

PoP for primary connection point by a member (“MetaPoP”)PoP needed because of signal regeneration requirementsbut can also be used for secondary connection by a memberPoP established by NLR for members regional needsPoP established at exchange points

RATO

BATO

SAND

STAR

CHICSALT

HOUS

DALL

SYRA

TULS

PENS

ELPA

KANS

PHOE

LOSA

ALBU

PITTWASH

BOIS

CLEV

ATLA

PORT

RALE

NEWY

SANA

DENV

SUNN

SEAT

JACK

OGDE

* Fiber on the SAND-LOSA-SUNN path belongs to CENIC

% used/Free

5%/19

15%/17

60%/8

23%/31

23%/31

23%/31 30%/28 35%/26

56%/14

40%/12

25%/15

35%/13

25%/3020%/16

25%/15

10%/18

15%/34

15%/27

19%/26

19%/26

15%/2715%/2719%/2619%/26

19%/2622%/25

22%/25

25%/2425%/24

25%/24

25%/2425%/24

19%/26

NLR Allocated Waves

Tom West, NLR

Charlie Catlett (catlett@mcs.anl.gov)Rick Summerhill

Layer 1 Topology with IP Network

PROVISIONAL TOPOLOGY – SUBJECT TO DISCUSSION

Charlie Catlett (catlett@mcs.anl.gov)Bill Johnston, ESNET

Virtual Circuit Network Services

A top priority of the science communityToday

– Primarily to support bulk data transfer with deadlinesIn the near future

– Support for widely distributed Grid workflow engines

– Real-time instrument operation

– Coupled, distributed applications To get an idea of how circuit services might be used to support the current

trends, look at the one year history of the flows that are currently the top 20

– Estimate from the flow history what would be the characteristics of a circuit set up to manage the flow

Charlie Catlett (catlett@mcs.anl.gov)Bill St. Arnaud (CANARIE)

CANARIE Approach: UCLP

User Controlled LightPaths – a configuration and provisioning tool built around grid technology using web services

Third party can concatenate cross connects together from various links, routers and switches to produce a wide are network that is under their control

– Articulated Private Network (APN)

– Next generation VPNUses Service Oriented Architecture (SOA) and so network can be integrated with

other web service applications

Charlie Catlett (catlett@mcs.anl.gov)

Extending Networks into Applications

Bill St. Arnaud (CANARIE)

Charlie Catlett (catlett@mcs.anl.gov)

“Life After Internet: Making Room for New Applications” (Larry Smarr and Charlie Catlett) in “Building Information Infrastructure,” ed. Brian Kahin, McGraw-Hill)

Hybrid Networks: A Blast from the Past ( 1991 ! )

Technical Issues– Rapid growth of Internet– New applications increasing bandwidth

requirements and network usage– End-to-end performance

Applications– Distributed computing– Digital libraries– Require peak bursts of bandwidth

several orders of magnitude greater than average traffic

Proposed Architecture• “The Internet architecture should,

wherever possible, combine the advantages of packet switched networks and of virtual circuit networks [to support] the ability to increase dynamically in capacity..."

Charlie Catlett (catlett@mcs.anl.gov)

Conclusions

National Networks will do on demand today, and better on demand next year

– Today in days or weeks - in 2007/8 will do it in minutes

– Today granularity is per site - multiple efforts trying to move to per application• CANARIE UCLP• ESnet OSCARS• I2 HOPI

Issues will remain

– Getting a switched lambda to a desktop is not a national backbone issue• Regional networks• Campus/Lab networks

Maybe we are doing OK!