The Internet2 HENP Working Group Internet2 Spring Meeting May 8, 2002

The Internet2 HENP Working Group

Internet2 Spring MeetingMay 8, 2002

Shawn McKeeUniversity of MichiganHENP Co-chair

March 19, 2002 April 2002

Slide 2

Overview

• Mission, Membership and Motivation

• HENP WG Goals

• Recent activities

• Where to from here?


Slide 3

Internet2 HENP WG*• Mission

• To help ensure that the required national and international network network infrastructuresinfrastructures, monitoring monitoring toolstools and facilitiesfacilities, and support for collaborative collaborative systemssystems, are deployed and developed on an ongoing basis, in time to meet the requirements of the major HENP experimental programs, as well as the HENP community at-large .

• To encourage that the Group's targeted developments are applied broadly, in other fields, within and beyond the bounds of scientific research.

• The goals of this Working Group are synergistic with the Internet2 End-to-End Initiative, which has HENP as one of its focal disciplines.

Formed as an Internet2 WG on Oct. 26 2001* Co-Chairs: S. McKee (Michigan), H. Newman (Caltech);

Secretary J. Williams (Indiana); With thanks to R. Gardner (Indiana)


Slide 4

Current Membership

• We have 88 members signed up on our mailing list at http://lists.bnl.gov/mailman/listinfo/henp-net-l

• Our membership is a mix of scientists, engineers, local and backbone network experts and funding agency members

• We are looking to expand our membership, especially internationally…


Slide 5

Why HENP Networking?

• Since the early 1980’s physicists have depended upon leading-edge networks to enable ever larger international collaborations.

• Major HENP collaborations require rapid access to event samples from massive data stores, not all of which can be locally stored at each computational site.

• Evolving integrated applications, i.e. Data Grids, rely on seamless, transparent operation of the underlying LANs and WANs.

• Networks are among the most basic Grid building blocks.


Slide 6

Tier 1

Tier2 Center

Online SystemOffline Farm,

CERN Computer Ctr ~25 TIPS

BNL CenterFrance ItalyUK

InstituteInstituteInstituteInstitute ~0.25TIPS

Workstations

~100 MBytes/sec

100 - 1000 Mbits/sec

Physicists work on analysis “channels”

Each institute has ~10 physicists working on one or more channels

Yearly data volume is ~10 Petabytes

Physics data cache

~PByte/sec

~2.5 Gbits/sec

Tier2 CenterTier2 CenterTier2 Center

~2.5 Gbps

Tier 0 +1

Tier 3

Tier 4

Tier2 Center Tier 2

CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1

Hierarchical Computing Model


Slide 7

HENP WG Goals

• One of our first task upon forming our working group was to define a set of goals

• These goals needed to address the range of interests and concerns of our community

• We arrived at 9 goals, which are now part of our charter.

• These are are the best definition of what our These are are the best definition of what our working group is about…working group is about…


Slide 8

HENP WG Goal #1

1. Support the development and deployment of toolkits, documentation and guidelines for best practices so that existing expert knowledge and tools for high throughput data transfers, packet loss and throughput-limit monitoring, and collaborative systems are widely known.

Contact: Tom Hacker/Shawn McKee (U Mich)


Slide 9

HENP WG Goal #2

2. Support the deployment of testing and monitoring tools and applications, link and site instrumentation, and a standard methodology, in association with the Internet2 End-to-End Initiative, so that all of HENP's major network paths can be adequately monitored, and used at full capability.

Contact: Iosif Legrand()/Les Cottrell (SLAC)*


Slide 10

HENP WG Goal #3

3. Share information and provide advice on the configuration of routers, switches, PCs and network interfaces, and network testing and problem resolution, to achieve high performance over local and wide area networks in production.

Contact: Sylvain Ravot(CERN)/<To be named>


Slide 11

HENP WG Goal #4

4. Work with the staffs of the international (Tier0), national (Tier1), regional (Tier2) and local (Tier3) Grid facilities as needed, to verify high throughput and network responsiveness, in association with the Grid projects, and for new collaborative interactive systems.

Contact: Les Cottrell* (SLAC)/<To be named>


Slide 12

HENP WG Goal #5

5. Work with the sites, expert networking teams at national labs, universities, and computational science centers, to develop and expand on a knowledge base for high performance networks. Develop a support base of knowledgeable engineers, technicians and scientists to assist with the above goals.

Contact: Jim Williams (IndianaU)/<TBN>


Slide 13

HENP WG Goal #6

6. Work to identify and track the programmatic network needs of the current and future generation HENP program: quantitatively in terms of bandwidths, throughputs and computer system capabilities required to provide the needed overall performance; and qualitatively in terms of the features and characteristics of existing and new systems needed for efficient distributed data access, processing and analysis as well as remote collaboration.

Contact: Harvey Newman(CalTech)/<To be named>


Slide 14

HENP WG Goal #7

7. Work with GriPhyN, PPDG, EU DataGrid, iVDGL and other Grid projects to ensure that the US and global network infrastructures are satisfying the programmatic needs of scientists using grids.

Contact: Dantong Yu(BNL)/Iosif Legrand()*


Slide 15

HENP WG Goal #8

8. Work with the network engineering staffs of the US and global research networks to help define the requirements and operational procedures for a Global Grid Operations Center, and mission-specific Grid Operations Centers as needed, so that the ensemble of research networks is able to work efficiently and provide the high level of capability required.

Contact: Jim Williams (IndianaU)/<TBN>


Slide 16

HENP WG Goal #9

9. Investigate emerging new network technologies, such as optical switching and lambda-based network infrastructure, for applicability and potential use in support of the HENP program. If appropriate, develop a strategic plan for coordinated deployment of these technologies among HENP sites.

Contact: Phil Demar(FNAL)/Bill St. Arnaud(CANARIE)


Slide 17

About These Goals…

• As you can see from our goals we have many network related issues to pursue.

• These goals are generally not specific to HENPHENP.

• We are striving to work with many related groups on these issues: HENP-WGHENP-WG; ICFA-SCICICFA-SCIC; I2 EngineeringI2 Engineering in the US; the GGF Networking WGGGF Networking WG, etc.

• NOTE: Grids implicitly assume high, repeatable network performance…

=> Both grid and networking groups MUST work together to insure an optimal outcome!


Slide 18

Recent HENP Talks

• Maximizing End-to-end Network Performance

• High Performance Throughput

• Present and Future Networks: an HENP Perspective

• Performance of Parallel TCP Streams in a Production Network

• Grid Monitoring

• QoS Implementation and Testing

• Report from the PPDG Monitoring Group

• Network-related Issues in a Grid Environment

• Remarks on HENP and the I2 End-to-end Performance Initiative

• Numerous other discussions, perspectives, reports and overviews


Slide 19

Some HENP Issues…

• The HENP WG has focused on high performance because of the unprecedented data volume we are anticipating.

• Some examples of recent interest:

• Local infrastructure

• Protocol issues for high performance

• Issues in achieving high performance networking


Slide 20

Local Networking Infrastructure

• LANs used to lead WANs in performance, capabilities and stability, but this is no longer true.

• WANs are deploying 10 Gigabit technology compared with 1 Gigabit on leading edge LANs.

• New protocols and services are appearing on backbones (Diffserv, IPV6, multicast) (ESNet, I2ESNet, I2).

• Insuring our HENP institutions have the required LOCAL level of networking infrastructure to effectively participate in the evolving hierarchical grid computing model is a major challenge.


Slide 21

TCP WAN Performance

Mathis, et. al., Computer Communications Review v27, 3, July 1997, demonstrated the dependence of bandwidth on network parameters:

PkLossRTT

MSSBW

7.0

BW - BandwidthBW - Bandwidth

MSS – Max. Segment SizeMSS – Max. Segment Size

RTT – Round Trip TimeRTT – Round Trip Time

PkLoss – Packet loss ratePkLoss – Packet loss rate

If you want to get 90 Mbps via TCP/IP on a WAN link from LBL to UM you need a packet loss < 1.8e-6 !! (~70 ms RTT).

This gets MUCH (~103) worse if you try to achieve ~1 gigabit across the Atlantic Ocean!


Slide 22

Achieving High Performance Networking

• Server and Client CPU, I/O and NIC throughput sufficient• Must consider firmware, hard disk interfaces, bus type/capacity• Knowledge base of hardware: performance, tuning issues, examples

• TCP/IP stack configuration and tuning is Absolutely RequiredAbsolutely Required

• Large windows, multiple streams?• No Local infrastructure bottlenecks

• Gigabit Ethernet “clear path” between selected host pairs• To 10 Gbps Ethernet by ~2003

• Careful Router/Switch configuration and monitoring • Enough router “Horsepower” (CPUs, Buffer Size, Backplane BW)• Packet Loss must be ~Zero (well below 0.1%)

• i.e. No “Commodity” networks (need I2, ESNet type networks)• End-to-end monitoring and tracking of performance


Slide 23

Where Are We Headed?

• We have had two official HENP WG meetings and our third is co-scheduled with the Internet2 Spring Meeting, May 6-8 (1-3 PM Today!)

• Our goals have been populated with one or two contacts each.

• During the next few weeks we hope to have compiled goal related information on our WWW page: http://www.internet2.edu/henp

• And longer term…(next slide)


Slide 24

Some Desired HENP WG Outcomes…

• Define local local hardware, software and configuration details necessary to achieve ~gigabit TCP connections to the desktop

• Enable high-quality, interactive video-conferencing on demand over the WAN

• Enable multi-gigabitmulti-gigabit site-to-site interconnects with related network services to facilitate the tiered grid computing model

• Provide a set of worldwide, transparent, high-performing networksnetworks, which allow interaction with users and applications that have special requirements


Slide 25

Thank-youThank-you for your attention!

• Questions?

• See our WWW page for more information: http://www.internet2.edu/henp

Documents

The Internet2 HENP Working Group Internet2 Spring Meeting May 8, 2002