MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

1

MB - NG

Discussion of Methodology forMPLS QoS &

High Performance High throughput Investigations

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

2

MB - NG

MB – NG SuperJANET4 Development Network

Gigabit Ethernet2.5 Gbit POS Access2.5 Gbit POS coreMPLS Admin. Domains

Dark Fiber (SSE)

RAL/UKERNAOSM-4GE-WAN-GBIC

WorldCom

SuperJANET4ProductionNetwork

Leeds

SJ4 DevC-PoP

Reading12416

SJ4 DevC-PoP

Warrington12416

SJ4 DevC-PoPLondon12416

WorldComWorldCom UCLOSM-4GE-WAN-GBIC

OSM-1OC48-POS-SS

UCLOSM-4GE-WAN-GBIC

MCCOSM-4GE-WAN-GBIC

OSM-1OC48-POS-SS

MANOSM-4GE-WAN-GBIC

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

3

MB - NG

High Performance High Throughput Show and Tell & Deliverables

Bringing Gigabit throughput to the Grid applications Show and tell demos TCP streams operating GridFTP operation 100 Mbit long transfers 500 Mbit long transfers 1 Gbit long transfers Multiple host-host – Host-striping High Throughput operations between multi-MPLS domains High Throughput & QoS – Realtime (EF - spq) / best-effort / Scavenger (qbss) Deliverables - Reports

PC Lab tests Protocol monitoring tools Gigabit WAN tests Enhanced TCP stacks Non TCP Best practice for TCP & GridFTP in Grid Middleware and Applications

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

4

MB - NG

High Performance High Throughput The Tools

Understanding the operation of the end system HW Latency vs frame size Throughput vs message size PCI bus load CPU load Effect of “protocol stacks”

Using Smartbits as a traffic generator Best way to monitor traffic and protocol packets

Optical splitter + Cambridge Card or NICs Monitor ports on Gigabit Switch Web100 instrumentation

Type and effect of background traffic generators Uniform Random – Poisson Bursty – web mirror – wget Self-similar

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

5

MB - NG

High Performance High Throughput Tests on Intermediate Systems

Understand the management & configuration of routers IP operation

MPLS

QoS/Diffserv management & configuration

Understand the performance of routers and (LAN) interconnects Latency vs frame size

Throughput vs message size

Link framing overheads

The effect of load on switch/router thoughput

Offered vs achieved thoughput

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

6

MB - NG

High Performance High Throughput Tests on the WAN 1

TCP/IP Throughput vs rtt Throughput vs window-size Dependency of Latency & Throughput on end system hardware Throughput using multiple TCP streams, and effect on the Network-

how does the background throughput change? The effect of background traffic loads Effect of different TCP stacks and TCP tuning

Reno

New Reno

Vegas

Enhanced TCP algorithms – Sally Floyd

Delayed-Ack, SACK, …

ECN

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

7

MB - NG

High Performance High Throughput Tests on the WAN 2

TCP Behaviour in QoS environment Marking Rate / Congestion Control CAR RED ECN Configurable Scheduling

• WFQ weighted fair queuing

• Priority / realtime

• WRR weighted round robin / DRR

• … …

Bringing Gigabit throughput to the Grid applications GridFTP at Gigabit BBftp and BBscp at Gigabit Data Replication GDMP Use of non-TCP protocols, and effect on the Network

UDP Blast via

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

8

MB - NG

High Performance High Throughput Time-Lines (1)Month1 NOW

Investigate NICs & End System behaviour Investigate Monitoring techniques – Code modifications ? Purchase end systems & monitoring tools ! Define test trafficMonth 2 Install / configure End systems at sites Set up GridFTP between sites – minimum Globus RH7.1? K2.4.14? Set up BBftp BBscp Test on SJ4 at 100 Mbit Investigate TCP/IP Stacks & Tuning – Agreement on Kernel / details of tests /.. Define MPLS needs – multi-loop to extend the rttMonth 3 Delivery & installation of routers and access links (TM1) Understand configuration of 7600s – IP only and MPLS Understand LAN and blade performance of routers – Lab tests Configuration of Development GSRs at PoPs

Discussion!Time

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

9

MB - NG

High Performance High Throughput Time-Lines (2)

Month4 Gigabit WAN tests e.g. UDP and TCP mem – mem streams

End-to-end IP only 2 sites / no background / with background

1 MPLS domain / n MPLS domains

Test with QoS – Realtime (EF - pf) / best-effort / Scavenger (qbss)

Think about results!

Month 5 Simple Test GridFTP / BBscp BBftp over the development network Transfers with QoS Realtime (EF - spq) / best-effort / Scavenger (qbss)

Discussion!Time

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

10

MB - NG

High Performance High Throughput Time-Lines (3)

Month 6 Initial demo of sustained throughput for different traffic classes (TM1) Establish connection between systems with real data e.g. BaBar HEPMonth 9 Demonstration of GridFTP operation 100 Mbit long transfers, 500 Mbit long transfers (TP1)Month 12 Initial demo & report of end-to-end guaranteed BW & Qos (TM2) Month18 Advanced demos including use of Qos Techniques (TM2) Demo of reliable transport at > 1Gbit/s - including with QoS (TP1) Grid Data Replication GDMPMonth 24 Final report on High throughput transport across the WAN (TP1) Final report on end-to-end traffic management over multiple MPLS domains

(TM2) Discussion!Time

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

11

MB - NG Next Steps

Draft document on High Performance Throughput Investigations to be circulated.

Technical “bits&bytes” meetings to discuss workplans and doc

Discussion!Time

MB-NG Meeting UCL 17 Jan 02R. Hughes-Jones Manchester

12

MB - NG

Hardware & Software Tools

High Bandwidth High Throughput Protocol Investigations (mainly)

•Optical splitter or opto-electrical tap•opto-electrical tap constructed for logic analyser

Software use of :tcpdump tcptrace

4 * 1 Gigabit Ethernet blade

Test PC

2.5 Gbit POS

Monitor PC

NIC

NIC