Networking update and plans(see also chapter 10 of TP)

Bob Dobinson, CERN, June 2000

GE Ethernet

• Prices dropping • see following graph

• GE twisted pair available (RAL)• As GE is getting so cheap

• Why not equip everywhere for GE?• Run some areas initially at 100 MBPS

– Nice up grade path later

• To be looked at in more detail

Cost Evolution

0

500

1000

1500

2000

2500

3000

1995 1996 1997 1998 1999 2000Year

FE NIC projected

FE NIC actual

FE port projected

FE port actual

GE NIC

GE port

An all optical network solution?• Optical fibre now very cheap

– Total installation cost is 20-50% more than CAT 5 TP same price as CAT 6– “VF-45 connector joins 2 fibres in less than 2 mins for less than $5 per

connector”

• A more reliable better total system, says 3M, (especially for raw packets)

– No interference– Lighter, more robust, easier to install– 20 year guarantee!

• To be looked at further– Think a 20 year timescale

Programmable NICs(Alteon)

• Nice results obtained recently ( see later)• Applications

• Switch testing at line speed (Gigabit)• RoI and event building• Pre processing• Protocol processing

Alteon ACENIC

PCI BUS

PCI interface

RISC2

RISC1

MAC

Ext Mem 0.5/1 Mbyte

Mem

Mem

DMA1

DMA2

PhyGE

TIGON CHIP

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 15000

10

20

30

40

50

60

70

80

90

100

Efficiency (%)Max. Effiency (%)

Data Pay load (by tes)

Effi

cien

cy (

%)

SEND ONLY

Data payload

% o

f 1 G

bps (

effic

ienc

y)

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 15000

10

20

30

40

50

60

70

80

90

100

Efficiency (% )Max. Effiency (%)

Data Pay load (by tes)

Y a

xis

title

SEND PLUS CONCURRENT RECEIVE SINGLE PROCESSOR

% o

f 1G

bps (

effic

ienc

y)

Data payload

Alteon contacts

• Meeting next Tuesday with the company• New chips?

• Would make a nice ROB component?

• New NICs?• Future of PCI?

Tests with Foundry Big Iron

• Tests with 16 port switch at CERN• High throughput, but some flow control

problems• This model switch can support 64 ports, we

need about 128• So we can almost buy what we need• 128 certainly in the pipeline

Switch testing

• GE, use Alteon boards programmed as traffic generators

• FE use 32 node FPGA based traffic generators– Doubles as ROB emulator– Note/talk by Micheal Levine

• Used to evaluate switches and calibrate modeling

Switch under test

GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC

Clock Mem

CPU

Cache Controller

PCI BUSFE NIC

Expansionto multiple units

NICs housed in PCs with 14/17 PCI slots

Global clock

Modeling

• Continuing progress• Adding QoS, trunking and TCP/IP stack

• Model various models for Trigger/DAQ networks

• Two separate• Integrated

Geographic distribution of computing resources

• At the pit• On CERN site• At remote locations

• Farms at home

Network: a global view and possible integration?

• Level 2• DAQ/EF• DCS• Front end ROD crates

Ethernet SW evolution

• See talk of Marc• Also will revisit the problem of high

performance Ethernet I/O– Will look at the TCP/IP co-existence problem– How to best deliver fibre performance to the application?– Scalability– Fault tolerance– QoS– Trunking

Present PC I/O architecture

Future PC I/O Architecture

Infiniband Trade Association• Major players

» IBM» INTEL» Microsoft» Lucent» SUN» Compaq» Hitachi» Fujitsu» Dell

• A significant development! Need to know more. Spec available soon.

Is this the end of PCI?

• I think not!• As ESA bus became a sub system on PCI, so PCI

will likely be a sub system hanging off Infiniband• May gain a nice way of supporting high bandwidth

from multiple PCI buses into a host