CHEP2000 - Padova, February 2000 Network Engineering @ SLAC S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A. Zele

CHEP2000 - Padova, February 2000

Network Engineering @ SLAC

S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A. Zele


Summary

IntroductionMajor IssuesNetwork ArchitecturePerformanceRetrospective


Introduction

SLAC - Stanford Linear Accelerator Center High Energy Physics Stanford Synchrotron Radiation Laboratory

1300 full-time employees, 700 on-site collaborators

Collaborate with 200 institutions internationally Mixture of real-time data acquisition, numerical

analysis, business services/administration Very large quantities of data, e.g. 1-2

petabytes/year for the BaBar experiment


Major Issues

Bandwidth demand doubles every 1.5 years High reliability and availability requirement, both

for experimental work and business services Network performance requirements dictate use

of leading-edge technology Leading-edge technology challenges reliability International collaboration dictates need for

open network Security - DOE requirements, open network is

more prone to hacking


Major Issues...

Network services have become so essential that failures can bring organizations to a standstill They are often more critical than the phone

One reason: critical information (data and applications) has moved into the (often-times central) computer/storage facilities

SLAC: particular issue - very high bandwidth/availability for data acquisition


Bandwidth Growth


Network Architecture Layer one

Star topology Fiber/Copper Redundant Ethernet Full/Half Duplex, 10/100/1000 Mbps Backbone almost entirely Gigabit Ethernet (with Gigabit EtherChannel)

Islands BaBar detector/data acquisition Main Control Center Business Services

Layer two Switched VLAN (~45 switches, ~40 VLANs)

Layer three Centralized routing


DMZ

InternetModems,

ISDNxDSL

SLAC Switched LAN Spring 2000

ESA

Legacy

SSRL

OldServers

MCC3

20 Buildingedge switches

BaBar

MCC2

SSRL

MCC1

10BaseT

FDDI/CDDI

100BaseT

100BaseFL

Gigaswitch

Router

Switch

Hub

1Gbit FL

4Gbit FL

Concentrator

IR26 Farm edge switches

4 Server switches

BSD

DMZ Switch

Netscout

Monitoring

CoreRouters

Switches




External Connections

622 Mb/s

155 Mb/s( 622 Mb/s)

45 Mb/s

155 Mb/s

2 Mb/s


Performance

A number of tools developed in-house to monitor the status of the network E.g., port, backplane, CPU utilization, device

reachability, L2/L3 traceroute, DMZ traffic statistics

No components are currently approaching saturation




Retrospective

The network design has proved to be scalable, highly available and provides high bandwidth in a secure environment

Some lessons learned: be careful with management and configuration of layer-2 switching

and spanning trees (e.g., watch for VTP configuration issues) and with CGMP/IGMP multicast support

do not attempt to mix standards (e.g., Cisco ISL and IEEE 803.1Q) even when theoretically possible

adequate security and openness are often conflicting requirements Open issues:

effective monitoring (e.g., SPAN) and data gathering in a high-speed switched environment

secure SNMP monitoring (SNMPv3 support) and secure access to the network devices (SSH)

Documents

CHEP2000 - Padova, February 2000 Network Engineering @ SLAC S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A. Zele