SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000

SLAC Update

Les Cottrell & Richard Mount

July 24, 2000

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 2

BaBar

• Very successful turn-on!– already reached design luminosity

• Computing:– 1,000,000 lines of C++– Object oriented Db = Objectivity– Online data rate 10-20MBytes/sec– Regional computing centers:

• IN2P3 (France/Lyon), RAL (UK), INFN (Italy/Rome)

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 3

SLAC-BaBar Data Analysis System50/400 simultaneous/total physicists, 300 Tbytes per year

HARDWARE UNITS End FY1999 End FY2000

Tape Silos (STK Powderhorn, 6000 tapes each)

silos 6 6

Tape Drives (STK Eagle, 20 Gbyte, 10 Mbytes/s)

drives 20 40

Disk (net capacity of RAID arrays)

Tbytes 20 56

File Servers and Data Movers (Sun)

CPUs 73 150

Interactive Servers (Sun + Linux)

CPUs 82 140

Batch Servers (Sun + Linux) CPUs 300 900

Network Switches (Cisco 6509)

switches 5 14

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 4

BaBar Offline Computing at SLAC:Costs other than Personnel

(does not include “per physicist” costs such as desktop support, help desk, telephone, general site network)

Does not include tapes

0.00

2.00

4.00

6.00

8.00

10.00

12.00

FY 1997 FY 1998 FY 1999 FY2000 FY2001

$M

other

net

datamove

tape

disk

farm cpu

smp cpu

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 5

BaBar’s Need for the Grid

Early 2000:– The Grid? “Don’t bother me now, I’m

working on the CP-violation result for Osaka”– Data Transfer? “Something the French

BaBarians do to justify the existence of their computer center”

More recently:– “The Grid? Yes we need it and it has to work”

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 6

BaBar’s Need for the GridWhat Happened?

– PEP II/BaBar has reached design (integrated) luminosity (131 TB in database by end June 2000)

– We all believe the plans to increase luminosity by a factor 8 by 2003

– BaBar data will grow faster than “Moore’s Law”

Options:– Pour French, British, Italian money into the SLAC

Computer Center

– Make the Grid work

While simultaneously improving storage management

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 7

SSRL

• Collaboration with SDSC (UC San Diego)– 130MBytes/minute/beamline/full operation– 374 GBytes per 12 hour day for 4 beamlines– 90TBytes per year (8 months operation)

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 8

Connectivity: NTON

NTONATM

NortelMUX

NortelMUX

Cisco GSR 12000

HP Exemplar

8 x OC-3 (155M) ATM2 x OC-12 (622M) ATM

Catalyst 6509

Gigabit Ethernet

Sun E450 Sun E420 Dual PIII 533 w/Linux

NSTOR FC Array

FC Disks

Gigabit Ethernet

Dual PIII 833 w/WindowsNT

2 x OC-12 (622M) ATM

Demonstrated 57MBytes/second disk to application between SLAC & LBNL a year ago.Now aiming for 100MBytes/sec between SLAC & Caltech

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 9

Connectivity: Stanford U/Abilene

• OC3 for last 9 months

• Will upgrade to OC12 soon

• Currently only used between SLAC and Stanford and UC campuses– Important for SSRL– proposal to use for QoS tests between SLAC

and Daresbury Lab in UK

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 10

Connectivity: ESnet 1 of 2

• ESnet: 43 Mbps being upgraded to 155Mbps– upgrade requested 18 months ago, approved 15

months ago, hope for soon...– now saturated for long periods

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 11

Connectivity: ESnet 2 of 2• Heavy use to transfer data from SLAC to IN2P3

(17-26Mbps continuous, 100GBytes/day, need TBytes/day)

• No longer send tapes (latency goes from weeks to hours, less jitter, reduced labor (especially for errors))

• In past weakness of International links protected ESnet from significant traffic

• HEP can saturate links between major regional centers

• Other major BaBar centers include: Oxford (RAL), Rome (INFN), Caltech, Colorado, LBNL, LLNL

SLAC Status, Les Cottrell ESnet International Meeting, Kyoto July 24-25, 2000 12

Requirements• Higher speed end-to-end links• Site-to-Site Replication Service

– 100 Mbyte/s goal possible through NTON (SLAC-Caltech are working on this).

• Multi-site Cached File Access System– Will use OC12, OC3, (even T3) as available

• (even 20 Mbits/s international links)

– Need “Bulk Transfer” service:• Latency unimportant• Tbytes/day throughput important (Need prioritized service to

achieve this on international links)• Coexistence with other network users important. (This is the main

PPDG need for differentiated services on ESnet)– can be background, don’t want onerous scheduling