TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

TRIUMF Site ReportHEPiX/HEPNT

NIKHEF, AmsterdamMay 19-23/2003

Corrie Kost

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

Overview

• Move to new facility

• Printing moves to CUPS

• Print/Copy/Scan-to-Email Integration

• WestGrid components at TRIUMF

• C4-Testbed Proposal

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

Move to ISAC II Building

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

New Facilities in ISAC II

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

REVISED LAN

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

Current LAN

CW

DM

Proton Exp Hall

Cyclotron Operations

Musr Facility

Meson HallChemistry Annex

Detector Facility

Electronics Devel

Main Office

Main Office 2nd Floor

Gerber Room

Trailer Complex

Proton Annex

Trailer Gg

ISAC I Experiments

Pet Group

ISAC I Offices

ISAC Controls

Cyclotron Controls

Data Center

ISAC II Offices

ISAC II Experiments

CW

DM

BCNETGigapop

UNIVERSITY

UBC

ISAC II

ISAC I

Meson &Proton Halls

Main OfficeComplex

PASSPORT8600

Edu/Research

e2e Lightpath

WestGRID

CommercialSpare

INT

ER

NE

T

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

Move to New Facility

• Weekend of March 29/2003 – 2 Passport 8600’s• Failure of air conditioner March 29 pm• Web/file server disk problems

– Old system failed – replaced– New system (disk) failed during backup– Diskcopy does not work for EXT3– DD gave ~37,000 lost&found entries– Murphy’s law – Mirrored on spare linux system – last gasp

• Legacy clean-up / move to racks / change servers

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

Common Unix Printing SystemCUPS

Components

• CUPS

• Foomatic (ppd)

• gs

• Samba

PRINT SERVER A (printcap)

PRINT SERVER B (CUPS)

Stage 1 : Separate Windows and Linux Print ServersStage 2 : Use SAMBA to use a single Linux Print Server

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

PRINT/COPY/SCAN(EMAIL) Integration

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

WestGrid – UBC/TRIUMF

• 1000 Cpu’s

• 10 TBytes Disk

• 75 Tbytes Tape

SUN HP IBM

INTERDYNAMIX DELL ATAPA

LINUXNETWORKX RACKSAVER

And the winner is……

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

WestGrid SolutionFor TRIUMF/UBC

• 1008 3.06GHz Xeons• 14 Blades (28 CPU)/Chassis• 533 MHz FSB• 2GB memory /node• 40GB drive/node• 36 Chassis in 6 racks• ~ 20Kwatts power/rack• ~ 400,000 Btu/hr (33 tons)• 40% less heat/power than 1U• ~ 1500 lbs/rack• 3.2 TeraFlop (peak)• 1.2-1.4Tflops (linpack)• 4*6*6 =144 Gige into 232 port Foundry Switch – max 756 nodes• 286Mbits/sec unblocked

IBM

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

CA*net 4 Testbed ProjectProposed Testbed Network Topology

OC192 (10Gbps)

OC192 (10Gbps)

Netherlight

Starlight

7+ Nodes hosts a 1.5 Terabyte Linux file server, each with 2 GbE optical interfaces, and a 10/100/1000 copper ethernet to provide LAN management.

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

CA*net 4 Testbed ProjectSoftware Matrix

Software Packages Disk Server RouterRedHat Linux 7.3 or 8.0 Globus Toolkit 2.2 OpenAFS Sun Grid Engine Sun J2EE SDK iperf Tsunami bbFTP Web 100 Net 100 GNU Zebra 0.93a

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003

CA*net 4 Testbed Project

• Project time contributions ~ 24 months• International in scope• Request persistent testbed• End-to-end lightpaths by user – to 10Gbps

– Quality of service– Guaranteed bandwidth– Stable network characteristics

• Test long-distance transfer protocols– 10Gbit Ethernet, Remote Direct Memory Access

RDMA/IP, FibreChannel/IP, serial SCSI, HyperSCSI etc