13
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost TRIUMF SITE REPORT Corrie Kost Head Scientific Computing

TRIUMF SITE REPORT Corrie Kost Head Scientific Computing

  • Upload
    lewis

  • View
    29

  • Download
    0

Embed Size (px)

DESCRIPTION

TRIUMF SITE REPORT Corrie Kost Head Scientific Computing. TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost. TRIUMF MANPOWER. TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost. WestGrid – UBC/TRIUMF Site. - PowerPoint PPT Presentation

Citation preview

Page 1: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

TRIUMF SITE REPORT

Corrie KostHead Scientific Computing

Page 2: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

TRIUMF MANPOWER

Public Computing Services 2.0 FTE

Network Infrastructure 1.5 FTE

Software Support & Security 3.0 FTE

PC Support Group 2.0 FTE

Data Acquisition Systems 2.5 FTE

Private CPU farms and Westgrid 0.5 FTE

Conferencing facilities 0.3 FTE

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 3: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

WestGrid – UBC/TRIUMF Site

• 504 dual 3.06 GHz Xeon blades• Red Hat Linux 7.3 compute nodes• PBS Scheduling with Maui• 10 TB disk storage• 24 TB tape storage• Direct Gigabit connection between sites sites• Possible 10GB in future• Subatomic physics allocation: ~1/3 of compute

resources• TWIST will be a Beta user this month.

TRIUMF/UBC first HEP site to install large (1008 CPU) blade cluster?

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 4: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

WestGrid – UBC/TRIUMF Site(www.westgrid.ca)

• Like the summer rains – two weeks away since August!• Blade power consumption exceeded expectations

– Use 12 of 14– Replace hot swap power supplies

• GPFS has some teething problems• Reduced cabling welcome – wireless would be better!• Very little infant mortality (no blades failed!)

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 5: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

14 dual-cpu blades in 7U Chassis

Side view of a blade

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 6: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

Rear View of Blades Chassis

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 7: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

WestGrid –UBC/TRIUMF Site

Storage

1 Rack Blades

6 Racks

UPSBatteries Inverters Switch Gear

Page 8: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

SATA IDE Drives system from HardData

64 bit 1.6GHz Opteronsapprox. equal to

32 bit 2.8GHz P4 (Xeons)

Performance details: Konstantin Olchanski ([email protected])

First look at Dual 1.6GHz Opterons

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 9: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

Misc. Developments

• 1 TB Raid 5 disks attached to Linux (TRSHARE) for Linux & Windows users (via Samba)

• Mosix & PBS Clustering – Steve McDonald 11:30 Wed

• Bandwidth Management – Steve McDonald – 11:40 Wed

• Colubris Radius Server – Steve McDonald – 11:50 Wed

• SPAM fighting at TRIUMF – Andrew Daviel – 13:50 Tue

• Document Management System: Docushare

• Windows 2003 Terminal Service (TNT2K3)

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

Page 10: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

Misc. Developments

TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost

•Videoconferencing

–One older room with limited capability

–A new room with Westgrid funded equipment

–Several protocols supported: ISDN, H323, VRVS, AccessGrid

–Participation in the Westgrid Collaborative/Visualization effort to improve videoconferencing quality and tools

Page 11: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

First Transatlantic Native 10 Gigabit Ethernet

ConnectionCatalin Meirosu

CERN and “Politehnica” University of Bucuresti

On behalf of the

International GRID Testbed

International

Testbed

Page 12: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

The First Transatlantic 10 GE

WAN PHY over an OC-192c circuit using lightpaths provided by SURFnet and CANARIE

CiscoONS 15454Force10

E 600

Force10E 600

HPItanium-2

HPItanium-2

CiscoONS 15454

CiscoONS 15454

CiscoONS 15454

CiscoONS 15454

Ixia400T

IntelItanium-2

IntelXeon

Ixia400T

10GE WAN PHY 10GE LAN PHY OC192c

Ottawa Toronto Chicago Amsterdam Geneva

9.24 Gbps using traffic generators

5.5 Gbps using TCP on PCs

6 Gbps using UDP on PCs

ITU Telecom World 2003 12

International

Testbed

Page 13: TRIUMF   SITE REPORT Corrie Kost Head Scientific Computing

The First Transatlantic 10 GE

CiscoONS 15454Force10

E 600

Force10E 600

HPItanium-2

HPItanium-2

CiscoONS 15454

CiscoONS 15454

CiscoONS 15454

CiscoONS 15454

Ixia400T

IntelItanium-2

IntelXeon

Ixia400T

10GE WAN PHY 10GE LAN PHY OC192c

Ottawa Toronto Chicago Amsterdam Geneva

ITU Telecom World 2003 13

International

Testbed

ESTA, IST-2001-33182