Upload
erika-warren
View
219
Download
2
Tags:
Embed Size (px)
Citation preview
Scientific Computing for SLAC Science
Bebo White
Stanford Linear Accelerator Center
October 2006
2
Scientific ComputingThe relationship between Science and the
components of Scientific Computing Application Sciences
Issues addressable with “computing”
Computing techniques
Computing hardware
High-energy and Particle-Astro Physics, Accelerator Science, Photon Science …
Particle interactions with matter, Electromagnetic structures, Huge volumes of data, Image processing …
PDE Solving, Algorithmic geometry, Visualization, Meshes, Object databases, Scalable file systems …
Processors, I/O devices, Mass-storage hardware, Random-access hardware, Networks and Interconnects …
Computing architectures
Single system image, Low-latency clusters, Throughput-oriented clusters, Scalable storage …
3
Drivers for SLAC Computing
• Computing to enable today’s data-intensive science
– Clusters, interconnects, networks, mass storage, etc.
• Computing research to prepare for tomorrow’s challenges
– Massive memory, low latency, petascale databases, detector simulation, etc.
4
SLAC Scientific ComputingScience Goals Computing Techniques
BaBar Experiment (winds down 2009-2012)
Measure billions of collisions to understand matter-antimatter asymmetry (why matter exists today)
High-throughput data processing, trivially parallel computation, heavy use of disk and tape storage. Intercontinental distributed computing.
ATLAS Experiment and Experimental HEP
Analyze petabytes of data to understand the origin of mass
High-throughput data processing. trivially parallel computation, heavy use of disk and tape storage. Intercontinental distributed computing.
Accelerator Science Simulate accelerator behavior before construction and during operation
Parallel computation, visual analysis of large data volumes
Particle Astrophysics (mainly simulation)
Star formation in the early universe, colliding black holes, …
Parallel computation (SMP and cluster), visual analysis of growing volumes of data
Particle Astrophysics Major Projects (GLAST, LSST …)
Analyze terabytes to petabytes of data to understand the dark matter and dark energy riddles
High-throughput data processing, very large databases, visualization
Photon Science Femtosecond x-ray pulses, “ultrafast” science, structure of individual molecules …
High throughput data analysis and large-scale simulation
New Architectures for SLAC Science
Radical new approaches to computing for Stanford-SLAC data-intensive science
Current focus: massive solid-state storage for high-throughput, low-latency data analysis
5
Data Challenge in High Energy Physics2006 example
SLAC
Online System
Selection and Compression
~10TB/s
• Raw data written to tape:10MB/s• Simulated and derived data: 20 MB/s• International network data flow to “Tier A Centers” 50 MB/s (400Mb/s)
6
Tier 1
Online System
EventReconstruction
France Germany
Institute ~0.25TIPS
~100 MBps
~0.6-2.5 Gbps
100 - 1000 Mbps
Physics data cache
~PBps
~0.6-2.5 Gbps
Tier 0 +1
Tier 3
Tier 4
Tier 2
• 2000 physicists in 31 countries are involved in this 20-year experiment in which DOE is a major player.
• Grid infrastructure spread over the US and Europe coordinates the data analysis
Analysis
Italy FermiLab, USA
Data Challenge in High Energy Physics: CERN / LHC High Energy Physics Data 2008 onwards
Event Simulation
CERN LHC CMS detector12,500 tons, $700M 2.5-40 Gbps
7
Client Client Client Client Client Client
Disk Server
Disk Server
Disk Server
Disk Server
Disk Server
Disk Server
Tape Server
Tape Server
Tape Server
Tape Server
Tape Server
SLAC-BaBar Computing Fabric
IP Network (Cisco)
IP Network (Cisco)
120 dual/quad CPU Sun/Solaris~700 TB Sun RAID arrays (FibreChannel +some SATA)
1700 dual CPU Linux (over 3700 cores)
25 dual CPU Sun/Solaris40 STK 9940B6 STK 9840A6 STK Powderhornover 1 PB of data
HEP-specific ROOT software (Xrootd) + Objectivity/DB object database some NFS
HPSS + SLAC enhancements to ROOT and Objectivity server code
8
Used/Required Space
Space (Square Feet)
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
50000
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Non-scientific Space
Low Density space
Medium Density space
Ultra High Density space
9
ESnet: Source and Destination of the Top 30 Flows, Feb. 2005T
erab
ytes
/Mon
th
Fer
mila
b (U
S)
Wes
tGrid
(C
A)
SLA
C (
US
)
INF
N C
NA
F (
IT)
SLA
C (
US
)
RA
L (U
K)
Fer
mila
b (U
S)
MIT
(U
S)
SLA
C (
US
)
IN2P
3 (F
R)
IN2P
3 (F
R)
Fer
mila
b (U
S)
SLA
C (
US
)
Kar
lsru
he (
DE
)
Fer
mila
b (U
S)
J
ohns
Hop
kins
12
10
8
6
4
2
0
LIG
O (
US
)
Cal
tech
(U
S)
LLN
L (U
S)
NC
AR
(U
S)
Fer
mila
b (U
S)
SD
SC
(U
S)
Fer
mila
b (U
S)
Kar
lsru
he (
DE
)
LBN
L (U
S)
U. W
isc.
(U
S)
Fer
mila
b (U
S)
U
. Tex
as, A
ustin
(U
S)
BN
L (U
S)
LLN
L (U
S)
BN
L (U
S)
LLN
L (U
S)
Fer
mila
b (U
S)
U
C D
avis
(U
S)
Qw
est (
US
)
ES
net (
US
)F
erm
ilab
(US
)
U. T
oron
to (
CA
)B
NL
(US
)
LLN
L (U
S)
BN
L (U
S)
LLN
L (U
S)
CE
RN
(C
H)
BN
L (U
S)
NE
RS
C (
US
)
LB
NL
(US
)D
OE
/GT
N (
US
)
JLa
b (U
S)
U. T
oron
to (
CA
)
Fer
mila
b (U
S)
NE
RS
C (
US
)
LB
NL
(US
)N
ER
SC
(U
S)
LB
NL
(US
)N
ER
SC
(U
S)
LB
NL
(US
)N
ER
SC
(U
S)
LB
NL
(US
)C
ER
N (
CH
)
Fer
mila
b (U
S)
DOE Lab-International R&E
Lab-U.S. R&E (domestic)
Lab-Lab (domestic)
Lab-Comm. (domestic)
10
Growth and Diversification
• Continue shared cluster growth as much as possible
• Increasing MPI (parallel) capacity and support (astro, accelerator, and more)
• Grid interfaces and support (Atlas et.al)
• Large SMPs (Astro)
• Visualization
11
Research - PetaCache
• The PetaCache architecture aims at revolutionizing the query and analysis of scientific databases with complex structure
– Generally this applies to feature databases (terabytes-petabytes) rather than bulk data (petabytes-exabytes)
• The original motivation comes from HEP
– Sparse (~random) access to tens of terabytes today, petabytes tomorrow
– Access by thousands of processors today, tens of thousands tomorrow
12
Latency Ideal
13
Latency Current Reality
14
Latency Practical Goal
15
PetaCache Summary
• Data-intensive science increasingly requires low-latency access to terabytes or petabytes
• Memory is one key:– Commodity DRAM today (increasing total cost by ~2x)
– Storage-class memory (whatever that will be) in the future
• Revolutions in scientific data analysis will be another key– Current HEP approaches to data analysis assume that random
access is prohibitively expensive
– As a result, permitting random access brings much-less-than-revolutionary immediate benefit
• Use the impressive motive force of a major HEP collaboration with huge data-analysis needs to drive the development of techniques for revolutionary exploitation of an above-threshold machine
16
Research – Very Large Databases
• 10-year, unique experience with VLDB
– Designing, building, deploying, and managing peta-scale production datasets/database – BaBar – 1.4 PB
– Assisting LSST (Large Synoptic Survey Telescope) in solving data-related challenges (effort started 4Q 2004)
17
LSST – Data Related Challenges (1/2)
• Large volumes
– 7 PB/year (image and catalog data)
– 500 TB/year (database)• Todays VLDBs ~10s TB range
• High availability
– Petabytes -> 10s of 1000s of disks -> daily disk failures
• Real time requirement
– Transient alerts generated in < 60 sec
18
LSST – Data Related Challenges (2/2)
• Spatial and temporal aspects
– Most surveys focus on a single dimension
• All data made public with minimal delay
– Wide range of users – professional and amateur astronomers, students, general public
19
VLDB Work by SCCS
• Prototyping at SCCS
• Close collaboration with key MySQL developers
• Working closely with world-class database gurus
20
Research – Geant4
• A toolkit simulating elementary particles passing through and interacting with matter, and modeling the detector apparatus measuring the passage of elementary particles and recording the energy and dose deposition
• Geant4 is developed and maintained by an international collaboration
– SLAC is the second largest center next to CERN
21
Acknowledgements
• Richard Mount, Director, SCCS
• Chuck Boeheim, SCCS
• Randall Melen, SCCS
WWW 2008 April 21-25, 2008
Beijing, China
23
Host Institution and Partners• Beihang University
– School of Computer Science
• Tsing-Hua University, Peking University, Chinese Academy of Sciences, …
• Microsoft Research Asia
• City Government of Beijing (pending)
24
BICC: Beijing International Convention Center
25
Key Personnel• General Chairs:
– Jinpeng Huai, Beihang University
– Robin Chen, AT&T Labs
• Conference Vice Chair:
– Yunhao Liu, HKUST
• Local volunteers
– 6-10 grad students led by Dr. Zongxia Du
– In cooperation with John Miller (TBD)
• IW3C2 Liaison: Ivan Herman
• PCO: two candidates under consideration
26
Local Organizing Committee• Composition of Local Organizing Committee:
– Vincent Shen, The HK University of Science and Technology
– Zhongzhi Shi, Chinese Academy of Sciences
– Hong Mei, Peking University
– Dianfu Ma, Beihang University
– Guangwen Yang, Tsinghua University
– Hsiao-Wuen Hon, Microsoft Research Asia
– Minglu Li, Shanghai Jiao Tong University
– Hai Jin, Huazhong University of Science and Technology
– … and Chinese Internet/Software/Telecom companies
27