CANS Meeting (December 1, 2004)Paul Avery1 University of Florida [email protected] UltraLight U.S. Grid Projects and Open Science Grid Chinese American

CANS Meeting (December 1, 2004)

Paul Avery 1

Paul AveryUniversity of [email protected]

UltraLight

U.S. Grid Projects andOpen Science Grid

Chinese American NetworkingSymposium

Florida International UniversityDecember 1, 2004


Paul Avery 2

U.S. “Trillium” Grid Consortium Trillium = PPDG + GriPhyN + iVDGL

Particle Physics Data Grid: $12M (DOE) (1999 – 2004+)GriPhyN: $12M (NSF) (2000 – 2005) iVDGL: $14M (NSF) (2001 – 2006)

Basic composition (~150 people)PPDG: 4 universities, 6 labsGriPhyN: 12 universities, SDSC, 3 labs iVDGL: 18 universities, SDSC, 4 labs, foreign partnersExpts: BaBar, D0, STAR, Jlab, CMS, ATLAS, LIGO,

SDSS/NVO

Complementarity of projectsGriPhyN: CS research, Virtual Data Toolkit (VDT)

developmentPPDG: “End to end” Grid services, monitoring, analysis iVDGL: Grid laboratory deployment using VDTExperiments provide frontier challengesUnified entity when collaborating internationally


Paul Avery 3

Goal: Peta-scale Virtual-Data Grids

for Global Science

Virtual Data Tools

Request Planning &Scheduling Tools

Request Execution & Management Tools

Transforms

Distributed resources(code, storage, CPUs,networks)

ResourceManagement

Services

Security andPolicy

Services

Other GridServices

Interactive User Tools

Production TeamSingle Researcher Workgroups

Raw datasource

PetaOps Petabytes Performance


Paul Avery 4

Trillium Science Drivers Experiments at Large Hadron

Collider100s of Petabytes 2007 - ?

High Energy & Nuclear Physics expts~1 Petabyte (1000 TB) 1997 –

present

LIGO (gravity wave search)100s of Terabytes 2002 –

present

Sloan Digital Sky Survey10s of Terabytes 2001 –

present

Data

gro

wth

Com

mu

nit

y g

row

th

2007

2005

2003

2001

2009

Future Grid resources Massive CPU (PetaOps) Large distributed datasets (>100PB) Global communities (1000s)


Paul Avery 5

Sloan Digital Sky Survey (SDSS)Using Virtual Data in GriPhyN

Galaxy clustersize distribution

Sloan Data


Paul Avery 6

The LIGO Scientific Collaboration (LSC)and the LIGO Grid

LIGO Grid: 6 US sites

* LHO, LLO: observatory sites* LSC - LIGO Scientific Collaboration - iVDGL supported

iVDGL has enabled LSC to establish a persistent production grid

Cardiff

AEI/Golm •

+ 3 EU sites (Cardiff/UK, AEI/Germany)

Birmingham•


Paul Avery 7

Search for Origin of Mass & Supersymmetry (2007 – ?)

TOTEM

LHCb

ALICE

27 km Tunnel in Switzerland & France

CMS

ATLAS

Large Hadron Collider (LHC) @ CERN


Paul Avery 8

CMS Experiment

LHC Global Data Grid

Online System

CERN Computer Center

USAKorea RussiaUK

Maryland

0.1 - 1.5 GB/s

>10 Gb/s

10-40 Gb/s

2.5-10 Gb/s

Tier 0

Tier 1

Tier 3

Tier 2

Physics caches

PCs

Iowa

UCSDCaltechU Florida

5000 physicists, 60 countries

10s of Petabytes/yr by 2008 1000 Petabytes in < 10 yrs?

FIU

Tier 4


Paul Avery 9

LCG: LHC Computing Grid Global Grid infrastructure for LHC experiments

Matched to decades long research program of LHC

Large scale resourcesHundreds of resource sites throughout the worldCommon resources, tools, middleware and environments

Operated and supported 24x7 globallyA robust, stable, predictable, supportable infrastructure


Paul Avery 10

Network Bandwidth Needs (Gb/s)


Paul Avery 11

Analysis by Globally Distributed Teams

Non-hierarchical: Chaotic analyses + productions Superimpose significant random data flows


Paul Avery 12

Trillium Program of Work Common experiments, leadership, participants CS research

Workflow, scheduling, virtual data

Common Grid toolkits and packagingVirtual Data Toolkit (VDT) + Pacman packaging

Common Grid infrastructure: Grid3National Grid for testing, development and production

Advanced networkingUltranet, UltraLight, etc.

Integrated education and outreach effort+ collaboration with outside projects

Unified entity in working with international projectsLCG, EGEE, Asia, South America


Paul Avery 13

VDT Growth Over 2.5 Years

VDT 1.1.3,1.1.4 & 1.1.5 pre-SC 2002

VDT 1.0Globus 2.0bCondor 6.3.1

VDT 1.1.7Switch to Globus 2.2

VDT 1.1.11Grid3

VDT 1.1.8First real use by LCG

VDT 1.1.14May 10


Paul Avery 14

UltraLight: 10 Gb/s Network

10 Gb/s+ network• Caltech, UF, FIU, UM, MIT• SLAC, FNAL• Int’l partners• Level(3), Cisco, NLR

Funded by ITR2004


Paul Avery 15

Grid3: An Operational National Grid30 sites, 3500 CPUs: Universities + 4 national

labsPart of LHC GridRunning since October 2003Applications in HEP, LIGO, SDSS, Genomics, CS

http://www.ivdgl.org/grid3


Paul Avery 16

Grid2003 Applications High energy physics

US-ATLAS analysis (DIAL),US-ATLAS GEANT3 simulation (GCE)US-CMS GEANT4 simulation (MOP)BTeV simulation

Gravity wavesLIGO: blind search for continuous sources

Digital astronomySDSS: cluster finding (maxBcg)

BioinformaticsBio-molecular analysis (SnB)Genome analysis (GADU/Gnare)

CS Demonstrators Job Exerciser, GridFTP, NetLogger-grid2003


Paul Avery 17

Grid3 Shared Use Over 6 months

cms dc04

atlasdc2

Sep 10

Usa

ge:

CP

Us


Paul Avery 18

Open Science Grid Build on Grid3 experience

Persistent, production-quality Grid, national + international scope

Continue U.S. leading role in international scienceGrid infrastructure for large-scale collaborative scientific

research

Create large computing infrastructureCombine resources at DOE labs and universities to

effectively become a single national computing infrastructure for science

Grid3 OSG-0 OSG-1 OSG-2 …

Maintain interoperability with LCG (LHC Grid) Provide opportunities for educators and students

Participate in building and exploiting this grid infrastructure

Develop and train scientific and technical workforce

http://www.opensciencegrid.org


Paul Avery 19


Paul Avery 20

Education and Outreach


Paul Avery 21

NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen

Grids and the Digital DivideRio de Janeiro, Feb. 16-20, 2004

Background World Summit on Information Society HEP Standing Committee on Inter-

regional Connectivity (SCIC)

Themes Global collaborations, Grids and

addressing the Digital Divide

Next meeting: May 2005 (Korea)

http://www.uerj.br/lishep2004

http://www.lishep.uerj.br/LAFEX.jpg


Paul Avery 22

iVDGL, GriPhyN Education / Outreach

Basics $200K/yr Led by UT

Brownsville Workshops, portals Partnerships with

CHEPREO, QuarkNet, …


Paul Avery 23

June 21-25 Grid Summer School First of its kind in the U.S. (South Padre Island,

Texas)36 students, diverse origins and types (M, F, MSIs, etc)

Marks new direction for TrilliumFirst attempt to systematically train people in Grid

technologiesFirst attempt to gather relevant materials in one placeToday: Students in CS and PhysicsLater: Students, postdocs, junior & senior scientists

Reaching a wider audiencePut lectures, exercises, video, on the webMore tutorials, perhaps 3-4/yearDedicated resources for remote tutorialsCreate “Grid book”, e.g. Georgia Tech

New funding opportunitiesNSF: new training & education programs

CHEPREO: Center for High Energy Physics Research and Educational OutreachFlorida International University

Physics Learning Center CMS Research iVDGL Grid Activities AMPATH network (S.

America)

Funded September 2003

$4M initially (3 years) 4 NSF Directorates!


Paul Avery 25

Grid Project ReferencesGriPhyN

www.griphyn.orgiVDGL

www.ivdgl.orgPPDG

www.ppdg.netGrid3

www.ivdgl.org/grid3Open Science Grid

www.opensciencegrid.orgCHEPREO

www.chepreo.orgUltraLight

ultralight.cacr.caltech.eduGlobus

www.globus.org

LCG www.cern.ch/lcg

EU DataGrid www.eu-datagrid.org

EGEE www.eu-egee.org


Paul Avery 26

Trillium Grid Tools: Virtual Data Toolkit

Sources(CVS)

Patching

GPT srcbundles

NMI

Build & TestCondor pool

(37 computers)

…

Build

Test

Package

VDT

Build

Contributors (VDS, etc.)

Build

Pacman cache

RPMs

Binaries

Binaries

Binaries Test

Use NMI processes later

Documents

CANS Meeting (December 1, 2004)Paul Avery1 University of Florida [email protected] UltraLight U.S. Grid Projects and Open Science Grid Chinese American