“OptIPuter Tech Transfer to the Broader e-Science and HPC Communities"

OptIPuter All Hands Meeting

Calit2@UCSD

La Jolla, CA

December 20, 2006

Dr. Larry Smarr

Director, California Institute for Telecommunications and Information Technology;

Harry E. Gruber Professor,

Dept. of Computer Science and Engineering

Jacobs School of Engineering, UCSD

Informing the Cyberinfrastructure Initiative

www.ctwatch.org

Global Lambda Integrated Facility (GLIF)International Group Innovating the LambdaGrid

• GLIF Technical and Control Plane Working Groups– Interim Meetings on 8-9 February 2006 – University of New Mexico, Albuquerque, United States

• GLIF Research and Applications Working Group– Maxine Brown and Larry Smarr– Abstract Taxonomy from 50 iGrid Demos

• GLIF Full Meeting – Tokyo, Japan– September 11-15, 2006– Interim Deadline for SC06 Demos

OptIPuter Will Have Coordinated Approach to SC06

chance2 10Gig (eth1 Intel Pro/10GbE)5 August 2005

chance1 10Gig (eth1 Intel Pro/10GbE)5 August 2005

DRAGON 10Gig DWDM XFP 5 August 2005

5

GSFC Scientific and Engineering Network (SEN)Mrtg-based `Daily' Graph (5 Minute Average)

Bits per second In and Out On Selected Interfaces

On August 5, 2005, GSFC’s Bill Fink simultaneously conducted two 15-minute-duration UDP-based 4.5-Gbps flow tests, with one flow between GSFC-UCSD and the other between GSFC-StarLight/Chicago. This filled both the NLR/WASH-STAR and DRAGON/channel49 lambdas to 90% of capacity. Flows were also tested in both directions. He measured greater than 9-Gbps aggregate in each direction and no-to-negligible packet losses.

Lambdas Give End Users Sustained ~ 10 Gbps Data Flow Rates

200 Times Faster Than Standard

Internet2!

Source: Pat Gary, NASA GSFC

ARC/NGIX-WestARC/NGIX-West

JPLJPL

GSFCGSFC

NGIX-EastNGIX-East

NLR SunnyvaleNLR Sunnyvale

NLR Los AngelesNLR Los Angeles

NLR ChicagoNLR Chicago

StarLightStarLight

LRCLRC

NLR ClevelandNLR ClevelandGRCGRC

NLR HoustonNLR HoustonJSCJSC

NLR MSFCNLR MSFC

NREN SitePeering Points1 GE10 GE

MAXMAX

NLR Baton RougeNLR Baton RougeSSCSSC

NLR JacksonvilleNLR Jacksonville

KSCKSC

OptIPuter Experiments AcceleratedFuture NASA NREN Over NLR

Linking OptIPuter to the DRAGON TestbedWashington, D.C. Metropolitan Area

HOPI / NLR

CLPK

ARLG

DCGW

MCLN

MIT Haystack Observatory(HAYS)U. S. Naval Observatory

(USNO)

University of Maryland College Park(UMCP)

Goddard Space Flight Center(GSFC)

National Computational Science Aliance (NCSA)

Univ of Southern California/Information Sciences Institute

(ISIE)

DCNE

DCNE

MAX

Bossnet

Global e-VLBIiGrid / SC05

• Goal: Real-Time VLBI Radio Telescope Data Correlation – from the USA (MIT Haystack, GGAO),

– Japan (Kashima) and

– Europe (Onsala in Sweden, Jodrell in the UK, Westerbork in The Netherlands)

• Achieved 512Mb Transfers from USA and Sweden to MIT,

• Results Streamed to iGrid.

Sloan Sky SurveyData Mining and Visualizing Data Using OptIPuter

• SDSS-I – Imaged > 8,000 Square Degrees of the Sky in Five Bandpasses– Detecting Nearly 200 Million Celestial Objects – Measured Spectra Of:

– > 675,000 galaxies

– 90,000 quasars

– 185,000 stars

• SDSS-II– Underway till 2008

www.sdss.org

iGRID2005From Federal Express to Lambdas:

Transporting Sloan Digital Sky Survey Data Using UDT

Robert Grossman, UIC

Evolution is the Principle of Biological Systems:Most of Evolutionary Time Was in the Microbial World

You Are

Here

Source: Carl Woese, et al

Much of Genome Work Has

Occurred in Animals

PI Larry Smarr

$24.5M Over 7 Years

Announcing Tuesday January 17, 2006

The Sargasso Sea Experiment The Power of Environmental Metagenomics

• Yielded a Total of Over 1 billion Base Pairs of Non-Redundant Sequence

• Displayed the Gene Content, Diversity, & Relative Abundance of the Organisms

• Sequences from at Least 1800 Genomic Species, including 148 Previously Unknown

• Identified over 1.2 Million Unknown Genes

MODIS-Aqua satellite image of ocean chlorophyll in the Sargasso Sea grid about the BATS site from

22 February 2003

J. Craig Venter, et al.

Science 2 April 2004:

Vol. 304. pp. 66 - 74

Marine Genome Sequencing ProjectMeasuring the Genetic Diversity of Ocean Microbes

CAMERA will include All Sorcerer II Metagenomic Data

Prochlorococcus Microbacterium

Burkholderia

Rhodobacter SAR-86

unknown

unknown

Metagenomics “Extreme Assembly” Requires Large Amount of Pixel Real Estate

Source: Karin RemingtonJ. Craig Venter Institute

Flat FileServerFarm

W E

B P

OR

TA

L

TraditionalUser

Response

Request

DedicatedCompute Farm(100s of CPUs)

TeraGrid: Cyberinfrastructure Backplane(scheduled activities, e.g. all by all comparison)

(10000s of CPUs)

Web(other service)

Local Cluster

LocalEnvironment

DirectAccess LambdaCnxns

Data-BaseFarm

10 GigE Fabric

Calit2’s Direct Access Core Architecture Will Create Next Generation Metagenomics Server

Source: Phil Papadopoulos, SDSC, Calit2+

We

b S

erv

ice

s

Sargasso Sea Data

Sorcerer II Expedition (GOS)

JGI Community Sequencing Project

Moore Marine Microbial Project

NASA Goddard Satellite Data

Community Microbial Metagenomics Data

First Implementation of the CAMERA Complex

Compute Database &Storage

Calit2/SIO will Establish Persistent OptIPuter Collaboratory with Venter Institute

HDTV Over Lambda

OptIPuter Visualized

Data

SIO/UCSD

NASA Goddard

www.calit2.net/articles/article.php?id=660

August 8, 2005

25 Miles

Venter Institute

Calit2/SDSC Proposal to Create a UC Cyberinfrastructure

of “On-Ramps” to National LambdaRail ResourcesOptIPuter + CalREN-XD + TeraGrid = “OptiGrid”

Source: Fran Berman, SDSC , Larry Smarr, Calit2

Creating a Critical Mass of End Users on a Secure LambdaGrid

UC San Francisco

UC San Diego

UC Riverside

UC Irvine

UC Davis

UC Berkeley

UC Santa Cruz

UC Santa Barbara

UC Los Angeles

UC Merced