14.09.2006. 2 GridKA School 2006, H. Enke, AIP 14.09.2006 Overview AstroGrid-D History Project Project Structure Goals Current Current status

14.09.2006

2GridKA School 2006, H. Enke, AIP14.09.2006

OverviewOverview

AstroGrid-D AstroGrid-D HistoryHistory Project StructureProject Structure GoalsGoals

Current statusCurrent status ArchitectureArchitecture Resource IntegrationResource Integration Services:Services:

• Information Service (MetaData Management)Information Service (MetaData Management)• Database and DataStreams ManagementDatabase and DataStreams Management

Selected UseCasesSelected UseCases• DynamoDynamo• NBody6++NBody6++

GATGAT ChallengesChallenges

• LOFAR / GlowGridLOFAR / GlowGrid• Robotic Telescopes GridRobotic Telescopes Grid

AstroGrid-D and GAVOAstroGrid-D and GAVO


History: About Cactus

Cactus and its ancestor codes have been using Grid infrastructure since 1993

Support for Grid computing was part of the design requirements for Cactus 4.0 (experiences with Cactus 3)

Cactus compiles “out-of-the-box” with

Globus

[using globus device of MPICH-G(2)] Design of Cactus means that applications are

unaware of the underlying machine/s that the simulation is running on … applications become trivially Grid-enabled

Infrastructure thorns (I/O, driver layers) can

be enhanced to make most effective use of the underlying Grid architecture T.RadkeT.Radke


History : Simulation

Zur Anzeige wird der QuickTime™ Dekompressor „YUV420 codec“

benötigt.

Numerical relativity: AEItangential collision of two black holes

Numerical relativity: AEItangential collision of two black holes

SC99 Colliding Black Holes

using Garching, ZIB T3E’s, with remote collaborative interaction and visualisation at ANL and NCSA booths

2000 Single simulation

LANL, NCSA, NERSC, SDSC, ZIB, Garching, …

SC99 Colliding Black Holes

using Garching, ZIB T3E’s, with remote collaborative interaction and visualisation at ANL and NCSA booths

2000 Single simulation

LANL, NCSA, NERSC, SDSC, ZIB, Garching, …


Galaxy Merger

Zur Anzeige wird der QuickTime™ Dekompressor „YUV420 codec“

benötigt.


Consortium AIP (Lead) AEI ZIB

ZAH

MPA MPE TUM

AstroGrid-D Consortium

Heidelberg

Potsdam/Berlin

Garching

Associated Partners

UP

MPIAMPIfR

LRZRZGUSM

FZK


AstroGrid-D Work packages

Integration of compute and data resources

Managing and providing metadata Distributed management of data (files) Distributed query of data bases and

management of data streams Supervision and interactive management

of grid jobs User and application programmer

interfaces


Resource Integration

Building Grid Infrastructure: Unified configurations based on GTK 4.x Compiled, no binary installation required for MPI

(mpichG2) Each resource acknowledges GermanGrid and

GridGerman Root CA (D-Grid StA recommendation) VOMRS, registration of access to resources Each institute has a Registration Authority Development of translation from MDS information to

AstroGrid-D Information Service for Job-Submisson, Brokering, VO Management

Globus MDS is enhanced by Ganglia Firewall configuration according to D-Grid-StA

recommendation


Certification / RA / Usermanagement

Certification / RA RA:

• MPA/MPE: RA via RZG (DFN)• TUM: -> (DFN)• AIP : RA established (GridKA)• AEI : RA established (GridKA)• ZIB : RA established (DFN)• ZAH: RA established (GridKA)

Usermanagement• Adopted the gridpool approach (gLite)

Grid-users are created in pools, allowing of an easy mapping for VO to (unix) groups, for accounting and tuning of local resource access policies


VO Management

Virtual Organisations D-Grid management tool (VOMS) is only available with a nearly

fully fledged gLite-Installation• Most Communites base their Grid environment on Globus

VOMRS is derived from VOMS, but is easily compatible with GTK

Benefits of VORMS: • Frontend: Java-webapp with https, uses certificates• Enhanced facilities for VO-Group and User-Management, extensible

to registering resources and their policies• SOAP clients builtin• Backend for customized queries of the database

AstroGrid-D has a grid-map merge procedure allowing to fine tune the grid-maps for local resource providers


Architecture: Overview


AstroGrid-D Metadata

Resources (computational, storage, robotic telescopes, instruments, network, software) Well-defined schemes

GLUE schema (computational, storage, software)

RTML (robotic telescopes) State of grid services

(jobs, files, data stream, ...)


Metadata (cont.)

Scientific metadata (domain-specific description of data sets, provenance)

Application-specific metadata (job history, job progress, ...)

Schemes will be decided later by the community


Requirements

Extensible/flexible schemes Easy to extract and export metadata Protect from unauthorized access Support exact match and range queries Handle different metadata characteristics


Approach

Metadata representation with RDF An RDF entry is a (subject, predicate, object)-

tuple A set of triples/statements form a graph

Metadata access via SPARQL Query language for RDF Graph pattern matching

Simple interface including add, update, remove and query


RDF Example

“Maria”

Photographer“A picture of the Eiffel tower has a photographer with value Maria”


RDF Example

“Maria has a phone number with value 555-444”

“Maria”

555-444

“Maria”

Photographer“A picture of the Eiffel tower has a photographer named Maria”“A picture of the Eiffel tower has a photographer with value Maria”

Phone number


RDF Example

“Maria has a phone number withvalue 555-444”

“Maria”

555-444

“A picture of the Eiffel tower hascreation-date with value 2003.06.05”

Photographer

“Maria”

555-4442003.06.05

Creation-date

“Maria”

Photographer“A picture of the Eiffel tower has a photographer named Maria”

Phone number

“A picture of the Eiffel tower has a photographer with value Maria”

Phone number


SPARQL Example

Photographer

“Maria”

555-444

“Get the name and phone number of the photographer who took the picture of the Eiffel tower” Phone number

SELECT ?name, ?number WHERE{“Picture of Eiffel tower” “Photographer” ?name . ?name “Phone number” ?number }

Number Name

555-444 Maria

2003.06.05

Creation-date

Input graph

Output results


Information Service (Architecture overview)


Storage interface

Add(String RDF, String context) Update(String RDF, String context)

Overwrite matching statements Remove([statements], String context)

Delete existing metadata part of the information service storage

Query(String sparql_query) Extract metadata from the information service or

RDF information producers


Demo: Overview

Idea: use Google's map API to present grid resources using RDF metadata provided by the information services

Tools MDS4 WebMDS produces an XML representation

of resource information Template language or XSLT for translating to RDF RDF store Web service interface to add and query the RDF

store


Demo: Component interaction


Demo: SPARQL queries

SELECT ?site, ?lat, ?long WHERE{?site rdf:type “Site” . OPTIONAL {?site geo:lat ?lat . ?site geo:long ?long }}

“Get all sites and their longitude and latitudeif available”

“Get all computing elements from site S”

SELECT ?ce WHERE {S “ComputeElement” ?ce}


Demo: RDF graph (example)

10.25

<SiteID>

<ClusterID>

<ComputeElementID>

51.30

default 2

48

#Running jobs

#FreeCPUsName#CPUs

Cluster Long

Lat

ComputeElement


DataStream Management


DataStream Management


Job Management (Architecture)


UseCase Dynamo

Modelling the turbulent dynamo in planets, stars and galaxies

(solving the induction equation with turbulent electromotive force)

Moderate Requirements:• some MB to few GB main memory and <= 10 GB of local disk space• Runtime: hours to days, Fortran90 code / compilers• Handling of plain files only• Concurrent start of multiple simulations with different input parameters


Preparation of Taskfarming

• One directory for each model (parameter sets) containing input parameter files and the executable or code and Makefile

• Job Description (Globus XML file for multijobs) • data location (input)• FQDN names of compute resources• data location (output)


Multi-Job XML

..........


Start Taskfarming

Single sign-on to grid$>grid-proxy-init

Submit Multi-Job$>globusrun-ws –submit -b –f job.xml \

–J –S –o epr.xml

Status requests & monitoring:$>globusrun-ws –status –j epr.xml

globusrun-ws –monitor –s –j epr.xml


Multi-Job Flowchart

Use

rG

rid

Res

ou

rce

Gri

d R

eso

urc

eG

rid

Res

ou

rce

Gri

d R

eso

urc

e

grid-proxy-

init

SubmitMulti-Job

XML

Multi-Job Description

Accept Multi-Job

Multi-Job Description

Dele-gate 1. Job

Dele-gate

N. Job

Accept singleJob

Accept singleJob

FileStage-In

Single-Job Descriptions

ExecuteTask

FileStage-

Out

InputData

1

Task

DeliverData &Task

Re

que

st F

iles

InputData

1 Task

Sen

d F

iles

OutputData

1

CleanUp

SaveOutputData

Sen

d O

utp

ut F

iles

OutputData

1

FileStage-In

ExecuteTask

FileStage-

Out

InputData

N

Task

OutputData

N

CleanUp

DeliverData &Task

SaveOutputData

OutputData

N

InputData

N

XML

HandleStatus

Info

ProvideMonitoring

Data

Job EPR

Job EPR

RequestJob

Status

RequestJob

Monitoring


JDSL and RSL

Why consider another Job Description Language ? Globus RSL, which is processed by GRAM,

lacks some features, which are required for a detailed description of many jobs

Simulations require often the compilation before the actual job is run

Datamining, where the first stage of the job has to acquire the actual data to operate on from remote archives/databases

Defining workflows is next to impossible


JDSL and RSL

JDSL is developped by a GGF working group AstroGrid-D started to develop a JSDL to RSL

translator to meet the requirements of our usecases

Commandline Interface

takes XML-file as inputproduces RSL-file for globus-run command

$>jsdlproc jsdl-examples/nbody6.jsdl > temp.rsl &&\globusrun-ws -submit -staging-delegate -job-description-file temp.rsl -F < globushost >


JDSL and RSL


JDSL and RSL


GAT (Grid Application Toolkit)

•GAT is not a new grid middleware

GAT enables an easy access to different grid infrastructures

• GRAM• Condor• Unicore• GridFTP• ...

GAT is an OpenSource project

•GAT is a framework to enable an uniform access to different grid middleware



• Applications call the GAT-API for grid operations Applications have to be linked against the GAT

• Applications are independend of the underlying grid infrastructure

GAT Engine loads available adaptors at runtime During a calls of the GAT-API the GAT engine decides, which adaptor performs the grid operation In error case during a „grid operation“, another adaptor will be accessed

• default adaptors offer local operations grid applications can be compiled, linked and tested without available grid services The same application can run in a „complete grid environment“: without a new build


•The GAT-API doesn‘t change. Changes e.g. in Globus job submit are included in the GAT Globus adaptor

•GAT offers reliability. If one grid service is not available, another grid service is used

•GAT is much eaysier to install than Globus

•GAT has now a Java-API, allowing for pure client-installations in combination with JAVA-COG (AstroGrid-D development)

•GAT offers grid with a minimum effort for the end user



•GAT doesn‘t replace any function of a grid middleware

•Without an adaptor for a grid middleware GAT is useless for this middleware

•GAT has no resource broker



GAT Adapter

SGE

PBSGTK4

Globus 2/3.xUnicore

DRMAA

Application

Application layer

GAT API GAT Engine

GAT layer

User S

pace

„Grid

“ Sp

aceGAT (Grid Application Toolkit)


Robotic Telescope GridRobotic Telescope Grid

24h-observation, no reaction delay.

Independent of local weather.

Use GRID-technology in an uplifted abstraction layer for robotic telescopes

IVOA (VOEvent) and HTN (Heterogeneous Telescopes Network)

AIP Liverpool JMU U GöttingenT. GranzerT. Granzer

Hawaii

Australia

Texas

La Palma Tenerife

South Africa

PotsdamArizona


Robotic Telescopes


LOFAR / GlowGrid

GLOW: German LOng Wavelength Consortium represents the interests of the german

astronomy community in the LOFAR project AIP, MPA and MPIfR are members of GLOW Goals of GlowGrid:

Grid methods and infrastructure for radioastronomy community

Developing grid based methods for “real time” processing of antennae signals

Long term storage for LOFAR/GLOW data


LOFAR / GlowGrid

Partners: AIP, MPA, MPIfR, ZIB, FZJ, KIP TLS, RUB, ZAH, TUM

Volume 5 FTE x 3 years + hardware Project is placed as community project, but

AstroGrid-D will be managing the project and the workpackages are added to our current structure


LOFAR

25 50 75

BremenBremen


FrequencyCoverage

Sensitivity Improve-ment

Resolu-tionImprove-ment

Polarization Purity

Flexibility+ New features

LOFAR(2006-2015)

20-200 MHz ~100-1000 100-1000 > 30 dB Digital pointingfrequency agilityall-sky FOVtime buffering

SKA

(>2015)

0.1-20 GHz ~100 ~1 40 dB digital + mechanical pointingfrequency agilitylarge FOV

submm-VLBI 40-600 GHz ~10 ~10 > 20 dB

ALMA(2008-2020)

40-1000 GHz

~10-100 ~10 > 30 dB Fast mechanical pointing

frequency agility

LOFAR


LOFAR: Wide Area Sensor Network


LOFAR: Grid-Environment


LOFAR: Grid-Environment

Data Storage: Distributed storage of data-product in SDC Access of data from distributed collaborations

Post Processing: Central Processing Facility Distributed Computing Facilities (SDC,

collaborations)

Access to LOFAR User-Interface Access to scheduling & monitoring information Event registering & triggering


LOFAR: Science Operating / Data Centers

Data Product of LOFAR:e.g. 6 TB of raw visibility data for an 8 beam, 4 hour synthesis

observation, after integration for 1 sec and over 10kHz.

One month of observing in this mode results in ~1 PetaByte

Normally final data-products are exported (from CPF) Archiving and scientific use is the responsibility of the

SO/DC for routine observation Intermediate data products can be exported to SO/DC,

provided sufficient bandwidth (1 Gbit-Connection; a 1 Beam, 4h integration takes ~100min)


LOFAR / GlowGrid


GAVO: Context IVOA

GAVO (German Astrophysical Virtual Observatory)

is part of international efforts of the Astronomical und

Astrophysical Community, to

Achieve interoperability of observational data and simulation

data archives

Development of software and tools for Astronomy towards a

„multi-wave-length“ view of the Universe

Defining standards and protocols for enabling unified access to

VO-compliant data archives


GAVO/AstroGrid-D: Clusterfinder

Simultaneous search for X-Ray clusters of galaxies X-Ray photon maps (RASS) Optical galaxy maps (SDSS)

(P. Schuecker et al. MPE)

Clusterfinder: VO-ConeSearch Distributed Computing on AstroGrid-D Integration of Results

(H. Enke/P.Schuecker/A.Carlson)

(see link: Clusterfinder)


AstroGrid / GAVO: Clusterfinder

Web

serv

ice

GridQueue

GridJob-Runner

VO-Archive access

ClusterFinder.exe-1processor-100 MB memory-10 GB dataspace- resources (maps, psf)

14.09.2006

Current Simulation: MareNostrum

Current Simulation: MareNostrum

Zur Anzeige wird der QuickTime™ Dekompressor „“

benötigt.

A.KahlatlayanA.Kahlatlayan


AstroGrid-D and GAVO

GAVO developped a VO-compliant data base and

query interface for useful cosmological

information (Halos and other substructures)(SOAP and HTTP-interface)

(see link: Millennium)

AstroGrid-D will support the grid-based storage of astronomical archives Databases simulation archives

and access based on both grid and VO protocols


AstroGrid-D and GAVO

AstroGrid-D: Infrastructure Network Computing Resources Grid-Technologie

GAVO: Interoperability of astronomical archives Virtual Observatoy compatible

Software Tools Database-Interfaces User-Interfaces


Thanks for your attention

Visit our website

www.gac-grid.org

Thanks to all the contributors from AstroGrid-DThanks to all the contributors from AstroGrid-D


Links to AstroGrid-D and IVOA

AstroGrid-D: http://www.gac-grid.org (Project website) http://www.gac-grid.org/project-products.html (Help, WebMDS, Software) http://cashmere.aip.de:8080/gridsphere/gridsphere (Portal) http://cashmere.aip.de:8080/gridsphere/gridsphere?cid=astrogridmap

(Demo) https://vomrs.gac-grid.org (VOMRS only with a valid D-Grid-Certificate) http://jean-luc.aei.mpg.de/Press/BH1999/ (Simulations) http://www.aip.de/People/msteinmetz/Movies.html (Simulations) http://www.aip.de/~arm2arm/ (Simulations) http://www.mpa-garching.mpg.de/mpa/research/current_research/hl2004-

8/hl2004-8-en.html (Simulations)

GAVO: http://www.ivoa.org/ (IVOA, International Virtual Observatory Alliance) http://www.g-vo.org (GAVO website) http://www.g-vo.org/clusterfinder/ (Clusterfinder) http://www.g-vo.org/portal/tile/products/services/mpasims/index.jsp

(Millennium Simulation Database)

Documents

14.09.2006. 2 GridKA School 2006, H. Enke, AIP 14.09.2006 Overview AstroGrid-D History Project Project Structure Goals Current Current status