AstroNET / AstroWISE

Distributed data analysis and survey

production networks Liverpool, 14 June 2008

Distributed data analysis and survey

production networks Liverpool, 14 June 2008

Edwin A. ValentijnOmegaCEN

Kapteyn Institute University Groningen

Edwin A. ValentijnOmegaCEN

Kapteyn Institute University Groningen

What is What is

• an e-science infrastructure with fully distributed resources,

• which allows teams distributed over Europe • to jointly collaborate on the data production,

calibration, QC, analysis of wide field imagers, • in the optical, IR and radio.• Current: WFI@2.2m, INT, Subaru, ACS@ST • Near future: Panstarrs, VST, VISTA, Lofar• Far future: EUCLID

• an e-science infrastructure with fully distributed resources,

• which allows teams distributed over Europe • to jointly collaborate on the data production,

calibration, QC, analysis of wide field imagers, • in the optical, IR and radio.• Current: WFI@2.2m, INT, Subaru, ACS@ST • Near future: Panstarrs, VST, VISTA, Lofar• Far future: EUCLID

AstroNET / AstroWISE

–OmegaCAM 24% (~2500 GB)

–VIRCAM 72%–(~7500 GB)

–All Current Paranal–Instruments 4% (433.2 GB)

Paranal Monthly Data Rates 2007 statistics

–Mark Neeser –Mark Neeser

AstroNET / AstroWISE

UKIDSS workshop 2007

VIKING

VISTA

0.6 sq.deg. InfraRed camera

16 2kx2k detectors

0.35” pixels

KIDS

OmegaCAM @VST

1500 sq.deg. u g r i

2 Million CCD readouts

32 2kx4kdetectors

0.21” pixels

3 – 10 year project

Example Public Survey KIDS – VIKING

~250 nights

~500 nights

AstroNET / AstroWISE

Astro-WISEGrid Computing

Astro-WISEGrid Computing

AstroNET / AstroWISE

AstroNET / AstroWISE

AstroWISE-Up AstroWISE-Up

AstroNET / AstroWISE

LOFAR @ 20-80 MHz 120-240 MHz

LOFAR @ 20-80 MHz 120-240 MHz

- EoR

- transients

- Survey

- astroparticles

- EoR

- transients

- Survey

- astroparticles

AstroNET / AstroWISE

E-Lofar: from experiment to observatoryE-Lofar: from experiment to observatory

E-LOFAR Participants

E-LOFAR Participants

• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted

• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations

• Sweden:– Onsala station funded

• + Poland, Italy, Austria, Bulgaria, Ireland, France

• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted

• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations

• Sweden:– Onsala station funded

• + Poland, Italy, Austria, Bulgaria, Ireland, France

AstroNET / AstroWISE

AstroNET / AstroWISE

Lofar StorageLofar Storage

AstroNET / AstroWISE

Lofar distributed analysis Lofar distributed analysis

Astro-WISE: ServicesAstro-WISE: Services

AstroNET / AstroWISE

Extreme data lineage Extreme data lineage

AstroNET / AstroWISE

AstroWISEObservatories AstroGrid Euro-VO

EGEE - GRID

AstroWISEObservatories AstroGrid Euro-VO

EGEE - GRID

Distributed data analysis and

survey production networks

Distributed data analysis and

survey production networks