Upload
curtis-clark
View
222
Download
0
Tags:
Embed Size (px)
Citation preview
Birmingham/Warwick AstroGrid Science Workshop
Meeting Goals
Silvia Dalla
Oct 10-11, 2006
Workshop goals
• Aim to work with you, the early users of AstroGrid, to demonstrate its capabilities and the potential of the Virtual Observatory (VO)
• Introduce you to the elements of the system and how to do science with it
• Look at a science problem in your research area and use VO tools to tackle it
Workshop goals (contd)
• Collect your feedback
What you like/dislike
What needs to be improved
Which data you need
Which applications you want to see running in AG
Workshop Plan
• Today: background talks on AstroGrid and how the system works (am). Initial workgroup set up (pm)
• Tomorrow: workgroups work on science cases. Report and feedback Session (pm)
• Don’t miss the dinner (Today – 7:00 pm The Chamon)
What you can do with AstroGrid
AstroGrid Science Team
Silvia Dalla, Eduardo Gonzalez-Solares, Jonathan Tedds, Anita Richards and Nicholas Walton
Outline
1. Background: Virtual Observatories (VOs) and AstroGrid
2. Example of non-VO solution to a data retrieval and processing task: making a movie of solar images
3. The AstroGrid solution
4. Introduction to AstroGrid components
What is the VO?
www.encyclopedia.com: VO = collection of integrated astronomical data archives and software tools that utilize computer networks to create an environment in which research can be conducted
Thanks to Ohishi Masatoshi (NAOJ) for the pointer…
VOs worldwide
• Similar efforts now in 15 countries:– UK, USA, Canada, France,
Germany, Italy, Holland, Japan, Australia, India, China, Russia, Hungary, South Korea, ESO, Spain
• Active collaboration among projects– Standards, common demos– International VO roadmap being
developed– Regular telecons over 10
timezones
Formal collaboration:International Virtual Observatory Alliance (IVOA)
AstroGrid: the UK’s VO
• Consortium of several UK Universities, funded by PPARC to build a VO for the UK.
• Part of the Euro-VO www.euro-vo.org
• Astrogrid release 2006.3: www.astrogrid.org/launch
• Data discovery, access to data and applications.
AstroGrid Goals
• Enable science by:
– Improving the quality, easy, speed of on-line astronomy
– Making integration and comparison of data from diverse sources transparent
– Removing data access barriers to multiwavelength analysis
– Enabling access and manipulation of large datasets
The need for a VO
• Data volume doubles every year
• Increase of size and multiplex capabilities of new instruments– WFCAM: 100Gb/night (100Tb/yr)– VISTA: 300Gb/night (300Tb/yr)– e-MERLIN/ALMA: 1Tb/day– SDO: 1 Tb/day
• We will have Petabytes of data by 2010
The need for a VO
• Hubble UDF
• Million second exposure
• 6000x6000 pix
• 11.5 sq. arcmin
• 10,000 galaxies
The need for a VO
• SWIRE ELAIS N1
• 9 sq. degrees (~3000 UDF)
• (moon ~0.2 sq. deg.)
• ~ 600,000 objects
The need for a VO
• Solar Dynamics Observer
• Launch 2008
• >1 TB/day
• Over 5 years: 1.8 PB
Making a solar movie – non VO
Web interface to database of SOHO/EIT observations
Making a solar movie – non VO
Download imagesto local machine
Making a solar movie – non VO
IDL SolarSoft requiredto calibrate images andmake a movie. Routinesavailable, however eachuser rewrites code calling them to produce a movie.
Making a solar movie – non VO: weaknesses
• If user wishes to make a movie for a different time period, the above steps need to be repeated – by hand
• Similarly for astronomy data if one is interested in many objects
• The entire archive is not ‘visible’ to the user – only the downloaded subset can be processed
• Scripting eg with Python, Perl etc is possible though code is different for different archives – not easy for the ‘general’ user / for many datasets. This limits feasibility of multi-wavelength, multi-instrument work
Making a solar movie – with AG
Solar Movie Maker science workflow. Other science workflows available:Redshift Maker, Colour Cutter, Cone Search, SWIRE images
Making a solar movie – with AG: under the hood
• Access to database of observations – via AG DSA (DataSet Access) software
• Requests are sent using ADQL (Astronomy Data Query Language), similar to SQL
Database of observations(at archive)
DSAAG Workbench/workflow engine
Input: ADQL query
Output: table of observationssatisfying the query, in VOTable format
Making a solar movie – with AG: under the hood: CEA
CEAAG workbench/workflow engine
Input: CEA applicationinput parameters
Output: whatever the outputof the application is, delivered to user’s Myspace
• Capability to call a set of processing routines that do operations on data, or a model (apps in any programming language can be wrapped) – CEA Application
Application. Eg software that
processes data, model etc
(on apps server)
Making a solar movie – with AG: under the hood: Workflow
• Workflow capability so that queries to datasets and calls to applications can be managed
• Jobs are run remotely and asynchronously
• Queries and workflows can be re-used and shared
• AstroGrid is currently the only VO project with a workflow capability
AstroGrid: Workbench
• Workbench gives access to AG services
• Data discovery: AstroScope and HelioScope
• Task Launcher – send a query to a database or launch a application
AstroGrid: Workbench- contd
• Run a ready made science workflow
• Build your own workflow
• View your files in MySpace
AstroGrid: MySpace
• Virtual disk space where you can store results, temporary files, and new things like query files and workflow files, so you can adjust and re-run jobs on a later day.
• Visible from any computer.
Finding Information : the Registry
• How do you find the data you require?
• How do you decide which resource (data, application, information, disk, …) to use?
• The registries are the yellow pages for astronomical resources
• All VO registries harvesting each other: thus querying any one returns full list of globally held resources.
VOTable: an interchange format
• XML standard for the interchange of data represented as a set of tables
• A table is an unordered set of rows, as specified in the table metadata
• Each row is a sequence of table cells, each of them containing a primitive data type or an array of such primitives
• It bridges two ways to express structured data: XML and FITS
• Use of Universal Column Descriptors (UCDs) to express the content of each parameter.
<?xml version="1.0"?><VOTABLE version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.ivoa.net/xml/VOTable/VOTable/v1.1"> <COOSYS ID="J2000" equinox="J2000." epoch="J2000." system="eq_FK5"/> <RESOURCE name="myFavouriteGalaxies"> <TABLE name="results"> <DESCRIPTION>Velocities and Distance estimations</DESCRIPTION> <PARAM name="Telescope" datatype="float" ucd="phys.size;instr.tel" unit="m" value="3.6"/> <FIELD name="RA" ID="col1" ucd="pos.eq.ra;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Dec" ID="col2" "pos.eq.dec;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Name" ID="col3" ucd="meta.id;meta.main" datatype="char" arraysize="8*"/> <FIELD name="RVel" ID="col4" ucd="src.veloc.hc" datatype="int" width="5" unit="km/s"/> <FIELD name="e_RVel" ID="col5" ucd="stat.error;src.veloc.hc" datatype="int" width="3" unit="km/s"/> <FIELD name="R" ID="col6" ucd="phys.distance" datatype="float" width="4" precision="1" unit="Mpc"> <DESCRIPTION>Distance of Galaxy, assuming H=75km/s/Mpc</DESCRIPTION> </FIELD> <DATA> <TABLEDATA> <TR> <TD>010.68</TD><TD>+41.27</TD><TD>N 224</TD><TD>-297</TD><TD>5</TD><TD>0.7</TD> </TR> <TR> <TD>287.43</TD><TD>-63.85</TD><TD>N 6744</TD><TD>839</TD><TD>6</TD><TD>10.4</TD> </TR> <TR> <TD>023.48</TD><TD>+30.66</TD><TD>N 598</TD><TD>-182</TD><TD>3</TD><TD>0.7</TD> </TR> </TABLEDATA> </DATA> </TABLE> </RESOURCE></VOTABLE>
Table Metadata
Table data(XML, FITS)
VOTableExample
Visualising results: Topcat
Visualising results: Topcat
Astrogrid Help Pages
Linked from workshop pages
Summary
• Workshop is an opportunity to see AG in action
• Current capability can be of use to a diverse range of science problems
• New cases – perhaps emerging from this workshop – may demand new science services: input is welcome from you!