Requirements from Requirements from astronomy in the Virtual astronomy in the Virtual Observatory eraObservatory era

Bob Mann Bob Mann

Institute for AstronomyInstitute for Astronomy & & NeSCNeSC

University of EdinburghUniversity of Edinburgh

Outline of talkOutline of talk

What is the Virtual Observatory (VObs)?What is the Virtual Observatory (VObs)?– How is it changing astronomy?How is it changing astronomy?

Who in astronomy needs e-Science skills?Who in astronomy needs e-Science skills?– Data centre staffData centre staff– VObs middleware developersVObs middleware developers– Data analysis tool developersData analysis tool developers– Research astronomers Research astronomers

SummarySummary– What skills are needed?What skills are needed?– How can these requirements be met?How can these requirements be met?

What is the What is the Virtual Observatory?Virtual Observatory?

A goalA goal::– An interoperable federation of the world’s An interoperable federation of the world’s

astronomical data sourcesastronomical data sources A standards agency/coordination body:A standards agency/coordination body:

– International Virtual Observatory Alliance International Virtual Observatory Alliance – Its members are VObs projects in Its members are VObs projects in

Australia, Canada, China, France, Australia, Canada, China, France, Germany, Hungary, India, Italy, Japan, Germany, Hungary, India, Italy, Japan, Korea, Russia, Spain, UK, USA, and the EU.Korea, Russia, Spain, UK, USA, and the EU.

Why is the VObs Why is the VObs needed?needed? We observe across the whole electromagnetic We observe across the whole electromagnetic

spectrum spectrum

Different views of a local spiral galaxyDifferent views of a local spiral galaxy– Need all of them to understand its physics fullyNeed all of them to understand its physics fully– The databases holding them are located all over the worldThe databases holding them are located all over the world

IRAS 252MASS 2DSS Optical IRAS 100 NVSS 20cmGB 6cmROSAT ~keV WENSS 92cm

Schematic of the VObsSchematic of the VObs

DB 1 DB 5 DB 4 DB 3 DB 2

Registry

Compute Resource 1

Compute Resource 2App 5

App 3App 2

App 1

App 4

Portal

Workflow

User

What services will the What services will the Virtual Observatory Virtual Observatory need?need? Data discoveryData discovery

– Looking up metadata in a registryLooking up metadata in a registry Data accessData access

– With/without authentication & With/without authentication & authorizationauthorization

Data integrationData integration– Cross-matching entries in databasesCross-matching entries in databases

Data manipulationData manipulation– Data mining, data analysis, etcData mining, data analysis, etc

Data centre staffData centre staff

DB 1 DB 5 DB 4 DB 3 DB 2

Registry

Compute Resource 1

App 1

• Data Curation:

•Provenance, DBMS design and operations

• Data Access

• Security, Registry Metadata, Web Services

• Data Manipulation

• User-uploaded code, job scheduling

VObs middleware VObs middleware developersdevelopers

DB 1 DB 2

Registry

Compute Resource 2

App 3

Portal

Workflow

Web/Grid services for the discovery, transport and integration of data

Workflow & service composition

Metadata and service registration

Job management

Security

Data analysis tool Data analysis tool developersdevelopers

DB 1

Registry

Compute Resource 2

App 3Workflow

Data Access:

• OGSA-DAI/data access web services

Service Registration

Integration into workflows

Job submission

Research astronomersResearch astronomers

DB 1 DB 2

Registry

Compute Resource 2App 5

App 4

Portal

Workflow

User

Initially:

• Use of portal

Soon:

• Script and tool development using APIs, local job control, wrapping apps as web services

Summary of skills Summary of skills neededneeded Web/Grid services for data discovery, Web/Grid services for data discovery,

access, integration and manipulationaccess, integration and manipulation Metadata for data & service registration Metadata for data & service registration Data curation – provenance, preservationData curation – provenance, preservation Workflow creation and executionWorkflow creation and execution Job control – local and remoteJob control – local and remote SecuritySecurity

How can these needs be met? How can these needs be met? (1)(1)

Learning through experienceLearning through experience– Difficult: fast moving field; where to start?Difficult: fast moving field; where to start?

Danger of learning the wrong technologiesDanger of learning the wrong technologies

– Inefficient: undirected training can be Inefficient: undirected training can be slowslow

Hiring people with appropriate skillsHiring people with appropriate skills– Not many people with e-Science skills yetNot many people with e-Science skills yet

……and expensive to employ on research grantsand expensive to employ on research grants

– Really need astronomers for some jobsReally need astronomers for some jobs Danger of losing scientific direction to projectsDanger of losing scientific direction to projects

How can these needs be met? How can these needs be met? (2)(2)

Existing staff: training coursesExisting staff: training courses– Can be expensive and time-consumingCan be expensive and time-consuming– Need to budget time and money for Need to budget time and money for

themthem Funders need to understand their necessityFunders need to understand their necessity

Future staff: educationFuture staff: education– Astronomers: MSc-level courses – either Astronomers: MSc-level courses – either

as MSc or as part of PhD trainingas MSc or as part of PhD training– IT specialists: inclusion of e-science IT specialists: inclusion of e-science

content in CS degree programmescontent in CS degree programmes