Http:// OGSA-DAI: Future Work and Wrap-up The OGSA-DAI Team info@ogsadai.org.uk

http://www.ogsadai.org.uk

OGSA-DAI: Future Work and Wrap-up

The OGSA-DAI Team

info@ogsadai.org.uk

2http://www.ogsadai.org.uk

Goals for DAIT

Aim to deliver application mechanisms that:– Meet the data requirements of Grid applications

• Functionally, performance and reliability

• Reduce development cost of data centric Grid applications

• Provide consistent interfaces to data resources

– Acceptable and supportable by database providers• Trustable, imposed demand is acceptable, etc.

• Provide a standard framework that satisfies standard requirements

A base for developing higher-level services– Data federation– Distributed query processing– Data mining– Data visualisation

3http://www.ogsadai.org.uk

Current Release

R4 April 2004– Provides Data Access components, an extensible framework for

building applications and some integration components– Built on top of Globus Toolkit 3.2– Supports relational, xml and some files

• MySQL, Oracle, DB2, SQL Server, Postgres, XIndice, CSV

– Supports various delivery options• SOAP, FTP, GridFTP, HTTP, files, email, inter-service

– Supports various transforms• XSLT, ZIP, GZip

– Supports message level security using X509 certificates– Client Toolkit library for application developers– GUI data browser (contributed by FirstDIG project)– Separate Distributed Query Processing components– Comprehensive documentation and tutorials

4http://www.ogsadai.org.uk

Roadmap / Workplan

Roadmap document available for comment:– http://www.ogsadai.org.uk/docs/OtherDocs/OGSA-DAIRoadmapV2.0.pdf

– User feedback required to drive this document

Integrate parts of DQP into OGSA-DAI core– Addressing platform dependencies– Want to include XML data resources

Undertaking data integration requirements capture process– Talked to 11 UK and US projects so far

Move Computation to Data– Java mobile code

5http://www.ogsadai.org.uk

Release 5

R5 October 2004 (Interfaces) – OGSI– Built on Globus Toolkit 3.2.1– Re-engineered to separate interface layer from core

OGSA-DAI functionality– Improved dependability and security integration– New file data resources representing flat files queried using

full text searches (e.g. EMBL format)– Installation and Configuration Wizard, including “all-in-one

installer”– Improved Data Browser which allows XPath querying.– Set of standard benchmarks.– JSP Quick View interface.– Support for other databases (e.g. Access, Exist, HSQL)

6http://www.ogsadai.org.uk

Release 5

Major refactoring exercise– Improve distinction between interface and core– Allow future support for different interfaces

Listened to major user concerns– Adding indexed text file access to support the

bioinformatics community– Improved robustness and usability– Addressed bugs as they were reported– Hope to add Activity Configuration, requested by projects

developing custom activities

Last release primarily targeted at the OGSI/GT3 platform

7http://www.ogsadai.org.uk

WS-I Technical Preview

A limited functionality evaluation version– An OGSA-DAI “Data Service” combining the metadata, configuration

and perform document capabilities of the OGSI-based GDSF and GDS services.

– Access to service metadata provided by a partial implementation of the WS-ResourceProperties specification.

– Example clients are provided for testing and coding reference.

Caveats/Issues:– No registry component, no support for third party delivery.– Security may be available (based on OMII WS-Security plug-in for

Axis).– Document schema and interfaces WILL change.– The WSDL is based on the OGSI-based WSDL from OGSA-DAI – Will not be supported to same level as main release for now.

Also released with OMII middleware distribution (Oct)

8http://www.ogsadai.org.uk

WS-RF Technical Preview

An evaluation version OGSA-DAI based on the Globus Toolkit 4.0 beta implementation of WSRF.

– Provides an amalgamation of the capabilities of the OGSI-based GDSF and GDS services (the metadata and configuration aspects of the GDSF and the metadata and perform document processing aspects of the GDS).

– Access to multiple data resources from a single service provided by data resource identifiers specified by a client within the WS-Addressing endpoint reference to a data service.

– Access to service metadata (database schemas, request status, etc) provided by an implementation of the WS-ResourceProperties specification.

– A WSRF version of the GridDataTransport portType supporting asynchronous data delivery between data services.

Caveats/Issues:– This preview of OGSA-DAI WSRF does not support data service security.– Document schema and interfaces WILL change.– Will not be supported to same level as main release.

Will also be released as part of the Globus Toolkit 4 beta

9http://www.ogsadai.org.uk

OGSA-DAI Project Webpage

http://www.ogsadai.org.uk

Background

News & Events

Software Releases

Documentation

Support

Training Courses

Links

10http://www.ogsadai.org.uk

FAQ, Support, Mailing List

Frequently Asked Questions– http://www.ogsadai.org.uk/support/faq.php– Updated as common problems become clear

Support for OGSA-DAI releases– http://www.ogsadai.org.uk/support– support@ogsadai.org.uk– Use to report problems

Users mailing list– users@ogsadai.org.uk– http://www.ogsadai.org.uk/support/list.php– General discussion of OGSA-DAI, data and the Grid

11http://www.ogsadai.org.uk

Second Users Group Meeting

Held alongside GGF12 in Brussels– ~40 attendees, 12+ projects represented

Users Group Chair appointed– Prof. Beth Plale, Indiana University

User presentations from– AstroGrid (UK), GeneGrid (UK), eDiamond (UK), INWA (UK),

LEAD (USA)– Also representatives from BioGrid (Japan), BRIDGES (UK),

caBIG (USA), GEDDM (UK), GEON (USA), myGrid (UK), SIMDAT (EU),…

A separate independent body to engage with users and feedback to developers in a formal way– Mailing list and website being setup

12http://www.ogsadai.org.uk

Take Home Messages

There are plenty of Research Challenges– Data integration and workflow on a global scale– Data resource and schema heterogeneity– Dynamic variability

• Authorisation, Resources, Data & Schema, Performance

– Some massive data– Metadata for discovery– Provenance tracking

Grasp the theoretical & practical challenges– Working in open & dynamic systems– Incorporate all computation– Welcome “code” visiting your data

13http://www.ogsadai.org.uk

Future DAI requires fundamental CS

What architecture best enables integration of data & computation?– Common conceptual models– Common planning & optimisation– Common enactment of workflows– Common debugging

What Fundamental CS is needed?– Trustworthy code & trustworthy evaluators– Decomposition and recomposition of applications– Metadata

Is there an evolutionary path? Are web services a distraction?

14http://www.ogsadai.org.uk

Conclusions

Still early days– Standardisation process not stabilising quickly enough– Infrastructure still developing and prone to change

OGSA-DAI acting as an enabler– Showing people what can be done– However is it cracking a nut with a sledge hammer?– Evolving and improving with each release

Usage patterns are similar– Call for people to work together to solve similar problems

Some problems are not OGSA-DAI specific– Metadata, time zones, security, …

Data discovery perceived to be important– Is this in the scope of what OGSA-DAI should be doing?

Please try it out!– It’s free and supported– Make suggestions, extend functionality, contribute to DAIS-WG

15http://www.ogsadai.org.uk

Links

OGSA-DAI Webpage– http://www.ogsadai.org.uk/

Globus Toolkit 3.2– http://www.globus.org/ogsa

Database Access and Integration Services (DAIS-WG)– https://forge.gridforum.org/projects/dais-wg

Grid Technology Repository– http://gtr.globus.org

ELDAS - Enterprise-Level Data Access Services (Eldas) – http://www.edikt.org/eldas

Web Services Choreography– http://www.w3.org/2002/ws/chor