Open Grid Service Architecture - Data Access & Integration (OGSA-DAI)

Dr Martin Westhead

Principal Consultant, EPCC

Telephone: +44 131 650 5958 Fax: +44 131 650 6555 Email: M.Westhead@epcc.ed.ac.uk

The Grid: An Overview

The Grid• Distributed computing model • Based on platform and protocol neutral standards• Resource virtualisation and resource sharing

– Hardware, storage, network, data, function, instruments

• Service oriented model – Discovery – Negotiated access and allocation– Introspection and management of state

• Unlimited resources– Dependability– Performance and scalability

• Community driven standards process – Global Grid Forum (GGF)– Open source reference implementations (Globus, OGSA-DAI)

Challenging Requirements

• Dynamic formation and management of virtual organisations

• Online negotiation of access to services: who, what, why, when, how

• Configuration of applications and systems able to deliver multiple qualities of service

• Autonomic management of distributed infrastructures, services, and applications

The Grid Concept

System Users

IntelligentInterface

Middleware

Cluster OperatingSystem

Supercomputing

Networking

Mass Storage

Jack Dongarra

Open Grid Service Architecture

Service Oriented Architecture

ServiceRegistry

ServiceRequestor

ServiceProvider

Find Publish

Bind

The Best of Both Worlds

Web Services

Business integration

Secure and universal access

Applications on demand

Grid Protocols

Vast resourcescalability

Global Accessibility

Resourceson demand

ContinuousAvailability

Accessresource

Manageresource

Shareresource

Open Grid Services Architecture

See: The Physiology Of The Grid …

Open Grid Services Architecture

• Web services– Standard interface definition mechanisms: multiple protocol

bindings, multiple implementations, local/remote transparency– Address discovery & invocation of services

• Grid Service extensions– Transient services– State management in a distributed environment

• Resource virtualisation• Multiple hosting targets: J2EE, .NET, “C”, …

OGSA-DAI Programme

OGSA-DAI Motivation

• UK e-Science Core Programme– Database Taskforce

– Grid data requirements scoping study

– Data Access and Integration BoF at GGF4

– DAIS Working Group in Global Grid Forum

• Exploit emerging Grid computing concepts (OGSA)

• Web Services adoption

• DBMS adoption

• Meet needs of Grid application developers for UK e-Science community – Grid Pilot projects

UK Grid Pilot Projects

Computational Fluid DynamicsGeodise

Bioinformatics, Environmental & Earth sciencesDiscoveryNet

Virtual ObservatoryAstroGrid

Interactive steering of data & compute intensive analysis – chemistry, earth sciences

RealityGrid

Combinatorial (bio-)chemistryCombiChem

Distributed Aircraft Maintenance EngineeringDAME

eScience workbench, BioinformaticsmyGrid

Research Focus

OGSA-DAI Requirements

• Transient and persistent data - Data sources & Resources

• Maintain complex data models - Information/Knowledge

• Record and maintain ‘data about data’ - Metadata

• Establish reliability and quality of data - Provenance

• Make data more accessible - Publishing and Discovery

• Restrict who can read and modify data - Access control

• Receive data in a state ready to use - Transformation

• Personalise data - Analysis/Interpretation

• Quality of Service - Managing data

DAI Programme of Work

• Two phases– Phase 1 Feb – Sept 2002 complete

– Phase 2 Oct 2002 – Jul 2003 complete

• Grid data service reference implementations– OGSA compliant

– DB2, MySQL, ORACLE, XINDICE

• Integration with Globus toolkit• Input to Global Grid Forum on standards

recommendations• Collaborative Programme

IBM UK & IBM US

Participating Organisations

Edinburgh Parallel Computing Centre

North East eScience Centre

National eScience Centre

North West eScience Centre

Oracle Corporation UK

OGSA-DAI Basic Services

OGSA-DAI Distributed Query

Database, Communication, OS… Technology

Resource Grid Infrastructure – OGSA…

Data Grid Infrastructure – Location, Delivery, Replication…

Database (Xindice, MySQL

Oracle, DB2)

Request to Registry for sources of data about “x”

Registry responds with Factory handle

Request to Factory for access to database

Factory creates GridDataService

Factory returns handle of GDS to client

Client queries GDS with SQL, XPath, XQuery etc

GDS interacts with database

Query results

returned XML

SOAP/HTTP

service creation

API interactions

Analyst

RegistryGDSR

FactoryGDSF

Grid Data Service

GDS

Consumer

ORdelivered to consumer

as XML

OGSA-DAI Behaviours

Delivery Patterns

A G

Q

S + R

AG

Q + D

S

RC

A G

Q + U

S

A G

Q

S

UP

Retrieve Update/Insert Pipeline

A

G1 = PQ1 + D

S1

U/RA

G

Q

S

DC

R

A G

Q + D

S

IP

U

I

G2 = CS2

Q2

A

G1 = PQ1

S1

U/R

G2 = CS2

Q2 + DI

1.

2.

3.

4.

6.

5.

7.

8.

Statement Operations

• Receives statement

• Performs statement against a data resource• Returns results

Statement Component and Database Connections

Data Resource

Statement Results

Statement Architecture

Engine/GDS-S

JDBC Connection XMLDB Connection

JDBC API Relational DB

XML DBXML:DB API

Statement

Statement Activity

Statement Handlers

JDBC Statement Handlers

XMLDB Statement Handlers

OGSA-DAI Security

• Grid Security Infrastructure Authentication

• The role mapper takes credentials and a database name and returns a dbRole

• dbRole contains username and password

Role Mapper

dbRoleCredentials

Database Name

username

password

OGSA-DAI Summary

• Reference implementation for DAIS standards recommendations

• Meet needs of Grid applications in e-Science

• Current phase provide basic building blocks

• Proposal for future work in progress– Automation, intelligence, higher level services

Release 3 available for download from:

www.ogsa-dai.org.uk