WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis Wide Area Grid Concepts (WGISS-22, Annapolis, sept.2006 ) [email protected] Centre National

WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis

Wide Area Grid Concepts(WGISS-22, Annapolis, sept.2006 )[email protected] National d'Etudes Spatiales,Programme & Strategy directorate,"Space Information Systesms"

Introduction Implementation concepts Way ahead

Introduction Implementation concepts Way ahead


50 major international systems ! Quote: WGISS user chair report,

Kiev, Sept 2005 !!

how to deal with the complexity wall ?How to deal wih the Babel syndrome ?

A pressing need: data & service large scale integration


WGISS also addresses the needs of data and information users by aiding the development of simpler and wider access to the resources and systems they require. WGISS places great emphasis on the use of demonstration projects involving user groups to solve the critical interoperability issues associated with the achievement of global services. WGISS has developed and will capitalize on a number of tools, standards, and services to assist access to, preservation of, and efficient use of Earth observation data resources.

John Faudeen, WGISS chairman, 2005

WGISS directions


WAG+Virtualization is the key ?


GRID

Classical GRID based

To day EO large scale objectivesWorld-wide follow-on ozoneWorld-wide follow-on ozone

World-wide follow-on of sea state (MWorld-wide follow-on of sea state (MEERCATOR)RCATOR)

World-wide follow-on of vegetation indexWorld-wide follow-on of vegetation index

World-wide follow-on of crop and food resourcesWorld-wide follow-on of crop and food resources

GMES programmeGMES programme

GEO endeavourGEO endeavour

........

Can we afford on the long run to pileCan we afford on the long run to pile--up investments as we do todayup investments as we do today

WAG-ObjectivesWAG stand for Wide Area Grid is in fact nothing new with respect to WAG stand for Wide Area Grid is in fact nothing new with respect to GRID technology apart from the fact it deals with a particular type of GRID technology apart from the fact it deals with a particular type of GRID system implementation.GRID system implementation.

In fact GRID concept as derived from Ian foster ideas is basically a set of In fact GRID concept as derived from Ian foster ideas is basically a set of hardware machines being connected together with basically no pre-hardware machines being connected together with basically no pre-specified implementation topology.specified implementation topology.

The major difference in between a “distributed system” and a GRID lies in The major difference in between a “distributed system” and a GRID lies in the fact the hardware organisation and the job handling is not anymore the fact the hardware organisation and the job handling is not anymore carried out by the application software itself but by a suitable MiddleWare carried out by the application software itself but by a suitable MiddleWare supposed in theory to hide the actual hardware arrangement specificity.supposed in theory to hide the actual hardware arrangement specificity.

=>> WAG experiments should enable testing a Widely shared =>> WAG experiments should enable testing a Widely shared GRID implementation carrying several EO Applications in a GRID implementation carrying several EO Applications in a concurent way concurent way

May prefigure GMES or GEO up-coming May prefigure GMES or GEO up-coming genericgeneric technical technical architecturesarchitectures

WAG solutions

Intra-GRID (WAG1)Intra-GRID (WAG1)

Sharing of individual and widely dispersed Sharing of individual and widely dispersed installations “seen” as a unique and general installations “seen” as a unique and general purpose GRIDpurpose GRID

Inter-GRID(WAG2)Inter-GRID(WAG2)

Connection of widely dispersed GRIDS either:Connection of widely dispersed GRIDS either:

- for the sake of a given application- for the sake of a given application

- for the sake of creating a unified entity (GRID - for the sake of creating a unified entity (GRID or Supr-GRIDor Supr-GRID

The MiddleWare question

WAG1: Just use Ian Foster vision by making the WAG1: Just use Ian Foster vision by making the distributed resources invisible while operated by distributed resources invisible while operated by a common MiddleWare as a unique Grid a common MiddleWare as a unique Grid resource resource

------------------------------------------------------------------------------------------------------------------------

WAG2: either enabling ad’oc solutions for Grids WAG2: either enabling ad’oc solutions for Grids connections or creating a “Super” MiddleWare connections or creating a “Super” MiddleWare able to catch and operate heterogeneous and able to catch and operate heterogeneous and widely distributed GRID systemswidely distributed GRID systems

WAG1: Intra Grid

WAG (in its original description from WAG (in its original description from WGISS20) works by melting together WGISS20) works by melting together existing hardware resources from as existing hardware resources from as many place as needed in a UNIQUE GRID many place as needed in a UNIQUE GRID (because by essence a local GRID is (because by essence a local GRID is nothing different than a highly nothing different than a highly dispersed GRID).dispersed GRID).

Pros:Pros: Zero Middleware development and Zero Middleware development and full compatibility right from scratch, full compatibility right from scratch, straightforward implementation straightforward implementation

Cons:Cons: Need to agree on the rule of the Need to agree on the rule of the game: patch all harware resources in a game: patch all harware resources in a network and operate it in a common network and operate it in a common middleware.middleware.

WAG2: Inter-Grids

Inter-Grid : means ways to make different and Inter-Grid : means ways to make different and isolated GRID systems becoming unified either isolated GRID systems becoming unified either by by

- creating some added middleware "gluing" all - creating some added middleware "gluing" all GRIDS togetherGRIDS together

- generating exchanges rules for all connected - generating exchanges rules for all connected GRID to speak together (kind of equivalent to GRID to speak together (kind of equivalent to Web service composition but with GRIds)Web service composition but with GRIds)

Pros: allow heterogeneous GRIDS to speak Pros: allow heterogeneous GRIDS to speak togethertogether

Cons: Not simple by essence since close to be Cons: Not simple by essence since close to be as complicated as making speaking machines as complicated as making speaking machines together (may mean a middleware for making together (may mean a middleware for making heterogeneous middleware compatible ?)heterogeneous middleware compatible ?)

WGISS & WAGRole of WGISS forwarding recommendation on Role of WGISS forwarding recommendation on “upcoming solutions” able to simplify Remote sensing “upcoming solutions” able to simplify Remote sensing data utilization and societal regognition.data utilization and societal regognition.

A system is not the “pilling up” of needed products or A system is not the “pilling up” of needed products or services but better a suitable balance in between generic services but better a suitable balance in between generic technological solutions and the thematic.technological solutions and the thematic.

Soon or later we will have to switch to “virtual Soon or later we will have to switch to “virtual cooperative solutions” and better capitalize on the cooperative solutions” and better capitalize on the enormous simplification brought by GRID conceptsenormous simplification brought by GRID concepts

CEOS-WGISS have ever shown the way to go with:CEOS-WGISS have ever shown the way to go with:

CEOS format, CEOS AVHRR 1km maps, CEOS CIP and CEOS format, CEOS AVHRR 1km maps, CEOS CIP and time has come for WGISS to introduce CEOS-WAG as time has come for WGISS to introduce CEOS-WAG as the next needed evolutionthe next needed evolution

What GEOSS or GMES need ?

Thematic solutions identified on a case by case Thematic solutions identified on a case by case basis (pollution, risk, ozone etc..)=> free mandate basis (pollution, risk, ozone etc..)=> free mandate given to agencies to deliver a service (present state given to agencies to deliver a service (present state of the art, very expensive since lot of technical of the art, very expensive since lot of technical duplication, low portability)duplication, low portability)

““Making thematic beeing virtualized” and leave Making thematic beeing virtualized” and leave technical implementation being WAG based (Full technical implementation being WAG based (Full use a Grid properties, MiddleWare unification, full use a Grid properties, MiddleWare unification, full portability, easy updates and evolutions, LOW portability, easy updates and evolutions, LOW COST and full distribution of development effort)COST and full distribution of development effort)


Vision: Technological evolutions

Blueprint plans (1850) CATIA (1985) (virtual mockup)

From Real to « Virtual »

Formal programming (1948) WINDOWS (1990)

Electronic circuitry (1914) SPICE simulators (1985)

MINITEL Physical links(1970) INTERNET (1990)

Processing systems (1960) eInfrastructures


The "Silos" infrastructure:Complex, little flexible, and Expensive

Not Scaleable

Single Points of Failure

Underutilized Hardware

Dedicated Staff

Applications Tied to Specific Hardware

• IT Staff Focused on Maintenance, Not Competitive Advantage

• Overall Resource Utilization: 10%-30%; No Resource Sharing

B o ttle n e c k e d A p p lic a tio n s

U n ix S M P

A p p lica tio n B

L in u x C lu s te r

A p p lica tio n C

M a in fra m e

A p p lica tio n D

W in d o w s S e rv e r

A p p lica tio n A


Scenario: Evolution of networks

Communication Networks

Computer Networks

eInfrastructures (networks +

services)

Tomorrow:Virtual GridsVirtual NetworksOutsourcing of Processes

Today:Physical ComputersPhysical Networksand Security issues!

Who will take the advantage in this new avenue of Business?

GRID

?


The Shift to Service-Oriented Computing

Source: Credit Suisse First Boston and DataSynapse

SERVICE-ORIENTED ARCHITECTURE

Services Policies Virtual Resources

Users Physical Resources

SERVICE-ORIENTED INFRASTRUCTURE

Applications

DemandDemand SupplySupply

Provisioning

Charge-Backs

Provisioning

Metering

VirtualResource

Market

VirtualResource

Market

Utility Grid

Compute& Service Grids

DataGridS

tora

ge

Pro

ce

ss

ing

Ne

two

rks

Functionality

SLAs

Accountability


The WAG Approach: A New Consumption

and Fulfillment Model for IT • Creates a Creates a virtual application operating virtual application operating

environment environment

• Virtualizes application services Virtualizes application services executionexecution

• Dynamically fulfills requests Dynamically fulfills requests over a virtual pool of system resourcesover a virtual pool of system resources

• Offers an adaptive, self-managed Offers an adaptive, self-managed operating environment that guarantees operating environment that guarantees high availabilityhigh availability

• The business gets what they need, The business gets what they need, when they need it, when they need it, on demandon demand

WAG

App BApp A App C

Application Layer

Systems Layer


Induced conceptual changes

No more knowledge of information or/and data-bank physical location (an archive is just known to be accessible trough its naming)No more need to describe the physical connections in between different locations (since fully supported by the underlying GRID MiddleWare) making in theory any world-wide distributed systems easy to implement.No more physical handling of information sets. This could trigger completly new merchandising concepts (for ex. come close to Telecomm recharge)No need to know the mapping of user ’software enabling virtual service to be easily created and offered to any-one on an autonomous fashion (virtual stores/services)


Integrated Generic Service layer will

happen because:

Higher needs and systems costs are not compatibleNeed to fight against the complexity barrier (rely on Middleware/UpperWare system management)New business are appearing making dedicated structure not competitive anymoreAre we sure we need on the long term specific resources for the only sake of Earth Observation ? WAG or/and WEB2 for systems development +Google technologies evolution for user access may become common place and incredibly cheap

WE NEED WGISS HELP FOR IMPLEMENTATION

Documents

WGISS22 Annapolis,11 sept.2006, CNES/J.P.Antikidis Wide Area Grid Concepts (WGISS-22, Annapolis, sept.2006 ) [email protected] Centre National