32
Craig Lee Craig Lee [email protected] [email protected] OGF-24 Mark Reichardt Mark Reichardt [email protected] [email protected] Grid-Enabled Geospatial Systems

Grid-Enabled Geospatial Systems

  • Upload
    yazid

  • View
    31

  • Download
    0

Embed Size (px)

DESCRIPTION

OGF-24. Grid-Enabled Geospatial Systems. Craig Lee [email protected]. Mark Reichardt [email protected]. Geospatial Data: Immense Applicability. Estimates vary, but ~80-90% of all data collected or produced by the human race is geospatially referenced. - PowerPoint PPT Presentation

Citation preview

Page 1: Grid-Enabled Geospatial Systems

Craig LeeCraig [email protected]@aero.org

OGF-24

Mark ReichardtMark [email protected]@opengeospatial.org

Grid-Enabled Geospatial Systems

Page 2: Grid-Enabled Geospatial Systems

2

Geospatial Data: Immense Applicability

Geospatial Data: Immense Applicability

Image: E. Gennai, C. Terbough, ESRI

Estimates vary, but ~80-90% of all data collected or Estimates vary, but ~80-90% of all data collected or produced by the human race is geospatially referencedproduced by the human race is geospatially referenced

• Natural exploration, e.g., oil & gas• Public administration• Civic planning and engineering• Weather and aviation• Environmental planning• Disaster management• Satellite ground systems

Page 3: Grid-Enabled Geospatial Systems

3

An Example: GEO GridAn Example: GEO Grid

Global Earth Observation

GridS. Sekiguchi,

AIST

Page 4: Grid-Enabled Geospatial Systems

4

The OGC-OGF The OGC-OGF CollaborationCollaborationThe OGC-OGF The OGC-OGF CollaborationCollaboration

• Promote best practices and international standardization for distributed geospatial data processing capabilities that is:• TransparentTransparent -- users is not aware of the infrastructure• InteroperableInteroperable -- the resources work together• ScalableScalable -- small local, to massive distributed platforms

• Promote best practices and international standardization for distributed geospatial data processing capabilities that is:• TransparentTransparent -- users is not aware of the infrastructure• InteroperableInteroperable -- the resources work together• ScalableScalable -- small local, to massive distributed platforms

Directly Relevent to Satellite Ground SystemsDirectly Relevent to Satellite Ground Systems

Page 5: Grid-Enabled Geospatial Systems

5

Two IssuesTwo Issues

• What geospatial and computing technologies must be integrated?

• How do we manage the process to make progress on doing so?

• What geospatial and computing technologies must be integrated?

• How do we manage the process to make progress on doing so?

Page 6: Grid-Enabled Geospatial Systems

6

Some Key OGC StandardsSome Key OGC Standards• Web Map Service (WMS)

• Display of registered and superimposed map-like views of information that come simultaneously from multiple remote and heterogeneous sources.

• Web Feature Service (WFS) • Retrieval and update of digital representations of real-world

objects referenced to the Earth’s surface.

• Web Coverage Service (WCS) • Access to a grid coverage, usually encoded in a binary

format, and offered by a server.

• Catalogue Service for the Web (CSW)• Common interfaces to publish, discover, browse, and query

metadata about data, services, and other resources

• Web Map Service (WMS)• Display of registered and superimposed map-like views of

information that come simultaneously from multiple remote and heterogeneous sources.

• Web Feature Service (WFS) • Retrieval and update of digital representations of real-world

objects referenced to the Earth’s surface.

• Web Coverage Service (WCS) • Access to a grid coverage, usually encoded in a binary

format, and offered by a server.

• Catalogue Service for the Web (CSW)• Common interfaces to publish, discover, browse, and query

metadata about data, services, and other resources

Page 7: Grid-Enabled Geospatial Systems

7

Key OGC StandardsKey OGC Standards• Sensor Observation Service (SOS)

• Access observations for a sensor or sensor constellation, whether in-situ or dynamic (e.g, water monitoring or satellite imaging). Optionally access associated sensor and platform data

• Sensor Planning Service (SPS)• An interface to task sensors or models. Using SPS, sensors can be

reprogrammed or reconfigured, sensor missions can be started or changed, simulation models executed and controlled.

• Sensor Alert Service (SAS)• Defines an interface for publishing and subscribing to alerts from

sensors. If an event occurs the SAS will notify all clients subscribed to this event type.

• Observations and Measurements Encoding Standard (O&M)• Defines an abstract model and an XML schema encoding for

observations and it provides support for common sampling strategies. It also provides a general framework for systems that deal in technical measurements in science and engineering.

• Sensor Observation Service (SOS)

• Access observations for a sensor or sensor constellation, whether in-situ or dynamic (e.g, water monitoring or satellite imaging). Optionally access associated sensor and platform data

• Sensor Planning Service (SPS)• An interface to task sensors or models. Using SPS, sensors can be

reprogrammed or reconfigured, sensor missions can be started or changed, simulation models executed and controlled.

• Sensor Alert Service (SAS)• Defines an interface for publishing and subscribing to alerts from

sensors. If an event occurs the SAS will notify all clients subscribed to this event type.

• Observations and Measurements Encoding Standard (O&M)• Defines an abstract model and an XML schema encoding for

observations and it provides support for common sampling strategies. It also provides a general framework for systems that deal in technical measurements in science and engineering.

Page 8: Grid-Enabled Geospatial Systems

8

Key OGC StandardsKey OGC Standards• Transducer Markup Language (TML)

• TML defines the conceptual model and XML Schema for describing transducers and supporting real-time streaming of data to and from sensor systems

• Web Processing Service (WPS)

• WPS defines a standardized interface to facilitate the publishing of geospatial processes, and the discovery of an binding to those processes by clients

• Transducer Markup Language (TML)

• TML defines the conceptual model and XML Schema for describing transducers and supporting real-time streaming of data to and from sensor systems

• Web Processing Service (WPS)

• WPS defines a standardized interface to facilitate the publishing of geospatial processes, and the discovery of an binding to those processes by clients

Page 9: Grid-Enabled Geospatial Systems

9

OGC Sensor Web Enablement Framework

(SWE)

OGC Sensor Web Enablement Framework

(SWE)

SP

SS

OS

WN

SS

AS

CS

W

MissionControlSystem

Registered sensor and observation metadataMetadata for a sensor and observations, and a URL

Metadata for a sensor observations, and a URL

Metadata for a sensor observations, and a URL

Metadata for a sensor observations, and a URL

Metadata for a sensor observations, and a URL………

!

SWE Architecture Encodings:

–Observations & Measurements

–SensorML

–Transducer Markup Language (TML)

IEE1451

Page 10: Grid-Enabled Geospatial Systems

10

OGC Web Services CombinedOGC Web Services Combined

Page 11: Grid-Enabled Geospatial Systems

11

Key OGF StandardsKey OGF Standards• HPC-Basic Profile

• Cluster scheduling interoperability layer supported by Microsoft, Altair Computing, Platform Computing, and others

• Simple API for Grid Applications (SAGA)• Complete distributed programming environment with C++

and Java bindings• GridRPC

• Grid-enabled Remote Procedure Call• Data Access and Integration specs (WS-DAI-*)

• Set of specs for accessing remote files, databases, XML documents

• Web Services Resource Framework (WSRF)• Allow remote data and services to be independently

managed

• HPC-Basic Profile• Cluster scheduling interoperability layer supported by

Microsoft, Altair Computing, Platform Computing, and others• Simple API for Grid Applications (SAGA)

• Complete distributed programming environment with C++ and Java bindings

• GridRPC• Grid-enabled Remote Procedure Call

• Data Access and Integration specs (WS-DAI-*)• Set of specs for accessing remote files, databases, XML

documents• Web Services Resource Framework (WSRF)

• Allow remote data and services to be independently managed

Page 12: Grid-Enabled Geospatial Systems

12

An Example of a Global Grid: EGEE

An Example of a Global Grid: EGEE

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Bob Jones, EGEE Project Director

Page 13: Grid-Enabled Geospatial Systems

13

Coalescing Service Categories to Specification

Areas

Coalescing Service Categories to Specification

Areas• Infrastructure Service Categories

• Catalogues & Registries• Discovery• Applications• Job Submission/Execution Svcs• Workflow Management• Data Product Management• Storage & Archiving• Messaging• Event Notification• User Management• Information Assurance/Security• Monitor & Control• Enterprise Svc Mgmt/Software Mgmt• APIs/Ops Systems• Financial Accounting

•Specification Areas• Catalogues & Registries• Discovery• Applications• Execution Services• Workflow/Transactions• Data Management• State Management• Messaging, Routing, Addressing• Event Notification• Metadata Schemas & Ontologies

• Portals and User Interfaces• Security• Policy & Agreement• System Architecture• System Management

Page 14: Grid-Enabled Geospatial Systems

14

Populating the Specification Areas

Populating the Specification Areas

Applications

(There are essentially an unbounded number of “applications” that could run as part of the application domain. OGC WMS, WFS & WCS are listed here since they would be part of what an end-user sees and interacts with.)

Job Submission/Execution SvcsWSDL, SOAP, (WS-I Basic Profile), CORBA, JSDL, OGSA-BES (OGF HPC Basic Profile), OGC Web Processing Service

DiscoveryUDDI, (WS-I Basic Profile), DDMS v1.3 (DoD Discovery Metadata Specification), WS-Discovery

Catalogues & Registries

LDAP, OpenGIS Cat 2.0, OGC CSW, ebRIM, ebXML RS, EO Products Ext., NATO DFDD 2007-1 (DGIWB Data Feature Data Dictionary), NAS v1.8 (NSG Application Schema), NEC V1.8 (NSG Entity Catalog)

Metadata Schemas & Ontologies

XML, CIM/GLUE, FGDC, GML 3.1, ISO 15836 (Dublin Core), ISO 19115 (metadata), ISO 19119 (services), ISO 19130 (sensors), OWL, OWL-S, SWRL, (many other ISO docs)

Messaging, Routing, Addressing

HTTP, WS-Addressing, WS-MessageDelivery, WS-Reliability, WS-ReliableMessaging

Event Notification WS-Notification, WS-Eventing, Sensor Alert ServiceUpdated from Fox, Ho, Pierce – U. Indiana Green: On DISR Baseline 07-3.0

Page 15: Grid-Enabled Geospatial Systems

15

Populating the Specification Areas

Populating the Specification Areas

Workflow/Transactions BPEL, WS-Coordination, (Kepler, Triana, Pegasus, …)

Data Management FTP, GridFTP, OGSA-DAI, (SRB, iRODS)

State ManagementIETF RFC 2695 (HTTP State Management (cookies)), WS-ResourceFramework, WS-Context, WS-I+, WS-MetadataExchange

Portals and User Interfaces WS-RemotePortlets

Security

SSL/TLS, Kerberos, WS-Security, ID-WSF 2.0 (Liberty Identity Web Services Framework), WS-Trust, SAML, GSI, Shibboleth, VOMS (Virtual Organization Management System), OpenID, OAuth

Policy & Agreement WS-Policy, WS-Agreement, (iRODS), XACML, GeoXACML

System ArchitectureOGF Open Grid Services Architecture, OGC Sensor Web Enablement (SWE), Community Sensor Model v2.A (CSM)

System ManagementWS-DistributedManagement, WS-Management, WS-Transfer, OGF CDDLM (Configuration Description, Deployment and Lifecycle Management)

Updated from Fox, Ho, Pierce – U. Indiana Green: On DISR Baseline 07-3.0

Page 16: Grid-Enabled Geospatial Systems

16

How Do We Manage Progress for Distributed Geospatial Systems?How Do We Manage Progress for Distributed Geospatial Systems?

• Targeted Projects on Decisive IssuesTargeted Projects on Decisive Issues• Build Critical Mass of Key StakeholdersBuild Critical Mass of Key Stakeholders

• Continual polling and coordination across Continual polling and coordination across the communitythe community

• They must agree on:They must agree on:• Clear GoalsClear Goals• Clear Schedule (“time-box” the process)Clear Schedule (“time-box” the process)• Clear ResponsibilitiesClear Responsibilities• Properly Provisioning the EffortProperly Provisioning the Effort

• Targeted Projects on Decisive IssuesTargeted Projects on Decisive Issues• Build Critical Mass of Key StakeholdersBuild Critical Mass of Key Stakeholders

• Continual polling and coordination across Continual polling and coordination across the communitythe community

• They must agree on:They must agree on:• Clear GoalsClear Goals• Clear Schedule (“time-box” the process)Clear Schedule (“time-box” the process)• Clear ResponsibilitiesClear Responsibilities• Properly Provisioning the EffortProperly Provisioning the Effort

Page 17: Grid-Enabled Geospatial Systems

17 17

A General Process ModelA General Process ModelA General Process ModelA General Process Model*RFQ/CFP = Request for Quotation/Call for Participation*RFQ/CFP = Request for Quotation/Call for Participation

Develop & Develop & TestTest

Task DTask D

Selection & Selection & Kick-offKick-off

Task CTask C

RFQ/CFP*RFQ/CFP*DevelopmentDevelopment

Task BTask B

ConceptConceptDevelopmentDevelopment

Task ATask A

Deploy & Deploy & PersistPersist

Task ETask EClear Schedule, Deliverables and Clear Schedule, Deliverables and Project ResponsibilitiesProject Responsibilities

Page 18: Grid-Enabled Geospatial Systems

18

Return on InvestmentReturn on Investment• What are the “carrots” to build the critical

mass of stakeholders?• Get early influence in specification

development, early skills building, visibility, and opportunity for early market deployment of standards, but just as important…

• Return on Investment (ROI)Return on Investment (ROI)• Investment

• Time, Money & People• Both Monetary and In-Kind (labor & materials)

• Timely Connection to Concrete Results• Stakeholders benefit from collaboration• Get more than they put in

• What are the “carrots” to build the critical mass of stakeholders?• Get early influence in specification

development, early skills building, visibility, and opportunity for early market deployment of standards, but just as important…

• Return on Investment (ROI)Return on Investment (ROI)• Investment

• Time, Money & People• Both Monetary and In-Kind (labor & materials)

• Timely Connection to Concrete Results• Stakeholders benefit from collaboration• Get more than they put in

Page 19: Grid-Enabled Geospatial Systems

19

A Case Study: HPC Basic Profile

A Case Study: HPC Basic Profile

• 2006• Key stakeholders decided to demonstrate interoperability

between their existing job submission infrastructures

• November 2006 (SC06)• Prototype implementations demonstrated

• 28-August-2007• HPC Basic Profile, Version 1.0, published• http://www.ogf.org/documents/GFD.114.pdf

• November 2007 (SC07)• Interoperability demonstrated by Altair, Microsoft, Platform,

OMII-UK, OMII-Europe, EGEE, UVa• 21-February-2008

• Interoperability Experiences with HPCBP, Version 1.0, published http://www.ogf.org/documents/GFD.124.pdf

• Commercial adoption plans by Altair, Microsoft & Platform

• 2006• Key stakeholders decided to demonstrate interoperability

between their existing job submission infrastructures

• November 2006 (SC06)• Prototype implementations demonstrated

• 28-August-2007• HPC Basic Profile, Version 1.0, published• http://www.ogf.org/documents/GFD.114.pdf

• November 2007 (SC07)• Interoperability demonstrated by Altair, Microsoft, Platform,

OMII-UK, OMII-Europe, EGEE, UVa• 21-February-2008

• Interoperability Experiences with HPCBP, Version 1.0, published http://www.ogf.org/documents/GFD.124.pdf

• Commercial adoption plans by Altair, Microsoft & Platform

Page 20: Grid-Enabled Geospatial Systems

20

OGC Web Service Testbed

OGC Web Service Testbed

• OWS is an annual process where sponsors identify specific demonstration targets• Larger number of participants supply in-kind resources

• OWS-5 recently finished (March 28, 2008)• OWS-5 Sponsors

• BAE Systems - National Security Solutions• Federal Geographic Data Committee (FGDC/USGS)• Google• Lockheed Martin Integrated Systems & Solutions (Lead

Org.)• Northrop Grumman• US National Aeronautic and Space Administration

(NASA)• US National Geospatial-Intelligence Agency (NGA)

• ~35 Participating Organizations• 3x-4x Return on Investment3x-4x Return on Investment

• Strength of CollaborationStrength of Collaboration

• OWS is an annual process where sponsors identify specific demonstration targets• Larger number of participants supply in-kind resources

• OWS-5 recently finished (March 28, 2008)• OWS-5 Sponsors

• BAE Systems - National Security Solutions• Federal Geographic Data Committee (FGDC/USGS)• Google• Lockheed Martin Integrated Systems & Solutions (Lead

Org.)• Northrop Grumman• US National Aeronautic and Space Administration

(NASA)• US National Geospatial-Intelligence Agency (NGA)

• ~35 Participating Organizations• 3x-4x Return on Investment3x-4x Return on Investment

• Strength of CollaborationStrength of Collaboration

Page 21: Grid-Enabled Geospatial Systems

21

OWS-5OWS-5• Five Threads

• Sensor Web Enablement (SWE)• Geo Processing Workflow (GPW)• Information Communities and Semantics

(ICS)• Agile Geography• Compliance Testing (CITE)

• Outbrief Videos • NASA Sensor Web 2.0 Experiments (7:43)• Data Architecture Views (10:37)• Conflation Processing (5:07)• Web Coverage Processing Service (5:58)

• Five Threads• Sensor Web Enablement (SWE)• Geo Processing Workflow (GPW)• Information Communities and Semantics

(ICS)• Agile Geography• Compliance Testing (CITE)

• Outbrief Videos • NASA Sensor Web 2.0 Experiments (7:43)• Data Architecture Views (10:37)• Conflation Processing (5:07)• Web Coverage Processing Service (5:58)

Page 22: Grid-Enabled Geospatial Systems

22

Still from OWS-5 NASA Sensor Web Video

Still from OWS-5 NASA Sensor Web Video

Satellite EO-1 tasked to collect imagery on Northern San Diego County Wildfires that was integrated with UAV track data (red lines) in Google Earth. Demo drove issues around sensor networks, data interoperability, and command & control.

http://www.opengeospatial.org/pub/www/ows5/index.html

Page 23: Grid-Enabled Geospatial Systems

23

OWS-6: RFQ/CFP TopicsOWS-6: RFQ/CFP Topics• Sensor Web Enablement (SWE)

• Georeferenced sensors, event notification, security

• Aviation Information• Aeronautical Information Exchange Model (AIXM) for next generation

flight control system (FAA and EuroControl)

• GeoProcessing Workflows (GPW)• Web Processing Service (WPS), Workflow and Grids (“WPS to Grid”)• Distributed resource management, e.g., data, services, hosts,

networks, for geospatial applications

• Decision Support Services• 3D Visualization with Fly Through• Open Location Services• Integrated Client for multiple OWS services

• Compliance, Interoperability & Test Evaluation (CITE)

• Suite of tests and reference implementations for all approved standards

• Sensor Web Enablement (SWE)• Georeferenced sensors, event notification, security

• Aviation Information• Aeronautical Information Exchange Model (AIXM) for next generation

flight control system (FAA and EuroControl)

• GeoProcessing Workflows (GPW)• Web Processing Service (WPS), Workflow and Grids (“WPS to Grid”)• Distributed resource management, e.g., data, services, hosts,

networks, for geospatial applications

• Decision Support Services• 3D Visualization with Fly Through• Open Location Services• Integrated Client for multiple OWS services

• Compliance, Interoperability & Test Evaluation (CITE)

• Suite of tests and reference implementations for all approved standards

Page 24: Grid-Enabled Geospatial Systems

24

Possible Geospatial/Grid Topics

Possible Geospatial/Grid Topics• Integration, interoperability of geospatial and computing resource

• Information models (CIM/GLUE)• Distributed and federated catalogs (ebRIM, MDS, etc.)• Discovery services

• Implementation of WPS on various grid tools• SAGA, GridRPC, HPC-Basic Profile

• Integration of WS-Eventing, WS-Notification, INFOD• Sensor Web Enablement

• Integration of configuration and lifecycle management tools• Configuration Description, Deployment and Lifecycle Management

(CDDLM)• Integration of WPS with workflow management tools

• BPEL, Kepler, Taverna, Triana, Pegasus, etc.• Workflow design tools, execution engines, planning (data

virtualization), data provenance• Integration of WPS with grid security models

• SAML, XACML, VOMS, GSI• Support for virtual organizations

• Integration of geospatial data with rule-based, data management policy engines

• iRODS

• Integration, interoperability of geospatial and computing resource• Information models (CIM/GLUE)• Distributed and federated catalogs (ebRIM, MDS, etc.)• Discovery services

• Implementation of WPS on various grid tools• SAGA, GridRPC, HPC-Basic Profile

• Integration of WS-Eventing, WS-Notification, INFOD• Sensor Web Enablement

• Integration of configuration and lifecycle management tools• Configuration Description, Deployment and Lifecycle Management

(CDDLM)• Integration of WPS with workflow management tools

• BPEL, Kepler, Taverna, Triana, Pegasus, etc.• Workflow design tools, execution engines, planning (data

virtualization), data provenance• Integration of WPS with grid security models

• SAML, XACML, VOMS, GSI• Support for virtual organizations

• Integration of geospatial data with rule-based, data management policy engines

• iRODS

Page 25: Grid-Enabled Geospatial Systems

25

Digital Repositories/Libraries

Digital Repositories/Libraries

• Digital Repositories/Libraries represent a growing part of the “infomass”• Federated catalogues and federated storage

• OGF-Europe Workshop at OGF-23• 4 sessions, ~70 participants from industry, academia and

European Commission

• Proposed OGF-Europe collaboration with Global Research Library 2020

• Proposed OGF-Europe involvement in GENESI-DR Digital Repositories Seminar in Autumn 2008, Frascati

• OGF-Europe PC membership for the DReSNet special DR session at IEEE e-Science conference, Dec 2008

• Eventual development of an OGF Working Group dedicated to issues surrounding DRs

• Digital Repositories/Libraries represent a growing part of the “infomass”• Federated catalogues and federated storage

• OGF-Europe Workshop at OGF-23• 4 sessions, ~70 participants from industry, academia and

European Commission

• Proposed OGF-Europe collaboration with Global Research Library 2020

• Proposed OGF-Europe involvement in GENESI-DR Digital Repositories Seminar in Autumn 2008, Frascati

• OGF-Europe PC membership for the DReSNet special DR session at IEEE e-Science conference, Dec 2008

• Eventual development of an OGF Working Group dedicated to issues surrounding DRs

Page 26: Grid-Enabled Geospatial Systems

26

Another Example: Green GridsManaging Energy Policy

Another Example: Green GridsManaging Energy Policy

Source: www.Bastionhost.com

Bill St. Arnaud

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

Page 27: Grid-Enabled Geospatial Systems

27

Another Example: Green GridsManaging Energy Policy

Another Example: Green GridsManaging Energy Policy

Source: www.Bastionhost.com

Bill St. Arnaud

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

Page 28: Grid-Enabled Geospatial Systems

28

Another Example: Green GridsManaging Energy Policy

Another Example: Green GridsManaging Energy Policy

Source: www.Bastionhost.com

Bill St. Arnaud

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

• A geospatial sensor network

• Extensive use of virtualizationvirtualization and fiber opticsfiber optics to put the computing where the power is

• Enable the workload to “follow the wind” or “follow the sun”

• A geospatial sensor network

Page 29: Grid-Enabled Geospatial Systems

29

SummarySummary• Significant work already underway for

standard, distributed geospatial computing infrastructures

• Highly relevant to many application domains

• Opportunity to leverage the work of key participants through collaboration

• How to engage application groups and industries needing distributed geospatial processing?

• Significant work already underway for standard, distributed geospatial computing infrastructures

• Highly relevant to many application domains

• Opportunity to leverage the work of key participants through collaboration

• How to engage application groups and industries needing distributed geospatial processing?

Page 30: Grid-Enabled Geospatial Systems

Back-Ups andOther Draft

Material

Page 31: Grid-Enabled Geospatial Systems

31

A “Cloudy” Solution SpaceA “Cloudy” Solution Space

OS Virtualization

OS Virtualization

Parallel Frameworks

Parallel Frameworks

Software as a Service

Software as a Service

MapReduce, GFS, BigTable

Amazon S3/EC2

Hadoop

Hadoop over EC2

Google AppEngine

Cohesive

Sun’s Caroline

MS Astoria

Mesh

RightScale, GigaSpaces,Elastra, 3Tera

Courtesy of Dennis Gannon, www.extreme.indiana.edu/~gannon/science-data-center.pdf

mapmap mapmap mapmap mapmap mapmap mapmap mapmap mapmap

reducereduce reducereduce

reducereduce

Data Collection Data Collection

…..

Vendors of high-levelVendors of high-levelapplication configurationapplication configurationtools are potentialtools are potential““cloud framework providers”cloud framework providers”

Page 32: Grid-Enabled Geospatial Systems

32

A Host of Cloud IssuesA Host of Cloud Issues• Data access and interoperability

• Will have to be approached at the application domain level, by the domain users involved

• Security• Can cloud providers provide sufficient security for sensitive applications?

• Reliability• Can cloud providers provide sufficient reliability for critical applications?

• Frameworks• How to manage sets of VMs and VOs?• These are essentially “cloud frameworks” and call for "cloud framework

providers"• Performance management

• Clouds tend to abstract away location• Managing the compute-data locality (affinity) can be important

• Costing models• How to compare your own infrastructure costs with a cloud computer?• How to compare two clouds?

• Data access and interoperability• Will have to be approached at the application domain level, by the domain users

involved• Security

• Can cloud providers provide sufficient security for sensitive applications?• Reliability

• Can cloud providers provide sufficient reliability for critical applications?• Frameworks

• How to manage sets of VMs and VOs?• These are essentially “cloud frameworks” and call for "cloud framework

providers"• Performance management

• Clouds tend to abstract away location• Managing the compute-data locality (affinity) can be important

• Costing models• How to compare your own infrastructure costs with a cloud computer?• How to compare two clouds?