Geospatial Information, Fundamental Grid Challenges, and the Role of Standards Organizations

© 2006 OpenGridForum

European Geoinformatics Workshop

Geospatial Information, Fundamental Grid Challenges, and the Role of Standards Organizations

Edinburgh, March 9, 2007

Dr. Craig A. Lee, [email protected] Aerospace Corporation(a non-profit, federally funded R&D center)

2© 2006 OpenGridForum

What’s the Motivation for All This?

• Geospatial data has immense practical value• Claim that large percentage of all data is geospatial in nature• Applicability across many domains

• Service Architecture concept is gaining wide momentum• Natural concept for designing, deploying, using distributed

systems• Managing access to data, machines -- resources of all kinds

-- for geographically distributed users

• A Service Architecture for Geospatial Data and Tools is a Clear Win• It is imperative to engage key stakeholders


But What’s the Larger Context?

• Geospatial Systems part of larger Systems-of-Systems• Automatically detect, ingest, and disseminate input data events• Automatically analyze the events and known data• Automatically plan responses• Distributed execution of workflows to enact the response• Workflows dynamically respond to further events of interest• Secure, autonomous operation in an environment with only partial control and

observability

• Focus: Event-Driven Workflows Event-Driven Workflows oror Dynamic Workflows Dynamic Workflows• Events delivered to decision-making elements that need to know• Decision makers plan and modify responses according to policy• Workflows executed with distributed control in a dynamic env.

• Dynamic, Data-Driven Application SystemsDynamic, Data-Driven Application Systems


ExperimentMeasurements

Field-DataUser

Theory

(First P

rincip

les)Sim

ulatio

ns

(Math

.Modelin

g

Phenomenolo

g

y) ExperimentMeasurements

Field-DataUser

Theory

(First P

rincip

les)

Simula

tions

(Math

.Modelin

g

Phenomenolo

gy

Observ

ation M

odeling

Design)

OLD

(serialized and static)

NEW PARADIGM

(Dynamic Data-Driven Simulation Systems)

Challenges:Application Simulations DevelopmentAlgorithms Computing Systems Support

Dynam

ic

Feed

back

& C

ontro

l

Loop

Motivation: DDDAS

Frederica Darema, NSF


Examples of Applications benefiting from the new paradigm

• Engineering (Design and Control) • aircraft design, oil exploration, semiconductor mfg, structural eng• computing systems hardware and software design

(performance engineering)

• Crisis Management• transportation systems (planning, accident response)• weather, hurricanes/tornadoes, floods, fire propagation

• Medical• customized surgery, radiation treatment, etc• BioMechanics /BioEngineering

• Manufacturing/Business/Finance• Supply Chain (Production Planning and Control)• Financial Trading (Stock Mkt, Portfolio Analysis)

DDDAS has the potential to revolutionize science, engineering, & management systems


Fire Model

• Sensible and latent heat fluxes from ground and canopy fire -> heat fluxes in the atmospheric model.

• Fire’s heat fluxes are absorbed by air over a specified extinction depth.

• 56% fuel mass -> H20 vapor

• 3% of sensible heat used to dry ground fuel.

• Ground heat flux used to dry and ignite the canopy.

Kirk Complex Fire. U.S.F.S. photoSlide Courtesy of Cohen/NCAR

Coupled ModelsCoupled Models• Sensible and latent heatSensible and latent heat• Fire PropagationFire Propagation• Atmospheric DynamicsAtmospheric Dynamics


Forest Fires in the Context of a Sensor Network

Kirk Complex Fire. U.S.F.S. photo

FireFighters

Policy,Planning,Response

AtmosphericModel

Fire Prop.Model

CombustionModel


Data Management and Manipulation

Visualization

Field Measurements

Simulation Models

Reservoir MonitoringField Implementation

Data Analysis

Production ForecastingWell Management

ReservoirPerformance

Data Collections from Simulations and Field Measurements

Multiple Realizations

Economic Modeling andWell Management


DynamicallyLink

&Execute

The NGS Program developsTechnology for integrated feedback & control Runtime Compiling System (RCS) and Dynamic Application

CompositionApplication

Model

Application Program

ApplicationIntermediate

Representation

CompilerFront-End

CompilerBack-End Performance

Measuremetns&

Models

DistributedProgramming

Model

ApplicationComponents

&Frameworks

Dynamic AnalysisSituation

LaunchApplication (s)

Distributed Platform

Ada

ptab

leco

mpu

ting

Syst

ems

Infr

astr

uctu

re

Distributed Computing Resources

MPP NOW

SAR

tac-com

database

firecntl

firecntl

alg accelerator

database

SP

…. F. Darema, NSF


A DDDAS Model(Dynamic, Data-Driven Application Systems)

Spectrum of Physical SystemsSpectrum of Physical Systems

Humans:3 Hz.

Cosmological:10e-20 Hz.

Subatomic:10e+20 Hz.

ComputationalInfrastructure(grids, perhaps?)

ModelsModels

ComputationsComputations

Discover, Ingest, Interact

Discover,Ingest,Interact

sensors & actuators sensors & actuators sensors & actuators

Loads a Loads a behaviorbehavior into intothe infrastructurethe infrastructure


CommunicationDomain

Top-Level Concept: Integration of Event Notification and Workflow

DecisionMaker

DecisionMaker

Policy

SensedEvents

Resource Info and

Mgmt Service

discovery

Response

Abstract Plan

DecisionMaker

Concrete Action

register


CommunicationDomain

Top-Level Concept

DecisionMaker

DecisionMaker

Policy

SensedEvents

discovery

Response

Abstract Plan

DecisionMaker

Concrete Action

register

Content-Based Routing Domain

Resource Info and

Mgmt Service


CommunicationDomain

Top-Level Concept

DecisionMaker

DecisionMaker

Policy

SensedEvents

discovery

Response

Abstract Plan

DecisionMaker

Concrete Action

register

PersistentDecision-makingComputationsDeterminedby Policy

Resource Info and

Mgmt Service


CommunicationDomain

Top-Level Concept

DecisionMaker

DecisionMaker

Policy

SensedEvents

discovery

Response

Abstract Plan

DecisionMaker

Concrete Action

register

Grid Information Service

Resource Info and

Mgmt Service


CommunicationDomain

Top-Level Concept

DecisionMaker

DecisionMaker

Policy

SensedEvents

discovery

Response

Abstract Plan

DecisionMaker

Concrete Action

register

Dynamic Grid Workflow Management

Resource Info and

Mgmt Service


Required Capabilities

• Events delivered to decision-making elements that need to know• Event Notification Service Managed by Publish/SubscribePublish/Subscribe

• Pre-defined Topics• Publication Advertisements• User-defined Attributes

• Content-Based Routing – Topology-Aware CommunicationContent-Based Routing – Topology-Aware Communication• Decision makers plan responses as determined by policy

• Semantic analysisSemantic analysis to determine the “meaning” of sets of events• PlanningPlanning - “path construction” from current state to goal state• Classic topics in Artificial IntelligenceClassic topics in Artificial Intelligence

• Resource Information & Management Systems• Distributed, Scalable, Timely• Metadata schemas, OntologiesMetadata schemas, Ontologies

• Responses executed as distributed workflows• Workflow EngineWorkflow Engine independently manages

• Scheduling Scheduling of Data Transfer Data Transfer• Scheduling Scheduling of Process Execution Process Execution

• Centralized vs. DistributedCentralized vs. Distributed


General Architecture for Topology-Aware Communication Services

Peer-to-Peer Network

Events are published to the P2P network which are then routed to subscribers

Subscription “signals” propagate through the P2P Network


Many Types of Communication Services Improved or Enabled

• Augmented Semantics• Caching (web caching), filtering, compression, encryption, quality of service, data-

transcoding, etc.• Collective Operations

• Accomplished “in the network” rather than using point-to-point msgs across the diameter of the grid

• Communication Scope• Named topologies can denote a communication scope to limit problem size and

improve performance• Content and Policy-based networking

• Publish/subscribe, interest management, event services, tuple spaces, quality of service

• Issues• Topology management/construction

• Dynamic member join/departure• Reliability

• Maintaining distributed state in the network• Security

• Integrity, authentication, authorization of signaling messages


Grid Workflow Management

• Organization of distributed computing services• Rather than building applications with ad hoc, "hard-coded“ task

organization, workflow provides a general mechanism for distributed task organization

• Independent scheduling of data transfer and process Independent scheduling of data transfer and process executionexecution

• Key Capability for all Workflow tools• Subsequent task may not exist when previous task completes• Where subsequent task is to execute may not even be decided• Output data may have to be buffered until it is needed/can be

used


Workflow Mgmt Considerations

• Representation• Graphical (DAGs), Syntactic (code, XML)

• Creation• Eager vs. lazy binding of service to physical resources• Eager vs. lazy binding of workflow to service• Co-Scheduling vs. Incremental Scheduling

• Data Transfer• Streaming• Buffered channel• File Transfer

• Data Persistence and Lifetime• How long does the data live where it is?

• Workflow Engine – executes the workflow• Centralized? (“orchestration”)Centralized? (“orchestration”)• Decentralized? (“choreography”)Decentralized? (“choreography”)


Combining Events and Workflow: Dynamic Event-Driven Workflows

• Besides events precipitating an initial response workflow, subsequent events may alter an existing workflow that is underway• Current amount of workflow completed must be determined• Current tasks on the “leading edge” of the workflow must be terminated

or allowed to complete• Status and disposition of data referenced by tasks must be determined• “Classical” storage management issues reoccur

• Dangling references to no data or stale data• Unaccessible data referenced by no one

• Such event-driven task mgmt is similar to fault tolerance• Similar mechanisms could be used to detect and respond to faults (failed

servers, networks, etc.)

• Directly Supports DDDAS ConceptDirectly Supports DDDAS Concept


Responding to Events under Centralized Workflow Control

ClientMaking Decision

(CentralizedControl)

What is State of Workflow What is State of Workflow When Event Received?When Event Received?

EventSubscription

EventNotification

Possible Actions after Event:Possible Actions after Event:• Do NothingDo Nothing• Cancel Entire WorkflowCancel Entire Workflow• Cancel Part of WorkflowCancel Part of Workflow• Conditional WorkflowConditional Workflow

How is Workflow Executed?How is Workflow Executed?• Client statically decides workflow Client statically decides workflow

services and servers prior to start-time services and servers prior to start-time • Client incrementally decides services Client incrementally decides services

and servers during run-timeand servers during run-time


In General, Nested or Recursive Workflows will be Possible

ClientMaking Decision

(CentralizedControl)

What is State of Workflow What is State of Workflow When Event Received?When Event Received?

EventSubscription

EventNotification

Even if Control is Centralized,Even if Control is Centralized,Client May Not Know Entire Workflow StateClient May Not Know Entire Workflow State


Avoiding Single Point of Failure: Decentralized Workflow Control

ClientMaking Decision(Decentralized

Control)

EventSubscription

EventNotification

• Workflow RepresentationWorkflow Representation passed among workflow services passed among workflow services• Initiating Client does not explicitly manage each service Initiating Client does not explicitly manage each service • Nested, recursive workflows still possiblyNested, recursive workflows still possibly


Responding to Events under Decentralized Workflow Control

• Currently active workflow agents subscribe to Currently active workflow agents subscribe to appropriate event topicsappropriate event topics

• Workflow agents may need to find and coordinate Workflow agents may need to find and coordinate with their active collaboratorswith their active collaborators

Event Notification


Programming Decentralized Workflows?

• “Process programming” in a distributed environment• Example: Little-JIL

• Agent Coordination Language• A coordination tree with four non-leaf operations

sequential, parallel, try, choice

• Other possibilities?• Stream-based languages?• Dataflow languages?

• Decentralized Workflows similar to Decentralized Workflows similar to Active Active NetworksNetworks, , Active AgentsActive Agents and and Active MessagesActive Messages• “Programming the message, not the node”

• Autonomic behavior• If peer agent fails, agent will have to infer workflow repair to reach goal state


Use of A Priori Information in GridsKnowable independently of experience

• Expects the world to have certain properties or be in a known state

• Semantic translation tools can be used, i.e., compilers

• Entire code units can be examined, analyzed, optimized

• Static information compiled-in• Everything that can be statically

defined a priori takes complexity out of the application and improves performance

Task-Define-time Task-Run-time

Start Time

• Increasing use of a posteriori information learned from experience

• Capturing more information about a running app and the environment

• More and more dynamic late binding• “Smart” run-time• “Smart back-end” of a compiler• Limited control and imperfect Limited control and imperfect

knowledge of the environmentknowledge of the environment• Must apply reasoning to what is Must apply reasoning to what is

semantically understandablesemantically understandable


Future Generation Grids:We Are Being Pushed Into…

• Dynamic discovery, late binding• How little a priori knowledge can be "compiled-in"?• Resource virtualization• Performance penalty for deciding everything dynamically

• Autonomic Control Cycle Occurs Everywhere• Monitor, Understand, Plan, Respond• Fault Tolerance/Recovery• Real-Time/Physical System Monitoring & Interaction• Dynamic configuration (late discovery, binding)• Anytime a goal state must be reached

• Planning is a classic AI capability• Chaining of "moves" to get from current state to goal state• Inferencing on known and discoverable facts• Done in environment with imperfect knowledge and limited control

• If plan fails, replan and try again• Declarative programming techniques

• Programming the “What”, not the “How”• Geosemantics is an archetypal example of this fundamental challenge to grid

computing• Advances made in this field should be understood, and hopefully generalized, for this wider

context• Interdisciplinary approach


How Do We Make Progress on these Fundamental Challenges?

• Research• Organized Research

• Governmental funding agencies• Organized, Interdisciplinary Research

• Getting the right fields of expertise to collaborate• Organized Adoption

• Open Grid Forum (OGF)• World Wide Web Consortium (W3C)• Organization for the Adv. of Structured Information Standards (OASIS)• Distributed Management Task Force (DMTF)• Storage Networking Industry Association (SNIA)• Tele Management Forum (TMF)• Internet Engineering Task Force (IETF)• International Telecommunication Union (ITU-T)


Key Technical Areas

• Security• How to manage grid identity and access

• Metadata and Ontologies• How to define the relevant information architecture

• Data Discovery and Management• How to manage the location and access to cached and

replicated data

• Semantics• How to use the meaning of data to produce information

• Service Architectures• How to integrate and manage all resources as a whole and

provide dynamic, transparent access


Security

• Security Capabilities• Authentication, Authorization, Privacy, Integrity, Non-Repudiation

• Authentication• Evolving to combination of GSI, Kerberos and Shibboleth

• Authorization• Databases (VOMS and Permis)• Role-based (TeraGrid, OSG)

• WS-Security• Performance is an issue

• Delegation of Trust -- Delegation of Identity• Identity is also dependent on role in a Virtual Organization• Identity has a structure


Metadata and Ontologies

• Metadata – data about data, e.g.,• Federal Geographic Data Committee, Content Standard on Digital Geospatial Metadata• GML 3.0 (Geographic Markup Language)• ISO Standards

• ISO 19115:2003 Metadata• ISO 19115.2 Metadata-Part 2: Extensions for Imagery and Gridded Data (within two years)• ISO 19119:2005 Services• ISO 19130 Sensor Model and Data Model for Imagery (within two years)

• Ontologies• Needed to capture process behavior, spatial/temporal characteristics, data and process

relationships• Need to be more than just keyword lists for classification

• OWL: Web Ontology Language• Semantic markup language for publishing and sharing ontologies on the web• OWL ontology: description of classes, properties and their instances• OWL-S: web service ontology

• Are GML, ISO standards and OWL sufficient for geospatial representation and reasoning?• (No!)


Data Discovery and Management• Data (and services) must be published in a registry to be discoverable

• Metadata and Ontologies are essential• UDDI is generally considered to be inadequate

• Not scalable, poor semantics for application data• Combined catalogue and storage management

• Storage Resource Broker (SRB, SDSC)• SRB MCAT (Metadata Catalogue) used to manage access across multiple remote

sites• OGSA-DAI (Open Grid Forum)

• Open Grid Service Architecture-Data Access and Integration• Web service access to files, databases

• Globus Data Replication Services• Built to support high-energy physics projects• Controls pushing of data closer to key consumers• Enables user to choose “closest” replica

• Storage Networking community driving to storage virtualization• E.g., Amazon S3 (Simple Storage Service)


Semantics: Enabling Intelligence

• Automated Systems Only Possible with Well-Known Semantics• Environmental Decision Systems• Emergency Decision Systems

• SWRL: Semantic Web Rule Language• Extension to OWL• Adds parts of RuleML into OWL• Extends OWL axioms to Horn-like clauses• Will this be sufficient?


Service Architectures: Key Capability Areas Covered by Core WS-* Specs

WS-* Specification Area Examples

1: Core Service Model XML, WSDL, SOAP

2: Service Internet WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM

3: Notification WS-Notification, WS-Eventing (Publish-Subscribe)

4: Workflow and Transactions

BPEL, WS-Choreography, WS-Coordination

5: Security WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation

6: Service Discovery UDDI, WS-Discovery

7: System Metadata and State

WSRF, WS-MetadataExchange, WS-Context

8: Management WSDM, WS-Management, WS-Transfer

9: Policy and Agreements WS-Policy, WS-Agreement

10: Portals and User Interfaces

WSRP (Remote Portlets)

B&W table courtesy of Fox, Ho, Pierce – U. Indiana

Consensus Merging Developing


Issues from an Organizational Perspective

• General Consensus Only on WS Basic Building Blocks• Must avoid vendor-specific solutions – Adopt vendor-neutral approach

• Adoption Roadmap and Timetable?• Much Work Remains to be Done – And It Is Underway

• Topics for Harmonization• Merging of competing WS standards expected• Service Component Architecture (SCA) and Open Grid Services Arch (OGSA)• Service Data Objects (SDO) and Web Service Resource Framework (WSRF)

• Workflow Management (aka web service chaining)• Triana, Taverna, Pegasus, BPEL• Semantically-aware workflow engine

• SAGA: Simple API for Grid Apps – a basic grid programming model• Common “look-and-feel” for programming in a distributed environment

• Appropriate use and cost• Not everything needs to be a service in a service architecture• Adoption of any new technology, e.g., SOA, is more expensive up front


Innovation “no man’s

land”

OGF(and other SDOs)

Driving Innovation

Research push Market pull

Phase 1 Solution proposal

Phase 2 Prototype

Phase 3 Pre-commercial product/service

Phase 4 Commercial

product/service

Phase 0 Research

Slide Borrowed from Ulf Dahlsten, Director Ulf Dahlsten, Director ‘‘Emerging Technologies and Infrastructures’Emerging Technologies and Infrastructures’

Market Pull

Managing the Technology Maturation Process


OGF Technical Strategy/Stakeholder Alignment Process

OGF Technical Strategy & Roadmap

Requirements Workshops

Uses Cases

Requirements

Best Practices

Architectures

Specifications

Milestones

OGF Events

OGF Document Series

Technical Strategy Committee Standards Groups

& Workshops

Analysis, Interpretation & Analysis, Interpretation & Prioritization of RequirementsPrioritization of Requirements

Application of Best Known Application of Best Known Practices and Current StandardsPractices and Current Standards


A More Refined View

EGR-RG

Financial

Telco

Pharma

EDA

SN-CG

Vendors

RequirementsSolicitation

Applications

Compute

Data

Infrastructure

Management

Architecture

Security

StandardsGroups

RequirementsRollup, Analysis

&Prioritization(EGR-RG)

TSCGAP Analysis

What WGs are doing i.e. WG roadmap

Prioritized Req and Req Patterns

Req

Req and Req Patterns

RequirementsSpecs

BestPractices

Req

Best PracticeWorkshops

Req

Best Practices

• Overall standards roadmap• Gap analysis of WG roadmap vs. prioritized Req•Recommended actions


OGF Grid Requirements Roll-up

• INFORMATION ARCHITECTURE• Metadata Schemas, Ontologies & Semantics• Data Profiling• Data-tagging, including managing files

• DISCOVERY• Detailed Asset Discovery• API for Product Capability Discovery

• Extract information about a project or a product• Lets users grab the right data -- categorizing data

• Content/Data Discovery• Catalogue-based Data Access

• RESOURCE VIRTUALIZATION• Dynamic Provisioning

• Capacity on demand• Capacity grows as available

• Content Provisioning• Provisioning and Capacity Management

• JOB MANAGEMENT• Distributed Execution• Job Submission• Job control management• Job Migration

• DATA MANAGEMENT• Data Copy, Data Movement • Backup• Storage Policy Mgmt• Replica Mgmt• Caching (local disk, indexes, memory)• Data Grid APIs

• GRID MANAGEMENT• Mgmt Console GUI• Asset Management and Topology• Policy Management and Quotas• Mitigate management overhead• Transition/evolution models

• WORKFLOW• Planning• Management (Cent. & Dist.)

• SCHEDULING• Meta scheduler, data aware

• MONITORING & EVENT NOTIF.• Monitoring, Auditing and Alert Mgmt

• FAULT TOL. & ERROR MGMT• Deep Error Analysis• Error Audit• Verification & Audit• Root Cause Analysis• Job error management• Very high levels of uptime

• SECURITY• Grid Identity Mgmt• Strong Security• Multiple domains

• ACCOUNTING & AUDITING• Billing and Chargeback

• Chargeback models• Business issues (charge back)

• Sarbanes-Oxley Support• SYSTEM DEVEL & DEPLOY

• Simplify application development• End-User Tools and Envs

• AUTONOMIC BEHAVIORS • Monitoring• Semantics• Planning• Action


Current OGF Standards Work

• ApplicationsDistributed Resource Mgmt App. API WG (drmaa-wg)Grid Checkpoint Recovery WG (gridcpr-wg) Grid Information Retrieval WG (gir-wg)Grid Remote Procedure Call WG (gridrpc-wg)Simple API for Grid Applications Core WG (saga-core-wg)

• ArchitectureOGSA Naming Working Group (ogsa-naming-wg) Open Grid Services Architecture WG (ogsa-wg)

• ComputeGrid Resource Alloc. Agreement Protocol WG (graap-wg) Job Submission Description Language WG (jsdl-wg) OGSA Basic Execution Services WG (ogsa-bes-wg) OGSA High Perf. Computing Profile WG (ogsa-hpcp-wg) OGSA Resource Selection Services WG (ogsa-rss-wg)

• DataData Format Description Language WG (dfdl-wg) Database Access and Integration Services WG (dais-wg) Grid File System Working Group (gfs-wg) Grid Storage Management WG (gsm-wg) GridFTP WG (gridftp-wg) Info Dissemination WG (infod-wg) OGSA ByteIO Working Group (byteio-wg) OGSA Data Movement Interface WG (ogsa-dmi-wg) OGSA-Data Working Group (ogsa-d-wg)

• InfrastructureGrid and Virtualization Working Group (gridvirt-wg) Network Mark-up Language Working Group (nml-wg) Network Measurements Working Group (nm-wg)

• ManagementApplication Contents Service WG (acs-wg) Configuration Description, Deployment, and Lifecycle

Management WG (cddlm-wg) Glue Schema Working Group (glue-wg) OGSA Resource Usage Service WG (rus-wg) Usage Record WG (ur-wg)

• SecurityOGSA Authorization WG (ogsa-authz-wg) Trusted Computing Research Group (tc-rg)


Gap Analysis:OGF Technical Strategy & Roadmap doc

Category Capability Working Group or Comment Maturity Multiple Security InfrastructuresError! Bookmark not defined. OGSA Auth-Z Evolving Perimeter Security SolutionsError! Bookmark not defined. Firewall Issues RG Research Virtual OrganizationError! Bookmark not defined. VOMS work applies Gap EncryptionError! Bookmark not defined. Existing technology is currently adequate Out of Scope CertificationError! Bookmark not defined. CA Ops WG Evolving AuthenticationError! Bookmark not defined. OGSA-AuthN Evolving AuthorizationError! Bookmark not defined. OGSA-AuthZ Evolving

Security

Web Service Protocol Security OASIS/WSS, OGSA Secure Channel Mature Instantiate New ServicesError! Bookmark not defined. CDDLM-WG, degenerate workflow Evolving DeploymentError! Bookmark not defined. ACS-WG, CDDLM-WG Evolving ProvisioningError! Bookmark not defined. ACS-WG, CDDLM-WG Evolving Service Level ManagementError! Bookmark not defined. GRAAP-WG Evolving Notification]Error! Bookmark not defined. OASIS/WS-Notification, WS-Eventing Mature MessagingError! Bookmark not defined. OASIS/WS-Notification, WS-Eventing Mature Logging ServiceError! Bookmark not defined. Related to metering, see below Gap Service and Resource MonitoringError! Bookmark not defined. Grid Monitoring Architecture Evolving Metering and AccountingError! Bookmark not defined. UR-WG and RUS-WG More needed Evolving PolicyError! Bookmark not defined. WS-Policy Evolving Policy ManagementError! Bookmark not defined. Management standards needed for policy Gap AdministrationError! Bookmark not defined. Community practices needed Gap Systems ManagementError! Bookmark not defined. Reference Model-WG Evolving

Operations

Aggregation of Services and ResourcesError! Bookmark not

defined. See OASIS WS-ServiceGroup Mature

OGSA-NamingError! Bookmark not defined. WS-Naming-WG, GFS-WG Evolving Resource DiscoveryError! Bookmark not defined. OASIS/WSDM Mature Resource BrokeringError! Bookmark not defined. RSS-WG On Hold Job ManagementError! Bookmark not defined. OGSA-BES-WG, JSDL-WG. Mature Choreography, Orchestration and WorkflowError! Bookmark not

defined. OASIS/BPEL, OGSA Workflow Design Team Mature

Resource VirtualizationError! Bookmark not defined. GridVirt-WG, CDDLM-WG Evolving Information ModelError! Bookmark not defined. DMTF/CIM, GLUE-WG Mature CPU Scavengingvii Proprietary Solutions Exist Mature Legacy ProgramsError! Bookmark not defined. ACS-WG Evolving

Resource Management

ReservationError! Bookmark not defined. GRAAP-WG, GSA-RG Evolving Data Movement DMI-WG, Grid-FTP Evolving Data Access GFS-WG and DAIS-WG Mature Data Integration DAIS-WG Evolving Data Management Storage Network-CG, OGSA-Data-WG Evolving Data ProvisioningError! Bookmark not defined. Continuation of EGA data work, OGSA-Data Gap

Data

MetadataError! Bookmark not defined. OASIS/WSRF-RMD, Evolving Application DebuggingError! Bookmark not defined. Gap Application Development Application APIsi SAGA-WG,GridRPC-WG,GridCPR-WG,DRMAA-WG Mature Communication ProtocolsError! Bookmark not defined. HTTP/SOAP, Mature ArchitectureError! Bookmark not defined. Reference Model-WG, OGSA-WG Mature Grid SemanticsError! Bookmark not defined. Semantic Grid-RG Research

Foundations

Grid Fabric Error! Bookmark not defined. OASIS/WSRF, NM-WG, NML-WG Mature Fault ToleranceError! Bookmark not defined. Implementation Property Mature Load BalancingError! Bookmark not defined. Implementation Property Mature

System Properties

Failure RecoveryError! Bookmark not defined. Implementation Property Evolving


(Formerly) Competing Camps!

• IBM & Friends• Open Grid Services

Architecture (OGSA)• Built on top of Web

Services Resource Framework (WSRF)

• Designed in collaboration with the Globus Alliance and used in GT4

• MS & Friends• .NET• Built on WS-Interoperability

(WS-I)• Forms basis of MS’s Web

Service Extensions (WSEs)


IBM, MS, HP, Intel Publicly Announce Intent to Converge Web Service Standards


Strategic Organizational Liaison• Potential OGC-OGF Collaboration• Workshop at OGF-20

• May 7, 2007, Manchester, UK• Organized by Chris Higgins (Edinburgh)

• General Agenda• Statements from key stakeholders & potential adopters• Panel on Specific Goals

• Goal• Memorandum of Understanding outlining concrete steps of collaboration

• Potential Technical Directions• Integration of registry concepts with current standards• Integration of services (e.g., WMS, WFS) w/ emerging WS standards• Identification of suitable security (user identity) model• Integration of resource mgmt, workflow, notification, tools, …

© 2006 OpenGridForum

NSF Support for Semantic Web Research

Frank OlkenNational Science FoundationCISE/[email protected]

Presentation toSICOP Special ConferenceFalls Church VA

Feb. 6, 2007


Why does NSF care about semantic web technologies?

• Formalization of scientific knowledge• Facilitate sharing of scientific data• Facilitate access to scientific data and knowledge• Natural language processing

• Information extraction, digital libraries, ...

• Support for digital government• Semantic rules languages, disaster support, ...

• Support for machine learning• Support for math/science education

Bullets courtesy Frank Olken, NSF/CISE/IIS, [email protected]


Debates about semantic web research• Skepticism about adoption of semantic tagging by the masses

(and the quality of the tagging)• NSF is concerned about scientific/govt uses, not MySpace.

• Poor Quality Ontologies• Ontology development and assessment remains difficult, rare skill. Some

progress (e.g., Ontoclean), clear need for more research and more training of practitioners.

• Ontology Merging is very very hard:• Currently subject of research, see Ontoclean work, also work by Joslyn,

et al. on use of partial orders.

• Skepticism of semantics by most of the database research community:• Still somewhat an issue, because semantic proposals often go to to

panels dominated by DB researchers. Progress in adding more semantic web researchers to panels.



More debates about semantic web research

• Description Logic vs. First Order Logic• Heated debates in KR research community about whether

description logics are adequate or whether FOL or other logics should be used.

• Scalability and structuring of rule bases• Concerns about the software engineering of large rule bases (or

collections of logic axioms). Efforts to partition such large rule bases / logic axiom collections (cf. Cyc's microtheories, etc.) This remains an open research topic.

• Skepticism about scalability of semantic search and inference engines

• Open research issue ...



• Which challenges and priorities does this group want to put on their research agenda?


Final Bit of Wisdom: Why there are no Penguins at the North Pole

Any further questions?

Documents

Geospatial Information, Fundamental Grid Challenges, and the Role of Standards Organizations