Semantic Web Processes: Semantics to exploit Web Services on a global scale IBM TJ Watson 10 July, 2003 Amit Sheth METEOR-SMETEOR-S Project Large Scale

Semantic Web Processes:Semantics to exploit Web Services on a global

scale

IBM TJ Watson10 July, 2003

Amit ShethMETEOR-S Project

Large Scale Distributed Information Systems (LSDIS) LabThe University of Georgia, Athens GA

Special Thanks: Kaarthik Sivashanmugam

© METEOR-S Team, UGA, 2003

http://lsdis.cs.uga.edu/~amit

http://lsdis.cs.uga.edu/proj/meteor/SWP.htm

http://lsdis.cs.uga.edu/

LSDIS Introduction Slide 1 of 3:Move from Syntax to Semantics in Information System

Semantic Web, some DL-II projects,Semantic Web, some DL-II projects,Semagix SCORE, Applied SemanticsSemagix SCORE, Applied Semantics

VideoAnywhereVideoAnywhereInfoQuiltInfoQuilt

OBSERVEROBSERVER

Generation IIIGeneration III(information

brokering)

1997...1997...

Semantics and Ontology-driven Information Systems

InfoSleuth, KMed, DL-I projectsInfoSleuth, KMed, DL-I projectsInfoscopes, HERMES, SIMS, Infoscopes, HERMES, SIMS,

Garlic,TSIMMIS,Harvest, RUFUS,...Garlic,TSIMMIS,Harvest, RUFUS,...

Generation IIGeneration II(mediators)

1990s1990s

VisualHarnessVisualHarnessInfoHarnessInfoHarness

Metadata

MermaidMermaidDDTSDDTS

Multibase, MRDSM, ADDS, Multibase, MRDSM, ADDS, IISS, Omnibase, ...IISS, Omnibase, ...

Generation IGeneration I(federated DB/

multidatabases)

1980s1980s

Data

Paradigm shift over time: Syntax -> Semantics

Increasing sophistication in applying semantics Relevant Information (Semantic Search & Browsing) Semantic Information Interoperability and Integration Semantic Correlation, Mining, Analysis, Early Warning

Ontology

ContentSources

Sem

i-St

ruct

ured

CA

ContentAgents

Stru

ctur

edU

nst r

u ctu

red

Documents

Reports

XML/Feeds

Websites

Email

Databases

CA

CA

KnowledgeSources

KA

KS

KS

KA

KA

KS

KnowledgeAgents

KSMetabase

Semantic Enhancement Server

Entity Extraction, Enhanced Metadata,

AutomaticClassification

Semantic Query ServerOntology and Metabase

Main Memory Index

Metadata adapter

Metadata adapter

Existing Applications

ECM EIPCRM

Semagix Freedom Architecture (a platform for building ontology-driven information system)

Web ProcessesWorkflows DistributedWorkflows

GlobalEnterprise Inter-Enterprise

B2B E-Services

Globalization of Processes

Processes driving the Networked Economy

LSDIS Introduction Slide 2 of 3:Enterprise Integration (Workflow Processes)

DAML-S, projects at IBMDAML-S, projects at IBMMETEOR-SMETEOR-S

Global ProcessesGlobal Processes

2001- 2001-

Semantic Web Processes (e-services,dynamic trading processes)

Exotica, ADEPT, MOBILE, …Exotica, ADEPT, MOBILE, …Advanced WorkflowAdvanced Workflow

1995-20001995-2000 METEOR-2METEOR-2

Distributed workflows and e-Commerce

METEOR-1METEOR-1Early Workflow MgmtEarly Workflow Mgmt

1990-19941990-1994

Centralized workflows

LSDIS Slide 3 of 3:Unique Attributes and Approach

• Extensive industry collaborations/partnerships (Boeing, MCC, Telcordia, CHREF, NIST, MCG, …)

• Commercial products from technology licensing and spin-offs: ADEPT/X Harness, METEOR EAppS and Semagix Freedom

• Research with well understood requirements/objectives (near or long term), focus on Enterprise Software, not just tools

• Real impact, not just papers

Processes driving the Networked Economy

“Companies distributed over space, time, and capability will have to come together to deliver products and solutions in global marketplace. Processes are already becoming chief differentiating and the competitive force in the networked economy. Processes are becoming an organic part of doing business.”

* From Sheth, Aalst et al, “Processes driving the Networked Economy” 1999

Architectures for Web Processes*

Stages of architectural evolution• Process Portal

– one stop for e-services, p2p interactions between buyer and sellers

– E-Gov, industry automation, Life Science

• Process Vortex– Interactions between buyer and seller through a third

party marketmaker, predefined processes, shared ontology

• Dynamically Trading Processes

* From Sheth, Aalst et al, “Processes driving the Networked Economy” 1999

BIG Challenge #1 from Enterprise to Global

Scope: Scalability

A

C D

N1 N2 FE

B8

A1A4 A1 A2

A4

B3

A1A4 A6

A2A2

A5

Before (Enterprise,Inter-enterprise workflows)

Discovery/Matchmaking should be accurate and scalable to the number of services available in Web

A

C D

N1 N2 FE

A4 A1A1

A4 A1 A4 A1 A2A1 A1A1 A1B3 A1

B3

A1

A2A1

B3

A1A4 A1 A2

A1B3

A1A4 A1 A2

A1

B3

A1

A1

B3

A1A4 A1 A2

A1

B3

A1A4 A2

A1

B3 A1A4 A1 A2

B3 A1A1

B3

A1A4 A1 A2

A1

B3

A1A4 A1

A2A1

B3 A1A4 A1 A2

A1B3

A1A4 A1 A2

A1

A1A4 A1

A1A4 A1 A2B3

A1A4

A1A2A1

B3

A1

A4A1 A2

A1

B3

A1A4 A1 A2

B3A1

A4 A1 A2A1

B3

A1A4 A1 A2

A1B3

A1A4 A1 A2

A1

A4

A4

A1

A4 A1

A1 A4 A1 A2

B3

A1A4 A1

A2

A1

B3

A1

A4

A1

A2B3

A1

A4 A1 A2A1

A4

A1

A4 A1

A1A4 A1 A2

A1

A1

A4 A1 A2A1

A1A4

A1

B3A1

A1A1

B3

A1A4 A1 A2

A1B3

A1A4 A1 A2

A1

A1 A2

A1

A4 A2A4 A2A2

A1

A1

B3

A1A4 A1 A2

A1

Now

Semantics of the services

Semantics of the activity(Data, Functional, Execution, QoS)

BIG Challenges

• Scalability from Enterprises to the Web (Global)– Machine understandable description to

support discovery, composition, etc.

• Dynamic nature of business interactions– Dynamic discovery, service selection,

composition,..

• Handle heterogeneity and autonomy– Syntactic, semantic and pragmatic– Complex rules/regulations related to B2B and

e-commerce interactions

METEOR-S @ LSDIS Lab

• Proposition: Semantics is the most important enabler.

• METEOR-S exploits Workflow, Semantic Web, Web Services, and Simulation technologies to meet these challenges in practical standards based framework.– Adding semantics to different layers of Web services

conceptual stack– Applying Semantics in all steps of process lifecycle– Using ontologies to provide underpinning for information

sharing and semantic interoperability

Relating SWP with the Semantic Web

Gen. Purpose,Broad Based

Scope of AgreementTask/ App

Domain Industry

CommonSense

Degre

e o

f Ag

reem

en

t

Info

rmal

Sem

i-Form

al

Form

al

Agreement About

Data/Info.

Function

Execution

Qos

Oth

er d

ime

nsio

ns:

how

ag

reem

ents

are

re

ach

ed

,…

Current Semantic Web Focus

Semantic Web Processes

Lots of Useful

SemanticTechnology

(interoperability,Integration)

Semantics for Web Services

• Data/Information Semantics– What: Formal definition of data in input and output messages of a web service– Why: for discovery and interoperability– How: by annotating input/output data of web services using ontologies

• Functional/Operational Semantics– Formally representing capabilities of web service– for discovery and composition of Web Services– by annotating operations of Web Services as well as provide preconditions

and effects; also annotating TPA/SLA – all using ontology(-ies)

• Execution Semantics– Formally representing the execution or flow of a services in a process or

operations in a service– for analysis (verification), validation (simulation) and execution (exception

handling) of the process models– using State Machines, Petri nets, activity diagrams etc.

• QoS Semantics (possibly part of above types)– Formally describing operational metrics of a web service/process: cost, time,

reliability,..– To select the most suitable service to carry out an activity in a process– using QoS model [Cardoso and Sheth, 2002] for web services (part of TPA/SLA

model?)

Data/ Information

Semantics

Development/ Description/ Annotation WSDL, WSEL

DAML-S

Meteor-S (WSDL Annotation)

Publication/ Discovery

UDDI

WSIL, DAML-S

METEOR-S (P2P model of registries)

Composition(Choreography?)

BPEL, BPML, WSCI, WSCL,

DAML-S, METEOR-S

(SCET,SPTB)

Execution(Orchestration?)

BPWS4J, Commercial BPEL Execution Engines,

Intalio n3, HP eFlow

Semantics for Web Process Life-Cycle

Data/ Information

Semantics


WSDL, WSEL

DAML-S


UDDI

WSIL, DAML-S




Development/ Description/ Annotation




DAML-S, METEOR-S

(SCET,SPTB)

Semantics for Web Service Life-Cycle

Functional/ Operational

Semantics


WSDL, WSEL

DAML-S


UDDI

WSIL, DAML-S








DAML-S, METEOR-S

(SCET,SPTB)


QoSSemantics


WSDL, WSEL

DAML-S


UDDI

WSIL, DAML-S








DAML-S, METEOR-S

(SCET,SPTB)


ExecutionSemantics


WSDL, WSEL

DAML-S


UDDI

WSIL, DAML-S








DAML-S, METEOR-S

(SCET,SPTB)



WSDL, WSEL

DAML-S


UDDI

WSIL, DAML-S




Semantics Required for Web Processes

ExecutionSemantics

QoSSemantics

Functional/ Operational

Semantics

Data/ Information

Semantics





DAML-S, METEOR-S

(SCET, SPTB)


Semantics at Different Layers of Web

Service Conceptual Stack 1

Publication

Discovery

Description

Messaging

Network

Flow

Description Layer: Why:

• Reason about the functionality of the services and the semantics of the I/O data

How: • Using Ontologies to semantically annotate WSDL

constructs (conforming to extensibility allowed in WSDL specification version 1.2) to sufficiently explicate the semantics of the – data types used in the service description, and– functionality of the service

Semantics Types: • Data, Functional and QoSPresent scenario:

• WSDL descriptions are mainly syntactic (provides operational information and not functional information)

• Semantic matchmaking is not possible1 [Kreger]

How to Annotate ?

• Map Web service’s input/output data as well as functional description using relevant data and function/operation ontologies, respectively

• How ?– Borrow from schema matching– Semantic disambiguation between

terms in XML messages represented in WSDL and concepts in ontology

Semantic Annotation

• IOParametersMatch (w,o) = LingusticMatch (w,o) + StructureMatch (w,o) +

ContextMatch (w,o) • LingusticMatch (w,o) =>

– NameMatch with stemming– Description Match– SynonymsMatch– HypernymRelation (w is a kind of o) : prevailing_speed is

a type of speed of a wind i.e. windSpeed– HyponymRelation (o is a kind of w)– Acronyms : Sea Level Pressure has acronym SLP

• StructureMatch (w,o) => – subTree(w) == subTree(o)

• ContextMatch– Name of the parent concept provides some insight to the

context of the term

Patil, Oundhakar, Sheth, SAWS Techincal Report

Semantic Annotation: Data Semantics

<xsd:complexType name="Wind"><xsd:sequence> <xsd:element name="prevailing_speed" type="xsd:double" /> <xsd:element name="gust_speed" type="xsd:double" /> <xsd:element name="prevailing_direction" type="xsd1:Direction" /> </xsd:sequence></xsd:complexType>

WindEvent

windSpeed

WeatherEvent

windDirection

PressureEvent

AltimeterSettingwindGustSpeed

SeaLevelPressure

PressureChangeEvent

Class

Property

<xsd:complexType name=“Pressure"><xsd:sequence> <xsd:element name=“altimeter" type="xsd:double" /> <xsd:element name=“slp" type="xsd:double" /> <xsd:element name=“delta" type="xsd:double" /> </xsd:sequence></xsd:complexType>


Ontology : weather-ont.daml

WSDL : GlobalWeather.wsdl

0.756

0.69

0.90.50.8

0.23

1.01.0

Semantic Annotation: Functional Semantics


<portType name="GlobalWeather"><operation name="getWeatherReport"> <input message="tns:getWeatherReport" /> <output message="tns:getWeatherReportResponse" /> </operation></portType><portType name="StationInfo"><operation name="searchByCountry"> <input message="tns:searchByCountry" /> <output message="tns:searchByCountryResponse" /> </operation>

WeatherFunctions

getWeathergetStation

getWind

getPressure

getTemperature

getStationByZip

getStationByCountry

A Sample Functional Ontology

WSDL Operations

Publication

Discovery

Description

Messaging

Network

Flow

Publication and Discovery Layers: Why: • Enable scalable, efficient and dynamic publication and discovery

(machine processability / automation)How: • Use of ontology to categorize registries based on domains and

characterize them by maintaining the1. properties of each registry2. relationships between the registries

• Capturing the WSDL annotations in UDDI so others can use for discovery

Semantics Types: • Data, Functional and QoSPresent scenario:

• Suitable for simple searches ( like services offered by a provider, services that implement an interface, services that have a common technical fingerprint etc.)

• Categories are too broad

• Automated service discovery (based on functionality) and selecting the best suited service is not possible

Semantics at Different Layers of Web

Service Conceptual Stack 1

Publication

Discovery

Description

Messaging

Network

Flow

Flow Layer: Why: • Design (composition), analysis (verification), validation

(simulation) and execution (exception handling) of the process models

• To employ mediator architectures for automated composition, control flow and data flow based on requirements

How: • Using

– Functionality/preconditions/effects of the participating services

– Knowledge of conversation patterns supported by the service

– Formal mathematical models like process algebra, concurrency formalisms like State Machines, Petri nets etc.

– Simulation techniques Semantics Types: • Data, Functional, QoS and ExecutionPresent Scenario: • Composition of Web services is static.• Dynamic service discovery, run-time binding, analysis and

simulation are not supported directly

Semantics at Different Layers of Web Service Conceptual Stack 1

Semantics in WS stack and METEOR-S Components

Publication

Discovery

Description

Messaging

Network

Flow

MWSDI: Scalable Infrastructure of Registries for Semantic publication and discovery of Web Services

MWSDI: Semantic Annotation of WSDL (WSDL-S)

MWSCF: Semantic Web Process Composition Framework

METEOR-S components for Semantic Web Services

• METEOR-S Discovery Infrastructure (MWSDI)– Semantic Annotation and Discovery of Web Services 1

– Semantic Peer-to-Peer network of Web Services Registries 2

• METEOR-S Composer (MWSCF)– Service Composition and Execution Tool 3

– Semantics Process Template Builder and Process Generator 4

– QoS Management• Specify, compute, monitor and control QoS (SWR algorithm) 5

• Orchestrator (Under development)

– Analysis and Simulation 6 , Execution, Monitoring 6

1 [Sivashanmugam et al.-1], 2 [Verma et al.], 3 [Chandrasekaran et al.], 4 [Sivashanmugam et al.-2], 5 [Cardoso et al.], 6 [Silver et al.]

METEOR-S Web Service Discovery Infrastructure (MWSDI)

uses Functional, Data semantics for service discovery uses QoS semantics for service selection

Use of ontologies enables shared understanding between the service provider and service requestor

Semantic Publication and Discovery

WSDL

<Operation>

<Input1>

<Output1>

Service Template

Operation:buyTicket

Input1:TravelDetails

Output1:Confirmation

Annotations

Publish

Search

UDDI

Class

TravelServices

Class

DataClass

Operations

subClassOf subClassOf

subClassOfsubClassOf subClassOf subClassOf

ClassTicket

Information

ClassTicket

Booking

ClassTicket

Cancellation

ClassConfirmation

Message

Operation:cancelTicket

Input1:TravelDetails

Output1:Confirmation

Extract details from WSDL file

Annotate the WSDL details using ontologies

Construct Service Requirement template

Annotate using ontologies

Discover services based on the template annotations

WSDL-S (WSDL with Semantic Annotation)

• Mapping Input and Output Message Parts to Ontology– XML Schema elements used in Input/Output messages do not reflect the

semantics of the data involved in Web Service operation– Use of ontologies or standard vocabulary* brings service provider and

requestor to common conceptual space providing well defined semantics for operational data

• Mapping Operations to Ontology– Service selection involves discovering appropriate WSDL description and

locating an operation to invoke– Operations with same signature could have different functionalities– Ontology or vocabulary depicting functionality is used for annotation

• Additional tags to represent pre-conditions and effects of each operation (should be added to next ver. of WSDL?)

– Preconditions and effects are added for each operation– Can be optionally used for service discovery and selection

* RosettaNet Business/Technical dictionary or ebXML Core Component catalog/dictionary

The focus of our work is not in developing ontologies for representing functionality/preconditions/effects but to use such ontologies for semantic annotation

METEOR-S Web Service Composition Framework (MWSCF)

TemplateBuilder

ActivityInterfaces

ProcessTemplates

UDDI

UDDI

UDDI

UDDIUDDI UDDI

Ontologies

ExecutionEngine

ProcessGenerator

Process Designer

Repositories

Discovery Infrastructure

(MWSDI)

Repositories are used to store 1. Web Service Interfaces 2. Ontologies 3. Process Templates

Process Designer 1. Template Construction activity specification using - interfaces - services - semantic activity templates - other details 2. Process Generation - Service discovery (automatic)

and selection (semi-automatic) - Data flow

MWSCF Architecture

Process Execution 1. Validation and deployment 2. Executing the process using a client

Web Process Life-Cycle

Find Matches

Rank Services

Select a Service

Discovery

Add to Process

Data Transformation

Data Flow

Composition

Generate Process

Validate Syntax

Execute

Execution

Design

Create Process WSDL

Create Process Templateand Add Activities

Add Control Flow

Find Ontologies &Annotate Activity

Requirements

Template Construction

Process Generation

Advantages of Semantic Process Template

• Flexible and rapid approach to process composition– Configuration and re-use of templates– Process is not bound to any Web service intefaces or

implementations. (partners/services can be dynamically changed)– Template/Process designer need not perform discovery of services.

Discovery can be delegated to the tool

• Well defined semantics for each activity– Using ontologies– Richer description of semantics of activities

• Can be generated in executable process in any standard

• Process re-design is easier• Can be advertised/published as reference/business

models for reuse across vertical industry segments

Conclusion

• Semantics can help address big challenges related to scalability, dynamic environments,..

• But comprehensive approach to semantics will be needed:– Data/information, function/operation, execution, QoS

• Semantic (Web) principles and technology bring new tools and capabilities that we did not have in EAI, workflow management of the past

More at: http://lsdis.cs.uga.edu/SWP.htm

References

• [Kreger] http://www-3.ibm.com/software/solutions/webservices/pdf/WSCA.pdf

• [Sivashanmugam et al.-1] Adding Semantics to Web Services Standards• [Sivashanmugam et al.-2] Framework for Semantic Web Process

Composition• [Verma et al.] MWSDI: A Scalable Infrastructure of Registries for

Semantic Publication and Discovery of Web Services• [Chandrasekaran et al.] Performance Analysis and Simulation of

Composite Web Services• [Cardoso et al.] Modeling Quality of Service for Workflows and Web

Service Processes• [Silver et al.] Modeling and Simulation of Quality of Service for

Composition of Web Services• [Paolucci et al.] Importing Semantic Web in UDDI• [UDDI-v3] http://uddi.org/pubs/uddi-v3.00-published-20020719.htm

More at: http://lsdis.cs.uga.edu/SWP.htm

Extensive related work at: IBM, Karlsruhe, U. Manchester, DAML-S (CMU, Stanford, UMD)

Now,what can we do together?