77
Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Embed Size (px)

Citation preview

Page 1: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Semantics – from Applications, and Middleware to Networks

LSDIS labContact Amit Sheth

Page 2: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

SW research @ LSDIS • Ontology design and population• Automatic Metadata Extraction, Semantic Annotations• Semantic Computations: (Inference), Rules,

Complex Relationships, Knowledge Discovery, Semantic Associations

• Active Semantic Documents• Semantic Web Services/Processes• Semantic Discovery on Adaptive Grid Services• Semantic Applications: Bioinformatics,

Health Care, Intelligence/Gov., (Commercial: Risk & Compliance, Content Aggregators)

• New themes: Semantics Enabled Networking & Middleware

Semantic Middleware

SemDis

WSDL-SMETEOR-S

Bioinformatics for Glycan Expressions

Page 3: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Unique proposition

• Innovation & Vision – ahead of the pack• Leadership and funding

– NSF, NIH, ARDA, IBM, AHC, ….

• Theory to Practice• Industry collaborations and support

– IBM Watson & Almaden, AHC, CTA, LMCO, ….

• Technology transfer (Infocosm, Taalee/Semagix, AHC, …)

• Standards Influence (W3C: WSDL-S, W3C: LSHC, OASIS)• Size: 5 faculty; 15-20 funded students/staff

Page 4: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Ontologies at the heart of SW• General: SWETO, SWETO-GS• Intelligence: Insider Threat, Financial

Irregularity• Bio: GlycO, ProPreO• Health care: Drug, Practice,

Diagnosis/Procedure

• and several commercial “ontologies”

Page 5: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Gen. Purpose,Broad Based

Scope of AgreementTask/ App

Domain Industry

CommonSense

Degre

e o

f Ag

reem

en

t

Info

rmal

Sem

i-Form

al

Form

al

Agreement About

Data/Info.

Function

Execution

Qos

Broad Scope of Semantic (Web) Technology and Ontologies

Oth

er d

ime

nsio

ns:

how

ag

reem

ents

are

re

ach

ed

,…

Current Semantic Web Focus

Semantic Web Processes

Lots of Useful

SemanticTechnology

(interoperability,Integration)

Cf: Guarino, Gruber

Page 6: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Bioinformatics Apps & Ontologies• GlycOGlycO: A domain ontology for glycan structures, glycan functions

and enzymes (embodying knowledge of the structure and metabolisms of glycans) Contains 600+ classes and 100+ properties – describe structural

features of glycans; unique population strategy URL: http://lsdis.cs.uga.edu/projects/glycomics/glyco

• ProPreOProPreO: a comprehensive process Ontology modeling experimental proteomics Contains 330 classes, 40,000+ instances Models three phases of experimental proteomics* –

Separation techniques, Mass Spectrometry and, Data analysis; URL: http://lsdis.cs.uga.edu/projects/glycomics/propreo

• Automatic semantic annotation of high throughput experimental data Automatic semantic annotation of high throughput experimental data (in progress)

• Semantic Web Process with WSDL-S for semantic annotations of Web Semantic Web Process with WSDL-S for semantic annotations of Web ServicesServices

– http://lsdis.cs.uga.edu -> Glycomics project (funded by NCRR)

Page 7: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

GlycO – A domain ontology for glycans

Page 8: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

GlycO

Page 9: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

GlycoTree – A Canonical Representation of N-Glycans

N. Takahashi and K. Kato, Trends in Glycosciences and Glycotechnology, 15: 235-251

-D-GlcpNAc-D-GlcpNAc-D-Manp-(1-4)- -(1-4)-

-D-Manp -(1-6)+-D-GlcpNAc-(1-2)-

-D-Manp -(1-3)+-D-GlcpNAc-(1-4)-

-D-GlcpNAc-(1-2)+

-D-GlcpNAc-(1-6)+

Page 10: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

N-GlycosylationN-Glycosylation ProcessProcess (NGPNGP)Cell Culture

Glycoprotein Fraction

Glycopeptides Fraction

extract

Separation technique I

Glycopeptides Fraction

n*m

n

Signal integrationData correlation

Peptide Fraction

Peptide Fraction

ms data ms/ms data

ms peaklist ms/ms peaklist

Peptide listN-dimensional arrayGlycopeptide identificationand quantification

proteolysis

Separation technique II

PNGase

Mass spectrometry

Data reductionData reduction

Peptide identificationbinning

n

1

Page 11: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth
Page 12: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Creating and Serving Metadata to Power the Life-cycle of Content

Where is the

content? Whose is

it?

ProduceAggregate

What is this

content about?

Catalog/Index

What other

content is it

related to?

Integrate Syndicate

What is the right

content for this user?

Personalize

What is the best way to

monetize this interaction?

Interactive Marketing

Broadcast,Wireline,Wireless,Interactive TV

Semantic Metadata

Content ApplicationsInfrastructure Services

Page 13: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

WWW, EnterpriseRepositories

METADATAMETADATA

EXTRACTORSEXTRACTORS

Digital Maps

NexisUPIAPFeeds/

Documents

Digital Audios

Data Stores

Digital Videos

Digital Images. . .

. . . . . .

Create/extract as much (semantics)metadata automatically as possible, from: Any format (HTML, XML, RDB, text, docs)Many mediaPush, pullProprietary, Deep Web, Open Source

Metadata extraction from heterogeneous content/data

Page 14: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Metadata and Ontology: Primary Semantic Web enablers

Page 15: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Automatic Semantic Annotation of Text:Entity and Relationship Extraction

KB, statistical and linguistic

techniques

Page 16: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Automatic Semantic Annotation

Limited tagging(mostly syntactic)

COMTEX Tagging

Content‘Enhancement’Rich Semantic

Metatagging

Value-added Semagix Semantic Tagging

Value-addedrelevant metatagsadded by Semagixto existing COMTEX tags:

• Private companies • Type of company• Industry affiliation• Sector• Exchange• Company Execs• Competitors

© Semagix, Inc.

Page 17: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

BRAHMS• BRAHMS - a workBench Rdf store And High-

performance Memory System for Semantic Association Discovery (ISWC 2005)

• Main-memory RDF storage with rich API of basic graph operations on nodes and edges

• Written in C++– bindings for Java (new SemDis API standard)

• Optimized for maximum speed, minimize and strict control memory usage

• Created as a general framework for testing graph algorithms on RDF/S knowledge base

Page 18: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

BRAHMS Design• Indexing for speed in basic operations

– full indexing of statements allows linear-time merges of triples during search

• SPO, SOP, OSP, OPS, PSO, POS

• Minimize memory usage– storage designed for main memory (also available:

memory mapped file on Unix)

• Read-only knowledge base– precomputed and compacted indexes– indirect adressing (by node ID, not pointer)

• Knowledge base as memory snapshot– RDF parsing and indexing happens only once

Page 19: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

BRAHMS Design• Separation of instances base and schema

– different types of classes for different resource types (instance, literals, schema class, property)

– specialized statements to handle separately instance resources, literals and schema – do not need to check for resource type during algorithm execution

– each resource is uniquely identified in its group by numeric identifier

– identifiers are contiguous [0..n] in each group, allowing straightforward sorting and indexing

• Taxonomy– precomputed full taxonomy for classes and properties

(including all ancestors and descendants)

Page 20: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

BRAHMS Results• Speed

– outperform Sesame, Jena and Redland in k-hop limited semantic association searches using main-memory RDF model

– big impact using large datasets, when other datastores either perform slowly or cannot execute algorith at all

• Handling datasets– size limited by main-memory (physical) and/or system

(32 Vs. 64bit)– able to efficiently run algorithms on large datasets, that

other RDF storages cannot handle using memory-model– tested: SWETO [255Mb], Lehigh University – Univ(50, 0)

[556Mb], synthetic [9Gb] /64bit machine/

Page 21: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

BRAHMS Results

Timing results of bi-directional Breadth-First Search for paths of length 6 to 10 on Univ(50,0) dataset [556Mb]

Page 22: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Examples of Semantic Applications Today

Page 23: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Semantic Search using named relationships• Relevance is measured based on how

documents relate to a ‘context’ query• Example: “Michael Jordan” (the research scientist on

Computer Science)

Semantic Search using ‘named relationships’

Page 24: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Indexing Challenges

• Indexing known entities (from an ontology) within documents

• Multiple typed Entities – i.e. teacher, activist, blogger, …

• Inter-entities links– i.e. topic-based

Semantic Search using ‘named relationships’

Page 25: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Ranking ChallengesConsider the following challenges:• Importance of entities• Relevance to a ‘user-context’

• There could be various contexts (dynamicity)

• Groupings or top-K document listings• Intra-document cohesion• Relevance of path-sequences

• i.e. relationships from document context

• (Semantic) Link analysis• i.e. transitivity of “located in” relationships

Semantic Search using ‘named relationships’

Page 26: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

• Knowledge discovery is the nontrivial extraction of implicit, previously unknown, and potentially useful information from data.

What does it mean to Discover?

• Knowledge discovery (driven by domain semantics) is the nontrivial extraction of implicit, previously unknown, and potentially useful relationships between entities in metadata.

Page 27: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

A general example of knowledge discovery

Page 28: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

How are the following things related?

Et in Arcadia ego by Nicolas PoussinThe Mona Lisa by Leonardo Da Vinci

The Last Supper by Leonardo Da Vinci

Vitruvian Man by Leonardo Da Vinci

Page 29: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Or these for that matter?

Page 30: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

A real example

Leonardo Da Vinci

The Da Vinci code

The Louvre

Victor Hugo

The Vitruvian man

Santa Maria delle Grazie

Et in Arcadia EgoHoly Blood, Holy Grail

Harry Potter

The Last Supper

Nicolas Poussin

Priory of Sion

The Hunchback of Notre Dame

The Mona Lisa

Nicolas Flammel

painted_by

painted_by

painted_by

painted_by

member_of

member_of

member_of

written_by

mentioned_in

mentioned_in

displayed_at

displayed_at

cryptic_motto_of

displayed_at

mentioned_in

mentioned_in

Page 31: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Leonardo Da Vinci

The Da Vinci code

The Louvre

Victor Hugo

The Vitruvian man

Santa Maria delle Grazie

Et in Arcadia EgoHoly Blood, Holy Grail

Harry Potter

The Last Supper

Nicolas Poussin

Priory of Sion

The Hunchback of Notre Dame

The Mona Lisa

Nicolas Flammel

painted_by

painted_by

painted_by

painted_by

member_of

member_of

member_of

written_by

mentioned_in

mentioned_in

displayed_at

displayed_at

cryptic_motto_of

displayed_at

mentioned_in

mentioned_in

Page 32: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Distributed Computation of Semantic Associations

Page 33: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Why compute Semantic Associations in a distributed environment?• Extremely Large Dataset

– Allows division of data such that it can be loaded into memory across many machines

• Separate Data Sources– Cooperating Organizations want to compute

associations but maintain separation of data

• Increased Speed– Parallel algorithm can run on a multi-processor

machine

Page 34: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

RDFData Store

RDFData Store

RDFData Store

Central Controller

User

r-path (A, B, k)

Query-Panning Index (graph)

System Architecture

1) User asks a query at local Data Store

2) Local Data Store forwards the query to central controller

3) Central Controller forms a query plan and returns this plan to the local Data Store

4) Local Data Store queries other data stores in the system to form final answer

5) These sub-queries will run in parallel (speedup potential)

Page 35: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Knowledgebase Borders

Peer 1

Peer 2

Border Node

Overlap (Peer_1:Peer_2 Border)

Data Stores are Linked by Their Common Resources (Nodes)

Page 36: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Intuition Behind Planning Method

• Sub-query end points can be either a border or a single node

• If we know the minimum distance (number of hops) between borders we can determine upper-bounds on hop limits for sub-queries and eliminate unnecessary searches.

2 3

KB1 KB3

KB2

2start end

In this example (numbers represent min distances)

• If we have the query r-path (start, end, k)• If k < 7, we can eliminate a query through KB2• Otherwise the hop limit through KB2 is k – (3 + 2)

Page 37: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Active Semantic DocumentA document (typically in XML) with• Lexical and Semantic annotations (tied to

ontologies) • Actionable information (rules over semantic

annotations)

Application: Active Semantic Patient Record for Cardiology Practice

Page 38: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Practice Ontology

Page 39: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Practice Ontology

Page 40: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Drug Ontology Hierarchy (showing is-a relationships)

Page 41: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Drug Ontology showing neighborhood of PrescriptionDrug concept

Page 42: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

First version of Procedure/Diagnosis/ICD9/CPT Ontology

maps to diagnosis

maps to procedurespecificity

Page 43: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Active Semantic Doc with 3 Ontologies

Referred doctor from

Practice Ontology

Lexical annotati

on

ICD9 codes from Diagnosis

Procedure Ontology

Page 44: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Active Semantic Doc with 3 Ontologies

Drug Allerg

y

Formulation RecommendationUsing Insurance

ontology

Drug Interaction using Drug Ontology

Page 45: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Explore neighborhood for drug Tasmar

Explore: Drug Tasmar

Page 46: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Explore neighborhood for drug Tasmar

belongs to group

belongs to group

brand / generic

classification

classification

classification

interaction

Semantic browsing and querying-- perform decision support (how many patients are using this class of drug, …)

Page 47: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

On-line demo of Active Semantic Electronic Medical Record(being deployed at Athens Heart Center)

Page 48: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Scientific Literature/PublicationsPublic Databases, Intra-lab Data

Repositories

Instance base

ONTOLOGY FRAMEWORK

REASONING SYSTEM

THE KNOWLEDGE PYRAMID

Page 49: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Building a Semantic Web Testbed

Page 50: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Unified Medical Language System (UMLS) – An upper level ontology for the BioMedical Domain •Semantic Network – concepts and named relationships between them•Metathesaurus•Specialist Lexicon

Medical Subject Heading (MeSH) – Hierarchy of 26,000 BioMedical concepts organized in 15 overlapping trees.

Semantic interpretation of the relationship between concept in this hierarchy limited to ISA and PART-OF

PubMed – A database of over 15 million scientific (BioMedical) articles published over the past 40-50 years.These are indexed with MeSH terms, which can be used to search this database of documents.

EnzymeComplicatesCell Function

Examples: Neoplastic Process, Enzymes, Cell function etc.

Examples: produces, complicates, result_of

GnT-VComplicates

Neoplastic Metastasis

Documents and Experimental data

hypothesis

validation

Page 51: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Value add of Semantics

• Multiple ontologies (UMLS, BioPax, etc.)• Multiple data sources (PubMed,CPT,ICD-9)• Extracting relationships from PubMed article

abstracts to instantiate those suggested by UMLS– Not as hard as arbitrary fact extraction

• Since end points and candidate relationships known

Page 52: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

METEOR-S: Semantic Web Process

Page 53: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Four types of Semantics• Data/Information Semantics

– What: (Semi-)Formal definition of data in input and output messages of a web service

– Why: for discovery and interoperability– How: by annotating input/output data of web services using ontologies

• Functional Semantics– (Semi-) Formally representing capabilities of web service– for discovery and composition of Web Services– by annotating operations of Web Services as well as provide preconditions and

effects• Execution Semantics

– (Semi-) Formally representing the execution or flow of a services in a process or operations in a service

– for analysis (verification), validation (simulation) and execution (exception handling) of the process models

– using State Machines, Petri nets, activity diagrams etc.• Non-Functional Semantics

– (Semi-) formally represent qualitative and quantitative measures of Web process– Qualitative includes security, transactions (WS-Policy)– Quantitative includes cost, time etc. (WS-Agreement)– Business constraints and inter service dependencies (Domain and application

ontologies)

Page 54: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Adding semantics to WSDL – guiding principles• Build on existing Web Services standards

• Mechanism independent of the semantic representation language

• Mechanism should allow the association of multiple annotations written in different semantic representation languagesSupport semantic annotation of Web Services whose data types are described in XML schema

• Provide support for rich mapping mechanisms between Web Service schema types and ontologies

Page 55: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

WSDL-S approach

• evolutionary and compatible upgrade of existing Web services standards

• describe semantics and operation level details in WSDL - upward compatibility.

• externalize the semantic domain models - agnostic to ontology representation languages.

Page 56: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Discovery in Semantic Web Using Semantics

• Functionality: What capabilities the requestor expects from the service (Functional semantics)

• Inputs: What the requestor can give to the to the Web service (Data semantics)

• Outputs: What the requestor expects as outputs from the service (Data semantics)

• Non-Functional: Quality of Service the distributor expects from the service (Non-Functional semantics)

Web ServiceDiscovery

Web ServiceDiscovery

(Functional semantics)(Data semantics)(Non- Functional semantics)(Syntactic description)

• Description: Natural language description of the service functionality (Syntactic description)

Page 57: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Extended Registries Ontologies (XTRO)

• Provides a multi-faceted view of all registries in MWSDI – Federations– Domains– Registries

subDomainOf

supports

belongsTo

consistsOf

belongsToFederation

Ontology

Registry

Domain

RegistryFederation

Page 58: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Constraint Analyzer/Optimizer• Constraints can be specified on each activity or

on the process as a whole.• An objective function can also be specified e.g.

minimize cost and supply-time etc• The Web service publishers provide constraints

on the web services.• The constraint optimizer makes sure that the

discovered services satisfy the client constraints and then optimizes the service sets according to the objective function.

Page 59: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Constraint Representation – Domain Constraints

Fact OWL expression

Supplier1 is an instance of network adaptor supplier

Supplier1 supplies #Type1Supplier1 is a preferred supplier.

<NetworkAdaptorSupplier rdf:ID="Supplier1">

<supplies rdf:resource="#Type1"/><supplierStatus>preferred</supplierStatus></NetworkAdaptorSupplier>

Type1 is an instance of NetworkAdaptor

Type1 works with Type1Battery

<NetworkAdaptor rdf:ID="Type1"> <worksWith><Battery rdf:ID="Type1Battery"></worksWith></ NetworkAdaptor >

Page 60: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Constraint Representation – Process Constraints

Feature Goal Value Unit Aggregation

Cost Optimize Dollars Σ (minimize total process cost)

supplytime Satisfy < 7 Days MAX (Max. supply time below Value)

partnerStatus Optimize MIN (Select best partner level; lower value for preferred partner)

Page 61: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Working of Constraint Analyzer

DiscoveryEngine

Optimizer (ILP)

Service Template 1

Service Template 2

ST=2C=100

ST=3C=250

ST=3C=200

ST=1C=300

ST=4C=200

ST=3C=180

Ranked Set

Objective Functionand Process constraintsMin (supply-time + cost)

Supply-time <= 4

Cost <=200

Network Adaptor

Supply-time <= 3

Cost <=300

Battery

Process constraintsSupply-time<=7

Cost<=400Min (Cost, Supply-time)

ST=2C=100

ST=3C=250

ST=4C=200

ST=3C=180

Abstract ProcessSpecifications

Domain Reasoner

(DL)

ST=2C=100

ST=3C=250

ST=4C=200

ST=3C=180

Ranked Set

Service templates and service constraints

Domain constraints in ontologies

Most optimal set cannot be chosen because of inter service dependenciesNetwork Adaptor from supplier 1 does not work battery from supplier 2

Page 62: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Web Process Engine (BPEL)

METEOR-S

ProcessManager Proxy 1 Proxy 2 Proxy 3

Discovery

ConstraintAnalyzer Mediator

Invoker

SemanticTemplates

DiscoveredServices Ranked

ServicesAfteropt.

NotifyProcess

InvokeService

GetService

Info

ReturnService

InfoInvokeService

ServiceFailed

Reconfigure

HaltHaltDiscover Services

DiscoveredServices Ranked

ServicesAfterOpt

NotifyNotifyNotifyInvokeService

ServiceSuccessful

ReplyTo

process

Page 63: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Tooling support

• METEOR-S Radiant– Annotating and Publishing Web Services

• METEOR-S Web Process Designer– Creating Configurable Web processes

• METEOR-S Execution Environment– Executing configurable Web processes

Page 64: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Autonomic Web Processes: The next step• Self aware, Self configuring, Self

optimizing, Self healing Web processes• WSDL-S to capture the process and service

level requirements• Policy driven process execution

Page 65: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Semantics Enabled Edge Services

• Semantics-based Consistency Management Middleware (SCMM)

• Automatic Fragment Detection in Dynamic Web Pages

Page 66: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

• Enterprise Edge Cache networks

– JCache, DynaCache

• Strong Consistency

• Weak Consistency

• Performance costs

• Too rigid

Consistency Management Middleware

Edge Server

Origin Server

Client

Page 67: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Motivation & Design Requirements• Real world applications demand flexibility

& control– Airline reservations, Online Auctions

• Time-varying consistency based on semantic and temporal states

• SCMM empowers applications to specify consistency requirements– Operators on semantic & temporal states

• Four consistency modes

Page 68: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

SCMM Design

Edge Server

Client

Buy a ticket to SFO for

tomorrow

Buy a ticket to JFK for 03/01/06

HPQ LPQ

Origin Server

STRENGHTS

• Empowering Applications

• Judicious use of resources

• Better performance

• Improved “flood” handling

• Low & less harmful rollbacks

Buy ticket on flight with < 5

seats

Page 69: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Fragment-based Edge Services

• Parts of a web pages– Encapsulate distinct

themes

• Advantages– Increases cacheable

content– Reduces Invalidations– Improves disk-space

utilization

• Manual fragmentation error-prone & costly

Page 70: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Automatic Fragment Detection• Challenges

– Identifying themes & functionalities– Sensitivity towards update and request patterns

• Our approach– Shared fragments & Lifetime-Personalization

fragments– Analysis of web pages delivered by edge servers

• Augmented Fragment tree (AF tree)– Shingles for similarity detection

• Two algorithms

Page 71: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Experimental Results Overview

Page 72: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

iTV: Extreme Personalization

Content Provider

(DBS, DISH, Wink, AOL-TV)

Semantic EngineTM

Meta-DataTagged Content

Content,“Programs”

Personalized Content Capsules,

Redirects and Programming

Immediate Interests,

Preferences,

Structured, Hi-QualitySemantic Metabase

Page 73: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Metadata for Automatic Content EnrichmentInteractive Television

This segment has embedded or referenced metadata that isused by personalization application to show only the stocksthat user is interested in.

This screen is customizablewith interactivity featureusing metadata such as whetherthere is a new ConferenceCall video on CSCO.

Part of the screen can beautomatically customized to show conference call specific information– including transcript,participation, etc. all of which arerelevant metadata

Conference Call itself can have embedded metadata to support personalization andinteractivity.

Page 74: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

Metadata in Enterprise Apps

Sony

Categorize

Catalog

Integrate

Filter, Search, Consolidate,Personalize, Archive,Licensing, Syndication

CollectionCollection ProcessingProcessing Production SupportProduction Support

NetworkContent

AffiliateFeeds

Public Sources

Rich Data

Metabase

Page 75: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

-- Breaking News --

Gore Demands That Recount Restart

Gore Says Fla. Can't Name Electors

Bush Meets Colin Powell at Ranch

Market Tumbles on Earnings Warning

Barak Outlines His Peace Plan

(1.33) – 12/06/00 - ABC

(2.53) - 12/06/00 - CBS

(5.16) - 12/06/00 - ABC

(2.46) - 12/06/00 - FOX

(1.33) - 12/06/00 - NBC

(5.33) - 12/06/00

(3.57) - 12/06/00 - CBS

(4.27) - 12/06/00 - ABC

(3.44) - 12/06/00 - FOX

(7.24) - 12/06/00 - CBS

(1.33) - 12/06/00 - CBS

TALLAHASSEE, Florida (CNN) – Though the two presidential candidates have until noon Wednesday to file briefs in Al Gore's appeal to the Florida Supreme Court, the outcome of two trials set on the same day in Leon County, Florida, may offer Gore his best hope for the presidency. Democrats in Seminole County are seeking to have 15,000 absentee ballots thrown out in that heavily Republican jurisdiction -- a move that would give Gore a lead of up to 5,000 votes statewide. Lawyers for the plaintiff, Harry Jacobs, claim the ballots should be rejected because they say County Elections Supervisor Sandra Goard allowed Republican workers to fill out voter identification numbers on 2,126 incomplete absentee ballot applications sent in by GOP voters, while refusing to allow Democratic workers to do the same thing for Democratic voters.

The GOP says that suit, and one similar to it from Martin County, demonstrates Democratic Party politics at its most desperate. Gore is not a party to either of those lawsuits. On Tuesday, the judge in the

(1.33) - 12/06/00 - ABC

(2.33) - 12/06/00 - CBS

(3.12) - 12/06/00 - NNS

(0.32) - 12/06/00 - CBS

(1.33) - 12/06/00 - CBS

Description

Produced by : CNN   Posted Date : 12/07/2000 Reporter :  David Lewis Event : Election 2000 Location : Tallahassee, Florida, USAPeople : Al Gore

Page 76: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

SceneDescriptionTree

Retrieve Scene Description Track

“Cisco Systems”

Node

Enhanced XML

Description

MPEG-2/4/7

Enhanced Digital Cable

Video

MPEGEncoder

MPEGDecoder

Node = AVO Object

TaaleeSemantic

Engine“Cisco Systems”

Produced by: Fox Sports   Creation Date: 12/05/2000 League: NFLTeams: Seattle Seahawks, Atlanta Falcons Players: John Kitna Coaches: Mike Holmgren, Dan Reeves Location: Atlanta

Object Content Information (OCI)

Metadata-richValue-added Node

Create Scene Description Tree

GREATUSER

EXPERIENCE

Metadata’s role in emerging iTV infrastructure

Channel salesthrough Video Server Vendors,

Video App Servers, and Broadcasters

License metadata decoder and semantic applications to

device makers

Page 77: Semantics – from Applications, and Middleware to Networks LSDIS lab Contact Amit Sheth

SENS Components

RouterWith SENS

Manning passed a TD to Shockey

Harrison and Manning Break NFL record

AnnotatedContent

Annotator(Uses SEE)

Name this(Uses Relationship

Discovery)

Publish-SubscribeServer

(Uses BRAHMS)

Logger

<NFL:NYG,QB>Manning passed a <NFL:TD>TD to

<NFL:NYG,WR>Shockey

<NFL:IND,WR>Harrison and <NFL:IND,QB>Manning Break

<Sports:NFL>NFL record

Sub ListIndiana

NewsNY Post

<NFL:IND,WR>Harrison and <NFL:IND,QB>Manning Break

<Sports:NFL>NFL record

Sub ListIndiana

NewsNY Post

<NFL:IND,WR>Harrison and <NFL:IND,QB>Manning Break

<Sports:NFL>NFL record

Indiana Post

Sub ListIndiana

NewsNY Post

<NFL:NYG,QB>Manning passed a <NFL:TD>TD to

<NFL:NYG,WR>Shockey

Sub ListIndiana

NewsNY Post

<NFL:NYG,QB>Manning passed a <NFL:TD>TD to

<NFL:NYG,WR>Shockey

Manning passed a TD to Shockey

Harrison and Manning Break NFL record