0

DAML Tools for Intelligent Information DAML Tools for Intelligent Information Annotation, Sharing and RetrievalAnnotation, Sharing and Retrieval

UMBCMIT Sloan School

Johns Hopkins University Applied Physics Lab

Tim FininBenjamin GrosofJames Mayfield

DAML PI meeting, 30 November 2004

1/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Overall Program SummaryOWL helps enable agents in open, heterogeneous, dynamic environments share knowledge and cooperate on tasks while managing privacy, commitments, trust, and security. To do this, we must ensure that:

– OWL integrates with common agent frameworks and standards– OWL integrates with other KR paradigms for real world reasoning -- rule

based systems, Bayesian reasoning, etc.– OWL permits IR systems to index and retrieve documents with text,

multimedia and knowledge

Travel Agents

Auction Service Agent

CustomerAgent

Bulletin BoardAgent

Market Oversight Agent

Request

Direct Buy

Report Direct Buy Transactions

BidBid

CFP

Report Auction Transactions

Report Travel Package

Report Contract

Proposal

Web Service Agents

2/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Technical Problem and Approach

• Current agent systems are not OWL enabled– Develop ontologies, software & tools to use OWL in FIPA systems– Develop a declarative, OWL based policy language to constrain and

advise agent behavior– Demonstrate and evaluate use of OWL & Agents in realistic applications

• Integrate OWL with rule-based & uncertainty reasoning– Work with groups to develop specifications for RuleML, SWRL, etc– Build prototype translation systems and demonstrations– Annotate OWL content with Bayesian info and generate BNs– Demonstrate utility for ontology mapping

• Information retrieval systems must work with RDF content– Augment existing IR systems like Google with Swangle terms– Develop Swoogle as a native RDF search engine

3/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Technical Progress

• Agents and OWL– Developed standards, ontologies and software to use OWL in FIPA

systems and demonstrated them in trading agents and pervasive com-puting applications

– Developed Rei policy language and demonstrated for security & priv-acy in web services, collaboration tools and pervasive computing

– WWW2005 Policy Management for Web Workshop• OWL and reasoning

– Helped lead efforts to integrate rule reasoning into the semantic web (RuleML, SWRL, SWRL FOL) resulting in solid specs

– Developed tools to translate the above to standard rule engine form– Developed an ontology to annotate OWL content with Bayesian info

and translate the ensemble into a Bayesian Network• OWL and Information Retrieval

– Developed Swoogle, a crawler-based IR system for RDF documents– Developed Swangling, a technique to allow standard IR engines (e.g.,

Google) to index RDF triples in retrievable form

D

D

4/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Milestones and Accomplishments2004 2005

TAGA

SweetRules v2

Rei

F-OWL reasoner

Swoogle

Swangler

Bayes OWL

Joint Committee

W3C WebOnt Working group SWSI & SWSA

5+/7?/2?28/5/525/4/120/1/04/0/0

RGBs

s

ss

s

s

s

s

Distributed IDS

Mogatu

OO O

O

O

RuleML

O

O

DLP s

s

2001 2002 2003

Agent &Systems

ITTALKS

Vigil Soupa Cobra sO O

SweetJessKR

OWLIRIR

specs

Papers/MS/PhD

s

Legend

Softwareor service

Ontology

Ph.D. Dissertation

Spec impact

O

5

http://daml.org/IT TALKSIT TALKS

UMBC, JHU/APL, and MIT/Sloan are working together on a set of issues to be integrated into agent-based applications involving search and using rule-based reasoning: UMBC: Integrating communicating agents, DAML and the Web; JHU APL: DAML and information retrieval; and MIT Sloan School: DAML, rules based technology and distributed belief

Features•Generation from DB to DAML and HTML mediated by MySQL, Java servlets, and JSP.

•Generation of DAML descriptions and user profiles from HTML forms.

•Creation & use of DAML-encoded user modelsdescribing interests and ontology extensions.

•Ontologies for events, people, places, schedules,topics, etc.

•Automatic HTML form (pre) filling from DAML.•Syncing of talks with Palm calendars via Coola.•Automatic classification of talks into topicontology

•A XSB-based DAML/RDF reasoning engine.

•Agent-based services:• ITTALKS agent with KQML API using DAMLas content language

• Intelligent matching of people and talks based on interests, locations and schedules.

• Agents using both Jackal and FIPA’s Java Message Service

• Notification via email and mobile devices via SMS and WML.

• Discovery of relevant background papers from NEC CiteSeer

• Automatic generation and maintenance of user models

• Talk recommendations via collaborative filtering• Integration with STP (Smart Things and Places) ubiquitous computing project

Ittalks.org a database driven web site of IT related talks at UMBC and other institutions. The database contains information on

– Seminar events– People (speakers, hosts, users, …)– Places (rooms, institutions, …)

This database is used to dynamically generate web pages and DAML descriptions for the talks and related information and serves as a focal point for agent-based services relating to these talks. To add your organization to ittalks.org and receive a domain (e.g., mit.ittalks.org) contact info@ittalks.org. See http://daml.org/ for more information on DAML and the semantic web.

http://ittalks.org/

UMBCAN HONORS UNIVERSITY IN MARYLAND

Webserver + Javaservlets

DAMLreasoning

engine

DAMLreasoning

engine

<daml></daml><daml>

</daml><daml></daml><daml>

</daml>

DAML files

Agents

Databases

People

RDBMSRDBMSDB

Email, HTML, SMS, WAP

FIPA ACL, KQML, DAML

SQLHTTP, KQML, DAML, Prolog

MapBlast, CiteSeer, Google, …HTTP

HTTP, WebScraping

WebServices

Acknowledgements: Funded by DARPA, contract F30602-00-2-0591. Students: H. Chen, F. Perich, L. Kagal, S. Tolia, Y. Zou, J. Sachs, S. Kumar and M. Gandhe. Faculty: T. Finin, A. Joshi, Y. Peng, R. Cost, C. Nicholas, R. Masuoka

6/18 UMBC/JHU/MIT 11/19/2004

Rei Policy Spec Language

• Rei is a product of Lalana Kagal’s dissertation (2004)• An OWL based declarative policy language• Models deontic concepts of permissions,

prohibitions, obligations and dispensations• Meta policies resolve conflicts• Speech acts can dynamically modify policies• Used to model different kinds of policies

Security; privacy; team formation/collaboration/maintenance; conversation constraints

• Use at UMBC, in joint DAML projects andby Global Infotek and Fujitsu

7

TAGA: Travel Agent Game in Agentcities

http://taga.umbc.edu/

TechnologiesFIPA (JADE, April Agent Platform)

Semantic Web (RDF, OWL)

Web (SOAP,WSDL,DAML-S)

Internet (Java Web Start )

FeaturesOpen Market Framework

Auction Services

OWL message content

OWL Ontologies

Global Agent Community

MotivationMarket dynamicsAuction theory (TAC)Semantic webAgent collaboration (FIPA & Agentcities)

Travel Agents

Auction Service Agent

CustomerAgent

Bulletin BoardAgent

Market Oversight Agent

Request

Direct Buy

Report Direct Buy Transactions

BidBid

CFP

Report Auction Transactions

Report Travel Package

Report Contract

Proposal

Web Service Agents

Ontologieshttp://taga.umbc.edu/ontologies/

travel.owl – travel concepts

fipaowl.owl – FIPA content lang.

auction.owl – auction services

tagaql.owl – query language

FIPA platform infrastructure services, including directory facilitators enhanced to use OWL-S for service discovery

Owl for representation and reasoning

Owl for service

descriptions

Owl for negotiation

Owl as a content language

Owl for publishing

communicative acts

Owl for contract

enforcement

Owl for modeling

trust

Owl for authorization

policies

Owl for protocol

description

OWL Everywhere

Context Broker Architecturehttp://cobra.umbc.edu

CoBrA FeaturesCoBrA Features[1] Uses OWL for context modeling and reasoning

[2] Uses abductive reasoning to detect and resolve inconsis-tent context knowledge

[3] Extend the REI policy language for privacy protection

[4] Adopt the FIPA standards for communication & knowledge sharing

EasyMeeting an intelligent meeting room prototype that provides services forspeakers, audience & organizers based on their situational needs.

9/18 UMBC/JHU/MIT 11/19/2004

RGB: Research Group in a Boxeating our own dog food

• The UMBC ebiquity portal exposes ts con-tent in RDF using a set of OWL ontologies for papers, people, projects, photos, etc.

• Lab members can add arbitrary RDF assertions about any of the portal’s objects.

• It’s designed as a modular, configurable package using O/S software that others can use to create their own portals.

• An reasoning component is planned to apply ontology sanctioned inferences & heuristics.

People

Papers

http://ebiquity.umbc.edu/

Photos

10/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Adding Rules to OWL

• Theory, algorithms and concepts– Combining LP rules + ontologies

(1) Predicate refers to ontology URI. (2) Description Logic Programs KR and fusion. (3) Courteous inheritance for OO default inheritance. (4) In-Progress: hypermonotonic reasoning to unify nonmonotonic LP with FOL.

– Translation between rule (+ontology) systems/KRs• SweetJess: Situated Courteous LP ↔ production rules

– In-Progress: Combine {Courteous LP, RuleML} with F-Logic Programs (Hilog, frame syntax) for SWSL-Rules

• Tools: SweetRules, V1, V2– SweetJess, SweetOnto (DLP), SweetPH (Process Handbook),

SweetXSB, SweetCR (CommonRules), core architecture• Specifications

– RuleML (co-led), SWRL (co-led), FOL SWRL, FOL RuleML. SWSL Rules and FOL (co-led)

demodemodemo

11/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Adding Rules to OWL

• Services and security application scenarios– Contracts: SweetDeal prototype– Trust policies– SWSL requirements analysis including match of nonmonotonic LP to

majority of SWS tasks• Metrics

– KR expressiveness and coverage of translation/merging/ inferencing– Importance and number of distinct rule/ontology systems that

interoperate– Scalability of inferencing for SCLP

• See: Rules plenary session for details • See: Demonstrations of SweetRules and SweetDeal

12

Unified Ontology Support using Bayesian Networks

MotivationMotivationReasoning within an ontology is un-certain due to noisy/incomplete dataDL based reasoning is over-generalized and inadequateRelations between concepts in different ontologies are inherently uncertain

ApproachApproachTranslating OWL ontologies to Bayesian networks (BNs)Concept mapping as conditional probabilitiesJoining translated BNs (by probabilistic mappings) dynamicallyOntology reasoning (in and across ontologies) as Bayesian inference

Translation ProcessTranslation ProcessExtend OWL for probability annotationDefine structural translation rules from RDF graphs to directed acyclic graphs of Bayesian networksConstruct conditional probability tables for individual variables in the BN DAG.

BN1

onto1

P-onto1Probabilistic ontological information

onto2

P-onto2

BN2

Probabilistic annotation

OWL-BN translation

concept mapping

Probabilistic ontological information

Student: Z. Ding. Faculty: Dr. Y. Peng, Dr. T. Finin, Dr. A. Joshi. Ack: DARPA Contract F30602-00-2-0591. 09/03.

13

Unified Ontology Support using Bayesian Networks

MotivationMotivationReasoning within an ontology is un-certain due to noisy/incomplete dataDL based reasoning is over-generalized and inadequateRelations between concepts in different ontologies are inherently uncertain

ApproachApproachTranslating OWL ontologies to Bayesian networks (BNs)Concept mapping as conditional probabilitiesJoining translated BNs (by probabilistic mappings) dynamicallyOntology reasoning (in and across ontologies) as Bayesian inference

Translation ProcessTranslation ProcessExtend OWL for probability annotationDefine structural translation rules from RDF graphs to directed acyclic graphs of Bayesian networksConstruct conditional probability tables for individual variables in the BN DAG.

Derived Conditional Probability TablesStudent: Z. Ding. Faculty: Dr. Y. Peng, Dr. T. Finin, Dr. A. Joshi. Ack: DARPA Contract F30602-00-2-0591. 09/03.

14

SWOs

SWIs

HTMLdocuments

Images

CGI scripts

Audiofiles

Videofiles

SWD = SWO + SWI

service

SwoogleSearch

SwoogleStatistics

demodemodemoSwoogle Search

Ontology Dictionary

analysis

SWOOGLE 2

The web, like Gaul, is divided into three parts: the regular web (e.g. HTML), Semantic Web Ontologies (SWOs), and Semantic Web Instance files (SWIs)

Human usersWeb ServerOntologyDictionary

Web Service Intelligent Agents

IR analyzer SWD analyzer

SWD Cache SWD Metadata

SWD Readerdigest

Contributors include Tim Finin, Anupam Joshi, Yun Peng, R. Scott Cost, Jim Mayfield, Joel Sachs, Pavan Reddivari, Vishal Doshi, Rong Pan, Li Ding, and Drew Ogle. Partial research support was provided by DARPA contract F30602-00-0591 and by NSF by awards NSF-ITR-IIS-0326460 and NSF-ITR-IDM-0219649. November 2004.

http://swoogle.umbc.edu/Swoogle uses four kinds of crawlers to discover semantic web documents and several analysis agents to compute metadata and relations among documents and ontologies. Metadata is stored in a relational DBMS. Services are provided to people and agents.

A SWD’s rank is a function of its type (SWO/SWI) and the rank and types of the documents to which it’s related.

SWD IR Engine

SWD RankSwoogle Statistics

Swoogle puts documents into a character n-gram based IR engine to compute document similarity and do retrieval from queries

The WebThe Web

CandidateURLs Web Crawler

discovery

Swoogle provides services to people via a web interface and to agents as web services.

6,800,000Individuals4,200Ontologies

53,000Properties45,000,000Triples

95,000Classes310,000SWDs

Statistics as of November 2004

15

OWL Search usingOWL

document

OWLDocument +

Swangledtriples

SwangleSearchQuery Triple

OWLDocument +

Swangledtriples

<j.0:SwangledTriple><rdfs:comment>Swangled text for

[<http://www.xfront.com/owl/ontologies/camera/#Digital>,<http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>

]</rdfs:comment><j.0:swangledText>BE52HVKU5GD5DHRA7JYEKRBFVQ</j.0:swangledText><j.0:swangledText>WS4KYRWMO3OR3A6TUAR7IIIDWA</j.0:swangledText> <j.0:swangledText>2THFC7GHXLRMISEOZV4VEM7XEQ</j.0:swangledText><j.0:swangledText>HO2H3FOPAEM53AQIZ6YVPFQ2XI</j.0:swangledText><j.0:swangledText>6P3WFGOWYL2DJZFTSY4NYUTI7I</j.0:swangledText> <j.0:swangledText>N656WNTZ36KQ5PX6RFUGVKQ63A</j.0:swangledText> <j.0:swangledText>IIVQRXOAYRH6GGRZDFXKEEB4PY</j.0:swangledText>

</j.0:SwangledTriple>

WebSearchEngine

Filters SemanticMarkup

InferenceEngine

LocalKB

SemanticMarkup

SemanticMarkup

Extractor

Encoder

RankedPages

SemanticWeb Query

EncodedMarkup

TextQuery

TextFiltersText

[<http://www.xfront.com/owl/ontologies/camera/#Digital>,<http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>

]

Inference

[<http://www.xfront.com/owl/ontologies/camera/#Digital>,<http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>

][<http://www.xfront.com/owl/ontologies/camera/#Digital>,<http://www.w3.org/2002/07/owl#Thing>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>

][<http://www.xfront.com/owl/ontologies/camera/#Digital> ,< http://www.w3.org/2002/07/owl#Thing >, < http://www.w3.org/2002/07/owl#Thing >

]

Swangler

Vision

Indexing and Search

Swangler

SwangleToolOWL Triples

JENA

16/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Transition/Handoff• Specifications

– Participated in W3C WebOnt, SWSA, SWSI– Co-founded RuleML, Co-led SWRL

• Software products– Swangler, F-OWL, SweetRules (+ SweetJess), Rei, Swoogle

• Major demonstration systems– ITTALKS, TAGA, EasyMeeting, Groove Team Agent, ebiquity site,

SweetDeal• Services

– Swoogle• Papers, tutorials, workshops

– 82 refereed papers, 17 MS theses, 8 PhD dissertations, 6 conference tutorials, several workshops

• Users– Swoogle has 100s of users, Rei used by ~4 other groups, SweetJess

and F-OWL each used by several groups

17/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

Remaining Issues

SweetRules authoring tools and studio/IDE environment; continued maintenance beyond 2005

SweetRules adoption

Extend OWL2BN translation algorithm to properties. Can prob-abilities be estimated from Swoogle data? Do mapping usecase.

Bayes OWL completion

Develop tools to enable Swangler and Swoogle to work with documents that embed RDF in XHTML.

IR over mixed RDF and text

Build a query interface for swangled Semantic Web documents. Develop a good usecase (intranet?)

Swangler lacks query I/F

Extend Swoogle’s database to include all instance data. How well will it scale?

Swoogle needs instance data

Incorporate SWRL/RuleML languages. Can we compile Rei policies to OWL+SWRL?

Rei policy langu-age needs rules

Complete design and implementation and documentation of existing components (e.g., SweetJess) and SWRL support

SweetRules polishing

RemediationIssue

18/18 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval

SummaryWe’ve demonstrated that• OWL helps agents in open, dynamic environments to share

knowledge and manage privacy, commitments, trust, and security.– Demonstration: ITTALKS system, TAGA multiagent system, SOUPA

ontology for pervasive computing• OWL can be integrated with other KR paradigms for real

world reasoning– Demonstration: SweetJess, SweetRules, Bayes OWL, Rei

• OWL is compatible with the information retrieval paradigm– Demonstration: Swoogle as an RDF search engine, Swangling

permits IR systems to index and retrieve documents with text, multimedia, and knowledge.

• OWL facilitates the sharing of knowledge thru web portals– Demonstration: ebiquity web pages