Agents on the Semantic Web David De Roure Intelligence Agents Multimedia Dept of Electronics and Computer Science University of Southampton, UK [email protected]

Agents on theSemantic Web

David De RoureIntelligence Agents Multimedia

Dept of Electronics and Computer ScienceUniversity of Southampton, UK

[email protected]

http://www.ecs.soton.ac.uk/~dder

Semantic Web Technologies November 2000

2

Overview

The Pervasive Information Fabric Agents state of play Why agents need metadata Onward to ontologies


3

BackgroundThis talk is based on the experience of building several applications (research prototypes) involving agents. These include:

Conceptual hypermedia and ontologies Context-aware hypermedia linking Information middleware for pervasive computing Collaborative filtering systems ‘Content based’ navigation of multimedia (images and

music)

We have also developed an Agent framework forDistributed Information Management


4

Background

Current activities include:

Mixed reality adaptive information systems Advanced knowledge technologies Disappearing computer initiative project Simulation of very large scale distributed

systems Grid computing (information/knowledge

grid)

The Pervasive Information Fabric


6

The Web of the past…

Is mainly a document dissemination Web Can only link to multimedia Assumes a traditional Web browser Has static hyperstructure


7

The Web of the future is…

Multimedia, including temporal media Mobile (with different style of working) Adaptive and open e.g. XLink Collaborative Automated (machine-to-machine e.g. XML) Semantic (of course!) And…


8


9

…pervasive

Embedded internet e.g. The Disappearing Computer Initiative Large numbers of devices Ad hoc networking (some support from IPv6) Systems need to be self organising Don’t wait for 1000s of bluetooth devices

before exploring scalability issues!

We call the middleware the Pervasive Information Fabric


10

…virtual

Interface is 3D worlds, telepresence, VR ‘click’ on object, query has spatial context Visualisation of results?

For example,abstract 3D midi visualisaton with links


11


12

Scenario 1 – this meeting!

Our devices communicate using wireless/ad hoc networking, publishing information resources and associated metadata

A hyperstructure (web) is created on-the-fly, enabling us to navigate our local information space. Links derived from metadata, metadata derived from documents; also bookmarks.

When a message comes in (e.g. mobile phone call) it is routed appropriately to minimise invasiveness


13

Scenario 2 – the musician Musician walks on stage with bluetooth guitar Musical devices on stage are located

automatically, musician chooses to control appropriate instrument(s)

Plays a few notes, musical score appears on private display

More…the musician appears in virtual world, other musicians there too, playing virtual instruments, audible in physical world

(AR meets VR)


14

Scenario 3 – the information grid Group of experts deciding whether to make

changes to production line in manufacturing organisation

Use multiple simulations to investigate Processes farmed out across clusters (server

farms) on WAN Visualise results locally, collaboratively Compare with results from previous runs

(Is this The Web?)

AgentsState of Play


16

Software agents

A buzzword for over 5 years now! Classic early papers:

‘Agents that reduce work and information overload’ 1994

‘Intelligent Agents: Theory and Practice’ 1995 The paradigm of weak agency is widely

accepted, especially in the Web area There is a convenient subcategorisation into:

Personal and information agents Multiagent systems


17

Personal information assistants

Personal assistants that collaborate with the user at the user interface

Learning by ‘watching over their shoulders’, building and maintaining model of user

Believable agents (cf traditional media)

Movie/CD/book/document recommender agents exist, though with fairly weak user models

For ‘people’, read ‘businesses’


18

Multiagent systems

Whole is greater than sum of the parts Requires agent communication languages

(eg KQML, FIPA ACL) Ontology required – to agree the terms to be

exchanged in communications

Agent frameworks have emerged Few large scale systems exist


19

Nwana’s appraisal

“The ontology issues has always been considered secondary to other issues such as cooperation, negotiation, formalisation and logics for beliefs, desires and intentions, etc. ... This problem is at the core of the agent interoperability issue – is it reasonable to expect knowledge and cooperation level interoperability without a significant degree of ontological sophistication of the agents concerned?”

Nwana and Ndumu, A Perspective on Software Agents Research, BT Labs


20

My appraisal

XML, RDF(S) are useful infrastructure Need also to address…

Scalability – putting the ‘multi’ into multiagent!

Security – not just an add-on Performance Real systems!


21

Current work

Collaboration and negotiation between agents Market-based models, e.g. auctions See http://www.ecs.soton.ac.uk/~nrj

Applications in e-commerce but also telecoms Agents in the PIF; e.g. briefing room scenario –

routing information in right format to right device at right time, taking account of security and invasiveness issues

Agents and metadata


23

Agents… Use metadata to find resources and work with

them They also create and maintain metadata

For example, our ‘query by humming’ system MIDI data gathered from net Tidied Channels classified and indexed Queries routed by index servers Results presented (e.g. SMIL)

There is also metadata associated with the agents!


24

Agents…recommender systems MEMOIR, an early agent-based recommender

system Logged trails in object-oriented database Users could ask ‘who else has looked at these

documents?’ and ‘what else did they look at?’ Later used keywords from docs in trails (and

bookmarks) to model users Can now search and present results with a

notion of context Through COHSE, will be able to navigate

concept space


25

Facilitator(s) and Agent Server(s)

The Web

UI Agent Web Browser

Memoir Agent Link Server Image DBase Wrapper 1 Wrapper n

Resource 1 Resource m


26

Open Hypermedia

Link database

documents

Note the direction of this arrow!

Separable hyperstructure


27

Hyperstructure as metadata

Open hypermedia introduces separable hyperstructure, e.g. as supported through XLink

Southampton model introduces reuseable separable hyperstructure, which can be applied to new documents

Agents used for link resolution Agents build link databases and maintain

them


28

A case for streaming metadata? Three kinds of multimedia streams:

Media on demand Live, one way Two way

Live metadata may be created by: Producer (e.g Big Brother) Video segmentation and classification Annotation

There are multiple simultaneous flows of data, from multiple sources

Metadata needs to come from upstream in production process!

Onward to ontologies


30

Agents using the Semantic Web Scenarios revisited:

Workshop scenario. Use ontology for our domain of interest. (And for IST?) Multimedia ontology for delivery of multimedia content.

Musician. Use ontology for navigating the musical information space. What about information about musical devices? Creation of metadata for new compositions.

Information grid. Ontologies for manufacturing and organisation. What about computational resources?

Also need to find agents, and to communicate with them

Hence working with multiple, distributed, ontologies.


31

SoFAR SoFAR (the Southampton Framework for Agent

Research) is a versatile multi-agent framework designed for Distributed Information Management tasks.

SoFAR embraces the notion of proactivity as the opportunistic reuse of the services provided by other agents, and provides the means to enable agents to locate suitable service providers.

SoFAR combines some ideas from the distributed computing community with the performative-based communications used in many agent systems: communications in SoFAR are based on the startpoint/endpoint paradigm, which is the foundation of Nexus, the communication layer at the heart of the Computational Grid.


32

Index of /distrib/sofar024/ontology

Name Last modified Size Description

Parent Directory 16-Nov-2000 22:45 -

actions/ 16-Nov-2000 22:45 -

base/ 16-Nov-2000 22:45 -

fohm/ 16-Nov-2000 22:45 -

infrastructure/ 16-Nov-2000 22:46 -

metadata/ 16-Nov-2000 22:45 -

multimedia/ 16-Nov-2000 22:46 -

system/ 16-Nov-2000 22:45 -

web/ 16-Nov-2000 22:45 –

Apache/1.3.9 Server at www.sofar.ecs.soton.ac.uk Port 80

http://www.sofar.ecs.soton.ac.uk/distrib/sofar024/ontology/?N=D

http://www.sofar.ecs.soton.ac.uk/distrib/sofar024/ontology/?M=A

http://www.sofar.ecs.soton.ac.uk/distrib/sofar024/ontology/?S=A

http://www.sofar.ecs.soton.ac.uk/distrib/sofar024/ontology/?D=A

http://www.sofar.ecs.soton.ac.uk/distrib/sofar024/


33

HyStream example Agents deal with multimedia streams ACL handles session control, synchronisation,

linking Publish-subscribe

Example predicates: ContainsContour(music, contour, time) Relay(multicast_address1, multicast_address2)

The mediadata-metadata distinction becomes blurred, e.g. when features extracted from multimedia documents


34

Agents supporting the Semantic Web Metadata, vocabularies, thesauri, ontologies

are ‘stuff’ in the information space Note distinction between:

Automation of tasks, i.e. computer-to-computer interaction (a goal of XML et al)

Ontology capture and design tools involving humans

Agents help with both; it’s the first that really helps agents (and is supported by current web technologies)


35

The (distributed) intelligence

Currently agents ‘wrap’ existing inference engines

Agent Based Computing is an appropriate paradigm to work in complex world with multiple ontologies, fragments, multiple inferencing engines

We anticipate further decomposition into multiple inferencing components


36

Future work at Southampton

Ontology support for agent collaboration and negotiation

Ontologies and hypermedia (COHSE, Pervasive)

Use of OIL/DAML Instantiating application-neutral ontologies

for agent infrastructure Agents supporting the knowledge lifecycle

Summary


38

Summary

The fabric of the Web is changing DIM agents eat metadata for breakfast XML and RDF(S) support agent-agent

interaction Agents not only use but also automate the

construction and maintenance of metadata and ontologies

[email protected]


39

Motivation

“Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and to coin one at random, “memex” will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory” – Bush, July 1945

“There will always be plenty of things to compute in the detailed affairs of millions of people doing complicated things”

Documents

Agents on the Semantic Web David De Roure Intelligence Agents Multimedia Dept of Electronics and Computer Science University of Southampton, UK [email protected]