Realizing a Semantic Web Application - ICWE 2010 Tutorial

Applied Semantic WebTimely. Practical. Reliable.http://applied-semantic-web.org

Realizing aSemantic Web ApplicationICWE 2010 TutorialJuly 5 - 9, 2010 in Vienna, Austria

Emanuele Della [email protected] - http://emanueledellavalle.org

Emanuele Della Valle - http://applied-semantic-web.org

Share, Remix, Reuse — Legally

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Your are free:

• to Share — to copy, distribute and transmit the work

• to Remix — to adapt the work

Under the following conditions

• Attribution — You must attribute the work by inserting– “© applied-semantic-web.org” at the end of each reused slide– a credits slide stating “These slides are partially based on

“Semantic Web An Introduction” by Emanuele Della Valle http://applied-semantic-web.org/slides/2010/03/01_intro.ppt

To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/

2


Acknowledgements

The presentation of this tutorial issupported by the Search Computing(SeCo) project, funded by European Research Council, under the IDEAS Advanced Grantsprogram.

Search computing • focuses on building the answers to complex search

queries like "Where can I attend an interesting conference in my field close to a sunny beach?"

• by interacting with a constellation of cooperating search services,

• using ranking and joining of results as the dominant factors for service composition.

3


Acknowledgements

We are working on using Search Computingto support the development of SemanticWeb applications like the one illustrated inthis tutorial

We believe that Search Computing methodsand tools will revolutionize the developmentof mash-ups using (but not limiting to) Linked Data

Learn more: http://www.search-computing.it/

http://blog.search-computing.it/

Stay tuned: http://twitter.com/searchcomputing

Get connect: http://www.facebook.com/pages/Search-Computing/10150092533150370

4


Introduction

Goal

We will “develop” (no panic ;-), no hands on!) together an application of the Semantic Web we named Music Event Explorer or simply meex

We will challenge the Semantic Web technologies in realizing a new service for Web users• Using• Transforming and• Combining existing data

5


Introduction

Ingredients

RDF as unified data model

OWL as modelling language for the data sources

GRDDL as a standard approach to translate in RDF the data stored in XML data sources

D2RQ as tool to translate in RDF the data stored in relational data sources

SPARQL as standard query language to access RDF data

Jena as application framework to merge the various data in a single RDF model and manipulate it

Joseky as tool to expose SPARQL endpoint

ARQ as SPARQL client library

A RDF storage to guarantee persistency

A OWL reasoner to infer new knowledge

Exhibit as user interface

6

Emanuele Della Valle - http://applied-semantic-web.org 7

Introduction

Approach

In order to realize meex 1. We start from the user need 2. We derive user requirements3. We “develop” the ontologies and the software components

While presenting we will explain the use of Semantic Web technologies and tools. (gray background slides)

A demonstrative installation of the application, together with the source code, is available at• http://swa.cefriel.it/meex


By the way what’s the Semantic Web

Computer should understand more

Large number of integrations - ad hoc - pair-wise

Too much information to browse, need for searching and mashing up automatically

Each site is “understandable” for us

Computers don’t “understand” much

?

Search & Mash-up Engine

010 0 1 1 0

01101

10100 10 0010 01 101 101 01 110 1 10 110 0 1 1 01 0 1 0 0 1 1 0 1 1 1 10 01 101 0 1

Millions of Applications



What does “understand” mean?

What we say to Web agents

" For more information visit <a href=“http://www.ex.org”> my company </a> Web site. . .”

What they “hear”

" blah blah blah blah blah <a href=“http://www.ex.org”> blah blah blah </a> blah blah. . .”

Jet this is enought to train them to achive tasks for us

[ source http://www.thefarside.com/ ]



What does Google “understand”?

Understanding that• [page1] links [page2] page2 is interesting

Google is able to rank results!• “The heart of our software is PageRank™, a system for

ranking web pages […] (that) relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value.”

http://www.google.com/technology/

10



Two ways for computer to “understand”

Smarter machines• Such as

– Natural Langue processing (NLP)– Audio Processing– Image Processing (IP)– Video Processing– … many many more

• They all work fine alone, the problem is combining them– E.g., NLP meets IP

- NLP: What does your eye see?- IP: I see a sea- NLP: You see a “c”?- IP: Yes, what else could it be?

• Not the Semantic Web approach

Smarter Data• Make data easier for machines to publish, share, find and

understand– E.g. http://wordnet.rkbexplorer.com/description/word-sea vs.

http://wordnet.rkbexplorer.com/description/word-c • The Semantic Web approach

Some NLP Related Entertainment http://www.cl.cam.ac.uk/Research/

NL/amusement.html

11



The Semantic Web 1/4

“The Semantic Web is not a separate Web, but an extension of the current one, in which information is given well-defined meaning, better enabling computers and people to work in cooperation.”

“The Semantic Web”, Scientific American Magazine, Maggio 2001 http://www.sciam.com/article.cfm?articleID=00048144-10D2-1C70-84A9809EC588EF21

Key concepts• an extension of the current Web• in which information is given well-defined meaning • better enabling computers and people to work in

cooperation.– Both for computers and people

12




“The Semantic Web […] , in which information is given well-defined meaning […]”

Human understandable but “only” machine-

readable

Human and machine

“understandable”

Web 1.0 Semantic Web

13




Semantic Web

Fewer Integration - standard - multi-lateral

[…] better enabling computers and

people to work in cooperation.

Even More Applications

Easier to understand for people

More “understandable” for computers

Semantic Mash-ups &Search


User Need

A user need for meex

Imagine the users need to explore music events related to a given music style• An event is a concert, a show or a workshop at which one or

more artist participate. • An artist is either a single musician or a band.

For instance, if a user is interest in Folk music meex• finds the artists that play Folk music • searches for events of those artists• allows the users to explore the events related to each artist

as a list, on a time line and on a map


User Need

A manual solution

1. I open musicmoz [1] and I look up artists that play Folk music

2. If the pages of the artists on musicmoz don’t satisfy me I navigate to musicbrainz [2]

3. I look up in EVDB [3] if some of those artists have organized an event close to my location in these days

4. I take note of the possible alternatives and I check how to get there using google maps [4]

[1] http://www.musicmoz.org

[2] http://www.musicbrainz.org

[3] http://www.eventful.com

[4] http://maps.google.com

16


User Need

A manual solution

1. I look up artists that play Folk music

17


User Need

A manual solution

2. I can learn more navigating to musicbrainz

18


User Need

A manual solution

3. I look up in EVDB if some of those artists have organized an event close to my location in these days

19


User Need

A manual solution

4. I take note of the possible alternatives and I check how to get there using google maps

20


User Need

Music Event Explorer

Of course I can do it manually, but I need the time to do so. Can’t I write a mash-up?

21


The Problem Space

What is needed?

Ivan Herman in introducing the Semantic Web explains

(Some) data should be available for machines for further processing

Data should be possibly combined, merged on a Web scale

Sometimes, data may describe other data (like the library example, using metadata)…

… but sometimes the data is to be exchanged by itself, like my calendar or my travel preferences

Machines may also need to reason about that data


The Problem Space

The rough structure of data integration

1. Map the various data onto an abstract data representation• make the data independent of its internal representation…

2. Merge the resulting representations

3. Start making queries on the whole!• queries that could not have been done on the individual data

sets


The Problem Space

The rough structure of data integration

24


The Problem Space

So where is the Semantic Web?

The Semantic Web provides technologies to make such integration possible! For example:• an abstract model for the relational graphs: RDF• extract RDF information from XML (eg, XHTML) pages:

GRDDL• add structured information to XHTML pages: RDFa• a query language adapted for the relational graphs: SPARQL• characterize the relationships, categorize resources: RDFS,

OWL, SKOS, Rules– applications may choose among the different technologies– some of them may be relatively simple with simple tools (RDFS),

whereas some require sophisticated systems (OWL, Rules)• reuse of existing “ontologies” that others have produced


The Problem Space

So where is the Semantic Web?

26


The Problem Space

“Global as a View” approach

Content Ontology 1

Content Ontology 2

Content Ontology N

Application Ontology

Bridge Ontology

Content Ontology 3

Bridge Ontology


Software Engineering for the Semantic Web

A Semantic Web application is still an application!

A Semantic Web application is still an application, thus we need to follow good practice from Software Engineering in developing it.

We adopt a Spiral Model inspired by the famous Boehm spiral model

We extend it with Knowledge Engineering practices


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy

R.1 Users’ needs analysis

R.3 Software requirements analysis

R.4 Content requirements analysis

D.3 Model sample

contents

Reuse

Merge

Extend

I.1 Implement theinitial Knowledge Base

V.1

V

alid

atio

n

I.3 Configure External Source Wrappers

I.2 Implement the integrated model

Reuse

Merge

Extend

I.4 Implement the application

R.2 Risk analysis

D.4 Design Application

T.1 Testing


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


Requirements Analysis

Application requirements analysis (1)

In this step (namely R.3) we should elicit• functional requirements of the application

– as grouping and filtering data• non-functional requirements of the application

– as performance and scalability w.r.t. number of users

However this is just a tutorial, therefore we concentrate on functional requirements, leaving non-functional requirements underspecified



Application requirements analysis (2)

Meex• must enable a user to explore data in the form of

– a list– a chronological graphic– a geographic map

• for each event must show– name– begin and end date– place

• for each artist must show– name– nationality– music styles he/she plays– related artists

• must allow users to– filter and rank results


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing



Content requirements analysis

Given we are developing a Semantic Web application is cruscial we reuse data already available on the Web• EVDB - http://eventful.com • MusicBrainz - http://musicbrainz.org• MusicMoz - http://musicmoz.org



EVDB Content Source

EVDB is a Web 2.0 website that makes available information about event all around the world

For each event it knows• The start data• The end data• The place in terms of address and geographic coordinates

EVDB offers a Web API in the form of a REST service that sends back XML<entry> <id>http://eventful.com/events/E0-001-020438140-1</id> <title>U2</title> <gd:when startTime="2009-07-07" endTime="2009-07-08" /> <gd:where> <gd:contactSection label="San Siro"> <gd:postalAddress>20151 Milan, Italy</gd:postalAddress> <gd:geoPt lat="45.4667" lon="9.2"/> </gd:contactSection> </gd:where</entry>

• see http://api.evdb.com



MusicBrainz Content Source

MusicBrainz • is a Web 2.0 website that gathered a large amount of

information about music• offers information about

– artists and bands– songs, albums and tracks– relations among artists and bands

The data of MusicBrainz are available as a PostgreSQL dump• see http://musicbrainz.org/doc/DatabaseDownload

It is exposed as a SPARQL endpoint by • see http://ontotext.com/factforge/



MusicMoz Content Source

MusicMoz • is another Web 2.0 website dedicated to music• offers information about

– artists and bands including their nationality– music styles and their taxonomic relationships– the styles each artist or band plays

• reuses MusicBrainz identifier for artists and bands

The data of MusicMoz are available as large XML files

<category name="Bands_and_Artists/U/U2/" type="band"> <resource name="musicbrainz" link="http://musicbrainz.org/artist/b10bbbfc-cf9e-42e0-be17-

e2c3e1d260d.html" /> <from>Ireland</from> <style number="1">Rock</style> <style number="2">Pop</style></category>

• see http://musicmoz.org/xml/



meex needs to merge this data

meex in order to be able to manipulate all this data at the same time needs to merge the data of the three data sources.

The artists and bands information from MusicBrainz should be linked to• the music styles they play from MusicMoz• the events related to them from EVDB



Data Licence Issue

The data of all three data sources are freely usable, we just need to make sure that the logos of the three applications appears on each page of meex

EVDB requests also to include a link to the permalink of the event on EVDB website

MusicBrainz request also that derived data are made available in Creative Commons.

Read out more here• EVDB - http://api.eventful.com/terms • MusicMoz - http://musicmoz.org/xml/license.html • MusicBrainz - http://musicbrainz.org/doc/DatabaseDownload


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


System Design

“Global as a View” approach instantiated

Music Moz Ontology

EVDB Ontology

meex Application Ontology

Bridge Ontology

Music Brainz Ontology

Bridge Ontology

MusicMoz MusicBrainz EVDB

[File XML] [DB] [REST+XML]

MUSIC EVENTS EXPLORER


System Design

The five ontologies that model meex

PerformerStyle EventperformsStyle

performsEvent

String When Where

relatedPerformer

ArtistStyle EventhasStyle

relatedArtist

fromCountry

from hasWhere

hasWhen

hasWhere

hasWhen

rdfs

mb evdbmm

gdxsd

meexsu

bPro

pert

yOf

subP

rope

rtyO

f

subP

rope

rtyO

f

subP

rope

rtyO

f

subP

rope

rtyO

f

subC

lass

Of

subC

lass

Of

subC

lass

Of


System Design

The five ontologies form a network

The properties hasStyle and from in MusicMoz ontology refer to the class Artist in MusicBrainz ontology

Both the meex application ontology and the EVDB ontology

refers to a shared ontology in which the class when and

where are described

The bridge ontology is a collection of subClassOf and

subPropertyOf statements • Further explanations follow when discussing the step I.2

(Implement the integrated model)


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


System Design

MusicBrainz Ontology Sample content

mb:artist/b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d a mb:Artist ;rdfs:label "The Beatles" ;mb:related_artist

mb:artist/ebfc1398-8d96-47e3-82c3-f782abcdb13d ,mb:artist/618b6900-0618-4f1e-b835-bccb17f84294 .

mb:artist/ebfc1398-8d96-47e3-82c3-f782abcdb13d a mb:Artist ;rdfs:label "The Beach Boys" .

mb:artist/618b6900-0618-4f1e-b835-bccb17f84294 a mb:Artist ;rdfs:label "Eric Clapton" .

45

String

When

Where


relatedArtist

from hasWhere

hasWhen

mb evdbmm

SampleInstance-MusicBrainz.n3


System Design

MusicMoz Ontology Sample content

mb:artist/b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d mm:from "England" ; mm:hasStyle mm:style/British-Invasion , mm:style/Rock , mm:style/Skiffle .

mm:style/British-Invasion a mm:Style ; rdfs:label "British Invasion" .

Note that we are reusing IRI from the MusicBrainz ontology

46

String

When

Where


relatedArtist

from hasWhere

hasWhen

mb evdbmm


System Design

EVDB Ontology Sample content

evdb:events/E0-001-008121669-0@2008022719 a evdb:Event ; gd:label "Tell Me Why: A Beatles Commentary" . evdb:hasWhen evdb:events/E0-001-008121669-0@2008022719_When; evdb:hasWhere evdb:events/E0-001-008121669-0@2008022719_Where.

evdb:events/E0-001-008121669-0@2008022719_When gd:startTime "2008-02-28" ; gd:endTime "2008-02-28" .

evdb:events/E0-001-008121669-0@2008022719_Where gd:hasGeoPt evdb:events/E0-001-008121669-0@2008022719_GeoPt ; gd:label "The Wilmington Memorial Library" ; gd:postalAddress "175 Middlesex Avenue, Wilmington, USA" .

evdb:events/E0-001-008121669-0@2008022719_GeoPt gd:lat "42.556943" ; gd:lon "-71.165576" .

47

String

When

Where


relatedArtist

from hasWhere

hasWhen

mb evdbmm


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


System Design

Wrap up of what we did so far

We are done with the modeling of ontologies and sample contents

We can now design meex (step D.4 of our approach)

In order to design meex architecture• We first design its interfaces in terms of

– both graphic user interface – and connection to the three data sources

• Secondly we design how it works inside in terms of – components and– execution semantics

49


System Design

meex interfaces

MusicBrainzdatabase

Adapter Database

RDF

SPARQLServer

EVDB REST service

MusicMoz File XML

meex

User

XML

Browser Web 3) HTML and RDF

2) RDF

GRDDL processor

EVDB RDF

MusicMoz RDF

XML

2) RDF

1) Music style


System Design

How we access the data

In order to get RDF data out from the three external data source we can use different techniques• For MusicBrainz database we can use tools that enable to

query non-RDF databases as virtual RDF graphs using a standard SPARQL endpoint

• For MusicMoz XML files we can use a GRDDL processor using the XSLT MusicMoz->RDF

• For EVDB we can use a GRDDL processor applying the XSLT EVDB->RDF to the XML file obtained using the EVDB REST service

51


System Design

User Interface

In order to collect users’ input and to present results back to the users, we can use Web 2.0 technologies and develop an AJAX interface

Such AJAX interface must allow for• Inserting the music style, the resulting events will refer to• Exploring the events found by meex• Filtering the events based on

– Artists– Their nationality– The music style they play

52


System Design

Designing how meex works insideAjax Web Framework

GRDDL Processor

For each Artist

SPARQL Client

MusicBrainz SPARQL Endpoint

HTTP REST Client

EVDB HTTP REST service

GRDDL ProcessorEVDB RDF

MusicMoz RDF

Linking Artists to Events

RDF Merge

Extraction and Transformation

Ajax Web Framework

Music style

Set of artist in RDF

Artist

SPARQL Query

Events in XML

Events in RDF

Artists and events in RDF

Artist datain RDF

HTTP Query

Dati RDF



System Design

Execution Semantics (1)

1. The user requests a music style

2. meex access the local copy of MusicMoz and using the GRDDL processors obtains a set of artist that plays the given music style

[more to follow]

54


System Design


[follows]

3. For each artist meex :a) uses the SPARQL client to query the MusicBrainz SPARQL

endpoint and it obtains the artist name and his/her relationships with other artist

b) invokes the EVDB REST service, it obtains the events that refer to the artist in XML and uses the GRDDL processor to obtain this data in RDF

c) links the data about each artist to the data about the events that refers to him/her

[more to follow]

55


System Design


[follows]

4. When all the peaces of information about artists and events are available in the RDF storage, meex extracts them and serializes them in the format of the Ajax Web framework

5. The ajax Web framework allows the user for exploring the events found by meex

6. When the user decides to start a new exploration, meex starts over from the beginning

56


System Design

Two important internal components

The RDF storage • must be initialized with both the application and the content

ontology• is filled in with the data meex loads from the three data

source given the music style requested by the user

The reasoner• allows all query in meex to be express in terms of the

application ontology even if data are loaded from the data sources using the content ontology

NOTE: the reasoner support the semantic integration of the data loaded from the external data sources. The meex’s programmer can ignore that multiple and heterogeneous data sources were used to load data

57


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


Development

Implement the initial Knowledge Base (1)

We start implementing meex by setting up the initial knowledge base (step I.1)

We need to select tools• to read and write RDF in the RDF/XML and RDF/N3 syntax• to manipulate programmatically RDF• to store RDF • to reason on OWL• to interpret SPARQL

59


Development

Implement the initial Knowledge Base (2)

We choose Jena because• offers API

– to read and write different RDF syntax– provides a programmatic environment for RDF, RDFS and OWL,

SPARQL a • guarantees RDF model persistence through several relational

database adapters• includes a rule-based inference engine which implement OWL

semantics• includes ARQ, a query engine that supports SPARQL

In order to use the RDF storage and the OWL reasoner from Jena we need to configure them as shown in the following slides

60


Development

Configuring the RDF storage

1. Class.forName("org.apache.derby.jdbc.EmbeddedDriver");

2. DBConnection con = new DBConnection( "jdbc:derby:C:/Meex/RDFStorage;create=true", "sa", "", "Derby");

3. Model model = ModelFactory.createModelRDBMaker(con). createDefaultModel();

We choose to use Derby (from Apache) as relational database underneath the RDF storage.

With row 1 we tell Jena where to find the JDBC driver

With row 2 we define the JDBC connection

With row 3 we instantiate the object model of Jena we will use to access and manipulate the RDF model in the storage


Development

Configuring the OWL reasoner

1. Reasoner reasoner = ReasonerRegistry.getOWLMicroReasoner();

2. model = ModelFactory.createInfModel(reasoner, model);

Jena offers numerous options to configure the internal rule-based inference engine with different expressivity-performance tradeoffs

We need simple reasoning features (i.e., subClassOf and subPropertyOf transitive closure), the OWL Micro configuration is, therefore, the most appropriate one

With row 1 we instantiate a OWL micro reasoner

With row 2 we instantiate a model with inference support using the model previously created and the OWL micro reasoner


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


Development

Implement the integrated model (1)

We move on with the implementation of meex realizing the integrated model (step I.2)

In the integrated model we merge application and content ontology• Our intent is to integrate semantically the heterogeneous

data coming from the external data sources

In order to realize the integrated model we need to define a bridge ontology using the properties• rdfs:subclassOf• rdfs:subpropertyOf

to connect classes and properties in the application ontology to those in the content ontology

64


Development


1. mb:Artist rdfs:subClassOf meex:Performer .

2. mb:related_artist rdfs:subPropertyOf meex:relatedPerformer.

3. mm:Style rdfs:subClassOf meex:Style .

4. mm:hasStyle rdfs:subPropertyOf meex:performsStyle .

5. mm:from rdfs:subPropertyOf meex:fromCountry .

6. evdb:Event rdfs:subClassOf meex:Event.

7. evdb:hasWhen rdfs:subPropertyOf meex:hasWhen.

8. evdb:hasWhere rdfs:subPropertyOf meex:hasWhere.

In rows 1 and 2 we connect the ontology of MusicBrainz to the application ontology, i.e.• the classes mb:Artist and meex:Performer• the properties mb:related_artist and meex:relatedPerformer.

Likewise, in rows 3, 4 and 5, we connect the ontology of MusicMoz to the application ontology and

in rows 6, 7 and 8 we connect the ontology of EVDB to the application ontology


Development

Back to the five ontologies that model meex

PerformerStyle EventperformsStyle

performsEvent

String When Where

relatedPerformer


relatedArtist

fromCountry

from hasWhere

hasWhen

hasWhere

hasWhen

mb evdbmm

gdxsd

meexsu

bP

rop

erty

Of

sub

Pro

per

tyO

f

sub

Pro

per

tyO

f

sub

Pro

per

tyO

f

sub

Pro

per

tyO

f

sub

Cla

ssO

f

sub

Cla

ssO

f

sub

Cla

ssO

f


Development


Thanks to this bridge ontology (and the reasoner), when a client application issues a SPARQL query using terms in the meex application ontology, it gets as results the information loaded from the external data sources

meex GUI can, therefore, query the RDF storage homogeneously in the terms of application ontology without caring of the heterogeneous formats of the three data sources

To give an idea of the differences, in the next slide we show• the data loaded from MusicBrainz• the SPARQL query that meex GUI issue in the application

ontology to the RDF storage• the answer it gets

67


Development


mb:artist/b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d a mb:Artist ;rdfs:label "The Beatles" ;mb:related_artist

mb:artist/ebfc1398-8d96-47e3-82c3-f782abcdb13d ,mb:artist/618b6900-0618-4f1e-b835-bccb17f84294 .

mb:artist/ebfc1398-8d96-47e3-82c3-f782abcdb13d a mb:Artist ;rdfs:label "The Beach Boys" .

mb:artist/618b6900-0618-4f1e-b835-bccb17f84294 a mb:Artist ;rdfs:label "Eric Clapton" .

68

SampleInstance-MusicBrainz.n3

SELECT ?artistName ?relatedArtistNameWHERE { ?x a meex:Performer . ?x rdfs:label ?artistName . ?x meex:relatedPerformer ?relatedArtist . ?relatedArtist rdfs:label ?relatedArtistName .

?artistName ?relatedArtistName

The Beatles The Beach Boys

The Beatles Eric Clapton


D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


Development

Configure External Source Wrappers

Following the proposed approach, next step (i.e. I.3) suggests to configure the external source wrappers

In the following slide we show how to implement and configure all the component necessary to allow meex to load data from the external data sources

70


Development

meex interfaces (1)

MusicBrainzdatabase

Adapter Database

RDF

SPARQLServer

EVDB REST service

MusicMoz File XML

meex

User

XML


2) RDF

GRDDL processor

EVDB RDF

MusicMoz RDF

XML

2) RDF

1) Music style


By the way, is this practice “orthodox”?

Semantic Web Double Bus

[source http://www.w3.org/DesignIssues/diagrams/sw-double-bus.png ]


Development

Importing data from MusicBrainz

The data of MusicBrainz are stored as dump of PostgreSQL database

So, first of all we install the relational database PostgreSQL• necessary documentation is available on PostgreSQL and

MusicBrainz official websites

When the database is available we need to install and configure 1. a translator from relational database to RDF2. a SPARQL endpoint

We choose D2RQ as translator and Joseki as SPARQL server

73


Development

Configuring D2RQ for MusicBrainz

map:artist a d2rq:ClassMap; d2rq:dataStorage map:database; d2rq:class mb:Artist; d2rq:uriPattern "http://musicbrainz.org/artist/@@artist.gid@@";

map:artist_name a d2rq:PropertyBridge; d2rq:belongsToClassMap map:artist; d2rq:property rdfs:label; d2rq:column "artist.name".

map:artist_relation a d2rq:PropertyBridge; d2rq:belongsToClassMap map:artist; d2rq:property mb:artist_relation; d2rq:join "artist.id = artist_relation.artist“; d2rq:join "artist_relation.ref = artist2.id"; d2rq:alias "artist AS artist2"; d2rq:uriPattern "http://musicbrainz.org/artist/@@artist2.gid@@".

74

D2RQ-MusicBrainzDB.n3

artist artist_relation

id gid

artist

ref


Development

Configuring Joseky for MusicBrainz

1.[] rdf:type joseki:Service ; rdfs:label "SPARQL for MusicBrainzDB" ; joseki:serviceRef "MusicBrainz" ; joseki:dataset _:MusicBrainzDS ; joseki:processor joseki:ProcessorSPARQL_FixedDS .

2._:MusicBrainzDS rdf:type ja:RDFDataset ; ja:defaultGraph _:MusicBrainzModel ; rdfs:label "MusicBrainz Dataset" .

3._:MusicBrainzModel rdf:type d2rq:D2RQModel ; rdfs:label "MusicBrainz D2RQ Model" ; d2rq:mappingFile <file:D2RQ-MusicBrainzDB.n3> ; d2rq:resourceBaseURI <http://musicbrainz.org/> .

75

joseki-config.ttl With row 1 we expose a SPARQL endpoint giving the

name of the service and the URL at which it will become accessible http://localhost:2020/MusicBrainz

With row 2 and 3 we configure the SPARQL endpoint to expose MusicBrainz via D2RQ using the configuration fileD2RQ-MusicBrainzDB.n3 (see previous slide)


Development

meex interfaces (2)

MusicBrainzdatabase

Adapter Database

RDF

SPARQLServer

EVDB REST service

MusicMoz File XML

meex

User

XML


2) RDF

GRDDL processor

EVDB RDF

MusicMoz RDF

XML

2) RDF

1) Music style


Development

Importing data from MusicMoz and EVDB

The MasicBrainz SPARQL endpoint is ready, let’s imporing data from MusicMoz and EVDB. They both exchange data in XML.

In the design steps we chose to use a GRDDL processor to convert from XML in RDF (in the RDF/XML syntax)

The GRDDL recommendation requires the XML documents to directly refer to the XSLT that performs the translation. • Neither MusicMoz nor EVDB XML files originally include the

reference request by GRDDL• We can programmatically add it

– In the following slide we show an excerpt of the modified XML files for MusicMoz

We can proceed likewise for EVDB

77


Development

Importing data from MusicMoz (1)

<musicmoz xmlns:grddl='http://www.w3.org/2003/g/data-view#‘ grddl:transformation="file:///[...]/musicmoz-to-rdf.xsl">

<category name="Bands_and_Artists/B/Beatles,_The“ type="band">

<resource name="musicbrainz" link="http://musicbrainz.org/artist/ b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d.html"/>

<from>England</from> <style number="1">British Invasion</style> <style number="2">Rock</style> <style number="3">Skiffle</style> </category>

<style><name>British Invasion</name></style> <style><name>Rock</name></style> <style><name>Skiffle</name></style></musicmoz>

78

Excerpts from the files musicmoz.bandsandartists.xml and musicmoz.lists.styles.xml


Development

Importing data from MusicMoz (2)<xsl:template match="musicmoz/category[(@type='band' or

@type='artist‘) and resource/@name='musicbrainz']"> <xsl:variable name="artist_uri“

select="resource[@name='musicbrainz']/@link"/> <xsl:for-each select="style"> <xsl:variable name="style_reformatted“

select="concat('http://musicmoz.org/style/',text())"/> <rdf:Description rdf:about="{$artist_uri}">

<mm:hasStyle rdf:resource="{$style_reformatted}"/> </rdf:Description> </xsl:for-each> <rdf:Description rdf:about="{$artist_uri}"> <mm:from><xsl:value-of select="from"/></mm:from> </rdf:Description></xsl:template><xsl:template match="musicmoz/style"> <xsl:variable name="style_reformatted"

select="concat('http://musicmoz.org/style/', name)"/> <mm:Style rdf:about="{$style_reformatted}"> <rdfs:label><xsl:value-of select="name"/></rdfs:label> </mm:Style></xsl:template>

79

Excerpts from the file musicmoz-to-rdf.xsl


Development

Importing annotations from MusicMoz (3) As GRDDL processor we choose GRDDL Reader, the GRDDL

processor for Jena.

With row 1 we instantiate a Jena model that will momentarily contain the RDF data produce by the GRDDL processor

With row 2 we instantiate a RDFReader that uses a GRDDL processor to load RDF data

With row 3 and 4 we load in the RDF model instantiate in row 1 the data contained in the XML files of MusicMoz using the RDF reader configured for GRDDL

With row 5 we merge the loaded RDF data with those already present in the RDF storage

80

1. Model mmModel = ModelFactory.createDefaultModel();

2. RDFReader reader = mmModel.getReader("GRDDL");

3. reader.read(mmModel, "file:///.../musicmoz.bandsandartists.xml");

4. reader.read(mmModel, "file:///.../musicmoz.lists.styles.xml");

5. model.add(mmModel);


Development

So far so good! (1)

MusicBrainzdatabase

Adapter Database

RDF

SPARQLServer

EVDB REST service

MusicMoz File XML

meex

User

XML


2) RDF

GRDDL processor

EVDB RDF

MusicMoz RDF

XML

2) RDF

1) Music style


Development

So far so good! (2)

Ajax Web Framework

GRDDL Processor

For each Artist

SPARQL Client


HTTP REST Client



MusicMoz RDF

Linking Artists to events

RDF Merge

Estrazione etrasformazione

Ajax Web Framework

Music style


Artist

SPARQL Query

Events in XML

Events in RDF


Artist datain RDF

HTTP Query

Dati RDF



D.1

Mod

el t

he

appl

icat

ion

onto

logy

D.2

Mod

el t

he

cont

ent

onto

logy




D.3 Model sample

contents

Reuse

Merge

Extend


V.1

V

alid

atio

n



Reuse

Merge

Extend


R.2 Risk analysis


T.1 Testing


Development

What’s left?

All the business logic that coordinates the interaction among the internal component is still to be implemented

NOTE: implementing the business logic requires • both writing many lines of pure Java code • and work with several Semantic Web technologies

we will focus our attention to the Semantic Web technologies

The complete Java code is available on meex the website for downloading.• See http://swa.cefriel.it/meex

84


Development

What’s left?

Ajax Web Framework

GRDDL Processor

For each Artist

SPARQL Client


HTTP REST Client



MusicMoz RDF

Linking Artists to events

RDF Merge

Estrazione etrasformazione

Ajax Web Framework

Music style


Artist

SPARQL Query

Events in XML

Events in RDF


Artist datain RDF

HTTP Query

Dati RDF



Development

MEMO: Execution Semantics (1)

1. The user requests a music style

2. meex access the local copy of MusicMoz and using the GRDDL processors obtains a set of artist that plays the given music style

[more to follow]

86


Development

Step 2: from the music style to the artists

The step 2. of meex execution semantics requires to query MusicMoz for the artist that plays the music style requested by the users

The following SPARQL query extracts information from MusicMoz in terms of the application ontology

87

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

PREFIX meex: <http://swa.cefriel.it/meex#>

SELECT DISTINCT ?performer

WHERE {

?performer meex:performsStyle ?style.

?style rdfs:label "British Invasion" .

}


Development


[follows]

3. For each artist meex :a) uses the SPARQL client to query the MusicBrainz SPARQL

endpoint and it obtains the artist name and his/her relationships with other artist

b) invokes the EVDB REST service, it obtains the events that refer to the artist in XML and uses the GRDDL processor to obtain this data in RDF

c) links the data about each artist to the data about the events that refers to him/her

[more to follow]

88


Development

Step 3.a: querying MusicBrainz

The step 3.a of meex execution semantics requires to query MusicBrainz for the data that describe an artist including the related artists

89

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

PREFIX mb: <http://musicbrainz.org/>

DESCRIBE <mb:artist/b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d>;

Excerpts from the file MusicBrainz.java


Development

Step 3.b: querying EVDB

The step 3.b of meex execution semantics requires to invoke the EVDB REST service, obtain the list of events in XML and use the GRDDL processor to obtain the RDF

90

Excerpts from the file EVDB.java


Development

Step 3.c: linking artists to events (1) The step 3.c of meex execution semantics requires to link

the artist information retrieved from MusicMoz and MusicBrainz to the event information retrieved from EVDB

We can use the following SPARQL CONSTRUCT query on top of the RDF generated by EVDB wrapper to create the links

91

PREFIX meex: <http://swa.cefriel.it/meex#>

CONSTRUCT {?performer meex:performsEvent ?event.}

WHERE { ?performer a meex:Performer .

?event a meex:Event . }


Development

Step 3.c: linking artists to events (2)

Note that the links we create are not “semantically” checked, we only encode in RDF the results of a keyword matching performed by EVDB search engine• E.g., if the event “Tell Me Why: A Beatles Commentary” is

returned by EVDB when asking for Beatles, such an event will be linked to Beatles :-/

But, the available data can be used to create smart filter• For example (very simple one, indeed)

– MusicBrainz contains information about death of artists or band broke up

– we can put a filter in front of the EVDB wrapper invoker to avoid looking for events of broken bands or dead artists

92


Development


[follows]

4. When all the peaces of information about artists and events are available in the RDF storage, meex extracts them and serializes them in the format of the Ajax Web framework

5. The ajax Web framework allows the user for exploring the events found by meex

6. When the user decides to start a new exploration, meex starts over from the beginning

93


Development

Step 4: preparing the data for the GUI

We choose Exhibit as Ajax Web framework because• allows facet browsing• allows grouping and filtering events by

– artist name– artist nationality– the style the artist plays– the related artists

• includes different views– an ordered list– a chronological graph– a geographic map

94


Development

Step 4: configuring Exhibit

We can configure Exhibit by the means of two files: • an HTML page that controls the look and feel and• a JSON file that contains the data to be explored by the user

In this tutorial we focus on the preparation of the JSON file. We refer to Exhibit documentation and the website of our Semantic Web book for the preparation of the HTML page of Exhibit for meex

A JSON file is a simple text file that contains data organized in set of recors. In the following slide we show the information of The Beatles expressed in JSON.

95


Development

Step 4: a sample JSON file

96

1. type: "Event",

2. label: "1964 The Tribute Tribute to Beatles",

3. eventful_link: "http://eventful.com/events/ E0-001-006129372-5",

4. when_startTime: "2008-01-25",

5. when_endTime: "2008-01-26",

6. where_label: "Paramount Theater",

7. where_address: "17 South Street, New York 10940, United States",

8. where_latlng: "41.4544,-74.471",

9. performer_label: "The Beatles",

10.fromCountry: "England",

11.styles: ["Skiffle", "British Invasion", "Rock"],

12.relatedPerformers:["The Beach Boys", "Eric Clapton"]


Development

Step 4: serializing RDF in JSON

In order to serialize RDF in JSON• we extract the information we loaded in the RDF storage

using the SPARQL query shown in the following slide• we serialize the result in JSON

NOTE: as we’ve already said several time, the query can be expressed in terms of the application ontology even if the data were loaded in other heterogeneous formats

97


Development

Step 4: extracting the data

98

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX meex: <http://swa.cefriel.it/meex#>PREFIX gd: <http://schemas.google.com/g/2005>SELECT DISTINCT ?event ?event_label ?when_startTime ?

when_endTime ?where_label ?where_address ?where_lat ?where_lon ?performer ?performer_label ?fromCountry

WHERE {?event rdfs:label ?event_label; meex:hasWhen ?when; meex:hasWhere ?where.?when gd:startTime ?when_startTime; gd:endTime ?when_endTime.?where gd:label ?where_label; gd:postalAddress ?where_address; gd:hasGeoPt ?geoPt.?geoPt gd:lat ?where_lat; gd:lon ?where_lon.?performer meex:performsEvent ?event; rdfs:label ?performer_label; meex:fromCountry ?fromCountry.}


Development

Step 5 and 6


Development

Step 5 and 6


Any (further) Question?


Where to find this presentation

This presentation is available for download and reuse

from the Search Computing blog

Check it out!

http://blog.search-computing.net/2010/07/rswa2010/


MAJOR CREDITS

Dario Cerizza [[email protected]]• who help in conceiving, designed and developed meex

Irene Celino [[email protected] - http://iricelino.org/]• who help in conceiving meex and supported the design and

development of meex

These slides are an evolution of• Emanuele Della Valle Dario Cerizza, Irene Celino.

Realizing a Semantic Web Application. Tutorial Presented at 7th Int. Semantic Web Conference Karlsruhe, Germany, October 26, 2008

103


More credits

The development of meex have been supported by CEFRIEL Semantic Web Activities• To learn more visit http://swa.cefriel.it

The gray slides of this tutorial are largely based on • Ivan Herman. An Introduction to the Semantic Web (Through

an Example…). 2010-06-04. Available online at http://www.w3.org/People/Ivan/CorePresentations/IntroThroughExample/

104


Tools employed (1)

Jena• Application Framework• http://jena.sourceforge.net

Derby• Relational database for the RDF storage• http://db.apache.org/derby

PostgreSQL• Relational database for MusicBrainz• http://www.postgresql.org

D2RQ• Translator from relational database to RDF• http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2rq

105


Tools employed (2)

Joseki• SPARQL Endpoint Server • http://www.joseki.org

ARQ• SPARQL query engine for Jena• http://jena.sourceforge.net/ARQ

GRDDL Reader• GRDDL processor• http://jena.sourceforge.net/grddl

Exhibit• Ajax Web Framework • http://static.simile.mit.edu/exhibit

106


Resources

RDF

RDF at the W3C - primer and specifications• http://www.w3.org/RDF/

Semantic Web tools - community maintained list; includes triple store, programming environments, tool sets, and more• http://esw.w3.org/topic/SemanticWebTools

302 Semantic Web Videos and Podcasts - includes a section specifically on RDF videos• http://www.semanticfocus.com/blog/entry/title/302-semantic-

web-videos-and-podcasts/

107


Resources

SPARQL

SPARQL Frequently Asked Questions• http://thefigtrees.net/lee/sw/sparql-faq

SPARQL implementations - community maintained list of open-source and commercial SPARQL engines• http://esw.w3.org/topic/SparqlImplementations

Public SPARQL endpoints - community maintained list• http://esw.w3.org/topic/SparqlEndpoints

SPARQL extensions - collection of SPARQL extensions implemented in various SPARQL engines• http://esw.w3.org/topic/SPARQL/Extensions

108


Resources

RDF-S/OWL

OWL Frequently Asked Questions• http://www.w3.org/2003/08/owlfaq.html

RDF-S/OWL implementations - community maintained list of open-source and commercial SPARQL engines• http://esw.w3.org/topic/SemanticWebTools#head-

d07454b4f0d51f5e9d878822d911d0bfea9dcdfd

RDF-S Specification• http://www.w3.org/TR/rdf-schema/

OWL Working Group Wiki• http://www.w3.org/2007/OWL/wiki

109


Advertisement: if you speak Italian …

Technology

Realizing a Semantic Web Application - ICWE 2010 Tutorial