sparqlPuSH: Proactive notification of data updates in RDF stores using PubSubHubbub

Copyright 2009 Digital Enterprise Research Institute. All rights reserved.

Digital Enterprise Research Institute www.deri.ie

sparqlPuSH: Proactive notification of data updates in RDF stores using

PubSubHubbub

Alexandre Passant(1) and Pablo Mendes(2)

(1) DERI, NUI Galway (2) Kno.e.sis, WSU, Dayton OH

SFSW2010 @ ESWC201031st of May 2010, Heraklion, Crete


Motivation

Real-time Semantic Web LOD wrappers for Social Data

Sensor data and Semantics

Ubiquitous computing and the Semantic Web

What’s new in a RDF store ? From “Pull” to “Push” From regularly querying the store to identify and retrieve new

data

To having RDF store notifying interested parties that new content is available, in real-time

2 of XYZ


Motivation

Use-case Select user-generated content mentioning companies that

compete with mine

3 of XYZ


sparqlPuSH

sparqlPuSH Combining SPARQL, SPARQL Update and PubSubHubbub for

proactive notifications of changes in RDF stores

An interface that can be plugged on the top of any RDF store - http://code.google.com/p/sparqlpush/

Based on existing standards SPARQL and SPARQL Update to register feeds and launch

actions when content of the RDF store changes

Atom and RSS to get feeds of related changes

PubSubHubbub for broadcasting changes (beneficing from public hubs such as Google’s one)

4 of XYZ

http://code.google.com/p/sparqlpush/


sparqlPuSH

An approach in two steps Query registration

Change notification

Both are independent of the RDF store implementation Registration can be done remotely, simply sending an HTTP

request to a sparqlPuSH interface

Notification is triggered as soon as new data in loaded in the store, using SPARQL Update through sparqlPuSH

Clients should be compliant with the PuSH protocol and be able to interpret the rel=“hub” link in the feed header

5 of XYZ


Query registration

6 of XYZ


Query registration

7 of XYZ


Registering new user-generated content

8 of XYZ


Registering changes about resource X

9 of XYZ


Conventions in query registration

Using conventions to get a well-formatted Atom / RSS feed Easier to read in standard news aggregators

Mandatory elements ?uri - their URI of the element(s) to be retrieved;

?date - their creation / modification date;

Optional elements ?label - their label

?author - their author

10 of XYZ


Browsing available feeds

sprqlPuSH interface lists available feeds Including timestamp of last update

Plus ability to create feeds from the interface

11 of XYZ


Notification

12 of XYZ


Notification

SPARQL Update support Data is pushed to sparqlPuSH (HTTP POST) and relayed to the

underlying RDF store

Identification of relevant changes Applying all registered queries to the updated dataset

Changes broadcast Using PubSubHubbub for real-time and scalable notification

13 of XYZ


Implementation

Source code (PHP) http://code.google.com/p/sparqlpush/ (BSD license)

Server Connection to any SPARQL endpoint

Additional connector for ARC2 using the ARC2 API

Generating RSS or Atom feeds

Demo client Registering / unregistering queries to remote sparqlPuSH

interfaces

Receiving updates from registered feeds

14 of XYZ



Demo

Live demo @ SFSW2010

http://vimeo.com/11023983 Introduction to sparqlPuSH

http://vimeo.com/12166777 Advanced querying including DBpedia data

15 of XYZ

http://vimeo.com/11023983



Future Work

Registering queries over stream data E.g. C-SPARQL

Combination with other approaches for real-time semantic data management E.g. dady, Semantic pingback, etc.

Implementation Implemented as a default feature of RDF stores ?

TWARQL http://wiki.knoesis.org/index.php/Twarql

Check upcoming WI paper “Linked Open Social Signals”

16 of XYZ

http://wiki.knoesis.org/index.php/Twarql


Thank you !

Source code http://code.google.com/p/sparqlpush/ (BSD license)

Videos http://vimeo.com/11023983 (introduction)

http://vimeo.com/12166777 (advanced querying)

Contacts [email protected] / @terraces

[email protected] / @pablomendes

17 of XYZ




mailto:[email protected]

mailto:[email protected]

Technology

sparqlPuSH: Proactive notification of data updates in RDF stores using PubSubHubbub