Visualizing Open Data with Plonea practical guide on how to query and visualize Linked Open Data

with eea.daviz product

Antonio De MarinisWeb Technology ManagementEuropean Environment Agency

www.eea.europa.eu

Linked Data evolution

as per 2011

2007

keeps growing...> 1 million datasetsWatch video STRATA conference 2013

Open Data - what is it?

Open data is a philosophy and practice requiring that certain data be freely available to everyone, without restrictions from copyright, patents or other mechanisms of control.

Linked Open Data (LOD) or simply Linked Data is a technique to interlink all open datasets into a web of data, aka semantic web, using technologies like RDF and SPARQL.

Linked Data vs classic ODBC

SPARQL query structure

A SPARQL query comprises, in order:● Prefix declarations, for abbreviating URIs● Dataset definition, stating what RDF graph(s) are being queried● A result clause, identifying what information to return from the query● The query pattern, specifying what to query for in the underlying dataset● Query modifiers, slicing, ordering, and otherwise rearranging query results# prefix declarationsPREFIX foo: <http://example.com/resources/>...# dataset definitionFROM ...# result clauseSELECT ...# query patternWHERE { ...}# query modifiersORDER BY ...

Real example querying DBpedia

SELECT * WHERE {?subject rdf:type <http://dbpedia.org/ontology/City>.?subject rdfs:label ?label.?subject rdfs:comment ?abstract.?subject <http://dbpedia.org/ontology/populationTotal> ?populationTotal.FILTER (lang(?label) = "en" && lang(?abstract) = "en")} LIMIT 5

Let's dive into a real example

SELECT * WHERE {?subject rdf:type <http://dbpedia.org/ontology/City>.?subject rdfs:label ?label.?subject rdfs:comment ?abstract.?subject <http://dbpedia.org/ontology/populationTotal> ?populationTotal.FILTER (lang(?label) = "en" && lang(?abstract) = "en" && (?populationTotal >= "5000000"^^xsd:integer))} LIMIT 5

Let's dive into a real example

PREFIX o: <http://dbpedia.org/ontology/>PREFIX p: <http://dbpedia.org/property/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

SELECT DISTINCT * WHERE {?subject a o:City.?subject rdfs:label ?label.OPTIONAL {?subject rdfs:comment ?abstract.}?subject p:populationTotal ?populationTotal.OPTIONAL {?subject geo:lat ?latitude.}OPTIONAL {?subject geo:long ?longitude.}FILTER (lang(?label) = "en" && lang(?abstract) = "en" && (?populationTotal >= "5000000"^^xsd:integer && ?populationTotal < "60000000"^^xsd:integer))} ORDER BY DESC(?populationTotal)

find all properties by exploring dbpedia e.g. dbpedia http://dbpedia.org/page/Tokyo

Example without duplicates http://daviz.eionet.europa.eu/data/local-sparql-queries/most-populated-cities

Corresponding data visualisation with Daviz

We have been able to create a data visualisation of open linked data with filters/facets entirely through the web in about 10 minutes!

live demo http://www.eea.europa.eu/sandbox/plog2013/most-populated-cities-with-coordinates-plus

Removing redundanciesTIP: In order to get rid of some rednundancy you can use "SAMPLE" or "SELECT DISTINCT"

PREFIX o: <http://dbpedia.org/ontology/>PREFIX p: <http://dbpedia.org/property/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

SELECT ?subject (sql:SAMPLE(?subject) as ?city)(sql:SAMPLE(?label) as ?label) (sql:SAMPLE(?latitude) as ?latitude) (sql:SAMPLE(?longitude) as ?longitude) max(?populationTotal) as ?maxPopulation max(?rainyDays) as ?rainyDays

WHERE {?subject a o:City.?subject rdfs:label ?label.OPTIONAL {?subject rdfs:comment ?abstract.}?subject p:populationTotal ?populationTotal.OPTIONAL {?subject geo:lat ?latitude.}OPTIONAL {?subject geo:long ?longitude.}OPTIONAL {?subject p:yearPrecipitationDays ?rainyDays.}FILTER (lang(?label) = "en" && lang(?abstract) = "en" && (?populationTotal >= "5000000"^^xsd:integer))}GROUP BY ?subjectORDER BY DESC(?maxPopulation)

live example http://daviz.eionet.europa.eu/visualisations/most-populated-cities

More examples at Daviz show room daviz.eionet.europa.eu

Example 2: Large companies (DBpedia)

PREFIX o: <http://dbpedia.org/ontology/>PREFIX p: <http://dbpedia.org/property/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

SELECT * WHERE {?subject rdf:type <http://dbpedia.org/ontology/Company>.?subject rdfs:label ?label.?subject rdfs:comment ?abstract.?subject p:numEmployees ?employees.?subject o:location ?location.?location geo:lat ?latitude.?location geo:long ?longitude.FILTER (lang(?label) = "en" && lang(?abstract) = "en" && ?employees > 10000)} ORDER BY DESC(?employees)LIMIT 20

Example 3: Energy plants (Enipedia)

SPARQL Endpoint:http://enipedia.tudelft.nl/sparql

BASE <http://enipedia.tudelft.nl/wiki/>PREFIX a: <http://enipedia.tudelft.nl/wiki/>PREFIX prop: <http://enipedia.tudelft.nl/wiki/Property:>PREFIX cat: <http://enipedia.tudelft.nl/wiki/Category:>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>select ?Name ?Point ?Generation_capacity where {?powerPlant prop:Country a:Italy .?powerPlant rdfs:label ?Name .?powerPlant prop:Point ?Point .?powerPlant prop:Generation_capacity_electrical_MW ?Generation_capacity . }

More resources

● SPARQL endpoints and their status: http://labs.mondeca.com/sparqlEndpointsStatus/index.html

● SPARQL tutorial by example: http://www.cambridgesemantics.com/semantic-university/sparql-by-example

● eea.sparql package gives you a sparql client and data holder for plone available on pypi

● eea.daviz bundle includes eea.sparql and the visualisations tools

Data table manipulation via drag and drop

Modular framework

EEA Daviz

And much more...

EEA Daviz

Live Demo

EEA Daviz - Resources

More live examples

○ Eionethttp://daviz.eionet.europa.eu

○ EEAhttp://www.eea.europa.eu/data-and-maps/daviz