Programming with LOD

Fumihiro Kato

Research Organization of Information and SystemsTransdisciplinary Research Integration Center

National Institute of Informatics

1

Linked Data Principles

1. Use URIs as names for things

2. Use HTTP URIs so that people can look up those names.

3. When someone looks up a URI, provide useful information, using the standards (RDF, SPARQL)

4. Include links to other URIs, so that they can discover more things.

2

http://dbpedia.org/resource/South_Korea3

1. request /resource/South_Korea

2. return an HTML document for /resource/South_Korea

http://dbpedia.orgWeb Browser

4

1. request /resource/South_Korea

2. return an RDF document for /resource/South_Korea

http://dbpedia.orgLOD Agent@prefix dbpedia-owl: <http://dbpedia.org/ontology/> .@prefix dbpedia: <http://dbpedia.org/resource/> .dbpedia:Seoul dbpedia-owl:country dbpedia:South_Korea .dbpedia:JnC dbpedia-owl:hometown dbpedia:South_Korea .@prefix dbpprop: <http://dbpedia.org/property/> .dbpedia:JnC dbpprop:origin dbpedia:South_Korea .dbpedia:Lee_Jae_Jin dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea .dbpedia:Kang_Sung_Hoon dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea .dbpedia:Northern_Asia-Pacific_Division_of_Seventh-day_Adventists dbpedia-owl:location dbpedia:South_Korea ; dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea .dbpedia:Korean_Amateur_Radio_League dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:regionServed dbpedia:South_Korea .dbpedia:Asia-Pacific_Association_for_International_Education dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea .dbpedia:The_Korean_Chemical_Society dbpedia-owl:location dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea .dbpedia:Korean_e-Sports_Association dbpedia-owl:regionServed dbpedia:South_Korea .dbpedia:Miss_Asia_Pacific_World dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea .dbpedia:Daegu_Stadium dbpedia-owl:location dbpedia:South_Korea .dbpedia:Chuncheon_Civic_Stadium dbpedia-owl:location dbpedia:South_Korea .

5

RDF Libraries

• Java: Jena, Sesame

• C: Redland

• C#: dotNetRDF

• Ruby: RDF.rb

• PHP: ARC2, Graphite

• Python: rdflib

• Perl: PerlRDF

• JavaScript: rdfQuery, rdfstore-js

• ....

See http://www.w3.org/2001/sw/wiki/Tools

6

RDF.rb

• http://rdf.rubyforge.org

• http://github.com/ruby-rdf

• $ gem install linkeddata equivalent-xml

7

Reading RDF

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end10 end

8

Reading RDF

.......#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c59c28(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/abstract> Corea del Sud, of icialment la República de Corea) és un país de l'Àsia que ocupa la regió meridional de la península de Corea. Al nord, limita amb Corea del Nord, al qual estava unit fins el 1945. A l'oest, creuant la mar Groga, es troba la República Popular de Xina, i a l'est, creuant l'Estret de Corea, es troba el Japó. La capital i la ciutat més gran n'és Seül, l'àrea metropolitana de la qual és una de les tres més poblades del món. La llengua oficial de Corea del Sud és el coreà, i és, actualment, la desena llengua més parlada del món. El nom del país, en coreà, significa "La Gran Nació del Poble Coreà", hom es refereix al país sovint com la "Terra de la Calma del Matí". llegendari Dangun Kanggeom. L'evidència arqueològica suggereix que els coreans han habitat la península des del Paleolític Baix. Després de la unificació dels Tres Regnes Coreans sota el comandament de Silla el 668 dC, Corea va romandre com a nació unificada sota les dinasties de Goryeao i Joseaon fins la caiguda de l'Imperi Coreà el 1910. Després de la divisió i de l'establiment de la república moderna de Corea del Sud, el país ha tingut que reconstruir-se després del període d'explotació japonesa i de la Guerra de Corea. No obstant això, l'estàndard de vida, així com l'economia, van créixer de manera sorprenent des de la dècada de 1980. El 1987, després de diverses manifestacions nacionals, el govern autoritari es va col·lapsar, i des de llavors, s'hi han celebrat eleccions democràtiques cada cinc anys. Avui dia, Corea del Sur és un dels Estats pioners de la democràcia a Àsia; els ciutadans coreans gaudeixen d'un dels nivells més alts de......

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end10 end

9

Accessing statements

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/Korea") do |reader| 7 reader.each do |statement| 8 puts "s: #{statement.subject}, p: #{statement.predicate}, o: #{statement.object}" 9 end10 end

10

RDF Serialization

• N-Triples

• RDF/XML

• Turtle (N3)

• TriX

• RDFa

• JSON-LD

• RDF/JSON

• ...

11

Reading text/n3

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/data/South_Korea.n3", :format => "n3") do |reader| 7 reader.each do |statement| 8 puts statement.inspect! 9 end10 end

12

Writing RDF

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Writer.open("writing.ttl") do |writer| 7 subject = RDF::URI("http://dbpedia.org/resource/South_Korea") 8 predicate = RDF::RDFS.label 9 object = RDF::Literal("韓国", :language => :ja)10 writer << RDF::Steatement.new(subject, predicate, object)11 end

13

Loading and Merging RDF

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/South_Korea") 8 puts graph.size 9 graph.load("http://dbpedia.org/resource/Daejeon")10 puts graph.size

14

Querying RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 graph.query([RDF::URI("http://dbpedia.org/resource/Daejeon"), 9 RDF::URI("http://dbpedia.org/ontology/country"),10 nil]).each do |statement|11 puts statement.object12 end

dbpedia:Daejeon

dbpedia-owl:country

dbpedia:South_Korea

dbpedia-owl:Cityrdf:type

Daejeon

rdfs:label36.350101

geo:lat

15

Defining a vocabulary 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/") 7 DBP_ONT = RDF::Vocabulary.new("http://dbpedia.org/ontology/") 8 9 graph = RDF::Graph.new10 graph.load(DBP.Daejeon)11 graph.query([DBP.Daejeon, DBP_ONT.country, nil]).each do |statement|12 puts statement.object13 end

dbpedia:Daejeon

dbpedia-owl:country

dbpedia-owl:Cityrdf:type

Daejeon

rdfs:label36.350101

geo:lat

dbpedia-owl:South_Korea

16

RDB

Query

DB

Model RDF Model

SQL

Relational Model

RDF Store

SPARQL

17

SPARQL

• SPARQL Protocol and RDF Query Language

• W3C Standards

• SQL-like syntax

• Graph Patterns

• Result Formats: XML, JSON, (CSV, TSV)

• $ gem install sparql

18

Using SPARQL in RDF.rb

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 9 sse = SPARQL.parse("""SELECT ?o 10 WHERE {11 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .12 }""")13 14 sse.execute(graph).each do |solution|15 puts solution[:o]16 end

19

Querying a SPARQL Endpoint

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """SELECT ?o 9 WHERE {10 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .11 }"""12 13 client.query(query).each do |solution|14 puts solution[:o]15 end

20

dbpedia:Seoul

Seoul

dbpedia:South_Korea

dbpedia-owl:coutnry

rdfs:label

dbpedia-owl:capital

freebase:Seoul

owl:sameAs

rdfs:labelSouth Koreadbpedia-owl:City

rdf:type

dbpedia:Samsung_Electronics

dbpedia-owl:location

Samsung Electronics

rdfs:labelhttp://global.samsungtomorrow.com/

foaf:homepage

1969-01-01

dbpedia-owl:Companyrdf:type

dbpprop:foundingYear

dbpedia-owl:Country

rdf:type

dbpedia-owl:Lee_Kun-hee

dbpedia-owl:owner

삼성전자

dbpedia-owl:location

rdfs:label

21

dbpedia:Seoul

Seoul

dbpedia:South_Korea

dbpedia-owl:coutnry

rdfs:label

dbpedia-owl:capital

freebase:Seoul

owl:sameAs

rdfs:labelSouth Koreadbpedia-owl:City

rdf:type

dbpedia:Samsung_Electronics

dbpedia-owl:location

Samsung Electronics

rdfs:labelhttp://global.samsungtomorrow.com/

foaf:homepage

1969-01-01

dbpedia-owl:Companyrdf:type

dbpprop:foundingYear

dbpedia-owl:Country

rdf:type

dbpedia-owl:Lee_Kun-hee

dbpedia-owl:owner

삼성전자

dbpedia-owl:location

rdfs:label

PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location rdfs:label ?label .}

22

23

24

Querying a SPARQL Endpoint 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """PREFIX dbpedia: <http://dbpedia.org/resource/> 9 PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>10 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>11 12 SELECT ?location ?label13 WHERE {14 dbpedia:Samsung_Electronics dbpedia-owl:location ?location .15 ?location rdfs:label ?label .16 }"""17 18 client.query(query).each do |solution|19 puts solution.inspect20 end

25

dbpedia:Seoul

Seoul

dbpedia:South_Korea

dbpedia-owl:coutnry

rdfs:label

dbpedia-owl:capital

freebase:Seoul

owl:sameAs

rdfs:labeldbpedia-owl:City

rdf:type

dbpedia:Samsung_Electronics

dbpedia-owl:location

Samsung Electronics

rdfs:labelhttp://global.samsungtomorrow.com/

foaf:homepage

1969-01-01

dbpedia-owl:Companyrdf:type

dbpprop:foundingYear

dbpedia-owl:Country

rdf:type

dbpedia-owl:Lee_Kun-hee

dbpedia-owl:owner

삼성전자

dbpedia-owl:location

rdfs:label

PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label .}

South Korea

26

27

dbpedia:Seoul

Seoul

dbpedia:South_Korea

dbpedia-owl:coutnry

rdfs:label

dbpedia-owl:capital

freebase:Seoul

owl:sameAs

rdfs:labeldbpedia-owl:City

rdf:type

dbpedia:Samsung_Electronics

dbpedia-owl:location

Samsung Electronics

rdfs:labelhttp://global.samsungtomorrow.com/

foaf:homepage

1969-01-01

dbpedia-owl:Companyrdf:type

dbpprop:foundingYear

dbpedia-owl:Country

rdf:type

dbpedia-owl:Lee_Kun-hee

dbpedia-owl:owner

삼성전자

dbpedia-owl:location

rdfs:label

PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label . FILTER(lang(?label) = ‘ko’)}

South Korea

28

29

SPARQL 1.0

• CRUD

• SELECT

• ASK

• CONSTRUCT

• DESCRIBE

30

SELECT

クエリパターン変数の値を取得

SELECT *WHERE { ?s ?p ?o .}LIMIT 100OFFSET 100

31

ASK

PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>

ASK WHERE { dbpedia:Daejeon ?p ?o .}

32

CONSTRUCT

PREFIX foaf: <http://xmlns.com/foaf/0.1/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

CONSTRUCT { ?s rdfs:label ?name .}WHERE { ?s foaf:name ?name .}

33

DESCRIBE

DESCRIBE <http://dbpedia.org/resource/Tokyo>

34

• http://www.w3.org/wiki/SparqlEndpoints

http://www.w3.org/wiki/SparqlEndpointshttp://www.w3.org/wiki/SparqlEndpoints35

• http://www.w3.org/wiki/SparqlEndpoints

http://www.w3.org/wiki/SparqlEndpointshttp://labs.mondeca.com/sparqlEndpointsStatus/36

http://ckan.net/package?res_format=api%2Fsparqlhttp://thedatahub.org/group/lodcloud37

http://lod.ac/apps/lodac_location/38

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

SELECT DISTINCT ?link ?title ?lat ?long WHERE { ?link rdfs:label ?title; geo:lat ?lat; geo:long ?long . FILTER ( ?lat > #{sw_lat} && ?lat < #{ne_lat} && ?long > #{sw_long} && ?long < #{ne_long} && lang(?title) = "ja" )}

39

RDB

Query

DB

Model RDF Model

SQL

Relational Model

RDF Store

SPARQL

40

RDF Store

• Triplestore / Quadstore / Graph DB

• SPARQL Endpoint

• Options

• RDFS/OWL Inference

• Full text indexing

• Clustering

• See http://en.wikipedia.org/wiki/Triplestore

Bigdata®

41

http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/42

Virtuoso Open Source• http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/

• install binary

• install from source$ tar xvzf virtuoso-opensource-6.1.6.tar.gz

$ cd virtuoso-opensource-6.1.6

$ ./configure

$ make

$ make install

$ /usr/local/virtuoso-opensource/bin/virtuoso-t +configfile /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.ini43

http://localhost:8890/conductor/

44

http://localhost:8890/conductor/

45

46

47

http://localhost:8890/sparql

48

Storage Adapter• Memory

• Sesame

• Virtuoso

• DataObjects (SQLite3, PostgreSQL)

• RedStore

• 4Store

• MongoDB

• Cassandra

• Talis

• SPARQL

49

Virtuoso Adapter$ gem install rdf-virtuoso

1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 require 'rdf/virtuoso' 6 7 REPO_URI = "http://localhost:8890" 8 USER = "dba" 9 PASS = "dba" # this should be changed10 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/")11 12 repo = RDF::Virtuoso::Repository.new(REPO_URI, :username => USER,13 :password => PASS, :auth_method => 'digest')14 query = RDF::Virtuoso::Query.select.where([:s, :p, DBP.South_Korea])15 result = repo.select(query)16 17 result.each do |solution|18 puts solution.inspect19 end

50

Summary

• how to consume LOD

• RDF

• SPARQL

51

52

twitter: fumi1fumi@nii.ac.jp