Upload
achim-friedland
View
107
Download
0
Embed Size (px)
DESCRIPTION
Achim Friedland has provided a very interesting overview of the graph databases products, the goals and some scenarios for graph databases, a brief comparison of property graphs with other models (relational databases, object-oriented, semantic web/RDF, and many other interesting aspects. (via: http://nosql.mypopescu.com/post/1211252052/nosql-frankfurt-a-quick-review-of-the-conference)
Citation preview
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 1NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Frankfurtco-located to ICOODB 2010
28. September 2010Frankfurt, Germany
The GraphDB Landscapeand sones
1NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Gephi, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 2NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 2NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Gephi, flickr
Hashtags:
#nosqlfr #graphdb
#sones #neo4j
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 3
Photo: litlnemo, flickr
3NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 4
• 43 years Object Orientation / OOP
• 35 years Relational Databases / SQL
• 21 years Semantic Web / RDF
Photo: litlnemo, flickr
4NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 5
• 43 years Object Orientation / OOP
• 35 years Relational Databases / SQL
• 21 years Semantic Web / RDF
Photo: litlnemo, flickr
5NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
…but still we do not know where
our files have gone!
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 6
Our wish list…
6NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Chris Westermeyer, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 7
Simple management of structured, semi-structured and unstructured
information
7NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: litlnemo, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 8
Simple management of structured, semi-structured and unstructured
information
8NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: litlnemo, flickr
SQL
XML/JSON binaries
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 9
Simple recursively linked information models
9NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Robbert van der Steeg, flickr
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 10
Simple recursively linked information models
10NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Robbert van der Steeg, flickr
• <a href = “…”>• metadata e.g. EXIF
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 1111NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Khem, flickr
Traversing linked information, finding shortest-paths, do
semantic partitions
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 12
Photo: NASA, flickr
12NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Recommendation and discovery
of potentially interessting linked information
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 13
Photo: NASA, flickr
13NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Recommendation and discovery
of potentially interessting linked information
socialpersonal item-related
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 14
Photo: squacco, flickr
14NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Consistency criteria and indices for simple attributes up to complex
subgraph structures
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 1515NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Cedrix Thual, flickr
Versioned information and versioned information schemata
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 1616NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Birger Hoppe, flickr
Good integration into state-of-the-art programming concepts
(no Object-Relational-Mappers!)
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 1717NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Photo: Chuck “Caveman” Coker, flickr
Can graph-databases help?
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 18
Photo: Jeffrey Beall, flickr
Graph-Databases?
Graph = ( Vertics, Edges )
18NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
A graph database is a database that uses graph structures with nodes, edges, and properties to represent and store information. General graph
databases that can store any graph are distinct from specialized graph databases such as triple stores and
network databases.http://en.wikipedia.org/wiki/Graph_database
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 19
Photo: Jeffrey Beall, flickr
Graph-Databases?
Graph = ( Vertics, Edges )
R. Angles, C. Gutierrez: Survey of graph database models,
ACM Computing Surv. 40, 1 (Feb. 2008), 1-39
Adrian Silvescu, Donia Caragea, Anna Atramentov:
Graph Databases, Technical Report. May 2002
Mark Gemis, Jan Paredaens, Inge Thyssens, und Jan Van den
Bussche: GOOD: A Graph-Oriented Object Database System, 1993;
In: Proceedings of the1993 ACM SIGMOD Intern. Conf. on Management of Data (Washington D.C., US, May 1993), 505-510.
19NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 20
Photo: Gephi, flickr
The Property-Graph
FriendsAliceID = 1
Alter = 21
BobID = 2
Alter = 23since = 2009/09/21
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 20
Properties are key-values pairse.g. <String, Object> or <AttributeKey, Object>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 21
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Focus on structured tables and their relational consistency constraints. Does not auto-scale.
vs. Relational Databases
Table A Table BRelation
Table
21
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 22
Photo: Gephi, flickr
The Property-Graph
Reference / PointerObject Object
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Focus on structured or dynamic objects.No (simple way for) edge properties.
vs. Object-Oriented Model
22
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 23
Photo: Gephi, flickr
The Property-Graph
PredicateSubject Object
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Focus on triples or quads.No (simple way for) edge properties.
vs. Semantic Web / RDF
23
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 24
Photo: Gephi, flickr
The Property-Graph
Friends EnemiesAlice Bob
Hobby Hobby
Guru
Crypto
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Bruce
Eve
Enemies
observes
24
A directed, attributed, multi-relational graphas central data structure for graph-databases.
since = 2009/11/02
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 25
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 25
Photo: Jeffrey Beall, flickr
The Property-Graph Model
http://wiki.github.com/tinkerpop/blueprints/property-graphmodel
Rodriguez, M.A., Neubauer, P., “Constructions from Dots and Lines,”
Bulletin of the American Society for Information Science and
Technology, American Society for Information Science and Technology,
volume 36, August 2010.
…much more at http://markorodriguez.com ;)
Currently no academically founded or well-definedgraph data model, but…
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 26
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 26
// Neo4J example
Node _alice = graphdb.createNode();_alice.setProperty("Name", "Alice");_alice.setProperty("Age", 21);
Node _bob = graphdb.createNode();_alice.setProperty("Name", „Bob");_alice.setProperty("Age", 23);
_alice.createRelationshipTo(_bob, Friends);_bob.createRelationshipTo(_alice, Friends);
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 27
Photo: Gephi, flickr
The Property-Graph
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 27
// sones gql example
CREATE VERTEX UserADD ATTRIBUTES (String Name, SET<User> Friends)INDICES (Name)
INSERT INTO User VALUES (Name = "Alice", Age = 21)INSERT INTO User VALUES (Name = "Bob", Age = 23)
LINK User(Name = ‘Alice') VIA Friends TO User(Name = ‘Bob')LINK User(Name = ‘Bob') VIA Friends TO User(Name = ‘Alice‘)
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 28
Photo: Gephi, flickr
Benefits of Graph-Databases
• Explicit graph data model– No implicit model like within relational dbs
– In the spirit of NoSQL:“One database for one data model”
– GraphDB gets a change to understand the semantics and our intention
28NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 29
Photo: Gephi, flickr
Benefits of Graph-Databases
• Index-free adjacency– No global adjacency index as each vertex acts
as its own “mini-index”
– The speed for traversing from one node to another is independed of the size of the graph( Data locality and improved scalability )
– Optimized for random access patterns
29NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 30
Photo: Gephi, flickr
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
30NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Graph-Database landscape
Marklogic?
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 31
Photo: Gephi, flickr
Data model
DEX
sones
Neo4J
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
31NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Property-Graph
No Edge-Properties
Schemata
HyperGraphs
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 32
Photo: Gephi, flickr
Query methods
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
32NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Traverser API
Gremlin
Adapted SQL
GraphQL
HGQuery API
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 33
Photo: Gephi, flickr
Programming languages
Neo4J
sones
DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
Filament
VertexDB
C++
Java
C#
C
JS
Lua
33NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 34
Photo: Gephi, flickr
Neo4J
sonesDEX
InfoGrid
InfiniteGraph
HyperGraphDBOrientDB
FilamentVertexDB
“Free Lunch”
AGPLv3
LGPL
BSD
Apache 2.0
$$$
34NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
License models
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 35
Photo: Gephi, flickr
Neo4J
sones DEX
InfoGrid
InfiniteGraph
HyperGraphDB
OrientDB
FilamentVertexDB
35NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Protocols
REST/JSON
REST/XML
HTTP
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 36
• URL• Goals• APIs
• Transactions• Repl./Scaling• Persistency
http://www.neo4j.orgEmbedded and spatial DBMSJava, JRuby, Ruby, Python, Scala, Clojure, C#, ErlangConcurrent reads, Sync on nodesMaster-Slave, Master FailoverProprietary ondisk format
Neo4J
Photo: Gephi, flickr
36NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 37
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.infogrid.orgDistributed GraphDB and Web-Application FrameworkYesp2p-replication, partitioningProprietary, RDBMS, Hadoop
InfoGrid
Photo: Gephi, flickr
37NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
MeshBase _GDB = StoreMeshBase.create(_MySQLStore);MeshObject _xkcd = _GDB.getMeshObjectLifecycleManager().createMeshObject();_xkcd.setProperty("Name", "xkcd");_xkcd.setProperty("Url", "http://www.xkcd.com");_xkcd.relate(_good);
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 38
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.dama.upc.eduHigh performance and integration of various information sourcesn/an/aProprietary ondisk format
DEX
Photo: Gephi, flickr
38NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
long xkcd = _GDB.newNode(Resource);_GDB.setAttribute(xkcd, Name, "xkcd");_GDB.setAttribute(xkcd, Url, "http://xkcd.com");int Tags = _GDB.newUndirectedEdgeType("Tags");long e2 = _GDB.newEdge(xkcd, good, Tags);
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 39
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.kobrix.comAI, NLP, Semantic WebACI(D), STMp2pBerkeley DB
HyperGraphDB
Photo: Gephi, flickr
39NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
// Note: Website(), Tag() are POJOHGHandle _xkcd = _GDB.add(new Website("xkcd", "http://www.xkcd.com"));HGHandle _good = _GDB.add(new Tag("good");HGHandle _xkcd2good = _GDB.add(new HGPlainLink(_xkcd, _good));
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 40
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.infinitegraph.comVery large graph databasesDistributed transactionsObjectivity/DBObjectivity/DB
Infinite Graph
Photo: Gephi, flickr
40NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
// Note: Website(), Tag() extend BaseVertex; Edge() extends BaseEdgeWebsite xkcd = new Website("xkcd", "http://xkcd.com/"); _GDB.addVertex(xkcd);Tag good = new Tag("good"); _GDB.addVertex(good);xkcd.addEdge (new Edge(), good, EdgeKind.BIDIRECTIONAL);
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 41
• URL• Goals
• Transactions• Repl./Scaling• Persistency
http://www.orienttechnologies.comProviding a KV-Store, DocumentDB
and a GraphDB in one solutionACID, MVCCn/aProprietary ondisk format
OrientDB
Photo: Gephi, flickr
41NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
OGraphVertex _xkcd = _GDB.createVertex().set("Name", "xkcd").set("Url", "http://www.xkcd.com");
OGraphVertex _good = _GDB.createVertex().set("Name", "good");_xkcd.link (_good);
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 42
• URL• Goals• Transactions• Repl./Scaling• Persistency
http://www.sones.deManagement of linked (binary) dataACID, MVCC (alpha)p2p (alpha)Proprietary file system
sones
Photo: Gephi, flickr
42NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 43
Photo: Shayne Kaye, flickr
User Friend
AliceID = 1
Alter = 21
BobID = 2
Alter = 23since = 2009/09/21
43NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
sones Property-Hypergraph
User Friend CarolID = 3
Alter = 20since = 1997/04/11
SET<User> Friends
SetMaxNumber = 12
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 44
Photo: Shayne Kaye, flickr
User Friend BobID = 2
Alter = 23since = 2009/09/21
44NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
sones Property-Hypergraph
User Friend CarolID = 3
Alter = 20since = 1997/04/11
SET<User> Friends
SetMaxNumber = 12
AliceID = 1
Alter = 21
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 45
Photo: Shayne Kaye, flickr
• Extensions to the Property Graph model
• Multiple edges are be grouped by a hyperedge
• Hyperedge may have their own properties
• Properties may include code as data(e.g. C#: Func<…>, ExpressionTrees)
• Allows calculations be done among the set of edges (GetMinWeight, SetMaxNumber, …)
sones Property-Hypergraph
45NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 46
Photo: Shayne Kaye, flickr
sones Special Edges and Properties
46NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
Vertex
PropertyStream DefaultEdition
LatestRevision
20100921…
Copy1
Copy2
20100918… Copy1
FileStream DefaultEdition
LatestRevision
20100815… Copy1
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 47
Photo: Shayne Kaye, flickr
sones Graph Query Language
47NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
FROM User SELECT User.Friends.Friends.Name
• “SQL for graphs”
• Its goal is to provide a very user-friendly language forad-hoc graph queries
• Functions and aggregates are type-safe and can be extended by your own plug-ins, e.g.
• SELECT COUNT(User.Friends)
• SELECT User.Friends.Random(2)
• SELECT User.Friends.Name.Substring(2,5)
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 48
Photo: jonathanpercy, flickr
• Get outgoing edges, where their label is equal to ‘created’
• Get their incoming vertices
• Get incoming edges, where their labels equals ‘created’
• Get outgoing vertices, except the starting vertex
• Get their name property
Gremlin
48NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
./outE[@label=‘created’]/inV/inE[@label=‘created’]/outV[g:except($_)]/@name
NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]> 49NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>
For more information…
[email protected]://www.twitter.com/ahzf
http://www.twitter.com/graphdbs