Upload
infinitegraph
View
103
Download
1
Embed Size (px)
DESCRIPTION
In this security solution demo, we have integrated Oracle NoSQL DB with InfiniteGraph to demonstrate the power of using the right tools for the solution. By integrating the key value technology of Oracle with the InfiniteGraph distributed graph database, we are able to create new views of existing Call Detail Record (CDR) details to enable discovery of connections, paths and behaviors that may otherwise be missed. Discover how to add value to your existing Big Data to increase revenues and performance!
Citation preview
The Database
© Objectivity, Inc. 2014
The Power of Relationships in Big Data
Leon Guzenda - Objectivity, Inc. Silicon Valley NoSQL Meetup - 1/23/14
© Objectivity, Inc. 2014
Overview
• The Problem
• Current Big Data Analytics
• Relationship Analytics
• Leveraging NoSQL
• Big Data Connection Platform
• Solution Use Case Demo!2
© Objectivity, Inc. 2014
Objectivity, Inc.
• Headquartered in San Jose, CA • Over two decades of NoSQL and Big Data experience • Enables complex data virtualization and Big Data
solutions for the enterprise • Software products:
• Objectivity/DB • InfiniteGraph • InfiniteGraph Social App
• Embedded in hundreds of enterprises, government organizations and products, with millions of deployments.
!3
© Objectivity, Inc. 2014
A Typical Deployment
!4
© Objectivity, Inc. 2014
Current Big Data Analytics
!5
© Objectivity, Inc. 2014
The ProblemInformation Overload!
• Making sense of it all takes time and $$$
• Which lead to a rush to Big Data Analytics
!6
© Objectivity, Inc. 2014
Current Big Data Analytics
!7
© Objectivity, Inc. 2014
Leveraging NoSQL
!8
© Objectivity, Inc. 2014
Not Only SQL - Four Main Technologies
!9
SimpleHighly Interconnected
© Objectivity, Inc. 2014
Hadoop?
!10
Objectivity/DB & InfiniteGraph
• Distributed processing with multithreading client processes and simple servers*
• Distributed, segmented Federated Database with a Single Logical View down to fine grain objects
• Tuned for random access and powerful parallel queries
• Excel at handling very large graph structures with built-in relationship analytics
Hadoop:
• Parallel processing using a divide and conquer or split and merge paradigm
• Sharded, distributed file system
• Tuned for sequential scans and simple queries
• Not suitable for highly interconnected data sets (graphs)
* Process workflow could be driven using MapReduce
© Objectivity, Inc. 2014
Incremental Analytic Improvements Aren’t Enough
• All current solutions use the same basic architectural model.
• None of the current solutions has an efficient way to store connections between entities in different silos.
• Most analytic technology focuses on the content of the data nodes, rather than the many kinds of connections between the nodes and in those connections.
• Why? Because most DBMSs are bad at handling relationships.
• Object and Graph Databases can efficiently store, manage and query the many kinds of relationships hidden in the data.
!11
© Objectivity, Inc. 2014
Relationship Analytics…A SQL Shortcoming
!12
Table_A Table_B Table_C Table_D Table_E Table_F Table_G
There are some kinds of complex relationship handling problems that SQL wasn't designed for.
© Objectivity, Inc. 2014
…Relationship AnalyticsA SQL Shortcoming
!13
Table_A Table_B Table_C Table_D Table_E Table_F Table_G
InfiniteGraph - The solution can be found with a few lines of code
A3 G4
© Objectivity, Inc. 2014
Graph Terminology
!14
● VERTEX: A single node in a graph data structure
● EDGE: A connection between a pair of VERTICES
● PROPERTIES: Data items that belong to a particular Vertex or Edge
● WEIGHT: A quantity associated with a particular Edge
● GRAPH: A network of linked Vertex and Edge objects
Vertex 1 Vertex 2Edge 1
City: San Francisco Pop: 812,826
City: San Jose Pop: 967,487
Road: I-101Miles: 47.8
© Objectivity, Inc. 2014
Example 1 - Relationship Analytics
!15
LOGISTICS HEALTHCARE INFORMATICS
MARKET ANALYSIS SOCIAL NETWORK ANALYSIS
© Objectivity, Inc. 2014
Finding The Links…
!16
Combatant A
Civilian Q
Situation Y
Civilian P
Bank X
Civilian S
Civilian R
Events/Places People/Orgs Facts
Situation X
Target T
Cafe C S Seen Near TA Banks at X
A Called P
A Seen At Y
A Seen Near X P Emailed S
P Called Q Q Seen Near T
P Called R R Seen Near T
X Paid S
A Eats At
© Objectivity, Inc. 2014
…Finding The Links…
!17
Combatant A
Civilian Q
Situation Y
Civilian P
Civilian S
Civilian R
Events/Places People/Orgs Facts
Situation X
Target T
VERTICES EDGES
S Seen Near TA Banks at X
A Called P
A Seen At Y
A Seen Near X P Emailed S
P Called Q Q Seen Near T
P Called R R Seen Near T
X Paid SBank X
Cafe C
A Eats At
© Objectivity, Inc. 2014
…Finding The Links…
!18
Situation X Combatant ASeen Near
Civilian P
Called
Called
Seen At Situation Y
Civilian Q
Target T
Seen Near
Emailed
Banks At
Bank X
Civilian S
Seen Near
Called
Civilian R
Seen Near
Paid
Eats At
Cafe C
© Objectivity, Inc. 2014
…Finding The Links…
!19
Situation X Combatant ASeen Near
Civilian P
Called
Called
Seen At Situation Y
Civilian Q
Target T
Seen Near
Emailed
Banks At
Bank X
Civilian S
Seen Near
Called
Civilian R
Seen Near
Paid
SUSPECTS
NEEDS PROTECTION
© Objectivity, Inc. 2014
…Finding The Links
!20
OTHER DATABASE(S)
GRAPH DATABASE
© Objectivity, Inc. 2014
Example 2 - Finding Patterns in Open Source Data
!21
● Data Volumes
● Fast-Changing Data
● Sensitivity of Data
● Significance of Data
The Challenges
© Objectivity, Inc. 2014
Example 3 - Cybersecurity
!22
© Objectivity, Inc. 2014
Big Data Connection Platform
!23
© Objectivity, Inc. 2014
Objectivity’s Disruptive Big Data Architecture
!24
Uses Data Virtualization to hide the nodes and focus on the connections
© Objectivity, Inc. 2014
InfiniteGraph
!25
Distributed Parallel Load and Queries
Distributed Parallel Link Finding
Start
Start
Powerful Graph Queries
X
XStart
Finish
Computational and Visualization Plugins
Start
Latency Exceeded
Custom Visualizer
© Objectivity, Inc. 2014
Solution Use Case Demo…
Let’s see InfiniteGraph coupled with Oracle’s NoSQL Solution…
!26