Intro to Neo4j or why insurances should love graphs

Introduction to Graph Databases

@peterneubauer #neo4j

Thursday, April 19, 12

What’s the plan?

๏Why a graph?

What’s the plan?

๏Why a graph?

๏Graph Database 101

What’s the plan?

๏Why a graph?

๏a look at Neo4j

What’s the plan?

๏Why a graph?

๏a look at Neo4j

๏the Real World

Why a graph?

Q: What are graphs good for?

๏Recommendations

๏Business intelligence

๏Social computing

๏Geospatial

๏MDM

๏Systems management

๏Genealogy

A: highly connected data

๏Recommendations

๏Social computing

๏Geospatial

๏MDM

๏Genealogy

• Real Use Cases:

• [A] ACL from Hell

• [B] Timely recommendations

• [C] Global collaboration

Trends in BigData & NOSQL

๏1. increasing data size (big data)

• “Every 2 days we create as much information as we did up to 2003” - Eric Schmidt

๏2. increasingly connected data (graph data)

• for example, text documents to html

๏3. semi-structured data

• individualization of data, with common sub-set

๏4. architecture - a facade over multiple services

• from monolithic to modular, distributed applications

4 Categories of NOSQL

Key-Value Category๏“Dynamo: Amazon’s Highly Available Key-Value Store” (2007)

๏Data model:

•Global key-value mapping

•Big scalable HashMap

•Highly fault tolerant (typically)

๏Examples:

•Riak, Redis, Voldemort

Key-Value: Pros & Cons๏Strengths

• Simple data model

•Great at scaling out horizontally

• Scalable

•Available

๏Weaknesses:

• Simplistic data model

• Poor for complex data

Column-Family Category๏Google’s “Bigtable: A Distributed Storage System for Structured

Data” (2006)

•Column-Family are essentially Big Table clones

๏Data model:

•A big table, with column families

•Map-reduce for querying/processing

๏Examples:

•HBase, HyperTable, Cassandra

Column-Family: Pros & Cons๏Strengths

•Data model supports semi-structured data

•Naturally indexed (columns)

•Good at scaling out horizontally

๏Weaknesses:

•Unsuited for interconnected data

Document Database Category๏Data model

•Collections of documents

•A document is a key-value collection

• Index-centric, lots of map-reduce

๏Examples

•CouchDB, MongoDB

Document Database: Pros & Cons๏Strengths

• Simple, powerful data model (just like SVN!)

•Good scaling (especially if sharding supported)

๏Weaknesses:

•Unsuited for interconnected data

•Query model limited to keys (and indexes)

•Map reduce for larger queries

Graph Database Category๏Data model:

•Nodes & Relationships

•Hypergraph, sometimes (edges with multiple endpoints)

๏Examples:

•Neo4j (of course), OrientDB, InfiniteGraph, AllegroGraph

Living in a NOSQL WorldCo

Key-ValueStore

ColumnFamily

Key-ValueStore

ColumnFamily

Key-ValueStore

DocumentDatabases

ColumnFamily

Key-ValueStore

DocumentDatabases

GraphDatabases

ColumnFamily

Key-ValueStore

DocumentDatabases

GraphDatabases

usecases

Graph Database: Pros & Cons

Graph Database: Pros & Cons๏Strengths

• Powerful data model, as general as RDBMS

• Fast, for connected data

• Easy to query

๏Weaknesses:

• Easy to query

๏Weaknesses:

• Sharding (though they can scale reasonably well)

• Easy to query

๏Weaknesses:

‣also, stay tuned for developments here

• Easy to query

๏Weaknesses:

•Requires conceptual shift

• Easy to query

๏Weaknesses:

•Requires conceptual shift

‣though graph-like thinking becomes addictive

Graph DB 101

Some well-known named graphs

17see http://en.wikipedia.org/wiki/Gallery_of_named_graphsThursday, April 19, 12

17see http://en.wikipedia.org/wiki/Gallery_of_named_graphs

diamond

diamond butterfly

diamond butterfly star

diamond butterfly bullstar

franklin

franklin robertson

franklin hortonrobertson

franklin horton hall-jankorobertson

We’re talking about a Property Graph

๏Nodes

๏Relationships

๏Nodes

๏Relationships

๏Nodes

๏Relationships

๏Nodes

๏Relationships

๏Properties

๏Nodes

๏Relationships

๏Properties

name:Andreasjob: talking

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

๏Nodes

๏Relationships

๏Properties

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

๏Nodes

๏Relationships

๏Properties

+ Indexes

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

Compared to RDBMS

becomes

A look at Graph Queries

Query a graph with a traversal

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

// lookup starting point in an indexstart n=node:node_auto_index(name = ‘Andreas’)

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

// lookup starting point in an indexstart n=node:node_auto_index(name = ‘Andreas’)// then traverse to find resultsstart n=node:People(name = ‘Andreas’)match (n)--()--(foaf) return foaf

knowssince: 2008

knowssince: 2006

knowssince: 1992

knowssince: 2002

knowssince: 2006

knowssince: 2002

knowssince: 1998

knowssince: 2000

since: 2002

knowssince: 1998

knowssince: 1996

Cypher

Cypher๏a pattern-matching query language

๏declarative grammar with clauses (like SQL)

๏aggregation, ordering, limits

๏tabular results

Cypher๏a pattern-matching query language

๏declarative grammar with clauses (like SQL)

๏aggregation, ordering, limits

๏tabular results

// get node with id 0start a=node(0) return a// traverse from node 1start a=node(1) match (a)-->(b) return b// return friends of friendsstart a=node(1) match (a)--()--(c) return c

Neo4j - the Graph Database

Background of Neo4j๏ 2001 - Windh Technologies, a media asset management company

• CTO Peter with Emil, Johan prototyped a proper graph interface

• first SQL-backed, then revised as a full-stack implementation

• (just like Amazon-Dynamo, Facebook-Cassandra)

๏ 2003 Neo4j went into 24/7 production

๏ 2006-2007 - Neo4j was spun off as an open source project

๏ 2009 seed funding for the company

๏ 2010 Neo4j Server was created (previously only an embedded DB)

๏ 2011 Fully funded silicon valley start-up - Neo Technology

Neo4j is a Graph Database

Neo4j is a Graph Database๏A Graph Database:

• a Property Graph with Nodes, Relationships

and Properties on both

• perfect for complex, highly connected data

๏A Graph Database:

• reliable with real ACID Transactions

• scalable: 32 Billion Nodes, 32 Billion Relationships, 64 Billion Properties

• Server with REST API, or Embeddable on the JVM

•high-performance with High-Availability (read scaling)25

the Real World

๏Recommendations

๏Social computing

๏Geospatial

๏MDM

๏Genealogy

๏Recommendations

๏Social computing

๏Geospatial

๏MDM

๏Genealogy

• Real Use Cases:

• [A] ACL from Hell

• [B] Timely recommendations

• [C] Global collaboration

[A] ACL from Hell

[A] ACL from Hell๏ Customer: leading consumer utility company with tons

and tons of users

๏ Goal: comprehensive access control administration for customers

๏ Benefits:

• Flexible and dynamic architecture

• Exceptional performance

• Extensible data model supports new applications and features

• Low cost

and tons of users

๏ Benefits:

• Low cost

• A Reliable access control administration system for

5 million customers, subscriptions and agreements

• Complex dependencies between groups, companies, individuals, accounts, products, subscriptions, services and agreements

• Broad and deep graphs (master customers with 1000s of customers, subscriptions & agreements)

and tons of users

๏ Benefits:

• Low cost

• A Reliable access control administration system for

5 million customers, subscriptions and agreements

• Complex dependencies between groups, companies, individuals, accounts, products, subscriptions, services and agreements

• Broad and deep graphs (master customers with 1000s of customers, subscriptions & agreements)

subscription: sports

service: NFL

account: 9758352794

agreement: ultimate

subscribes to

has plan

includes

provides group: graphistas

promotion: fall

member of

offered

discounts

company: Neo Technologyworks with

gets discount on

subscription: local

subscribes to

provides service: Ravens

includes

[A] ACL from Hell

[B] Timely Recommendations

[B] Timely Recommendations๏ Customer: a professional social network

• 35 millions users, adding 30,000+ each day

๏ Goal: up-to-date recommendations

• Scalable solution with real-time end-user experience

• Low maintenance and reliable architecture

• 8-week implementation

๏ Problem:

• Real-time recommendation imperative to attract new users and maintain positive user retention

• Clustered MySQL solution not scalable or fast enough to support real-time requirements

๏ Upgrade from running a batch job

• initial hour-long batch job

• but then success happened, and it became a day

• then two days

๏ With Neo4j, real time recommendations

๏ Problem:

• Real-time recommendation imperative to attract new users and maintain positive user retention

• Clustered MySQL solution not scalable or fast enough to support real-time requirements

๏ Upgrade from running a batch job

• initial hour-long batch job

• but then success happened, and it became a day

• then two days

๏ With Neo4j, real time recommendationsname:Andreasjob: talking

[C] Collaboration on Global Scale

[C] Collaboration on Global Scale๏ Customer: a worldwide software leader

• highly collaborative end-users

๏ Goal: offer an online platform for global collaboration

• Highly flexible data analysis

• Sub-second results for large, densely-connected data

• User experience - competitive advantage

• Massive amounts of data tied to members, user groups, member content, etc. all interconnected

• Infer collaborative relationships through user-generated content

• Worldwide Availability

Asia North America Europe

Insurance <3 Graphs?

Q: Why should you care?

A: because you have connected data.

๏CRM, BI, social graphs

๏GeoSpatial analytics

๏Fraud detection

๏Network management

A sample insurance domain setup

Building A

sub_product

Coverage: Super

covered_by

Coverage: Fire

sub_cover

Size: 120m2

attribute

Risk: Building small

risk_has_attrcovers_risk

Quote: C12

includes

Customer: C12

is_offered

Policy: C12

is_offered

Agreement: C12

signed

User: U34

based_on

contains

Questionaire: Q1

concerns

contains_question

fills_in

includes

T&C: X

Recommendations, BI, Social Computing

๏enrich your CRM with data from Facebook, Google, Twitter etc

๏Recommender systems for products

๏Find influencers in your customer base for special treatment

This is what your CRM sees

36http://inmaps.linkedinlabs.com/network

Customer1

Peter Neubauer

This is what your CRM doesn’t see.

37http://inmaps.linkedinlabs.com/networkThursday, April 19, 12

This is what your CRM doesn’t see.

37http://inmaps.linkedinlabs.com/networkThursday, April 19, 12

Geospatial features

๏Dynamic layers from different sources

Geospatial features

• domain data -> flood area layer + crime index + firestation + living standard index

Geospatial features

• domain data -> flood area layer + crime index + firestation + living standard index

๏routes of low insurance risks

Geospatial features

Configuration/Network Management

๏Model physical and logical networks

• impact analysis

• configuration management

• impact analysis

• configuration management

• network inventory

Questions!

and, Thanks :)

Intro to Neo4j or why insurances should love graphs

Technology

Modelling Data as Graphs (Neo4j)

GraphConnect 2014 SF: Dynamic Graphs: The Future of Neo4j Visualization

OpenData, Graphs and do-it-yourself Journalism · 2019. 9. 23. · About Neo4j • Creators of Neo4j Graph Plattform and Neo4j - World’s leading open-source graph database • Company

11 Introduction to Neo4j. 2 We all have our own graphs

An Overview of Neo4j - uniroma1.itrosati/dmds-1819/Neo4J.pdf · NEO4J: Cypher’s introduction Cypher is a declarative, SQL inspired language for describing patterns in graphs. It

Open Insurances

Introduction to Graph Data Management...Introduction to Graph Data Management with neo4j Tobias .Lindaaker@neo4j.com @thobe #neo4j 1 Why graphs? The world is a graph – everything

Independent insurances-website

Intro to Graphs and Neo4j

Big Data,Neo4j Developer,Neo4j Graph database,Casandra Developer

Neo4j OGM - An Object Graph Mapping Library for Neo4j v2we-yun.com/doc/3.3/neo4j-ogm-manual-2.0.pdf · Neo4j OGM 1.x works against server based installations of Neo4j and uses Cypher

NAGICO STRONG - Nagico Insurances : Nagico Insurances

Property Graphs: Neo4j and Cypherhomepages.inf.ed.ac.uk/libkin/teach/beijing2018/neo4j-beijing.pdf · Neo4j name: Ian name: ACME Person Company R L name: Jacob Person name: Tobias

Relevant Search Leveraging Knowledge Graphs with Neo4j

INSURANCES - wib-bank.net

Neo4j Aura - Neo4j Graph Database Platform · Neo4j Aura is the simplest way to run Neo4j in the cloud. Completely automated and fully managed, Neo4j Aura delivers the world’s most

Inside Neo4j's Graph Query Engine - Huihoodocs.huihoo.com/javaone/2014/BOF4909-Inside-Neo4j-Graph-Query... · Inside Neo4j's Graph Query Engine ... • Neo4j • Graphs: Beneﬁts,

Neo4j. План Cypher – Создание – Запросы Neo4j embedded in Java Немного о релизации (Neo4j Internals) – Native Graph Processing – Native

NoSQL Databases - IRISA · BUT makes some manipulations difficult ... § Neo4j, Infinite Graph ... § Handling very big graphs (shardingis difficult) Neo4j 70

insurances in india