Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP...

Temporal Graph Analytics with GRADOOP

Christopher Rost and Kevin GomezLeipzig University

About the speakers and the team

Kevin, PhD Student

Christopher, PhD Student

Prof. Dr. Erhard Rahm,Database Chair

Timo, M.Sc. Student

Lucas, B.Sc. Student

We’re hiring!

CENTER FORSCALABLE DATA ANALYTICSAND ARTIFICIAL INTELLIGENCE

● Visit us at www.scads.de

● Open positions and projects (Developer and PhD Students)

● Various topics:● Machine Learning (Graph-ML, Privacy)● Image Recognition● Blockchain (Distributed-Ledger)● Graph Analytics

Motivation

“Graphs are everywhere”

“Graphs are everywhere”

“Graphs are everywhere”

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

frien

dOf

friendO

f

friendO

f

friendOf

friendOf

friendO

f

Anna

Bob

Chris

Sally

Dutch

Arthur

Jack

SheldonRaj

Mike

James

John

“Graphs are everywhere”

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

frien

dOf

friendO

f

friendO

f

friendOf

friendOf

friendO

f

Anna

Bob

Chris

Sally

Dutch

Arthur

Jack

SheldonRaj

Mike

James

Johnage:29sex: f

yob: 1991

“Graphs are everywhere”

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

friendOf

frien

dOf

friendO

f

friendO

f

friendOf

friendOf

friendO

f

Anna

Bob

Chris

Sally

Dutch

Arthur

Jack

SheldonRaj

Mike

James

Johnage:29sex: f

yob: 1991

since: 5Y

“A open-source framework and research platform for efficient, distributed and domain independent management and analytics of heterogeneous and

temporal graph data.“

Architecture

HDFS / YARN

Distributed Operator Implementation

TPGMEPGM

G raph A nalytical L anguage (GrALa) Java 8

45k LOC

ALv 2.0

Graphs and collections of graphs

OperatorsOperators

Unary Binary

Log

ical G

rap

hG

rap

h C

ollecti

on

Algorithms

Frequent Subgraph

Temporal extension

Temporal extension

Temporal extension

Temporal extension

Time dependent operatorsOperators

Unary Binary

Log

ical G

rap

hG

rap

h C

ollecti

on

Algorithms

Frequent Subgraph

Time dependent operatorsOperators

Unary Binary

(Tem

p-)

Log

ical G

rap

h(T

em

p-)

Gra

ph

Collecti

on

Algorithms

Frequent Subgraph

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

Surgery

Accident

Cardiology

Oncology

Contact trackingSensors capturing who is close to whom

at what time.{empID1, empID2, t_from, t_to}

In case of an infection, which

hospital services are at risk of contracting

the virus X?

Use case: spread of airborne pathogens

Images by Vecteezy.com.

timet0 t1

t5 ... t

10 t

15 t

20

TemporalGraph contacts = mySource.getTemporalGraph();

TemporalGraph contacts = mySource.getTemporalGraph();

time

Employee[-,-)id: 32432srv: Oncology

Employee[-,-)id: 99934srv: Oncology

contact[ t

2 ,t

6 )

t0 t1

t5 ... t

10 t

15 t

20

timet0 t1

t5 ... t

10 t

15 t

20

Breaking news:

Employee of oncology infected by VirusX.

Detected at t20

.

timet0 t1

t5 ... t

10 t

15 t

20

Breaking news:

Employee of oncology infected by VirusX.

Detected at t20

.

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

timet0 t1

t5 ... t

10 t

15 t

20

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

timet0 t1

t5 ... t

10 t

15 t

20

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

timet0 t1

t5 ... t

10 t

15 t

20

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

timet0 t1

t5 ... t

10 t

15 t

20

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

contactmaxDur: 40 min

contactmaxDur: 122 min

contactmaxDur: 124 min

contactmaxDur: 146 min

contactmaxDur: 2 min

contactmaxDur: 9 min

timet0 t1

t5 ... t

10 t

15 t

20

Virus XSymptoms: 3rd eye growing

Transmission: if contact > 5 minIncubation period: 5 time units

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

contactmaxDur: 40 min

contactmaxDur: 122 min

contactmaxDur: 124 min

contactmaxDur: 146 min

contactmaxDur: 2 min

contactmaxDur: 9 min

timet0 t1

t5 ... t

10 t

15 t

20

contactmaxDur: 40 min

contactmaxDur: 122 min

contactmaxDur: 124 min

contactmaxDur: 146 min

contactmaxDur: 2 min

contactmaxDur: 9 min

Breaking news:

Employees of oncology and surgery quarantined

because of VirusX.

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

TemporalGraph contacts = mySource.getTemporalGraph();

contacts = contacts.snapshot(new FromTo(t15, t20));

contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);

Conclusion● Distributed graph analysis system● Temporal property graph model

– Bitemporal support– Logical graphs and graph collections– Composing operators and algorithms

● Declarative workflow creation

● Visit Gradoop: http://gradoop.com● Read our Wiki: https://github.com/dbs-leipzig/gradoop/wiki● Start: https://github.com/dbs-leipzig/gradoop/wiki/Getting-started● Try the examples: https://github.com/dbs-leipzig/gradoop/wiki/Examples