Dpso -- An Optimization Approach for Load Balancing in Parallel Link Discovery

Motivation Load Balancing Approaches Evaluation Conclusion and Future Work

DPSOAn Optimization Approach for Load Balancing in Parallel

Link Discovery

Mohamed Ahmed Sherif and Axel-Cyrille Ngonga Ngomo

September 17, 2015

Mohamed Ahmed Sherif and Axel-Cyrille Ngonga Ngomo — DPSO 1/30


Outline

1 Motivation

2 Load Balancing Approaches

3 Evaluation

4 Conclusion and Future Work



Outline

1 Motivation


3 Evaluation




Link Discovery (LD)

Why LD?

1 Fourth principle

2 Links are central for

Cross-ontology QAData IntegrationReasoningFederated Queries...

LD Time complexity

Large number of triples (> 63 billion triples)

Quadratic runtime



Load Balancing for LD

Need to develop highly scalable LD algorithms

Local hardware LD

Suffer less from the data transfer bottleneckBetter runtime than parallel LD approaches on remotehardware (e.g. cloud-based approaches)

Current load balancing approaches for local LD

Paid little attentionMostly naıve implementations





Local hardware LD








Local hardware LD






Load Balancing Problem

Given:

n tasks τ1, ..., τnComputational complexities c(τ1), ..., c(τn)m processors

Goal:

Distribute τi across m processors as evenly as possible

Example

3 tasks τ1, τ2 and τ3 with complexities 3, 4 resp. 6

2 processors

An optimal distribution:

P1 → {τ1, τ2}P2 → {τ3}



Load Balancing Problem

Given:

n tasks τ1, ..., τnComputational complexities c(τ1), ..., c(τn)m processors

Goal:

Distribute τi across m processors as evenly as possible

Example

3 tasks τ1, τ2 and τ3 with complexities 3, 4 resp. 6

2 processors

An optimal distribution:

P1 → {τ1, τ2}P2 → {τ3}



Outline

1 Motivation


3 Evaluation





Notations

Given S , T of source resp. target resources

Divides S , T such thatk⋃

i=1

Si = S andl⋃

j=1

Tj = T

Determines (Si ,Tj) whose elements are to be compared

The idea of load balancing is to distribute thecomputation of Si × Tj over m processors

Task τ : Comparing elements in (Si ,Tj)

c(τ) = |Si | · |Tj |block B : Set of all tasks assigned to a single processor

c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m

∣∣∣2Mohamed Ahmed Sherif and Axel-Cyrille Ngonga Ngomo — DPSO 8/30



Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m




Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m




Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m




Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m




Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m




Notations



i=1

Si = S andl⋃

j=1

Tj = T





c(B) =∑t∈B

c(τ)

MSE =∑m

i=1

∣∣∣c(Bi)−∑m

j=1c(Bj )

m



Running Example

4 tasks {τ1, τ2, τ3, τ4}Respective complexities {7, 1, 8, 3}2 processors P1,P2

Tasks: 7 1 8 3

Processors: 1 1

τ1 assigned to P1: 7



Naıve Load Balancer

Idea

Divides tasks between processors based on their index andregardless of complexity

Example

Processors assignment: 7 1 8 3 MSE = 30.25



Greedy Load Balancer

Idea

1 Sorts tasks in descending order based on their complexity

2 Starting from the most complex task, assigns tasks toprocessors in order

Example

1. Sorted tasks: 8 7 3 1

2. Processors assignment: 8 7 3 1

MSE = 2.25



Pair-Based Load Balancer

Idea

1 Sorts tasks according to task complexity

2 In order, assigns i th and (n − i + 1)th tasks to Pi

Example

1. Sort tasks: 1 3 7 8

2. Processors assignment: 1 3 7 8

MSE = 0.25



Particle Swarm Optimization

Idea

Initialization

Best Known Positions (BKP)BKP ← Partitions the n tasks to m task blocksComputes fitness function F to the initial BKPF is the complexity difference between the most andleast loaded blocksInitializes Best Known Fitness (BKF) to F

Until a termination criterion is met

Performs the particles migration, based on randomparticle velocityRecomputes FIf F < BKF , updates BKF and BKP

Return BKP




Idea

Initialization




Return BKP




Idea

Initialization




Return BKP




Idea

Initialization




Return BKP




Example

Termination criterion: max number of iterations of 1

Initialization:

7 1 8 3 BKF = F = 11

First iteration:

7 1 8 3 F = 5

As F < BKF , updates BKF and PKB

MSE = 6.25



Deterministic PSO (DPSO)

DPSO

PSO is non-deterministic

PSO depends on a random selection of velocity formoving particles

We propose the Deterministic PSO (DPSO)




Idea

Partitions the n tasks to m task blocks

Until termination criterion is met

Finds the most overloaded block B+ and the leastunderloaded block B−

Sorts tasks within B+ based in their complexitiesAs far as a better balancing between B+ to B− is met

Moves tasks in order from B+ to B− (task migration)

Computes fitness function as complexity differencebetween B+ and B−

Returns best known blocks




Why is DPSO deterministic?

Only moves tasks from B+ to B− (no random migration)

Sorts B+ tasks before task migration start

Insures optimal load balancing between B+ and B−




Example

Termination criterion: max number of iterations of 1

Initialization:

B+ = 8 7 ,B− = 1 3 F = 11

First iteration:

Sorted B+ = 7 8 ,B− = 1 3

Task migration: B+ = 8 ,B− = 1 3 7 F = 3

MSE = 2.25



Outline

1 Motivation


3 Evaluation




Evaluation Setup

Orchid

The parallel task generation was based on Orchid

Orchid partitions the surface of the planet

A task is the comparison of all points in two partitions



Evaluation Setup

Datasets

1 5 synthetic geographic datasets

Polygons’ sizes varied from 1 to 10 points

2 3 real geographic datasets

NutsDBpediaLinkedGeoData



Evaluation Setup

Hardware

64-core server running OpenJDK 64-Bit Server 1.6.0 27on Ubuntu 12.04.2 LTS.

8 quad-core processor Intel(R) Core(TM) i7-3770 CPU @3.40 GHz with 8192 KB cache

Each experiment was assigned 20 GB of memory

PSO

PSO ran 5 times in each experiment and provide themean of the 5 runs



Orchid vs. Parallel Orchid

Goal

Evaluate the speedup gained by using parallel Orchid

For Nuts, PSO and DPSO up to 3 times faster

For LinkedGeoData, PSO and DPSO up to 10 times faster

2 4 8

Number of threads

0.0

0.1

0.2

0.3

0.4

Tim

e (

min

.)

NaïveGreedyPairBasedPSODPSOOrchid

Nuts runtime

2 4 8

Number of threads

1

10

100

1000

Tim

e (

min

.)


LinkedGeoData runtime



Orchid vs. Parallel Orchid

Goal

PSO and DPSO are capable of achieving superlinearperformance, as processor caches are faster than RAM

Greedy and pair-based fail to achieve even the run time ofthe normal Orchid, due to significant sorting time

2 4 8

Number of threads

0.0

0.1

0.2

0.3

0.4

Tim

e (

min

.)


Nuts runtime

2 4 8

Number of threads

1

10

100

1000

Tim

e (

min

.)





Parallel Load balancing Algorithms

Goals

Measure each algorithm runtime

Qualify each algorithm data distribution using MSE

2 4 8

Number of threads

0.0

0.1

0.2

0.3

0.4

Tim

e (

min

.)


Nuts runtime

2 4 8

Number of threads

1010

1011

1012

1013

MS

E

NaïveGreedyPairBasedPSODPSO

Nuts MSE




Goals



2 4 8

Number of threads

0.0

0.1

1.0

10.0

Tim

e (

min

.)


DBpedia runtime

2 4 8

Number of threads

1011

1012

1013

MSE


DBpedia MSE




Goals



2 4 8

Number of threads

1

10

100

1000

Tim

e (

min

.)



2 4 8

Number of threads

1009

1010

1011

MSE


LinkedGeoData MSE



Outline

1 Motivation


3 Evaluation




Conclusion and Future Work

Conclusion

Presented load balancing techniques for link discovery

Proposed deterministic PSO (DPSO)

Combined load balancing algorithms with Orchid

Evaluated on real and artificial datasets

Future Work

Enable splitting of one task over multiple processors

Implement a caching techniques

Study the combination of DPSO with other taskgeneration algorithms



Conclusion and Future Work

Conclusion

Presented load balancing techniques for link discovery

Proposed deterministic PSO (DPSO)

Combined load balancing algorithms with Orchid

Evaluated on real and artificial datasets

Future Work

Enable splitting of one task over multiple processors

Implement a caching techniques

Study the combination of DPSO with other taskgeneration algorithms



Thank You!

Questions?Mohamed Sherif

Augustusplatz 10D-04109 Leipzig

[email protected]://aksw.org/MohamedSherif

http://aksw.org/Projects/LIMES

#akswgroup


http://aksw.org/MohamedSherif

http://aksw.org/Projects/LIMES

Education

Dpso -- An Optimization Approach for Load Balancing in Parallel Link Discovery