an introduction to Distem

an introduction to

Distem – http://distem.gforge.inria.fr/ 1 / 17

Imagine you want to build a race car

Distem – http://distem.gforge.inria.fr/ 2 / 17

But you want it to work well

on both dry tracks and wet tracks

Distem – http://distem.gforge.inria.fr/ 3 / 17

You could use a simulation :

model the car and the trackand compute how they would interact

Distem – http://distem.gforge.inria.fr/ 4 / 17

Or you could use a real world experiment :

build a car, and try it on a real circuit

But sometimes, rain is unlikely

Distem – http://distem.gforge.inria.fr/ 5 / 17

Or you could use a real world experiment :

build a car, and try it on a real circuit

But sometimes, rain is unlikely

Distem – http://distem.gforge.inria.fr/ 5 / 17

You could move to another race track . . .But what if you could change the weather ?

That’s what Distem does :Take a platform, and alter it to suit

your experimentation needs

Distem – http://distem.gforge.inria.fr/ 6 / 17

You could move to another race track . . .But what if you could change the weather ?

That’s what Distem does :Take a platform, and alter it to suit

your experimentation needsDistem – http://distem.gforge.inria.fr/ 6 / 17

An emulator for distributed systems

Take your real application

Run it on a cluster

And use Distem to alter the platformso it matches the experimental conditions you need

Heterogeneous nodesLong distance networksFaults, perf. variations

Grid, Cloud, P2P features. . .

Distem – http://distem.gforge.inria.fr/ 7 / 17

Ideal complement to other toolsSimGrid

Toolkit for the simulation ofdistributed applications in

heterogeneous environments

Grid’5000Large-scale platformfor experiments ondistributed systems

1700 nodes, 7000 coresFully reconfigurable

Grid’5000

Distem can run on Grid’5000

Distem – http://distem.gforge.inria.fr/ 8 / 17

Ideal complement to other toolsSimGrid

Toolkit for the simulation ofdistributed applications in

heterogeneous environments

Grid’5000Large-scale platformfor experiments ondistributed systems

1700 nodes, 7000 coresFully reconfigurable

Grid’5000

Distem can run on Grid’5000Distem – http://distem.gforge.inria.fr/ 8 / 17

What can Distem do for you ?

I Introduce heterogeneity in an homogeneous clusterHow does your app perform when some nodes are slower ?

I Emulate complex network topologiesHow does your app perform on a Grid ? on a slower network ?

I Inject faults and performance variations during the experimentHow does your app behave when a node crashes ?When the available CPU time decreases ?When the available network bandwidth increases ?

Distem – http://distem.gforge.inria.fr/ 9 / 17

Introducing heterogeneity

I Distem splits real nodes into several virtual nodes

I with a different number of coresI with different CPU performance

I Each virtual node can be used as a real Linux system

0 1 2 3 4 5 6 7

VN 1 VN 2 VN 3 Virtual node 4

CPU cores

CP

Upe

rform

ance

Distem – http://distem.gforge.inria.fr/ 10 / 17

Emulating network topologies

I Emulate several local networks linked togetherI Control available bandwidth and latency on each interface

n3

n1

n2

←5 Mbps, 10ms

10 Mbps, 5ms→if0

←1 Mbps, 30ms

1 Mbps, 30ms→

if0

←100 Mbps, 3ms

100 Mbps, 1ms→

if0

n4

n5

←4 Mbps, 12ms

6 Mbps, 16ms→if1

←10 Kbps, 200ms

20 Kbps, 100ms→

if0

←200 Kbps, 30ms

512 Kbps, 40ms→ if0

Distem – http://distem.gforge.inria.fr/ 11 / 17

Distem internals

Distem uses modern Linux featuresto steal resources from applications :

LXC, cgroups, cpufreq, iptables, traffic control

Node 1

Node 2

Node 3

Switch

Node 1

Node 2

Node 3

Switch

Distem – http://distem.gforge.inria.fr/ 12 / 17

User interfaces

I Command-line interface – distem command,,, Easy to use/ Harder to automate/ No access to more advanced features

I Ruby library, Easy to automate, Access to all features,// Easy to use if you known Ruby

I REST API, Can be used from any language/ Requires REST knowledge

Distem – http://distem.gforge.inria.fr/ 13 / 17

Overview of a Distem instance

User’s machine

Uses the command line interface,

the Ruby client library

or a REST client

Pnode 2

distemd

Pnode 3

distemd

Coordinator& Pnode 1

Starts and controls other Pnodes

Keeps the global state of the platform

Is the gateway to Pnodes and Vnodes

distemdREST

REST

VnodesDistem – http://distem.gforge.inria.fr/ 14 / 17

Example experiment : SCP vs RsyncI Transfer 50 files (total : 5 MB)I Available bandwidth : 10 MbpsI Varying network latency

0 20 40 60 80 1000

10

20

Emulated latency (ms)

Tran

sfer

time

(s)

SCPRsync

Distem – http://distem.gforge.inria.fr/ 15 / 17

Distem scales to thousands of virtual nodes

I Executing a command with ClusterShell and TakTuk

0 10,000 20,000 30,000 40,0000

200

400

600

Virtual nodes

Exe

cutio

ntim

e(s

)

ClusterShellTakTuk

Distem – http://distem.gforge.inria.fr/ 16 / 17

More information ?

http://distem.gforge.inria.fr/

Distem – http://distem.gforge.inria.fr/ 17 / 17