Download pdf - an introduction to Distem

an introduction to

Distem – http://distem.gforge.inria.fr/ 1 / 17

Imagine you want to build a race car


But you want it to work well

on both dry tracks and wet tracks


You could use a simulation :

model the car and the trackand compute how they would interact


Or you could use a real world experiment :

build a car, and try it on a real circuit

But sometimes, rain is unlikely


Or you could use a real world experiment :

build a car, and try it on a real circuit

But sometimes, rain is unlikely


You could move to another race track . . .But what if you could change the weather ?

That’s what Distem does :Take a platform, and alter it to suit

your experimentation needs


You could move to another race track . . .But what if you could change the weather ?

That’s what Distem does :Take a platform, and alter it to suit

your experimentation needsDistem – http://distem.gforge.inria.fr/ 6 / 17

An emulator for distributed systems

Take your real application

Run it on a cluster

And use Distem to alter the platformso it matches the experimental conditions you need

Heterogeneous nodesLong distance networksFaults, perf. variations

Grid, Cloud, P2P features. . .


Ideal complement to other toolsSimGrid

Toolkit for the simulation ofdistributed applications in

heterogeneous environments

Grid’5000Large-scale platformfor experiments ondistributed systems

1700 nodes, 7000 coresFully reconfigurable

Grid’5000

Distem can run on Grid’5000


Ideal complement to other toolsSimGrid

Toolkit for the simulation ofdistributed applications in

heterogeneous environments

Grid’5000Large-scale platformfor experiments ondistributed systems

1700 nodes, 7000 coresFully reconfigurable

Grid’5000

Distem can run on Grid’5000Distem – http://distem.gforge.inria.fr/ 8 / 17

What can Distem do for you ?

I Introduce heterogeneity in an homogeneous clusterHow does your app perform when some nodes are slower ?

I Emulate complex network topologiesHow does your app perform on a Grid ? on a slower network ?

I Inject faults and performance variations during the experimentHow does your app behave when a node crashes ?When the available CPU time decreases ?When the available network bandwidth increases ?


Introducing heterogeneity

I Distem splits real nodes into several virtual nodes

I with a different number of coresI with different CPU performance

I Each virtual node can be used as a real Linux system

0 1 2 3 4 5 6 7

VN 1 VN 2 VN 3 Virtual node 4

CPU cores

CP

Upe

rform

ance


Emulating network topologies

I Emulate several local networks linked togetherI Control available bandwidth and latency on each interface

n3

n1

n2

←5 Mbps, 10ms

10 Mbps, 5ms→if0

←1 Mbps, 30ms

1 Mbps, 30ms→

if0

←100 Mbps, 3ms

100 Mbps, 1ms→

if0

n4

n5

←4 Mbps, 12ms

6 Mbps, 16ms→if1

←10 Kbps, 200ms

20 Kbps, 100ms→

if0

←200 Kbps, 30ms

512 Kbps, 40ms→ if0


Distem internals

Distem uses modern Linux featuresto steal resources from applications :

LXC, cgroups, cpufreq, iptables, traffic control

Node 1

Node 2

Node 3

Switch

Node 1

Node 2

Node 3

Switch


User interfaces

I Command-line interface – distem command,,, Easy to use/ Harder to automate/ No access to more advanced features

I Ruby library, Easy to automate, Access to all features,// Easy to use if you known Ruby

I REST API, Can be used from any language/ Requires REST knowledge


Overview of a Distem instance

User’s machine

Uses the command line interface,

the Ruby client library

or a REST client

Pnode 2

distemd

Pnode 3

distemd

Coordinator& Pnode 1

Starts and controls other Pnodes

Keeps the global state of the platform

Is the gateway to Pnodes and Vnodes

distemdREST

REST

VnodesDistem – http://distem.gforge.inria.fr/ 14 / 17

Example experiment : SCP vs RsyncI Transfer 50 files (total : 5 MB)I Available bandwidth : 10 MbpsI Varying network latency

0 20 40 60 80 1000

10

20

Emulated latency (ms)

Tran

sfer

time

(s)

SCPRsync


Distem scales to thousands of virtual nodes

I Executing a command with ClusterShell and TakTuk

0 10,000 20,000 30,000 40,0000

200

400

600

Virtual nodes

Exe

cutio

ntim

e(s

)

ClusterShellTakTuk


More information ?

http://distem.gforge.inria.fr/
