39
Benjamín Barán National University of Asuncion (UNA) [email protected] Paraguay DataCenter optimization for Cloud Computing

DataCenter optimization for Cloud Computing

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: DataCenter optimization for Cloud Computing

Benjamín Barán

National University of Asuncion (UNA)

[email protected]

Paraguay

Data

Cente

ropti

miz

ati

on

for

Clo

ud C

om

puti

ng

Page 2: DataCenter optimization for Cloud Computing

Conte

nt

2

�C

lou

d C

om

pu

tin

g

�C

om

mer

cial

Offe

rin

gs

�B

asic

Pro

ble

m F

orm

ula

tio

n

�O

pen

Res

earc

h

�C

on

clu

sio

ns

Page 3: DataCenter optimization for Cloud Computing

Clo

ud C

om

puti

ng

3

Cloud

com

putin

gis

anIn

tern

et-

base

dco

mp

uti

ng

inw

hic

hla

rge

gro

up

so

fre

-m

ote

serv

ers

are

net

wo

rked

toal

low

shar

ing

of

dat

ap

ro-

cess

ing

task

s,ce

n-

tral

ized

dat

ast

ora

-ge

and

on

-lin

eac

cess

toco

mp

ute

rse

rvi-

ces

or

reso

urc

es.

[htt

p:/

/en

.wik

iped

ia.o

rg/w

iki/

Clo

ud

_co

mp

uti

ng]

1-

Pu

blic

Clo

ud

2-

Pri

vate

Clo

ud

3-

Hyb

rid

Clo

ud

Page 4: DataCenter optimization for Cloud Computing

NIS

Tdefi

nit

ion o

f C

loud C

om

puti

ng

4Cloud

com

putin

gis

am

od

elfo

ren

ablin

gu

biq

uit

ou

s,co

n-

ven

ien

t,o

n-d

eman

dn

etw

ork

acce

ssto

ash

ared

po

ol

of

con

-fig

ura

ble

com

pu

tin

gre

sou

rces

(as,

netw

orks

,se

rver

s,stor

age,

applic

ations

,and

serv

ices

)th

atca

nb

era

pid

lyp

rovi

sio

ned

and

rele

ased

wit

hm

inim

alm

ana-

gem

ent

effo

rto

rse

rvic

ep

ro-

vid

erin

tera

ctio

n.

Th

isclou

dm

odel

isco

mp

ose

do

f5

esse

nti

alch

arac

teri

stic

s,an

d3

serv

ice

mo

del

s.

[htt

p:/

/csr

c.n

ist.

gov/

pu

blic

atio

ns/

nis

tpu

bs/

80

0-1

45

/SP

80

0-1

45

.pd

f]

Nat

iona

lIns

titut

eof

Sta

ndar

san

d Tec

hnolog

y(N

IST

)

Page 5: DataCenter optimization for Cloud Computing

Clo

ud C

om

puti

ng

5

�T

he

very

def

init

ion

of

clou

dco

mput

ing

still

rem

ain

sco

ntr

ove

rsia

l.

�T

her

ear

eal

tern

ativ

ed

efin

itio

nas

the

follo

win

go

ne:

Cloud

Com

put

ing

isthedynamicprovisioningofIT

capabilities

(hardware,software,orservices)fromthirdpartiesoveranetwork.

�C

lou

d c

om

pu

tin

g is

a com

put

ing

mod

el, n

ot

a te

chn

olo

gy. I

n t

his

m

od

el o

f co

mp

uti

ng,

al

l el

emen

ts (

pro

cess

ing,

sto

rage

, etc

.)

rela

ted

to

Dat

aCen

ters

are

mad

e av

aila

ble

to

en

d u

sers

via

th

e In

tern

et.

�V

irtu

aliz

atio

n-

as w

ell

as t

he

clo

ud

co

mp

uti

ng

mo

del

wit

hin

w

hic

h i

t o

ften

ru

ns

-an

swer

s m

uch

of

Dat

aCen

ters

nee

ds.

[http://

www.com

put

erwor

ld.com

/article/2

527305/c

loud

-com

put

ing/

clou

d-co

mput

ing-

defin

ition

s-an

d-so

lutio

ns.h

tml]

Page 6: DataCenter optimization for Cloud Computing

NIS

TServ

ice M

odels

6

Nat

iona

lIns

titut

eof

Sta

ndar

san

d Tec

hnolog

y

Page 7: DataCenter optimization for Cloud Computing

Every

thin

g/A

nyth

ing a

s a

Serv

ice -XaaS

7•B

Paa

S-

Bu

sin

ess

Pro

cess

as

a Se

rvic

e

•C

aaS

-C

om

mu

nic

atio

n a

s a

Serv

ice

•D

aaS

-D

ata

as a

Ser

vice

•Ia

aS-

Infr

astr

uct

ure

as

a Se

rvic

e

•IT

aaS

-IT

(In

form

ation Te

chno

logy

) as

a S

ervi

ce

•P

aaS

-P

latf

orm

as

a Se

rvic

e

•R

aaS

–R

eso

urc

es a

s a

Serv

ice

•Sa

aS-

Soft

war

e as

a S

ervi

ce

•SE

Caa

S-

SEC

uri

tyas

a S

ervi

ce

Page 8: DataCenter optimization for Cloud Computing

Infr

aestr

uctu

re a

s a

Serv

ice -

IaaS

8Infr

astr

uct

ure

as

a Se

rvic

e –

IaaS

, pro

vid

es g

rid

s o

r cl

ust

ers

or

virt

ual

ized

ser

vers

, net

wo

rks,

sto

rage

an

d

syst

ems

soft

war

e d

esig

ned

to

au

gmen

t o

r re

pla

ce t

he

fun

ctio

ns

of

an e

nti

re D

ataC

ente

r.

Th

e h

igh

est-

pro

file

exam

ple

is Am

azon

's E

lastic

Com

pute

Cloud[EC2]

and

Sim

ple

Stor

age

Serv

ice [S3]

, b

ut

oth

er t

rad

itio

nal

IT

ven

do

rs a

re a

lso

offe

rin

g se

rvic

es.

[htt

p:/

/ww

w.c

om

pu

terw

orl

d.c

om

/art

icle

/25

27

30

5/c

lou

d-c

om

pu

tin

g/cl

ou

d-c

om

pu

tin

g-d

efin

itio

ns-

and

-so

luti

on

s.h

tml]

Page 9: DataCenter optimization for Cloud Computing

IaaSGartner

Magic

Quadra

nt

9

[htt

p:/

/aw

s.am

azo

n.c

om

/res

ou

rces

/gar

tner

-m

q-2

01

4-l

earn

-m

ore

/?sc

_ic

ou

ntr

y=en

&sc

_ic

han

nel

=h

a&sc

_id

etai

l=h

a_en

_4

2&

sc_

ico

nte

nt=

ha_

en_

d_

ed_

42

_1

&sc

_ip

lace

=h

a_en

_ed

&sc

_ic

amp

aign

=h

a_en

_G

artn

er&

trk=

/]

Page 10: DataCenter optimization for Cloud Computing

AW

S –

Am

azon W

eb S

erv

ices

10

Page 11: DataCenter optimization for Cloud Computing

Case S

tudy: using A

WS

11

Page 12: DataCenter optimization for Cloud Computing

Com

panie

s u

sin

g

Public Cloud Computing

12

[htt

p:/

/sp

ectr

um

.ieee

.org

/co

mp

uti

ng/

net

wo

rks/

esca

pe-

fro

m-t

he

-dat

a-ce

nte

r-th

e-p

rom

ise-

of-

pee

rto

pee

r-cl

ou

d-c

om

pu

tin

g/?u

tm_

sou

rce=

tech

aler

t&u

tm_

med

ium

=em

ail&

utm

_ca

mp

aign

=0

92

51

4]

Page 13: DataCenter optimization for Cloud Computing

Cost

Models

13

�Static

: fix

ed p

rice

s (r

eso

urc

e p

rice

s ra

rely

ch

ange

in

tim

e, a

s tr

adit

ion

al A

maz

on

EC

2)

�Dynamic Prices. R

eso

rce

pri

ces

fluct

uat

es o

n d

eman

d

on

a d

ay o

r w

eekl

y b

asis

(e.

g., w

eeke

nd p

rice

s ar

e differ

ent)

.

�Spot Prices. I

t is

bas

ed o

n u

ser´

s b

ids.

If u

ser bid

met

or ex

ceed

the

cur

rent

spo

t pr

ice, h

e ga

ins

acce

ss to

requ

este

d re

sour

ces (a

s ne

w A

maz

on E

C2).

Page 14: DataCenter optimization for Cloud Computing

14

1 y

ear

Pri

ces E

xam

ple

INSTANCE

CPU

ECU

RAM [GiB]

Storage

[GB]

Price per

hour

t2.micro

1V

aria

ble

1E

BS

$0

.01

3

t2.small

1V

aria

ble

2E

BS

$0

.02

6

t2.medium

2V

aria

ble

4E

BS

$0

.05

2

m3.medium

13

3.7

51

x 4

SSD

$0

.07

0

m3.large

26

.57

.51

x 3

2 S

SD$

0.1

40

m3.xlarge

41

31

52

x 4

0 S

SD$

0.2

80

m3.2xlarge

82

63

02

x 8

0 S

SD$

0.5

60

EC

U …

EC

2C

om

pu

tin

g U

nit

(e

.g. 1

EC

U =

1.0

-1.2

GH

z 2

00

7 X

eon

)

EB

S…

Ela

stic

Blo

ck S

tora

ge (

$0

.10

per

GB

-mo

nth

)

SSD

… S

olid

Sta

te D

rive

, in

tern

al s

tora

ge

Page 15: DataCenter optimization for Cloud Computing

15

htt

ps:

//aw

s.am

azo

n.c

om

/mar

ketp

lace

/sea

rch

/res

ult

s/re

f=m

kt_

ste_

free

_ti

er_

ec2

?pag

e=1

&re

stri

ctio

n=

%2

8o

r+as

in%

3A

%2

7B

00

AA

27

RK

4%

27

+as

in%

3A

%2

7B

00

A6

KU

VB

W%

27

+as

in%

3A

%2

7B

00

7O

RSS

8I%

27

+as

in%

3A

%2

7B

00

AA

ESF

K8

%2

7+

asin

%3

A%

27

B0

07

O0

H3

5O

%2

7+

asin

%3

A%

27

B0

06

35

Y2

IW%

27

+as

in%

3A

%2

7B

00

7Z

5Y

WX

4%

27

%2

9

AMI:

Amazon Machine Images

Pri

ces E

xam

ple

Page 16: DataCenter optimization for Cloud Computing

Spot

Pri

ce e

xam

ple

16

SeeTUTORIALSat:

[h

ttp

://a

ws.

amaz

on

.co

m/e

c2/p

urc

has

ing-

op

tio

ns/

spo

t-in

stan

ces/

]

INSTANCE

LINUX

WINDOWS

m1.small

$0

.00

71

per

Ho

ur

$0

.01

71

per

Ho

ur

m1.medium

$0

.00

81

per

Ho

ur

$0

.03

31

per

Ho

ur

m1.large

$0

.01

61

per

Ho

ur

$0

.06

61

per

Ho

ur

m1.xlarge

$0

.03

52

per

Ho

ur

$0

.13

21

per

Ho

ur

Page 17: DataCenter optimization for Cloud Computing

Clo

ud C

om

puti

ng T

rend

17

htt

p:/

/ww

w.c

isco

.co

m/c

/en

/us/

solu

tio

ns/

colla

tera

l/se

rvic

e-p

rovi

der

/glo

bal

-clo

ud

-in

dex

-gci

/Clo

ud

_In

dex

_W

hit

e_P

aper

.pd

f

Page 18: DataCenter optimization for Cloud Computing

18

Clo

ud C

om

puti

ng T

rend

htt

p:/

/ww

w.c

isco

.co

m/c

/en

/us/

solu

tio

ns/

colla

tera

l/se

rvic

e-p

rovi

der

/glo

bal

-clo

ud

-in

dex

-gci

/Clo

ud

_In

dex

_W

hit

e_P

aper

.pd

f

1 Z

B =

10

21

BC

AG

R…

Co

mp

ou

nd

An

nu

al G

row

th R

ate

Page 19: DataCenter optimization for Cloud Computing

Vir

tualizati

on

19

Page 20: DataCenter optimization for Cloud Computing

Vir

tualizati

on

20

htt

p:/

/ww

w.g

artn

er.c

om

/rep

rin

ts/v

mw

are-

vol4

?id

=1

-1G

RG

RR

U&

ct=

13

07

02

&st

=sb

Page 21: DataCenter optimization for Cloud Computing

Vir

tualizati

on e

xam

ple

: VM

ware

21

DR

S…

Dis

trib

ute

dR

eso

urc

eSc

hed

ule

r

HA

… H

igh

Ava

ilab

ility

SMP

… S

ymm

etri

cM

ult

i-P

roce

ssin

g

ESX

… E

last

icsk

yX

ser

ver

VM

FS…

Vir

tual

Mac

hin

e Fi

le S

yste

m

Page 22: DataCenter optimization for Cloud Computing

Basic

Pro

ble

m F

orm

ula

tion

22

Page 23: DataCenter optimization for Cloud Computing

Vir

tual M

achin

e P

lacem

ent

23

Virtual Infrastructure

Wh

ich

vir

tual

mac

hin

es s

ho

uld

be

loca

ted

at

each

phy

sica

l m

ach

ine?

Und

er w

hich

crite

ria?

Page 24: DataCenter optimization for Cloud Computing

Obje

cti

ve F

uncti

ons

�M

ain

ob

ject

ive

fun

ctio

ns

[3]

[F.

pez

Pir

es,

B.

Bar

án,

“Tax

onom

yof

Optim

alVirtu

alM

achine

Plac

emen

tin

Efficient

Dat

acen

ters

,”IE

EE

Ara

nd

uco

n’

20

12

]

(1)

En

ergy

Co

nsu

mp

tio

n M

inim

izat

ion

(2)

Eco

no

mic

al R

even

ue

Max

imiz

atio

n

(3)

Net

wo

rk T

raff

ic M

inim

izat

ion

�M

ath

emat

ical

fo

rmu

lati

on

wit

ho

ut

SLA

[4]

[F.

pez

Pir

es,

B.

Bar

án,

“Multi-

Objec

tive

Virtu

alM

achine

Plac

emen

twith

Serv

ice

Leve

lAgr

eem

ent,”

6th

IEE

E/A

CM

Inte

rnat

ion

alC

on

fere

nce

on

Uti

lity

and

Clo

ud

Co

mp

uti

ng,

UC

C’2

01

3.

Dre

sden

–A

lem

ania

]

24

Page 25: DataCenter optimization for Cloud Computing

Physic

al R

esourc

es M

atr

ix

25

��

�������

������ ��

�������

������ ��

⋮�����

⋮��

�⋮

�����

⋮ ��

��1

��2

���

wh

ere:

�: :::N

um

ber

of

phy

sica

l mac

hin

es

� �:

Vir

tual

mac

hin

e w

ith

iden

tific

atio

n�

�����

: P

roce

ssin

g re

sou

rce

of

the

phy

sica

l mac

hin

e� �in

[MIP

S]

��

�:

RA

M m

emo

ryre

sou

rce

of

the

phy

sica

l mac

hin

e � �in

[MB

]

�����

: S

tora

ge r

eso

urc

e o

fth

e p

hysi

cal m

ach

ine� �i iiin

[GB

]

��

: M

axim

um

po

wer

co

nsu

mp

tio

n o

f th

ep

hysi

cal m

ach

ine� �in

[W]

Page 26: DataCenter optimization for Cloud Computing

wh

ere:

:

Nu

mb

er o

f vi

rtu

al m

ach

ines

� �:

Vir

tual

mac

hin

e w

ith

iden

tific

atio

n�

�����

: P

roce

ssin

g re

qu

irem

ent

of

the

virt

ual

mac

hin

e� �in

[MIP

S]

���

: R

AM

mem

ory

req

uir

emen

t o

f th

e vi

rtu

al m

ach

ine� �

in [

MB

]

�����

: S

tora

ge r

equ

irem

ent

of

the

virt

ual

mac

hin

e� �in

[GB

]

� �:

Eco

no

mic

al r

even

ue

for

pla

cem

ent

of

virt

ual

mac

hin

e� �in

[$]

��� �

: S

ervi

ce le

vel a

gree

men

t o

f vi

rtu

al m

ach

ine� �

26���������

������� �

��� �

�������

������� �

��� �

⋮����

⋮��

����

⋮⋮

� ���

��1

��2

��

Vir

tual R

equir

em

ent

Matr

ix

Page 27: DataCenter optimization for Cloud Computing

Netw

ork

Tra

ffic

Matr

ix

27

!�! ��

! ��…! �

! ��! ��

…! �

⋮ ! �

⋮ ! �⋱ …

⋮ !

��1

��2

��

wh

ere:

:

Nu

mb

er o

f vi

rtu

al m

ach

ines

� �:

Vir

tual

mac

hin

e w

ith

iden

tific

atio

n�

� $:

Vir

tual

mac

hin

e w

ith

iden

tific

atio

n%

! �$:

Net

wo

rk C

om

mu

nic

atio

n r

ate

bet

wee

n � �

and

� $in

[Kb

ps]

%�1%�

2…%�

Page 28: DataCenter optimization for Cloud Computing

* T

he

pro

po

sed

pro

ble

m f

orm

ula

tio

n c

on

sid

ers

on

ly s

tati

c co

nte

xts

Basic

Pro

ble

m F

orm

ula

tion

��4'�

�� 5'

!�'

)�'�

)�*) ��+

wh

ere) ��∈*0,1+

() ���0)

ind

icat

esth

at� �

ISN

OT

loca

ted

in� �

() ���1)

ind

icat

esth

at� �

ISlo

cate

din� �()��

:� ��� �

)

INPUT

OUTPUT

28

Page 29: DataCenter optimization for Cloud Computing

Pla

cem

ent

Matr

ix

29)�

100

100

100

010

010

010

010

001

000

) ��: � �

�� �

) ��: � �

�� �

) /�: � /

�� �

) 0�: � 0

�� �

) 1�: � 1

�� �

) 2�: � 2

�� �

) 3�: � 3

�� �

) 4/: � 4

�� /

� 5�6�7866�9�:�

Page 30: DataCenter optimization for Cloud Computing

Constr

ain

ts

30

�U

niq

ue

pla

cem

ent

of

virt

ual

mac

hin

es

;) ��<1∀�∈*1,2,…,+

� �>�

Constraint 1

wh

ere:

�: :::N

um

ber

of

phy

sica

l mac

hin

es

) ��:

Bin

ary

vari

able

eq

ual

s 1

if

the

virt

ual

mac

hin

e � �

is lo

cate

d t

o r

un

on

the

phy

sica

l mac

hin

e � �

;0 o

ther

wis

e

:

Nu

mb

er o

f vi

rtu

al m

ach

ines

Page 31: DataCenter optimization for Cloud Computing

Constr

ain

ts

31

�Se

rvic

e Le

vel A

gree

men

t (S

LA)

pro

visi

on

Constraint 2

wh

ere:

�: :::N

um

ber

of

phy

sica

l mac

hin

es

) ��:

Bin

ary

vari

able

eq

ual

s 1

if

the

virt

ual

mac

hin

e � �

is lo

cate

d t

o r

un

on

the

phy

sica

l mac

hin

e � �

; 0 o

ther

wis

e

��� �

:Se

rvic

e L

evel

Agr

eem

ent ��� �

= 1

if � �

is c

riti

cal,

or

0 o

ther

wis

e

;) ���1∀�6���8�8�����1

� �>�

Page 32: DataCenter optimization for Cloud Computing

Constr

ain

ts

32

�R

eso

urc

e ca

pac

ity

of

phy

sica

l m

ach

ines

;�����') ��<�����

�>�

;���') ��<��

� �>� ;�����') ��<�����

�>�

Constraint 3

Constraint 4

Constraint 5

wh

ere: �����

:P

roce

ssin

g re

qu

irem

ent

[MIP

S] o

f v

irtu

al m

ach

ine� �

���

: R

AM

mem

ory

req

uir

emen

t [M

B]

of

vir

tual

mac

hin

e� �

�����

: S

tora

ge r

equ

irem

ent

[GB

] o

f v

irtu

al m

ach

ine� �

Page 33: DataCenter optimization for Cloud Computing

Mult

i-O

bje

cti

ve M

em

eti

c A

lgori

thm

33

�C

hro

mo

som

e re

pre

sen

tati

on

Solutio

n?�

100

100

100

010

010

010

010

001

001

Proposed Form

ulation

Proposed Chromosome Representation

Page 34: DataCenter optimization for Cloud Computing

Mult

i-O

bje

cti

ve M

em

eti

c A

lgori

thm

34

Initialization

Reparation

Local Search

Population Evolution

Pareto Set

Stop

Criteria?

No

Yes

Crossover and Mutation

Reparation

Local Search

Pareto Set Update

Selection

Page 35: DataCenter optimization for Cloud Computing

Experi

menta

l R

esult

s

35

�Te

stin

g E

nvir

on

men

t

�A

lgo

rith

ms

in A

NSI

C (

GN

U C

)

�G

NU

/Lin

ux

Ub

un

tu 1

1.1

0 O

per

atin

g Sy

stem

�In

tel

Co

re i

7 d

e 1

.2 G

Hz

Pro

cess

or

�8

GB

of

RA

M M

emo

ry

�R

eal

Inp

ut

Dat

a

Page 36: DataCenter optimization for Cloud Computing

Experi

menta

l R

esult

s

36

�E

xp

erim

enta

l Tes

t 1

:

�E

xh

aust

ive

sear

chal

gori

thm

can

no

tco

mp

lete

calc

ula

tio

nin

use

fult

ime

.

�It

isn

eces

sary

toim

ple

men

tal

tern

ativ

esto

exh

aust

ive

sear

ch.

@ ABCDB:

Kn

ow

n P

aret

o F

ron

tE ABCDB:

Kn

ow

n P

aret

o S

et

Scenario

Number

of

Physical

Machines

Number

of

Virtual

Machines

Critical SLA

Percentage

Number

of@ABCDB

Elements

Number

ofEABCDB

Elements

10

x2

01

02

05

0%

48

48

Page 37: DataCenter optimization for Cloud Computing

Experi

menta

l R

esult

s

37

�E

xp

erim

enta

l Tes

t 2

:

�R

elat

ion

of

vari

able

s:

�Execution Time

and

Critical SLA Percentage

�Number of Solutions

and

Critical SLA Percentage

Scenario

Number of

Physical Machines

Number of

Virtual Machines

Critical SLA

Percentage

3x

53

50

, 10

, 20

, 30

, 40

, 50

, 6

0, 7

0, 8

0,9

0, 1

00

%

4x

10

41

00

, 10

, 20

, 30

, 40

, 50

, 6

0, 7

0, 8

0,9

0, 1

00

%

Page 38: DataCenter optimization for Cloud Computing

Futu

re W

ork

38

�A

lter

nat

ive

form

ula

tio

ns

for

the

pro

ble

m:

�C

on

sid

erin

gm

ore

SLA

leve

lsan

dco

nst

rain

s(a

sge

ogr

aph

ical

)

�C

on

sid

erin

gm

ore

SLA

met

rics

:res

pons

etim

e,jitte

r,et

c.

�Fo

rmu

lati

on

wit

ho

ther

ob

ject

ive

fun

ctio

ns

(mor

eth

an80

differ

entob

ject

ive

func

tions

wer

efo

und

inth

esp

ecializ

edliter

atur

e).

�Te

stin

go

ther

bio

-in

spir

edm

eta-

heu

rist

ic,

give

nth

en

ove

lty

of

the

pro

po

sed

con

tex

t.

�P

ure

Dyn

amic

alC

on

tex

tan

dit

su

nce

rtai

nty

.

�U

seo

fa

thir

d-p

arty

Bro

ker.

�C

on

sid

erH

ybri

dcl

ou

ds.

�C

ase

stu

die

san

dco

mm

erci

alap

plic

atio

ns.

Page 39: DataCenter optimization for Cloud Computing

Thanks!

39

Benjamín Barán

National University of Asuncion (UNA)

[email protected]

Paraguay