Gordon h 0945 Deep Dive-openstack Compute

7/25/2019 Gordon h 0945 Deep Dive-openstack Compute

1/69

DEEP DIVE: OPENSTACK COMP

Stephen Gordon


2/69

AGENDA

OpenStack architecture refresher

Compute architecture

Instance life cycle

Scaling compute

Segregating computeUpcoming features


3/69

OPENSTACK MISSION STATEMENT

To produce the ubiquitous Open Source Cloud C

platform that will meet the needs of public and privregardless of size, by being simple to impleme

massively scalable.


4/69

OPENSTACK A$C%ITECTU$E

NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

Modular architecture

esign to easily scale out

!ased on "gro#ing$ set of core ser%ices


5/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack ash&oard "Hori'on$

Pro%ides simple self(ser%ice user interface for end users

!asic cloud administrator functions


6/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack Identity ")eystone$

Common authori'ation frame#ork ( manages users, tenants, endpoints

Plugga&le &ackends "S*+, PM, +P, etc$


7/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

Telemetry "Ceilometer$

Monitors, collects, and stores usage data for all OpenStack in

Primary targets metering and monitoring #ith e-panda&le fram


8/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

Orchestration "Heat$Template(&ased deployment orchestrator for cloud application

utomates deployment of compute, storage, and net#orking

Pro%ides .S Cloud/ormation implementation for OpenStac


9/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack !lock Storage "Cinder$

Pro%ides &lock storage for %irtual machines "persistent disks$

Similar to ma'on 0!S ser%ice, plugin architecture for %endo


10/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack 1et#orking "1eutron$

Pro%ides frame#ork for Soft#are efined 1et#ork "S1$

Plugin architecture allo#s integration of hard#are and soft#anet#ork solutions


11/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack O&2ect Storage "S#ift$

Modeled after ma'on3s S4 ser%ice

Pro%ides simple ser%ice for storing and retrie%ing ar&itrary dat


12/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack Image Ser%ice "5lance$

Stores and retrie%es disk images "%irtual machine templates$

Supports Ra#, *CO., 6M), 6H, ISO, O6/ 7 MI8)I

!ackend storage 9 /ilesystem, S#ift, ma'on S4


13/69


NOVA

IMAGESERVICE

OBJECTSTORE

VOLUMESERVICE

GLANCE SWIFT CINDER

COMPUTE NETWORKING

NEUTRON HEAT

Orchesr!"#$

CEILO

Te%e

DASHBOARD

HORI(ON

OpenStack Compute "1o%a$

Schedules, and manages the life cycle of %irtual machine instsupported hyper%isors

1ati%e OpenStack PI and ma'on 0C: compati&le PI


14/69

COMPUTE A$C%ITECTU$E


15/69

COMPONENTS


16/69

OT%E$ COMPONENTS

Certificate manager for -; e?g?9

ne+tron'open&s)(th'agent

ne+tron',(n+#-r(dge'agent

Ceilometer agent > opensta*'e(,o!eter'o!p+te


17/69

OT%E$ COMPONENTS

Token %alidator for console access > opensta*'no&a'

onso,ea+thConsole pro-ies9

1o61C pro-y > opensta*'no&a'no&npro#

SPIC0 pro-y > opensta*'no&a'sp(eht!,/pro#

@a%a client pro-y > opensta*'no&a'#&p&npro#


18/69

0A11 IN ONE2

ll compute ser%ices co(reside #ith9

Shared ser%ices "Identity, Image Ser%ice,!lock Storage, 1et#orking$

Support ser%ices "data&ase, message&roker$

no

nova-

nova-c

nova-


19/69

0CONT$O11E$ 3 COMPUTE2

ll compute control ser%ices "PI,

scheduler, conductor$ co(reside #ith9Shared ser%ices "Identity, Image Ser%ice,!lock Storage, 1et#orking$

Support ser%ices "data&ase, message&roker$

Compute agent runs on dedicatednode as #ell as9

+: net#orking agent

Ceilometer compute agent "optional$

n

nova-

nova-

nova

Con r

Com


20/69

0CONT$O11E$ 3 COMPUTE2

nova-

nova-cond

nova-sche

Controller

nova-compute

Compute Node

0-pand &y adding more compute

nodesTypical deployments start &reaking offinto dedicated9

1et#ork nodes

Storage nodes0tc?


21/69

A$C%ITECTU$E

OpenStack Compute itself can also &e &roken off into d

PI nodes

Conductors

Schedulers

Compute 1odes

0tc?Same approach applies to other pro2ects?


22/69

RED HAT ENTERPRISE LINU !PENSTAC" PLAT#!R$DOC144908

ssurance that 4rdparty products ha%e &een tested #ith anon Red Hat 0nterprise +inu- OpenStack Platform

ocumented !est practices, Installation, configuration, kn

Colla&orati%e support agreements &et#een Red Hat and 2ointly sol%e customer issues

lignment on S+, +ifecycle, roadmap and upstream eng

SUPPO$TED D$IVE$S


23/69

SUPPO$TED D$IVE$S


24/69

SUPPO$TED D$IVE$S 4 KVM

1o%a Controller 1oRH0+(OSP

PI

)6MCompute1odes


25/69

SUPPO$TED D$IVE$S 4 VM)are &Center

1o%a Controller 1oRH0+(OSP

%Center Ser%er

PI

0SAiComputeClusters

#ith 6M#are 1SA


26/69

INSTANCE 1I5E C6C1E


27/69

AUT%ENTICATION

1o%a o&tains an authentication token on your &ehalf fro

)eystoneCredentials pro%ided #hen user logs in to dash&oard oen%ironment %aria&les for command line9

$ source keystonerc_admin


28/69

AUT%ENTICATION


29/69

INSTANCE $E7UEST

Initiating creation of an instance using the command lin

$ nova boot --flavor 2 \

--image 174e7a3a... \

"My Instance"

/la%or B: correlates to mD?small9

D %CPU : 5 RM, D< 5 root disk, :< 5 ephemeral dis


30/69

INSTANCE $E7UEST

Instance 1ame

/la%or

Image


31/69

INSTANCE 51AVO$S

/la%ors define instance si'es9

1um&er of %CPUs

Memory

Root disk space

0phemeral disk space

r&itrary e-tra specifications

/i%e default fla%ors a%aila&le, are customi'a&le or moreadded?


32/69

API

Initial reEuest hits Compute PI in AM+ or @SO1 forma

0ndpoint like http://192.168.122.161:8774/v2/%(te

Parameters are e-tracted from PI call for &asic %alidation

Some no%a e-tensions are called "e?g? os-config-drive$

Retrie%es a reference to the selected fla%or


33/69

API

Identifies &oot media9

Retrie%es a reference to the selected image using 5lance clie

Retrie%es a reference to the selected %olume using Cinder cli

Performs further parameter %alidation and applies defaoptional parameters

Sa%es instance state to data&asePuts a message on the Eueue for the scheduler


34/69

SC%EDU1E$

efaults to filter scheduler

pplies filters and #eights &ased on no&a.on8settings

/ilters, e?g?9

Compute filter > is this host onG

Core filter > does this host ha%e enough %CPUs a%aila&leG

.eights, e?g?9

RM #eigher > gi%e more preference to hosts #ith more or less RM


35/69

5I1TE$ SC%EDU1E$

Host D

Host :

Host 4

5

I

1

T

E

$S

9

E

I

G

%

TS


36/69

SC%EDU1E$

Updates instance data in data&ase

Puts a message on the Eueue for the compute agent oselected compute node


37/69

COMPUTE NODE

Updates instance state in data&ase

Retrie%e the &oot media information

ecode any in2ected files

Calls 1eutron to get net#ork and security group informBplug %irtual interfaces

Calls Cinder to attach %olume if necessary

Sets up configuration dri%e if necessary


38/69

COMPUTE NODE

Use hyper%isor PIs to create %irtual machine


39/69

SCA1ING COMPUTE


40/69

COMPONENTS

So compute is simple to implement,

&ut ho# to make it massivelyscalableG


41/69

SCA1ING LOAD BALANCER

MESSAGE)UEUE

SCHEDULER

API

DATABASE

CONDUCTORAM)P

dd a load &alancer in front of

the PI? dd additional9

Conductors

Schedulers

Compute agents8hyper%isors Scale out message &rokers anddata&ases using documentedmechanisms

CE11S


42/69

CE11S

Maintains a single compute

endpointRelie%e pressure on Eueuesdata&ase at scale "


43/69

API CE11

dds a load &alancer in front of

multiple instances of the PI ser%iceHas its o#n message Eueue

Includes a ne# ser%ice, no&a'e,,s

Handles cell scheduling

Packaged as opensta*'no&a'e,,sReEuired in e%ery cell

MESSAGE)UEUE

API

CELLS$#-!.ce%%s

LOAD BALAN

$#-!.!/"

COMPUTE CE11


44/69

COMPUTE CE11

0ach compute cell contains9

Its o#n message Eueue anddata&ase

Its o#n scheduler, conductor,compute nodes

MESSAGE)UEUE

SCHEDULER

CELLS

DATABASE

CONDUCTOR$#-!.c#$12c#r

$#-!.sche12%er

$#-!.ce%%s


45/69

SEG$EGATING COMPUTE

9%6 SEG$EGATE COMPUTE


46/69

9%6 SEG$EGATE COMPUTE

0-pose logical groupings of compute resources

5eographical region, data center, rack, po#er source, net#or0-pose special capa&ilities of compute resources

/aster 1ICs or storage, special de%ices, etc?

The di%isions mean #hate%er you #ant them to mean

$EGIONS


47/69

$EGIONS

Complete OpenStack deployments

Share a )eystone and Hori'on installation Implement their o#n targeta&le PI endpoints,net#orks, and compute

!y default all ser%ices in one region (

$ keystone endpoint-create --regionRegionTwo

Target actions at a region3s endpoint9

$ nova --os-region-name RegionTwoboot

$EGIONS


48/69

$EGIONS

Re4"#$ A Re4"#$ BKe's#$e

H#r"#$

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

%OST AGG$EGATES


49/69

%OST AGG$EGATES

+ogical groupings of hosts &ased on metadata

Typically metadata descri&es capa&ilities hosts e-poseSS hard disks for ephemeral data storage

PCI de%ices for passthrough

0tc?

Hosts can &e in multiple host aggregates9 BHosts that ha%e SS storage and D


50/69

%OST AGG$EGATES

I!p,((t,user targeta&le9

dmin defines host aggregate #ith metadata, and a fla%or tha$ nova aggregate-create hypervisors-with-SSD

$ nova aggregate-set-metadata 1 SSDs=true

$ nova aggregate-add-host 1 hypervisor-1

$ nova flavor-key 1 set SSDs=true

User selects fla%or #hen reEuesting instanceScheduler places on host aggregate matching host aggregate

AggregateInstanceExtraSpecsFilter

AVAI1A;I1IT6 ONES


51/69

AVAI1A;I1IT6 ONES

+ogical groupings of hosts &ased on ar&itrary factors lik

+ocation "country, data center, rack, etc?$1et#ork layout

Po#er source

E#p,((t,user targeta&le9

$ nova boot --availability-zone rack-1

AVAI1A;I1IT6 ONES


52/69

AVAI1A;I1IT6 ONES

%aila&ility one

AVAI1A;I1IT6 ONES


53/69

AVAI1A;I1IT6 ONES

Host aggregates are made e#p,((t,user targeta&le &

them as an 9$ nova aggregate-create tier-1 us-east-tier-1

tier-1is the aggregate name, us-east-tier-1is the

Host aggregate (sthe a%aila&ility 'one in this case

Hosts annot&e in multiple a%aila&ility 'ones

Hosts an&e in multiple host aggregates

E=AMP1E


54/69

E=AMP1E


H#r"#$

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

E=AMP1E


55/69

E=AMP1E


H#r"#$

A( 3 A( 8

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

A( 9 A( :

E=AMP1E


56/69

E=AMP1E


H#r"#$

A( 3 A( 8

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

A( 9 A( :

H#ss 6"h SSDs

E=AMP1E


57/69

E=AMP1E


H#r"#$

A( 3 A( 8

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

A( 9 A( :

H#ss 6"h SSDsH#ss 6"h 30G NICs

E=AMP1E


58/69

E=AMP1E

Re4"#$ A Re4"#$ ;Ke's#$e

H#r"#$

A( 3 A( 8

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

A( 9 A( :

H#ss 6"h SSDsH#ss 6"h 30G NICsH#ss 6"h GPUs

E=AMP1E


59/69

E=AMP1E

Re4"#$ A Re4"#$ ;Ke's#$e

H#r"#$

A( 3 A( 8

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

G%!$ce

C"$1er

N#-!

Ne2r#$

S6"7

A( 9 A( :

H#ss 6"h SSDsH#ss 6"h 30G NICsH#ss 6"h GPUs


60/69

NE9 5EATU$ES

$O11ING UPG$ADES


61/69

Impro%ed %ersion interopera&ility

/acilitates Brolling upgradesUpgrade control ser%ices independently of compute nodes

llo#s operators to take a more gradual approach to upan OpenStack cloud?

INSTANCE G$OUPS API


62/69

llo#s cloud users to create and apply policies to groupinstances "also referred to as ser%er groups$

llo#s intelligent placement of #orkloads that interact #other

pply scheduling policies to instance groups9

ffinity

nti(ffinity

NOTI5ICATIONS


63/69

dditional notifications for9

Compute host lifecycle operations9Shutdo#n

Re&oot

Maintenance mode "in8out$

Creation and deletion of keypairs?

Support enhanced monitoring from Telemetry "Ceilome

VI$TIO $NG


64/69

Para(%irtuali'ed random num&er generator?

efault entropy pool is 3de&3rando!on compute nodeUse of a hard#are random num&er generator "R15$ or entrodaemon "05$ also supported

llo#s cloud users to run #orkloads reEuiring random dcryptographic applications$ #hile a%oiding guest entrop

star%ation

VI$TIO 9ATC%DOG


65/69

Triggers instance lifecycle e%ents for +inu- guests on cpanic

0na&led using h)>)athdog>at(onimage property?

h)>)athdog>at(on?


66/69

Para(%irtuali'ed SCSI controller?

esigned as a future successor to 6irtIO !lock9 Impro%ed scala&ility > instances can connect to more storage

Standard command set > uses standard SCSI command, sime-pansion

Standard de%ice naming > disks use same paths as &are(met

0na&led using h)>d(s*>-+s>!ode,image property9

h)>d(s*>-+s>!ode,?&(rt(o'ss(

VCENTE$ D$IVE$


67/69

1o# supports &oot from ISO

1o# supports diagnostics PI9Returns a %ariety of details and statistics o%er and a&o%e thos`nova show`

$ nova diagnostics


68/69

COMING UP

$ed %at C,o+d In8rastr+t+re net)or*(ng deep d(&e

Cloud eep i%e DD9


69/69

7UESTIONS

Co!e see +s

Infrastructure as a Ser%ice 'one of the Red Hat &oot

Documents

Gordon h 0945 Deep Dive-openstack Compute