Cloud Services Innovation Platform (CSIP)

Service Isolation vs. Consolidation: Implications for IaaS Cloud Application DeploymentWes Lloyd, Shrideep Pallickara, Olaf David, James Lyon, Mazdak Arabi, Ken Rojas

March 26, 2013

Colorado State University, Fort Collins, Colorado USAIC2E 2013: IEEE International Conference on Cloud Engineering

1OutlineBackgroundResearch ProblemResearch QuestionsExperimental SetupExperimental ResultsConclusions2March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment2Background3Traditional Application Deployment4Object StorePhysical Server(s)March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentIaaS Component Deployment 5App ServerComponentDeploymentApplicationComponentsApplication StackVirtual Machine (VM) Images

PERFORMANCErDBMS r/oFile ServerLog ServerLoad BalancerImage 2rDBMS write. . .Image 1App ServerFile ServerLog ServerrDBMS writeImage nrDBMS r/oLoad BalancerDist. cacheMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implication for IaaS Cloud Application DeploymentResearch Problem6Amazon Web Services: White Paper on Application DeploymentAmazon white paper suggests:bundling the logical construct of a component into an Amazon Machine Image so that it can be deployed more often.J. Varia, Architecting for the Cloud: Best Practices, Amazon Web Services White Paper, 2010, https://jineshvaria.s3.amazonaws.com/public/ cloudbestpractices-jvaria.pdfTo support application scaling

7March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentService Isolation Advantages 8tomcat7nginxPostgreSQLMemcacheDBMySQLMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBSCALEEnablesHorizontal scalingFault tolerance

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implication for IaaS Cloud Application DeploymentService Isolation Overhead9Isolation requiresSeparate operating system instancesMore network traffic

tomcat7nginxPostgreSQLMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentProvisioning VariationProblems & Challenges 10VMPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostVMVMVMAmbiguousMappingVMVMVMVMVMVMVMVMVMVMVMVMVMVMRequest(s) to launch VMsVMs ReservePM Memory BlocksVMs Share PMCPU / Disk / Network

PERFORMANCEMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentResearch Questions11Research QuestionsWhat performance and resource utilization implications result based on how application components are deployed? How does increasing VM memory impact performance?

How much overhead results from VM service isolation?

Can resource utilization data be used to build models to predict performance of component deployments?

12RQ1:

RQ2:

RQ3:March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

Gaps in Related WorkPrior work investigates:Virtualization performanceIsolation properties of hypervisorsAutonomic scaling of application infrastructurePerformance variation fromProvisioning variationShared cluster/cloud loadsNo studies have investigated implications of how the application stack is deployedMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentExperimental Setup14

RUSLE2 ModelRevised Universal Soil Loss EquationCombines empirical and process-based sciencePrediction of rill and interrill soil erosion resulting from rainfall and runoffUSDA-NRCS agency standard modelUsed by 3,000+ field officesHelps inventory erosion ratesSediment delivery estimationConservation planning tool15March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRUSLE2 Web ServiceMulti-tier client/server applicationRESTful, JAX-RS/Java using JSON objectsSurrogate for common architectures

16OMS3RUSLE2POSTGRESQLPOSTGIS1.7+ million shapes57k XML files, 305MbMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

Eucalyptus 2.0 Private Cloud(9) Sun X6270 blade serversDual Intel Xeon 4-core 2.8 GHz CPUs24 GB ram, 146 GB 15k rpm HDDsCentOS 5.6 x86_64 (host OS)Ubuntu 9.10 x86_64 (guest OS)Eucalytpus 2.0Amazon EC2 API support8 Nodes (NC), 1 Cloud Controller (CLC, CC, SC)Managed mode networking with private VLANsXEN hypervisor v 3.4.3, paravirtualization17March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRUSLE2 ComponentsVirtual MachineDescriptionMModelApache Tomcat 6.0.20, Wine 1.0.1, RUSLE2 Model, Object Modeling System (OMS 3.0)DDatabasePostgresql-8.4, and PostGIS 1.4.0-2. soil data: 1.7 million shapes, 167 million pointsmanagement data: 98 shapes, 489k pointsclimate data: 31k shapes, 3 million points4.6 GB for the state of TNFFile Servernginx http server 0.7.62 57,185 XML files consisting of 305MB.LLoggerCodebeamer 5.5 running 32-bit ApacheTomcat 6.0Custom REST/JSON logging service as wrapper. 18March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment18SC2M DF LSC4M D

FLSC7LMD

FSC3M D

F LSC5MDF LSC6MD FLSC8MD

F LSC9MD LFSC10M FD LSC11M FDLSC12M LD FSC13M LDFSC14M DLFSC15M LFDSC1M DF L19(15) Tested Component Deployments

Each VM deployed to separate physical machinesAll components installed on composite imageScript enabled/disabled components to achieve configsMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentTested Resource Utilization Variablesc20Network- Network bytes sent (nbr)- Network bytes received (nbs)CPUCPU time

Disk- Disk sector reads (dsr)- Disk sector reads completed (dsreads)March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

RUSLE2 Application Profiles21D-bound:join w/ a nested queryM-bound:standard modelMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment21Experimental Results22

Slow group:ah3, ah5, ah8*Middle group:ah4, ah9*, ah10*, ah13Fast group:ah1, ah2*, ah6*, ah7, ah11, ah12*, ah14, ah15Test: 2 identical runs, 4GB VMs, 15 component deployments, 10 ensemble runs of 100 model runs each

Performance was reproduced.

Strong correlationp=0.000000000809050298* - indicates same group membership as DBoundReproducibility of tests Conclusion:

Service Composition of VMs mattered.Performance is different and can bemeasured with reproducible results.

24

SC15SC14SC13SC12SC11SC10SC9SC8SC7SC6SC5SC4SC3SC2SC1 CPU time disk sector reads disk sector writes net bytes rcvd net bytes sentRQ1: Resource utilization implications from component deployments

Boxes represent absolute deviation from mean (m-bound)Magnitude of variance for deployments

Resource Utilization ChangeMin to Max Utilization

m-bound d-boundCPU time:6.5%5.5%Disk sector reads:14.8%819.6%Disk sector writes:21.8%111.1%Network bytes received:144.9%145%Network bytes sent:143.7%143.9%

RQ1: Performance implications from component deployments

25Slower deploymentsFaster deployments Performance Change:Min to max performance

M-bound:14%D-bound:25.7%March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRQ1: How does increasing VM memory allocation impact performance?

26In some casesmore memory lead toslower performanceMore memoryFaster performanceMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRQ2: How much overhead results from VM service isolation?27

1.2 %.3 %2.4 %Performance Overhead

Xen:~1% averageKVM:~2.4% averageMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment28100 random runsJSON object20x Ensembles100 random runs100 random runs100 random runs100 random runs100 random runs100 random runs100 random runs100 random runsSC5MDF LSC8MD

F LSC11M FDLSC14M DLFSC1M DF L(15) RUSLE2deployments

Resource UtilizationDatascript captureData used to build multiple linear regressionperformance model

1st run training dataset2nd run test datasetMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

RQ3: Can resource utilization data be used to build models to predict performance of component deployments?CPUDisk I/ONetwork I/O29# VMs.71.37.14.007.008.04Multiple Linear Regression Performance Model

For the test dataset:

Combined R2:.8416Mean absolute error: 324ms (test dataset)Average rank error:2 unitsFastest deployment predicted accurately

Explained 84% of the varianceMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentConclusions30ConclusionsComponent deployments led to:25% performance variationNetwork and disk resource utilization most affected. VM memory did not always improve performance

Up to 2.4% performance overhead from service isolation

Our MLR-model accounted for 84% of the variance when predicting deployment performance

31RQ1:

RQ2:

RQ3:March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentQuestions32

Extra Slides33ApplicationServersLoad BalancerLoad BalancerService RequestsnoSQL data storesrDBMSdistributed cacheInfrastructure ManagementProblems & Challenges 34Scale Services

Tune Application Parameters

Tune Virtualization ParametersMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentApplication Profiling VariablesPredictive Power35

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentApplication Deployment ChallengesVM image compositionService isolation vs. scalabilityResource contention among componentsProvisioning variation Across physical hardware

36

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentResource Utilization VariablesStatisticDescription P/VCPU timeCPU time in msP/Vcpu usrCPU time in user mode in msP/Vcpu krnCPU time in kernel mode in msP/Vcpu_idleCPU idle time in msP/VcontextswNumber of context switchesP/Vcpu_io_waitCPU time waiting for I/O to completeP/Vcpu_sint_timeCPU time servicing soft interruptsVdsrDisk sector reads (1 sector = 512 bytes)VdsreadsNumber of completed disk readsVdrmNumber of adjacent disk reads mergedVreadtimeTime in ms spent reading from diskVdswDisk sector writes (1 sector = 512 bytes)VdswritesNumber of completed disk writesVdwmNumber of adjacent disk writes mergedVwritetimeTime in ms spent writing to diskP/VnbrNetwork bytes sentP/VnbsNetwork bytes receivedP/VloadavgAvg # of running processes in last 60 sec37March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentExperimental DataScript captured resource utilization statsVirtual machinesPhysical MachinesTraining data: first complete run20 different ensembles of 100 model runs 15 component configurations30,000 model runs Test data: second complete run30,000 model runs38March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deploymentn=# components; k=# components per set

Permutations

Combinations

But neither describes partitions of a set!Application Deployments

39

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

Bells Number 40ModelComponentDeploymentn = #componentsApplication StackVM deployments# of ConfigurationsDatabaseFile ServerLog Server. . .k= #configsconfig 1MDFLconfig 2MFLconfig nMLFD1 VM : 1..n componentsnk4155526203787784,140921,147n. . .DNumber of ways a set of n elements can be partitioned into non-empty subsets

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implication for IaaS Cloud Application DeploymentXEN Mbound vs Dbound Performance Same Ensemble41

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentXEN 10 GB VMs42

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentKVM Mbound vs Dbound PerformanceSame Ensemble

43March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentKVM 10GB PerformanceSame Ensemble

44March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentKVM 10 GB Performance ChangeSame Ensemble

45March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentKVM Performance ComparisonDifferent Ensembles

46March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentKVM Performance Change From Service Isolation

47March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentService Configuration TestingBig VMsAll application services installed on single VMScripts enable/disable services to achieve configurations for testingEach VM deployed on separate hostProvisioning Variation (PV) TestingKVM used15 total service configurations46 possible deployments48March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentPV: Performance Difference vs. Physical Isolation

49March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentService Configuration Testing - 2Big VMs used in physical isolation were effective at identifying fastest service configurationsFastest configurations isolate L service on separate physical host; and VMsSome provisioning variations fasterOther SC provisioning variations remained slowSC4A-D, SC9C-D Only SCs w/ avg ensemble performance < 30 seconds50March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

Can Resource Utilization Statistics

51

Model Application Performance? March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRQ1 Which are the best predictors? PM Variables52

CPUNetwork I/OMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRQ2 How should VM resource utilization data be used by performance models?Combination: RUdata=RUM+RUD+RUF+RUL

Used Individually: RUdata={RUM; RUD; RUF; RUL;}

53March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

RQ2 How should VM resource utilization data be used by performance models?54D-bound separateD-bound combinedM-bound separateM-bound combinedTreating VM data separately for D-bound was better !RUM or RUMDFLfor M-bound was better !Note the larger RMSEfor D-bound RUMDFL!March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentRQ3 Which modeling techniques were best?Multiple Linear Regression (MLR)Stepwise Multiple Linear Regression (MLR-step)Multivariate Adaptive Regression Splines (MARS)Artificial Neural Network (ANNs)55March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment

RQ3 Which modeling techniques were best?56MultipleLinearRegressionStepwiseMLRMultivariateAdaptiveRegresionSplinesArtificalNeuralNetworkRUMDFL data used tocompare models.

Had high RMSEtest error for D-Bound (32% avg)Model performance did not vary much

Best vs. Worst

D-BoundM-Bound .11% RMSEtrain.08% .89% RMSEtest.08% .40 rank err.66

March 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application DeploymentResource Utilization Statisticsc57CPU- CPU time- CPU time in user mode- CPU time in kernel mode- CPU idle time- # of context switches- CPU time waiting for I/O- CPU time serving soft interrupts- Load average (# proc / 60 secs)

Disk- Disk sector reads- Disk sector reads completed- Merged adjacent disk reads- Time spent reading from disk- Disk sector writes- Disk sector writes completed- Merged adjacent disk writes- Time spent writing to diskNetwork- Network bytes sent- Network bytes receivedPMVMVMPMVMVMVMMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications for IaaS Clouds Application Deployment