OReilly Software Architecture Conf: Cloud Economics

Cloud EconomicsChris Bailey, IBM

@Chris__Bailey

“Cloud economics is a branch of knowledge concerned with the principles, costs and benefits of

cloud computing.” — searchcio.techtarget.com

http://searchcloudcomputing.techtarget.com/definition/cloud-computing

http://searchcio.techtarget.com

Why Cloud?

Traditional Computing

§ On premise, or hosted bare metal machines

| @Chris__Bailey



§ Capital Expenditure (CapEX): - Upfront cost, followed by depreciation

| @Chris__Bailey




§ Requirement for accurate capacity planning - Estimate hardware for given load - Project future load requirements

| @Chris__Bailey




§ Requirement for accurate capacity planning - Estimate hardware for given load - Project future load requirements

§ IT is seen as a cost centre

| @Chris__Bailey

Capacity Planning

§ Find a representative benchmark?

| @Chris__Bailey

SPECjEnterprise 2010 (spec.org)

http://spec.org

Capacity Planning

§ Find a representative benchmark? - SPECjbb2015 - SPECjEnterprise2010 - SPECjms2007 - SPECjvm2008

| @Chris__Bailey

SPECjbb2015 (spec.org)

http://spec.org

Capacity Planning


| @Chris__Bailey

SPECjbb2015 (spec.org)

http://spec.org

Capacity Planning


| @Chris__Bailey

SPECjEnterprise2010 (spec.org)

http://spec.org

Capacity Planning


| @Chris__Bailey

Capacity Planning


| @Chris__Bailey

§ Find a representative benchmark? - SPECjEnterprise2010

§ Analyse the results

Capacity Planning

Tested By System Name ResultJava EE Server DB Server

Nodes CPU Nodes CPU

Oracle Corporation Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2Jun 29, 2016 | HTML | Text 14,121.47 1 16 cores, 2 chips 1 16 cores, 2 chips

Oracle Corporation Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2Jun 29, 2016 | HTML | Text 14,400.78 1 16 cores, 2 chips 1 16 cores, 2 chip

| @Chris__Bailey

http://spec.org/jEnterprise2010/results/res2016q2/jEnterprise2010-20160608-00059.html

http://spec.org/jEnterprise2010/results/res2016q2/jEnterprise2010-20160608-00059.txt





Capacity Planning


Nodes CPU Nodes CPU

Oracle Corporation Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2Jun 29, 2016 | HTML | Text 14,121.47 1 16 cores, 2 chips 1 16 cores, 2 chips

Oracle Corporation Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2Jun 29, 2016 | HTML | Text 14,400.78 1 16 cores, 2 chips 1 16 cores, 2 chip

| @Chris__Bailey







Capacity Planning


Nodes CPU Nodes CPU

Oracle Corporation Oracle Weblogic Server Standard Edition Release 12.2.1 on Oracle Server X6-2Mar 31, 2016 | HTML | Text NC 1 44 cores, 2 chips 1 44 cores, 2 chips

| @Chris__Bailey





Capacity Planning


Nodes CPU Nodes CPU

Oracle Corporation Oracle Weblogic Server Standard Edition Release 12.2.1 on Oracle Server X6-2Mar 31, 2016 | HTML | Text NC 1 44 cores, 2 chips 1 44 cores, 2 chips

| @Chris__Bailey





Capacity Planning


Nodes CPU Nodes CPU

Oracle Corporation Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2Apr 1, 2015 | HTML | Text 21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips

| @Chris__Bailey





Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey





Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey





Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey





- Intel® Xeon® processor E5- 2600 v3 @3.6GHz

Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey





§ Calculate transactions per core

Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey





§ Calculate transactions per core - 21,504.30 / 36 = 597.3 EjOPS/core

Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey



Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

PurchasingDelay

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

PurchasingDelay

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

PurchasingDelay

Capacity Requirement

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

PurchasingDelay


Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load

PurchasingDelay

Predicted Load + 10%Capacity Requirement

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted LoadCapacity Requirement

PurchasingDelay

Predicted Load + 10%

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


PurchasingDelay

Predicted Load + 10%

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


PurchasingDelay

Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


PurchasingDelay

Excess Capacity

Excess Capacity

Capacity Planning

§ Size for peak demand - Allow for Black Friday / Cyber Monday - Or just changes in load throughout each day

| @Chris__Bailey

Capacity Planning


§ Allow for High Availability (HA) and Disaster Recovery (DR) - Rule of three for HA - 50% extra capacity - Full fail-over for DR - 100% extra capacity

| @Chris__Bailey

Capacity Planning


§ Allow for High Availability (HA) and Disaster Recovery (DR) - Rule of three for HA - 50% extra capacity - Full fail-over for DR - 100% extra capacity

§ Oh, and what about capacity for integration, stress and performance test?

| @Chris__Bailey

Most enterprises have hardware utilisation below 20%

Most enterprises have hardware utilisation below 20% and below 10% for workloads with peaks

Capacity Planning and Agility

§ Time to obtain capacity affects operational agility

§ Market opportunities lost because of lead time for new projects

| @Chris__Bailey


§ Time to obtain capacity affects operational agility

§ Market opportunities lost because of lead time for new projects

§ Case Study: A fashion retailer can show measureable increase in sales if a item similar to that seen in the media can be placed on their on-line store landing page within 1 hr of it appearing in public.

§ Each product placement is different so they need a fast, agile, approach that does not jeopardize their on-line stores availability and quality.

| @Chris__Bailey

Cloud Nirvana

What Cloud Promises

“a virtual, dynamic environment which maximizes use, is infinitely scalable, always available and needs minimal upfront investment or commitment”

§ Take your code – host it on someone else's machine and pay only for the resource you use for the time you use it

§ AND be able to do that very quickly and repeatedly in parallel

| @Chris__Bailey

Cloud Values

§ Much shorter “purchasing delay”

§ Return integration, stress and performance test environments after use

§ Cloud guarantees remove need for DR

| @Chris__Bailey

Traditional Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


PurchasingDelay

Excess Capacity

Excess Capacity

Cloud Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


Excess Capacity

Excess Capacity

Cloud Capacity Planning

| @Chris__Bailey

Time

Cap

acity

Predicted Load


Excess Capacit

y




| @Chris__Bailey




Achievement Unlocked!

| @Chris__Bailey

Other advantages of Cloud§ Globally distributed data centres and locality

| @Chris__Bailey

https://aws.amazon.com/about-aws/global-infrastructure/

Other advantages of Cloud§ Globally distributed data centres and locality

| @Chris__Bailey

https://aws.amazon.com/about-aws/global-infrastructure/

Trouble in Paradise?

OpEx vs CapEx…

§ Rent vs Buy

| @Chris__Bailey

OpEx vs CapEx…

§ Rent vs Buy

§ No upfront cost, but no end to cost

| @Chris__Bailey

OpEx vs CapEx…

§ Rent vs Buy


§ Drive for operational efficiency

| @Chris__Bailey

OpEx vs CapEx…

§ Rent vs Buy


§ Drive for operational efficiency

§ Focus on ROI for individual apps and app features

| @Chris__Bailey

How is Cloud cost calculated?

| @Chris__Bailey

Provider Type Memory CPUs Cost/monthAmazon EC2 Linux t2.nano 512MB 1 vCPU $4.68

Linux t2.micro 1GB 1 vCPU $9.36Linux t2.small 2GB 1 vCPU $18.72

Linux t2.medium 4GB 2 vCPUs $37.44Digital Ocean Standard 512MB 1 Core Processor $5.00

Standard 1GB 1 Core Processor $10.00Standard 2GB 2 Core Processors $20.00

Pivotal Cloud Foundry App Instance 512MB 4 vCPUs $10.80App Instance 1GB 4 vCPUs $21.60App Instance 2GB 4 vCPUs $43.20

Heroku Standard 1x 512MB 1 Share $25.00Standard 2x 1GB 2 Shares $50.00

IBM Bluemix Instant Runtimes 512MB 4 vCPUs $24.15Instant Runtimes 1GB 4 vCPUs $49.35

Containers 512MB 4 vCPUs $10.22Containers 1GB 4 vCPUs $20.59

$£€¥ = GB/Hr

Movement from CPU to Memory based costs is a fundamental shift for Application Architectures



- Intel® Xeon® processor E5- 2600 v3 @3.6GHz

Capacity Planning


Nodes CPU Nodes CPU


| @Chris__Bailey



Reading the fine print…

Hardware Vendor: Oracle CorporationModel Name: Oracle Server X5-2Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)MHz: 2300# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)Memory (MB): 262144Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSDNetwork Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICsJVM Options:-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200 -Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2

Server instances were started using numactl binding 1 instance per 9 cores.

| @Chris__Bailey


Hardware Vendor: Oracle CorporationModel Name: Oracle Server X5-2Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)MHz: 2300# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)Memory (MB): 262144Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSDNetwork Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICsJVM Options:-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200 -Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2


| @Chris__Bailey


Hardware Vendor: Oracle CorporationModel Name: Oracle Server X5-2Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)MHz: 2300# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)Memory (MB): 262144Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSDNetwork Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICs

7.1GB / core

JVM Options:-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200 -Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2


| @Chris__Bailey

§ Project Management Triangle § Fast § Good § Cheap

§ Pick any two…

Performance Optimisation

Euler Diagram

Cheap

Fast Good

| @Chris__Bailey

§ Performance Triangle § Throughput § Latency § Memory footprint

§ Enterprise software has traditionally chosenthroughput and latency over memory § Data caching § Garbage Collection § Function Inlining

LowMemory

HighThroughput

LowLatency

Performance Optimisation

Euler Diagram

| @Chris__Bailey

Garbage Collection Tradeoff

| @Chris__Bailey

Mem

ory

Heap Size


| @Chris__Bailey

Mem

ory

Live in-use data(Retained Set)

Heap Size


| @Chris__Bailey

Mem

ory


Heap Size

Temporary Data(Garbage)



| @Chris__Bailey

Mem

ory


Heap Size




| @Chris__Bailey

Mem

ory


Heap Size




| @Chris__Bailey

Mem

ory


Heap Size



40 to 70%


| @Chris__Bailey

Mem

ory


Heap Size



40 to 70%

30 to 60% additional memory for additional performance

Optimising for the Cloud

Optimising for the Cloud#1: Choose The Right Language

Memory Usage by Language

| @Chris__Bailey

0

17.5

35

52.5

70

C++

Memusage(MB)

C


| @Chris__Bailey

0

17.5

35

52.5

70

C++ Go Sw

i:Rust

Memusage(MB)

ModernNativeC


| @Chris__Bailey

0

17.5

35

52.5

70

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

Memusage(MB)

ModernNative ScriptingC


| @Chris__Bailey

0

17.5

35

52.5

70

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

Java

Scala

Clojure

Memusage(MB)

ModernNative Scripting JVMC

Throughput by Language

| @Chris__Bailey

C

0

25

50

75

100

C++

DuraOon


| @Chris__Bailey

ModernNativeC

0

25

50

75

100

C++ Go Sw

i:Rust

DuraOon


| @Chris__Bailey


0

25

50

75

100

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

DuraOon


| @Chris__Bailey


0

25

50

75

100

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

Java

Scala

Clojure

DuraOon

Normalised Throughput/Memory Language

| @Chris__Bailey

C

0

20

40

60

80

C++

DuraOon


| @Chris__Bailey

ModernNativeC

0

20

40

60

80

C++ Go Sw

i:Rust

DuraOon


| @Chris__Bailey


0

20

40

60

80

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

DuraOon


| @Chris__Bailey


0

20

40

60

80

C++ Go Sw

i:Rust

Ruby PH

P

Python

JavaScript

Erlang

Dart

Java

Scala

Clojure

DuraOon

Selecting the Language

§ Ruby on Rails app, 100 instances at 500MB / instance - 50GB of memory required at $24.15 GB/month - = $14,490/year

| @Chris__Bailey



§ Switching to Java provides a 7x saving: - = $2,070/year

| @Chris__Bailey



§ Switching to Java provides a 7x saving: - = $2,070/year

§ Switching to Swift provides 42x saving: - = $345/year

| @Chris__Bailey


| @Chris__Bailey

C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure

C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native

Scripting


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native

Scripting


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native

Scripting


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native

Scripting

JVM


| @Chris__Bailey


C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17

Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06

Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43

Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55

Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37

PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67

Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25

JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20

Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47

Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25

Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60

Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12

Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47

From

To

C

Modern Native

Scripting

JVM


Optimising for the Cloud#2: Use Intelligent Scaling

Intelligent Scaling for Capacity

| @Chris__Bailey

Time

Cap

acity

Predicted Load


Excess Capacit

y


| @Chris__Bailey

Time

Cap

acity

Predicted Load


Excess Capacit

y


| @Chris__Bailey

Time

Cap

acity

Actual Load


Excess Capacit

y


| @Chris__Bailey

Time

Cap

acity

Actual Load


Excess Capacit

y


| @Chris__Bailey

Time

Cap

acity

Actual Load


Excess Capacit

y

Dynamic Scaling to Load

| @Chris__Bailey

00:00

Cap

acity

Actual Load

Excess Capacit

y

12:00 18:0006:0000:00

Capacity


| @Chris__Bailey

00:00

Cap

acity

Actual LoadCapacity

Excess Capacit

y

12:00 18:0006:0000:00


§ Daily load cycles are common for user facing applications - Particularly if your using localised availability zones

| @Chris__Bailey

“Most enterprises have hardware utilisation below 20% and below 10% for workloads with peaks”



§ Intelligent Scaling can provide a 2x saving:

| @Chris__Bailey




§ Intelligent Scaling can provide a 2x saving: - Ruby on Rails app: $14,490/year - Move to Java: $ 2,070/year

| @Chris__Bailey




§ Intelligent Scaling can provide a 2x saving: - Ruby on Rails app: $14,490/year - Move to Java: $ 2,070/year - Add scaling: $ 1,035/year

| @Chris__Bailey



Optimising for the Cloud#3: Identify and Fix Memory Leaks

Look for and Fix memory leaks§ A small leak means a very large

amount of memory over time

| @Chris__Bailey

AmericanSocietyofCivilEngineers



§ Memory leaks act a little like interest on a debt

| @Chris__Bailey

Live in-use data(Principle)




| @Chris__Bailey


Memory Leak(Interest)




§ Made worse if you have to size the container ahead of time to allow for the leak

| @Chris__Bailey


Memory Leak(Interest)


Optimising for the Cloud#4: Memory Analysis is the New Performance Tuning

Memory Analysis

§ Most applications are memory in-efficient - Duplication of data - Oversized collections - Empty collections

§ Initial analysis and optimisation usual has a large return on investment

| @Chris__Bailey

Memory Analysis

§ Java case study: PlantsByWebSphere sample in WebSphere AppServer 7 - 5 user test load, 206MB of “live data”:

| @Chris__Bailey

Memory Analysis


| @Chris__Bailey

Collection Count Collection Size (MB)Hashtable 262,234 26.5

WeakHashMap 19,562 12.6HashMap 10,600 2.3ArrayList 9,530 0.3HashSet 1,551 1.0Vector 1,271 0.04

LinkedList 1,148 0.1TreeMap 299 0.03TOTAL 306,195 42.9

Memory Analysis


§ 42.9MB (16% of memory) are collection objects - Just the collections themselves - excluding any data they store

| @Chris__Bailey

Collection Count Collection Size (MB)Hashtable 262,234 26.5

WeakHashMap 19,562 12.6HashMap 10,600 2.3ArrayList 9,530 0.3HashSet 1,551 1.0Vector 1,271 0.04

LinkedList 1,148 0.1TreeMap 299 0.03TOTAL 306,195 42.9

Memory Analysis

§ Analysis of just the Hashtable instances showed:

| @Chris__Bailey

Memory Analysis

§ Analysis of just the Hashtable instances showed:

§ 127,016 of the Hashtables were completely empty

| @Chris__Bailey

Memory Analysis

§ Looking at empty collections across all collection types:

§ 52.6% of collections are empty (22.6MB)

§ Could save 11% of heap utilisation by removing empty collections

| @Chris__Bailey

Collection Count Empty % Empty

Hashtable 262,234 127,016 48.8

WeakHashMap 19,562 19,456 99.5

HashMap 10,600 7,599 71.7

ArrayList 9,530 4,588 48.1

HashSet 1,551 866 55.8

Vector 1,271 622 48.9

TOTAL 304,748 160,156 52.6

Memory Analysis

§ Looking at empty collections across all collection types:

§ 52.6% of collections are empty (22.6MB)

| @Chris__Bailey

Collection Count Empty % Empty

Hashtable 262,234 127,016 48.8

WeakHashMap 19,562 19,456 99.5

HashMap 10,600 7,599 71.7

ArrayList 9,530 4,588 48.1

HashSet 1,551 866 55.8

Vector 1,271 622 48.9

TOTAL 304,748 160,156 52.6


Optimising for the Cloud#5: Design for Memory Usage

Design for Memory

§ Practice lazy allocation of collections

| @Chris__Bailey

Design for Memory


§ Don’t use default collection sizes

| @Chris__Bailey

Design for Memory



§ Reallocate collections to shrink them if needed

| @Chris__Bailey

Design for Memory



§ Reallocate collections to shrink them if needed

§ Use cache miss based strategies for cache sizes

| @Chris__Bailey

Summary

Cloud Economics