Upload
boden-russell
View
30.564
Download
8
Tags:
Embed Size (px)
DESCRIPTION
Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective. Results here provide insight as to: - Cloudy ops times (start, stop, reboot) using OpenStack. - Guest micro benchmark performance (I/O, network, memory, CPU). - Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion. - Compute node resource consumption; VM / Container density factors. - Lessons learned during benchmarking. The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM. Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
Citation preview
Passive Benchmarking with docker LXC, KVM & OpenStackHosted @ SoftLayer
Boden Russell ([email protected])IBM Global Technology Services
Advanced Cloud Solutions & Innovation
04/07/2023 2
Why Linux Containers (LXC) Fast– Runtime performance near bare metal speeds– Management operations (run, stop , start, etc.) in seconds / milliseconds
Agile– VM-like agility – it’s still “virtualization”– Seamlessly “migrate” between virtual and bare metal environments
Flexible– Containerize a “system”– Containerize “application(s)”
Lightweight– Just enough Operating System (JeOS)– Minimal per container penalty
Inexpensive– Open source – free – lower TCO– Supported with out-of-the-box modern Linux kernel
Ecosystem– Growing in popularity– Vibrant community & numerous 3rd party apps
04/07/2023 3
Hypervisors vs. Linux Containers
Hardware
Operating System
Hypervisor
Virtual Machine
Operating System
Bins / libs
App App
Virtual Machine
Operating System
Bins / libs
App App
Hardware
Hypervisor
Virtual Machine
Operating System
Bins / libs
App App
Virtual Machine
Operating System
Bins / libs
App App
Hardware
Operating System
Container
Bins / libs
App App
Container
Bins / libs
App App
Type 1 Hypervisor Type 2 Hypervisor Linux Containers
Containers share the OS kernel of the host and thus are lightweight.However, each container must have the same OS kernel.
Containers are isolated, but share OS and, where appropriate, libs / bins.
04/07/2023 4
Hypervisor VM vs. LXC vs. Docker LXC
04/07/2023 5
Docker in OpenStack Havana– Nova virt driver which integrates with docker REST API on backend– Glance translator to integrate docker images with Glance
Icehouse– Heat plugin for docker
Both options are still under development
nova-docker virt driver docker heat plugin
04/07/2023 6
About This Benchmark Use case perspective– As an OpenStack Cloud user I want a Ubuntu based VM with MySQL… Why would I choose
docker LXC vs a traditional hypervisor? OpenStack “Cloudy” perspective– LXC vs. traditional VM from a Cloudy (OpenStack) perspective– VM operational times (boot, start, stop, snapshot)– Compute node resource usage (per VM penalty); density factor
Guest runtime perspective– CPU, memory, file I/O, MySQL OLTP
Why KVM?– Exceptional performance
DISCLAIMERSThe tests herein are “passive” – no in depth tuning, analysis, etc. More active testing is warranted. These results do not necessary reflect your
workload or exact performance nor are they guaranteed to be statistically sound.
04/07/2023 7
Benchmark Environment Topology @ SoftLayer
glance api / reg
nova api / cond / etc
keystone
…
rally
nova api / cond / etc
cinder api / sch / vol
docker lxc
dstat
controller compute node
glance api / reg
nova api / cond / etc
keystone
…
rally
nova api / cond / etc
cinder api / sch / vol
KVM
dstat
controller compute node
+Awesome!
+Awesome!
04/07/2023 8
Benchmark SpecsSpec Controller Node (4CPU x 8G RAM) Compute Node (16CPU x 96G RAM)
Environment Bare Metal @ SoftLayer Bare Metal @ SoftLayer
Mother Board SuperMicro X8SIE-F Intel Xeon QuadCore SingleProc SATA [1Proc]
SuperMicro X8DTU-F_R2 Intel Xeon HexCore DualProc [2Proc]
CPU Intel Xeon-Lynnfield 3470-Quadcore [2.93GHz] (Intel Xeon-Westmere 5620-Quadcore [2.4GHz]) x 2
Memory (Kingston 4GB DDR3 2Rx8 4GB DDR3 2Rx8 [4GB]) x2 (Kingston 16GB DDR3 2Rx4 16GB DDR3 2Rx4 [16GB]) x 6
HDD (LOCAL) Digital WD Caviar RE3 WD5002ABYS [500GB]; SATAII Western Digital WD Caviar RE4 WD5003ABYX [500GB]; SATAII
NIC eth0/eth1 @ 100 Mbps eth0/eth1 @100 Mbps
Operating System Ubuntu 12.04 LTS 64bit Ubuntu 12.04 LTS 64bit
Kernel 3.5.0-48-generic 3.8.0-38-generic
IO Scheduler deadline deadline
Hypervisor tested NA - KVM 1.0 + virtio + KSM (memory deduplication)- docker 0.10.0 + go1.2.1 + commit dc9c28f + AUFS
OpenStack Trunk master via devstack Trunk master via devstack. Libvirt KVM nova driver / nova-docker virt driver
OpenStack Benchmark Client
OpenStack project rally NA
Metrics Collection NA dstat
Guest Benchmark Driver NA - Sysbench 0.4.12- mbw 1.1.1.-2- iibench (py)- netperf 2.5.0-1
VM Image NA - Scenario 1 (KVM): official ubuntu 12.04 image + mysql snapshotted and exported to qcow2 – 1080 MB
- Scenario 2 (docker): guillermo/mysql -- 334.8 MB
Hosted @
04/07/2023 9
Test Descriptions: Cloudy BenchmarksBenchmark Benchmark Driver Description
OpenStack Cloudy Benchmarks
Serial VM boot(15 VMs)
OpenStack Rally - Boot VM from image- Wait for ACTIVE state- Repeat the above a total of 15 times- Delete VMs
VM reboot(5 VMs rebooted 5 times each)
OpenStack Rally - Boot VM from image- Wait for ACTIVE state- Soft reboot VM 5 times- Delete VM- Repeat the above a total of 5 times
VM snapshot(1 VM, 1 snapshot)
OpenStack Rally - Boot VM from image- Wait for ACTIVE state- Snapshot VM to glance image- Delete VM
04/07/2023 10
Test Descriptions: Guest BenchmarksBenchmark Benchmark Driver Description
Guest Runtime Benchmarks
CPU performance Sysbench from within the guest - Run sysbench cpu test- Repeat a total of 10 times- Average results over the 10 times
OLTP (MySQL) performance
Sysbench from within the guest - Run sysbench OLTP test- Repeat a total of 10 times- Average results over the 10 times
MySQL Indexed insertion benchmark - Run iibench for a total of 1M inserts printing stats at 100K intervals
- Collect data over 5 runs & average
File I/O performance Sysbench from within the guest- Synchronous IO- No-direct
- Run sysbench OLTP test- Repeat a total of 10 times- Average results over the 10 times
Memory performance
Mbw from within the guest - Run mbw with array size of 1000 MiB and each test 10 times
- Collect average over 10 runs per test
Network performance
Netperf - Run netperf server on controller- From guest run netperf client in IPv4 mode- Repeat text 5x- Average results
04/07/2023 11
SERIALLY BOOT 15 VMS
OpenStack Cloudy Benchmark
04/07/2023 12
Cloudy Performance: Serial VM Boot Benchmark scenario overview– Boot VM via OpenStack nova– Wait for VM to become active– Repeat the above steps for a total of 15 VMs– Delete all VMs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
2
4
6
8
10
12
14
16
Benchmark Visualization
VMs
Time
Activ
e VM
s
04/07/2023 13
Cloudy Performance: Serial VM Boot
docker KVM0
1
2
3
4
5
6
7
3.900927941
5.88419742584
Average Server Boot Time
Series1
Tim
e in
Sec
onds
Docker 1.5x faster
04/07/2023 14
Cloudy Performance: Serial VM Boot
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 880
102030405060708090
100
Docker: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 1360
102030405060708090
100
KVM: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
Averages
– 1.14
– 0.44
Averages
– 12.6
– 2.08
04/07/2023 15
Cloudy Performance: Serial VM Boot
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 880.00E+005.00E+081.00E+091.50E+092.00E+092.50E+093.00E+093.50E+094.00E+094.50E+095.00E+09
Docker: Compute Node Used Memory
Memory
Time
Mem
ory
Used
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 1360.00E+005.00E+081.00E+091.50E+092.00E+092.50E+093.00E+093.50E+094.00E+094.50E+095.00E+09
KVM: Compute Node Used Memory
Memory
Time
Mem
ory
Used
Delta687 MB
Per VM45.8 MB
Delta2775 MB
Per VM185 MB
04/07/2023 16
Cloudy Performance: Serial VM Boot
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 880
5
10
15
20
25
Docker: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 1360
5
10
15
20
25
KVM: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
Average
0.09 %
Average
9.94 %
04/07/2023 17
SERIAL VM SOFT REBOOT
OpenStack Cloudy Benchmark
04/07/2023 18
Cloudy Performance: Serial VM Reboot Benchmark scenario overview– Boot a VM• Wait for it to become active
– Soft reboot the VM• Wait for it to become active• Repeat soft reboot a total of 5 times
– Delete VM– Repeat the above for a total of 5 VMs
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 550
1
2
3
4
5
6
Benchmark Visualization
Active VMs
Time
Activ
e VM
s
04/07/2023 19
Cloudy Performance: Serial VM Reboot
docker KVM0
20
40
60
80
100
120
140
6.59131344795
124.452507925
Average Server Reboot Time
Series1
Tim
e In
Sec
onds
Docker 18.9x faster
04/07/2023 20
Cloudy Performance: Serial VM Reboot
docker KVM0
1
2
3
4
5
6
7
8
9
7.85760216713
4.45187151432
Average Server Delete Time
Series1
Tim
e In
Sec
onds
KVM 1.75x faster
04/07/2023 21
Cloudy Performance: Serial VM Reboot After investigating docker delete times– Docker sends SIGTERM to container process to stop– Bash is immune to SIGTERM– Docker waits for X seconds before stop– Default container image (init) command was using bash– See: https://github.com/dotcloud/docker/issues/3766
Rebuild the docker mysql image– Don’t use bash for container image command– No change to docker (hypervisor); image config only change
Rerun rally reboot tests; Cloudy reboot round 2
04/07/2023 22
Cloudy Performance: Serial VM Reboot
docker KVM0
20
40
60
80
100
120
140
2.54194530487
124.452507925
Average Server Reboot Time (Round 2)
Series1
Tim
e In
Sec
onds
Docker 48.99x faster
04/07/2023 23
Cloudy Performance: Serial VM Reboot
docker KVM0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
4.09325451851
4.45187151432
Average Server Delete Time (Round 2)
Series1
Tim
e In
Sec
onds
Docker 1.09x faster
04/07/2023 24
Cloudy Performance: Serial VM Reboot
3 136 269 402 535 668 801 934 1067 1200 1333 1466 1599 1732 1865 1998 2131 2264 2397 2530 2663 2796 2929 30620123456789
10
KVM: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153 161 169 177 185 193 201 209 217 225 233 2410123456789
10
Docker: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
Averages
– 0.34
– 0.11
Averages
– 0.82
– 0.19
04/07/2023 25
Cloudy Performance: Serial VM Reboot
1 9 17 25 33 41 49 57 65 73 81 89 97 1051131211291371451531611691771851932012092172252332410.00E+00
5.00E+08
1.00E+09
1.50E+09
2.00E+09
2.50E+09
Docker: Compute Node Used Memory
Memory
Time
Mem
ory
Used
3 143 283 423 563 703 843 983 1123126314031543168318231963210322432383252326632803294330830.00E+00
5.00E+08
1.00E+09
1.50E+09
2.00E+09
2.50E+09
KVM: Compute Node Used Memory
Memory
Time
Mem
ory
Used
Delta
57 MB
Delta
467 MB
04/07/2023 26
Cloudy Performance: Serial VM Reboot
3 129 255 381 507 633 759 885 1011113712631389151516411767189320192145227123972523264927752901302731530
0.5
1
1.5
2
2.5
3
3.5
KVM: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153 161 169 177 185 193 201 209 217 225 233 2410
0.5
1
1.5
2
2.5
3
3.5
Docker: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
Average
0.05 %
Average
0.35 %
04/07/2023 27
SNAPSHOT VM TO IMAGE
OpenStack Cloudy Benchmark
04/07/2023 28
Cloudy Performance: Snapshot VM To Image Benchmark scenario overview– Boot a VM– Wait for it to become active– Snapshot the VM– Wait for image to become active– Delete VM
04/07/2023 29
Cloudy Performance: Snapshot VM To Image
docker KVM0
5
10
15
20
25
30
35
40
45
50
26.3947799206
42.92771101
Average Snapshot Server Time
Series1
Tim
e (S
econ
ds)
Docker 1.62x faster
04/07/2023 30
Cloudy Performance: Snapshot VM To Image
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 1130
1
2
3
4
5
6
7
KVM: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 670
1
2
3
4
5
6
7
Docker: Compute Node CPU
usrsys
Time
CPU
Usag
e In
Per
cent
Averages
– 0.4
– 0.11
Averages
– 1.58
– 1.07
04/07/2023 31
Cloudy Performance: Snapshot VM To Image
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 1131650000000
1700000000
1750000000
1800000000
1850000000
KVM: Compute Node Used Memory
Memory
Time
Mem
ory
Used
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 6718300000001840000000185000000018600000001870000000188000000018900000001900000000191000000019200000001930000000
Docker: Compute Node Used Memory
Memory
Time
Mem
ory
Used
Delta
48 MB
Delta
114 MB
04/07/2023 32
Cloudy Performance: Snapshot VM To Image
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 1130
0.1
0.2
0.3
0.4
0.5
0.6
KVM: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67-0.1
2.77555756156289E-17
0.1
0.2
0.3
0.4
0.5
0.6
Docker: Compute Node 1m Load Average
1m
Time
1 M
inut
e Lo
ad A
vera
ge
Average
0.1 %
Average
0.37 %
04/07/2023 33
GUEST PERFORMANCE BENCHMARKS
Guest VM Benchmark
04/07/2023 34
Guest Performance: CPU Linux sysbench 0.4.12 cpu test Calculate prime numbers up to 20000 2 threads Instance size– 4G RAM– 2 CPU cores– 20G disk
04/07/2023 35
Guest Performance: CPU
Docker KVM Bare Metal0
2
4
6
8
10
12
14
1615.11 15.08 15.03
Calculate Primes
Series1
Seconds
04/07/2023 36
Guest Performance: Memory Linux mbw 1.1.1-2 Instance size– 2 CPU– 4G memory
Execution options– 10 runs; average– 1000 MiB
04/07/2023 37
Guest Performance: Memory
MEMCPY DUMB MCBLOCK0
2000
4000
6000
8000
10000
12000
14000
Memory Benchmark Performance
BareMetaldockerKVM
Memory Tests
MiB
/s
04/07/2023 38
Guest Performance: Network Netperf 2.5.0-1– Netserver running on controller– Netperf on guest– Run netperf 5 times & average results
Instance size– 2 CPU– 4G memory
Execution options– IPv4 / TCP
04/07/2023 39
Guest Performance: Network
docker KVM0
100
200
300
400
500
600
700
800
900
1000
Network Throughput
Series1
Thro
ughp
ut In
10^
6 bi
ts/s
econ
d
04/07/2023 40
Guest Performance: File I/O Linux sysbench 0.4.12 fileio test– Synchronous IO– Random read / write– Total file size of 150G– 16K block size
Thread variations: 1, 8, 16, 32 Instance size– 4G RAM– 2 CPU cores– 200G disk
KVM specs– Disk cache set to none– Virtio– Deadline scheduler (host & guest)
Docker specs– AUFS storage driver– Deadline scheduler
04/07/2023 41
Guest Performance: File I/O
1 8 16 320
50
100
150
200
250
300
File I/O: Read
DockerKVM
Threads
Mb
1 8 16 320
20406080
100120140160180200
File I/O: Write
DockerKVM
Threads
Mb
04/07/2023 42
Guest Performance: File I/O – Read / Write
1 8 16 320
200
400
600
800
1000
1200
1400
1600
File I/O: Transfer Rate
DockerKVM
Threads
Kb/s
ec
04/07/2023 43
Guest Performance: MySQL OLTP Linux sysbench 0.4.12 oltp test– Table size of 2,000,000– MySQL 5.5 (installed on Ubuntu 12.04 LTS with apt-get)
Variations– Number of threads– Read only & read / write
Instance size– 4G RAM– 2 CPU cores– 20G disk
04/07/2023 44
Guest Performance: MySQL OLTP (Read)
2 8 160
10000
20000
30000
40000
50000
60000
MySQL OLTP Read Transactions
DockerKVM
Threads
Tota
l Tra
nsac
tions
2 8 160
100000200000300000400000500000600000700000800000
MySQL OLTP Read Requests
DockerKVM
Threads
Requ
ests
04/07/2023 45
Guest Performance: MySQL OLTP (Read / Write)
2 16 32 640
5000
10000
15000
20000
25000
MySQL OLTP Read/Write Transactions
dockerKVM
Threads
Tota
l Tra
nsac
tions
2 16 32 640
50000
100000
150000
200000
250000
300000
350000
400000
MySQL OLTP Read/Write Requests
DockerKVM
Threads
Requ
ests
04/07/2023 46
Guest Performance: MySQL Indexed Insertion Indexed insertion benchmark (iibench python script)– A total of 1,000,000 insertions– Print stats at 100K intervals– Collect stats over 5 runs– Average
Instance size– 4G RAM– 2 CPU cores– 20G disk
04/07/2023 47
Guest Performance: MySQL Indexed Insertion
1 2 3 4 5 6 7 8 9 100
20
40
60
80
100
120
140
MySQL Indexed Insertion @ 100K Intervals
dockerkvm
Table Size In 100K Increments
Seco
nds P
er 1
00K
Inse
rtion
Bat
ch
04/07/2023 48
BENCHMARK OBSERVATIONS
04/07/2023 49
Docker LXC CPU Growth 26x Lower Than VM
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 700
5
10
15
20
25
30
35
f(x) = 0.00912569940189079 x + 0.734884194053208
f(x) = 0.237004371128586 x + 2.29925813820674
User CPU Growth Trend
dockerLinear (docker)KVMLinear (KVM)
CPU
Usag
e In
Per
cent
Slope– 0.0091
– 0.237
04/07/2023 50
Docker LXC Memory Growth 3x Lower Than VM
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 730.00E+00
5.00E+08
1.00E+09
1.50E+09
2.00E+09
2.50E+09
3.00E+09
3.50E+09
4.00E+09
4.50E+09
5.00E+09
f(x) = 11257691.2277667 x + 1243983088.22487
f(x) = 32597283.8749557 x + 1271949903.76939
Memory Usage Growth Trend
dockerLinear (docker)KVMLinear (KVM)
Mem
ory
Used
Slope– 1E+07
– 3E+07
04/07/2023 51
Observations Cloudy operations with Docker LXC outperform VM– 48x server reboot, 1.5x server boot, 1.62x server snapshot, etc.
Docker LXC density potential compared to VMs– 3x memory savings– 26x CPU savings– 3.22x smaller images in this test (note – image sizes can vary based on required packages)
Docker LXC containers run on bare metal– Greater / equivalent “in the VM” performance at the micro benchmark level
Micro benchmark results do not always reflect macro performance– Always benchmark your “real” workload
Docker image (init) command can impact performance– Bash ignores SIGTERM
Nova-docker virt driver and docker-registry components still under dev– Nice work, but room for improvement (python anyone?)
Real performance of Docker LXC ops capped my Cloud manager– Can start the SQL image from docker CLI in 0.191s
04/07/2023 52
REFERENCES
04/07/2023 53
References & Related Links http://www.slideshare.net/BodenRussell/realizing-linux-containerslxc https://www.docker.io/ http://sysbench.sourceforge.net/ http://dag.wiee.rs/home-made/dstat/ http://www.openstack.org/ https://wiki.openstack.org/wiki/Rally https://wiki.openstack.org/wiki/Docker http://devstack.org/ http://www.linux-kvm.org/page/Main_Page https://github.com/stackforge/nova-docker https://github.com/dotcloud/docker-registry http://www.netperf.org/netperf/ http://www.tokutek.com/products/iibench/ http://www.brendangregg.com/activebenchmarking.html
04/07/2023 54
REFERENCE
04/07/2023 55
Rally Boot 15 VM Configuration Rally config:{ "NovaServers.boot_server": [ { "args": { "flavor_id": 2, "image_id": “IMAGE_ID" }, "runner": { "type": "constant", "times": 15, "active_users": 1 }, "context": { "users": { "tenants": 1, "users_per_tenant": 1 } } } ]}
Flavor+-----+-----------+-----------+------+-----------+------+-------+-------------+-----------+| ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |+-----+-----------+-----------+------+-----------+------+-------+-------------+-----------+| 1 | m1.tiny | 512 | 1 | 0 | | 1 | 1.0 | True || 2 | m1.small | 2048 | 20 | 0 | | 1 | 1.0 | True || 3 | m1.medium | 4096 | 40 | 0 | | 2 | 1.0 | True || 4 | m1.large | 8192 | 80 | 0 | | 4 | 1.0 | True || 42 | m1.nano | 64 | 0 | 0 | | 1 | 1.0 | True || 451 | m1.heat | 1024 | 0 | 0 | | 2 | 1.0 | True || 5 | m1.xlarge | 16384 | 160 | 0 | | 8 | 1.0 | True || 6 | m1.perf | 4096 | 200 | 0 | | 2 | 1.0 | True || 7 | m1.sql | 512 | 5 | 0 | | 1 | 1.0 | True || 8 | m1.bench | 512 | 10 | 0 | | 1 | 1.0 | True || 84 | m1.micro | 128 | 0 | 0 | | 1 | 1.0 | True || 9 | m1.mini | 512 | 20 | 0 | | 1 | 1.0 | True |+-----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
04/07/2023 56
Cloudy Benchmark: Serially Boot 15 VMs Docker+------------------+-------+--------------+-------------+---------------+---------------+---------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+------------------+-------+--------------+-------------+---------------+---------------+---------------+| nova.boot_server | 15 | 4.8055100441 | 3.900927941 | 3.64957404137 | 4.56917948723 | 4.80114896297 |+------------------+-------+--------------+-------------+---------------+---------------+---------------+
+---------------+---------------+--------------+---------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+---------------+---------------+--------------+---------------+---------------+---------------+-------------+| 5.02777004242 | 4.11018741926 | 3.8767888546 | 4.77419886589 | 5.02389762402 | 1.0 | 15 |+---------------+---------------+--------------+---------------+---------------+---------------+-------------+
KVM+------------------+-------+---------------+---------------+---------------+---------------+---------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+------------------+-------+---------------+---------------+---------------+---------------+---------------+| nova.boot_server | 15 | 7.00468301773 | 5.88419742584 | 4.84723997116 | 6.13021831512 | 6.48467411995 |+------------------+-------+---------------+---------------+---------------+---------------+---------------+
+---------------+--------------+---------------+--------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+---------------+--------------+---------------+--------------+---------------+---------------+-------------+| 7.22686600685 | 6.1018067201 | 5.05242800713 | 6.332523489 | 6.68914074898 | 1.0 | 15 |+---------------+--------------+---------------+--------------+---------------+---------------+-------------+
04/07/2023 57
Rally Reboot 5x5 Configuration Rally config:{ "NovaServers.boot_and_bounce_server": [ { "args": { "flavor_id": 2, "image_id": “ID", "actions": [ {"soft_reboot": 5} ] }, "runner": { "type": "constant", "times": 5, "active_users": 1 }, "context": { "users": { "tenants": 1, "users_per_tenant": 1 } } } ]}
04/07/2023 58
Cloudy Performance: Serial VM Reboot Docker+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| nova.reboot_server | 25 | 6.85025811195 | 6.59131344795 | 6.50809788704 | 6.65774655342 | 6.72213015556 || nova.boot_server | 5 | 4.96266412735 | 4.13350987434 | 3.82891011238 | 4.61819009781 | 4.79042711258 || nova.delete_server | 5 | 8.74263191223 | 7.85760216713 | 6.61050701141 | 8.72944793701 | 8.73603992462 |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+
+---------------+---------------+---------------+---------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+---------------+---------------+---------------+---------------+---------------+---------------+-------------+| 45.8827331066 | 44.9664116383 | 43.5658369064 | 45.7298994541 | 45.8063162804 | 1.0 | 5 |+---------------+---------------+---------------+---------------+---------------+---------------+-------------+
KVM+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| nova.reboot_server | 10 | 124.951740026 | 124.452507925 | 123.876129866 | 124.950992012 | 124.951366019 || nova.boot_server | 2 | 6.1228749752 | 5.58395600319 | 5.04503703117 | 6.0150911808 | 6.068983078 || nova.delete_server | 2 | 4.45799517632 | 4.45187151432 | 4.44574785233 | 4.45677044392 | 4.45738281012 |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+
+---------------+---------------+---------------+--------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+---------------+---------------+---------------+--------------+---------------+---------------+-------------+| 632.718183994 | 632.315432072 | 631.912680149 | 632.63763361 | 632.677908802 | 0.4 | 5 |+---------------+---------------+---------------+--------------+---------------+---------------+-------------+
04/07/2023 59
Rally Snapshot Configuration
Rally config:{ "NovaServers.boot_and_bounce_server": [ { "args": { "flavor_id": 1, "image_id": “IMAGE", "actions": [ {"soft_reboot": 5} ] }, "runner": { "type": "constant", "times": 5, "active_users": 1 }, "context": { "users": { "tenants": 1, "users_per_tenant": 1 } } } ]}
04/07/2023 60
Cloudy Performance: Snapshot VM To Image Docker (note -- defect deleting image)+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+| nova.create_image | 1 | 26.3947799206 | 26.3947799206 | 26.3947799206 | 26.3947799206 | 26.3947799206 || nova.boot_server | 2 | 4.120429039 | 3.85578501225 | 3.59114098549 | 4.06750023365 | 4.09396463633 || nova.delete_server | 2 | 8.61395692825 | 7.64416801929 | 6.67437911034 | 8.41999914646 | 8.51697803736 |+--------------------+-------+---------------+---------------+---------------+---------------+---------------+
+-----------+-----------+-----------+--------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+-----------+-----------+-----------+--------------+---------------+---------------+-------------+| n/a | n/a | n/a | n/a | n/a | 0 | 1 |+-----------+-----------+-----------+--------------+---------------+---------------+-------------+
KVM+--------------------+-------+----------------+----------------+----------------+----------------+----------------+| action | count | max (sec) | avg (sec) | min (sec) | 90 percentile | 95 percentile |+--------------------+-------+----------------+----------------+----------------+----------------+----------------+| nova.delete_image | 1 | 0.700344085693 | 0.700344085693 | 0.700344085693 | 0.700344085693 | 0.700344085693 || nova.create_image | 1 | 42.92771101 | 42.92771101 | 42.92771101 | 42.92771101 | 42.92771101 || nova.boot_server | 2 | 40.9650099277 | 22.9950100183 | 5.02501010895 | 37.3710099459 | 39.1680099368 || nova.delete_server | 2 | 4.47270512581 | 4.46178817749 | 4.45087122917 | 4.47052173615 | 4.47161343098 |+--------------------+-------+----------------+----------------+----------------+----------------+----------------+
+--------------+--------------+--------------+--------------+---------------+---------------+-------------+| max (sec) | avg (sec) | min (sec) | 90 pecentile | 95 percentile | success/total | total times |+--------------+--------------+--------------+--------------+---------------+---------------+-------------+| 98.541975975 | 98.541975975 | 98.541975975 | 98.541975975 | 98.541975975 | 1.0 | 1 |+--------------+--------------+--------------+--------------+---------------+---------------+-------------+
04/07/2023 61
Configuring Docker LXC Container For 2CPU x 4G RAM
Pin container to 2 CPUs / Mems– Create cpuset cgroup– Pin group to cpuset.mems to 0,1– Pin group to cpuset.cpus to 0,1– Add container root proc to tasks
Limit container memory to 4G– Create memory cgroup– Set memory.limit_in_bytes to 4G– Add container root proc to tasks
04/07/2023 62
Docker Command Line Start docker image from docker CLI
root@devstack-compute:~/devstack$ time docker run -d guillermo/mysql859f2a88adeb7190eb855eb060172fc2f137bff609f2f3be8ad8ee069d7e88f4
real 0m0.191suser 0m0.004ssys 0m0.004s