View
213
Download
0
Category
Tags:
Preview:
Citation preview
Cloud Computing Term Project
Cloud Monitoring and ScalingMerged CD Group
Pankaj KumarQinglan Zhang
Sagar DavasamSowjanya Puligadda
Wei Liu
Overview
• What we have done– Openstack installation– Create and manage VM instances– Monitoring tools: Ganglia & Zenoss
• Future work– Compare monitoring tools and present union of
data metrics from Ganglia and Zenoss– Performance analysis and VM scaling – Controlling and computing availability
Architecture Diagram of the setup
Cloud for Monitoring, Scaling and performance measurement
UBUNTU 12.04
OPENSTACK
UBUNTU-12.04 cloud
OS
VM-1
Host 1 – 192.168.2.120 Host 2 – 192.168.2.121 Host 3 – 192.168.2.122
Similar VM-2 and VM-3
Ganglia and Zenoss Tools
UBUNTU 12.04
OPENSTACK
UBUNTU-12.04 cloud OS
VM-1
Similar VM-2 and VM-3
Ganglia and Zenoss Tools
UBUNTU 12.04
OPENSTACK
UBUNTU-12.04 cloud OS
VM-1
Similar VM-2 and VM-3
Ganglia and Zenoss Tools
Openstack installation
• Use devstack multi-node installation– http://devstack.org/guides/multinode-lab.html
Network configuration & NTP
Add User & Set Up SSH
Configure Controller & Slaves
Issues and best practices
• Devstack script does not work well. The problems we have solved are as follow:– Mysql conflicts– Invalid download link– Permission denial• Su; chown –R stack:stack devstack/;
– No reboot post devstack installation– VM installation using openstack dashboard
Create and manage VM instance
• Verify the Identity Service installation• Verify the Image Service installation• Enable Networking• Generate a keypair and choose a flavor• Launch the instance
Scaling
• Analyze the performance result recorded by Ganglia and Zenoss.
• Set threshold to scale out or shrink back• Try dynamic strategies to scale automatically
Availability
• Monitor the availability• Test availability in different situations:– Shut down server– Shut down single vm– Shut down multiple vms– Turn on vms
Ganglia Vs Zenoss
Zenoss Performance Metrics
• Aggregate Reports– CPU usage– Free memory– Free swap– Network Input and Output
• Availability - Percentage of time that a device or component is considered available
• CPU Utilization – Shows monitored interfaces, devices, load average and % utility
Zenoss Performance Metrics
• Filesystem Utilization - Shows total bytes, used bytes, free bytes, and percentage utilization for each device.
• Interface Utilization - Shows the traffic through all network interfaces monitored by the system.– Interface’s rated bandwidth– Average input traffic– Average output traffic– Total average traffic across interface– % utilization of bandwidth
Zenoss Performance Metrics
• Memory Utilization - Provides system-wide information about the memory usage for devices in the system.– Total memory– Available memory– Cache memory– Buffered memory– Percentage of memory utilized
Zenoss Extended Monitoring• Apache Web Server• DNS• File Transfer Protocol• Internet Relay Chat• Jabber Instant Messaging• LDAP Response Time• MySQL Database• Network News Transport Protocol• Network Time Protocol• ONC – System Remote Procedure Call• Webpage Response Time (HTTP)• VMware esxtop• Xen Virtual Hosts
Ganglia Vs Zenoss
• Zenoss can automatically discover hosts and start monitoring them automatically. Ganglia requires an agent to run on every host to gather information
• Ganglia does not monitor events, Zenoss reports about event count, event queue length
• Zenoss can monitor services but Ganglia can not• Status of a running task can be received by
Zenoss tool but Ganglia can not be used for
Zenoss Monitoring System Architecture
Zenoss features
• Zenoss offers visibility over the entire IT stack, from network devices to applications.
• Automatic discovery,• Inventory via CMDB, • Availability monitoring, • Easy-to-read performance graphs, • Sophisticated alerting, • An easy-to-use web portal
Why Both Ganglia and Zenoss
• More data and parameters received from both tools will make– Decision making easier – Decision making robust and efficient– Decision and action will be more reliable
All
VMs
UBUNTU-12.04 cloud OS
VM-1
Ganglia and Zenoss Tools
Union of data read by Ganglia and Zenoss Tool
read_availability < demanded_availability
VM provisioning Module
0
Scale out to meet availability – add VMs
Scaling algorithm
Decision making Module
Yes
Project Architecture Diagram
How to Scale
• Scale up(horizontal) or Scale out (vertically )?– scale up: add resources to a single node– scale out: add more nodes to a system
• We decide– Scale out– Add VMs on nodes in system
• So when we scale our system?– System availability is under our expectation.– Scaling !!!
Our goal• Add minimum VMs to get “three nines” availability (Assuming adding VMs will enhance the availability)
Availability % Downtime per year Downtime per month* Downtime per week
90% ("one nine") 36.5 days 72 hours 16.8 hours
95% 18.25 days 36 hours 8.4 hours
97% 10.96 days 21.6 hours 5.04 hours
98% 7.30 days 14.4 hours 3.36 hours
99% ("two nines") 3.65 days 7.20 hours 1.68 hours
99.5% 1.83 days 3.60 hours 50.4 minutes
99.8% 17.52 hours 86.23 minutes 20.16 minutes
99.9% ("three nines") 8.76 hours 43.8 minutes 10.1 minutes
99.95% 4.38 hours 21.56 minutes 5.04 minutes
99.99% ("four nines") 52.56 minutes 4.32 minutes 1.01 minutes
99.999% ("five nines") 5.26 minutes 25.9 seconds 6.05 seconds
99.9999% ("six nines") 31.5 seconds 2.59 seconds 0.605 seconds
99.99999% ("seven nines") 3.15 seconds 0.259 seconds 0.0605 seconds
Our Algorithm• Initialize several VMs, get the monitoring data and availability.
– CPU, Network, I/O, Memory usage percentage – Availability
• Run our algorithm to make decision– 1. Calculate the initialized weight of each factor– 2. Calculate the number of VM we needed
• (Wc, Wn,Wi, Wm) = f (Uc, Un,Ui, Um, Ni ,Ni+1, Ai ,Ai+1)
• Wc, Wn,Wi, Wm are weights of CPU, Network, I/O and Memory
• Uc, Un,Ui, Um are usage of CPU, Network, I/O and Memory
• Ni ,Ni+1, Ai ,Ai+1 are number of VMs and Availability in training i and i+1.
– 3. Lance algorithm every a period of time to make decision and revise the algorithm itself based on data collected by Ganglia and Zenoss • Machine learning: semi-supervised approach
Problem Now
• How can we get the function f?• Two possible ways:– 1. Generate the function theoretically, and use the collected
data to verify and revised the function.• Hard to accomplish for current stage.
– 2. Collected the data first, and guess the function and use some tool (MATLAB) to generate the function. After training the data, improve the accuracy and revise the algorithm.• This is what we will use.
• Example next
Guess and Example• From (Wc, Wn,Wi, Wm) = f (Uc, Un,Ui, Um, Ni ,Ni+1, Ai ,Ai+1)• Guess general function to calculate the weights
• Wc, Wn,Wi, Wm are weights of CPU, Network, I/O and Memory
• Uc, Un,Ui, Um are usage of CPU, Network, I/O and Memory
• Ni ,Ni+1, Ai ,Ai+1 are number of VMs and Availability in training i and i+1.
• For example:
When i = 1, 2, 3, 4, we can get 4 equation and calculate the weights.
Number of VMs
CPU usage Network usage
I/O usage Memory usage
Availability
Training 1 1 80% 50% 90% 60% 90%
Training 2 2 75% 43% 88% 52% 92%
Training 3 3 69% 38% 70% 48% 96%
Training 4 4 60% 35% 65% 43% 97.8%
Expected ? / / / / 99.9%
Data from ZenossData from Ganglia
Goal𝐴𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑
Example and Guess
• After we get the weights, make decision how many VMs should be added.
• Get it !
• = 99.9%• the total number of VMs after scaling.
OpenStack installation Controller Node
OpenStack installation on Compute Nodes
OpenStack installation on Compute Nodes
Instance on Compute node (CirrOS)
Instance on Compute node (Ubuntu)
Zenoss Installation on Ubuntu VM
Zenoss Login
Ganglia Installation
Recommended