Abstract - It is a well known fact that there exists a direct relationship between the availability of resource and application performance. Conversely, an application’s performance decreases with a dearth of resource availability. However it has to be noted that allocating a physical machine to service the application is clearly an overkill. This anomaly stems because resources are underutilized. With the advent of fast CPUs, it can easily be perceived how many cycles are wasted when a single application is run. Here is where virtualization comes to the rescue. Virtualization provides a perfect platform for an optimal use of resources by a fair share allocation of available resources to different applications. Towards this, multiple virtual machines (VMs) are employed with each application running on a VM. The aim of this paper is to analyse the performance of Jmeter on VMs and PMs. Additionally this paper also illustrates the performance of VMs that are allocated resources dynamically.

Keywords - Virtualization, performance, Virtual

machine, physical machine

I. INTRODUCTION

The number of servers used in an organization increases every year due to which there is an increase in floor space used, energy consumed and cost for cooling. More and more administrators are required to maintain these systems. Few servers may be overloaded during peak hours. There may be some more servers which are underutilized even during peak loads. The IT funds are put more in managing these servers rather than developing innovative projects. To overcome from these problems the virtualization[1,2] technology is used.

The today’s CPUs are extremely fast and powerful than most applications need. The servers today has RAM of 16 or even 32 gigabytes and it is cheaper than before. Running a simple web server, database server or a game server on these powerful systems is waste of CPU and memory. This inturn underutilizes the resources available on physical machine. To heighten the resource utilization, several virtual servers are run on such a system. This is possible with server virtualization where we achieve the maximum resource utilization. These virtual servers are called as Virtual Machines(VM).

There are many advantages the VM offer over their physical counterparts like server consolidation, Disaster recovery, load balancing and hardware flexibility. The main is hardware flexibility. The flexibility comes from the fact that the hardware resources could be very easily manipulated (ex. RAM, CPU, disk could be added/removed). Disk space can be changed (increased) by shutting the VM down. Virtualization deployment is to save costs – provisioning more services with less resources. This is achieved by consolidating hardware resources, that is to make optimal use of underlying hardware by running multiple VMs.

The virtualization technology is the research hotspot in recent days. The server virtualization is the technique of abstracting physical resources and operating systems from server users (guests). The server virtualization also consolidates number of servers thereby reducing the maintenance cost and cooling.

Several VMs each running their own operating system are commonly used in server virtualization for consolidating servers. Where services used to be run on separate physical servers, they are instead run now on separate VMs but utilizing the same physical server. By running multiple single tasking OSes on a physical server to allow time sharing of the hardware resources was the original motivation for the need of virtualization. In addition to the above, the guest OSes need not be same, for instance some could be running Linux, some windows etc.

The use of virtual machine increases the complexity of resource management. The reason for this complexity is during the execution of an application by virtual machine, an application may require more resources to complete. The resources may be allocated statically or managed dynamically. The drawback of static allocation is that the resources to virtual machines are allocated when the virtual machine is hosted on the physical server. These resources cannot be increased if needed nor cannot be released when not necessary. In dynamic allocation the resources are allocated and released at run time. This paper is an attempt to analyze the performance of JMeter on physical machines with one and two CPUs. The same application is run on VM with one VCPU and two VCPU and the results are analyzed for pinned and unpinned cases.

Comparative Study of Application Performance on Virtual Machine and Physical Machine.

Anala M R1, Dr. Shobha G2 1Department of CSE, R V College of Engineering, Bangalore, India 2Department of CSE, R V College of Engineering , Bangalore, India

1 anala_m_r@yahoo.co.in ,2 shobhatilak@rediffmail.com

978-1-4673-1344-5/12/$31.00 ©2012 IEEE

II. METHODOLOGY

Figure 1 Architecture of virtual environment

Fig 1 shows the architecture of virtual environment.

Three VMs run parallel by sharing the resources available on physical host. These VMs share the resources of underlying physical machine with the help of Management/Decision module. This module is responsible to allocate CPU resources to all VMs. Each VM hosts its own application. Each application is associated with range of resources required, that is minimum to maximum. Depending on the availability and requirement of resources each VM is allocated with set of resources. Decision module is responsible to allocate or release the resources acquired by VM.

Fig 2 Petrinet model for resource utilization

Fig 2 shows the petrinet model illustrating resource consumption and release by a virtual machine. The two tokens at place P0 indicate two different applications. A tokens at place P0 indicates the readiness of an application. Tokens at places P3 and P4 indicates the availability of memory and CPU resources respectively. Transition T4 is fired only when an application is waiting at place P0, similarly for transition T0. The transitions T5 and T1 are fired when these ready applications requests for required resources. These applications wait in place P6 and P2 for their resources. Transitions T6 and T2 are fired when VMs completes serving respective applications. After the transitions T6 and T2 are fired the acquired resources are released by corresponding VMs. Whenever VM is ready to serve an application it waits in place P6 or P2 (depending on type of application) to receive required resources. Once an application and required resources (CPU and memory) are available the transition T7 or T2 is fired to indicate the execution of an application. When an application completes its execution the resources allocated to this application are released back to the resource pools. These resource pools are indicated by places P4 for CPU resource and P3 for memory resource.

III. LITERATURE REVIEW This section focuses on different approaches to

allocate resources dynamically. In virtualization resources are allocated to all VMs dynamically depending on availability and requirement.

Reference [3] addresses the overhead of dynamic resource allocation over static resource allocation. It concludes that due to dynamic resource allocation there is a loss in workload performance and system capacity. The paper demonstrates that performance and capacity overheads are higher for transactional workloads than for computational workloads. [4] presents a grey prediction control model used for dynamic allocation of resources in virtual machines based on control theory. The dynamic adjustment of resources to an application depends on the local resource demands of a node that hosts the application. Predictive control technique uses the historical data to allocate resources.

Reference [5] considers Web Server as the application workload in the virtualized computing environment, presents a Utility-based QoS constrained resource allocation policy which aims at optimizing the total utility of all the applications running in the system and achieves a reasonable resource utilization.

In [6] using feedback control theory, they present VM-based architecture for adaptive management of virtualized resources in cloud computing and model an adaptive controller that dynamically adjusts multiple virtualized resources utilization to achieve application Service Level Objective (SLO) in cloud computing. In [7], the VMs themselves self-adapt based on feedback about resource usage and availability. This paper defines a Friendly VM (FVM) to be a virtual machine that adjusts

2012 IEEE International Conference on Computational Intelligence and Computing Research

its demand for system resources allocated efficiently in a fair share manner with competing FVMs with the aid of control rules such as Additive Increase and Multiplicative Decrease.

[8] identifies challenges in applying control theory to dynamic sizing of a resource partition using CPU entitlement as the input and the mean response time or the relative CPU utilization as the output. The closed-loop performance of an adaptive integral controller for controlling relative utilization of a resource partition is evaluated. It presents a new adaptive controller for regulating the mean response time that incorporates information on measured CPU utilization. This demonstrates that better performance and robustness can be achieved with these controllers compared with other controllers. This paper considers only CPU as resource and results are demonstrated based on web server applications.

In [9] when multiple VMs or processes co-reside on the same physical server, the processor utilization which is measured does not provide an accurate estimate of the actual requirement. Worst, it may provide very poor estimates. This paper derives a simple, much more accurate alternative estimate of CPU demand and demonstrate its superiority experimentally. It also demonstrates that using this demand estimation framework in conjunction with dynamic resource allocation in a visrtualized environment greatly improves the effectiveness of dynamic placement, resulting in the overall performance of the individual VMs.

[10] presents a management algorithm for dynamic allocation of virtual machines to physical servers. The algorithm pro-actively adapts to demand changes and migrates virtual machines between physical hosts thus providing probabilistic SLA guarantees. [11] presents VScheduler, a system that dynamically adjusts processor resource configuration of VMs factoring in the virtual resources and a new mapping of VMs to physical nodes.

[12] Implements and validates a dynamic resource provisioning framework for virtualized server environments. The provisioning problem is posed as one of sequential optimization under uncertainty and solved using a lookahead control scheme. The proposed approach account for the switching costs incurred while provisioning virtual machines and explicitly encodes the corresponding risk in the optimization problem.

These authors try to give the best solution for dynamic resource allocation where each VM will get required resources. This paper aims at analysing the performance of dynamic resource allocation.

IV. WORKLOAD DESCRIPTION

There are different types of workloads the physical or a VM can serve, namely a web application, database application, an interactive game and computational workloads. In this paper an Apache JMeter is taken as an

application to analyze the performance of VM and the physical host.

The Apache JMeter is an open source desktop Java application intended to load test functional behaviour and measure performance. Apache JMeter was initially designed and tested for Web Applications. Later it was extended to test other functions. Apache JMeter tests performance on static and dynamic resources. It is used to test and analyze the performance of server under different load types. The server performance and strength can be analyzed by simulating a heavy load on a server.

Apache JMeter can load and performance test many different server types like Web-HTTP, HTTPS, SOAP, Database via JDBC and many more. Supports complete portability and 100% Java purity, Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.

V. RESULTS AND DISCUSSION

The scope of the paper is to discuss the performance of JMeter on PM with single and two CPUs. Then the JMeter is also run on VMs with one VCPU set and two VCPUs set. The performance of VM and PM are compared. The performance are also analysed for pinned and unpinned cases. Work load performance on native Linux.

Figure 3 Throughput/s Vs Number of cores

Fig 3 illustrates the performance of JMeter on native Linux for different number of CPUs assigned to PM. Since JMeter is multithreaded application it shows good performance as and when we add more number of CPUs. Work load performance on Virtual Machine

This section demonstrates the performance of VM when the JMeter application is run on VMs with different values of VCPUs.

2012 IEEE International Conference on Computational Intelligence and Computing Research

CPU usage and Credit scheduler

The resource requirements of VM vary depending on the application the VM is serving. It is possible to tune CPU usage of VM based on the application type.

The Xen domains will have one or more virtual CPUs (VCPUs), which regularly run on the physical CPUs. These VCPUs consume credits when they run.The xm vcpu-set command is used to specify the number of VCPUs for a domain and it is possible to change the number of VCPUs while a domain is running. To assign the CPU affinity, xm vcpu-pin command is used.

The scheduler designed by Xen team is a work-conserving scheduler since it minimizes the wasted CPU time. In case the available CPU is more than the aggregate of domUs the scheduler allocates resources among these domUs effectively. When there is contention the scheduler arbitrates fairly between the domains that want CPU. Using the xm sched-credit commands, it is possible to adjust CPU allocation on a per-domain basis.

The credit scheduler will nominate each domain attributes like weight, and optionally a cap. The weight attribute gives the relative CPU allocation of a domain -- if the CPU is scarce, a domain with a weight of 512 will receive twice as much CPU time as a domain with a weight of 256 (the default.) The cap sets an absolute limit on the amount of time a domain can receive, expressed in hundredths of a CPU (note that this number can exceed 100 on multiprocessor hosts.)

Jmeter results for different cap values for 1 cores

Figure 4 Throughput/s Vs Cap value for one CPU

Fig 4 illustrates the performance of JMeter on VM

with VCPU set to one. As it is seen from the diagram the performance remains constant when the cap value is increased more than 100. It is because the number of VCPUs allocated is one so it uses only one CPU which

can have maximum cap as 100. An increase in cap value will not mean the addition of resources. Jmeter results for different cap values for 2 cores

Figure 5 Throughput/s Vs Cap value for two CPU

Figure 5 illustrates the performance of JMeter on VM with VCPU set to two. We can see the improvement in the performance with increase in the cap value since VCPU is set to two, the application can use two CPUs which is equivalent to cap value of 200. Comparison of VM with VCPU=1 and VCPU=2

Figure 6 Comparison of VMs

Fig 6 shows the comparison of two VMs. The

throughput increases with VCPU=2 since the available resources are more compared to VCPU=1. Performance of VM when VCPUs are pinned and unpinned for a VM with one CPU.

Fig 7 shows the performance of VM with VCPU set to 1 for pinned and unpinned cases. In pinned context particular VCPU is assigned with physical CPU (VCPU 0 0) and it uses only that physical CPU. In unpinned case(VCPU 0 all) the VCPU 0 can schedule on any one of

2012 IEEE International Conference on Computational Intelligence and Computing Research

the available CPUs , VCPU all all option schedules any VCPUs on any physical CPUs.

Figure 7 Throughput/s Vs Cap for one CPU pinned and

unpinned cases Performance of VM when VCPUs are pinned and unpinned for a VM with 2CPUs

Figure 8 Throughput/s Vs Cap for two CPUs pinned and

unpinned cases

Fig 8 shows the performance of Jmeter when the VCPus are pinned to CPus and VCPUs are not pinned to particular CPUs. There is an increase in performance if VCPUs are not pinned to particular CPU since VCPU can schedule on any one of the free CPU dynamically.

Figure 9 Throughput/s Vs Number of Cores Comparison of VM with PM- Which is faster? There is no silver bullet which addresses all types of applications and performs well be it a VM or PM. It is proved that while certain type of applications reap benefits running on a VM, others do well on a PM. But by and large, the performance metrics of applications are almost on par as depicted in this paper measured by using JMeter. The workload JMeter gives nearly same performance on both VM and PM as shown in below Fig 9.

V. CONCLUSION The results demonstrate that an application running on VM and PM shows nearly same performance when the same numbers of resources are acquired by both VM and PM. Since virtualization supports multiple benefits like server consolidation, higher resource utilization and many more, it is always better to serve an application on VM than PM. If PM is used to serve the same application the entire resource is occupied or blocked by that application whereas if the same application is run on VM only the required amount of resources are used and remaining resources may be used by some other VM. This paper effectively demonstrates the performance of JMeter on VM and on PM and illustrates that JMeter gives near performance on VM and PM when it is executed. From the Fig 9 it is shown that their performances are almost the same for PM and VM.

REFERENCES

[1] P.Barham, B.Dragovic, K.Fraser, S.Harris, A.Ho, R.Neugebauer, I.Pratt, A.Warfield, “Xen and the art of virtualization”, SOSP 03 - symposium of operating systems principles- ACM, pp.164-177.

[2] Clark.B, Deshane.T, Dow.E, Evanchik.S, M.Finlayson, J.Herne, and J.Matthews, “Xen and the art of repeated research”, in USENIX Annual Technical Conference, 2004, pp. 135–144.

[3] Wang Zhikui, Zhu Xiaoyun, Padala Pradeep, Singhal, Sharad, “Capacity and Performance Overhead in Dynamic Resource Allocation to Virtual Containers”, 10th IFIP/IEEE International Symposium on Integrated Network Management- IM '07,2007, pp. 149 – 158.

[4] Xianghua Xu, Yanna Yan, Jian Wan, “Grey Prediction Control of Adaptive Resources Allocation in Virtualized Computing System”, in Eighth IEEE International Conference on Dependable, Autonomic and Secure Computing, DOI 10.1109/DASC.2009.41, 2009, pp.109-114.

[5] Jian Wan, Peipei Shan, Xianghua Xu, “A Utility-based Adaptive Resource Allocation Policy in Virtualized Environment”, in 8th IEEE International Conference on Dependable, Autonomic and Secure Computing, DOI 10.1109/DASC.2009.92 ,2009, pp.63-68.

[6] Qiang Li, Qinfen Hao, Limin Xiao, Zhoujun Li, “Adaptive Management of Virtualized Resources in Cloud Computing Using Feedback Control”, 1st International Conference on Information Science and Engineering (ICISE), 2009, pp.99-102.

[7] Zhang, Yuting ; Bestavros, Azer ; Guirguis, Mina ; Matta, Ibrahim ;West, Richard:, “Friendly virtual machines:

2012 IEEE International Conference on Computational Intelligence and Computing Research

leveraging a feedback-control model for application adaptation”, In Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments- VEE ’05, ACM Press. – ISBN 1595930477, pp. 2-12.

[8] Zhikui Wang, X.Z., Sharad Singhal, “Utilization vs. SLO-Based Control for Dynamic Sizing of Resource Partitions”, - 16th IFIP/IEEE Distributed Systems: Operations and Management DSOM , 2005, pp.133-144.

[9] Canturk Isci, James E. Hanson, Ian Whalley, Malgorzata Steinder and Jeffrey O. Kephart, “Runtime Demand Estimation for Effective Dynamic Resource Management”, in IEEE/IFIP Network Operations and Management Symposium – NOMS, 2010, pp.:381-388.

[10] Norman Bobroff, Andrzej Kochut, Kirk Beaty, “Dynamic Placement of Virtual Machines for Managing SLA Violations”, in 10th IFIP/IEEE International Symposium on Integrated Network Management- IM '07, 2007, pp.119-128.

[11] Hai Jin, Li Deng, Song Wu, Xuanhua Shi, ”Dynamic Processor Resource Configuration in Virtualized Environments”, IEEE International Conference on Services Computing, 2011, DOI 10.1109/SCC.2011.37, pp. 32-39.

[12] Dara Kusic, Jeffrey O. Kephart, James E. Hanson, Nagarajan Kandasamy, Guofei Jiang, “Power and Performance Management of Virtualized Computing Environments Via Lookahead Control”, IEEE International Conference on Automonic Computing, DOI 10.1109/ICAC.2008.31 ,2008, pp. 3-12.

2012 IEEE International Conference on Computational Intelligence and Computing Research