1

Taking Enterprise Testing to Cloud

Testing and Cloud Computing

Arun Pareek (arrun.pareek@gmail.com

Sr. Analyst, SOA Development

Taking Enterprise Testing to Cloud

For more articles by the author visit beatechnologies.wordpress.com

2

The Precursor

According to me every idea has a precursor, a path and finally a vision.

• The precursor typically lays down the proponents that existed in the generation of the idea.

• The path is a sequence of activities/actions that must be undertaken to make the idea work and grow.

• The vision is the idea in its most mature state. This is the stage when the idea would be full ripe and contribute to maximum yield.

3

The Scenario NowSoftware testing is a notoriously challenging activity. Running large collections of test cases as part of a regression/functional test suite can consume considerable time and resources, often precluding their use in an interactive setting.

Traditional approaches to this problem focus on reducing the number of regression tests that need to be run after a change is made to the system.

4

Is Cloud there to Stay?It's always exciting to be on the front end of a trend. Some turn out to be short-lived fads while others, such as the electronic distribution of music, disrupt entire industries. There are some who see cloud computing as a fad. Those who share that view tend to look at the Cloud as if it's an all or nothing proposition; if you can't move your entire infrastructure to the Cloud it must certainly be a passing fancy.

But the Cloud is here to stay. There are many applications, like compute intensive statistical analysis, business intelligence and testing, that are incredibly well-suited for cloud computing.

While many companies still debate whether it makes sense for them to look to the cloud for their application and data center needs, there should be no debate that the cloud is the place to go for testing.

5

Explaining Through ScenariosWell for most part of my life I have believed that numbers and real life scenarios do the best job when it comes to explaining.

And for any idea to seed well it is upmost important that we realize not only the idea’s potential but also what we are lacking now and how the proponent can help us address it.

Let me quickly throw around some basic problems/challenges that are faced by all of us in our real IT worlds.

6

Prototyping/Discovery– Problem

You need to build a high speed transaction processing system and you do not know how much memory and CPU would be optimal for the architecture.

– Pre Cloud

In most of the projects that you will work, you would typically had to purchase your hardware early in the life-cycle due to long procurement cycles and setup time. Usually you have to order this way before you even started prototyping which means you had to take your best guess. The consequences would normally be that you would have to tailor your solution around the hardware that you purchased or existing hardware that you were forced to use. The end result was usually a less than optimal performing solution due to hardware constraints. Instant legacy, how nice!

7

Performance Testing– Problem

It is very time consuming and expensive to create performance tests that can simulate millions of transactions from multiple touch points.

– Pre Cloud

Unless you were testing on a soon to be production system, you had no way to do true performance testing without having a replica of your production environment, plus enough hardware/infrastructure to simulate loads that far exceed your current projections. After all, it makes no sense to only test the amount of traffic that you expect to see in production without testing spikes and testing for future growth. And are there really any load when the system starts to break or perform sub-optimally. In the on-premise world, buying additional capacity just to do performance tests coupled with the incredibly high costs of software licensing from tools like Loadrunner is just not feasible. So we are rarely able to simulate with large enough loads to test scale and simply can’t afford to provision test systems that are equivalent to the production environments.

8

Explaining Through ModelsIn load or performance testing you want to subject your software to an equal amount of virtual users as would be in an actual production environment. You would then want to monitor the performance under the specified load, usually on a target-test environment identical to the production environment, before going live.

9

The Traditional Way

• Good testing softwares.• Sufficient testing hardware for your testing environment• Target testing hardware similar to your production hardware for

your target-test• environment with your software installed• Sufficient bandwidth for realistic simulation• Skilled professionals to run, monitor and analyze the results of the

tests• Other equipment to simulate realistic environment

Traditionally, to support such a test, you would

need the following components in place:

Cost of such an environment and the priority of load/performance/functional testing versus on-going production tasks, bring decision-makers to make compromises and find less adequate solutions to the problem of validating performance under load and throughout time.

10

Explaining Through TablesCompromise Cause ResultsMonitor the production environment instead of stress-testing it

Lack of budget for testing The system is tested under average load only. Odds are that during peek load the system will crash

Test the production environment instead of a similar target-testing Environment

Lack of budget for adequatetarget-testing environment

Testing is performed only during idle times. A real problem is the risk to the production environment

Test the external production environmentusing test hardware hosted with-in the organization's facility

Lack of budget for hosting at a co-location hosting facility

Enterprise bandwidth is a bottle neck. Simulating a nonrealistic environment

Test small amount of users or use limited testing Hardware

Lack of budget for testing hardware

Either receiving a non realistic simulation or the hardware is over loaded and affects the results

Neglect performance/stress testing all together

Lack of budget and knowhow Risk of not sustaining promised performance and lost of revenues.

Now I hope that the idea has seeded well. With these numerous ways of presentation I believe we can now at least understand as what is the potential problem. Get along to the ‘PATH’ part of the idea to know how we can try to overcome them.

11

The Path

According to me every idea has a precursor, a path and finally a vision.

• The precursor typically lays down the proponents that existed in the generation of the idea.

• The path is a sequence of activities/actions that must be undertaken to make the idea work and grow.

• The vision is the idea in its most mature state. This is the stage when the idea would be full ripe and contribute to maximum yield.

12

About Cloud-ComputingCloud-computing is actively revolutionizing the traditional IT scenery. It's all about paying only for what you consume. It is a legitimate way for companies to significantly lower costs of infrastructure while increasing operations.

Cloud-computing enables non-expensive and rapid system deployment. Unlimited computing resources and pay-as-you-go model allow each business to scale according to actual business activity increase and pay only for its consumption.

Cloud-computing is supported and lead by industry giants including Sun Microsystems, IBM, Amazon, Google, Microsoft, Yahoo and others.

The diagram below (taken from Wiki) compares traditional IT expenditure to cloud-computing on a cost per increasing number of users ratio.

13

Time for Some AnswersRemember we discussed a few scenarios earlier. Well have a look at some possible answers to them.

– Post Cloud

The procurement and setup issues go away because resources can be provisioned in minutes in the cloud. This would allow to get deep into the prototyping phase and then test design against numerous combinations of CPU, memory, and database subsystems. The result would be a complete opposite of the old days. Instead of rigging your design to meet the constraints of hardware, you would be able to provision the virtual servers that met your performance requirements based on actual test results. Isn’t it nirvana!

These scenarios can be tested in the cloud regardless if the actual solution to be tested was built in the cloud or on-premise. Even if you are building your solution on-premise, you can still do the initial prototyping on the cloud to figure our how much CPU and memory you need to buy.

So even if your company fears the cloud as a solution for deploying applications and building data centers, there is no excuse for continuing to spend way too much time and money testing things the “way we have always done it”.

14

The 'Cloud' Way (Enterprise Testing in Cloud)

With cloud based testing compromises are no longer necessary. Cloud based testing leverages cloud computing resources and models to enable all aspects of testing in a highly cost-effective manner.

With cloud based testing you have unlimited resources at your disposal, paying only for what you consume, only when and if you consume it.

Cloud based testing can combine state-of the-art open-source technology and cloud-computing resources to support your entire testing requirements at a reasonable cost. Unlimited resources, rich scripting and graphical functionality, zero setup time. It takes only a few hours to launch a test of 30,000 concurrent users, complex database transactions and authenticated users. While in the past for a medium size organization, testing solution cost tens of thousands of dollars, now it's down to hundreds of dollars per month – all included. For small scale organizations it's only tens of dollars per month.

Build your testing environment in the cloud. A virtual environment TCO varies from about 20% to 70% of a real one. Your virtual environment is hosted elsewhere and managed by you, ready to execute whenever you require it. Leverage cloud resources as your load-testing hardware using whatever testing software you already have, reducing the need to purchase and maintain your own hardware. Unlimited resources are available. You pay only for what you consume.

15

A sample Cloud ProviderSee what are the various infrastructure and software based services a Cloud provider can provide. We take Cloud resources that are hosted with-in Amazon EC2 for the purpose of this presentation.

• From single core to 20 cores.

• 1.7 GB memory to 7GB memory.

• High and low performance I/O.

• 32bit to 64bit.

• Unlimited disk space

• 1Gb/s throughput for each instance.

• Operating systems• MS Windows• Linux/Unix• And more

• Data bases• Oracle• MS SQL• MySQL• And more

On top of what already exists in the cloud, we can install almost any other proprietary or commercial application.

16

And its sample PricingOn-Demand Instances

On-Demand Instances let you pay for compute capacity by the hour with no long-term commitments. This frees you from the costs and complexities of planning, purchasing, and maintaining hardware and transforms what are commonly large fixed costs into much smaller variable costs.

It wouldn’t take rocket science to figure out how these pricing would compare with the procurement, configuration and maintenance of physical hardware.

17

The Journey to Your Cloud

Phase I : IT Production

In this first phase virtualize about

20 percent of the testing

infrastructure. The goal here

should be typically pooling of resources to reduce costs,

increase stabilization and simplify testing.

For example: Testing usually

involves repeatedly setting up new instances of code. It would be a lot easier to

provision a virtual instance of a

machine than to physically

provision and procure

hardware.

Phase II : Business

Production

This phase involves moving

virtualization into Tier-1 production systems. Virtualize

about 50 to 80 percent of the

testing infrastructure.

There is an increased focus on

benefits such as ensuring increased

performance, scalability and

usage.

Phase III : IT as a Service

This phase represents the end

state for the journey to the cloud. This ensures that all

testing infrastructure is

moved to the cloud. This would enable

acute levels of operational

flexibility and enterprise wise cost

savings.

18

Targeting Five Key Benefits

Maximize performance while meeting service level agreements.

Increased Agility and Test Execution.

Improve Quality of Service with Dynamic Scaling and High Availability.

Increase Hardware Utilization and Reduce Costs through Consolidation.

Simulate Real World Testing Scenarios.

19

The Vision

According to me every idea has a precursor, a path and finally a vision.

• The precursor typically lays down the proponents that existed in the generation of the idea.

• The path is a sequence of activities/actions that must be undertaken to make the idea work and grow.

• The vision is the idea in its most mature state. This is the stage when the idea would be full ripe and contribute to maximum yield.

20

Where Companies should MoveEmergence as a Private Cloud Provider

Every idea need to have a vision. A long term projection of where the idea should land. So does this idea of mine has.

The idea whilst in its inception and path would definitely show how moving Enterprise testing to virtualized cloud based environment will add key cost and other benefits, it would also provide a long term perspective.

What companies should aim in the immediate long run is try and become a provider or adopter of Private Cloud based offering. Thiss would build in house expertise in server/platform/virtualization technologies and this is definitely a way forward. The ‘precursor’ and the ‘path’ will mark a venture in this direction. Doing so will improve in house expertise on using Cloud Based environments/platform that can eventually be build out and extended as a service.

In a recent Forbes Insights survey about 84 percent of the IT executives said that they have plans to use private clouds. In the same study 9 out of 10 technology executives said that they are under pressure to cut infrastructure costs, adjust service levels to meet changing needs and deliver applications with greater speed. These are issues addressed by private clouds – that is clouds in which the IT organization controls and manages all IT resources, whether external or internal.

And every Organization has to become that IT Organization.