Using VMware for More Productive and Efficient Software Testing

HB Farris, Practice ManagerSoftware Quality Management

Genilogix, LLC

Agenda

• Welcome and Introductions• Overcoming Physical Test Lab Challenges• Optimizing Virtual Test Labs • Best Practices and Recommendations

Company Profile - Genilogix

• Specialize in Quality Assurance Services– Test Automation– Virtual Test Labs

• Developed Solutions for Virtualization– VMware Lab Manger Add-in for HP Quality

Center– MonitorBridge for VMware

• VMware VIP Enterprise and HP Platinum Partner• Inc 500 information technology company

Where we are today

So you’re using test automation tools to…• Automate regression testing• Shorten test cycles • Ensure process and consistency

However...• Your physical test lab resources have become the

new bottleneck to efficient testing

Focus of this session

We’ll show you:– Best practices for testing in virtual test labs with:

• VMware Lab Manager• HP Quality Center and QuickTest Professional• VMware Lab Manager Add-in for HP Quality

Center

Common Test Lab Challenges

• Inefficient provisioning of test environments – Setup, teardown, reset and configuration time– Process, time and budget dependency on IT

Customer Example: Preparing Test Environments

– Maintenance Tasks – Non-test related tasks such as Imaging, Installing, Cleanup, or Locked for Triage.

– Testing Tasks – Test execution

0%10%20%30%40%50%60%70%80%

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov

Maintenance Tasks Testing Tasks

Image Machine Install Software Run Tests Await Triage Release for new test

20 Minutes 5 Minutes 20-120 Minutes 1-12 Hours!!

Common Test Lab Challenges

• Inefficient provisioning of test environments – Setup, teardown, reset and configuration time– Process, time and budget dependency on IT

• Constraints of physical hardware – Test matrix limited by available lab resources– Little asset sharing leads to server sprawl

Customer Example: Physical Test Lab Challenges

– Resources locked until test run complete

– Every task must go through imaging and product install

– No reuse of common configurations

– Remote access issues– Requires hardware specific

Images– Consumes lots of power and space

Common Test Lab Challenges

• Inefficient provisioning of test environments – Setup, teardown, reset and configuration time– Process, time and budget dependency on IT

• Resource requirements – Test matrix limited by available equipment– Little asset sharing leads to server sprawl

• Collaboration of development and testing teams– Reproducing environments to troubleshoot problems

Virtualize your test lab!Virtualize your test lab!

Virtual Lab Automation with VMware Lab Manager

• Built on VMware Infrastructure 3– Run dozens of workloads as virtual machines on fewer

physical servers to reduce test lab equipment cost

Virtual Lab Automation with VMware Lab Manager

• Built on VMware Infrastructure 3• Rapid provisioning portal

– Set up, remove and reset configurations on demand– Deploy multiple instances of the same configuration

Virtual Lab Automation with VMware Lab Manager

• Built on VMware Infrastructure 3• Rapid provisioning portal• Virtual Machine Image Library

– Capture and share entire test environments on defect– Use a fraction of the storage space normally required

Best Practices for Testing in Virtual Labs

• Setting up Templates• Avoiding Test Resource Collisions• Environment Impact Tests• Test Data Management• Support for Global Users

Virtual Machine Library –Deploy in Seconds

Setting up Templates

• Create multi-level templates– Level 1: OS– Level 2: OS + Patches + Virus Scan– Level 3: OS + Patches + Virus Scan + Apps

• Be consistent with your naming conventions– E.g. Project – OS – App – Build - Defect

Avoiding Test Resource Collisions

• Different types of testing cannot be executed simultaneously in same environment

User Acceptance Testing

Regression and Integration Testing

Installation and Upgrade Testing

Test Environment

Solution: Clones and Network Fencing• Set up multiple identical environments for all your

testing needs• Linked clones and delta disks reduce storage costs

Regression and Integration Testing

Installation and Upgrade Testing

Configuration AOriginal

Configuration CClone of A

Configuration BClone of A

ESX Server Pool

User Acceptance Testing

Best Practices for Testing in Virtual Labs

• Setting up Templates • Avoiding Test Resource Collisions• Environment Impact Tests

– Installation, Upgrade and Patch testing

• Test Data Management– Maintain test data through data destructive test cycles

Solution: Store Application State

• Snapshots and Revert Points• Capture to Library and Checkout of Library

Pristine Data State

Test Execution

Unknown Data State

Revert to Snapshot orCheckout Library Configuration

Best Practices for Testing in Virtual Labs

• Setting up Templates • Avoiding Test Resource Collisions• Environment Impact Tests• Test Data Management• Environment Support for Global Users

– Testing scheduled when environment support is not available

– Environment issues impede test progress

Solution: Web Portal • All teams can access web portal and manage their

test environments• Configurations can be shared or private• No client software required

How VMware Lab Manager Works

VMware Lab Manager Image Storage Library

VMware InfrastructureVirtualized Server Pool

Automated Virtual Lab

LAN/SAN

QA Engineer

User selects a multi-machine configuration, clicks deploy.

VMware Lab Manager determines the best host servers, then deploys the machines.

Once deployed, user directly interacts with the machines, as if sitting at each console

Best Practices for using HP Quality Center and the VMware Lab Manager Add-in

• Schedule test environment deployments• Execute automated tests against virtual test

environment • Capture application state when defect is detected• Preserve application test data

Automation of VMware Lab Manager Operations in HP Quality Center

Integrating VLM Operations with Automated Tests

Utilize Quality Center Scheduling

Capture Application State on Defect

Process Overview

VIRTUAL LAB AUTOMATION

VMware Lab Manager Integration with HP Quality Center

WebPresentation

Servers

Staging Servers

TestServers

Conclusion: Virtual Lab Automation• Virtualize your test lab!

– Start by leveraging existing servers and local storage– Import physical test configurations as virtual machines

• Use test automation tools!– Included automated provisioning and capture of test

environments in your tests• From 2007 Customer Survey:

– Time is the biggest item “saved” through virtual lab automation

– Average 3 days per environment setup– 50% increase in defects found prior to

production/shipment

Next Steps

Contact Genilogix for more information:• Online Demo• Evaluation• www.genilogix.com/vmware• info@genilogix.com • 1-800-780-5110