What is New in Visual Studio 11 for Testers

Charles SterlingSenior Program Manager Microsoft Corporationhttp://blogs.msdn.com/visualstudioalm

GSJGD

Agenda

Review of what we delivered in Visual Studio 2010 + enhancements for Visual Studio 11

Paradigm Shift to Exploratory Testing

Enhancements in Lab Management

Microsoft Test Manager enhancements for Visual Studio 11

Huge Performance improvementsMulti Line Test CasesRich Text Test CasesTest Data ReductionSuite CopyingTest Case access from the Test RunnerMark Test Case Status directly in MTM (no need to launch the runner)Testing Metro ApplicationsEXPLORATORY TESTING!!!!!!!!!!

Demo

Agenda

Review of what we delivered in Visual Studio 2010 + enhancements for Visual Studio 11

Paradigm Shift to Exploratory Testing

Enhancements in Lab Management

The Agile Workflow

Why - Exploratory testing

Enables focusing customer value rather than spec complianceFree Testers from up front planning and find bugs fast and early without overheadFocuses on working software!

Exploratory testing – myths

1. Exploratory testing is ad-hoc, random testing2. There is no way to measure exploratory testing3. Bugs found through exploratory testing are hard

to repro

Using “Testing Tours” to put a method to the madness

Guidebook tour Back alley tour Supermodel tour You can make up your own…

Sample tours – how we did it

The Reflex tour – tests responsiveness of the IDE The Idiot tour – tests F1 inline help on each element The Palatial tour – tests our multi monitors experience The Footprint tour – tests virtual memory usage The Budget tour – tests experience on a low end machine

Measuring exploratory testing

Right metrics are like a fingerprint of your team: no silver bullet

Distinguish between app quality metrics and metrics that direct exploratory testing

Metrics – how we did it

Code coverage across toursStory complexity to bugs ratioSession duration for storiesSession roll up per tour

Heat maps – how we did it

Finding reproducible bugs

A couple of puzzles

How many software testers does it take to change a light bulb?None. Testers just noticed that the room was dark. Testers don't FIX bugs, they just find them.

How many developers does it take to change a light bulb?“What's the problem? The bulb at MY desk works fine!”

Rich bugs – how we did it

Automatic repro stepsScreenshotsDesktop recordingSystem information Intellitrace logs

Demo

Test documentation – how we did it

Test cases automatically created while doing exploratory testingCan save artifacts from exploration like expected imageWith each sprint, regression suites are automatically ready

Demo of Exploratory Testing

Agenda

Review of what we delivered in Visual Studio 2010 + enhancements for Visual Studio 11

Paradigm Shift to Exploratory Testing

Enhancements in Lab Management

Lab Management Enhancements for Visual Studio 11

Use physical infrastructure Zero Configuration Support for clustered Hyper-V hosts Scale to 1000’s of VMs Scale by cloning VMs via Network Isolation New Environment viewer Deploy using deployment scripts Full support SCVMM 2012

http://blogs.msdn.com/anutthara http://twitter.com/anutthara

Q&A

Please fill the feedback forms!