QA with Microsoft Test Manager and Lab

ManagementRofiqi Setiawan, Technology Specialist

Microsoft Indonesia

Self discoverable Bring your own deviceEdge facing

Modern business applications

Continuous services

Modern app lifecycleContinuous value delivery with modern business applications

DefineIdeation

DevelopIdea to working

software

MonitorImplement

SHORTENEDCYCLE TIMES

Conventional QA in the Modern app lifecycle

QA

Testing postimplementation. Mostly

manual with limitedautomation.

UAT

UAT post implementation

and systemstesting

Operationsreadiness

verification

Pre-deploymentverification

Document requirements as BRDs and

functional specs

Write code to implement

requirements

Quality

Enablement

Quality enablement practicesContinuous quality for continuous value delivery

Continuous qualityShortened cycle times

Acceptance test planning

Business goalsActionable

requirementsAcceptance

criteriaAcceptance test

plan

Acceptance test driven

development

Continuous acceptance testing

User

Interface

Services

Business Processes

Business Rules and Logic

Service integrations

Data access

Identity

Data

Automated testing

Manual testing

Automated testing

Lab management automation

Quality Enablement Solution Architecture

Work item tracking Test lab managementBuild automation

Extensibility interfaces

SCMTest execution

servicesDeployment automation

Reporting and BI

Lab infrastructureTest recorders and

runnersBuild engines

• Actionable feedback • Exploratory testing

• Requirements management

• Acceptance criteria definition

• Developer testing• Build automation• Actionable

diagnostics

• Test planning and management

• Test execution and defect reporting

• Lab management

• Production monitoring• Integrated incident

managementQuality metrics and

reporting

Role tailored tools

Microsoft Solution

The Microsoft Solution

Traceability with work item tracking

Requirements management

Test lab management

Source code management

Extensibility interfaces

Team development

servicesTesting services

Build and deployment automation

Reporting and Business

Intelligence

Lab infrastructure Test recorders and runners

Build engines

• Microsoft Office• Feedback client

• Microsoft Office and SharePoint

• Microsoft Project• Browser apps

• Visual Studio IDE• Eclipse• Cross-platform

command line tools

• Microsoft Test Manger

• Microsoft Visual Studio Premium or Ultimate

Microsoft System Center Microsoft Office and SharePoint

Project planning and management

Git

Quality Enablement Solution Architecture

Work item tracking Test lab managementBuild automation

Extensibility interfaces

SCMTest execution

servicesDeployment automation

Reporting and BI

Lab infrastructureTest recorders and

runnersBuild engines

• Actionable feedback • Exploratory testing

• Requirements management

• Acceptance criteria definition

• Developer testing• Build automation• Actionable

diagnostics

• Test planning and management

• Test execution and defect reporting

• Lab management

• Production monitoring• Integrated incident

managementQuality metrics and

reporting

Role tailored tools

Manual Testing

Software testing with Visual Studio

Business challengesLong, opaque manual testing cycles

“No-repro” defect reports

Repetitive and error prone

Unable to track tested vs. untested code

SolutionsActionable bugs containing data to reproduce defects

Code coverage reports show untested features

Automated fast-forward

Connect dev and test with tools to instrument and repro

Business valueReduce dev-test “ping pong” to cut waste

Speed delivery of high quality software

Delivered software meets user and value expectations

Manual Testing

DemoManual Testing

Organize testing in multiple configurations

Playback a prior recording to save time with frequently-repeated setup

steps and prerequisites

Bind data to controls, to support multiple iterations

and future automation

Pause, save, close, return, and pick up where you left off

IntelliTrace Logs

System logs & configurationDetailed status and history visible

to all team members

Audio & Video recordings

Get an overview of the test run with summary pie charts, or drill

into the details for each test.

Analyze code coverage for manual tests to determine which specific parts of your code were tested .

Associate manual test runs with builds and environments.

Exploratory Testing

Software testing with Visual Studio

Business challengesPrescribed test cases can’t anticipate all possible bug scenarios

Ad-hoc testing often results in “no-repro” defects

Unable to validate original bug has been fixed

SolutionsLeverage inherent tester creativity with ad-hoc testing

Record ad-hoc test steps

Create test cases from bugs to reproduce complex scenarios or to verify bug fixes later

Business valueDiscover unknown scenarios and hidden defects

Eliminate manual capturing of test steps to cut waste

Increase quality in delivered software

Exploratory Testing

DemoExploratory Testing

Create rich text comments with screenshots while

exploring

Exploratory sessions can be recorded to increase

reproducibility

Explore your application to uncover new defects

The Test Runner docks on the left side of the screen and automatically records

your steps as you explore

Comments and steps are automatically added

The exploratory test session is paused while you create a bug

report

File new bugs without ending the exploratory test session

Attach the video recorded during the exploratory test session for

maximum reproducibility

Quickly create an automatically linked test

case for this bug

Add verification values to indicate the expected result

Session is paused while creating test cases

Test steps are added automatically

Comment block reflects the added bugs and tests

Continue exploratory testing right from where you left off

Quick access to bugs and test cases created during the session

Lab Management

Software testing with Visual Studio

Business challengesModern apps run in diverse environments

Difficult to repro issues found

Replicating environments for test is time-consuming, costly

SolutionsVirtualization reduces cost to provision environments

Templates allow a wide range of baselines to be spun up quickly and on demand

Snapshots capture states for reproducing test conditions and verifying defects

Business valueDramatically reduced hardware and maintenance expenses

Improved quality and reliability of apps across platforms

Operations engaged in earlier phases of testing

Lab management automation

On-premise | Cloud

DemoLab Management

Create a new environment using virtual machine templates in

SCVMM

Create an environment using machines running on

SCVMM

Create new environments on existing machines

Quickly choose multiple test suites to run after

deployment

Choose your environment, and build, deploy and test in that environment

from a single dialog

Snapshot environments during manual testing to increase

reproducibility

Spin up virtual test environments from within Test Manager, or attach a virtual environment to a manual testing session

Test run results are attached to specific builds and environments

Collect all the data required to recreate bugs from virtual

environments

The Microsoft differentiation

Role tailored tools Hybrid IT capableContinuous quality practices

ApplicationLifecycle

Continuous value delivery

New value

Call to Action

• Take Microsoft Virtual Academy (MVA) courses• http://aka.ms/mvasoftwaretesting

• http://aka.ms/mvaswtestingvs

• Use TechNet Virtual Lab (http://aka.ms/virt-labs) • http://aka.ms/VLTestPlanning

• http://aka.ms/VLTestCase

• http://aka.ms/VLExploratoryTest

• Use Azure Trial (http://azure.microsoft.com/en-us/pricing/free-trial/)

• Download Visual Studio ‘14 CTP• http://aka.ms/vsctp14