04b | Manage Test Execution (2 of 2) Steven Borg | Co-founder & Strategist, Northwest Cadence Anthony Borton | ALM Consultant, Enhance ALM

04b | Manage Test Execution (2 of 2)

Steven Borg | Co-founder & Strategist, Northwest CadenceAnthony Borton | ALM Consultant, Enhance ALM

• Run Tests

• Perform Exploratory Testing

• Manage Bugs

• Use Lab Center

• Analyze Recommended Tests

• Perform Analysis

• Manage Work Items

Module Overview

Click to edit Master subtitle style

Microsoft Virtual Academy

Manage Bugs

What the Study Guide says…

• Manage bugs– tracking bug metrics (bug trends, status)– verifying bugs (create Test from bugs)– analyzing bug reports–managing bug workflow

DEMO


Manage Bugs



Use Lab Center


• Use Lab Center– creating new environments– creating copies of environments– running Tests on remote environments– executing Test Case in a Lab Center Environment

Lab management automation

Lab management services

Environment templates Environment provisioning Environment snapshots

Development and Testing

Environments Development environments

Repro environmentsTest environments

Continuous Build-Deploy-TestModern applications need to be deployed and tested in multiple environments.

Whether you store your test environments on bare metal or in the cloud, Lab Manager enables automation and increases collaboration, reducing the build-deploy-test cycle time.

Create a new environment using virtual machine templates in

SCVMM

Create an environment using machines running on

SCVMM

Create new environments on existing machines

Automation: engageWhen developers and testers collaborate, quality wins.

Developers build the application, deploy it to an environment and run automated test cases in a single step, preparing the way for testers.

Quickly choose multiple test suites to run after deployment

Choose your environment, and build, deploy and test in that environment

from a single dialog

Manage EnvironmentsTesters spin up environments to run manual tests, and test agents record all the data required to recreate any bugs.

Snapshot environments and attach to bug reports so developers can easily reproduce in the exact environment where the defect was found.

Snapshot environments during manual testing to increase

reproducibility

Spin up virtual test environments from within Test Manager, or attach a virtual

environment to a manual testing session

Intelligent analysisEnable data collectors and generate detailed test results to ensure any bugs that are found are actionable.

Team Foundation Server keeps track of all the pieces, keeping your entire team on the same page and reducing time to repair.

Test run results are attached to specific builds and environments

Collect all the data required to recreate bugs from virtual

environments

DEMO


Use Lab Center



Analyze Recommended Tests


• Analyze Recommended Tests– selecting the build in use– comparing the current build to a previous build– viewing Recommended Tests– analyzing related Work Items

DEMO


Analyze Recommended Tests



Perform Analysis


• Perform analysis– analyzing reports (Requirements-User Stories)– analyzing by Test Suite– analyzing by configuration– identifying areas where quality is low– identifying Test Plan status

Software testing with Visual Studio

Team Foundation Server

Visual Studio IDEMicrosoft Test Manager

Unit testing UI automation testing

Performance testing Load testingCode quality

analysisTest planning and

managementTest case

management Lab manager Manual testing

Exploratory testing

Microsoft Feedback Client

User testing

Work item tracking Source control Lab management services Test execution services Build services Reporting and BI

Pluggable components

Test runners Build engines Lab environmentsDiagnostic data collectors

Monitoring Quality Metrics

Users Solution managers

Developers Testers Operations Stakeholders

• Actionable feedback • Exploratory testing

• Requirements management

• Acceptance criteria definition

• Developer testing• Build automation• Actionable

diagnostics

• Test planning and management

• Test execution and defect reporting

• Lab management

• Production monitoring• Integrated incident

managementQuality metrics and

reporting

Role tailored tools feed data to transparent quality reports

Data Warehouse

Stories Overview

Test Failure Analysis

Bug Reactivation Bug Trends Custom

SSRS/SSIS Pre-built and fully customizable cross project reporting

Visualize qualityThe build report is a barometer of quality and a harbinger of success or failure.

As the development cycle nears the end, defects and failed builds should trend towards zero. Build quality reports provide teams clear insight into the development process and can indicate where teams need to focus their efforts.

Filters allow you to control which builds you see

View the test and code coverage results for each build to determine

quality

Right out of the gateWaiting until the end of a cycle to address defects can adversely impact schedule and resources.

Desired ATDD behavior should see a high spike in failing acceptance tests early in the cycle with the spike tapering down as the cycle progresses indicating that tests are being run early and uncovering defects early on.

Bug bashingDevelopment teams can use defect tracking reports to estimate levels of deliverable code quality and to provide insight into a team’s ability to address defects.

A defect should ideally be detected and solved once, with a corresponding test implemented to ensure the fix addresses the desired behavior. Bug reactivations should remain close to zero.

Filling in the gapsThe only thing constant is change and code that changes must be tested.

To ensure your code base is fully covered by test leverage the code coverage report and identify your testing gaps. Establishing a code base fully covered by tests ensures new changes will not break your code!

Line-by-line highlighting illustrates which code is covered, uncovered, or

partially covered by test

Results show which methods are covered by test and total coverage is

indicated by a percentage

I can see clearly nowA picture is worth a thousand words and a Stories Overview report gives that big picture!

The Stories Overview gives a comprehensive view of each implemented user story including completion status, test results and bug status. A thousand words, indeed!

Track total percentage hours completed by user story

View real-time status of each user story

Test results quickly display quality status of each user story

Track active and resolved bugs by user story

DEMO


Perform Analysis



Manage Work Items


• Manage Work Items– validating requirements–Work Item relationships (eg, what it means when a test

case is associated with a requirement)– creating Work Item queries– performing bulk updates in Microsoft Excel

DEMO


Manage Work Items

• Become very familiar with work items, work item queries, and how work items are related

• Understand the lifecycle of a bug and its associated test case

• Know what Excel reports are available for each process template

• Know what SSRS reports are available for each process template

• Understand the basics of the Lab Center tab, including environments and an overview of Lab Management

EXAM BEST BETS

©2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Documents

04b | Manage Test Execution (2 of 2) Steven Borg | Co-founder & Strategist, Northwest Cadence Anthony Borton | ALM Consultant, Enhance ALM