Hands-On LabIntroduction to Quality Tracking with Visual Studio 2010

Lab version: 1.0.0

Last updated: 5/24/2023

CONTENTS

OVERVIEW................................................................................................................................................. 3

EXERCISE 1: VIEWING TEST PLAN STATUS IN MICROSOFT TEST MANAGER 2010........................4

EXERCISE 2: VIEWING QUALITY DASHBOARD.....................................................................................6

EXERCISE 3: VIEWING REPORTS IN MICROSOFT VISUAL STUDIO 2010...........................................8

EXERCISE 4: CREATING A COMPLEX QUERY FOR ADVANCED REPORTING.................................11

EXERCISE 5: CREATING A REPORT IN MICROSOFT EXCEL.............................................................17

Overview

Microsoft Visual Studio 2010 allows detailed and elaborate tracking and reporting of testing results. This lab will introduce you to working with various reporting features available with Visual Studio 2010.

This hands-on-lab is one out of a number of labs that deal with Microsoft Test Manager 2010. The recommended order to complete the labs in is as follows:

Introduction to Test Planning with Microsoft Test Manager 2010

Introduction to Test Case Management with Microsoft Test Manager 2010

Authoring and Running Manual Tests using Microsoft Test Manager 2010

Introduction to Exploratory Testing with Microsoft Test Manager 2010

Introduction to Platform Testing with Microsoft Test Manager 2010

Introduction to Quality Tracking with Visual Studio 2010

System Requirements

In order to complete this lab you will need the Visual Studio 2010 virtual machine provided by Microsoft. For more information on acquiring and using this virtual machine, please see this blog post.

Exercises

1. Viewing Test Plan Status in Microsoft Test Manager 2010

2. Viewing Quality Dashboard

3. Viewing Reports in Microsoft Visual Studio 2010

4. Creating a Complex Query for Advanced Reporting

5. Creating a Report in Microsoft Excel

Estimated time to complete this lab: 60 minutes.

Next Step

Exercise 1: Viewing Test Plan Status in Microsoft Test Manager 2010

Exercise 1: Viewing Test Plan Status in Microsoft Test Manager 2010

In this exercise, you will learn how to view the status of a test plan using Microsoft Test Manager 2010.

1. Log in as Abu Obeida Bakhach (Dev) if you have not already done so. The password is P2ssw0rd (capital letter P, the number two, the letter s, the letter s, the letter w, the number zero, the letter r, and the letter d). Please see “Working with the Visual Studio 2010 Virtual Machine” for instructions on how to log into the VM.

2. Open Microsoft Test Manager from Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Test Manager 2010. This is a new tool dedicated to manual software testing that has been built from the ground up for Visual Studio 2010.

Note: Microsoft Test Manager allows testers to work with test plans, author and organize manual test cases, execute test cases, file bugs, and post results back to Team Foundation Server.

Figure 1Microsoft Test Manager starting up

3. When Microsoft Test Manager initially loads, it will connect to the most recently loaded test plan, which in the case of this virtual machine is the test plan named Iteration 2.

Figure 2Test plan contents for Iteration 2

4. Select the Plan tab and then select the Properties link to load the test plan properties window.

5. Scroll down to the bottom of the properties window to see the three pie charts that indicate the status of tests in the plan.

Figure 3Test plan properties window

Next Step

Exercise 2: Viewing Quality Dashboard

Exercise 2: Viewing Quality Dashboard

In this exercise, you will learn how to launch dashboards from Microsoft Test Manager 2010 and study the Quality Dashboard in particular.

Note: A significant portion of this lab is dedicated to demonstrating where to find reports and what type of information that they provide. The actual data that is contained within the virtual machine may differ from the screenshots shown in this document. In addition, it is important to point out that since it is sample data, it does not always realistically represent what you would expect to see if an actual software team was generating this data.

1. In Microsoft Test Manager, select the Track tab and then select the Project Portal link to load the project portal in Internet Explorer.

Figure 4Location of Project Portal link

Note: This virtual lab environment does NOT include a Quality Dashboard. The following steps and screenshots are for your reference only. You may go through the remainder of this exercise in a different VM where a Quality Dashboard has been pre-configured.

2. In the left pane, select the Quality link under the Dashboard section to open the Quality Dashboard.

Figure 5Quality Dashboard showing a variety of quality metrics

3. As shown in the figure below, the Quality Dashboard includes reports that answer the following questions about product quality:

◦ Are we making progress on running test plans?

◦ How are our builds doing over time?

◦ Are we fixing bugs?

◦ What is the quality of our bug fixes?

◦ Are we getting enough code coverage?

◦ How many lines of code are we churning?

Figure 6The Quality Dashboard answers a wide variety of questions

Next Step

Exercise 3: Viewing Reports in Microsoft Visual Studio 2010

Exercise 3: Viewing Reports in Microsoft Visual Studio 2010

In this exercise, you will learn how to access various reports available in Microsoft Visual Studio 2010.

1. Open Microsoft Visual Studio 2010 by going to Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010 if it is not already open.

2. In Team Explorer, expand Tailspin Toys to open the Tests report folder (Tailspin Toys | Reports | Tests).

Figure 7Locating test reports

3. You should see two reports in the Tests folder, Test Case Readiness and Test Plan Progress.

The Test Case Readiness report shows the state of preparation for test cases, including those that are ready to run.

The Test Plan Progress report helps track the progress of the overall test plan by showing the number of test cases that were ran, passed, failed, blocked, etc. For those test cases that were run, a state is assigned such as Aborted, Not Executed, PassedButRunAborted, NotRunnable, Disconnected, Timeout, etc.

4. In Team Explorer, locate and expand the Project Management report folder (Tailspin Toys | Reports | Project Management).

Figure 8Multiple project management reports are available in Microsoft Visual Studio 2010

5. Double-click the Stories Overview report to open it. This report shows each user story and the development progress, acceptance tests, and number of bugs that have been filed against it.

Figure 9Stories Overview report offers progress and related project information on each user story

6. In Team Explorer, locate and expand the Builds report folder (Tailspin Toys | Reports | Builds).

Figure 10Builds reports show quality, success and summary information of each build

7. The Build Quality Indicators report shows test coverage, code churn, and bug counts for a specified build definition. You can use this report to help determine how close portions of the code are to being release quality.

The Build Summary report helps determine the status of each build. It shows a list of builds with test results, test coverage, code churn, and quality notes.

This Build Success Over Time report helps track changes in code quality and also shows the test results for the last build of each day.

Next Step

Exercise 4: Creating a Complex Query for Advanced Reporting

Exercise 4: Creating a Complex Query for Advanced Reporting

In this exercise, you will learn how to create a complex query in Microsoft Visual Studio 2010 to provide more specific data than the default queries can provide.

1. Open Microsoft Visual Studio 2010 by going to Start | All Programs | Microsoft Visual Studio 2010 | Microsoft Visual Studio 2010 if it is not already open.

2. In Team Explorer, expand Tailspin Toys to the Queries node (Tailspin Toys | Work Items | My Queries).

Figure 11Location of My Queries node

3. Right-click on My Queries and select New Query from the context menu.

Figure 12Creating a new query

4. By default, there are already three query clauses (rows) provided when creating a new query. Change the query clauses so that the query looks for all test cases that are not closed as follows:

◦ First clause: Team Project = @Project

◦ Second clause: And Work Item Type = Test Case

◦ Third clause: And State <> Closed

Figure 13New query checks for test cases that are not closed

5. Select the dropdown box next to Type of Query and select Work Items and Direct Links. This will add an additional sub-query and show a secondary grid below the first.

Figure 14Changing the Query Type in the New Query Editor

6. Try not to confuse the secondary query grid with the previous query. The secondary query grid is used to find linked work items from results returned from the first query.

Figure 15Secondary query was added to be used on work items with links

7. Let’s modify the secondary query to find all closed bugs that are linked to the test cases returned from the primary query.

Change the first query clause to be Work Item Type = Bug.

Add a second query clause: And State = Resolved.

Figure 16Modifying secondary query to find linked bugs that are resolved

8. Now that the query has been defined, select the Run button to preview the data set that is returned.

Figure 17Location of Run query button

Figure 18Example results set from new query

9. Note that the query returns four test cases, only one of which has a linked bug. Since the Top level work items setting is set to Return all top level work items, all of the test cases are returned – even those that do not have linked bugs.

10. Select the Save Query button.

Figure 19Location of Save Query button

11. In the Save Query As dialog box, type Test Cases With Resolved Bugs for the Name field. Finally select the Save button to save the query and return to the Query Editor window.

Figure 20Naming and saving new work item query

12. Change the Top level work items setting to the Only return items that have the specified links option as shown in the figure below and then select the Run button to preview the results once again.

Figure 21Modifying query to only show work items with specified links

13. The result, which you can see in the figure below, is that only one item is returned since it is the only test case with a linked bug.

Figure 22Result set from modified query

Next Step

Exercise 5: Creating a Report in Microsoft Excel

Exercise 5: Creating a Report in Microsoft Excel

In this exercise, you will learn how to create a report in Microsoft Excel using data provided by a work item query in Visual Studio 2010.

1. In Team Explorer, expand Tailspin Toys to the Team Queries node (Tailspin Toys | Work Items | Team Queries).

Figure 23Using Team Explorer to locate default team queries

2. Right-click on the My Bugs query and then select Create Report in Microsoft Excel.

Figure 24Right-click on a query to create a report in Microsoft Excel

3. After Microsoft Excel loads and displays the New Work Item Report dialog box, select All Reports and then select the Finish button.

Figure 25New Work Item Report selection dialog box

Note: The report generation process may take some time to complete depending upon the machine used and the amount of data available.

4. After the report has been generated, you will see a worksheet that contains a table of contents to the generated reports. Select the first link named Stack Rank from the table of contents to navigate to the worksheet named 1.1 Stack Rank.

Figure 26Table of contents for report generated in Microsoft Excel

5. The Work Item Count by Stack Rank report is just what it sounds like - for selected work items, it shows the top 10 stack rank values by work item count. Many different dimensions of filtering can be applied to explore this data set.

Figure 27Stack rank report for work item count

6. Although the data in the virtual machine does not provide much data to work with for demonstrating the value of the reports, go ahead and navigate through some of the other reports to see what is available by default.

To give feedback please write to VSKitFdbk@Microsoft.com

Copyright © 2010 by Microsoft Corporation. All rights reserved.