Software Testing with VS2010

Software Testing with Visual Studio 2010

Quality debt is like a Tetris game gone bad. Start out sloppy, and end up never being able to clean things up.

Tobias Fors on Twitter

About this documentVersion Version 0.1History First versions: February 2011

Tom VercauterenRelated Documents


Contents

1. Quality Assurance Process........................................................................................................41.1. Write User Stories............................................................................................................41.2. Illustrate using examples.................................................................................................51.3. Automate the examples literally......................................................................................61.4. Write unit tests................................................................................................................71.5. Write code until unit tests pass.......................................................................................81.6. Write code until examples pass.......................................................................................81.7. Tester executes exploratory tests....................................................................................81.8. End User executes acceptance tests................................................................................91.9. Software is released.........................................................................................................9

2. Test Automation..................................................................................................................... 102.1. Automated below the UI...............................................................................................102.2. Automated using the UI.................................................................................................102.3. Performance and Load Tests.........................................................................................10

3. Scrum in TFS 2010...................................................................................................................123.1. Product Backlog.............................................................................................................123.2. Product Burndown.........................................................................................................123.3. Velocity..........................................................................................................................133.4. Sprint Planning...............................................................................................................143.5. Sprint Burndown............................................................................................................16

4. Microsoft Test Manager..........................................................................................................174.1. Connecting to a Team Project........................................................................................174.2. Testing Center................................................................................................................184.3. Lab Center......................................................................................................................28

5. Test Tools in Visual Studio 2010..............................................................................................315.1. Testing Tools..................................................................................................................315.2. Creating a Test Project...................................................................................................315.3. Creating an Executable Specification (with SpecFlow)...................................................325.4. Record and Playback......................................................................................................405.5. Database Test................................................................................................................445.6. Configuring Test Environment.......................................................................................455.7. Executing Tests..............................................................................................................465.8. Code Diagnostics............................................................................................................51

6. Tips & Tricks............................................................................................................................566.1. Test Management versus Source Control......................................................................566.2. The Product Backlog in TFS Workbench shows way to many Product Backlog Items. I

only need the current sprint..........................................................................................576.3. How do you distinguish between Functional and Non-Functional tests?......................576.4. How do you re-open a defect?......................................................................................576.5. I don’t want to install Microsoft Test Manager or VS2010............................................586.6. My Product Owner wants to see the Executable Specs without VS2010......................586.7. We group our User Stories in User Epics. How do you do that?...................................586.8. How do I detect Bugs that were closed by developers?................................................596.9. Can I continue a previous test run?...............................................................................596.10. Can you send a bug as an email?...................................................................................596.11. When I run a test, it takes ages before the steps are shown. Why is that?..................59

2


6.12. Can I copy a query, so that I don’t have to recreate the definition?..............................596.13. Can I record actions for Internet Explorer 6, or Microsoft Dynamics?...........................606.14. Can I search for bugs by ID or description?....................................................................606.15. Can I use “enter” when editing test steps?....................................................................606.16. Can I edit my test during execution?.............................................................................606.17. Internet Explorer 7, 8 More information about Unit Testing.........................................606.18. More information about Specification by Example.......................................................616.19. More information about VS2010 / MTM2010...............................................................61

3


1. Quality Assurance Process

What we attempt here is to tear down the walls between developer, tester, analyst, customer,…

For many of us, this looks familiar:

Why not stop doing this, and start creating business value together, as a team?

1.1. Write User StoriesA user story is one or more sentences in the everyday or business language of the user that captures what the user wants to achieve. The user stories should be written by the customers for a software project and are their main instrument to influence the development of the software.

User stories are a quick way of handling customer requirements without having to elaborate vast formalized requirement documents and without performing overloaded administrative tasks related to maintaining them. The intention of the user story is to be able to respond faster and with less overhead to rapidly changing real-world requirements.

However, a User Story is not an analysis, it is only a promise to have a conversation.

4


1.2. Illustrate using examplesA User Story does not contain enough information to actually develop the software, let alone test it. When you add Key Examples, things become a lot clearer for everyone.We use SpecFlow to create these tests. This allows us to manager the User Story and its examples right next to the actual code, while still keeping it readable and manageable for everyone involved.

Example:

5

http://www.exampler.com/propaganda.html%00%E5%A1%B9%EF%92%81%E1%B4%BB%E4%A1%BF%E2%B2%AF%E5%B6%82%E8%97%84%E6%8C%A7%00%00%EA%AE%A5


1.3. Automate the examples literallyThen, as we have the specifications right next to the code, woudn’t it be cool to actually create tests out of them? Guess what, we can! And with very little effort from a developer. Even better: these tests can be become Living Documentation of the system, reducing the need to write documentation.

Have a look at the image above. Note that some text is reused, for instance “the password is changed to …”.

For each (green) line, a Fixture has to be written in C#, and every time we add an example using this same line, no code has to be added or changed. When the logic of “change password” changes, only this piece of code has to be changed.

Example:

Then you can run these examples just as you can run any unit test:

6


1.4. Write unit testsA Unit test is a method by which an individual unit of source code is tested to determine if it is fit for use. A unit is the smallest testable part of an application. Unit tests are created by developers.

Example: suppose you have a module that determines wether a number is a prime number or not.

A developer may write these tests:

The result should look like this:

7


1.5. Write code until unit tests passAlthough in the previous items there are already screenshots of passing tests, the correct order of work is:

write a test

run the test and watch it fail

write code

rerun the test and watch it pass

refactor if needed

rerun all tests to make sure nothing else broke

start over with something else

1.6. Write code until examples pass.This is basically a loop around the above cycle:

pick an example of the User Story you’re developing. Run the test and watch it fail Figure out what you need to code to make it pass Write unit tests, and code. Rerun the test and watch it pass Rerun all other tests to make sure nothing else is broken Start over with something else.

1.7. Tester executes exploratory testsBach’s Definition: ‘Any testing to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.’

Which simply can be put as: A type of testing where we explore software, write and execute the test scripts simultaneously.

Exploratory testing is a type of testing where tester does not have specifically planned test cases, but he/she does the testing more with a point-of-view to explore the software features and tries to break it in order to find out unknown bugs.

8


A tester who does exploratory testing, does it only with an idea to more and more understand the software and appreciate its features. During this process, he/she also tries to think of all possible scenarios where the software may fail and a bug can be revealed.

Why do we do exploratory testing? In Scrum projects we do not have time to write test cases beforehand, we need to

write them while we test The most common scenario’s are already covered in the Executable Specifications, so

there is no need to repeat them (all). You cannot automate “end user experience”, so these tests have to be done by a

human.

1.8. End User executes acceptance testsAt the end of each sprint, every developed User Story will be shown to the end user in a demo. This should give te end user a quick view of what was added. After this demo the end users should take some time to actually work with the software, to see if it really is what they need.

1.9. Software is releasedAt the end of each Sprint, when the End User has accepted the software, you may go live. Remember to copy the source code of what you released before you start adding new features, just in case you’ll need to fix bugs later on.

9


2. Test Automation

2.1. Automated below the UIMost applications today split their business logic from the User Interface. The Business Logic is contained in a DLL or a WebService, and can be tested separately. These tests are easy to automate, as long as the tester and developer work together closely.

You may want to do this: The tester writes very detailed tests in a Scenario, as if it

were Executable Specifications. Warning: do not add this to the original Specifications, because too much details will render the original file unreadable. The original file is owned by the team, the detail file is “just for the tester”.

The developer then automates and runs the tests.

2.2. Automated using the UITesting through the User Interface is a lot more complicated. The User Interface has to be started, for one thing, and clicking on controls is a lot more complicated than calling a service.

The good thing is that we can automate a number of things already: Browser based applications are no problem, we can automate them using Selenium. WinForm applications are a bit more tricky, but we can still do this without too much

effort WPF applications require more work, but since VS2010 we should be able to do this.

We hope to create a framework that will enable you to easily automate all these technologies.

Contact Tom Vercauteren for more information.

2.3. Performance and Load Tests A performance test answers the question “How fast is action X on my system?” whereas a load test attempts to answer questions like “How many users can my system handle?”.

For a performance test it is often enough to create an automated test, and add some code that monitors how fast the action was. Load Tests tend to be more complicated.

Visual Studio provides a Load Testing feature, if you have the Tester or Ultimate edition.

For a load test, follow these rules: Determine the user load you want to build: We start with X users per minute, then

every Y seconds, we add Z users, with a maximum of W users.

10


Do not reuse data, make sure each instance of a test has its own data. In the test itself, there is no room for test preparation. You need to prepare

beforehand. This type of test will always be data driven. This means you have a table of data (e.g. userids, or any other data you need) handy to make sure all individual tests are “unique”.

Get test variables into memory as soon as possible, and keep them there. Do not clean up during the test, but do that later. Do not validate anything (or as little as possible) during the test

Example: Suppose you want to test how many users can change their password simultaniously.

Load scenario: We start with 5 users per minute, then every 60 seconds, we add 5 users until we reach 100 users.

To reach 100 users, the test will run for 20 minutes (5 during the first minute, 10 during the second, 15 during the third, … 100 during the 20th minute). This means that we will need 1050 different users (5 during the first minute, 10 during the second, 15 during the third, … 100 during the 20th minute)

So before the test even starts, we check if there are 1050 users, and if needed, create them.

Then the very first test loads a list of these into memory (for the developers: I usually end up with a static table or array) and iterate through them.

When the tests are done, we cleanup the 1050 users. During the test, we don’t want to bother with “checking that the password has

indeed been changed”. We have other tests for that. If no error was returned, we assume everything is in order.

Steps to follow: Developer: Create a “unit test” that follows the above principles Tester: Create a “Load Test” with the Load test Wizard. Here you will set the Load

scenario (the number of users per minute, …) Tester: prepare the test data Tester: Run the test Tester: cleanup the test data Tester: Fill out a Performance Test Report

Sometimes the PC that runs the test can’t keep up (CPU at 100%, Memory full, …). Luckily even this can now be fixed (for free, providing you have more than one PC available). Just download and install “Visual Studio Agents 2010”:

Test Controller 2010 should be installed on the “master” PC Test Agent 2010 can be installed on each PC that will run the tests

Contact Tom Vercauteren for more information.

11

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Methodology/Template%20Performance%20Tests.dot


3. Scrum in TFS 2010

3.1. Product Backlog

In the TFS Workbench, go to “Item List”, “Product Backlog Item”:

3.2. Product BurndownReports are available in the Team Portal.

12


3.3. VelocityReports are available in the Team Portal.

13


3.4. Sprint PlanningDefining the Sprint can be done in TFS Workbenc, via “Project Setup”:

Or you can do this in VS2010, using the query “Cadence and Teams”, “All Sprints” (and the other queries here.

14


Sprint Planning is done in TFS Workbench, in “Task Board”, “Product Backlog – Tasks”

If you prefer a list, you van go to “Item List”, “Sprint Backlog Task”.

15


3.5. Sprint BurndownReports are available in the Team Portal. Unfortunately, the screenshot below does not show the capacity and trend.

16


4. Microsoft Test Manager

4.1. Connecting to a Team ProjectWhen you start Team Project for the first time, you’ll b e asked to connect to a Team Project:

Next you’ll have to select (or create) a Test Plan:

17


4.2. Testing Center

4.2.1. Test PlanTest Plans are organised by requirement. In the left panel you can create a hierarchy of these requirements, and on the right side, you can list the test cases per requirement.

Click “Add” to add an existing test, or “New” to define a new one.

18


You should link each test to its User Story (or Product Backlog Item):Go to the tab “All links” and select “Link to”, then select Link type “Tests”:

Click OK, and select this “Saved Query”, then select the correct user Story below (check the checkbox) and click OK.

19


You have now linked the test to this User Story:

4.2.2. Query Based Test Plan

Don’t want to add all your (existing) tests manually, or want to reuse them, for instance in a regression test plan? Create a new test plan, and add a “Query Based suite” (from the “New” menu) under your new Plan. Now you can create a query, that will automatically select your tests. In this example I simply select all Test Cases that are assigned to me:

Whenever a new test is created, and it complies with the specified criteria, it will be added to this test plan.It is a good idea to use this to automatically select all tests linked to a certain User Story (or Product Backlog Item). This means that, when you create a test, all you have to do is link it to all User Stories it covers, and it will appear in all relevant places in the test plan.

4.2.3. Test Execution, Semi-automatedRunning tests is done via Test Center / Test / Run Tests.Once you select the requirement you want to test, the linked tests will appear. You start testing by selecting the test you want to execute, and pressing “Run” on top.

20


You now get a new window, where you can start the test, with Action Recording on or off.

Action Recording is what makes your test semi-automated. While you execute the test for the first time, set Action Recording on, and Test Manager will record all your actions. Beware: it will NOT record any validations you do!Below you can track how many actions were recorded, and what they are.Once a test was recorded, you can simply press CTRL-L (or select Play from the menu) to execute a step.Note that you have to do the validations yourself!

To pass a step, click the icon on top, or use the shortcut keys:

Click End Test when you’re done, then click Save and Close to return to Test Manager.

21


4.2.4. Test Execution, Data drivenYou can add parameters to a test, so it is run several times, with these different parameters. You can also do this for a test that you will add a recording to afterwards.You can do this by clicking “Insert Parameter”.

Then you can specify the values below:

22


When you run this, a “database” icon will show you when the mapping was made:

Tip: as you execute this test, the current parameter will automatically be set in the clipboard. In the example above, the state “Karnataka” is in the clipboard, so in the application you can just select the State field, and click ctrl-v to paste the value.

4.2.5. Track

Queries: this is basically the same as “Reports” in Visual Studio. You can see things like: Sprint Planning, in Cadance and Teams, All Sprints Active Bugs, in Springing, Bugs My tasks, in Sprinting, My Work

23


Assign Build is used to select or change the Build of the software against which you are testing:

Recommended tests will list tests that need to be re-executed when you change the Build. This will only work if you had Code Coverage analysis on while you were running the tests.

24


Code Coverage can be set via the local.testsettings file. Select the Data and Diagnostics option:

25


4.2.6. Organize

Test Plan Manager is the quick way to select other Test Plans.

The second tab, Test Configuration Manager, is for managing and modifyingthe configurations. We can create multiple configurations based on differentparameters such as the Operating system, Browser, version and othersystem variables.

The next tab is the Test Case Manager, where all the test cases in the Testplan are displayed. We can select any of the test cases and modify it or createa new test case or create a copy of existing test case. We can filter the testcases from all existing test cases and then choose the test case to modify.

The final tab is the Shared Steps Manager tab, which is for creating and maintaining the test cases that are shared across many test cases. These are called shared steps because they are common steps that are repeated in

26


multiple test case steps. For example, the entering employee details step is the same step used while creating a new employee detail and updating an employee detail.

27


4.3. Lab Center

4.3.1. Environments

Creating a collection of Virtual machines and managing it with Lab Management is called a virtual environment. The integration of Lab Management with System Center Virtual Machine Manager enables us to deploy and test our applications on these virtual environments. We can schedule the Team Foundation Server build to build our application and then deploy and test them on these environments.The Lab tab in the Lab Center provides access to the virtual environments deployed on the host groups of a team project. A host group is a collection of physical computers on which virtual environments can be created.

28


4.3.2. Test Settings

The Test Settings tab helps us to create multiple test settings. We can define the roles and choose data and diagnostics information for the test.

Later on while deploying the virtual machine, the role can be used to choose the corresponding virtual machine to run the tests on.

4.3.3. Library

The Library tab in the Lab Center is used for maintaining and storing the virtual environments and templates that we use to create the virtual environments.

29


4.3.4. Controllers

The Controller tab is used to manage the controllers used for the environment. You can select the controller from the list and change the configuration as well. The test controller manages the test agents to run the tests and communicates what each agent should do.

30


5. Test Tools in Visual Studio 2010

5.1. Testing Tools

5.1.1. Test ViewThe Test View window shows all the tests in the solution. We can filter the tests based on test properties or names.

5.1.2. Test List EditorWhere the Test View only lists your tests, the Test List Editor allows you to group your tests into logical sets. Here 2 lists were created, and the checkbox notes which tests will be executed:

5.2. Creating a Test ProjectIn the same way as adding the projects and code files to the solution, we have tocreate the test project and test files, and add the test project to the solution. There aredifferent ways of creating test projects under the solution:

31


Select the solution and add a project using the Add | New Project....Then select the project type as Test from the list of project types under thelanguage which we are going to use. Then select the template as Test Project.

Note that for this type of project you need to select .NET Framework 4.

5.3. Creating an Executable Specification (with SpecFlow)Right click the new Project, and select Add / New Item.Select SpecFlow Feature File:

32


As filename, use the name of the User Story that this file will describe. For example: “US-123 Change Password.feature”.

Then, change the default text of the Feature:

For example:

Note that this is just text. This is not automated.

Then, there are 3 ways to create examples of this Feature.

5.3.1. Scenario only

Type “Scenario:” and add a name for this example.

Then you just add lines starting with “locigal” keywords: Given: this discribes the situation from which we start.

33


When: this is the action that is executed. Then: this describes the result.

To make it more readable, we can also add lines with: And But

Example:

Or in a more realistic example:

5.3.2. Scenario with table(s)

Obviously we don’t want to just copy-paste our tests, and change a few variables. This would create too much maintenance, and result in lower readability. That is where tables come in.

Note that the table delimiter is the |. This can be found on most keyboards under “1”.

34


Example with 2 tables built in:

5.3.3. Scenario Outline

If you use tables in more than one line, you may want to create a Scenario Outline, where all tables are grouped into one, at the bottom:

You might even have more than one Examples block, to make a difference between “positive” and “negative” tests, for instance.

35


5.3.4. Feature Background

For some features, all scenarios start from the same situation. You may want to separate these, and add a “Background” tag. For instance, these 2 scenarios started with “Given a user with a valid profile”:

Note that the “Background” code will be executed before every test.

5.3.5. BeforeScenario and AfterScenario

Use AfterScenario to clean your tracks after a test was done.Use BeforeScenario to do some generic setup before the test starts. For instance: here we make sure the user executing the test has the necessary rights:

Sometimes, you want to do some setup or teardown only for specific situations. You can use tags on the test for this:

36


Note that when the test marked with @Mail is executed, BeforeScenario will be run first, and then BeforeScenario(“Mail”) is run. Than the actual test starts.

5.3.6. Ignore a test

When a testers adds a test, and it fails, then you don’t want to check in the code with the failing test in it, because your build script may fail on it.

You can add an “@Ignore” flag to this test:

This will make sure the test is not executed. Once a developer has successfully implemented this test, the tag can be removed.

5.3.7. Run the test and watch it fail

The tests, although not yet implemented, are already visible in Test View:

37


Obviously, the test does not pass. You do get a nice bit of sample code, that you can add to a new file: a SpecFlow Step Definition:

38


Only 2 things are left to do: when needed, use parameters. In the example above, you want to pass

the “password” as a parameter:

implement the actual test.

Notice the user of (.*) to capture a parameter. You can use: (\d+) for a number (.*) for a string

5.3.8. Implement the test

As each line of the example is implemented separately, and you don’t know which line will be executed next, you’ll have to make sure that the object under test exists. So every method will have (or calls) something like this:

ScenarioContext is the place where you store the object you are testing. After you’re done with this object (for now), you store it again:

A complete example may look like this:

39


Note that the above example uses the SD Worx Test Automation Framework to do things like “StartApplication”, and “Click”.

At the moment, we have a small issue with SpecFlow: when you atomate a User Interface, these tests are not immidiately runnable.

Workaround: open the.cs file, next to the feature file locate the line

[Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute()] replace it with [Microsoft.VisualStudio.TestTools.UITesting.CodedUITest()]

5.4. Record and Playback

Warning: use Record and Playback only:when a test automation framework is not yet availablewhen the user interface does not change much

The reason is that it’s not very easy to edit these test once they are recorded. When your application changes a lot, there is nothing for it but to re-record everything again.

Right click the project that contains your tests, and select Add, Coded UI Test:

Here you have 2 options:

40


The first option allows you to create a new test.If you’re working with a tester, that created a recording in Test Manager, you can select the second option the grab that recording from TFS.

Now Visual Studio minimizes, and you get a small menu below:

Don’t forget to click Record before you start executing your test.

I usually start with the application under test closed, and the link to the application in the clipboard. This allows me to press Start, Run, and then paste the link and enter to start the application.

This makes your test more robust, as you don’t have to rely on shortcuts, or having to start your application manually beforehand.

Every time you want to validate something on the screen, you’ll have to do this: Click the last icon (Generate Code – Alt-G) and enter a name for the steps you’ve gone

through so far. When the code is generated, drag the third icon to the control you want to validate,

until a blue square appears around it:

41


Then, select the property you want to check:

Right click and select Add Assertion…

Then click Generate Code again, and add a name for this validation

When all the code is generated, simply close the “UIMap – Coded UI Test Builder” window.

In Visual Studio, you’ll notice that 4 files were generated:

CodedUITest1 contains a method with all steps you generated The actual code is generated in UIMap.Designer.cs

42


UIMap.uitest is actually an XML file that contains all the data in the test. You can edit this by doubleclicking on it:

43


5.5. Database TestIf you have some business logic on the database, you may need to test these functions / stored procedures / … on the database.

Create a test project in Visual Studio, and right click it. Choose Add, New Test, Database Unit Test.

After specifying the database connection, you can specify the test: The top part is where you execute some code on the database. The bottom part is where you validate the result

44


5.6. Configuring Test EnvironmentEvery test project automatically gets a Test Settings file. When you doubleclick this file, you can change several things, the most important being the Test Timeouts:

When a test takes a long time, you’ll need to change this. For a Web Test you can set which browser needs to be simulated:

45


5.7. Executing TestsYou can run a set of tests, by selecting them in the Test View or Test List Editor, and clicking the icon:

5.7.1. Test ResultsWhen the tests are done, you get a result:

Doubleclicking on a test gives you a detailed view of what happened:

For a failed test, you can easily create a Bug:

46


5.7.2. Test Impact ViewWhen source is checked in, this view shows you which tests should be re-run. You’ll need to enable this, and code coverage, in the settings first:

47


Then, after some code changes, you see which tests are impacted:

48


5.7.3. Running tests in a BuildWhen configuring the Build, you can set which tests to run:

49


After the build, you get a result of the tests:

50


5.8. Code Diagnostics

5.8.1. Code CoverageCode Coverage has to be set on in the test settings (local.testsettings):

Next, click Configure to select the code that will be instrumented. Then you can run the test, and afterwards you can open the Code Coverage Result window from the Test menu.

51


5.8.2. IntelliTrace

Do you remember the time when the tester logged a defect, and the developer would just close it with “non-reproducible” in the comments?

I remember this as if it were yesterday (ok, that probably is because it WAS yesterday that this happened, AGAIN).

IntelliTrace promises that this is a thing of the past. Run your test with this option on, and you’ll be able to hand the developer a lot more information than he wants. Enough to easily retrace your steps.

52


At first glance, the trace looks like this:

Suppose that something went wrong when we clicked “-“. When the developer doubleclicks the “Clicked “-“” line above, he’ll be able to debug the application exactly at that moment, with all the variables at the exact same value as you had them.

53


If the developer still does not believe the bug is really there, include a video recording:

5.8.3. Profiler

Some parts of an application are slower than others, and it’s not always easy to make an application faster.

If you have a method that you know takes 3 seconds, and after some optimization it only takes 1 second, you won 2 seconds every time this method is called.

But what if this method is only called once a month, and there is another method that is called thousands of times a day, but only takes 0,2 seconds.

The application would become much faster if we could make that method do its job in 0,15 seconds, wouldn’t it?

That’s where a profiler comes in.

You turn on the profiler in the test configuration, and then run a REALISTIC test on the system. It’s no use trying to hit all the code, because you’re not

54


interested in code that will only be hit once a month. You need the most used code.

When the test is done, you get something like this:

Clicking around on the various methods may disclose the slow points in your application.

Fixing these slow point is usually a whole different ball game…

55


6. Tips & Tricks

6.1. Test Management versus Source ControlHow do you manage your tests per version of the software:Version 1 is released, so the developers branch the source into “v1” and “v1.1”. They start working on “v1.1”, but there may be bugfixes on “v1”.

Unit tests, and Executable Specifications are no problem, as they are right next to the code. But what about your manual tests?

In Test Center, create a test plan for the new version of the software.

Then, create a copy of the tests you need:

Next, remove the test from this testplan

And finally add the test to the correct testplan.

I know, this is not the best workaround, so we’re hoping to find a better idea.

56


6.2. The Product Backlog in TFS Workbench shows way to many Product Backlog Items. I only need the current sprint.Sort the Product Backlog by “Iteration Path”, ascending.

6.3. How do you distinguish between Functional and Non-Functional tests? Add them to a different Suite to the Test Plan Start the name of Non-functional tests with “[Non-Functional]”

6.4. How do you re-open a defect? Change the state to “Deleted” Then you can change the state to “Active”

I agree with you that this is a bug, but since we won’t do this very often, and there’s a workaround, we won’t fix this.

The complete state diagram of a bug looks like this:

57


6.5. I don’t want to install Microsoft Test Manager or VS2010.Just browse to http://sbe351:8080/tfs and you’ll be able to open and close bugs just the same.

6.6. My Product Owner wants to see the Executable Specs without VS2010No problem, just provide him with a link to TFS Team Web Access, and show him where the feature file can be found. He won’t be able to make changes.

6.7. We group our User Stories in User Epics. How do you do that?Simple: first you create the User Epic as a Product Backlog Item, then you right click this and add a “new linked work item”. Choose Child link, and item type “Product Backlog Item”:

58

http://sbe351:8080/tfs


6.8. How do I detect Bugs that were closed by developers?In VS2010, create a new query with these conditions:

If more than one tester works on this project, you may need to add several “Changed By” lines. Developers are allowed to create new bugs (Active) and are allowed to say they are resolved and can be re-tested (Ready for Test). Other statusses are reserved for the tester.

6.9. Can I continue a previous test run?Good question, but we haven’t found an answer yet.

6.10. Can you send a bug as an email?Yes you can, but only from the web interface (http://sbe351:8080/tfs) and only when this feature has been enabled.

6.11. When I run a test, it takes ages before the steps are shown. Why is that?You may have some data collectors on, for instance “capture a video while I test” takes a lot of preparation. Turn this off in test configuration, and it should be a lot faster.

6.12. Can I copy a query, so that I don’t have to recreate the definition?Yes, but only in VS2010. In Test Manager, this option is not available (yet).

59

http://sbe351:8080/tfs


6.13. Can I record actions for Internet Explorer 6, or Microsoft Dynamics?These technologies are currently supported: Internet Explorer 7, 8 and 9, including HTML and AJAX Firefox 3.5 and 3.6 (with Feature Pack 2) Silverlight 4 (with Feature Pack 2) Windows Forms 2.0 and later WPF 3.5 and later Dynamics CRM Web Client

These are partially supported, visit MSDN to find out what is or isn’t: MFC SharePoint Dynamics Ax Client Citrix / Terminal Services

These are not (yet) supported: Internet Explorer 6 Chrome Opera Safari Silverlight 3 Flash / Java Office Client Applications SAP

6.14. Can I search for bugs by ID or description?Sure, just create a report with these criteria:

6.15. Can I use “enter” when editing test steps?No. In MTM2010, a test step is just that: a small step that should be executed. If you need a long explanation, you really should split it up in multiple steps.

6.16. Can I edit my test during execution?No. Unfortunately this is not possible in the current version.

6.17. More information about Unit TestingMore information on Unit Testing can be found in “The Art of Unit testing, by Roy Osherov”.Alternate route:

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/default.aspx

60


http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/The%20Art%20of%20Unit%20Testing.pdf

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/The%20Art%20of%20Unit%20Testing.pdf


Specialst Literature Various

6.18. More information about Specification by ExampleMore information can be found in “Specification by Example, by Gojko Adzic”Alternate route:

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/default.aspx Specialst Literature Various

6.19. More information about VS2010 / MTM2010More information can be found in “Software testing using Visual Studio 2010, by Subashni S and Satheesh Kumar N” or “Software testing with Visual Studio 2010, by Jeff Levinson”Alternate route:

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/default.aspx Specialst Literature Various

61


http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/Software%20Testing%20with%20Visual%20Studio%202010.pdf

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/Software%20Testing%20with%20Visual%20Studio%202010.pdf

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/Software%20Testing%20using%20Visual%20Studio%202010.pdf

http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/Software%20Testing%20using%20Visual%20Studio%202010.pdf


http://portal.sd.dika.be/sites/ICTCorporate/ICTTestersCorner/Shared%20Documents/Specialist%20literature/Various/Specification%20by%20Example.pdf