Upload
softwarecentral
View
1.072
Download
0
Tags:
Embed Size (px)
Citation preview
Effective Unit Testing with Effective Unit Testing with NUnitNUnit
Win-Dev 2004 Win-Dev 2004 Friday, October 29, 2004Friday, October 29, 2004
Peter Provost Peter Provost Software Design EngineerSoftware Design Engineerpatterns & practicespatterns & practicesMicrosoft CorporationMicrosoft Corporation
AgendaAgenda
What is Unit Testing?What is Unit Testing?
Typical Unit Testing ProblemsTypical Unit Testing Problems
Best Practices for Effective Unit Best Practices for Effective Unit TestingTesting
Tools DemoTools Demo
Q & AQ & A
What is Unit Testing?What is Unit Testing?
A DefinitionA Definition
Unit Testing is Unit Testing is code that…code that…
Is written by Is written by developers, for developers, for developers.developers.
Exercises a small, Exercises a small, specific area of specific area of functionality.functionality.
Helps “prove” that Helps “prove” that a piece of code a piece of code does what the does what the developer expects developer expects it to do.it to do.
What Unit Testing Is NotWhat Unit Testing Is Not
Acceptance Testing (aka Functional Acceptance Testing (aka Functional Testing)Testing)
Performance TestingPerformance Testing
Scalability TestingScalability Testing
Marick’s Four Quadrants of Marick’s Four Quadrants of TestingTesting
Source: Brian Marick – http://testing.com/
CustomerTests
Programmer Tests
Exploratory Tests
ScalabilityPerformance
UsabilitySecurity
Business Facing
Technology Facing
Sup
port
Dev
elop
men
tC
ritique Result
Why Unit TestWhy Unit Test
It will make your It will make your life easierlife easier
Better codeBetter code
Better designsBetter designs
Code is easier to Code is easier to maintain latermaintain later
Confidence when Confidence when you codeyou code
Common ExcusesCommon Excuses
I’m not a tester!I’m not a tester!
It takes too much It takes too much time.time.
It takes too long to It takes too long to run the tests.run the tests.
I don’t know how I don’t know how to test it.to test it.
I don’t really know I don’t really know what it is supposed what it is supposed to do, so I can’t to do, so I can’t test it.test it.
But it compiles! It But it compiles! It doesn’t need tests.doesn’t need tests.
Typical Unit Testing Typical Unit Testing ProblemsProblems
Testing Is Monotonous and Testing Is Monotonous and BoringBoring
Often indicates bad tooling or bad Often indicates bad tooling or bad techniquestechniques
Unit testing should…Unit testing should…Be easy to doBe easy to do
Provide rapid feedbackProvide rapid feedback
Poor Test CoveragePoor Test Coverage
Often happens when tests not written Often happens when tests not written firstfirst
Hard to retrofit laterHard to retrofit later
Test coverage is orthogonal to test Test coverage is orthogonal to test qualityquality
Purpose of Tests is Purpose of Tests is MisunderstoodMisunderstood
Remember why we do unit testingRemember why we do unit testing
Easy to get distracted by irrelevant Easy to get distracted by irrelevant thingsthings
What is the scope of the test?What is the scope of the test?
Informal Testing ProcessInformal Testing Process
How does your team do unit testing?How does your team do unit testing?
Test-firstTest-first
Intuitive “poking and prodding” style Intuitive “poking and prodding” style testingtesting
Smoke testingSmoke testing
Whatever the developer wantsWhatever the developer wants
Inconsistent TestingInconsistent Testing
Are tests required before check in?Are tests required before check in?
How is it enforced?How is it enforced?
Low Test QualityLow Test Quality
Single biggest problem in unit testing Single biggest problem in unit testing todaytoday
Testing requires a special mindsetTesting requires a special mindset
Tools only solve half the problemTools only solve half the problem
Code Not Designed for TestCode Not Designed for Test
Often the cause of low test qualityOften the cause of low test quality
Hard to design testable codeHard to design testable code
However, testable code often is However, testable code often is better designedbetter designed
Tests Not MaintainedTests Not Maintained
So you’ve shipped your code and So you’ve shipped your code and someone finds a bug…someone finds a bug…
Do you fix the tests? Or just fix the Do you fix the tests? Or just fix the bug?bug?
What about the rest of the team?What about the rest of the team?
Best Practices for Effective Best Practices for Effective Unit TestingUnit Testing
Automate Your TestsAutomate Your Tests
If it is hard to run tests, you won’t do If it is hard to run tests, you won’t do itit
Manual (aka scripted) tests make Manual (aka scripted) tests make regression testing very hardregression testing very hard
Use Good ToolsUse Good Tools
The tools should make it…The tools should make it…Easy to write testsEasy to write tests
Easy to organize your testsEasy to organize your tests
Easy to run tests (all, some, one)Easy to run tests (all, some, one)
They should provide…They should provide…Quick feedback of pass/failQuick feedback of pass/fail
Details on the fail conditionsDetails on the fail conditions
Use Good ToolsUse Good Tools
Testing Focused Testing Focused ToolsTools
NUnitNUnit
csUnitcsUnit
MbUnitMbUnit
Test Driven .NET Test Driven .NET (aka NUnitAddin)(aka NUnitAddin)
NMockNMock
DotNetMockDotNetMock
Supporting ToolsSupporting Tools
NAntNAnt
CodeRushCodeRush
ResharperResharper
CodeSmithCodeSmith
And of course…And of course…
Visual Studio .NETVisual Studio .NET
Tools DemoTools Demo
Get a MentorGet a Mentor
Testing is as much an art as a Testing is as much an art as a sciencescience
Learning to write good tests is hardLearning to write good tests is hard
Some people are naturally good at it Some people are naturally good at it and some aren’tand some aren’t
Use a Test ListUse a Test List
Start your development activities by Start your development activities by writing down a list of things you want writing down a list of things you want to testto test
You will often think of a test while You will often think of a test while writing another one. When you do, writing another one. When you do, add it to the list.add it to the list.
Review your list frequentlyReview your list frequently
Write Tests FirstWrite Tests First
Test-driven development (TDD) is a Test-driven development (TDD) is a proven way of improving quality*proven way of improving quality*
TDD’s main objective is not testing TDD’s main objective is not testing software! (this is a side effect)software! (this is a side effect)
TDD’s main objective is to aid TDD’s main objective is to aid programmers and customers during programmers and customers during the development process with the development process with unambiguous requirements. unambiguous requirements.
Code is written with testing as a Code is written with testing as a primary motivation. In short, the primary motivation. In short, the code is testable.code is testable.
* - http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf* - http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf
What to TestWhat to Test
The “did we get what we expected” The “did we get what we expected” test isn’t always good enoughtest isn’t always good enough
Ask yourself the question…Ask yourself the question…
““If the code ran correctly, how would I If the code ran correctly, how would I know?”know?”
What to Test - Testing What to Test - Testing HeuristicsHeuristics
Test at the boundariesTest at the boundaries
Test every error messageTest every error message
Test different configurationsTest different configurations
Run tests that are annoying to setupRun tests that are annoying to setup
Avoid redundant testsAvoid redundant tests
Source: Lessons Learned in Software Testing, by Cem Kaner, James Source: Lessons Learned in Software Testing, by Cem Kaner, James Bach, Brett PettichordBach, Brett Pettichord
What to Test – BICEPWhat to Test – BICEP
BB Are the boundary conditions Are the boundary conditions correct?correct?
II Can you check inverse Can you check inverse relationships?relationships?
CC Can you cross check using otherCan you cross check using othermeans?means?
EE Can you force an error condition toCan you force an error condition tohappen?happen?
PP Are the performance characteristicsAre the performance characteristicsacceptable?acceptable?
Source: Pragmatic Unit Testing in C# with NUnit by Andrew Source: Pragmatic Unit Testing in C# with NUnit by Andrew Hunt and Dave ThomasHunt and Dave Thomas
Test StructureTest Structure
William Wake’s 3-A PatternWilliam Wake’s 3-A PatternArrangeArrange
ActAct
AssertAssert
Designing for TestDesigning for Test
One simple question to help you One simple question to help you write good code…write good code…
““How am I going to test this?”How am I going to test this?”
If you don’t know the answer, then If you don’t know the answer, then you probably need to reconsider your you probably need to reconsider your design.design.
Stubs and MocksStubs and Mocks
Stub DefinedStub Defined“ “ A portion is a portion of code that is A portion is a portion of code that is inserted at runtime in place of real code, in inserted at runtime in place of real code, in order to isolate calling code from the real order to isolate calling code from the real implementation. The intent is to replace a implementation. The intent is to replace a complex behavior with a simpler one that complex behavior with a simpler one that allows independent testing of some portion allows independent testing of some portion of real code.” of real code.”
Mock Objects DefinedMock Objects Defined““A mock object is an object created to A mock object is an object created to stand in for an object that your code will stand in for an object that your code will be collaborating with. Your code can call be collaborating with. Your code can call methods on the mock object, will deliver methods on the mock object, will deliver results as set up by your tests.” results as set up by your tests.”
Source: JUnit in Action, by Vincent MassolSource: JUnit in Action, by Vincent Massol
When are stubs appropriate? When are stubs appropriate?
Objects that are expensive to create Objects that are expensive to create to manipulateto manipulate
At the edges of a system, or around At the edges of a system, or around complex clusters of objectscomplex clusters of objects
Course-grained testing, integration Course-grained testing, integration testingtesting
Stub DownsidesStub Downsides
A Complex system might require a A Complex system might require a complex stubcomplex stub
Hard to write in a verifiable wayHard to write in a verifiable way
Difficult to maintainDifficult to maintain
When are mocks appropriate? When are mocks appropriate?
Real object has Real object has non-deterministic non-deterministic behaviorbehavior
Real object is Real object is difficult to set up difficult to set up
Real object has Real object has behavior that is behavior that is difficult to causedifficult to cause
Real object is slowReal object is slow
Source:http://c2.com/cgi/wiki?Source:http://c2.com/cgi/wiki?MockObjectsMockObjects
Mock DownsidesMock Downsides
Code complexity of being able to Code complexity of being able to switch between real and mock switch between real and mock implementationsimplementations
Maintaining the mockMaintaining the mock
Properties of Good TestsProperties of Good Tests
AutomaticAutomaticTests need to be run checkedTests need to be run checkedautomaticallyautomatically
ThoroughThorough Test everything that couldTest everything that couldpossibly breakpossibly break
RepeatableRepeatable Tests should produce the Tests should produce the samesame
results each time they are results each time they are runrunIndependentIndependent A test should exercise only A test should exercise only one one
thing at a timething at a timeProfessionalProfessional Tests are code too! Write Tests are code too! Write themthem
professionally.professionally.
Source: Pragmatic Unit Testing in C# with NUnit by Andrew Source: Pragmatic Unit Testing in C# with NUnit by Andrew Hunt and Dave ThomasHunt and Dave Thomas
Test Organization OptionsTest Organization Options
Tests in the VS Project being testedTests in the VS Project being testedBest for experiments, spikes, etc.Best for experiments, spikes, etc.
Tests in a separate VS ProjectTests in a separate VS ProjectMost commonly used arrangementMost commonly used arrangement
Tests are not part of distributionTests are not part of distribution
Tests in a separate VS SolutionTests in a separate VS SolutionThe usual reference issues, but possibleThe usual reference issues, but possible
Recommended Test Recommended Test OrganizationOrganization
Tests in a separate project/assemblyTests in a separate project/assembly
Naming conventionsNaming conventions
If you are testing the class:If you are testing the class:
Foo.Bar.BazFoo.Bar.Baz
Then you should name your testsThen you should name your tests
Foo.Bar.Tests.BazTestsFoo.Bar.Tests.BazTests
Note the namespace and test class Note the namespace and test class name!name!
When to Run TestsWhen to Run Tests
When you write a new method…When you write a new method………compile and run local unit testscompile and run local unit tests
When you fix a bug…When you fix a bug………run the test that illustrates that bug.run the test that illustrates that bug.
Any successful compile…Any successful compile………run local unit tests.run local unit tests.
Before you check in…Before you check in………run all tests.run all tests.
Continuously…Continuously…...check out and build the project from ...check out and build the project from
scratch including all unit tests.scratch including all unit tests.
Do not check in code that…Do not check in code that…
Is incomplete (e.g. Is incomplete (e.g. missing missing dependencies)dependencies)
Doesn’t compileDoesn’t compile
Compiles but Compiles but breaks other codebreaks other code
Doesn’t have unit Doesn’t have unit teststests
Has failing unit Has failing unit teststests
Passes its tests but Passes its tests but causes other tests causes other tests to fail.to fail.
Additional ResourcesAdditional Resources
Web SitesWeb Sites
http://www.testdriven.comhttp://www.testdriven.com
http://www.xprogramming.comhttp://www.xprogramming.com
http://workspaces.gotdotnet.com/tddhttp://workspaces.gotdotnet.com/tdd
WeblogsWeblogs
Peter ProvostPeter Provosthttp://www.peterprovost.orghttp://www.peterprovost.org
Jim NewkirkJim Newkirkhttp://weblogs.asp.net/jamesnewkirkhttp://weblogs.asp.net/jamesnewkirk
Brian ButtonBrian Buttonhttp://dotnetjunkies.com/WebLog/oneagihttp://dotnetjunkies.com/WebLog/oneagilecoder/lecoder/
Martin FowlerMartin Fowlerhttp://www.martinfowler.com/bliki/http://www.martinfowler.com/bliki/
Darrell NortonDarrell Nortonhttp://dotnetjunkies.com/WebLog/darrell.http://dotnetjunkies.com/WebLog/darrell.norton/norton/
Mailing ListsMailing Lists
Yahoo Group: testdrivendevelopmentYahoo Group: testdrivendevelopment
Yahoo Group: agiledotnetYahoo Group: agiledotnet
Yahoo Group: extremeprogrammingYahoo Group: extremeprogramming
BooksBooks
Test-Driven Development in Microsoft Test-Driven Development in Microsoft .NET – James Newkirk and Alexei .NET – James Newkirk and Alexei VorontsovVorontsov
Pragmatic Unit Testing in C# with Pragmatic Unit Testing in C# with NUnit – Andrew Hunt and David NUnit – Andrew Hunt and David ThomasThomas
Test-Driven Development, by Test-Driven Development, by Example – Kent BeckExample – Kent Beck
Refactoring to Patterns – Joshua Refactoring to Patterns – Joshua KerievskyKerievsky
Lessons Learned in Software Testing Lessons Learned in Software Testing – Cem Kaner, James Bach, and Brett – Cem Kaner, James Bach, and Brett PettichordPettichord
Questions? CommentsQuestions? Comments
Peter Provost Peter Provost Software Design EngineerSoftware Design Engineerpatterns & practicespatterns & practicesMicrosoft CorporationMicrosoft Corporation
E-mail: [email protected]: [email protected]: Weblog: http://www.peterprovost.org/http://www.peterprovost.org/
NOTENOTE: Updated slides available at : Updated slides available at http://www.peterprovost.org/Files/C1http://www.peterprovost.org/Files/C16_Effective_Unit_Testing.ppt6_Effective_Unit_Testing.ppt