Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
1
Slide 1 of 110
Through A Glass, Darkly:Through A Glass, Darkly:Object-Oriented TestingObject-Oriented Testing
●
●
●
●
●
● Mark StockmyerMark Stockmyer● EECS 814 PresentationEECS 814 Presentation
● December 1, 2005December 1, 2005●
●
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
2
Slide 2 of 110
Personal InfoPersonal Info
● Mark StockmyerMark Stockmyer● B.S. Computer Science – MWSUB.S. Computer Science – MWSU● B.S. Chemistry – MWSUB.S. Chemistry – MWSU● 15 years professional programming 15 years professional programming
experienceexperience● Reuters – Financial market analysis softwareReuters – Financial market analysis software● US Navy – Weapons/Propellant simulationsUS Navy – Weapons/Propellant simulations
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
3
Slide 3 of 110
SourceSource
● A Practical Guide to Testing Object-Oriented A Practical Guide to Testing Object-Oriented SoftwareSoftware by John D. McGregor and David A. by John D. McGregor and David A. SykesSykes
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
4
Slide 4 of 110
QuoteQuote
● Never ascribe to malice that which can be Never ascribe to malice that which can be explained by incompetence.explained by incompetence.
Napoleon (attributed)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
5
Slide 5 of 110
Road MapRoad Map
● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
6
Slide 6 of 110
Why Do I Care?Why Do I Care?
● Quality is importantQuality is important● No one likes fixing bugsNo one likes fixing bugs● Prevent them from occurringPrevent them from occurring● Questions on the final examQuestions on the final exam● PracticalityPracticality
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
7
Slide 7 of 110
Question #1Question #1
● How many of you write code for a living?How many of you write code for a living?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
8
Slide 8 of 110
Question #2Question #2
● If you write code for a living, how many of If you write code for a living, how many of you write Object-Oriented code?you write Object-Oriented code?
● C++?C++?● Java?Java?● Smalltalk?Smalltalk?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
9
Slide 9 of 110
Question #3 (Rhetorical)Question #3 (Rhetorical)
● Have you noticed that most of the in-class Have you noticed that most of the in-class examples have been imperative examples have been imperative programming?programming?– Non Object-OrientedNon Object-Oriented
● Does this mean that the techniques we've Does this mean that the techniques we've learned aren't applicable to OO?learned aren't applicable to OO?– No, they're still very applicableNo, they're still very applicable
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
10
Slide 10 of 110
Practical ('prak-ti-kul) Practical ('prak-ti-kul) adjectiveadjective
● 4a: disposed to action as opposed to 4a: disposed to action as opposed to speculation or abstractionspeculation or abstraction
● Something you can accomplishSomething you can accomplish● Realistic goalsRealistic goals● Everyday usesEveryday uses
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
11
Slide 11 of 110
Question #4Question #4
● Do you work in a “Practical Development Do you work in a “Practical Development Environment”?Environment”?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
12
Slide 12 of 110
So What's an Impractical So What's an Impractical Environment?Environment?
● Space Shuttle on-board software teamSpace Shuttle on-board software team– Must be 100% goodMust be 100% good
– They test everythingThey test everything
– Realistically, fast and cheap are ignoredRealistically, fast and cheap are ignored
– No such thing as “good enough”No such thing as “good enough”
● ““Real” worldReal” world– Don't have that luxuryDon't have that luxury
– Must set prioritiesMust set priorities
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
13
Slide 13 of 110
3-Way Practicality Process Matrix3-Way Practicality Process Matrix**
● There are three things that can define any There are three things that can define any processprocess– GoodGood
– FastFast
– CheapCheap
*Pick any two
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
14
Slide 14 of 110
●QuoteQuote
● Everybody in the real world will agree - the Everybody in the real world will agree - the moment a project is behind deadline, quality moment a project is behind deadline, quality assurance tends to go out the window. assurance tends to go out the window.
Alan Cox
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
15
Slide 15 of 110
Road MapRoad Map
● IntroductionIntroduction
● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
16
Slide 16 of 110
Object-Oriented SystemsObject-Oriented Systems
● ObjectObject● MessageMessage● InterfaceInterface● ClassClass● InheritanceInheritance● PolymorphismPolymorphism
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
17
Slide 17 of 110
ObjectObject
● Operational entityOperational entity– As opposed to definitional entityAs opposed to definitional entity
● EncapsulatesEncapsulates– Specific data valuesSpecific data values
– Code that manipulates data valuesCode that manipulates data values
● Basic building block of OO systemsBasic building block of OO systems
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
18
Slide 18 of 110
Object: ExamplesObject: Examples
● CourseCourse● StudentStudent● BillBill● GradeGrade
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
19
Slide 19 of 110
Object: Testing ImplicationsObject: Testing Implications
● EncapsulationEncapsulation– Build up object and pass it aroundBuild up object and pass it around
– Easy to manipulateEasy to manipulate
● Information hidingInformation hiding– Difficult to change objectDifficult to change object
– Changes in object difficult to observeChanges in object difficult to observe
● Has stateHas state– Can become inconsistentCan become inconsistent
● Has a lifetimeHas a lifetime– Construction/DestructionConstruction/Destruction
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
20
Slide 20 of 110
MessageMessage
● Request that an operation be performed by Request that an operation be performed by an objectan object
● Allows objects to collaborate to solve Allows objects to collaborate to solve problemsproblems
● Different languagesDifferent languages– C++: member function callC++: member function call
– Java/Smalltalk: method invocationJava/Smalltalk: method invocation
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
21
Slide 21 of 110
Message: ExamplesMessage: Examples
● Student.AddCourse(Course)Student.AddCourse(Course)● Student.SendBill(Bill)Student.SendBill(Bill)● Student.MakePayment(CreditCard)Student.MakePayment(CreditCard)● Grade.SetGrade(“A”)Grade.SetGrade(“A”)● Student.SetGradeForCourse(Grade,Course)Student.SetGradeForCourse(Grade,Course)● Student.SendGrades()Student.SendGrades()
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
22
Slide 22 of 110
Message: Testing ImplicationsMessage: Testing Implications
● Has a senderHas a sender– Sender determines when to send messageSender determines when to send message
– May send incorrectlyMay send incorrectly
● Has a receiverHas a receiver– Receiver may not be readyReceiver may not be ready
– Unexpected message may be ignoredUnexpected message may be ignored
● May include parameters/return valueMay include parameters/return value– Used and/or updated by receiverUsed and/or updated by receiver
– Object state is importantObject state is important
– Receiver may throw exceptionReceiver may throw exception
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
23
Slide 23 of 110
InterfaceInterface
● Aggregation of behavior declarationsAggregation of behavior declarations● Different languagesDifferent languages
– C++: abstract base class with public pure C++: abstract base class with public pure virtual methodsvirtual methods
– Java: interface (defined just like a class)Java: interface (defined just like a class)
● Building block for specificationsBuilding block for specifications
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
24
Slide 24 of 110
Interface: ExamplesInterface: Examples
● EnrollmentEnrollment– Behavior associated with enrolling in coursesBehavior associated with enrolling in courses
● ResidenceResidence– Behavior associated with living in on-campus Behavior associated with living in on-campus
housinghousing
● BillableBillable– Behavior associated with being able to make Behavior associated with being able to make
paymentspayments● StudentStudent● ParentParent● CompanyCompany
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
25
Slide 25 of 110
Interface: Testing ImplicationsInterface: Testing Implications
● Possible poor designPossible poor design– Some interfaces may have 'extra' behaviorSome interfaces may have 'extra' behavior
● Interface as parameterInterface as parameter– An object may be passed merely as its An object may be passed merely as its
interfaceinterface
– Unwanted baggage may be carried alongUnwanted baggage may be carried along
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
26
Slide 26 of 110
ClassClass
● Set of objects (members) that share a Set of objects (members) that share a common conceptual basiscommon conceptual basis
● Class definition Class definition – What the members have in commonWhat the members have in common
– What they will do togetherWhat they will do together
● ObjectsObjects– Elements for executing a programElements for executing a program
● ClassesClasses– Elements for defining a programElements for defining a program
● An Object is an instance of a ClassAn Object is an instance of a Class
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
27
Slide 27 of 110
Class: SpecificationClass: Specification
● Operation: action applied to an object to Operation: action applied to an object to obtain a certain effectobtain a certain effect
● Two categories of operationsTwo categories of operations– AccessorAccessor
● Returns information about the objectReturns information about the object● Changes nothing in the objectChanges nothing in the object
– ModifierModifier● Changes the state of an objectChanges the state of an object
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
28
Slide 28 of 110
Preconditions/PostconditionsPreconditions/Postconditions
● PreconditionPrecondition– Condition that must be true before an Condition that must be true before an
operation is attemptedoperation is attempted
● PostconditionPostcondition– Condition that must be true after an operation Condition that must be true after an operation
is completedis completed
● InvariantInvariant– Condition that must be true before Condition that must be true before andand after after
an operationan operation
– Sort of a pre/post combinationSort of a pre/post combination
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
29
Slide 29 of 110
PreconditionsPreconditions
● State and attributes of the object containing State and attributes of the object containing the operationthe operation
● Attributes of Attributes of parameters in parameters in the messagethe message
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
30
Slide 30 of 110
PostconditionsPostconditions
● State and attributes of the object containing State and attributes of the object containing the operationthe operation
● Attributes of Attributes of parameters in parameters in the messagethe message● Return value (if any)Return value (if any)● Any exception that might ariseAny exception that might arise
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
31
Slide 31 of 110
InvariantsInvariants
● Attributes that must be the same before and Attributes that must be the same before and afterafter
● OperationalOperational– Invariants during a single operationInvariants during a single operation
● ClassClass– Invariants for the lifetime of an objectInvariants for the lifetime of an object
– Implies: during ALL operationsImplies: during ALL operations
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
32
Slide 32 of 110
Road MapRoad Map
● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview
● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
33
Slide 33 of 110
Question #5Question #5
● Is OO Testing:Is OO Testing:– Blackbox testing?Blackbox testing?
– Whitebox tesing?Whitebox tesing?
● Why?Why?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
34
Slide 34 of 110
Trick Question: BothTrick Question: Both
● WhiteboxWhitebox– Looking at how the code is writtenLooking at how the code is written
– Analyzing code structureAnalyzing code structure● ClassClass● MessageMessage● InheritanceInheritance
● BlackboxBlackbox– How objects behave from the outsideHow objects behave from the outside
● InterfaceInterface● PolymorphismPolymorphism
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
35
Slide 35 of 110
OO Is More Than the CodeOO Is More Than the Code
● It can also include the entire development It can also include the entire development process*process*– RequirementsRequirements
– Analysis and DesignAnalysis and Design
– ImplementationImplementation
*Please see EECS 816: OO Analysis and Design
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
36
Slide 36 of 110
The Big Zoom OutThe Big Zoom Out
● Many levels of testingMany levels of testing● Each has 'challenges'Each has 'challenges'
– ClassClass
– Class InteractionsClass Interactions
– Class HierarchiesClass Hierarchies
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
37
Slide 37 of 110
So, What's a Class Again?So, What's a Class Again?
● Fundamental unit of OO programmingFundamental unit of OO programming● First level of information-hidingFirst level of information-hiding● Key to encapsulationKey to encapsulation● Building block of inheritanceBuilding block of inheritance
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
38
Slide 38 of 110
How Can You Test a Class?How Can You Test a Class?
● Two ways:Two ways:– Code reviews (static)Code reviews (static)
– Execution of test cases (dynamic)Execution of test cases (dynamic)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
39
Slide 39 of 110
Code Reviews (Pro/Con)Code Reviews (Pro/Con)
● ProsPros– Ability to shift with changing specificationsAbility to shift with changing specifications
– Exposes more of the team to the Exposes more of the team to the implementationimplementation
● ConsCons– Humans involved (mistakes)Humans involved (mistakes)
– Time-consuming (especially for regression Time-consuming (especially for regression testing)testing)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
40
Slide 40 of 110
Test Case Execution (Pro/Con)Test Case Execution (Pro/Con)
● ProsPros– Ability to shift with changing specificationsAbility to shift with changing specifications
– Exposes more of the team to the Exposes more of the team to the implementationimplementation
– Easily repeatable (Regression testing)Easily repeatable (Regression testing)
● ConsCons– Humans involvedHumans involved
– Difficult to identifyDifficult to identify
– Time-consuming to writeTime-consuming to write● If coverage is good, maybe more time than If coverage is good, maybe more time than
writing the code itselfwriting the code itself
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
41
Slide 41 of 110
So, Which One's the Best?So, Which One's the Best?
● That depends. . . . That depends. . . . – How often do you need to test?How often do you need to test?
– What development model do you use?What development model do you use?
– Aren't they complementary?Aren't they complementary?
● Yes, they're complimentaryYes, they're complimentary● Technically, a lot of overlapTechnically, a lot of overlap● Better to do both, but may reduce the Better to do both, but may reduce the
practicalitypracticality
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
42
Slide 42 of 110
Our FocusOur Focus
● Execution-based testingExecution-based testing– Easily repeatableEasily repeatable
– More useful in non-waterfall modelsMore useful in non-waterfall models
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
43
Slide 43 of 110
Main TroubleMain Trouble
● Identifying test casesIdentifying test cases
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
44
Slide 44 of 110
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
45
Slide 45 of 110
Assumption QuoteAssumption Quote
● ““We assume that a class to be tested has a We assume that a class to be tested has a complete and correct specification. . . . . We complete and correct specification. . . . . We assume the specification is expressed in a assume the specification is expressed in a specification language such as the Object specification language such as the Object Constraint Language (OCL) or a natural Constraint Language (OCL) or a natural language and/or as a state transition language and/or as a state transition diagram.”diagram.”
● Does that sound practical to you?Does that sound practical to you?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
46
Slide 46 of 110
Complete Specifications?Complete Specifications?
● Not me!Not me!
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
47
Slide 47 of 110
But, Seriously. . .But, Seriously. . .
● In order to write tests, you need a In order to write tests, you need a specification of some kindspecification of some kind
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
48
Slide 48 of 110
So, What's a Specification So, What's a Specification Exactly?Exactly?
● Link between real world and model Link between real world and model – Part of the problemPart of the problem
– Part of the solutionPart of the solution
● Description of operationDescription of operation– Class a wholeClass a whole
– OperationsOperations
– Preconditions/postconditionsPreconditions/postconditions
– InvariantsInvariants
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
49
Slide 49 of 110
What If I Don't Have a Spec?What If I Don't Have a Spec?
● Get one Get one
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
50
Slide 50 of 110
No, Really, What If I Don't Have a No, Really, What If I Don't Have a Spec?Spec?
● Reverse-engineer from the codeReverse-engineer from the code– And how think the code is supposed to workAnd how think the code is supposed to work
● Test-driven developmentTest-driven development– Write the testsWrite the tests
– Write the codeWrite the code
– RepeatRepeat
– When do you write comments?When do you write comments?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
51
Slide 51 of 110
Better To Write As You Go. . .Better To Write As You Go. . .
● Tests are codeTests are code● Consider test code linked with Consider test code linked with
implementationimplementation● Unit testingUnit testing
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
52
Slide 52 of 110
What is Unit Testing?What is Unit Testing?
● Unit TestUnit Test– procedure used to verify that a particular procedure used to verify that a particular
module of source code is working properlymodule of source code is working properly
● Goal is to catch regression errorsGoal is to catch regression errors● Unit test frameworksUnit test frameworks
– Sunit (Smalltalk)Sunit (Smalltalk)
– Junit (Java)Junit (Java)
– CppUnit (C++)CppUnit (C++)
– NUnit (C#)NUnit (C#)
● PracticalPractical
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
53
Slide 53 of 110
eXtreme Programming (XP)eXtreme Programming (XP)
● Agile development modelAgile development model● ““Requires” test-first modelRequires” test-first model● Must write the test before the code that will Must write the test before the code that will
be testedbe tested● We're getting off track. . .We're getting off track. . .
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
54
Slide 54 of 110
Road MapRoad Map
● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing
● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
55
Slide 55 of 110
Finding DefectsFinding Defects
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
56
Slide 56 of 110
Dimensions of Class TestingDimensions of Class Testing
● WhoWho● WhatWhat● WhenWhen● HowHow● How much?How much?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
57
Slide 57 of 110
WhoWho
● Usually developers (of the code itself)Usually developers (of the code itself)● Minimizes public knowledge (good)Minimizes public knowledge (good)● Minimizes peer review (bad)Minimizes peer review (bad)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
58
Slide 58 of 110
WhatWhat
● Code meets requirementsCode meets requirements– No moreNo more
– No lessNo less
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
59
Slide 59 of 110
WhenWhen
● NeverNever● MilestonesMilestones● When you have a specificationWhen you have a specification● When changes occurWhen changes occur● AlwaysAlways
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
60
Slide 60 of 110
HowHow
● Drivers and stubsDrivers and stubs● Unit test frameworksUnit test frameworks● Instance vs. staticInstance vs. static
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
61
Slide 61 of 110
How Much?How Much?
● Spec vs. implementationSpec vs. implementation● Simple answer: Enough!Simple answer: Enough!● Impractical answer: Everything!Impractical answer: Everything!● Risk-based analysisRisk-based analysis
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
62
Slide 62 of 110
Back To Not Having a Spec. . .Back To Not Having a Spec. . .
● Writing testable codeWriting testable code● Contract ApproachContract Approach
– Preconditions over postconditionsPreconditions over postconditions
● Defensive ApproachDefensive Approach– Postconditions over preconditionsPostconditions over preconditions
● Problems?Problems?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
63
Slide 63 of 110
Contract ApproachContract Approach
● Preconditions are an obligation of the Preconditions are an obligation of the sendersender
● If met, receiver If met, receiver is obligated to meis obligated to meet the et the requirements and postconditionsrequirements and postconditions
● Design implicationsDesign implications– Need sufficient preconditions so receiver can Need sufficient preconditions so receiver can
fulfill postconditionsfulfill postconditions– a sender can determine a sender can determine whether all whether all
preconditions are met before sending a preconditions are met before sending a messagemessage
– May need extra accessor methods to check May need extra accessor methods to check for postcondition successfor postcondition success
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
64
Slide 64 of 110
Defensive ApproachDefensive Approach
● Interface defined in terms of the receiverInterface defined in terms of the receiver● Operation typically returns a valueOperation typically returns a value
– Return codeReturn code
– Status objectStatus object● Primary goalPrimary goal
– Sanity checkingSanity checking
– Returning status of processingReturning status of processing
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
65
Slide 65 of 110
Test Writing ImplicationsTest Writing Implications
● Contract approachContract approach– simplifies class testingsimplifies class testing– complicates interaction testingcomplicates interaction testing
● must ensure any sender meets the must ensure any sender meets the preconditionspreconditions
● Defensive approachDefensive approach– complicates class testingcomplicates class testing
● test cases must address all possible test cases must address all possible outcomesoutcomes
– complicates interaction testingcomplicates interaction testing● must ensure all possible outcomes are must ensure all possible outcomes are
produced and that they are properly handled produced and that they are properly handled by the senderby the sender
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
66
Slide 66 of 110
Road MapRoad Map
● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses
● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
67
Slide 67 of 110
Programming TipProgramming Tip
● Testing is for cowardsTesting is for cowards– A brave coder will bypass that stepA brave coder will bypass that step
– If we have full confidence in our coding If we have full confidence in our coding ability, then testing will be unnecessary ability, then testing will be unnecessary
- from How to Write Unmaintainable Code
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
68
Slide 68 of 110
Class InteractionsClass Interactions
● OO ProgramOO Program– Collection of objects Collection of objects
– Collaborate together to solve a problemCollaborate together to solve a problem
– Ensure that messaging happens correctlyEnsure that messaging happens correctly
● AssumptionsAssumptions– Objects have been individually testedObjects have been individually tested
– Unidirectional messagesUnidirectional messages
– Interactions are sequential (not concurrent)Interactions are sequential (not concurrent)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
69
Slide 69 of 110
What's an Interaction?What's an Interaction?
● Object 'A' calls a method 'X' of Object 'B'Object 'A' calls a method 'X' of Object 'B'● Object B's method X executesObject B's method X executes● Reply from method X is returned to AReply from method X is returned to A
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
70
Slide 70 of 110
Impact of a “Single” InteractionImpact of a “Single” Interaction
● Possible cascade of other object interactionsPossible cascade of other object interactions– State of the receiverState of the receiver– Associated objectsAssociated objects– Possible outcomesPossible outcomes
● ““No change”No change”● Changes in attribute valuesChanges in attribute values● State changesState changes● Creation or deletion of objectsCreation or deletion of objects
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
71
Slide 71 of 110
Identifying InteractionsIdentifying Interactions
● Two types of classesTwo types of classes● Primitive classesPrimitive classes
– SimpleSimple
– No other classes neededNo other classes needed
– Usually limited in numberUsually limited in number
● Non-primitiveNon-primitive– Support/require other objectsSupport/require other objects
– Most common in OO programmingMost common in OO programming
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
72
Slide 72 of 110
Two Categories of ClassesTwo Categories of Classes
● CollectionCollection● CollaborationCollaboration
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
73
Slide 73 of 110
Collection ClassesCollection Classes
● Collection classesCollection classes– Have classes associated with themHave classes associated with them
– Never actually Never actually use use them them
– Store/Create/DeleteStore/Create/Delete
– ExamplesExamples● StackStack● QueueQueue● MapMap● VectorVector
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
74
Slide 74 of 110
Testing Collection ClassesTesting Collection Classes
● Fairly straightforwardFairly straightforward● Capacity limitationsCapacity limitations● Store/retrieve order correctnessStore/retrieve order correctness● Negative testsNegative tests
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
75
Slide 75 of 110
Collaboration ClassesCollaboration Classes
● More complex than collection classesMore complex than collection classes● IdentificationIdentification
– Uses other objects in operationsUses other objects in operations
– When, after an operationWhen, after an operation● State is possibly affectedState is possibly affected● An attribute is used or modifiedAn attribute is used or modified
● Handout # 1Handout # 1– Identify the classesIdentify the classes
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
76
Slide 76 of 110
What To Test?What To Test?
● BestBest– Every collaboration interactionEvery collaboration interaction
– Not reasonableNot reasonable
● BetterBetter– Interactions most likely to have faultsInteractions most likely to have faults
● GoodGood– Probability distributionProbability distribution
● Good enough (Not much choice here)Good enough (Not much choice here)– Less than ideal probability distributionLess than ideal probability distribution
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
77
Slide 77 of 110
Test Case SamplingTest Case Sampling
– RandomRandom
– Use ProfilingUse Profiling
– Stratified SampleStratified Sample
– OATS (Orthogonal Array Test System)OATS (Orthogonal Array Test System)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
78
Slide 78 of 110
Use ProfileUse Profile
● Associate each use of the system with a Associate each use of the system with a frequencyfrequency
● RankedRanked● Probability of useProbability of use● Higher frequency of use ---> More testingHigher frequency of use ---> More testing
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
79
Slide 79 of 110
●Stratified SampleStratified Sample
● Tests selected from categoriesTests selected from categories● Example: Test subsetsExample: Test subsets
– Select a set of test cases that exercise every Select a set of test cases that exercise every component in the systemcomponent in the system
– Divide those into subsets based on Divide those into subsets based on componentcomponent
● Example: Use Case ActorsExample: Use Case Actors– Rank uses by actor/frequencyRank uses by actor/frequency
– Handout #2Handout #2
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
80
Slide 80 of 110
Orthogonal Array Test System Orthogonal Array Test System (OATS)(OATS)
● Method to limit explosion of test casesMethod to limit explosion of test cases● Uses pair-wise combination of factorsUses pair-wise combination of factors
– Two-way interactions cause most faultsTwo-way interactions cause most faults
● Sometimes called “pairwise testing”Sometimes called “pairwise testing”● Used in Taguchi MethodsUsed in Taguchi Methods
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
81
Slide 81 of 110
OATS: Basic TechniqueOATS: Basic Technique
● 1) Determine number of independent 1) Determine number of independent variablesvariables
● 2) Determine number of values it will take 2) Determine number of values it will take (states)(states)
● 3) Find an minimum-pair array (book)3) Find an minimum-pair array (book)● 4) Map factors into array4) Map factors into array● 5) Build test cases5) Build test cases● These steps may vary slightly from user to These steps may vary slightly from user to
useruser
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
82
Slide 82 of 110
OATS: ExampleOATS: Example
● Consider a web page with three sections Consider a web page with three sections (Top, Middle, and Bottom).(Top, Middle, and Bottom).
● Sections that can be individually shown or Sections that can be individually shown or hidden by the user. hidden by the user.
● You wish to test the interactions of the You wish to test the interactions of the different sections.different sections.
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
83
Slide 83 of 110
Question #6Question #6
● How many test cases would there be if you How many test cases would there be if you wanted 100% coverage?wanted 100% coverage?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
84
Slide 84 of 110
Answer #6Answer #6
● 8 tests, 2^3 = 88 tests, 2^3 = 8
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
85
Slide 85 of 110
OATS 1-Independent VariablesOATS 1-Independent Variables
● Three independent variables (page sections)Three independent variables (page sections)– TopTop
– MiddleMiddle
– BottomBottom
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
86
Slide 86 of 110
OATS 2-Variable StatesOATS 2-Variable States
● Each variable can take two statesEach variable can take two states– HiddenHidden
– VisibleVisible
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
87
Slide 87 of 110
OATS 3-Minimum Pair ArrayOATS 3-Minimum Pair Array
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
88
Slide 88 of 110
OATS 4-Map Factors Into ArrayOATS 4-Map Factors Into Array
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
89
Slide 89 of 110
OATS 5-Build Test CasesOATS 5-Build Test Cases
● 1) Hide all sections1) Hide all sections● 2) Display all but the top section2) Display all but the top section● 3) Display all but the middle section3) Display all but the middle section● 4) Display all but the bottom section4) Display all but the bottom section
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
90
Slide 90 of 110
OATS: OO ProgrammingOATS: OO Programming
● Independent variables are class familiesIndependent variables are class families● Values are members of the class familyValues are members of the class family● See Handout #3See Handout #3
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
91
Slide 91 of 110
OATS: Adequacy CriteriaOATS: Adequacy Criteria
● Exhaustive Exhaustive – all combinations of all factorsall combinations of all factors
● MinimalMinimal– only interactions between the base classes only interactions between the base classes
● RandomRandom– tester haphazardly selects cases from tester haphazardly selects cases from
several of the classesseveral of the classes● RepresentativeRepresentative
– uniform sample -- every class is tested uniform sample -- every class is tested ● Weighted representativeWeighted representative
– adds cases of relative importance or risk adds cases of relative importance or risk
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
92
Slide 92 of 110
Road MapRoad Map
● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions
● HierarchiesHierarchies● ConclusionConclusion
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
93
Slide 93 of 110
Programming TipProgramming Tip
● Never, Ever Do Any Performance TestingNever, Ever Do Any Performance Testing– Hey, if it isn't fast enough, just tell the Hey, if it isn't fast enough, just tell the
customer to buy a faster machine customer to buy a faster machine
- from How to Write Unmaintainable Code
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
94
Slide 94 of 110
Class Hierarchies (Inheritance) Class Hierarchies (Inheritance)
● Relationship between classesRelationship between classes● Allows new class to be based on existing Allows new class to be based on existing
classclass● Reuse of specification and implementationReuse of specification and implementation● Parent (source) classParent (source) class
– SuperclassSuperclass
– Base classBase class
● New ClassNew Class– SubclassSubclass
– Derived classDerived class
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
95
Slide 95 of 110
Inheritance (cont.)Inheritance (cont.)
● Inheritance hierarchyInheritance hierarchy● Each class has ancestors (except roots)Each class has ancestors (except roots)● Good OO DesignGood OO Design
– is-a is-a relationshiprelationship
– is-a-kind-ofis-a-kind-of relationship relationship
● Best when used to propagate specificationBest when used to propagate specification– Not implementation (code reuse)Not implementation (code reuse)
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
96
Slide 96 of 110
Inheritance: Testing ImplicationsInheritance: Testing Implications
● Provides a bug propagation mechanismProvides a bug propagation mechanism– Inherit defects from parentInherit defects from parent
– Good reason to test class hierarchy as it Good reason to test class hierarchy as it developsdevelops
● Allows for reuse of test casesAllows for reuse of test cases– Reuse of spec and implementationReuse of spec and implementation
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
97
Slide 97 of 110
A Tale of Two ClassesA Tale of Two Classes
● CC– Parent or superclassParent or superclass
● DD– Child or subclassChild or subclass
● Handout #4Handout #4
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
98
Slide 98 of 110
Is-Kind-Of RelationshipIs-Kind-Of Relationship
● D is-a-kind-of CD is-a-kind-of C● D can be used in place of C in a programD can be used in place of C in a program● Behavior of D must match (or supersede) CBehavior of D must match (or supersede) C● Class BehaviorClass Behavior
– Observable statesObservable states
– Operational semanticsOperational semantics
● Enforced by the Substitution PrincipleEnforced by the Substitution Principle
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
99
Slide 99 of 110
Substitution PrincipleSubstitution Principle● Only these changes are allowed in a Only these changes are allowed in a
subclasssubclass– PreconditionsPreconditions
● Must be the same or weakerMust be the same or weaker● Less constraining to a clientLess constraining to a client
– PostconditionsPostconditions● Must be the same or strongerMust be the same or stronger● Must expect as much as superclassMust expect as much as superclass
– Class InvariantClass Invariant● Must be the same or strongerMust be the same or stronger● Can add more constraintsCan add more constraints
● Assume Substitution PrincipleAssume Substitution Principle
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
100
Slide 100 of 110
Refinement Possibilities Refinement Possibilities (Subclass)(Subclass)
● 1) Add new operations/methods to D1) Add new operations/methods to D● 2) Change spec or implementation of and 2) Change spec or implementation of and
operation declared by Coperation declared by C– Change in D an operation declared in CChange in D an operation declared in C
– Override in D a method inherited in COverride in D a method inherited in C
● 3) Add new instance members in D3) Add new instance members in D● 4) Change the class invariant4) Change the class invariant
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
101
Slide 101 of 110
Hierarchical, Incremental Testing Hierarchical, Incremental Testing (HIT)(HIT)
● Incremental changes from C to D can guide Incremental changes from C to D can guide test case selectiontest case selection
● All of the spec-based tests of C should apply All of the spec-based tests of C should apply to Dto D– If not, substitution rule has been brokenIf not, substitution rule has been broken
● Simple process to determineSimple process to determine– Which C tests are still applicableWhich C tests are still applicable
– What new tests need to be writtenWhat new tests need to be written
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
102
Slide 102 of 110
1) HIT: Add Operations1) HIT: Add Operations
● New operation implements new functionalityNew operation implements new functionality● New functionality needs to be testedNew functionality needs to be tested● Also need to addAlso need to add
– Class/Object implementation testClass/Object implementation test
– Class Interaction testsClass Interaction tests
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
103
Slide 103 of 110
2) HIT: Change Spec2) HIT: Change Spec
● a) Change in D the spec for a method a) Change in D the spec for a method declared in Cdeclared in C– Must add new spec-based testsMust add new spec-based tests
– Must test new pre/post conditionsMust test new pre/post conditions
– Must re-run C test casesMust re-run C test cases
● b) Override a C method in D that uses a C b) Override a C method in D that uses a C operationoperation– Can reuse C's spec-based testsCan reuse C's spec-based tests
– Must review all implementation based testsMust review all implementation based tests
– Possibly new coveragePossibly new coverage
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
104
Slide 104 of 110
HIT: Change OperationsHIT: Change Operations
● New instance variables addedNew instance variables added– Most likely tested by new codeMost likely tested by new code
– Already covered by new tests?Already covered by new tests?
– If a new variable is not used, it doesn't need If a new variable is not used, it doesn't need to be testedto be tested
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
105
Slide 105 of 110
HIT: Change InvariantHIT: Change Invariant
● New postconditionsNew postconditions● Simply test the new invariant conditionsSimply test the new invariant conditions● Must re-run all inherited test cases to assure Must re-run all inherited test cases to assure
that the invariant holdsthat the invariant holds
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
106
Slide 106 of 110
ConclusionConclusion
● OO brings new tool to the programmerOO brings new tool to the programmer● New way of programmingNew way of programming● New ways of testingNew ways of testing● Old testing methods still applyOld testing methods still apply
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
107
Slide 107 of 110
Thoughts?Thoughts?
● How do you test now?How do you test now?● How can you change the testing of code in How can you change the testing of code in
your environment?your environment?● Will these techniques help you?Will these techniques help you?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
108
Slide 108 of 110
Questions?Questions?
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
109
Slide 109 of 110
ReferencesReferences
● A Practical Guide to Testing Object-Oriented A Practical Guide to Testing Object-Oriented SoftwareSoftware by John D. McGregor and David A. by John D. McGregor and David A. Sykes; 2001, Addison-Wesley.Sykes; 2001, Addison-Wesley.
● The Object PrimerThe Object Primer by Scott W. Ambler; by Scott W. Ambler; 2001, Cambridge University Press.2001, Cambridge University Press.
● ””JUnit Primer”: JUnit Primer”: http://www.clarkware.com/articles/JUnitPrimhttp://www.clarkware.com/articles/JUnitPrimer.htmler.html
● ““Unit Testing”: Unit Testing”: http://en.wikipedia.org/wiki/Unit_testinghttp://en.wikipedia.org/wiki/Unit_testing
Through a Glass, Darkly: Object-Oriented Testing
Mark Stockmyer December 1, 2005
110
Slide 110 of 110
ReferencesReferences
● ““Orthogonal Array Testing System (OATS)”: Orthogonal Array Testing System (OATS)”: http://www.cvc.uab.es/shared/teach/a20370/http://www.cvc.uab.es/shared/teach/a20370/Temas/TemarioEnCurso/Prova orientada a Temas/TemarioEnCurso/Prova orientada a objecte/orthogonal arrays.pdfobjecte/orthogonal arrays.pdf