Embed Size (px)
Citation preview
Pragmatic Software Testing
Becoming an Effective and Efficient Test Professional
Rex Blaill
1 8 O 7
®WILEY 2 O O 7
Wiley Publishing, Inc.
Acknowledgments xiii
Introduction xv
EarJJ Goals. Strategies, and Tactics L
Chapter 1 What Does It Mean to Be Pragmatic? 3 What Do Effective and Efficient Mean? 3
What Effects Do You Want? 4 What Is the Right Level of Efficiency? 4
Avoiding Redundancy 4 Reducing Cost 5 What Software Testing Isn't.. .But Is Often Thought to Be 6
Five Phases of a Tester's Mental Life 7 Other Founding Views on Testing 8 Testing as a Form of Quality Risk Management 9 So What Is the Test Team All About? 9
What Does "Quality" Mean to You? 10
Chapter 2 Triangle Test Exerdse 13 Exercise: The Triangle Test 13
Your Solution for the Triangle Test 14 Your Solution for the Triangle Test (Continued) 15 Author's Triangle Test Solution 15
Chapter 3 Aligning Testing with the Project 19 Why Do Organization Test? 19 Perspectives on Testing 20 Testing in Context 20 Common Test Phases and Objectives 21 Testing Throughout the Organization 21 The V Model 23 Evolutionary and Incremental Models 25 The Spiral Model 28 Regrettably Common Model: Code and Fix 28
Contents
Testing Maintenance Releases 29 System Integration 30 Hardware/Software Development 32 The Test Process 32
Chapter 4 Understanding Test Strategies, Tactics, and Design 35 Aligning Mission, Strategies, and Tactics 35 Analytical Test Strategies 36 Model-Based Test Strategies 37 Methodical Test Strategies 38 Process-Oriented Test Strategies 38 Dynamic Test Strategies 39 Philosophical Test Strategies 40 Regression 41
Regression Strategy 1: Repeat All Tests 42 Regression Strategy 2: Repeat Some Tests 42
Three Other Regression Strategies 43 Tactics: Categories of Testing Techniques 44 Strategie and Tactical Considerations 46 The Test System 47 Classic Principles for Test Design 50 Phases of Test Development 51 Synopsis of Test Strategies, Tactics, and Design 52
EarJLU Risk-Based Testing 5 1
Chapter 5 Understanding Risks to System Quality 55 Categories of Quality Risks 56
Functionality 56 Performance and Reliability 57 Stress, Capacity, and Volume 59 States 59
Transactions 60 Installation and Deinstallation 61 Operations 62 Maintenance and Maintainability 63
Regression 63 Usability and User Interface 64 Data Quality 65 Error and Disaster Handling and Recovery 66 Date and Time Handling 67 Localization 68 Configuration and Compatibility 69 Networked, Internetworked, and Distributed 69 Standards and Regulatory Compliance 70 Security 70 Timing and Coordination 71 Documentation 71
CanYouThinkof Other Quality Risks? 72
Contents vii
Chapter 6
Chapter 7
Part III
Chapter 8
Chapter 9
^
Aligning Testing with Quality Risks Prioritize Risks to System Quality Testing, Customer Usage, and System Configurations Approaches for Quality Risks Analysis Informal Quality Risk Analysis Tips for Risk Analysis Challenges of Risk Analysis
Quality Risk Analysis Exercise My Solution Comments on My Solution A Funny Thing Happened to Me on the Way
to the Quality Risks... Bonus Exercise Template My Solution Comments on My Solution Bonus Example: Grays and Blues Quality Risk Analysis
Static Testing
Reviews and Other Static Tests Testing Requirements and Designs Reviews Costs and Benefits Types of Reviews Reviews for Verification and Validation Reviews as a Way to Improve the Process, Improve
the System, and Reduce Costs Answers to the Three Questions about Static Testing Reviews as a Way to Achieve Consensus and Understanding The Review Process, Roles, and Responsibilities Deliverables and Ground Rules from Reviews Common Requirements and Design Bugs Reviewing (and Testing) Documentation Other Static Tests
Review Exercise Reviews Your Solution My Solution Comments on My Solution Bonus Exercise: Reviewing Triangle Requirements
Title Page Triangle Requirements Review Using Wiegers's List
The Requirements Bugs You Found My Solution: Requirements Bugs Comments on My Solution
73 73 74 76 78 81 82
85 86 91
92 95 96
100 101 102
105
107 107 108 109 109
110 114 115 116 117 118 119 120
123 123 124 128 132
132 132 134 137 139
v i i i Contents
Part IV Behavioral Testing 141
Chapter 10 Equivalence Classes and Boundary Values 143 Equivalence Partitioning 144 Boundary Value Analysis 144
Integer 145 Real Numbers 147 Character and String 149 Date 152 Time 154 Currency 155 Beyond Functional Classes and Boundaries 157
Chapter 11 Equivalence Classes and Boundary Values Exercise 159 Functional Boundaries and Classes 159 My Solution and Comments 161 Nonfunctional Boundaries and Classes 166
Omninet Marketing Requirements Document (v0.3) 166 Omninet System Requirements Document (v0.2) 168
Chapter 12 Use Cases, Live Data, and Decision Tables 169 Use Cases and Scenario Tests 169 Nouns and Verbs, Adjectives and Adverbs 172 Live Data and Customer Tests 173 Decision Tables 174
Chapter 13 Decision Table Exercise 177 Decision Table Tests 177 My Solution and Comments 178 Decision Tables and Boundary Values 181 My Solution and Comments 183 Building a Decision Table for Testing 188 My Solution and Comments 190
Chapter 14 State Transition Diagrams 197 Describing Systems with States Using State
Transition Diagrams 197 State Tables 199 Printer Server State Transition Diagram 200
Chapter 15 State Transition Diagram Exercise 203 Kiosk States 203 My Solution and Comments 204 ATM State Models 211 My Solution and Comments 212 Grays and Blues and State Transition Diagrams 213 My Solution and Comments 214
Chapter 16 Domain Testing 217 Combinatorial Explosions 217 A Domain Example Using Frequent-Flyer Programs 218 Possible Domain Test Values 220
Contents
An Aerospace Example 221 When Domain Rules Change 223 Domain Analysis Summary 224 Complex Domain Testing Example 225 A General Rule for Complex Domains 228
Chapter 17 Domain Testing Exercise 233 My Solution and Comments 234
Chapter 18 Orthogonal Arrays and All Pairs 239 Combinatorial Explosions 239 Orthogonal Arrays and All-Pairs Tables 241 Two Orthogonal Arrays 241 Selecting an Orthogonal Array 243 Applying Orthogonal Arrays to a Complicated
Real-World Example 244 All-Pairs Tables 258 Other Thoughts on Configuration Testing 259
Chapter 19 Orthogonal Arrays Exercise 261 My Solution and Comments 263
Chapter 20 Reactive Testing 267 General Facts about Reactive Tests 267 Error Guessing, Attacks, and Bug Taxonomies 268 Bug Hunting 269 Exploratory Tests 270 Checklists 271 Other Sources of Inspiration for Reactive Testing 273 Advantages and Disadvantages 274 A Case Study of Exploratory Testing 276
PartV Structural Testing 1 229.
Chapter 21 Control-Flow Testing 281 Code Coverage 282 McCabe Cyclomatic Complexity 284
Chapter 22 Control-Flow Testing Exercise 287
My Solution and Comments 288
Chapter 23 Data-Flow Testing 293
Chapter 24 Data-Flow Testing Exercise 297 My Solutions and Comments 298
Chapter 25 Integration Testing 307 Drivers and Stubs 308 Integration Techniques 309 Backbone Integration 311 McCabe Basis Paths for Integration 313 Enhanced Hex Converter Program 316 Call Flow 318
x Contents
Chapter 26 Integration Basis Test Exercise 321 My Solution and Comments 323
Part VI Appendices 225.
Appendix A Omninet: The Internet Everywhere Marketing Requirements Document 327
1 Scope 329 1.1 Terms, Acronyms, and Abbreviations 329 1.2 Applicable Documents 330
2 Required release date 330 3 Description of requirements 330
3.1 General technical requirements 330 3.1.1 Welcome 330 3.1.2 Payment 330 3.1.3 Internet Browser 331 3.1.4 Performance 331 3.1.5 Localization 331 3.1.6 Content Control 331 3.1.7 Session Termination 331 3.1.8 Confidentiality 332
3.2 Administration 332 3.2.1 Software Updates 332 3.2.2 View Kiosks 332 3.2.3 View Users 333 3.2.4 Modify User 333 3.2.5 Terminate User 333
Appendix B Omninet: The Internet Everywhere System Requirements Document 335
Functionality System Requirements 336 Reliability System Requirements 340 Usability System Requirements 341 Efficiency System Requirements 342 Maintainability System Requirements 343 Portability System Requirements 344 Design Models 345
Omninet System Architecture 345 Payment Processing Decision Table 346 Kiosk Module Flow 347 Kiosk State-Transition Diagram 348 Kiosk State-Transition Table 348 Kiosk OS/Browser/Connection Speed Configuration
Orthogonal Array 348
Appendix C Bibliography and Other Recommended Readings 351 RBCS Company Profile 353
Index 355
