Embed Size (px)
Citation preview
Agenda
What is testability? Why is testability important? Background: What is a test? Determining testability: SOCK Improving testability Q&A
What is testability?
The degree to which components and systems are designed and implemented to make it easier for tests to achieve complete and repeatable code path coverage and simulate all usage situations in a cost efficient manner.
In other words: How easy is it to test?
Why is testability important?
Helps enforce good design principles. Reduces the time to diagnose unexpected
behavior. Improves manageability and supportability. Provides the building blocks for self-
diagnosing and self-correcting software. Speeds test development while increasing
the depth and quality of automated tests.
Background: What is a test?
Making sure the component works as expected and has the expected effect on the system as a whole in a repeatable and reliable way.
Anatomy of a test case: SEARCH– Setup– Execution– Analysis– Reporting– Cleanup– Help
Determining testability: SOCK
Simplicity– Reducing cost of testing at each phase of SEARCH– Design
Observability– Exposing state (aka Visibility)
Control– Exercising every nook & cranny as efficiently as possible.
Knowledge of expected results– Is observed behavior is correct?
Improving Simplicity
Make setup and configuration easy– Componentize– Allow multiple installs with varying configs on
same machine
Use good design principles– Strong cohesion– Loose coupling– Low redundancy
Improving Observability
Expose component state– Magic numbers (Any value that controls code path
execution.)– Resource usage– State transition information– API in/out parameters
Tracing– Good tracing library– Observer or Visitor patterns can help maintain good design
Be careful of exposing security info!
Improving Control
Wrap interfaces you have no control over– Forces behavior you need to test your component– Facilitates fault injection tools
Windows: Holodeck, AppVerifier
Expose Magic Numbers to become configurable at runtime.
Be mindful of security
Improving Knowledge of expected results
Code contracts in source code Document expected behavior in all situations Document end-to-end user scenarios Future: Use modeling (machine readable
specifications)
Q&A
Resources
Design Patterns Explained: A new perspective on object-oriented design. Alan Shalloway, James Trott. Addison Wesley, 2002.
– Observer pattern (tracing option for new classes)– Visitor pattern (tracing option for existing classes)– Adapter pattern (wrapping for fault injection)
Refactoring: Improving the design of existing code. Martin Fowler, et al. Addison-Wesley, 1999.
– Refactoring to improve design helps testability at the same time. www.netobjectives.com
– Emergent Design, other materials Fault injection tools for Windows
– Holodeck: http://www.sisecure.com fault injection tool.– AppVerifier: http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/appvrfr.mspx
Google: CCCC cyclomatic complexity tool Brett Pettichord, Industry Testability Guru
– Design for Testability: http://www.io.com/~wazmo/papers/design_for_testability_PNSQC.pdf– QA-related papers: http://www.io.com/~wazmo/qa– Other: http://www.pettichord.com
