Embed Size (px)
Citation preview
What Any Developer Or
Tester Should Know Even
After Midnight
Hans Schaefer
Software Test Consulting
© 2005 Hans Schaefer Slide no. 2The little extra
What any Developer or Tester Should
Know, even After Midnight
Hans [email protected]
http://home.c2i.net/schaefer/
The “little extra”
© 2005 Hans Schaefer Slide no. 3The little extra
Contents
1. Testing The Normal Way is Not Enough.
2. Can we Do Testing in a Better Way?
3. The Purpose of Testing
4. Continuous Learning
5. A Critical Mindset
6. Defects
7. The Tester Has Some Rights
8. The Late Night Tester's Toolbox
© 2005 Hans Schaefer Slide no. 4The little extra
Testing The Normal Way is Not Enough.
Just do your job?
As told by the books?
As told by the plan?
How to survive time pressure?
Do you really LIKE this job?
How to get better?
Boring, inefficient
A GOOD job
Can we Do Testing in a Better Way?
© 2005 Hans Schaefer Slide no. 5The little extra
The Purpose of Testing
Only cowards do testing
The brave roll it out!
© 2005 Hans Schaefer Slide no. 6The little extra
The Purpose of Testing
Myers (1979): Go and find defects!
Hetzel (1988): Measure the quality!
Johnsen (1990): Improve, prevent defects!
NN (later): Measure and reduce the risk!
Paul Gerrard: Testing is Porject Intelligence
Whatever you do, the purpose is not to “just test it”
Try to find defects!
Look around, in the hidden details: Defects may appear at places where you do not see them easily, i.e. not on screen
output!
© 2005 Hans Schaefer Slide no. 7The little extra
Finding even more defects
Defects clump together: they are social!
Defects often have a common cause!
Try to find common causes, and then follow them!
Where you find defects, dig deeper!
© 2005 Hans Schaefer Slide no. 8The little extra
Testing is risk mitigation
Risk = probability for damage * cost of damage
Probability from usage profile
And from project work characteristics (complexity, turnover, rotten compromises, ...)
The very beginning: The good, the bad and the ugly.
What if everything is fine?
What happens if some input is wrong?
What happens if all hell breaks loose?
© 2005 Hans Schaefer Slide no. 9The little extra
A pessimist tests better!
© 2005 Hans Schaefer Slide no. 10The little extra
Continuous Learning
Even for us while testing!
New
methods
tools
defects
business domains
standards
Your system
which modules?
how do they communicate
the architecture
assumptions
domain, risks
Any software defect
Is interesting!
© 2005 Hans Schaefer Slide no. 11The little extra
Learning
Learn more, about everything!
Before going to the South pole in 1911, Roald Amundsen said:
«I am using three things to pull my equipment: dogs, dogs and dogs.»
For a tester, the three things are: Learning, learning and learning.
If you get mo time to learn on your job?
Get another job!?! Pull out experience!
© 2005 Hans Schaefer Slide no. 12The little extra
A Critical Mindset
Lenin said: “Trust is good, checking is better!”
My priest said: “Believing is something we do on Sundays in church,
everything else we either know or check!”
Assumptions lead to misunderstandings lead to errors
lead to defects lead to failures.
Don‟t assume. Ask!!
© 2005 Hans Schaefer Slide no. 13The little extra
A critical mindset
If nobody else asks the right question, you might do
so!
Think about new possibilities, unknown problems,
and the stuff you learn.
Think «out of the box»!
Don‟t trust even your own memory!
Learn interview techniques.
© 2005 Hans Schaefer Slide no. 14The little extra
Defects
You bring bad news nobody loves this!
1 - An issue is only interesting for a tester if it is accepted as a defect and repaired.
2 - There are defects, which are the result of running many test cases in a row in some very special order. Symptom: Intermittent failures.
You have a sales job!
© 2005 Hans Schaefer Slide no. 15The little extra
A primer on selling defects
Clearly describe them!
Check out more, find the real damage!
Report them in terms of risk!
Make them repeatable!
Be diplomatic!
© 2005 Hans Schaefer Slide no. 16The little extra
Handling intermittent failures
Analyze even intermittent problems!
Log everything you do in testing!
Log everything you see, and look at more remote details!
Make it possible to rerun your tests, with more logging and analysis tools switched on!
If nothing helps, at least log what you see and cannot repeat!
© 2005 Hans Schaefer Slide no. 17The little extra
The Tester Has Some Rights
A tester is NOT a vacuum cleaner for developer errors!
Lisa Crispin & Tip House:
* You have the right to make and update your own estimates
(…).
* You have the right to the tools you need to do your job (…).
* You have the right to expect your project team, not just
yourself, to be responsible for quality.
© 2005 Hans Schaefer Slide no. 18The little extra
Tom Gilb„s Tester„s Bill of Rights
1. Testers have the right to sample their process inputs, and reject poor quality work (no entry).
2. Testers have the right to unambiguous and clear requirements.
3. Testers have the right to test evolutionarily early as the system increments.
4. Testers have the right to integrate their test specifications into the other technical specifications.
5. Testers have the right to be a party to setting the quality levels they will test to.
6. Testers have the right to adequate resources to do their job professionally.
7. Testers have the right to an even workload, and to have a life.
8. Testers have the right to specify the consequences of products that they have not been allowed to test properly.
9. Testers have the right to review any specifications that might impact their work.
10. Testers have the right to focus on testing of agreed quality products, and to send poor work back to the source.
© 2005 Hans Schaefer Slide no. 19The little extra
The Late Night Tester's Toolbox
In testing, you must be able to state what coverage a test has
achieved!
Follow the usage profile if possible!
If not possible, test for robustness against wrong input.
Equivalence partitioning is a good basic technique!
Remember combination testing!
© 2005 Hans Schaefer Slide no. 20The little extra
The Late Night Tester's Toolbox
Test the test environment - well before test execution!
Check you test data!
Automate testing tasks!
Be aware that there is more automation than using test robots!
© 2005 Hans Schaefer Slide no. 21The little extra
And finally...
Be self-critical: You may be wrong – don‟t
trust yourself 100%!
Don‟t trust me!
© 2005 Hans Schaefer Slide no. 22The little extra
Thank you!
Details and references:
http://home.c2i.net/testing/testermidnighteng.pdf
Questions?
