Embed Size (px)
DESCRIPTION
By marshaling credible and persuasive evidence, influential testers answer three basic questions: How good is the product? What testing did you do? Why is the testing any good? Feeble evidence and the behaviors associated with it are common testing maladies and threats to your ability to tell a compelling story. What are the qualities of strong evidence, and the anti-patterns and risks of weak evidence? Griffin Jones presents the different qualities and types of evidence. We review threats to the credibility and persuasiveness of your work, such as: The danger of false-negative results reported as “pass – as expected”, “Lullaby Language” as an anti-pattern of Lean’s Genchi Genbutsu (go-and-see), the danger of obsessing on efficiency and how it biases the observer. Finally, we show how over-scripted procedures can superficially conceal fatal evidentiary flaws. Be intentional and critical about the type and quality of your evidence. Leave with the skills to recognize and evaluate the dangers and risks, strengths and weaknesses of the evidence you use for your testing story.
Citation preview
© 2013 Congruent Compliance LLC 1
What is Good Evidence?
May 2013
© 2013 Congruent Compliance LLC 2
Griffin JonesConsultantAgile / Testing / Regulatory
May 2013
This Presentation Covers …• Types of Evidence• Qualities of Evidence• Traps and Anti-Patterns
Why I Care, and Why You Should• Civil and Criminal Courts• Credible and Persuasive Story-of-Testing
© 2013 Congruent Compliance LLC 3May 2013
TYPES OF EVIDENCESources• FDA’s 21 CFR Part 11 – Electronic Records• FDA’s Good Manufacturing Practices• (Adjust for your context)
Records versus Testimony• Speaks for Itself • Cross-Examination
© 2013 Congruent Compliance LLC 4May 2013
QUALITIES …Clear• To Whom?
Objective• To What Degree?
© 2013 Congruent Compliance LLC 5May 2013
… QUALITIES …Authentic• Originals or Copies?• Resistant to Falsification?
Integrity• Protected from Damage / Deterioration?• Can not be Readily Repudiated?
© 2013 Congruent Compliance LLC 6May 2013
Readable• Human and Machine?
Changes• Do Not Obscure Previous Information?• Single line-out, initial, date, reason?
… QUALITIES …
© 2013 Congruent Compliance LLC 7May 2013
… QUALITIES …Attributable• Do we know the Specific Person?
Contemporary• Created At-The-Moment?• Are Not Recollections?
© 2013 Congruent Compliance LLC 8May 2013
… QUALITIES …Audit Trail• Is there a Chain-of-Custody?• Is there a Revision History?
Retrievable• Is this stored where I can Access it?
© 2013 Congruent Compliance LLC 9May 2013
… QUALITIES3rd Party Review / Reconstruction• Minimal Outside Assistance?• Reach the Same Results & Conclusions?
© 2013 Congruent Compliance LLC 10May 2013
TRAPS, THREATS AND ANTI-PATTERNS
Deficient Qualities • Leads to Feebleness
Feebleness• Leads to Unpersuasive Stories• Maybe a Systemic Risk smell?
© 2013 Congruent Compliance LLC 11May 2013
“PASS-AS EXPECTED”CHECKBOXES
Fear potential False-Positives• The danger of the Green-Bar• Falsification versus Confirmation
What you Intended to do• Not what you Actually Did
© 2013 Congruent Compliance LLC 12May 2013
Shuts-down Conversation• Discourages feedback • Promotes Shallow Agreement • Lulls your mind into a False Sense of
SecurityStyled Ceremony and Ritual• To provoke a behavior, not inform a
decision
LULLABY LANGUAGE
© 2013 Congruent Compliance LLC 13May 2013
… VERSUS LEAN’S GENCHI GENBUTU
“… It’s Lies! All Lies!”• Metrics and reports are just maps …• Get out of your office• Go observe where the work happens• Take your own measurements
© 2013 Congruent Compliance LLC 14May 2013
EFFICIENCY AND TIME PRESSURE
Biasing and Priming the Subject• Over-Scripting over-focuses attention• The “Moon-Walking Bear” problem• Target Lock-On
Unringing the Bell• The button story and pixel-drop out
© 2013 Congruent Compliance LLC 15May 2013
Recognize the Different Qualities• Clear, Objective, Authentic, Integrity,
Readable, Changes, Attributable, Contemporary, Audit Trail, Retrievable, Reconstructable
How is your evidence deficient?
SUMMARY … A
© 2013 Congruent Compliance LLC 16May 2013
Recognize the Traps• Pass-As-Expected• Lullaby Language• Moon-Walking BearGo Observe the Work
SUMMARY… B
© 2013 Congruent Compliance LLC 17May 2013
QUESTIONS AND STORIES
© 2013 Congruent Compliance LLC 18
Thank you for attending this session. Please fill out the evaluation form.
Griffin Jones
May 2013
