View
115
Download
0
Embed Size (px)
DESCRIPTION
EuroSTAR Software Testing Conference 2010 presentation onCritical Thinking for Testers by Susan Windsor. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
Citation preview
Critical Thinking for Testers
© Copyright 2010 Aqastra 1
Agenda
• Why Critical Thinking?
• Ways of Using Language
• Dishonest Tricks in Argument
• Debate
© Copyright 2010 Aqastra 2
Why Critical Thinking?
test
We can be deceived by most of
what we deal with
© Copyright 2010 Aqastra 4
People
• Users
• Stakeholders
• Project Managers
• Developers
• Ourselves
Artefacts
• Project Plans
• Requirements
• Designs
Infrastructure
• Environment
• Test Tools
• Test Data
Software
• Under Test
• Not Under Test
How we can be deceived - 1
• By People– They may be uninformed and/or may not be able to
express themselves well
– We cannot understand them easily
– They have agendas, prejudices and vested interests
– They push for unauthorised requirements/changes
– Our own prejudices and shortcomings get in the way
• By artefacts (requirements, designs, project plans)– Gaps in content
– Ambiguities, contradictions, defects
– Unsupported assertions
– Unjustified, unrealistic, unverified requirements.
© Copyright 2010 Aqastra 5
How we can be deceived - 2
• Infrastructure
– Environments may not be sound, stable, dedicated
– Tools may not work or not work the way we expect and mislead us
– Test data may not be appropriate, complete, controlled
• Software
– Software under test may reveal its secrets slowly
– Software not under test may hide or influence behaviour
– Software does not obey the laws of physics (or any reliable law for that matter)
© Copyright 2010 Aqastra 6
Critical thinking is…
• About having an open mind
• Recognising flaws in thinking and writing
• Challenging the arguments of others
• An enabler:
– to ensure we have good reasons to believe or do
what other people attempt to persuade us to
believe or do
• Etc.
© Copyright 2010 Aqastra 7
Critical thinking as a process
• Identifying other people’s positions, arguments and conclusions
• Evaluating the evidence for alternative points of view
• Weighing up opposing arguments and evidence fairly
• Being able to read between the lines, seeing behind surfaces, and identifying false or unfair assumptions
Recognising techniques used to make certain positions more appealing to others, such as false logic and persuasive devices
• Reflecting on issues in a structured way, bringing logic and insight to bear
• Drawing conclusions about whether arguments are valid and justifiable, based on good evidence and sensible assumptions
• Presenting a point of view in a structured, clear, well-reasoned way that convinces others.
© Copyright 2010 Aqastra 8
We need to be critical of…
• Testers are responsible for collecting, analysing and disseminating intelligence on software artefacts
• We don‟t make decisions, but others use that intelligence to do so
• We are very reliant on people, artefacts, infrastructure and software to do our job and our „genes‟ tell us to be distrustful
• So we need to be critical of:– The fallibility of our sources of information and guidance
– Our processes, approaches and tools
– Our project colleagues, our stakeholders and their leadership.
© Copyright 2010 Aqastra 9
What is „critical‟
• “Involving skilful judgment as to truth, merit;”
• “Characterized by careful, exact evaluation and judgment”
• “Characterized by thoroughness and a reference to principles, as becomes a critic”
• Of course, critical has the other, more popular meaning relating to defect detection, importance etc.
© Copyright 2010 Aqastra 10
References
Straight and Crooked Thinking, Thouless, Penguin
Critical Reasoning, Anne Thompson, Routledge
Critical Thinking, Bowell and Kemp, Routledge
Critical Thinking Skills, Cottrell, Palgrave
• A Rulebook for Arguments, Weston, Hackett
• Lines of Thought, Young, Oxford
• Critical Thinking, Paul and Elder, Pearson Prentice Hall
• Tools of Critical Thinking, Levy, Waveland Press
© Copyright 2010 Aqastra 11
Ways of Using Language
Many ways of using language -1
• Some examples:1. To give information about a fact – “The system has
been delivered”
2. To ask for information about a fact – “was the system delivered on time?”
3. To indicate an emotional attitude – “the system is useless”
4. To indicate how a word is used – “programmers are people who write programs”
5. To ask how a word is used – “what is a „test‟?”
6. To get someone to carry out an action- “please fix this bug”
© Copyright 2010 Aqastra13
Many ways of using language - 2
• But there are many more ways – for example:
7. To make poetry
8. To make a joke
9. To make a quarrel
10. To resolve a quarrel
11. To give a greeting
• But let‟s look more closely at types 1 and 3 on the previous slide.
© Copyright 2010 Aqastra 14
Factual and emotional use of words
• Example:
– “My dog is of mixed-breed”
– “Your dog is a mongrel”
• Factually, the two statements say the same thing, but the second infers a less satisfactory situation; disapproval
• This is a trivial example, but compare this with words used to describe mixed race – most are unacceptable to modern society.
© Copyright 2010 Aqastra 15
Emotional words are everywhere
• Define the adjective „firm‟:– “I am firm”
– “You are obstinate”
– “He is pig-headed”
• Essentially the words mean the same, but they convey an increasing level of disapproval
• Countless words exist that reflect this notion of popularity, approval or „goodness‟
• In political contexts, different words are used, depending on whose side you are on– Freedom-fighter – terrorist
– Spirit (our army) – mentality (theirs)
– Heroism (our army) – foolhardiness (the enemy)
© Copyright 2010 Aqastra 16
And in a business context?
• From a hardware company website 1/6/08
– “Today's communications consumers want
personalized and entertainment-related
services.… they have an insatiable need to
communicate…”
• But I don‟t want spam, crappy music, junk mail,
irritating web advertising, cold calls, intrusive
salesmen thank-you very much.
© Copyright 2010 Aqastra 17
And in a testing context?
• For Example, who is the largest provider of testing
services in the UK?
– Mission Assurance and Testing (Mission) is the UK's largest
systems quality assurance and testing solutions provider.
– SQS-UK is the UK's leading independent provider of
system and software testing and quality improvement
services
– Cognizant is a leading provider of information technology,
consulting and business process outsourcing services.
© Copyright 2010 Aqastra
Political oratory (rhetoric)
• Political debate/oratory is full of loaded terms:
– Progress, Liberty, Democracy, Fascist, Reactionary, Racist, Liberal are all loaded terms
– So how can we hope to understand and decide?
• Political oratory has its place
– E.g. Winston Churchill‟s many war speeches
– Where we are to make an important decision (and isn‟t that what democracy is?) it has no place, has it?
• But it is so pervasive, we had better be able to recognise it!
© Copyright 2010 Aqastra 19
Rhetoric
• Used to mean:– the study of the effective use of language
– the art of influencing the thought and conduct of an audience
• Common usage now is it refers to:– High-flown style
– Excessive use of verbal ornamentation
– Loud and confused and empty talk
• Rhetoric is deprecated by commentators, but it‟s persuasive power is irresistible to politicians, newspapers, salesmen, CEOs etc.
© Copyright 2010 Aqastra 20
From The People 1/6/08
CASH FOR BAD PARENTS:
Lousy parents could soon be
BRIBED to make sure their kids
go to schoolhttp://www.people.co.uk/news/tm_headline=cash-for-bad-
parents%26method=full%26objectid=20590312%26siteid=93463-name_page.html
© Copyright 2010 Aqastra 21
From the editorial - same paper
It’s Action – not ASBOsTen years ago anti-social behaviour orders were hailed as a panacea for all the yobbery, violence and teenage terror in our communities. A decade on they have had little impact. Youth crime is now a plague infecting every corner of Britain -fuelled by binge-drinking and the apathy of irresponsible parents.
Teenagers stabbed by teenagers, dads stamped to death by boozed-up gangs, pensioners scared witless by dope-smoking hoodies. Decent, law-abiding families are crying out for tough new measures to control feral youths who now see Asbos as a badge of honour.
So they should welcome the Action Plan being launched by the Government tomorrow.
At last PARENTS will be punished if they fail to control their yob kids - they will be fined or made to clean up the neighbourhoods terrorised by their offspring. But the failure of the Asbo has proved that in the poorest communities with the most deep-rooted social breakdown it is futile to use a stick without a carrot.
And it seems the Government has got the message.
Work and Pensions Minister James Purnell has been to America to study a project in New York's impoverished ghettos. The radical scheme PAYS parents to take better care of their children. There are cash incentives to ensure youngsters go to school, get regular health checks, even use the library.
Introducing such a scheme here is bound to outrage millions of hard-up parents who are already raising bright, healthy and respectful children.
Critics will call it bribery - rewarding feckless mothers and fathers with taxpayers' cash.
But think of parents like Margaret and Barry Mizzen, Sally and Colin Knox and John and Sylvia Lancaster who have lost their children at the hands of the teen thugs of Broken Britain.
They are the reason we MUST consider all options.
© Copyright 2010 Aqastra 22
Exercise: Spot the emotional language
Exercise
Emotive Terms
•
•
•
•
•
•
•
•
•
Non-Emotive Terms
•
•
•
•
•
•
•
•
•
© Copyright 2010 Aqastra 23
Dishonest Tricks in Argument
Sophistry
• From the Greek “sophist” meaning sage, wise
• Sophists:– “A group of professional fifth-century B.C. Greek
philosophers and teachers later characterized by Plato as superficial manipulators of rhetoric and dialectic”
– “an impostor in argument; a captious or fallacious reasoner” (captious=carping, nitpicking, niggling, picky, testy)
• Sophistry: subtle, tricky, superficially plausible, but generally fallacious method of reasoning
• Sophisticated is not necessarily a good thing.
© Copyright 2010 Aqastra 25
„All‟ and „Some‟
• Arguments such as „All Xs are Y‟ e.g.
– All pacifists are cowards
– All tobacco/oil/arms companies are evil
– All developers are poor testers
– All regression testing is better automated
– All bugs are bad
• To overthrow these assertions, we need only one contradictory example
• Some are easier than others – it depends on the context.
© Copyright 2010 Aqastra 26
Using „all Xs are Y‟ as a trap
• People use the argument to make a „stronger point‟– Counter is to point out exceptions
– BUT… maintain a moderate position
• Aggressive lawyers might trick a witness into saying „all Xs are Y‟ – which they then attack– Lawyer repeatedly makes extreme assertions (“you
lied, cheated, misled,…”)
– You say, “I always told the truth”
– Lawyer reveals you lied about your age on myspace.com – undermining your credibility.
© Copyright 2010 Aqastra 27
The “exception proves the rule”
• If you challenge an „all Xs are Y‟ assertion with exceptions the response might be “the exception proves the rule”
• Just point out that the exceptions do the opposite of proving a rule to be true
• Consider that prove really means test or challenge
• Respond that exceptions still confirm that a „rule‟ is false.
© Copyright 2010 Aqastra 28
Diversionary tactics
• Where an assertion of yours is challenged by an irrelevant assertion e.g.
• Example:
– “X is a thief” / “but he gives money to charity”
– “the software doesn‟t work” / “but we have a deadline to meet”
– “I need X weeks to test” / “but the plan says you finish in Y”
– “You said the system could be tested in X weeks” / “but you didn‟t give me the people”
© Copyright 2010 Aqastra 29
More diversionary tactics
• Personal attacks, slurs, aspersions– “he always underestimates/over-estimates/lies/
pretends/sucks up to bosses/”
– “he never…”, “he would say that, wouldn‟t he?”
• Refer to something your opponent is unaware of– “according to general systems theory, you are
correct”
– “Thank-you” (thinks: what is GST?)
– “BUT: more recent research has proven you completely WRONG”
– “Er….”
© Copyright 2010 Aqastra 30
Nitpicking as diversion
• Picking on minor points, asserting victory in an argument is another diversionary tactic– If your opponent says “that‟s 1% wrong” repeatedly, the memory
of the discussion may be that “it‟s wrong”
• “You can‟t be certain” could overrule your common-sense argument– “But you can‟t be certain the system will fail in production”
– “In fact, there are 379 bugs in the system, not 377, (you are WRONG) so the system should go live”
• In fact, these are actually irrelevant assertions.
© Copyright 2010 Aqastra 31
The “Lesser of two evils” Argument
• Sometimes a course of action is advocated as the „lesser of two evils‟
• But the two evils may not be related
– “we can‟t abolish war because more people die on the roads than in wars”
– “we should stop testing because complete testing is impossible”
– “we shouldn‟t worry about releasing now because the software industry has a bad reputation anyway”
© Copyright 2010 Aqastra 32
Lesser of two evils - 2
• A classic defence of people accused of parking
illegally or speeding”
– “The police shouldn‟t waste time spying on and
arresting good citizens. They should be arresting
serious criminals”
• Of course the counter-argument is:
– If Y is more evil than X then this is no reason for
not trying to eliminate X, but it is a reason for
fighting Y more energetically.
© Copyright 2010 Aqastra 33
The mean between two extremes
• Not obviously a trick, but watch out for the language used to state a position
• The argument is that a position is the „middle ground‟ between extremes – so must be more attractive than either
• But everything can be represented as a middle ground – it depends on what that ground is
• The political party that wins the middle ground, wins the election
• Perhaps their definition of ground won, really
© Copyright 2010 Aqastra 34
Hidden agendas and laziness
• We‟ll change our process (its impossible to
change our people or culture)
• We‟ll train our people (we can‟t ask
management to change)
• We‟ll buy tools (we can‟t ask our developers
or testers to work any harder)
• We‟ll outsource (hiring more people is
expensive, and getting smarter people is too).
© Copyright 2010 Aqastra 35
Debate
If a talented orator were to sell the idea that
2+2=5 how might they do it?
• 2+2=4– A backward, superstitious ideology! A corrupt idea like
flat-earth, slave-labour, communism, witch-burning and is condemned by all. Do you want to live in the dark age?
• 2+2=6– A dangerous view promoted by self-serving, ambitious and
dishonest bourgeois politniks and anarchists – Are you prepared to risk civilisation and democracy?
• 2+2=5– The one, true way. Five is God‟s chosen number and will
free mankind to achieve universal freedom and peace, and unfetter our ambition to understand our universe.
© Copyright 2010 Aqastra 37
If a talented salesman were to sell you
consultancy how might they do it?
• Structured Testing– A backward, superstitious ideology! A corrupt idea like
flat-earth, slave-labour, communism, witch-burning and is condemned by all. Do you want to live in the dark age?
• Exploratory Testing– A dangerous view promoted by self-serving, ambitious and
dishonest bourgeois politniks and anarchists – Are you prepared to risk civilisation and democracy?
• “The five-point method – or “Best Practice!”– The one, true way. Five is God‟s chosen number and will
free mankind to achieve universal freedom and peace, and unfetter our ambition to understand our universe.
© Copyright 2010 Aqastra 38
Debate
Invalid Statement
•
•
•
•
•
•
•
•
•
Trick being used
•
•
•
•
•
•
•
•
•
© Copyright 2010 Aqastra 39
Some assertions to debate
• Some examples…
1. “Testing must be planned, documented and scripted and systematically executed with disciplined record-keeping”
2. “Test design must use formal test design techniques to be effective and efficient”
3. “Testers need perfect requirements to be effective”
4. “Testers should make the acceptance decision”
5. “Testers have no role in the acceptance decision”
6. “Automation should be the goal of all testers”
7. …
• Care to add your own?
© Copyright 2010 Aqastra 40
Close
Any Questions?