Upload
malcolm-clarke
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Trends in the MarketplaceTesters will have to change – but how?Paul GerrardGerrard Consulting1 Old Forge CloseMaidenheadBerkshireSL6 2RD UK
e: [email protected]: http://gerrardconsulting.comt: 01628 639173
Slide 1 Assurance with Intelligence
It’s the Benefits, Stupid! Automation frameworks Test Process Improvement is a Waste
of Time Software Success Improvement What Makes a Good Tester? Recommendations
Agenda
Slide 3 Assurance with Intelligence
It’s the Benefits, Stupid
Slide 4 Assurance with Intelligence
Won’t bore you with yet another survey of IT projects that fail - take it for granted, most do
But why? We can trace most failures to:- Bad decisions- Decisions that were made too late- Decisions that were not made at all
We suggest this happens because stakeholders and project managers lack the right information at the right time
We’ll call this information Project Intelligence
Why projects fail
The frame of reference for making those decisions is often wrong too!
Slide 5 Assurance with Intelligence
Four-eyed plans
IncreasinglyInaccurate
IT- focusedInitial
PlansThe plan is a model of the project. The real project consists of people, organisation,
goals and risks.Slide 6 Assurance with Intelligence
The people like to see the broad range of issues covered by the politicians
Makes politicians feel important But it’s all froth
Team Poster
It’s the economy, stupid!benefits
Ultimately, our customers are only
interested in the benefits of ITSlide 7 Assurance with Intelligence
Number of people in IT will drop by 15% by 2010
60% percent of technology professionals will move to more business-focused roles, concentrating on the use of IT and processes rather than IT delivery
In 2010, the typical IT department in a large company will be at least one third smaller than it was in 2000
Departments within business will take on the traditional roles of IT.
Gartner predictions
Slide 8 Assurance with Intelligence
Increasingly, businesses will focus on benefits and will take control of projects or IT completely
Projects involving IT will no longer be dominated or managed by IT
The disciplines of Benefits Realisation, Goal-Directed Project Management and Project Intelligence will become mainstream
Most testers will work for (or come from?) business
(Some) Test Managers become PI Managers.
Here’s my interpretation…
Slide 9 Assurance with Intelligence
Automation FrameworksI am very grateful to Susan
Windsor ([email protected]) for the use of some of her material
Slide 10 Assurance with Intelligence
Automation frameworks and more complex business requirements
Agile development methods mean developers undertake more unit and component testing
Growth of outsourced testing to different geographies => greater competition for the roles
Testing is in demand, and solutions reduce the number of functional testers required
Is the role of the functional tester (who is neither technical nor business specialist) dead?
Slide 11 Assurance with Intelligence
Focus on technology rather than business needs
80% of functional testing still manual 60% to 70% of automation tools used for non-
functional testing Typically, traditional functional automation
stops at 100 scripts, regardless of test coverage requirement
Critical factors- Cost of implementation and maintenance prohibitive- Insufficient and expensive skills required - Inability to asset share over different technologies
Functional test automation is broken!
Source: Paul Herzlich, OVUM UKSlide 12 Assurance with Intelligence
Home grown frameworks built within organisations to meet business demands
Niche suppliers provide frameworks - try Google:- 34,400 exact matches for “test automation
framework”
- A few are now mature
- Latest review by Paul Herzlich (OVUM analyst) Market Leaders such as Mercury
developing Business Process Tester (BPT)
Business analysts already using them, and use will grow
This seems to be the tools industry direction now
Slide 13 Assurance with Intelligence
Slide 14
Manual Test Execution
& Documentation
UI AutomatedTest Execution
Tools
Test Results
Non UI ComponentTest Execution
Harness
Requirements
Test Management
Governance
Project ManagementAnalysis & Design
Test Case Definition – Automation Framework
Test Case Documentation
Test RequirementsSign Off
Where frameworks fit
14 Assurance with Intelligence
Test Process Improvement is a Waste of Time
Slide 15 Assurance with Intelligence
I want to improve my (insert any activity
here) _______ people improvement _______ organisation improvement _______ process improvement
How to improve…
Changing people (like me) and organisation (like my company) is so hard – let’s not even think about it
Slide 16 Assurance with Intelligence
There are no “practice” Olympics to determine the best
There is no consensus about which practices are best, unless consensus means “people I respect also say they like it”
There are practices that are more likely to be considered good and useful than others, within a certain community and assuming a certain context
Good practice is not a matter of popularity. It’s a matter of skill and context.
The delusion of ‘best practice’
Derived from “No Best Practices”, James Bach, www.satisfice.comSlide 17 Assurance with Intelligence
Google search- “CMM” – 12,100,000- “CMM Training” – 12,200- “CMM improves quality” – 4
A recent client…- CMM level 3 and proud of it (chaotic, hero culture)- Hired us to assess their overall s/w process and
make recommendations (quality, time to deliver is slipping)
- 40+ recommendations, only 7 adopted – they couldn’t change
- How on earth did they get through the CMM 3 audit?
The delusion of process models(e.g. CMM)
Slide 18 Assurance with Intelligence
Slide 19 Assurance with Intelligence
People like simple models:- levels of maturity, stepping stones,
checklists, roadmaps and outside support for credibility
But life is much more complicated, unfortunately
“Things should be made as simple as possible,
but no simpler”
Albert Einstein
But process models make improvement simple don’t they?
Slide 20 Assurance with Intelligence
A big problem with process is it becomes all encompassing
Process folk sell process and cast all things in terms of it- They ignore that people who are smart- Smart people succeed regardless of process, not because of it
It could be argued, that less smart people need process- (By less smart, we're talking about people who need so much
structure and enforced discipline they can only operate in the military, or in prison probably)
Is our industry really staffed by such people? Do we really want production-line workers? Do YOU really want to be a production-line worker?
People need process?
Slide 21 Assurance with Intelligence
“I believe that a scientist looking at nonscientific problems is just as dumb as the next guy”
“It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with
experiment, it's wrong”
Richard P. Feynman
Physics quotes…
Slide 22 Assurance with Intelligence
“I believe that a process consultant looking at non-process problems is just as dumb as the next guy”
“It doesn't matter how beautiful your process model is, it doesn't matter how smart you are. If it doesn't agree
with reality, it's wrong”
“Test process improvement is tinkering”
Paul Gerrard ;-)
You can quote me if you want…
Slide 23 Assurance with Intelligence
Software Success Improvement
Slide 24 Assurance with Intelligence
From this…
CurrentMaturity
FutureMaturity
ProcessChange
CurrentCapability
FutureCapability
Acts ofFaith
Better Capability = better, faster, cheaper
Perceived Results Chain
Slide 25 Assurance with Intelligence
To this…
CurrentConstraints/
Problems
ChangedAspirations
CurrentCapability
FutureCapability
Acts ofChange
Better Capability = better, faster, cheaper
Actual Results Chain
Slide 26 Assurance with Intelligence
Constraints are fixed headcount, budget, timescales, quality of requirements, contracts etc.
Problems are “testing takes too long; too expensive; can’t hire testers; bugs get through” etc. etc.…
Aspirations:- Personal: personal development, fulfilment, motivation
- Organisational: hero culture to team culture, outsourced, higher consistency, predictability
Acts of change are…
Constraints, problems and aspirations
Slide 27 Assurance with Intelligence
Changes in behaviour to address specific problems (time, cost, quality etc.)
Targeted personal and team development
Infrastructure change (process, techniques, tools, environments) to support the changes
Managed Transition…
Acts of change – focused on constraints, problems and aspirations
Slide 28 Assurance with Intelligence
Current behaviour assessed in the context of current problems, goals and constraints
Aspirations drive the definition of goals People in the job define and consent to the
required changes in behaviour People supported by- Personal/team development plans- Infrastructure investment (process, technology,
tools, environment) specific to the change Transitions are managed, not assumed.
Principles of change
Slide 29 Assurance with Intelligence
Establish a sense of urgency Create a guiding coalition Develop a vision and strategy Communicate the change vision Empower broad-based action Generate short term wins Consolidate gains, produce more
change Anchor new approaches in the culture.
Eight stage change process (Kotter)
Derived from “Leading Change”, John Kotter, www.johnkotter.comSlide 30 Assurance with Intelligence
Mission Coalition Vision Communication Action Wins Consolidation Anchoring
Eight stage change process (after Kotter)
Changes identified here
This is where your ‘process model’ comes into play
Slide 31 Assurance with Intelligence
What Makes a Good Tester?
Slide 32 Assurance with Intelligence
PedanticSceptic
Nitpicker
A perfect tester?
Slide 33 Assurance with Intelligence
Attributes of a good tester?
intelligent
persistent
systematic
pedantic
thoroughsceptical
logical
curious
articulate
conscientious
imaginative
accurateobservant
thick-skinned
assertive
redoubtablestubborn
Slide 34 Assurance with Intelligence
Attributes of a good tester?
intelligent
persistent
systematic
pedantic
thoroughsceptical
logical
curious
articulate
conscientious
imaginative
accurateobservant
thick-skinned
assertive
redoubtablestubborn
Personal intelligence and skill
Personality to cope with your
environment
Slide 35 Assurance with Intelligence
Intelligence - are they smart enough? Thinking skills- Approach to problem solving; ability to reason
Interpersonal skills- Communication, assertiveness, conflict handling,
survival, team skills etc. etc. Testing skills- Theory (ISEB/ISTQB Etc.)- Practical (hands on testing)- Technical (technology, business domain etc.)
Plus… Personality- Many well-known attributes are usually required.
Tester selection criteria
Slide 36 Assurance with Intelligence
Tester skills (and training budget?)
Testing theory (ISEB/ISTQB etc.)- see www.bcs.org.uk/iseb
- www.istqb.org Thinking skills- verbal reasoning, numerical/abstract
reasoning, fault diagnosis, accuracy Testing Practice (Testing Case Studies)- hands-on practical test activity
Interpersonal skills- communication, assertiveness,
conflict handling, survival, team skills
20%
40%
20%
20%
Slide 37 Assurance with Intelligence
Summary and Discussion
Slide 38 Assurance with Intelligence
Take a closer look at benefits realization, results-based management, performance management, project intelligence
Become more business-oriented- Be ready to talk their language – benefits and
risk
- Align your test activities with the need to provide information on benefits and risk to them
- Be prepared to be managed by your customer!
It’s the benefits, Stupid!
Slide 39 Assurance with Intelligence
Take a close look at frameworks What is your organisation doing about it? Understand how Frameworks support
testing so you have a view, because you will be asked
If you are functional tester – evolve or… Look to enhance your skills…- Become an automation/framework specialist- Move towards management- Take on non-functional skills- Get closer to your business/customers
Automation frameworks
Slide 40 Assurance with Intelligence
Don’t take on Test Process Improvement projects- Too many problems in test are caused elsewhere- Testers shouldn’t compensate for other people’s failures- Whole-process is in scope for change - or nothing!
Ask why things are the way they are Focus on people, culture and organisation “The art of the possible” – get your suggestions
from your practitioners, not a book Follow the 8 step change process (or another
transition management method – there are many) Be realistic about benefits, and be prepared to
measure them
Software Success Improvement
Slide 41 Assurance with Intelligence
Functional testing skills are a commodity – so move on
Have a vision of where you want to be… manager, specialist, consultant, business expert…
Following technology changes can be lucrative, but is a never-ending journey
Broaden your horizon – thinking, interpersonal, business, management, project management skills
Create a development plan – training yes, but look out for practical hands-on, not theory
Most learning takes place on the job so choose jobs wisely
Find a good coach.
Developing, as a tester
Slide 42 Assurance with Intelligence
gerrardconsulting.com
uktmf.comriskbasedtesting.co
m
Good luck in your career!
Trends in the MarketplaceTesters will have to change – but how?
Slide 43 Assurance with Intelligence