View
251
Download
0
Category
Tags:
Preview:
DESCRIPTION
Presented on Testwarez 2012 (the biggest Polish conference about testing and quality). If you are interested, please read my article on the same topic: http://pl.coremag.eu/fileadmin/user_upload/redaktion/coremag_pl/Downloads/Core_magazineTestWarez_2012.pdf
Citation preview
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Between Scrum and Kanban – define test process
Zbyszek Moćkun
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
1. Methodologies
2. Problems to overcome
3. Workflows
4. Issue management
5. Taskboard
6. Exploratory testing
7. Metrics
Agenda
Client/partner logo
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Few words about Agile
Methodologies
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Methodologies axis
(120+)
RUP (13)
XP
(9)
Scrum
(3)
Kanban
(0)
Do whatever
Source: Kanban vs Scrum – how to make the best of both (Henrik Kniberg)
Prescriptive Adaptive
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Kanban
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
• Visualize the workflow
• Limit WIP
• Measure the lead time
• Manage the lenght of the queue
• No fixed timeboxes iteration
• You can choose when:− Planning
− Release
− Retrospective
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Scrum-inspired
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
• Do not limit yourself to one tool
• Mix and match tools as you need
• Dynamic changes in projects (suite your process to actual needs)
• Retrospective – continuous improvements
• Agile manifesto (Individuals and interactions over processes and tools)
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Problems that we had to overcome
Problems
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Problems to be overcame
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
• Project status on Taskboard− Status of all tasks visualized on Taskboard
− No need of additional reports
− Visualize QA activities
• Exploratory testing management
• Ensure that all tasks are tested
• Metrics: quality, testing velocity, ...
• One process for different methodologies: Scrum, Kanban, mixed
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Ensure a QA place in workflow
Workflows
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
The meaning of „Done”
• When do we finish our work?
• What is the latest action?
• Who is responsible for the transfer to the state done?
• What is the client role?
• Visualize the done state
• Done for Scrum vs Done for Kanban
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
QA States
There are no QA states in standard workflows • assumption - close action is for client (sign off)• Green states
Add qa states to your workflow• QA queue, Testing in progress, Tested• Green plus white states
In Progress Resolved Testing in
Progress Tested Closed
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Keep the standards
• Do not change the workflow parts that customer use
In Progress Resolved Testing in
Progress Tested Closed
In Progress Implemented Testing in Progress Resolved Closed
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Different tasks, different needs
• Client oriented – extended workflow− QA Engineer is not always a client− Client has to accept task solution
• Single role – simple workflow− Division of task on the parts by developer
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Make it simple
Issues management
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Issues by software lifecycle
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Issues tracking
• Regression – Standard Issue Type
• Acceptance – Sub-Task Issue Type
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Story as a container
• All issues connected with story are reported as sub task (bug, improvement, question, task)
• All data in one place (historical data too)• All sub tasks have to be implemented to send story to QA• Story accepted only if no sub-issues left
Story
Sub improvment Sub bug Sub bug Sub bug Subtask
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Dev – QA cooperation
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Quality levels
• Are all issues have to be resolved?
• Can we fix some of them later?
• Flexible or restricted process?
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Retest: whole story or sub-issues only
• Switching between contexts require very good time managment skills
• Helicopter view• Do not duplicate data (JIRA <->TCM)• All failed issues attached to story• QA Engineer can make decision
− Retest whole story− Only failed issues− Chosen parts (change propagation)− Decision made on available data (fixed issues, introduced
changes, developer output)
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Integrate Workflow with Taskboard
Taskboard
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Workflow on Taskboard
• One tool, all information, no additional reports
• Only the Standard Issue Type visible− Acceptance issues are hidden
• Sorted by rank (priority)
• Rejected issues again in To Do queue
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Why scripted aproach doesn’t work
Exploratory testing
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Scripted testing doesn’t work
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
• Status in two tools: JIRA and TCM
• Retesting dilemma
• Test Script creation in advance − boring and not efficient
− Can’t write scripts for 100% coverage
− Requirements/Acceptance criteria updates
− No time for this phase (parallel testing)
• Acceptance tests
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Scripting doesn’t work - reports
• Scripted testing report
• Not all founded issues are against test case
• Test case number means nothing (James Bach)
• New test scripts added almost each day
• Old onces are updated
phase All Executed passed failed
Test design phase 80 0 0 0
First day 90 (4 updated) 20 16 4
Second day 95 (10) 75 65 10
Third day 120 (14) 75 73 2
Fourth day 125 (16) 110 98 12
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Exploratory testing
• Efficiency− Concentrate on functionality
− Time management (quality level)
− Critical issues are founded earlier
− Changes are welcome
− No test script creation in advance phase
− Test scripts as output (for regression tests purpose)
• Learning phase− Simulate users (usability issues)
• Move responsibility on tester− Retesting dilemmas
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Improve your ET skills
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Find what you need
Metrics
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Metric by actions
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
One measurement, two metrics
• Acceptance rate− Acceptance actions/all qa actions
− All QA actions = accept or reject
− Measure quality of software send to QA
• QA velocity− Sum off all actions
− Divide by number of QA Engineers assigned to project
− QA velocity depend on software quality that comes to qa
− Why testing takes so long? (Kaner’s blog)
Recommended