Agile testing (n)

Agile Testing

Agile Manifesto

1. people and communication overprocesses and tools2. working software over documentation3. customer collaboration over contractnegotiation4. responding to change over following aplan

Roles and activities on agile team"Testers have a foot in each world, understanding the customer viewpoint as well as the complexities of the technical implementation."

Customer Team● business experts● product owners● domain experts● product managers● business analysts● subject matter

experts● testers

Developer Team● programmers● system

administrators● architects● security specialists● database

administrators● technical writers● testers

"Testing"

● Unit and component testing● Functional testing● System testing● Load testing● Performance testing● Security testing● Stress testing● Usability testing● Exploratory testing● End-to-end testing● User acceptance testing

Ten principles for AgileTesters

1. Provide continuous feedback2. Deliver value to the customer3. Enable face-to-face communication4. Have courage5. Keep it simple6. Practice continuous improvement7. Respond to change8. Self-organize9. Focus on people

10. Enjoy

What's the difference...?Traditional testing:

● long iterations, "mini-waterfall"-trap● testers are not involved until release planning● testers have requirement documentation, they write long

test plans, wait for● the developers and then execute tests -> "fast and

furious testing"● if anything goes wrong it causes rushed testing phase

or/and release is● usually postponed● traditionally testers are considered as gatekeepers of

quality -> but they● have no control over code quality or developer testing● separated QA team

What's the difference...?Agile testing:

● short iterations, frequent, small releases● testers are involved as soon as project planning begins● testers have access to all knowledge and help from the

team they need,● they can focus on big picture and small tasks

simultaneously● if anything goes wrong it can be discovered and fixed in

time● testable design and code● team understands that quality is more important than

quantity or speed● independent, but not separated QA team (everyone is a

tester)

So,What's the difference...?

"One of the biggest differences in agile development versus traditional development is the agile 'whole-team' approach. With agile, it's not only the testers or a quality assurance team who feel responsible for quality."● no more quality police mentality - quality is not only the testers' responsibility● whole-team approach; in agile teams everyone gets test-infected -> code is designed for testability● tester team is integrated, there's no "QA-time" (no time to cut when deadline is close, no rushed QA-phase)● every team member has equal value to the team● self-organizing team, continuous improvement

Agile testing QuadrantsQuadrant 1: this represents

test-driven development,

which is a core agile

development practice

Quadrant 2: business-facing

tests, these define external

quality and the features that

the customer wants

Quadrant 3: business-facing

tests to critique the product

(manual)

Quadrant 4: technology-facing

tests

Quadrant 1 - Test-driven development TDD

● Unit tests● Component tests● internal quality isn't negotiated with the customer, it's defined by the programmers● programmer tests make lots of testing activities easier to accomplish● TDD means programmers are always consciously making testable code and it provides immediate feedback● each unit test is independent and tests one dimension at a time -> when a unit test fails, it's easy to identify where and what to fix ● TDD is a design activity, not only testing

Quadrant 2 - External quality

● Functional tests● Examples● Story tests● Prototypes● Simulations● business-facing tests address business requirements based on

examples● quadrant 2 tests define and verify external quality● these are written for each story before coding starts, they help the

team● understand what code to write● generally written in executable format, automated, so that team

members● can run the tests as often as they like● these become later automated regression tests as well

Quadrant 3 - tests that critique the product

● Exploratory testing● Scenarios● Usability testing● UAT (User Acceptance Testing)● Alpha/Beta testing● critiquing or evaluating the product● difficult to automate because these are based on human intellect,● experience and instinct● this manual testing is not enough to produce high-quality software (without● automation)● trying to recreate actual experiences of the end users● "soap opera testing" - "what's the worst thing that can happen, and how did● it happen?"

Quadrant 4 - Performance, Security and “-ility” testing

● Security● Maintainability● Interoperability● Compatibility● Reliability● Installability● technology-facing tests that critique the product● nonfunctional requirements, configuratiion issues● many of these tasks require specialized knowledge

Why are all of these important?

● help to get emerge technical debt (shortcuts, quick fixes and skipped automated tests making the code harder to maintain)

● responsibility is shared between the quadrants

● quality is not only the testers responsibility● "When developers don't have an automated

suite of tests acting as a safety net, they may start viewing the testers themselves as a safety net."

Why manual testing isn't enough

● it gets boring very quickly, it's repetitive, so it's easy to overlook simple bugs

● manually testing a number of different scenarios can take a lot of time● especially if tester is keying inputs into a UI -> only a limited number of

scenarios may be tested, and important defects can be missed● automated tests can help with consistency across the application● automated regression tests are the safety net, not the QA person; without

them manual regression testing will grow in scope and eventually may simply be ignored

● automated test frees the QA team for more important work, such as exploratory testing

● "when post-development testing time is occupied with finding and fixing bugs that could have been detected by programmer tests, there's no time to find the serious issues"

How to be an agile tester

while release planning● avoid big design up front, but understand itcorrectly● "create" high-level test plan● try to identify risk, impact● think about 'ripple effects' as well● help correctly sizing stories with correctquestions

"Intuition is something that a machine cannot learn."

● the tester needs to keep the big picture in mind to create test cases so they can ask uncomfortable questions○ how much effort does it require to create a test environment?○ what's the worst scenario that can happen?○ what if the end user doesn't understand the application?○ what's the difference between test and production environment? how

will it affect the performance?○ what kind of test data should we use and where do we get it?○ etc.

● they always have more different viewpoints (developer-customer team)● they can identify 'thin slice' and 'critical path' to help prioritize stories● they give quick feedback to developers, helping them to create quality

product● UX/UI testing is the most important after quality code -> "if a user has a

choice of applications or websites, and has a bad experience, they likely won't user your application again"


while story sizing:● everyone's goal is to deliver real value in each iteration -

> help identify one path through the functionality to code and test first, then help adding more features after the first critical path works

● identify 'small testable chunks' -> testable doesn't necessarily means GUI

● always think about the big picture, know how each story affects the whole system (if there's a bug in the inventory code, what's the worse thing that can happen?) -> help identify impact


while test planning:● the biggest benefit of test planning is planning itself● wait for iteration planning to create detailed tests● consider automation and what is needed for test

environment● test planning is a risk mitigation strategy● consider test infrastructure● collect test data, real examples and real use-cases● choose the right testing method, considering:

○ what's in scope○ what are the impacts, is there any unusual risk○ do you need test reports (for the customer team? for

your team?)

Test plan alternatives

Lightweight plans● plan should not cover every eventuality or every story● not meant to address traceability● it's a tool to help thinking about testing risks Test Matrix● it's a list of functionality down the side and test conditions across the top● high-level test matrix -> can be used to show to customer team what has been tested already and what's left● more detailed matrix -> can be used to show to developer team what's planned for testing and track the progress

Test plan alternativesTest spreadsheet● it's more like a functionality list and impact analysis in a spreadsheet format, similar to test matrix but on more tabs Whiteboard● sometimes it's enough to list the risks and assumptions on a whiteboard or index cards or a wiki page Automated test list● provided by tools (at EU Edge: TestLink) ● it's more like a detailed test plan● it's only available after test cases are written● no added value


while iteration planning:● think about how you can test the stories● help the programmers to design the application to be effectively testable● "when testability is an issue, make it the team's problem to solve"● work closely with the customers● think about 'big picture' and detailed tests as well● high-level tests should cover the main functionality behind the story● think both about the desired and undesired behaviour● 'What's important as you begin the iteration is that you quickly learn the

basic requirement for each story and express them in context in a way that works for the whole team."

● review the test cases with the customers and developers as well● well-written tests form the core of the application's documentation


while development:● start simple -> create a happy path to show the core

functionality● add complexity -> as soon as happy path works start

adding more test cases● remember the purpose of the tests: they should provide

examples that tell the programmers what code to write● use risk analysis to help prioritize testing● identify variations● use the 'power of three' (disagreement -> programmer +

QA person + Product Owner)● focus on one story, try to get it 'done'


while development:● as soon as testable chunks are ready take time to

explore functionality● if you find 'nice-to-have' stories consult with the

customer, if there's time to add it in the iteration and it has business value, go ahead (these are cheaper now), but don't jeopardize other stories by adding 'blind' that doesn't have big ROI (ROI = Return on investment)

● technology-facing tests to critique the product ("-ility testing") are often best done during coding

● if the team doesn't have automated tests, everyone should plan time to address manual regression tests and manually testing new features


while dealing with bugs:● bug or feature? -> in the end, does it really matter if it is

a bug or a feature if it needs to be fixed?● bugs -> technical debt: the longer a bug stays in the

system and goes undetected, the greater is the impact● sometimes a bug is really a missed requirement and

needs to be handled as a story; estimated and prioritized for a future iteration

● set rules like "the number of bugs should never get higher than ten at any one time"


while delivery:● the goal is to deliver value to the business in a timely manner● plan as much time for the end game as you need (it varies with the

maturity of the team and the size of the application)● use this end-game time to do a final exploratory testing● step back and look at the whole system; do some end-to-end scenarios ● find a way to guarantee that all changes made in the test environments will

be done in the staging and production environments during release● 'customers have a habit of using the application in weird and wonderful

ways, and the data is not always as clean as we would like' - everyting needs to be tested again with real data in a real environment

● try to avoid high risk -> release in smaller stages or use a system property to turn on new features and use the old as long as the new one is safe enough

● communicate, have reminders

Summary

Success factor 1: Whole-team approach● you have a large variety of skill sets and experience levels● when testing is a team priority, the team designs testable code● remember that quality, not speed, is the goal of agile development● use the "power of three"

Success factor 2: Adopt an agile testing mindset● "Ten principles for agile testers", no more "Quality police mentality"● be courageous● focus on delivering value● be flexible in responding to change● drive to continually find better ways to work● experiment with new practices, tools, and techniques

Summary

Success factor 3: Automate regression testing● if you spend all your time doing manual regression testing, you'll never● have time for the important exploratory testing● remember to start simply

Success factor 4: Provide and obtain feedback● feedback is a core agile value● the short iterations are designed to provide constant feedback● one of the most valuable skills to learn is how to ask for feedback

○ ask the programmers if they get enough information○ ask the customers if they feel their quality criteria are being met

Summary

Success factor 5: Build a foundation of core practices● "you can't test quality into the product"● every agile team needs source code management and continuous

integration -> you can't test effectively if you don't know exactly what you're testing

● you can't test effectively without a test environment that you can control ● manage technical debt; don't let time pressure get you into hacking, or

"quickfixing" - good test coverage, automated regression tests can help● use small-scaled stories -> develop and test step-by-step to keep quality

"Agile practices were designed to complement each other. Take time tounderstand the purpose of each one, consider what is needed to take fulladvantage of each practice, and make thoughtful decisions about what worksfor your team.”

Summary

Success factor 6: Collaborate with customers● use the "Power of three"● encourage direct communication between customers and developers● help customers clarify requirements, desired behaviour with concrete

examplesSuccess factor 7: Look at the big picture● testers tend to look at the big picture usually from a customer point of view● developers have to focus on the story they're working at● use agile testing quadrants as a guide to help you plan testing that will

cover all angles● make your test environments as similar as possible to production (test data

as well)

Technology

Agile testing (n)