#TestFirst @KevinDunneQARobust testing platform purpose-built to help agile teams create
higher quality software
ABOUT QASYMPHONY
#TestFirst @KevinDunneQA
Creating Visibility and Speed with Automation and Test ManagementThursday, March 24th at 2pm ET
– Guest Speaker: Joe Colantonio
Joe Colantonio is a test automation architect for a large Fortune 100 company with more than 15 + years of test automation and performance testing experience. Joe is also the founder of the popular test automation blog, joecolantonio.com and the host of TestTalks, a podcast dedicated to all things test automation related. He is also the author of the highly rated book The UFT API Testing Manifesto – A step-by-step hands-on testing guide for the masses.
UPCOMING WEBINAR
#TestFirst @KevinDunneQA
QUALITY JAM 2016 – APRIL 13TH IN ATLANTA
Learn more at qualityjam.com
Speakers Include:
• Scott Berkun, Best Selling Author
• Keith Klain, Software Testing Thought Leader
• Michael Cooper, Chief Quality Officer, Healthcare IT Leaders
• Adam Satterfield, VP of QA, BetterCloud
#TestFirst @KevinDunneQA
• Early bird pricing of $100. Pricing will go up to $200 on March 1st.
• For a limited time, you can get an additional $20 off using the promo code QASVIP when you check out.
• For the first 5 people who send an email to [email protected], you will get a complimentary pass to the event.
QUALITY JAM 2016
#TestFirst @KevinDunneQA
• This webinar will be recorded and available on demand next week
• If you have a question, type it into the Q&A module on the left. We will answer as many as we can towards the end of the webinar
• Join the conversation on Twitter using the hash tag #TestFirst
• At the end of the webinar, you will be asked to take a short survey
HOUSEKEEPING
#TestFirst @KevinDunneQA
Kevin Dunne, VP of Strategy and Business Development, QASymphony
Kevin Dunne is the VP of Strategy and Business Development at QASymphony. In this role, Kevin focuses on innovation, thought leadership and new global business opportunities. As one of the first employees at QASymphony, Kevin has seen many facets of the business working in sales, customer support, marketing, and product management.
Kevin comes to QASymphony from Deloitte, where he managed testing on large government and Fortune 500 engagements delivering ERP implementations and custom software development. Kevin holds a Bachelor of Science degree from Vanderbilt University.
OUR PRESENTER
#TestFirst @KevinDunneQA
What is TDD?
Why are we hearing about it now?
How is it different?
Why use TDD?
How will it benefit my organization?
What drawbacks if any exist?
How to Implement TDD?
How do I prepare my people?
What’s the new process?
Do I need new tools?
AGENDA
#TestFirst @KevinDunneQA
Past development cycles often modeled the Rational Unified Process:
Source: http://www.psa-software.com/_img/_knowledge_center/rup.jpg
WHERE WE CAME FROM
#TestFirst @KevinDunneQA
We would have obviously chosen a more efficient process, but we were constrained by many limitations, including:
WHY WE CHOSE IT
Environment Creation
Code MergesOn-Premise Prevalence
Desktop Focus
Lack of Collaboration
Off-Shore Development
#TestFirst @KevinDunneQA
Many of our priori limitations have been replaced, based on macro trends around technology and industry:
WHAT’S CHANGED
• Containers have simplified the process dramaticallyEnvironment Creation
• Git has replaced Subversion as the industry standardCode Merges
• Cloud adoption is at an all time high, increased securityOn-Premise Prevalence
• Prevalence of Web, Mobile, Internet of thingsDesktop Focus
• Increase in teamwork, chat and collaboration technologyLack of Collaboration
• Shifts towards rural sourcing, onshoring of laborOff-Shore Development
#TestFirst @KevinDunneQA
Now that we have freed ourselves of past limitations, the process has been shifted to one that aligns more with our needs:
Traditional Approach
Test-First Approach
HOW THE PROCESS HAS ADAPTED
Design Requirements Code Test Deploy
Design (Automated)
TestCode Refactor Deploy
#TestFirst @KevinDunneQA
Test-First methodologies were coined “Test Driven Development”. Less technically focused versions called Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD) also emerged:
TDD VS ATDD VS BDD
Test Driven Development (TDD)
Behavior Driven Development BDD
Acceptance Test Driven Development
(ATDD)
Unit Test Driven Development
(“Technical TDD”)
#TestFirst @KevinDunneQA
ATDD and BDD are similar in that they both try to make TDD more accessible to business users. The major functional difference comes down to how the tests are structured:
"I think this definition leaves out a key piece, we are focusing on collaboration and learning. Having worked on a project that was using 'ATDD', in 2005 I think, we had the same goals then as BDD
without the Given When Then language.“ - Wes Williams
WHAT’S THE DIFFERENCE?
#TestFirst @KevinDunneQA
ATDD/BDD offer benefits over more Unit Focused/Technical TDD, but also has its drawbacks:
PROS AND CONS
Pros
• Increased understanding of tests from business stakeholders
• Increased collaboration early in the cycle
• More focus on customer and business needs
• Higher involvement of business in development and quality
Cons
• Addition of more tooling in the development/delivery chain
• Greater time spent defining tests and specifications
• Demands stronger contributors in requirements, dev, test
• Often increases the automation needs in an organization
#TestFirst @KevinDunneQA
Test Driven Development brings several major benefits to organization, most notably:
1. Move Testing Up Front – prevents having to rush testing at the end of the cycle
2. Bake in Automation from Day 1 – protects against getting behind with test and automation coverage
3. Build More Testable Software – requires developers to think about testability, and create more robust software
4. Push to Customers When Ready – allows you to push software to customers just in time, as it is developed
HOW CAN TDD HELP US?
#TestFirst @KevinDunneQA
Moving Testing Up Front removes the risk of having to make compromises at the end of the cycle on quality or on-time delivery:
Traditional Development Timeline
Ends on: Day 1 Day 3 Day 14 Day 20 Day 21
There is risk in this process that any process, typically Code, will run over and either squeeze development, or push release dates. TDD removes it!
MOVE TESTING UP FRONT
Design Requirements Code Test Deploy
#TestFirst @KevinDunneQA
The cost of resolving a defect is far greater when found later in the cycle. TDD should help you to find defects as they are created, so they can be resolved quickly and at a lower cost:
Source: http://www.isixsigma.com/industries/software-it/defect-prevention-reducing-costs-and-enhancing-quality/
DEFECT COSTS INCREASE AS CODE MATURES
#TestFirst @KevinDunneQA
In traditional development, automation is often built after the code is developed, which has significant limitations:
• Test development is more costly, since we cannot access the code to make it more testable (more details to come)
• Tests are slower and more brittle, with higher levels of maintenance, if we can only access the UI
• Test coverage is incomplete, as we must chose strategically where to build out automation coverage
Moving to TDD flips the process and forces developers to write code to satisfy tests, increasing automation coverage, speed, and reducing cost
BAKE IN AUTOMATION UP FRONT
#TestFirst @KevinDunneQA
Moving towards BDD will force your developers to build an application that can be tested well at the Unit, Integration, and UI levels:
Source: http://zeroturnaround.com/wp-content/uploads/2015/12/PUZZLE-1-min.png
BUILD FOR TESTABILITY
#TestFirst @KevinDunneQA
Moving towards BDD will also demand a more complete testing strategy focused on more than just UI testing:
BUILDING A COMPLETE TESTING STRATEGY
UI Tests
Integration Tests
Unit
Tests
Unit
Tests
Integration Tests
Unit Tests
#TestFirst @KevinDunneQA
TDD paired with continuous delivery will allow you to push features as they become ready, if you’d like to:
Old Way
New Way
PUSH FEATURES WHEN THEY ARE COMPLETE
Code Feature A
Code Feature B
Code Feature C
Test Deploy
Wait
Wait
Code Feature A
Code Feature B
Code Feature C
WriteTests
Deploy
Deploy
Deploy
#TestFirst @KevinDunneQA
A change in organizations is never successful without a complete strategy:
Source: http://www.consultia.co/wp-content/uploads/2015/07/security-technologies-processes.png
RECIPE FOR SUCCESSFUL CHANGE
#TestFirst @KevinDunneQA
People are the largest driver and inhibitor in change, so there are a number of key changes needed to be made in human capital to move to TDD:
PEOPLE RELATED CHANGES
Automation Talent Gap Feature Centric Culture
Siloed Communication Style
#TestFirst @KevinDunneQA
Building automation competence is not simple, but it can be done with proper investment:
BUILDING AUTOMATION COMPETENCE
•Train Up•Train your existing employees around automation and new frameworks•Low cost, long time to benefit, scalable
•Hire In•Hire new employees with automation skill to augment your current team•Medium cost, medium time to benefit, scalable
•Outsource•Outsource your automation efforts to a team •High cost, low time to benefit, not scalable
#TestFirst @KevinDunneQA
Developers need to be educated that their time is not solely spent developing feature code:
Current Time Spent Future Time Spent
SHIFTING FROM A FEATURE CENTRIC CULTURE
•Feature Code
•Feature Code•Test code•Refactoring
#TestFirst @KevinDunneQA
Proper BDD is implemented with collaboration between development, test, and business/product – coined the “3 amigos”:
Source: http://image.slidesharecdn.com/bdd-in-action-140408000826-phpapp01/95/bdd-in-action-principles-practices-and-realworld-application-8-638.jpg?cb=1396916226
CREATING THE 3 AMIGOS CULTURE
#TestFirst @KevinDunneQA
Process can be the source of the largest changes, as a process that has been in place for years may need to be extensively overhauled:
PROCESS RELATED CHANGES
Requirements Specification
Requiring Tests Before Code
Frequent Refactoring
#TestFirst @KevinDunneQA
BDD and ATDD requirements are typically written in a Given, When, Then format that differs from traditional requirements:
Source: https://github.com/cucumber/cucumber/wiki/Feature-Introduction
SHIFTING TO GIVEN, WHEN, THEN
#TestFirst @KevinDunneQA
Developers are always eager to code – requiring them to write tests (or wait for tests to be written) before coding can be a paradigm shift
Source: http://www.tnooz.com/wp-content/uploads/2011/11/angry-computer.jpg
BUILDING TESTS BEFORE CODE
#TestFirst @KevinDunneQA
Properly implemented TDD, BDD, or ATDD should provide additional focus on refactoring and creating more elegant solutions and manageable code:
Source: http://hanwax.github.io/assets/tdd_flow.png
FOCUSING ON REFACTORING
#TestFirst @KevinDunneQA
Tools are often the first place organizations look for change, but they likely should be approached after process and technology:
TECHNOLOGY RELATED CHANGES
Automation Framework Testability / Interfaces
Monitoring
#TestFirst @KevinDunneQA
May teams struggle to pick an automation framework, but the reality is most frameworks are similar and differentiate in a few key areas:
CHOOSING AN AUTOMATION FRAMEWORK
•Dev Style•Is it more focused on BDD, TDD, or ATDD?
•Scalability•Does it have features to support large enterprises, as well as small teams?
•Language•Does it support dev centric languages (i.e. Java) or easier to use languages (i.e. Ruby)
•Workflow•Does it support the concept of features, or stories?
#TestFirst @KevinDunneQA
Moving to BDD process moves testing activity from solely UI based to focusing more testing at the services/API layer:
BUILDING TESTABLE SOFTWARE
#TestFirst @KevinDunneQA
For cloud and web based applications, it is likely wise to implement monitoring to guard against any potential bugs leaking into production while the TDD process is getting started:
MONITORING
What to monitor:• Scalability• Performance• Usability• Localization• …the list goes on