Click here to load reader

Automation Culture: Essential to Agile Success

  • View

  • Download

Embed Size (px)


For organizations developing large-scale applications, transitioning to agile is challenging enough. If your organization has not yet adopted an automation culture, brace yourself for a big surprise because automation is essential to agile success. From the safety nets provided by automated unit and acceptance tests to the automation of build, build verification, and deployment processes, the iterative nature of agile demands a culture of automation across your engineering organization. Geoff Meyer shares lessons learned in adopting a test automation culture as the Dell Enterprise Systems Group simultaneously adopted Scrum and agile processes across its entire software product portfolio. Learn to address the practical challenges of establishing an automation culture at the outset by ensuring that your organizational makeover incorporates changes to your hiring, staffing, and training practices. Find out how you can apply automation beyond the Scrum team in areas including continuous integration, scale and stress testing, and performance testing.

Text of Automation Culture: Essential to Agile Success

  • 1. W16 Agile Testing 5/1/2013 3:00:00 PM Automation Culture: Essential to Agile Success Presented by: Geoff Meyer Dell, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 904-278-0524 [email protected]
  • 2. Geoff Meyer A test architect in the Dell Enterprise Solutions Group, Geoff Meyer has more than twenty-seven years of experience as a software developer, manager, test architect, and business analyst. Geoff co-chairs the Agile Steering committee within Dell Enterprise Solutions Group which guides the software development practices of more than 600 development, test, and UX engineers across three Global Design Centers. He is an active member of the Agile Austin community.
  • 3. 4/16/2013 Automation Culture: Essential to Agile Success STAR East 2012 Geoff Meyer, [email protected] 24 April 2013 Last updated: February 28, 2013 Session Objectives Challenges automating in a HW-dependent environment NOT Real-world examples from a large organization perspective IS Practical Solutions to What, When and What Not to Automate IS Challenges to expect when adopting Test automation during your transition to Agile For Organizational Leaders driving Strategy A Tools Discussion Focused on how to develop Test Automation 1
  • 4. 4/16/2013 Introductions 3 Geoff Meyer Dell Inc, 1998 present Test Architect Agile Test & Automation Strategy Agile Steering Committee Co-Chair Global Projects 15 Scrum teams 9 scrum teams 7 Scrum teams NCR Corp. 1984 1998 SW developer, Project Lead, SW Manager B.S. Computer Science, San Diego State University Masters Engineering Management - NTU 4 2
  • 5. 4/16/2013 Agenda Why Automation is essential to Agile The Dell Landscape and Agile @ Dell Organizational Context Matters Challenges and Common Pitfalls The Automation Landscape Foundations of an Automation Culture Care and Feeding of the Automation Culture 5 The Need, The Challenge, The Pitfalls 6 3
  • 6. 4/16/2013 Why is Automation So Important in Agile? Near-term Ensures that you dont break what you just built Provides safety net for developers & rapid feedback to new changes Continuous Integration and use of Build Verification Test (BVT) Long-term Maximizes velocity of Scrum team Creates capacity for Exploratory and ad-hoc Testing Enables activities that cant be done cost-effectively by humans And if you dont Projects rapidly build up an unmanageable regression suite Differences with Automation in Waterfall? In Waterfall Automated tests are derived from the backlog of completed features In Agile, Automation can be incorporated in the requirement Testing and automation is performed after Development is complete Focus is on first-time discovery of defects and optimizing your test coverage vs. Agile where automation provides immediate feedback to defect introduction 8 4
  • 7. 4/16/2013 The Challenge Adopting Agile In an organization that doesnt have an Automation Culture Apply Test Automation in the context of your organizational automation needs --- Bob Galen, iControl The Context @ Dell 10 5
  • 8. 4/16/2013 Dell Enterprise Solutions Group Global Design and Development using Agile Scrum Silicon Valley Design Center Noida Design Center Austin Design Center Bangalore Design Center 11 Dell Enterprise Solutions Software Products Server Systems Management Converged Infrastructure Systems Management Private Cloud Server Management Console Plug-ins (i.e. for SCCM, vCenter) Common Product Characteristics: Large hardware support test matrix Software is installed the Data Center Enterprise update cycles ~6 months Products must function even as underlying FW, BIOS, and Drivers are upgraded - SUSTAINING 12 6
  • 9. 4/16/2013 Agile @ Dell Roles/Responsibilities Product Management Epic Prioritization & Business Value Represents Customer - Profiles/Roles Requirement Acceptance Product Owner User Story Development Customer rep. for Scrum team Story Acceptance Product Owner Proxy 1 per Scrum Scrum Master Facilitate, Remove Scrum impediments 1 per Scrum Development Architecture, Design, Development, Unit Test 4-5 per Scrum Test Test Design, Automation Development 2:1 InfoDev User-facing text, error messages, online help 1 per 3 scrum team UI UI Design, Development, Unit Test 1 per Scrum Scrum teams 13 Agile @ Dell with Adaptations Feature Complete Release Exit Code Freeze Release Plan Sprints 1 1 2 2 1 2 3 3 3 N-1 N-1 N N N-1 N Hardening Stability Extended Sprint Test Software System Test PRP Define Plan Develop Launch 14 7
  • 10. 4/16/2013 Pitfalls encountered at Development didnt historically automate unit tests Build teams were staffed with non-Build practitioners Minimal guidance beyond Go forth and automate Architecture(s) not optimized for Automatability Automation was interpreted by many as Automate the UI Insufficient SW engineering background across Test teams 15 Which Resulted In Automation not keeping up within the sprint Feature Devotion Multiple automation tools & licenses Test scripts not designed for re-use 16 8
  • 11. 4/16/2013 The Automation Landscape 17 Opportunities for Automation Its not just about Test cases Can provide efficiencies to: Test Preparation, Setup and Configuration Test Content Development and Execution Continuous Integration & Automated BVT Acceptance Testing Test Matrix coverage Non-Functional Testing Longevity, Scale and Performance Characterization 18 9
  • 12. 4/16/2013 Test Preparation Setup & Configuration OS Provisioning and Configuration Deployment Test case staging Environment Cleanup/Baseline Virtualization-based test environments 19 Test Content Development Whole team ownership of QA & Automation UI automation on Customer Usage workflows Automated CI (UT, Build, BVT) Web-services (or CLI) Test Automation for Functional Testing In-Sprint, Automated Acceptance Tests Unit Test Automation 1 CAUTION: Application Architecture can be an enabler or inhibitor 1 - Agile Testing: A Practical Guide for Testers and Agile teams 20 10
  • 13. 4/16/2013 Application Architecture Matters Does it facilitate testing at the API/Services level? Does the Business/Error Logic reside below the services level? Does the UI architecture support test automation other than record-playback? For Large organizations: Standardize UI architectures across the product portfolio HTML5, Flex, Silverlight Standardize Service architectures across the product portfolio SOAP, REST, CLI, API 21 Prioritize and Identify what NOT to Automate Prioritize High ROI 1. BVT Candidate Acceptance Test Test of core functionality that executes in a short duration 2. Core Functionality to be run in Nightly Regression 3. Sustaining Test Candidate Test Functional test that verifies the application can withstand subsystem changes What NOT to Automate Everything GUI Except for High-value customer usage flows once the UI is stable Tests that would only find low severity bugs 22 11
  • 14. 4/16/2013 Non-Functional Testing Performance Characterization 2 Longevity Stress Scale Concurrency Often times analysis of these areas are simply too cost-prohibitive to be done manually 2 - 23 Foundations of an Automation Culture 24 12
  • 15. 4/16/2013 Keys To A Culture Transition Development vs. Test A different School of Test3 Evolve from Functional Responsibility to Whole Product Ownership 3 Scott Barber Approaches to Software Testing: An Introduction 25 Establishing an Automation Culture Identify Near and long-te

Search related