Embed Size (px)
Citation preview
Agile Quality Practices that Save Time
10 Tips for Development Managers and Scrum Masters
1
The impetus for exploring and eventually adopting agile methodologies typically originates from within development. After all, agile was artfully branded as a development methodology created by developers for developers. Yet, despite the benefits (and the sociology) of agile development practices, the “business” only cares about one thing: make my business requirement work. Of course, this goal actually involves three very crucial elements:
1. Implement my business requirement. 2. Make it work as the business expects it to work. 3. Deliver it on time.
The point here is this: Without a robust strategy for agile quality that effectively meets the demands of the business, the most agile of agile teams will remain benched.
To get the ball rolling in the right direction, we've aggregated 10 tips for development and scrum managers to consider while evolving more iterative development methodologies. These tips are designed to help you extend well-known agile quality practices to ensure that your software satisfies business needs—effectively and efficiently:
1. Extend Quality Beyond QA
2. Embrace Policy as a Team
3. Use Automation to Catch as Many Errors as Possible before QA
4. Continuously Evolve Your Quality Policy in Response to Defects Detected
5. Manage Code Complexity
6. Extend TDD and Functional Tests with Automatically-Generated Regression Tests
7. Reduce Developer Testing Complexity
8. Reduce End-to-End Testing Complexity
9. Re-Test Only as Needed
10. Use Automation to Optimize Stand-Up Meetings
2
1. Extend Quality Beyond QA
In an agile process, everyone on the team—from the developer to the product manager—needs to be acutely aware of quality throughout the process, and have quality concerns as an integral part of their daily workflow. Multiple quality tasks are performed simultaneously. This allows for faster discovery and remediation of defects, reducing the time, effort, and cost required to eliminate each defect.
For instance, at any given point, QA might be performing high-level functional testing on one requirement from the current iteration while developers are writing unit tests and code for another. At the same time, the product manager might be concerned about the product's overall reliability or performance, and in response you might start adjusting the quality policy to address this concern. We've found that the best results are achieved when quality fits into the various team members' daily tasks as follows: Role Tasks Development Manager/Scrum Master Distributing requirements
Establishing coding guidelines Setting development test policy Estimating development cost Monitoring work in progress
Developers Reading requirements Implementing requirements Tracking time Tracking changes Testing requirements Participating in peer code reviews Following the quality policy
QA Testers Reviewing developer testing efforts Performing additional testing of requirements Working iteratively during development Rapidly discovering and reporting defects Investigating the cause of defects Influencing development policy
Product Manager Writing requirements Setting deadlines Monitoring progress and cost Monitoring high-level quality status Adjusting requirements and schedule
3
2. Embrace Policy as a Team
Policy is not traditionally associated with agile development—and many pure Agilists worry that the whole notion of policy contradicts the collaborative, lightweight nature of agile processes. Yet we have found that many teams working with agile methodologies actually appreciate having clear expectations regarding how code should be written and tested. With expectations clearly defined, the team does not need to waste time trying to figure out exactly what is expected when—or constantly reworking the code to remedy misunderstandings. You can start off by implementing a policy that simply formalizes the coding and quality practices that the team is already following. Then, working from that baseline, you can incrementally introduce new policies that the team has discussed and agreed to follow. Possible policy modifications can then be discussed in daily stand-up meetings or during code reviews. This allows team members to provide valuable feedback regarding policy refinements (e.g., fine-tuning a static analysis rule to allow certain exceptions) and extensions (e.g., requiring additional test coverage on a module that QA reports as being especially buggy). In addition to ensuring that the entire team is on board, it's also essential to ensure that the policy enforcement serves as a guide, not an intrusion. Ideally, an automated infrastructure checks compliance automatically in the background and works invisibly unless the agreed-upon expectations are not met. At that point, it figures out which team member should be notified, and reminds him or her to perform the tasks required to meet the agreed-upon expectations.
Essentially, the automated infrastructure operates like an EKG system that is hooked up to a hospital patient, constantly measuring electrical activity. If everything is fine, it runs inconspicuously in the background. But if the readings start to flatline, alarms are sounded and people understand that action is needed.
4
3. Use Automated Test & Analysis to Catch as Many Errors as Possible Before QA
If defects stemming from poorly-implemented code are constantly being passed on to QA, this ultimately results in:
Less thorough acceptance testing by QA Frequent/lengthy QA cycles that impact the iteration deadline and scope Defects slipping into the released products, then requiring fixing/patching during the time
allotted for a subsequent iteration Many such defects could be caught by completely automated developer testing—testing that can be performed on the integration server or from the desktop with a single click. The starting point is having an effective coding policy and implementing it with static code analysis. Static code analysis checks for known anti-patterns in source code constructs and reports each occurrence of a match. It can be performed quickly and is guaranteed to find all cases of code that match a pattern. Defensive coding can make code immune to many categories of defects, including many defects that are extremely difficult to detect in testing, by eliminating their root causes. For instance, you can prevent many causes of:
Exceptions Deadlocks Race conditions Missed notifications Infinite loops Initialization issues Memory usage issues Serialization issues Data corruption Resource & memory leaks
Not every type of defect can be prevented by static code analysis. But every one that you can prevent is one less defect that slows the team's iteration. QA is relieved from having to find and document the defect. The responsible developer is relieved from having to stop whatever he is doing, refresh his memory of the problematic code, fix the code, and verify that the problem is solved. And QA does not have to re-test the developer's modification. To find the types of defects that are beyond the scope of static analysis, try the following techniques—both of which require no additional manual effort:
Run data flow static analysis overnight. Data flow analysis statically simulates application execution paths, which may cross multiple units, components, and files. It’s like testing without actually executing the code. With data flow, you can "test" a large number of execution paths without having to write or run a single test case.
Enable runtime error detection as your existing functional tests execute (unit tests and manual or scripted tests). This helps you expose critical defects that surface only at runtime (for example, file overwrites) and zero in on the root cause of the application crashing, running slowly, or behaving unpredictably. The results indicate errors that actually occurred during execution. Thus, defects are reported with unprecedented accuracy.
In addition, we strongly recommend peer code review. This technique does require effort, but is the only effective way to expose high-level defects related to logic, design, etc. Tip #10 discusses some ways to use automation to streamline code reviews.
5
4. Continuously Evolve Your Quality Policy in Response to Defects Detected
When your developer testing or QA testing efforts expose defects that made it past your team's quality policy, don’t just fix that one defect and move on. This is like an automobile manufacturer noticing that a car came off the assembly line with a faulty brake installation, fixing that car's brakes, and hoping that the problem doesn't happen again. A safer approach—one that would save a lot of time, hassle, and grief down the line—would be to examine the assembly line, determine how that problem got introduced, then fix its root cause. In terms of the software development process, this approach involves working with development and QA to:
1. Isolate the root cause of this general type of defect.
2. Determine how to modify the development process to prevent it.
3. Determine how to measure if that the modification is actually being practiced.
In many cases, a coding policy change will do the trick: by enabling or reconfiguring certain static analysis rules, you can find all other instances of existing code that is prone to the same type of defect—and you can also ensure that new or modified code steers clear of the known pitfall. Often, runtime error detection and data flow analysis tools even point you to the precise static analysis rules that will prevent any defects they expose. Other times, the resolution might involve changing the way that code is tested, or adding to a list of items to review during manual code inspections. Ultimately, this immunizes your software development process against many of your most common software defects.
6
5. Manage Code Complexity
Metrics, as most people think of them, aren't incredibly helpful. Assume that I have a million lines of code. If I find out that the code has an average Cyclomatic Complexity of 9 or 6, what does this tell me? Not much. However, since overly-complex code has time and time again been proven to be slower, more difficult, and more risky to update and extend than simpler code, it is helpful to zero in on complex code—say, any class or method whose Cyclomatic Complexity is 10 or higher. This code might work fine right now, but it could cause the team considerable grief down the road when what appeared to be a simple functionality extension ends up requiring a large-scale rewrite. Using the metrics calculations for one of your easy-to-maintain projects as a guideline, set realistic boundaries and thresholds for selected metrics, then ensure that team members are alerted when metrics are outside of the prescribed range. This will reduce the time and effort required to modify and extend this code in subsequent iterations. It will also help new developers get up to speed faster.
7
6. Extend TDD and Functional Tests with Automatically-Generated Regression Tests
If you're already practicing test-driven development (TDD) and continuous automated regression testing, you've got a great foundation for determining when modifications change or break your previously-validated functionality. Yet, if you really want to rest assured that your modifications don't change or break some application behavior that you did not explicitly test with your TDD test cases, more extensive regression testing is needed. One option is to spend lots of time writing many more test cases. But who has time for that? A much simpler approach is to automatically generate a battery of regression tests that will help you check application behavior:
1. Automatically generate a baseline by creating test cases that capture the project code's current functionality. In most cases, such a test suite can be generated overnight using automated unit-testing tools. Because your goal is to establish a behavioral baseline rather than verify the application's current functionality, there is no need to actually review—or even glance at—the generated tests or the assertions/observations/outcomes attained.
2. Detect changes from this baseline by running these tests against your evolving code base on a regular basis. Typically, this is as simple as adding these new test cases to your current automated regression testing infrastructure. On subsequent test runs, test failures will be reported only if the current code behavior does not match the behavior captured in the baseline test suite.
The result is an easy and efficient way to determine when and how your code modifications impact your code—even the parts of the application that are not covered by the functional / TDD tests that you wrote manually. The only work required is to keep the test suite in sync with application changes. When developers arrive at work each morning, they review and respond to any test failures reported for their code. In doing so, they will either address functional defects in their code or update the test suite to reflect the correct behavior of the code.
8
7. Reduce Developer Testing Complexity
The systems that agile teams are building today are growing in size and complexity. As a result. developers often struggle with how to test the specific components they worked on because these components operate as part of complicated systems—they are connected to the outside world, staging systems are difficult to establish and maintain, each test requires considerable setup, and so on. It might be feasible to run a test once, but you really need to run all your team's tests daily so that you’re immediately alerted in case your constant modifications end up breaking previously-verified functionality. Test case tracing allows the team to continuously test parts of the application without the hassles of dealing with the complicated system. Using a tracing technology like Parasoft Tracer, you can create unit test cases that capture the functionality specified in the requirements. From the application GUI or using a SOA or Web test client, you just exercise your use cases while tracing is enabled. As the use case is executed, the technology monitors all the objects that are created, all the data that comes in and goes out, and then it creates you a unit test that represents this operation—and even sets up the proper initial conditions. You can then take this unit test case and execute it on a separate machine, away from the original system. This means that you can use a single machine—such as the continuous integration server or even a standard developer desktop—to reproduce the behavior of a complicated system during your verification procedure. Add all of these tests to your regression test suite, run it continuously, and you’ll immediately be alerted if this functionality that you captured is impacted by your code modifications. Essentially, this helps you address the complexity of today’s systems in two ways:
It relieves you from having to perform all the time-consuming and tedious work required to set up realistic test conditions.
It gives you a test case that you can run apart from the system—a test case you can run continuously and fully automatically.
9
8. Reduce End-to-End Testing Complexity
Agile teams commonly work on distributed heterogeneous applications (e.g., mobile, cloud, composite apps) where one or more component is incomplete, evolving, unstable, inaccessible, or otherwise unavailable for testing. Agile methodologies call for continuous and automated (when possible) regression testing, but how can you run automated end-to-end functional tests nightly when parts of the system under test are not readily available? If you delay testing until everything is ready and available for testing, you risk slipping into a waterfall-like paradigm.
One way to tackle this challenge is to emulate the behavior of the necessary components with service virtualization. Essentially, you emulate (virtualize) application behavior, performance, and data. You want to construct and deploy virtualized assets that can:
Replace the various endpoints in the system architecture (services, applications, mainframes, stored procedures, etc.) as well as emulate the application behavior at the unit level.
Be added to the test environment to replace the behavior of various components that would otherwise inhibit your team’s ability to effectively exercise and validate the end-to-end business process.
Be managed and shared in a team environment, and easily rolled out between shared servers and user desktops. Business data used by the virtualized assets can be easily maintained with spreadsheets or various other data sources.
With such virtualized assets, you can replace dependent systems, methods, or components that would otherwise inhibit the team's ability to effectively validate the requirements that need to be tested or to verify that detected problems have been effectively resolved. For instance, consider a services-based account provisioning system that’s losing orders. To solve this problem, the development team has to test the fixed application in a safe sandbox that emulates the transaction flows of the production environment—or risk breaking the system upon redeploying the application. Using emulation, the team can exercise their services in context, without impacting partners' normal business transactions.
This concept isn't new, but it's becoming increasingly important as enterprise systems continue to become more distributed and heterogeneous. Moreover, this increased complexity makes it necessary to expand the concept of emulation to extend beyond services and also include other essential system components—for example, the database, mainframes, the Web interface, the broker, and so forth—and to be able to emulate the various protocols that commonly co-exist across such systems.
10
9. Re-Test Only as Needed
Although agile methodologies recommend automated testing whenever possible, some degree of manual testing is always required. Each time the application is updated, the QA team typically either repeats all manual tests, or spends a considerable amount of time trying to figure out which of those test cases might be related to the application part that changed. Either way, much time is wasted. With change-based testing, the team can instantly see which requirements are impacted by recent source code changes and which tests need to be executed again in order to verify those requirements. This saves a tremendous amount of QA time and resources that could be better applied to value-added tasks.
To enable change-based testing, requirements are automatically correlated with tests, source code, and development/testing tasks. With these correlations in place, the current level of verification for each requirement or task (including task pass/fail status and coverage) can be assessed at any time by back tracing to all associated tests.
Moreover, the team can instantly gain an objective assessment of which requirements are actually working as expected, which defects are resolved, and which requirements and defects still need testing. This real-time visibility into true requirement and defect status helps you prevent late-cycle surprises that threaten to derail schedules and budgets.
11
10. Use Automation to Optimize Stand-Up Meetings
Stand-up meetings are an important part of agile processes—and are an ideal forum for keeping quality issues at the forefront of development. To keep these meetings at the recommended 15-minute length (and still free up some time to cover quality), it's important to streamline what's covered and take more involved issues "offline." Two strategies that help you optimize these meetings are 1) automated task distribution and tracking and 2) automated peer code review workflow.
Automated Task Distribution & Tracking
Many of the most boring and tedious aspects of stand up meetings can be eliminated with automated task distribution and monitoring. Using a software development management system such as Parasoft Concerto, the manager or scrum master converts requirements and issues into actionable, measurable work tasks which are then automatically distributed directly to the responsible developer's IDE (according to manual assignments or predefined guidelines).
When a developer is ready to start working on an assigned task, he or she marks it as "in progress," works on the task as normal, then marks it as "done" when it's completed. Without interrupting the developer's workflow, the task distribution technology works behind the scenes to track which files are related to the given task and how much time is spent working on these files. Real-time status information is always available in the system.
This tracking is key for evaluating the team's current progress and capacity in preparation for the stand-up meetings. It also keeps QA well-informed about what's ready for testing.
Peer Code Review Workflow Automation Code review typically involves long tedious meetings that require a tremendous amount of preparation. An automated code review system can help the team streamline code reviews by:
1. Identifying updated code from source control or the local file system.
2. Matching the code with designated reviewers. 3. Alerting the reviewer and allowing him to review
the code within his IDE. 4. Tracking the progress of each review item until
closure. The team is then free to focus on the actual peer review: a creative process that cannot (and should not) be automated. Code reviews do not necessarily need to cover every line of code. For instance, some teams might prefer to review only the most dangerous segments of their code (e.g., the most security-sensitive or deadlock-prone areas of the code base). The expectations regarding code review should be discussed with the team and set in the quality policy.
12
Implementing Agile Quality Practices with Parasoft Parasoft's Agile Development Quality solution ingrains Agile quality & testing practices into the team’s application development process. With the industry's broadest spectrum of integrated defect prevention and detection technologies, it's easier than ever to know when functionality is truly "done".
Developers build and validate code according to the team's agreed-upon policies (for static analysis, peer review, unit testing, TDD, etc.). This improves velocity by reducing the amount and difficulty of downstream debugging—and by making code easier to evolve as business needs change.
QA rapidly constructs end-to-end tests that validate all critical aspects of complex transactions, which may extend through web interfaces, backend services, ESBs, databases, and everything in between. Parasoft covers functional testing, load testing, manual user acceptance testing, and application behavior virtualization.
Parasoft supports development-level agile quality practices (static analysis, unit testing, runtime error detection, code review) in C, C++, Java, and .NET. For QA, we provide end-to-end functional and load testing for distributed heterogeneous applications (Web/RIA, SOA, Cloud).
To learn more, visit our Development Testing site and Development Testing resource center.
About Parasoft For 25 years, Parasoft has researched and developed software solutions that help organizations define and deliver defect-free software efficiently. By integrating Development Testing, API/cloud testing, and service virtualization, we reduce the time, effort, and cost of delivering secure, reliable, and compliant software. Parasoft's enterprise and embedded development solutions are the industry's most comprehensive—including static analysis, unit testing with requirements traceability, functional & load testing, dev/test environment management, and more. The majority of Fortune 500 companies rely on Parasoft in order to produce top-quality software consistently and efficiently.
Contacting Parasoft USA Phone: (888) 305-0041 Email: [email protected] NORDICS Phone: +31-70-3922000 Email: [email protected] GERMANY Phone: +49 731 880309-0 Email: [email protected] POLAND Phone: +48 12 290 91 01 Email: [email protected] UK Phone: +44 (0)208 263 6005 Email: [email protected] FRANCE Phone: (33 1) 64 89 26 00 Email: [email protected] ITALY Phone: (+39) 06 96 03 86 74 Email: [email protected] OTHER See http://www.parasoft.com/contacts
© 2013 Parasoft Corporation All rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation. All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries.
