The utility of Google CodePro Analytix in Agile Projectsfileadmin.cs.lth.se/cs/Personal/Lars_Bendix/... · The utility of Google CodePro Analytix in Agile Projects Karl Hedin Sånemyr

The utility of Google CodePro Analytix in AgileProjects

Karl Hedin SånemyrD07, Lund Institute of Technology, Sweden

[email protected]

March 5, 2013

Abstract

Extreme Programming (XP) and other Agile software developmentmethods are widely used within the software industry. When introducingsuch methodologies there is often a set of challenges that are encountered.This paper set out to answer to what extent a selected set of XP practicescan be supported by the use of a static code analysis tool - Google Code-Pro Analytix. A study is performed in the context of coaching a softwaredevelopment project, introducing second year students to Extreme Pro-gramming. The selected XP practices are assesed using Google CodeProAnalytix and the result is evaluated throughout the project and put incontrast to 10 other teams.

Keywords: Static code analysis, agile coaching, CodePro Analytix, Ex-treme Programming

The utility of Google CodePro Analytix in Agile Projects

Contents1 Introduction 2

2 Background 22.1 Supporting the selected XP practices in focus . . . . . . . . . . . 22.2 Tool of choice - Google CodePro Analytix . . . . . . . . . . . . . 3

2.2.1 Code Coverage . . . . . . . . . . . . . . . . . . . . . . . . 42.2.2 Code Audit and Metrics . . . . . . . . . . . . . . . . . . . 5

3 Method 63.1 Incrementally introducing CodePro . . . . . . . . . . . . . . . . . 6

3.1.1 Iteration 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.1.2 Iteration 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.1.3 Iteration 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.1.4 Iteration 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.1.5 Iteration 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.1.6 Iteration 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2 Collected software metrics . . . . . . . . . . . . . . . . . . . . . . 8

4 Result and discussion 94.1 Team1 progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 Comparison with other teams . . . . . . . . . . . . . . . . . . . . 11

5 Conclusion 11

A CodePro Audit Rule Set 13

B Collected software metrics for 10 teams 14

1


1 IntroductionThis paper is the result of a study performed during the course Coaching ofProgramming Teams (EDA270), held at Lund Institute of Technology. Aftercompleting a theoretical part of the course, the attending students take on therole of a coach for a software development project, hereby denoted the PVGproject. Students of this course will hereby be reffered to as coaches. ThePVG project teams consist of two coaches and 8-10 second year students takingthe course Software Development in Teams (EDA260). In this course there isa main focus on the Extreme Programming (XP) methodology. Prior to thiscourse many of the second year students have had no hands-on experience ineither programming in teams of above mentioned size or programming accordingto an agile software development methodology. It is therefore a steep learningcurve for the students of Software Development in Teams. XP has a set of corevalues and practices that for the inexperienced could mean a paradigm shiftin the way of approaching software development. The main goal of this studyis therefore to address a selected subset of these XP practices and explore towhat extent they could be supported by a tool. By providing tool support forthese practices the aim is to prove that the practices are easier to assimilate andfollow - especially for persons previously unfamilair with them.

The findings of this study may be of value for project teams currently enforc-ing or considering implementing Extreme Programming or any other applicableagile software development methodology.

The paper starts off with a background section, providing an introductionto the selected XP practices in focus and theoretical concepts that will be usedto support them. Further, an introduction is given to the tool of choice, GoogleCodePro Analytix. The Method section describes the chosen features of GoogleCodePro Analytix and how they were introduced into the project and whatmetrics were captured. The obtained result is then presented accompanied bya discussion. Finally the findings and conclusions are summarized.

2 Background

2.1 Supporting the selected XP practices in focusExtreme Programming (XP), created by Kent Beck, is defined by a set of valuesand practices all building upon and supporting each other on various levels. Inthis section the selected XP practices in focus for this study will be presentedalong with difficulties in complying with them and how that could be addressed.Each practice will be presented as defined by Kent Beck and a set of metricsand approaches to support them will be surrounded with {}-brackets.

Test-driven development — Programmers continually writeunit tests, which must run flawlessly for development to continue.Customers write tests demonstrating that features are finished. [1]

The practice of writing tests before writing the actual code is probably the mostchallenging of the XP practices for the second year students as this has notbeen enforced in any prior course, and therefore runs a greater risk of not beingfollowed. It requires a somewhat reverse way of thinking that could be perceived

2


as inflexible for the novice. If this practice is not followed the code runs therisk of not being covered by unit tests, potentially leaving broken functionalityundetected after refactoring etc. To support Test-driven development (TDD) arelevant metric is {Code Coverage}, measuring the percentage of the code beingcovered by tests. A decrease in code coverage could serve as an indication thatcode has been written before the unit tests.

Simple design — The system should be designed as simply aspossible at any given moment. Extra complexity is removed as soonas it is discovered. [1]

This practice is essential given the PVG project context. Since only a few storiesare given to the team at a time and circumstances change speculative designshould be avoided. The design is made simple and refactored when needed.However, due to the fact that the second year students might have limitedexperience in applying until then learned software architecture patterns, seeingwhere and when these can be applied might not be obvious. This calls for a wayof easier being able to identify refactoring needs. A software metric often usedin this context is {McCabe’s Cyclomatic Complexity}. Cyclomatic Complexitymeasures the number of linearly independent paths through the code and canpotentially indicate overly complex code in the need of refactoring [2]. Anothermetric of relevance to Simple design is {Lines of Code per Method}. In his bookClean Code, Robert C. Martin suggests that methods should be as short aspossibly, preferably as short as 4 lines of code [3].

Coding standards — Programmers write all code in accor-dance with rules emphasizing communication through the code. [1]

During the PVG project a common source code repository is used. This is inaccordance with the XP practice Collective Code Ownership, giving all teammembers ownership and insight into all code. Because of this there is the evidentneed of using a common set of Coding standards. This requires discipline ofthe team members, and if not followed the code runs the risk of loosing sharedunderstandability. It is therefore benificial to set such common coding standardsand {Automate Coding Standards} into the workflow.

2.2 Tool of choice - Google CodePro AnalytixAs staded in the previous section the features required to support the XP prac-tices in focus are:

1. Measure Code Coverage

2. Measure McCabe’s Cyclomatic Complexity

3. Measure Lines of Code per Method

4. Automate Coding Standards

3


After further reasearch a range of tools were identified resulting in the fol-lowing list of tools fulfilling each requirement:

1. EclEmma, CodeCover, CodePro

2. Checkstyle, PMD, CodePro

3. Checkstyle, PMD, CodePro

4. Checkstyle, PMD, CodePro [4][5][6][7][8]

A tool that was considered but dropped due to commercial license was Agi-tarOne [9].

The tool that provides all above mentioned functionality for supporting theXP practices in focus for this study is Google CodePro Analytix, hereby denotedCodePro. The drawback of CodePro is that it is no longer under development.CodePro is a tool that performs static and dynamic code analysis to produceoutput in a range of different areas, such as Code Coverage, Complexity, De-pendency, Inheritance etc. Below is a list of the features relevant to this study.

2.2.1 Code Coverage

In CodePro code statement coverage is recorded each time the unit tests are run.The code coverage can be viewed on different levels of detail; project, package,class, methods etc. A history of the code coverage repots is saved to track thetrends. Code coverage reports can be exported to HTML. Below is a figure ofa coverage report and how coverage is visualized in the Java Editor in Eclipse.

Figure 1: Code coverage report in CodePro

4


Figure 2: Code coverage visualized in in the Java Editor

As seen in Figure 2 the green markers indicate statements that are coveredby unit tests while red markers indicate uncovered statements.

2.2.2 Code Audit and Metrics

The Audit feature of CodePro produces an audit violation report from a pre-defined set of audit violation rules. A rule violation means that the defined ruleor target metric was not fulfilled. The drawbacks of these kinds of reports is thatthe user is often overwhelmed with violations of varying severity or relevance.It is however possible to define a custom audit violation rule set, selecting whichrules to include and what values to enforce. Below is a figure of a audit violationreport.

Figure 3: Audit violation report in CodePro

The audit violation rules especially relevant to this study are the met-rics mentioned in section 3.2 - Cyclomatic Complexity and Lines of Code perMethod.

5


3 Method

3.1 Incrementally introducing CodeProThe work the PVG Project is divided into 6 2-hour planning sessions and 6 8-hour development sessions, denoted iterations. As previously mentioned, sincethe team members of the PVG Project new to concept of programming in teamsand XP there is a steep learning curve in the startup of the project. To avoid in-terrupting their workflow and shift away their focus too much it was decided toincrementally introduce the CodePro features as the project progressed. Sincesoftware metrics are collected throughout the project (see section 3.2) introduc-ing the CodePro features incrementally could also help indicate the impact ofeach introduced feature.

The goal end goal was for CodePro usage to get fully integrated into theprocess in order to utilize it’s potential. Each time a new concept was introducedto the team it was then evaluated together with the team following an iterationat the Planning sessions. The name of the team is Team1.

3.1.1 Iteration 1

This first iteration was somewhat chaotic - all team members mainly struggledwith getting started with the new way of working. To avoid confusing mattersat this stage CodePro was only used by the coach for monitoring purposes totrack the progress of the team. The team was briefly introduced to CodePro.

3.1.2 Iteration 2

During this iteration CodePro was still just used by the coach, but code cover-age was announced to the team on occasions. Cyclomatic Complexity and it’simplications was explained to the team and cocncerns regarding a class withhigh Cyclomatic Complexity was raised.

3.1.3 Iteration 3

Before this iteration the team had as a preparation assignment read about Code-Pro and prepared to start using it themselves during the iteration. In the be-ginning of the iteration CodePro was installed on all computers. MeasuringCyclomatic Complexity and Code Coverage was demonstrated and encouragedto be used.

3.1.4 Iteration 4

During iteration 3 it was observed that the work generally was running smootherthan before and the stress was going down. It was therefore decided to introducethe next suitable CodePro feature - the audit feautre. The default audit rules ofthe audit feature was evaluated by the coach, wich revealed many unnessecaryand irrelevant rule violations. The default audit rule set was therefore duplicatedand the rules deemed sensible given the context was kept. Specifically, auditrules that supported the previously mentioned XP practices in focus was set:

6


• Maximum Cyclomatic Complexity = 7

• Maximum Lines of Code per Method = 30

These values were discussed and decided upon by the team. There is muchresearch done on levels of cyclomatic complexity. A quote by Capers Jones wasmentioned by the coach as a reference when deciding:

"Empirical evidence for more than 35 years indicates that soft-ware with cyclomatic complexity levels of less than 10 are generallyperceived by software engineers as being fairly simple and easy tofollow." [10]

The Maximum Cyclomatic Complexity of 7 was therefore agreed upon. Max-imum Lines of Code per Method, that according to Robert C. Martin preferablyshould strive to be as short as 4 lines of code [3] was deemed unreasonably lowin this context. A maximum of 30 lines of code was agreed upon, at least en-abling a method to be visible on the screen without the need of scrolling. Fora full specification on the chosen audit rules see Appendix A. The rule set wassubmitted to the repository. This allowed audit rules to be adjusted wheneverneeded, affecting all further performed audits.

For keeping track of the stories under development the team was using aKanban, a scheduling system originally developed at Toyota [11]. On Team1’sKanban there was a list of items that had to be checked in order for a storyunder development to be considered done, called Definition of Done. To avoidrunning the risk of the audit violations being ignored, the procedure of runningand fixing all audit violations was added to the Definition of Done for a story.Further, a code coverage of at least 80% was added to the list. The audit ruleviolations were cleared out as the iteration progressed.

3.1.5 Iteration 5

During this iteration there was alot of turbulence due to the fact that the teamhad an overwhelming ammount of stories to cope with simultaneously. This ac-companied with underestimated times and deadlines led to some audit violationsbeing overlooked and code coverage having to be temporarly put aside.

3.1.6 Iteration 6

The focus of this last iteration was mainly completing unfinished stories andensuring the program’s robustness. This also enabled the team to consistentlyuse CodePro according to the process. Some additional features of CodeProwas explored such as Similar Code Analysis.

7


3.2 Collected software metricsThrughout the project after each iteration the following software metrics wascollected:

• Code Coverage

• Cyclomatic Complexity

• Lines of Code

• Average Lines of Code per Method

This was collected using CodePro. Each snapshot was made directly aftereach iteration. In hope of serving as further input for the result and discussionthe metrics were captured for a total of 10 teams partaking the course. Impor-tant to point out is that code coverage was measured by excluding gui packagesfor all teams in all fairness for the comparison, since some teams focused onperforming gui testing while some teams decided not to.

8


4 Result and discussion

4.1 Team1 progressThe compiled result over the iterations for Team1 can be seen below.

Figure 4: Code coverage progress after each iteration for Team1

After the first iteration code coverage reached 68% which could be an in-dication that TDD was being performed to some extent. TDD was also thefocus of this iteration so it had been repeated by the students prior to the it-eration which could explain the reasonable coverage without the help of anytool. However, things become interesting after iteration 2. During iteration 2the tempo increased since there were more stories added to the backlog and thefirst release was performed. The increased stress of the release, the additionalstories to cope with and the team still being new to the work process probablyaccounts for the drop in code coverage from 0.68% to 0.58%. This indicates thatnew code probably was written before tests. Concerns regarding code coveragewere raised but this seemed to have little effect since it also was not directlyvisible for the team members. During iteration 3 CodePro was installed onall team members computers which made code coverage directly visible in allclasses. The ability of being given feedback by coverage metrics and green orred marks indicating wether the newly written code is tested or not proved veryuseful for the team ending up with the much better code coverage of 83% afterthe iteration. Iteration 4 was as stated the iteration when the team started per-

9


forming code coverage measurments before being allowed checking off a storyas done, probably contributing to the code coverage of 90%. After iteration 5a drop in code coverage was observed, most likely due to an abundance in par-allel work with pressing deadlines. Finally the code coverage reached 97% afteriteration 6, wich is likely the result of being given time to focus on robustnessand narrowing down on untested code with ease by the help of CodePro.

Figure 5: Average cyclomatic complexity after each iteration for Team1

When it comes to average cyclomatic complexity and average lines of codeper method it is interesting that even though the values increase throughoutthe iterations the trend is that the values are reasonably stable even thoughadditional and increasingly complex functionality is added to the system. Thisis of course to be expected since the values are limited by audit violation rules,but it is at least shown that the values can be kept at a minimum with the helpof the audit rules. On a sidenote an interesting finding with CodePro was thatfairly effective in finding similair code using the Similar Code Analysis feature,something that was used in the last iteration.

10


Figure 6: Average number of lines of code per method after each iteration forTeam1

4.2 Comparison with other teamsIn the comparison with other teams a similair trend of increasing and stablecode coverage was noticed among the teams that had a tool for measuringit. The teams that had the highest and most stable code coverage had themeasurement of code coverage strongly integrated in their work process justas this paper hypothesized. Due to the fact that the teams consist of teammembers with varying knowledge, each developed system is composed of verydifferent architectures and that the stories included in each system at any givenmoment is unknown I was unable to draw any overall conclusions regardingTeam1 in comparison with other teams. Generall conclusions that can be madehowever is that for teams that did not integrate policies regarding code coverage,cyclomatic complexity or lines of code per method into their process there wasa tendency of more fluctuations in the values, as compared to the more stablevalues of the teams using tools for such purposes. For a full reference of collectedmetrics see Appendix B

5 ConclusionIn this paper it has been shown that core practices of Extreme Programmingcan be supported and assimilated with the use a static code analysis tool suchas Google CodePro Analytix. It has been shown that a consistent Test-drivendevelopment is easier to obtain when being given constant feedback in the way of

11


CodePro. Design flaws that contradicts the XP practice of Simple design couldalso more easily be identified by the use of setting audit rules for related metricssuch as cyclomatic complexity. Additinally the practice of following commoncoding standards is someting that is simplified by integrating such audit’s intothe process. Finally it can be concluded that there is indeed utility of staticcode analysis tools in agile project - however it is something that needs to befurther explored in order to further evaluate it’s full potential in that context.

References[1] Beck, K. Extreme Programming Explained: Embrace Change. Addison-

Wesley Professional, 1999.

[2] McCabe, T. A complexity measure IEEE Transactions of Software Engi-neering Vol. 2, 1976.

[3] Martin , Robert C. Clean Code: A Handbook of Agile Software Craftsman-ship Prentice Hall 1st edition, 2008.

[4] “EclEmma”, EclEmma, Sept 27, 2012. [Online] Available:http://www.eclemma.org/. [March 5, 2013]

[5] “CodeCover”, CodeCover, July 7, 2011. [Online] Available:http://codecover.org/. [March 5, 2013]

[6] “CodePro”, CodePro Analytix User Guide, March 27, 2012. [Online] Avail-able: https://developers.google.com/java-dev-tools/codepro/doc/. [March5, 2013]

[7] “Checkstyle”, Checkstyle, Sept 18, 2012. [Online] Available:http://checkstyle.sourceforge.net/. [March 5, 2013]

[8] “PMD”, PMD, Feb 3, 2013. [Online] Available:http://pmd.sourceforge.net/. [March 5, 2013]

[9] “AgitarOne”, AgitarOne, Jan 30, 2013. [Online] Available:http://www.agitar.com/solutions/products/agitarone_demos.html.[March 5, 2013]

[10] Jones, C. & Olivier Bonsignour The Economics of Software QualityAddison-Wesley Professional 1st edition, 2011.

[11] “Kanban”, Kanban, Jan 14, 2008. [Online] Available:http://www.infoq.com/articles/hiranabe-lean-agile-kanban [March 5,2013]

12


A CodePro Audit Rule SetTeam1 Audit Rule Set

Explicit "this" UsageDead CodeUnnecessary Import DeclarationsMethod Javadoc ConventionsEmpty MethodMissing Default in SwitchMaximum Cyclomatic Complexiy 7Large Number of Fields 10Large Number of Methods 20Large Number of Parameters 5Max method lines of code 30Max constructor lines of code 30Max initializer lines of code 40

13


B Collected software metrics for 10 teams

Figure 7: Collected code coverage for 10 teams

Figure 8: Collected Average Cyclomatic Complexity for 10 teams

Figure 9: Collected average lines of code per method for 10 teams

Figure 10: Collected lines of code for 10 teams

14

Documents

The utility of Google CodePro Analytix in Agile Projectsfileadmin.cs.lth.se/cs/Personal/Lars_Bendix/... · The utility of Google CodePro Analytix in Agile Projects Karl Hedin Sånemyr