UNIVERSITYOFHERTFORDSHIRE

FacultyofEngineeringandInformationSciences

7WCM0003ComputerScienceMScProject(Online)

FinalReport

January2017

AnInvestigationintoIntroducingTest-DrivenDevelopmentintoaLegacyProject

P.Hall

ii

ABSTRACT

BackgroundTest-DrivenDevelopmentisadevelopmenttechniquethatissaidtoproducecodethatis lesscomplex,highlycohesiveandeasytomaintaincomparedtotraditionaldevelopmentmethods.TDDwas introduced intoa legacyapplication thathadnotesting in place, which allowed users to generate their own content and wasdevelopedinthe.NetMVCframework.

ObjectiveTo determine if adding new functionality to a legacy project with Test-DrivenDevelopmentimprovestheinternalcodequality.

MethodNewfunctionalitywasaddedtoalegacyapplicationusingTest-DrivenDevelopment.Codemetricswererecordedatthestartoftheprojectandaftereveryiterationofdevelopment.TheresultswerethencomparedtoestablishifanydifferencecouldbeseeninthemetricvaluesbeforeandafterintroducingTDD.

ResultsTest-DrivenDevelopmentmarginallyhadnegativeimpactsonaveragelinesofcode,couplingandcomplexitybutithadaslightpositiveimpactuponmaintainabilityandcohesion.Noneofthesedifferencesweresignificant.

ConclusionTest-Driven Development has little to no impact on improving the internal codequality of .Net legacy application. However, it did significantly increase the testcoverageoftheproject,whichcanleadtoareductionintheerrorpronenessoftheapplicationandanimprovementinexternalcodequality.

iii

ACKNOWLEDGEMENTSIwouldliketogivethankstomysupervisorwhohelpedmetoformalisemyideasandwasalwaysonhandtogiveadvice.Iwouldalsoliketothankmylovingfamilyfortheircontinuedsupportthroughouttheprojectandtheirunderstandingforallthoselongnightsandweekendsspentawayfromthem.

iv

CONTENTS1.Introduction............................................................................................................1

1.1Background........................................................................................................1

1.2ResearchQuestions...........................................................................................3

1.3Objectives..........................................................................................................3

1.4Methodology.....................................................................................................4

2.RelatedWork...........................................................................................................6

3.Test-DrivenDevelopment.....................................................................................11

3.1OverviewofTDD.............................................................................................11

3.2TDDToolsin.Net.............................................................................................12

3.3MethodsandTechniquesforTDDandLegacyCode.......................................21

4.MeasuringSoftwareQuality..................................................................................24

4.1WhatMakesGoodSoftware?.........................................................................24

4.2SoftwareMetricsUsedforProject..................................................................25

4.3SoftwareMetricToolsin.Net.........................................................................31

5.LegacyProject.......................................................................................................37

5.1OverviewofApplication..................................................................................37

5.2DevelopmentPlan...........................................................................................37

6.DevelopmentChallengesandSolutions................................................................41

7.Results...................................................................................................................47

7.1SoftwareMetrics.............................................................................................47

7.2CodeTestCoverage.........................................................................................52

8.Conclusion.............................................................................................................55

8.1SummaryandEvaluation.................................................................................55

8.2FutureWork....................................................................................................56

References.................................................................................................................57

AppendixA................................................................................................................63

AppendixB................................................................................................................66

AppendixC................................................................................................................67

AppendixD................................................................................................................69

1

1.INTRODUCTION

1.1BackgroundTest-DrivenDevelopment (TDD) is an approach to developing softwarewere test

codeiswrittenfirstbeforewritingproductioncodetofulfilthetest.Itreliesheavily

uponrefactoringwhilstprovidingasafetynetofteststhatcanberuneachtimethe

code is changed (Crispin 2006) and is not just a testing technique but a process

incorporatingmanydifferentmethodssuchasunit,acceptance,performanceand

systemintegrationtests(Desaietal.2008).TDDfollowsared-green-refactorcyclein

whichatestiswrittenthatfails(red),productioncodeiswrittenthatpasses(green)

and finally that code is refactored if necessary (refactor), this cycle formulates a

continuousrhythmforthelifecycleoftheproject(Beck2002).

TDDisanincreasinglypopularparadigminsoftwareengineeringespeciallywiththe

advent of agile processes over the past two decades. It is a highly sought after

practiceinthejobmarketwithitscitationonITjobadvertsintheUKincreasingyear

onyear(ITJobsWatch2016),butitcanalsobeadauntingpracticetobeginforthose

new to the technique (Crispin 2006). However, a recent study shows that skilled

developerswithnopriorexperienceofTDDcanquicklylearnandproperlyapplythe

methods (Latorre 2014). Despite this increase in popularity, heavy links to agile

methodologiesandapparenteasetoadopt,a2013surveyshowsthatlessthan40%

ofagileteamsemployaTDDapproach(Ambler2013a).

2

Figure1-showshowTDDasajobrequirementhasincreasedgreatlyoverthepastdecade(ITJobsWatch,2016)

Figure2-2013Ambysoft“howagileareyou?”surveyresultsshowinglessthan40%employaTDDapproach(Ambler2013b)

AsystematicreviewbyBissietal.(2016)ontheeffectsofTDDoncodequalityand

productivityfoundthatmostofthecurrentworkonthesubjecthassofarfocused

on examining external code quality by using the Java programming language to

developnewprojects.Thereviewsuggeststhatrelativelylittleworkhasbeendone

3

toinvestigateTDDinlegacyprojectsandstatesthatthereisscopeforresearchinto

howTDDimpactsinternalcodequalityusingasetofcodeanalysemetricsaswellas

howTDDextendstoalternativelanguagesotherthanJava.

ThisprojectwasbuiltupontheworkofstudiessuchasGuerra’s(2014),wherean

investigationtogiveanoverviewoftheprocessofdevelopingacomplexframework

usingTDDpractiseswasundertaken.Thestudyusedcodemetricstoanalysehowthe

framework evolved over time and provided a detailed documentation of the

challengesencounteredduringthedevelopmentprocess.Thisprojectusedasimilar

techniquebutinvestigatedhowTDDinfluencedthecodequalityofalegacyproject.

1.2ResearchQuestions1. DoesintroducingTest-DrivenDevelopmentintoalegacyprojectimprovethe

internalcodequality?

2. WhatarethechallengesandsolutionsofimplementingTest-DrivenDevelopment

practicesintoalegacyproject?

3. Whataretheadvantagesanddisadvantagesofthedifferenttoolsavailablefor

implementingTest-DrivenDevelopmentinthe.Netframework?

4. Dodifferentcodemetricmeasuringtoolsprovidedifferentresultsforthesame

codebase?

1.3Objectives• CompletealiteraturesearchandreviewofexistingworkregardingTest-Driven

Developmentandlegacyprojects

• IdentifyandevaluatetheavailableapproachesforintroducingTest-Driven

Developmentintoalegacyproject

• CompareandevaluatethemostpopulartoolsforTest-DrivenDevelopmentinside

the.Netframework

• Extendthelegacyprojectbyimplementingcompletelynewfunctionalityand

advancedweb2.0featuresusingTest-DrivenDevelopment

• AnalysetheeffectTest-DrivenDevelopmenthasupontheinternalcodequalityof

theproject

• Providedocumentationoftheprojectexperienceanddevelopmentincludingany

challengesencounteredandthesolutionsused

4

• AnalysetheprojectcodecoverageandnumberofunittestsintroducedusingTest-

DrivenDevelopment

• Comparetheresultsofsoftwaremetrictoolsfor.Net

1.4MethodologyThe research for this project applied knownmethodologies to a new case study,

namelyapplyingTest-DrivenDevelopmenttoanexistingprojectandapplyingasuite

ofcodemetricstoanalyseresultsoninternalcodequality.

AlegacyprojectdevelopedinASP.NetMVCwasusedasthebasisfortheresearch

project.ThisprojectisaWeb2.0usergeneratedcontentwebsitebasedaroundthe

idea of users creating painting guides for miniature wargame models. It was

developed in Visual Studio using the ASP.Net MVC framework with C# for the

backendandHTML,JavaScriptandJQueryforthefrontend.Thisprojectwaschosen

as the existing project to work upon because it fits the criteria for the research

proposed:itallowsfortheinvestigationintothetoolsavailableforTDDinthe.Net

frameworkandmostimportantlyitwasnotdevelopedusingaTDDapproachandin

factdoesnothaveanyformofautomatedtestingintegrated,whichfitsthedefinition

ofalegacyprojectasgivenbyMichaelFeathers(2004).Newfeatureswereaddedto

theexistingprojectusingTDD.

Toanalysetheinternalcodequality,whichisatthecoreoftheresearch,thirdparty

toolswereutilised.Developingtoolsforanalysingcodemetricsisbeyondthescope

forthisproject,butcouldbecomepartofanyfuturework.Readingsweretakenat

regularintervalsthroughoutthelifecycleofdevelopmentsothatthegradualimpact

TDDhasontheprojectovertimecouldbeseen.

Different tools and frameworks available for implementing TDD in the .Net

frameworkwere researched, comparedandevaluated; someofwhichwereused

duringdevelopment.Theseweretoolsforunittests,testdoubles,anddependency

injection.NuGet statistics that trackdownloads and installswereutilised, so that

focuswasplacedonthemostpopularandrelevanttoolsandextensionsinthe.Net

community.

5

The overall project was developed using an agile process. This is because agile

processesarecurrently themostpopulardevelopmentmethods (Jeremiah2015).

Agilealsopromotesintegratingtestingintothedevelopmentlifecycleatallstages,

ratherthanastandalonetestingphaseattheendintraditionalmethodslikewaterfall

orv-lifecycle,andtheprojectaimedtoseeexactlyhowTDDfits intotheprocess.

VisualStudioonlinewasusedtomanagetheprojectbecauseagilemanagementtools

are providedwhich allow the ability to add all work items to a task board, plan

iterationsandgetinformationonburndownsandvelocity.

6

2.RELATEDWORKThere have been several studies into Test-Driven Development since it was

popularisedbytheExtremeProgrammingmovementandespeciallybyKentBeckin

his2003work.MostpreviousworkaroundTDDandcodequalityhas focusedon

externalqualityandonlya fewstudieshaveconsidered internalcodequalityand

classdesign(AnicheandGerosa,2015),andevenfewerofthesehaveusedsoftware

metricstoanalyseresults.Therehasbeenalackofcohesioninthemetricsusedto

analyse codequalitywhich leads toadifficulty in comparing the resultsbetween

studies(Bissietal,2016;Kollanus2011).

In2003KaufmannandJanzenconductedanexperimentcomparingsoftwaremetrics

between TDD and test last approaches. They concluded that TDD had greater

productivitybutCyclomaticComplexity(CC)andothermetricsweresimilarbetween

thetwomethods.However,theyadmitthattheexperimentwassomewhatflawed

becausetheTDDgrouphadmoreprogrammingexperience,higheraveragegrades

andthattheprojectsdevelopedweretoosmall.

JanzenandSaiedian(2006)alsoconcludedthatTDDdoesnotshowanynoticeable

improvement in internalcodequality,buttheydidnoteconcernsthatqualitycan

significantlyreducewhentheTDDprocessbreaksdownandnotestsarewritten.

Siniaalto andAbrahamsson (2007) conducted an experiment that used a suite of

traditionalsoftwaremetricstomeasurecodequalityandthedifferencesbetween

TDD and an Iterative Test Last (ITL) method. They only found differences in the

CouplingBetweenObjects(CBO)andLackofCohesionofMethods(LCOM)metrics

betweenthetwodevelopmentapproaches.TDDwasfoundtohaveabetterCBObut

aworseLCOMwhencomparedto ITL,but thedifferencesweresmallandcannot

concretelybeputdowntoTDDalone,experienceofdevelopersmayhaveplayeda

part. The study did, however, find that TDD significantly improved test coverage

comparedtoITL.

AfurtherstudybySiniaaltoandAbrahamsson(2008)usedChidamberandKemerer

metrics (1994), CC and Martin’s Dependency Management Metrics (2003) to

compareTDDandITLapproaches.TheyfoundthatTDDmayproducecodethat is

7

lesscomplexandhasahigherDepthofInheritance(DIT)butthatisalsoharderto

changeandmaintainduetoitsstabilityinrelationtoitsabstractness.

A study by Pancur and Ciglaric (2011) found similar results to Siniaalto and

Abrahamsson when comparing TDD to ITL. They saw small positive increases in

internalcodequalityfromcomplexityandcodecoveragemetricsbutthesewerenot

statisticallysignificant.

JanzenandSaiedian (2008)posited that test-firstprogrammersaremore likely to

producesmallerunitsof lesscomplexcode that ishighly testedcompared to ITL.

They conducted a quasi-controlled experiment comparing TDD and ITL over 6

projects.Theyusedsoftwaremetricstoanalysecomplexity,couplingandcohesion.

TheresultsshowedthatTDDproducedcodethatwaslesscomplexbutthedatafor

couplingandcohesionwasinconclusive,andaslightincreaseincouplingforTDDwas

putdowntomoreclasses thatweresmallerbeingproducedover ITL,which they

claimcouldbepositivedesignduetotheclassesabstractness.However,theywere

unabletoprovethattheTDDclassesweremoreabstract.

Vuetal. (2009)proposedthatTDDdoesnot improve internalcodequalityduring

theirexperimentintothedifferencesbetweenatest-firstandtest-lastapproachto

development.TheymeasuredinternalcodequalitywithCCandWeightedMethods

perClass(WMC).TheyfoundthatCCdoesnotdifferbetweenthetwoapproaches

butWMCwassignificantlyhigherintheTDDprojectmeaningthattest-firstproduced

largerandmorecomplexclasses.

Guerra(2014)developedacomplexframeworkusingTDDandshowsfromsoftware

metric readings that coding standards remained similar throughout development

andthattheresultswerebetterthanstatisticalthresholds.

A2016systematicreviewbyBissietal.found27previouspapersthataddressedthe

effectsofTDDon internal codequality,external codequality,andproductivity. It

concludesthat76%showanincreasein internalquality,howeverthiscomeswith

thecaveatthatonlythecodecoveragemetricwasusedtodistinguishthisasitwas

theonlycommonmetricfoundinallthepapersreviewed.Thiscontrastswitha2011

systematicreviewbyKollanusinwhichitisconcludedfromthe16papersreviewed

8

thatmostfoundnoimprovementincodequalityandthatTDDmayinfactproduce

codethatisdifficulttomaintain.A2010literaturereviewbyShulletal.alsofound

thatpreviousworkintheareashowsnoconsistenteffectoninternalcodequality.

Bissietal.(2016)goesontoproposethatmostcurrentworkisfocusedongreenfield

applicationsandthatthereisalackofresearchintoTDDandlegacyprojects.Italso

findsthatmoststudiesusetheJavaprogramminglanguageduetoitspopularityand

the availability of tools available that support TDD, therefore more studies are

neededintootherprogramminglanguagesandenvironments.

AliteraturesearchproducedonlyfourpreviousstudiesthatlookedatTDDinlegacy

applications,andoutoftheseonlyoneoftheminvestigatedhowitimpactedupon

the internal code quality. Klammer and Kern (2015) looked at industrial legacy

products that had limited or no testing implemented and detailed the problems

encounteredandsolutionsusedforretrofittingunittests,butnotnecessarilyusing

TDD.Theystatethatitmaybeimpossibletoaddunitteststolegacycodewithout

refactoringthecodefirstandthatthismayresultinaheavyrelianceonmocks,which

canresultinlargeandcomplextestsbeingwritten.Theirfindingsindicatethatmocks

should rarely be used and advanced mocking should only be temporary whilst

refactoring,lowtestabilityistheresultofpoorsoftwaredesignandthataddingunit

teststountestedcodemaynotbecosteffective.Theyconcludethatbecauseofthese

findings retrofitting unit tests is unlikely to happen in the real world, therefore

testabilityshouldbeconsideredfromthestartofdevelopment.

Shihabetal.(2010)presentedanapproachfordeterminingwhichfunctionstowrite

unittestsforinlargescalelegacyprojects.Theystatethat90%ofdevelopmentcost

isspentonmaintenancebutwritingunittestsforawholelegacysystematonceis

practically infeasible. The study suggests that TDD should be used for the

maintenanceoflegacycodeandproposesanapproachwhichtheytermTest-Driven

Maintenance(TDM).Inthisapproachfunctionsofasystemareisolatedandunittests

incrementallywrittenuntilqualitytargetismet.Theydetailaprocessforprioritising

whichfunctionstounittestbyextractinghistoricaldatafromtheprojectsuchasbug

fixandgeneralmaintenancemodifications,mappingthesetothefunctionsthatwere

changed,calculatingheuristicsandrecommendingwhichfunctionsshouldbetested.

9

A 2005 study by Nair and Ramnath presented a series of developer stories

documentingexperienceswhentryingto introduceTDDintoa legacyproject.The

studyfindsthatthemostcommonproblemsencounteredwhenintroducingTDDinto

legacyprojectsarebreakingdependenciesandfindingfracturepointstobeableto

getcodeundertest,andtheassumptionthatalllegacycodeisbadandneedstobe

refactored. They advocate the use of dependency injection, interfaces andmock

objectstoovercomedifficultiesandthatlegacycodeisnotnecessarilybadcodeand

itisnotnecessarytomakeit“right”.Theystatethatmostlegacycodecansimplybe

avoidedandonlyrefactoredifabsolutelyneeded,whenitisdirectassociationwith

new features added via TDD. The study suggests a bottom-up approach to

refactoring,followedbyaperiodictop-downreviewtounifythearchitecture.

Themostrelevantpreviousstudytothis reportwasperformedbyChoudhariand

Suman(2015).Anexperimentwasconductedonfivemaintenanceprojectsinwhich

the same user stories were implemented by two different teams, one using an

extreme programming method and the other a traditional waterfall approach.

Metricswerecollectedtoevaluateproductivity,codequalityandmaintainabilityand

asurveywiththeprogrammerswasalsoconducted.Thestudyfoundthattheteams

using extreme programming produced code that had lower complexity, looser

couplingandhighercohesioncomparedtothewaterfallmethod.Italsoreportsthat

the extreme programming approach producedmore maintainable code and had

higher productivity. The survey results concluded that the extreme programming

developersweremoreconfidentinthequalityoftheircodeanditsrobustnessfor

futurechanges.However,thisstudyusedotherextremeprogrammingtechniques

suchaspairprogramming,collectiveownershipofcodeand iterative lifecyclesas

wellasTDD,soitisnotobvioushowmuchofaroleTDDplayedintheresults.

Sofartherehavebeenmixedresultsfrompreviousstudiesandnoclearevidencehas

beenpresentedthatTDDincreasesinternalcodequality.Moststudiessofarhave

reported small positive impacts on code complexity that are not statistically

significant,andnorealdifferenceforanyotherinternalcodequalitymetricswhen

comparedtootherdevelopmentmethods.ThereisnotmuchresearchintohowTDD

impacts legacy projects andmanymore studies are needed before any concrete

10

conclusionscanbemade.TDDseemstohaveasurroundingmyth inthesoftware

industry and literature that it drastically increases code quality, class design and

productivitybutthisisnotbackedupbyanyscientificevidence.

11

3.TEST-DRIVENDEVELOPMENT

3.1OverviewofTDD

Test-DrivenDevelopmentisatechniqueforproducingsoftwareinwhichunittests

are written before any production code. TDD is not a testing technique but is a

method fordesigning codewhich isbelieved tobe cleaner, simplerandeasier to

maintain(Beck,2002)TDDwaspopularizedbyKentBeckintheearly2000saspart

oftheExtremeProgrammingagilemethodologybutthepracticeitselfcanbedated

tomuchearlier,includinganearlyreferenceinNASAProjectMercuryinthe1960s

(SiniaaltoandAbrahamsson,2007).

TheprocessofTDDisiterativeandreliesuponrepetitionofaverysmalldevelopment

cycle. Each iteration follows a red-green-refactor cycle inwhich a new feature is

addedby firstwriting a failing unit test (whichusually gets a redbar in unit test

frameworks),thenwritingthesimplestpossiblecodetomakethetestpass(which

usuallygetsagreenbarinunittestframeworks)followedbyrefactoringthecodeto

removeanyduplication.Thisprocessisfollowedcontinuallyuntilallfunctionalityhas

beenimplemented.Writingthetestsbeforetheproductioncodeinthismanneris

saidtosolidifyunderstandingofwhatthecodeshouldbelike(Feathers,2004)and

inspireconfidenceinthequalityofthecode(Beck,2002).

12

Writeunittestthatfails

(red)

Makecodecompile

Maketestpass

(green)

Runalltests

Duplication?Reworkcode(refactor)

Yes No

Figure3red-green-refactorcycleforTDD

3.2TDDToolsin.Net

3.2.1TypesofToolsTherearenumerousdifferenttypesoftoolsthatcanbeusedforpracticingTDDin

.Net.However,onlyonetypeoftoolisafundamentalrequirementandthatisUnit

TestFrameworks.Therestofthetoolsandframeworkspresentedinthissectionare

optionaland ifneededthetechniquescanbe implementedmanuallywithoutany

tools.

3.2.1.1UnitTestingFrameworks(UTF)Unit testing is a process in which individual components of a system are tested

independently for correct operation. A unit is the smallest testable part of an

applicationwhich is often an entire class or individualmethod in object oriented

systems.UTF’sallowfortheautomationofthisprocessbyenablingthedeveloperto

13

writeapieceofcodethat invokesanotherpieceofcodeandchecksassumptions

ratherthanperformingtestsmanually(Osherove,2009).

10-unittestframeworkswereidentifiedafterperformingaNuGetpackagesearch.

ThesearchwaslimitedtoNuGetbecauseitisthepacketmanagerfortheMicrosoft

developmentplatformincluding.Netanditrecordsthenumberofdownloadssoitis

possibletoseethemostpopularpackagesinthe.Netcommunity.

Name NuGetDownloads LastUpdatecsUnit 360 13/07/2015dbUnit 235 04/02/2016Fixie 28,384 01/09/2016Gallio 19,584 15/10/2011mbUnit* 21,419 20/10/2014MSTest** 2,477 23/10/2015NaturalSpec 9,148 06/12/2012NUnit 7,142,172 04/10/2016VisualStudioUnitTesting***

50,684 14/06/2013

xUnit.net 3,659,325 06/11/2016Table1listofunittestframeworksonNuGetasof10/08/2016

*partofabundlewithGallioonNuGet

**legacycommandlineutilityforolderversionsofVisualStudio

***includedinvisualstudiosodownloadsnotagoodindicationofpopularity

3.2.1.2DependencyInjection(DI)Dependency injection isatechniqueforachieving loosecouplingbetweenobjects

and their dependencies byusing a set of softwaredesignprinciples andpatterns

(Seemann,2011),specificallyaprocesswhereobjectsdefinetheirdependenciesonly

throughconstructorargumentsandproperties.Itcanbeusedtobreakthenormal

coupling between a system under test and its dependencies during automated

testing(Meszaros,2007).DIframeworkshelptodetectdependenciesandmakeit

easiertoemployDItechniques.

Name Downloads LastUpdateAutofac 3,240,042 23/11/2016BalticAmadeus.Container 202 14/09/2014Caliburn.Micro.Container 5,467 19/06/2013CastleWindsor 1,278,257 18/05/2016Container.NET 740 04/11/2013

14

hq.container 91 24/09/2016IoC 4,376 04/03/2016LinFu 5,108 21/03/2012Ninject 3,775,182 14/08/2016Petite.Container 2,943 23/04/2012SimpleInjector 756,781 13/06/2014Spring.Net 77,797 14/04/2015StructureMap 1,462,689 22/11/2016Unity 4,718,018 10/06/2015

Table2listofDIframeworksonNuGetasof07/12/2016

3.2.1.3AutomatedRefactoringTools(ART)Refactoring istheprocessofrestructuringapieceofcodebychangingits internal

structurewithoutchangingitsexistingbehaviour(Fowler,1999).Refactoringisakey

stepintheTDDcycle.Automatedrefactoringtoolshelpintherefactoringprocess

by providing automated operations for refactoring techniques such as renaming,

changing method signatures, extracting an interface, etc. The degree of the

automation of refactoring varies greatly amongst tools, but even semiautomatic

approaches to refactoring offered bymanymodern IDE’s can increase developer

productivitybyafactoroften(MensandTourwe,2004).Feathers(2004)outlined

sometechnicalandpracticalcriteriaforrefactoringtools;theymustbeaccurateand

preservebehaviourofprogram,fasttoenabledeveloperstoworkquickly,integrated

sothat refactoring’scanbeperformedduringdevelopment,andprovideanundo

facility so that an exploratory approach to design can be explored with limited

consequences.

Visual Studio itself provides somebasic automated refactoringoptionsanda few

other sophisticatedautomated refactoring toolsareavailable for .Netandcanbe

installed as extensions to the IDE. These types of tools are usually not free to

downloadanduse.

Name DeveloperCodeRush DevExpressJustCode TelerikReSharper JetBrainsVisualAssist WholeTomatoSoftware

Table3listofmajorARTfor.NetthatcanbeinstalledasextensionstoVisualStudio

15

3.2.1.4TestDoublesTestdoublescanbeusedinplaceofrealobjectsfortheexpresspurposeofrunning

atest(Meszaros,2007).Therearegenerallyfourtypesoftestdoublesthatcanbe

used, but there is confusion in the developer community about the differences

between types of test doubles and Meszaros (2007) states that terminology is

confusingandinconsistent.Mocksisthetermgenerallyusedtocoverallthedifferent

typesoftestdoubles.

3.2.1.4.1DummyDummyobjectsarepassedaroundfrommethodtomethodbutneveractuallyused.

Theyaremainlyusedtofillparameterlists,andareneveractuallyexpectedtodo

anything other than exist (Meszaros, 2007). In statically typed languages like C#

objectsforparametersmustbetypecompatiblesothereforedummyobjectsthatdo

notimplementanyactualbehaviourcanbeusedwhenaparameterisrequiredbut

doesnotneedtobeusedbythetest.

3.2.1.4.2FakeFakes are objects that have working implementations but are not suitable for

production,forexampleinterfacesthatcontainfixeddataandlogicoranin-memory

database (Fowler,2007).Theyaregenerallyusedtoreplacethe functionalityofa

depended-oncomponentwhentherealversionhasnotyetbeenbuilt,istooslowor

isnotavailableinthetestenvironment(Meszaros,2007).

3.2.1.4.3MockMockshelptobreakdependenciesincode(Feathers,2004)bycreatingalternative

implementationsofrealobjectsthatsupplythecorrectvaluesfortesting.Mockscan

performassertionsinternallyandverifycallstotheobject.Therearetwodifferent

typesofmocks,strictmockobjectsthatfailtestsifcallsarereceivedinadifferent

order than expected or lenient mock objects that tolerate out of order calls

(Meszaros,2007).

3.2.1.4.4StubAteststubisusedtoprovideexplicitbehaviourofanobjectsothatcodepathsthat

mayotherwisebeimpossibletocovercanbetested.Theyprovideacannedresponse

tocallsmadetothemethodduringatestandusuallydonotrespondtoanyother

influencesoutsideofthetest(Fowler,2007).Meszaros(2007)identifiestwotypesof

16

teststubs,aresponderthatreturnsvalidandinvalidvianormalmethodcallsanda

saboteurthatraisesexceptionsorerrorstosimulateabnormalbehaviour.

3.2.1.4.5AvailableTestDoubleTools13 test double frameworks were identified for .Net that provide different

implementations of the various types of doubles. There are few frameworks

specificallyforeachtypeoftestdouble,mostarecalledmockingframeworksdespite

thetypeofdoubletheyimplement.ByfarthemostpopularframeworkforC#and

.NetisMoq.

Name Downloads LastUpdateAutoMock 1,893 18/05/2016DimMock 1,648 25/01/2012EasyMoq 2,022 01/05/2013FakeItEasy 613,917 05/12/2016Foq 22,567 14/06/2015JustMockLite 89,147 04/05/2016LightMock 3,400 04/02/2016Mockjockey 2,242 13/02/2013Moq 8,942,046 20/08/2016NMock3 19,395 04/04/2013Prig 1,985 14/09/2014RhinoMocks 843,888 22/04/2014Simple.Mocking 9,539 22/12/2015

Table4listoftestdoubleframeworksonNuGetasof07/12/2016

3.2.2ComparisonofUnitTestFrameworksOnlytheUnitTestframeworkswerecomparedastheyaretheonlytypeoftoolthat

isrequiredforTDD.ThetoptwoNuGetpackagesbydownloadwereusedinthetest

alongwithVisualStudioUnitTestingasthisisincludedinVisualStudioasthedefault

unittestingoption.

3.2.2.1NUnitNUnitisanopen-sourceandfreeUTFforall .Netlanguages.It ispartofthexUnit

familyofunittestframeworksandwasoriginallyportedfromJunit(aJavalanguage

UTF).ItfullyintegrateswithVisualStudioIDEandtestscanberunviatheconsoleor

withthirdpartytestrunnerssuchasReSharper.Thelatestversionsofferamultiple

constraintassertionsyntax,whichusesasinglemethodoftheassertclassandpasses

inaconstraintobject.

17

[Test] public void GetDeletePostReturnsView() {

var result = _controller.DeletePost(_post1.ForumPostId); Assert.That(result, Is.InstanceOf(typeof(ViewResult))); }

Figure4exampleNUnitunittestshowingmultipleconstraintassertion

3.2.2.2xUnit.NetxUnit.Net is a free, open-source UTF that is part of the xUnit family of test

frameworks.ItsupportsC#,F#,VB.Netandother.Netlanguages.Itfullyintegrates

intoVisual Studio IDEand canworkwith various test runners suchasReSharper,

CodeRush, TestDriven.Net. xUnit.Net does not use SetUp or TearDown methods

unlikemostUTF’s, instead favouring constructors and IDisposable. This results in

largerunitteststhatintheoryareeasiertounderstand.

[Fact] public void GetDeletePostReturnsView() {

SetUpControllerWithUnitOfWorkMock(); var testPost = new ForumPost(); _uowMock.Setup(x => x.Repository<ForumPost>().GetById(1)).Returns(testPost); var result = _controller.DeletePost(1); Assert.IsType<ViewResult>(result); }

Figure5exampleunit test inxUnit.Net.Showsuseofparameterised testpatternbecauseofnosetupandteardownmethods

3.2.2.3VisualStudioUnitTestingVisualStudioUnitTestingisaUTFthatisincludedinsomeversionsofVisualStudio.

TestscanbeexecutedviaarunnerinVisualStudioincludingthirdpartyrunnersor

fromthecommand lineusingMSTest.Theframeworkusesattributestodecorate

classesandmethodsastestsandusesdifferentmethodsontheassertclasstoverify

behaviour.

18

[TestMethod] public void GetDeletePostReturnsView() {

var result = _controller.DeletePost(_post1.ForumPostId); Assert.IsInstanceOfType(result, typeof(ViewResult)); }

Figure6exampleunittestinVisualStudioUnitTestframeworkshowingmethodcalltoassertclass

3.2.2.4MethodandResultsAunittestshouldrunfastandhelplocalizeerrorsquickly(Feathers,2004)andAbig

partofTDDistheabilitytoperformarunofallunittestsfrequently(usuallyevery

timeabuildiscompiled),soanimportantconsiderationforaUnitTestFrameworkis

thespeedinwhichitcanruntests.Aspeedtestcomparisonwasperformedonthe

threeframeworkslistedabove.

OnetestclassfromtheprojectthathadbeenwritteninNUnittoimplementafeature

viaTDDwaschosenthatrepresentedavariedselectionoftests.Thisclasswasthen

rewrittenintotwonewclassesforVisualStudioUnitTestandxUnit.Net.Thetests

werekeptasclosetotheoriginalaspossible,testingtheexactsamepieceofcode

andthesameexpectedresultsbutusing thesyntaxandmethodsof thedifferent

frameworks.Ifanyothertoolsorframeworkswereusedintheoriginaltests(suchas

mocks)thentheexactsameversionwassetupandusedinthenewtest.

Therewere40unittestsinthetestsuiteso80newtestswerewritten(40inVisual

Studio Unit Test and 40 in xUnit) for a total of 120 unit tests. These were then

executed on two different test runners, the built in Visual Studio runner and

ReSharpertestrunnertoseeifthetestrunneritselfplayedanypartintheresultsor

ifitwasjusttheframeworkthatdeterminedexecutionspeed.

Foreachoftheunittestmethodsforeachframeworkthetimeinmillisecondsthat

the test took to execute was recorded for each test runner. Metrics were then

calculatedonthedatacollectedandtheresultscanbeseenthetablebelow.

19

ReSharper VSRunnerMetric VSTest NUnit xUnit VSTest NUnit xUnitTotal 687 604 1931 446 234 1235Max 436 49 1080 243 14 662Min 4 8 1 3 4 1Average 16.75 11.23 48.28 11.15 5.85 30.88Median 5 9 4 4 5 2StandardDeviation 67.23 7.12 190.09 37.30 2.07 120.85

Table5valuesinmillisecondsforbothtestrunners

AtfirstitappearsthatNUnitisbyfarthefastestframeworkacrossbothtestrunners

whenlookingatthetotalandaveragevalues.However,ifwelookatthemedianand

standarddeviationitappearsxUnithassomeextremeoutliersthatareskewingthe

results.TestsgenerallyseemtorunfasterinxUnitbutithastheoddtestthatruns

extremelyslow,whereasthisdoesnothappenwiththeotherframeworks.Thiscan

beseenbetterontheboxplotgraphsbelow.

Figure7boxplotforReSharpertestrunnerwithoutoutliersshowsthatmostresultsinxUnitarefasterthantheNUnit

20

Figure 8 box plot for VS Test Runnerwithout outliers shows contrast in evenmore detail thanReSharperresults,suggestingthatxUnitismuchfasterwithouttheextremeresults.

Figure9boxplotforVSTestRunnerwithoutliersandmeanindicated.xUnithastwooutliersthatarefarabovetherestoftheresults,VSTestalsohasoneoutliermuchgreater

Two tail T-Tests were used to compare the data and determine if there was a

significantdifferencebetweentheframeworksregardingthespeedinwhicheachof

the40unittestswereran.Avalueof0.05orlessisgenerallyconsideredstatistically

significant (Craparo, 2007) and none of the T-Tests performed were below this

threshold.ThebiggestdifferencewasbetweenxUnitandNUnit,butagainthiswas

notastatisticallysignificantdifference.

21

T-Test VSUnitTest NUnit xUnitVSUnitTest

0.61 0.33

NUnit 0.23xUnit

Table6T-TestsresultsforReSharpertestrunnershowingPvalue,results<0.05arestatisticallysignificant

T-Test VSUnitTest NUnit xUnitVSUnitTest

0.38 0.33

NUnit 0.2xUnit

Table7T-TestresultsforVStestrunnershowingPvalue,results<0.05arestatisticallysignificant

Overall itappearsthatthereissomedifferencebetweentheunittestframeworks

andbetweenthetestrunnersthemselves,althoughnotstatisticallydifferent.The

datashowsthatNUnithasthefastestoverallspeedandalltestmethodsexecutein

asimilartime,whereasbothVSUnitTestandxUnithavesomeextremeresultsthat

slowdown theoverall time.Most tests run faster in these two frameworks than

NUnit,asseenbythemedianvalues,buthaveextremeoutliersthatslowdownthe

overalltestsuite.AsfarastestrunnersareconcernedthedatashowsthatVSTest

Runner isconsistently faster thanReSharperandthebestcombinationoutof the

optionscomparedisunittestswrittenintheNUnitframeworkandexecutedonthe

VSTestRunner.

Thiscomparisonwasonlyconductedonasmallsubsetoftheunittestswrittenfor

theproject, inwhichmore than400werecodedandmany largescale realworld

productscouldhavemanythousandsofunitteststhatneedtoberunoften.Over

thatlargeamountofdata,amoresignificantdifferencemaybeseenintheresults.

Forfurtherworkamuchlargersetoftestclassesandmethodsshouldbeusedwhich

alsoutilisemoreoftheframeworksnuancesandtechniques.

3.3MethodsandTechniquesforTDDandLegacyCode

Legacy code is defined as any application that does not have testing in place

(Feathers,2004).Oneofthebiggestproblemsinworkingwithlegacyapplicationsis

notbeingabletodetect ifchangesmade,either fromrefactoringtheoldcodeor

22

addingnewcodethatinteractswiththelegacycode,causeexistingfunctionalityto

break.Withouttestinginplace,itisimpossibletobecertainthatchangesmadedo

notcauseanyissuesinotherpartsoftheprogram.

Thereareafewdifferentapproachestoworkingwithlegacycode.Thefirstoptionis

tosimplyabandonthelegacyapplicationandstartagain,thisapproachhasmeritsin

thatthenewversioncanbemadecleanerandlearnfromthemistakesoftheoriginal

aswellasputtingtestsinplacebutthedownsideisthatitcanbetimeconsuming

andwastefulespecially if the legacyapplication is stillworking correctly.Another

approachistogetthewholeofthelegacyapplicationundertestbeforeaddingany

new functionality, however, this can be extremely difficult and costly as well as

leadingtoalongtimebeforeanynewfeaturesareseenintheprogram.

The third approach, and one recommended by Feathers (2004), is to only touch

legacycode inplaceswhere it isneeded, i.e.whenanewfeature isrequiredthat

must interactwithpartsof theexistingcodebase. In these instances, the feature

shouldbeaddedusingTDDandanylegacycodethatthenewcodemustinteractwith

shouldbewrappedinsideatestharnessbeforeanyrefactoringtakesplace.Onsome

occasionsthelegacycodemaybeuntestable,forexamplebecauseofdependencies

ortightcoupling,andintheseinstances,otherformsoftesting,i.e.acceptanceor

integrationtests,mayneedtobetemporarilyintroducedbeforetherefactoringto

getthecodeundertestbeforebreakingdependenciestowriteunittests.Thisisdone

toensure that thebehaviourof the legacy code isnot alteredwhen changing its

structure. Some automated refactoring tools can help with this scenario by

performingtherefactorinamoresecuremanner,butcautionmustbeutilizedasnot

alltoolsare100%reliable(Feathers,2004).

Feathers(2004)statesthatnewfeaturesshouldbeaddedviaTDDwhenworkingwith

legacycodeasthis isthemost importantfeatureadditiontechniqueanditallows

developerstoworkonnewcodeindependentlyoflegacycodeaswellasintroducing

teststotheproject.HegoesontooutlineanextendedTDDcycle inwhichanew

phaseisaddedatthebeginningtogettheclasstobechangedundertestandanote

ofcautiontonotchangeanylegacycodewhenmakingthetestpass.Legacycode

canbemodifiedduringtherefactoringprocesswhenitissafelyunderatestsuite.

23

Writeunittestthatfails

(red)

Makecodecompile

Maketestpass(green)withoutmodifyingold

code

Runalltests

Refactor?Reworknewandoldcode

Yes No

Getcodeundertest

Figure10modifiedTDDcycleforlegacycode(Feathers,2004)

24

4.MEASURINGSOFTWAREQUALITY

4.1WhatMakesGoodSoftware?

Softwarequalityishardtodefineandthereisnoperfectdesign(Marinescu,2005),

butseveralattemptshavebeenmadetoestablishwaysinwhichtomeasurethecode

of programs to determine this quality. The International Organisation for

Standardisation(ISO)setforthaqualitymodelinISO9126in1991whichhassince

beensupersededby ISO/IEC25023:2016.Thesestandardsprovideasetofquality

measures that can be used for quantitativelymeasuring and evaluating software

quality,includingthoseforinternalmetrics,acrossmultiplecharacteristicsandstates

thatcomplexityandmaintainabilityarekeyfactorsinestablishingsoftwarequality

(BSI,2016).

Martin(2006)putforthasetofprinciplesforobject-orienteddesignthatheclaimed

ifappliedtogetherwouldleadtoalesscomplexandeasiertomaintainsystem.These

principlesarebased largelyuponhaving lowcomplexity,highcohesionand loose

couplingofclassestoproducehighqualityobject-orientedcode.

Alargebodyofsoftwaremetricssuiteshavebeendevelopedformeasuringobject-

orienteddesignandnumeroustoolsareavailableforcalculatingthese(Linckeetal.,

2008). In1977Halsteadproposeda suiteofmetricsbasedaroundmeasuring the

complexityofsoftware,andalthoughithashadalastingeffectandisstillusedin

othermetrics such asmaintainability index, the work has been heavily criticised

(FentonandBieman,2014).McCabe(1976)alsoproposedametric formeasuring

softwarecomplexity,CyclomaticComplexity,thatisstillwidelyusedtoday.

Otherpioneersinsoftwaremetricsforobject-orienteddesignwereChidamberand

Kemererwhoin1991proposedasuiteofmetricswithsixdifferentmeasurements

thatcouldbeusedtopredictthequalityofanyobject-orientedlanguage.

25

MetricName AbbreviationCouplingBetweenObjects CBODepthofInheritanceTree DITLackofCohesionofMethods

LCOM

NumberofChildren NOCResponseforaClass RFCWeightedMethodsPerClass

WMC

Table8ChidamberandKemerermetricssuite

TheMartin suite of metrics (Martin, 1994) establishes a set of metrics that can

measurethequalityofanobject-orienteddesignintermsoftheindependenceof

subsystems.Itsetsforthadesignpatterntoencourageallformsofdependencyto

be of a desirable form, and the set of metrics are designed to measure the

conformancetothedesignpattern.

Metric AbbreviationAbstractness AAfferentCoupling CaEfferentCoupling CeInstability IDistancefromtheMainSequence

D

Table9Martinmetricsuite

AthesisbyAnderssonandVestergren(2004)conductedanexperimentintowhich

metrics are best suited for measuring object-oriented design by taking

measurements from a selection of good and bad quality source code on several

differentsoftwaremetricstools.Theyconcludethatmetricsdohaveapracticaluse

andthattheycantosomeextentreflectasoftwaresystemsdesignquality.Theyalso

statethatsomemetricsarebetterthanothersforpredictingsoftwarequality.

4.2SoftwareMetricsUsedforProject

AsubsetofthemetricspresentedbyAnderssonandVestergren(2004)intheirstudy

onmeasuringobject-orienteddesignwereusedtoanalysethecodequalityofthe

legacyapplicationeachofwhich isdescribed indetailbelow.Thesemetricswere

26

chosen because of the availability in the tools surveyed and to be able to give a

comparisonofthesetoolsasuiteofmetricswasneededthatallthetoolssupported.

ThereportlooksatthemetricsfromanapplicationlevelforserversideC#codeonly

anddoesnotanalyseanyclientsidecode(HTML,JavaScript,etc.).

MetricName AbbreviationMaintainabilityIndex MICyclomaticComplexity CCLackofCohesionofMethods

LCOM

CouplingBetweenObjects CBOLinesofCode LOC

Table10metricsusedfortheprojectandtheirabbreviation

4.2.1MaintainabilityIndex(MI)This is a calculation of composedmetrics to show how easy the code base is to

maintaininthefuture.OmanandHagemeisterfirstproposedMIin1992anditwas

used in an extensive trial by Hewlett Packard (Coleman et al., 1994). The MI is

comprised of traditional software metrics: weighted Halstead metrics, McCabe’s

CyclomaticComplexity,linesofcodeandnumberofcomments;andcomposedinto

asinglefigurethatindicatesthemaintainabilityofapieceofcode.

Twooriginalcalculationsweresetforth;onethatconsideredcommentsandonethat

didnot(OmanandHagemeister,1992).

MI=171-3.42ln(aveE)-0.23aveV(g')-16.2ln(aveLOC) Equation1whereaveE isaverageHalsteadeffectpermodule,aveV(g') is theaverageextendedcyclomaticcomplexitypermoduleandaveLOCistheaveragenumberoflinespermodule.

MI=171-3.42ln(aveE)-0.23aveV(g’)-16.2ln(aveLOC)+0.99aveCM

Equation2whereaveEistheaverageHalsteadEffortpermodule,aveV(g’)istheaver-ageextendedcyclomaticcomplexitypermodule,aveLOCistheaveragelinesofcodepermodule,andaveCMistheaverage

numberoflinesofcommentspermodule.

Severalothercalculationshavebeenproposedoverthesubsequentyears(Welker,

2001),butareallbasedoffsimilar traditionalmetricsandcomposed intoasingle

27

figure.VSMetricsistheonlytoolusedforthisprojectthatgivesanMIratingandit

calculatesitdifferentlytotheoriginalversion.Thevalueisnormalisedtobeinthe

rangeof0to100insteadoftheoriginalrangeof171toanunboundnegativenumber.

MI=MAX(0,(171–5.2*ln(HalsteadVolume)–0.23*(CyclomaticComplexity)–16.2*ln(LinesofCode))*100/171)

Equation3MIascalculatedbyVSMetricstool(Conorm,2007)

ThethresholdsasproposedbytheVSMetricstoolareasfollows:

Range Maintainability

20-100 Goodmaintainability

10-19 Moderatemaintainability

0-9 LowmaintainabilityTable11maintainabilitythresholdsasproposedbyVSMetricstool(zainnab,2011)

4.2.2CyclomaticComplexity(CC)CCmeasures thecomplexityof theprogrambycalculating thenumberof linearly

independentpaths inthecodeandhasafoundation ingraphtheory.CCwasfirst

proposedbyMcCabein1976anditsprimarypurposewastoprovideamathematical

techniquetoidentifysoftwaremodulesthataredifficulttotestormaintain(McCabe,

1976).Inpracticalityterms,itcanbeusedtoprovidetheupperboundforthenumber

oftestsneededtoensurethatallcodepathsareexecuted(Pressman,1997).

McCabe’sCCmetricisdefinedas:

v(G)=e-n+2

Equation4wherev(G)equalsthecyclomaticcomplexityoftheflowgraphG,eequalsthenumberofedgesandnequalsthenumberofnodes.

Tobetterillustratethisconsiderthefollowingexampleofabubblesortalgorithmin

C#:

28

1: for (int i = intArray.Length - 1; i > 0; i--) { 2: for (int j = 0; j <= i - 1; j++) { 3: if (intArray[j] > intArray[j + 1]) { 4: int highValue = intArray[j]; 5: intArray[j] = intArray[j + 1]; 6: intArray[j + 1] = highValue; } } }

Figure11bubblesortalgorithminC#

ToworkouttheCCitcanbeconvertedtoacontrolflowgraphusingthenumbersin

codeaboveasnodes:

Figure12controlflowgraphforsamplecodeshowing6nodesand8edges

Thenumberofedges=8andnodes=6,sotheCCcanbeworkedoutasv(G)=8–6

+2=4.

AsCCmeasurescomplexity,itisthereforedesirabletokeepthenumberaslowas

possible.McCabehimselfusedanupperboundof10,whichhestatedseemedlikea

“reasonable,butnotmagical,upperlimit”(McCabe,1976).

4.2.3CouplingBetweenObjects(CBO)Alargeassortmentofcouplingmeasurementsforobject-orientedsystemshavebeen

definedbutChidamberandKimerer’sCBOmetricisthemostwidelyused(Mitchell

andPower,2005).

1

2

3

4

5

6

29

CBOforaclass isthecountofthenumberofotherclassestowhichit iscoupled.

Couplingoccurswhenoneclassusesmethodsorinstancevariablesofanother,but

itisnotassociative,forexampleifclassAiscoupledwithclassBandclassBiscoupled

withclassCthenclassAisnotnecessarilycoupledwithclassC.

CBOwasfirstproposedbyChidamberandKemerer(1991)intheirpapertoestablish

a metric suite for object oriented design along with 5 other metrics. The same

authors revised CBO in 1994were some ambiguities regarding inheritance and a

requirementforbi-directionalityincouplingwereremoved(ChidamberandKemerer

1994).

ChidamberandKemerer(1994)arguethatcouplingshouldbekepttoaminimumto

improvemodularityandencapsulationandthathighcouplingcanbeassociatedwith

poormaintainability, due to a sensitivity in changes in other areas of thedesign.

Classesthatdependoneachothertoomucharehardertounderstand,changeand

correctandaremorecomplextotheresultingsystem(MitchellandPower,2005).

However,Henderson-Sellersetal.(1996)statesthatwithoutanycouplingasystem

is useless, therefore some coupling is necessary, and that it is the elimination of

extraneouscouplingthatisthegoalofgoodobject-orienteddesign.

CBOcanbeusedasanindicatorofwhetheraclassislosingitsintegrity(Siniaaltoand

Abrahamsson,2007),howcomplexthetestingofvariouspartsofthedesignarelikely

to be (Chidamber and Kemerer, 1994) and the maintainability, fault proneness,

testabilityandchangeofpronenessofsoftwaredesign(MitchellandPower,2005).

4.2.4LinesofCode(LOC)LOCisameasurementofamodulessizeandisoneoftheoldestsoftwaremetrics

(Rosenberg,1997).However,assimpleasthatsounds,therearemanyvariationsof

thismetricanddiscrepanciesoverwhatexactlyshouldbecounted.Therearemany

considerationsastowhatshouldbeincludedinthecountsuchascommentedlines,

blank lines, singlemethod calls than spanmultiple lines because of coding style,

includeandusingstatements,etc.andthiscanleadtocountsofthesamepieceof

codewidelyvarying(Smacchia,2007).

30

Rosenberg (1997) states that LOC can be used as a predictor of development or

maintenanceeffort,acovariateforothermethodstonormalisethemtothesame

codedensityandasastandardagainstwhichothermetricscanbeevaluated.

Onamethodlevel,andwhencalculatingtheaverageLOCformethods,itcanbea

goodindicatorofthecomplexityofthecode.TheNDependtoolrecommendsthat

methods with LOC greater than 20 are hard to understand and those with LOC

greater than 40 are extremely complex and should be split into many smaller

methods(NDepend,2016).

4.2.5LackofCohesionofMethods(LCOM)LCOM is a measurement of how cohesive a class is. It measures the correlation

betweenmethodsandthelocalinstancevariablesofaclass.Cohesionisdesirablein

aclassbecauseitpromotesencapsulationandadherestothesingleresponsibility

principleasproposedbyMartin(2003).

LCOM was first proposed by Chidamber and Kemerer in 1991 but several other

definitions have been developed subsequently (Li and Henry, 1993; Henderson-

Sellers,1996).

ConsideraclassC1withnmethodsM1,M2,...,Mn.Let{Ij}=setofinstance

variablesusedbymethodMi.

Therearensuchsets{I1},...,{In}.LetP={(Ii,Ij)|Ii∩ Ij=∅}andQ={(Ii,Ij)|Ii∩Ij≠∅}.Ifallnsets{I1},...,{In}are∅thenletP=∅.

LCOM=|P|-|Q|,if|P|>|Q|=0otherwise Figure13definitionofLCOMbyChidamberandKemerer(1994)

LCOMproducesvaluesintherangefrom0to1,where0indicatesperfectcohesion

and1anextremelackofcohesion.Itisdesirabletokeepthevalueascloseto0as

possible.

Lowcohesivenessincreasescomplexity,therebyincreasingthenumberoferrorsin

thedevelopmentprocessanditimpliesthatclassesshouldprobablybesplitintotwo

ormoresubclasses(ChidamberandKemerer,1994).

31

4.3SoftwareMetricToolsin.Net

4.3.1PreviousWork

Lotsofworkhasbeendoneregardingmetricsandtheir suitability forquantifying

softwarequalitybutnotmuchresearchhasbeencarriedoutcomparingcodemetric

tools,andonlyonepaperhasaddressedthisinrelationtotoolsspecificallyfor.Net.

In their 2011 study Novak and Rakić identified 5 .Net tools that met their

requirementsformeasuringsoftwaremetrics,theywentontoanalysethetoolsby

runningthemonseveraldifferentsoftwareapplicationsandcomparedtheresults

againstoneanotherusingT-Tests.Theyconcluded that softwaremetric toolscan

giveresultsthatdiffersignificantlyforthesamemetricsonthesameapplications.

Mostotherworkinthefieldhasfocusedonotherprogramminglanguages,notably

JavaandC/C++.Linckeetal.(2008)performedacomparisonof10softwaremetric

toolsforJavaandanalysed9metricsthatmostofthetoolsimplemented.Theyfound

thattherewereseveralissueswithmetricsdefinitions,notablythattheyareunclear

and inexact which opens the possibility of different interpretations and

implementationsandthatnamedoesnotdistinguishdifferentvariantsofthesame

metric.Theyconcludedthatmostofthemetricstoolsprovidedifferentresultsfor

thesameinputsandthatthiscouldbeduetoadifferenceininterpretationofthe

metric. Bakar (2012) also found that different tools calculate the same metrics

differently by comparing four tools and attributes this result to an ambiguity of

metrics definitions, which lead to different implementations across tools. He

concludes that research using softwaremetrics are tool dependant and different

resultsmight be given by using a different tool because there can be significant

differencesbetweenthemeasurementresults,healsosuggeststhatsomestandard

meaningsshouldbeprovidedforsoftwaremetrics.Noneofthestudiesformulatean

opiniononwhichtoolgivesbetterormoreaccurateresultsbuttheyallconcludethat

theresultsdifferfromoneanothersignificantlyonatleastsomesoftwaremetrics.

4.3.2AvailableTools

14 different tools were identified for measuring software metrics in .Net after

performing an internet search. These varied from relatively simple lines of code

32

counterstocomplexstaticcodeanalysissoftwareforenterpriselevelwhichrequired

substantiallicencefees.

Name Manufacturer Link FreeC#SourceCodeMetrics

SemanticDesigns

http://www.semdesigns.com/Products/Metrics/CSharpMetrics.html

No

RSM MSquaredTechnologies

http://msquaredtechnologies.com/m2rsm/index.htm

No

LocMetrics LocMetrics http://www.locmetrics.com YesCodeCounterPro

GeroneSoft http://www.geronesoft.com No

SLOCCount DHWheeler http://www.dwheeler.com/sloccount/

Yes

SLOCMetrics Microguru http://microguru.com/products/sloc/

No

EZMetrics JamesHeires http://www.jamesheiresconsulting.com/Products.htm

No

McCabeIQ McCabeSoftware

http://www.mccabe.com/iq_developers.htm

No

NDepend ndpend http://www.ndepend.com

NoFreeacademiclicenceavailable

Sourcemonitor CampwoodSoftware

http://www.campwoodsw.com/sourcemonitor.html

Yes

VSCodeMetrics Microsoft YesBorlandTogether

MicroFocus http://www.borland.com/en-GB/Products/Requirements-Management/Together

No

TICS TIOBE http://www.tiobe.com/tics/fact-sheet/

No

Designite Designite http://www.designite-tools.com NoFreeacademiclicenceavailable

Table12showsthetoolsavailableforsoftwaremetricsin.Net

4.3.3EvaluationofTools

Onlytoolsthatwerefreelyavailable,providedafreetrialperiodorafreeacademic

licencewereconsideredforevaluation.Inadditiontothisthetoolhadtoprovideat

leastoneofthemetricslistedearlierandbeavailableforusein.NetandC#.Onlysix

toolsofthe14foundmettheselectioncriteria,eachofwhichisdiscussedinmore

detailinthissection.

33

Number Name1 VSCodeMetrics2 LocMetrics3 SLOCCount4 NDpend5 SourceMonitor6 Designite

Table13listofsoftwaremetrictoolsevaluated

4.3.3.1VisualStudioCodeMetrics

TheVSCodeMetricstoolisdevelopedbyMicrosoftandbuiltintotheVisualStudio

Integrated Development Environment (IDE). It offers a small range of software

metrics for identifying code quality andmeasuresmetrics on a solution, project,

namespace,type,ormethodlevel.DatacanbeexporteddirectlytoExcelorasaCSV.

4.3.3.2LocMetrics

LocMetrics is a tool formeasuring the various lines of code in a project. It runs

external to the IDEwithaGUIor from thecommand line.The toolhasnotbeen

updatedsinceOctober2007butisstillavailablefordownload.

4.3.3.2SLOCCount

SLOCCount supports severaldifferentprogramming languagesand isopen source

softwarewhichmeansthatthesourcecodecanbemodified.Itonlyconcentrateson

thenumberof linesofcodemetric. Itwas lastupdated inAugust2004but isstill

availablefordownload.

4.3.3.3NDpend

NDpend isacommercial tool thatprovidesextensivesoftwaremetriccoverageas

well as providing graphical reports and trend monitoring. The tool is highly

configurable so that it can be setup to the needs of the project.Metrics can be

calculatedforapplications,assemblies,namespaces,typesandmethods. Itcanbe

installed as a package extension toVisual Studio and all data can be exported in

variousformats.

34

4.3.3.4SourceMonitor

SourceMonitorisafreewareprogramthatoffersmetricsatsourcecodeandmethod

levelsforavarietyofprogramminglanguages.ItoperateswithinaGUIexternalto

theIDEandcandisplaymetricsingraphicaltablesandcharts.Metricscanalsobe

exportedtoXMLorCSV.

4.3.3.5Designite

Designite isacommercial tool thatprovidessoftwaremetricsatsolution,project,

classandmethodlevel.Italsoprovidesinformationonobject-orienteddesigninthe

form of “code smells”, trend analysis and dependencymatrix. The tool is highly

customisableandallowsforresultstobeexportedtoExcel,XMLorCSVfileformats.

Name MI CC CBO LOC LCOM

VSCodeMetrics X X X X LocMetrics X X SLOCCount X NDepend X X X XSourcemonitor X X Designite X X X X

Table14showswhichofthe6toolsselectedsupporteachofthemetricsrequiredbystudy(Xindicatesmetricissupported)

4.3.4Comparison

OutofthesixtoolsevaluatedaboveLocMetricsandSLOCCountweredismissedfor

use intheprojectas theyonlyofferedasmallsubsectionof thesoftwaremetrics

requiredandthedevelopersnolongersupportthem.Theremainingfourtools(VS

CodeMetrics,NDepend, SourceMonitor andDesignite)were comparedusing the

techniquedescribedinthissection,whichwasbaseduponthemethodsproposedby

NovakandRakić(2011).

Firstallthesoftwaretobeusedwasinstalled,eitherasastandaloneproductoras

anextensiontoVisualStudio.Codeanalysiswasrunonthelegacyapplicationthat

wasbeingusedfortheprojectbeforeanynewworkwasdoneinregardstoadding

newfeatureswithTDD.Aftertheinitialanalysiswasruntheresultswereexported

toMicrosoftExcelwherefurtheranalysiswasperformed.

35

Alldatawassavedintoseparatefilesforeachtoolbeingcomparedandtheaverage

valueandstandarddeviationwascalculated forall themetricsbeingused in this

project. Vectors were then created with the metric values for each tool and

compared using two sample T-Tests, which allow for the comparison of data to

determineiftheydifferfromoneanother.Followinggeneralstatisticalconvention,

aresultbelow0.05isconsideredsignificantlydifferent(Craparo,2007).

T-Test VSMetrics Designite SourceMonitor NDepend

VSMetrics 0.53 0.56 0.94

Designite 1.00 0.55

SourceMonitor 0.58

NDepend

Table15T-Testresultsp-value,avalueof<0.05isconsideredsignificantlydifferent

Theresults indicatethatcalculatedmetricvaluesforeachtooldonotsignificantly

differ from one another, which contradicts previous work in this area. In some

instances, the T-Test indicates that the tools give virtually identical results for

comparisonsbetweenVSMetricsandNDependandSourceMonitorandDesignite.

Thestandarddeviationforeachmetricwasalsocalculatedandcanbeseeninthe

tablebelow.TheresultsofthisshowalargevariationinfortheLOCmetrics,which

agreeswiththeresultsNovakandRakíc(2011)found.Thiscouldbeputdowntoan

ambiguity in the calculation of LOC across tools. There does not appear to be a

standardwayofcalculatingthismetric.

Tool MI CC LCOM LOC Avg.LOC CBOVSMetrics 13.63 1.36 - 1296 45.19 16.04Designite - 2.06 0.19 4320 79.52 1.94SourceMonitor - 1.11 - 4330 104.97 -NDepend 1.09 0.28 1448 3.38 -Average 13.63 1.41 0.24 1477 58.27 8.99

Table16standarddeviationforallmetricsandtools,showsalargevarianceforLOCmetricsacrosstools

Inconclusion,thisstudyfoundmixedresultswhencomparingthesoftwaremetric

databetweendifferenttoolsin.Net.Whentakenasasuiteofmetricsnosignificant

36

differencewasfoundbetweenanyofthetoolswhichcontradictspreviouswork,but

ifeachmetricisanalysedindividuallyalargevarianceintheLOCmetriccanbefound,

which indicates a difference in the calculationmethod amongst tools compared.

From these results, it is impossible to concludewhichof the toolsprovidebetter

results,andthiscouldbeanareaforfutureresearch.

Threatstovaliditycouldarisefromthefactthatthisstudyonlyusedasmallsubset

ofthemetricsavailableineachtoolandonlyoneprojectwasusedtocalculatethe

metrics unlike the previous studies which used many different code bases, so

thereforetheremaybelesschancefortheresultstodiffer.

4.3.5SelectionforProject

Alltheproductsthatwerecomparedintheprevioussectionwereusedtotakemetric

readings for the duration of the project. They were all used instead of just one

becausethecomparisonwasunabletodetectwhichtoolwasthemoreeffectiveand

despiteconcludingthatnosignificantdifferenceinthedatawasfoundmostprevious

work in thearea indicatesotherwise (Linckeetal.,2008;NovakandRakíc,2010).

Showing the results of multiple tools over time would allow for a more

comprehensiveconclusionofwhetherTDDimprovesinternalcodequality.

Number Name1 VSCodeMetrics2 NDepend3 SourceMonitor4 Designite

Table17listoftoolsselectedforuseinproject

37

5.LEGACYPROJECT

5.1OverviewofApplication

The existing project that was used for the research is a web application that is

primarilyfocusedonusergeneratedcontent(henceforthknownasversion1.0ofthe

application). It allows for users to create painting guides forminiature wargame

modelsbylettingthemuploadimages,writetextandselectequipmentusedforthe

variousstepsinthepaintingprocess.The1.0versionoftheapplicationincorporated

severalWeb2.0featuressuchascommenting,ratingandsharingofusergenerated

content.

Version1.0wasdevelopedusingASP.NetMVC5inC#forthebackendwithaSQL

databaseandwithHTML,CSSandJQueryforthefrontendusingaBootstraptheme.

It containednounit testsor anyother formof testing and therefore fulfilled the

definitionofalegacyapplicationasputforthbyMichaelFeathers(2004),whostates

thatanycodewithoutunittestscanbeconsideredaslegacy.

5.2DevelopmentPlan

The new version of the application (version 2.0) was developed inside an agile

methodology(SpecificallyDisciplinedAgileDevelopment)usingTDDto implement

newfeatures.Theprojectwasdevelopedover10weekswith5iterationseachtwo-

weeks in length, followed by a one-week transition phase to finalise the

development.

Before development began a one-week inception phase took place in which the

initialplanningoftheprojecttookplace.Firstaprojectschedulewasproducedinthe

formofaGanttchartthatshowedthedurationofeachphaseofdevelopment.

38

ID TaskName Start Finish DurationSep2016 Oct2016 Nov2016

4/9 11/9 18/9 25/9 2/10 9/10 16/10 23/10 30/10 6/11 13/11 20/11 27/11

1 7d11/09/201605/09/2016Inception

2 14d25/09/201612/09/2016Iteration1

3 14d09/10/201626/09/2016Iteration2

4 14d23/10/201610/10/2016Iteration3

5 14d06/11/201624/10/2016Iteration4

6 14d20/11/201607/11/2016Iteration5

7 7d27/11/201621/11/2016Transition

Figure14Ganttchartshowingtheschedulefordevelopment

Thenalistofstakeholdersfortheapplicationwasproducedforwhichuserstories

couldbegenerated.

Stakeholder DescriptionAdmin Theadmincanmanageandcuratethecontentonthesiteaswell

asregisteredusers.LoggedinUser

Aloggedinusercancreatecontentforthesiteaswellasinteractwithotherusers.

User Ausercanviewthecontentonthesitebutcannotcreatecontentorinteractwithotherusers.

Table18showsallstakeholdersfortheprojectthatcanhaveuserstoriesassignedtothem

Alistofuserstorieswasthenmadeforanynewfeaturesthatweretobeaddedand

givenaprioritybaseduponrelativecomplexityandeffortneeded.Alistofknown

bugswasalsocreatedandeachgivenapriority,anditwasdecidedthatfixingthese

shouldbeaddedintodevelopment.

ID UserStory PriorityPointsUS1 AsanadminIcanloginsecurelysothatIcan

administerthesitescontent3

US2 AsanadminIcancuratethesitescontentsothatIcanremoveanyunwanteditems

8

US3 AsanadminIcanseealluseraccountssothatIcanseewhoisusingthesite

8

US4 AsanadminIcanmodifyuseraccountssothatIcanbanorsuspendunwantedusers

13

US5 AsanadminIcanmodifycommentssothatIcanremoveoreditanythinginappropriate

13

US6 AsaloggedinuserIcanflagcommentssothatIcanreportanythinginappropriate

5

US7 AsaloggeduserIcanflagpaintguidessothatIcanreportinappropriatecontent

2

39

US8 AsauserIcanfiltercontentonthesitesothatIcaneasilyfindcontentthatIlike

2

US9 AsaloggedinuserIcancreateaprofilesothatIcangivemyselfanidentityonthesite

5

US10 AsaloggedinuserIcanaddpaintguidestomyfavouritessothatIcanpreservealistofallthecontentIlike

3

US11 AsauserIcanviewlistsofthetoppaintguidessoIcanviewthebestcontentonthesite

1

US12 AsaloggedinuserIcansendmessagestootheruserssothatIcancommunicateprivately

8

US13 AsaloggedinuserIcanparticipateinforumssothatIcantakeamoreactivepartinthesitecommunity

20

US14 AsanadminIcangetdetailedstatsofsitecontentusesothatIcanseewhatishappeningonthesite

13

US15 AsaloggedinuserIcanratecommentssothebestcommentscanbeeasilyviewed

8

Table19alluserstoriesforthenewfeaturestobeaddedatthestartofdevelopment.Storiescouldbechangedoraddedrightupuntilimplementationoftheitem.

ID Bug Priority1 Sortorderisnotpreservedwhenswitching

theviewtypeonguidespage2

2 Itemsonguidespagearenotseparatedintopages

3

3 Editingofapaintguideresultsinanunhandlederrorexception

1

4 Creatingapaintguidecausesanerror 1Table20listofknownbugsinthepreviousversion

ThesewerethenaddedintoMicrosoft’sagileplanningtoolonVisualStudioOnline

anditwascalculatedthattherewasaworkitemestimationof150points(allowing

extraforcontingencysuchasadditionalfeaturesneedingtobeaddedorproblems

arising),whichmeantthat30pointsworthofuserstoriesneededtobecompleted

eachiteration.

40

Before the first iteration began and at the end of each iteration softwaremetric

readings were taken so that any differences in code quality could be tracked

throughoutthelifecycleofdevelopment.

41

6.DEVELOPMENTCHALLENGESANDSOLUTIONSThelegacyapplicationcontainednoformoftestingsothefirstthingthatwasneeded

wasforatestprojecttobesetupinsidethesolution.Thiswasdonesothatunittests

could be written for the project. The first problem occurred during the

implementationofthefirstuserstory,whichwasfunctionalitytoallowtheuserto

loginasanadministrator.Theissuewasthatunittestscouldnotbewrittenforthe

projectwithoutinteractingdirectlywiththedatabase,thiswasbecausethebusiness

anddatalayersweretoocloselycoupledandalayerofabstractionbetweenthetwo

did not exist. The dependencies needed to be broken before unit tests could be

addedandnew functionality implemented.Threeapproaches fordoing thiswere

identified:

• Introducearepositoryandunitofworkdesignpattern• FakethedatabasecontextbyaddinginterfacestoDBSetandapplicationContext,

whichwouldallowforusinganalternativecontexttotestwith• Useatestdoubleframeworktomockthecontextdirectlywithoutchangingany

legacycode

As TDD dictates to implement the simplest possible solution first and the best

practice is to avoid changing legacy codeuntil necessary, the third approachwas

initially attempted.However, this did notwork as expectedbecauseof problems

encountered trying tomock non-interface classeswhichwould lead to having to

changetheimplementationofthedatabasecontextandthisforcedamovetothe

repositoryandunitofworkapproachinstead,asthiswouldleadtomakingtestsfor

otherpartsoftheapplicationeasiertowriteinthefuture.

Therepositoryandunitofworkpatternisintendedtocreateandabstractionlayer

betweenthedatalayerandthebusinesslogiclayer.Thispatterncanhelpprotectthe

applicationfromchangesinthedatastoreandcanhelptofacilitateTDD(Dykstra,

2013).

42

Figure15differencesbetweenapplicationwithnorepositoryandwithrepositoryandunitofworkpattern

(Dykstra,2013)

Asthistechniquewasgoingtochangelegacycodeandbecauseunittestscouldnot

yetbewrittenintegrationtestswereneededtogettheclassestobechangedunder

atestharness.ThiswasdoneusingSeleniumwebdrivertocheckelementsonthe

live website, and the tests could be run after refactoring took place to ensure

functionalityonthewebsiteremainedthesame.

Therepositoryandunitofworkpatternwasthenimplementedwithinterfacesfor

botharepositoryandunitofworkclass.Usinginterfacesallowedfortheclassesto

beeasilymockedor faked for testingbyusingdependency injection to inject the

objectintotheclassesthatneededtoconsumethem.Afakeimplementationofthe

unit ofwork and repository could insteadbe injectedduring testing and the real

versionforproduction,thisallowedforseparatingouttheconcernsoftheunittests.

43

interface IUnitOfWork : IDisposable {

int Save(); IGenericRepository<ApplicationUser> UserRepository { get; } IGenericRepository<Guides> GuidesRepository { get; } IGenericRepository<Comments> CommentsRepository { get; } }

Figure16firstversionoftheIUnitOfWorkinterface,whichhadarepositoryforalldatatablesintheapplication(notallimplementedhere)

public interface IGenericRepository<TEntity> where TEntity : class { IEnumerable<TEntity> Get(Expression<Func<TEntity, bool>> filter, Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy, string includeProperties); IEnumerable<TEntity> GetAll(); TEntity GetById(object id); void Insert(TEntity entity); void Delete(object id); void Delete(TEntity entityToDelete); void Update(TEntity entityToUpdate); int SaveChanges(); }

Figure17firstversionoftheIGenericRepositoryinterface

Dependenciesbetweenthedataandbusinesslogiclayerswerenowbrokensonew

featurescouldbeaddedviaTDD.Afteraddingsomenewfunctionality,amodelclass

fromthelegacyapplicationneededtobechangedandthisrefactoringcouldbedone

withconfidencebecauseunittestswerenowinplaceforthisareaoftheapplication.

Theconfidenceinbeingabletosecurelyrefactorisoneofthegreatadvantagesof

TDDovertraditionalsoftwaredevelopmenttechniques.

Theprocessofabstracting thedata layerandbreakingdependencies so thatunit

testscouldbewrittenwasnoteasyandconsiderablysloweddownthedevelopment

ofnewfeatures.Alongtimewasspentfindingseampoints,writingintegrationtests

and refactoring old code to the detriment of being able to add new features.

However,oncethisworkhadbeendonethebenefitscouldbeseenandconfidence

begantogrowintheabilitytochangethelegacycodewithoutconsequences.Itwas

temptingatthisstagetomovethewholeprojectoverttothenewdesignpattern,

butTDDemphasisthatonlythebareminimumshouldbedonetomaketestspass,

44

andthiswouldhaverequiredwritingcodeandtestsforpartsoftheapplicationthat

didnotcurrentlyneedtoworkedon.

The next problem encountered was when trying to implement a suspend user

method that relied heavily upon the .Net framework classes UserStore and

UserManager.Itwasdifficulttodeterminewheretoplacedependenciesforthese

objectsinmyownclassesanditwashardtotestcallstothemwithoutmockingas

theyaccessthedatabasedirectly.Thefirstsolutiontriedwastofaketheapplication

context in the unit tests so that when the framework objects were called they

accessedthefakecontextbutthisdidnotworkasintendedbecauseofnon-virtual

propertiesthatcouldnotbemocked.ThesecondsolutionwastoinjecttheUserStore

andUserManagerintotherepositorySuspendmethoddirectlyandthiswasdonevia

anextensionsothatnotallchildrenoftherepositoryinterfacehadtoimplementit.

However,this leadtoproblemswithmockingasextensionmethodsarenoteasily

mockedbymostmockingframeworks,somesolutionstothisweretoeithercreatea

wrapperclass,useMoleswhichisanMSisolationframeworkortomovethemethod

directly into the interface. The last option was used although this was not ideal

becauseallclassesthat implementedtherepository interfacewouldnowneedto

haveasuspendusermethodevenifnotneeded.

public interface IGenericRepository<TEntity> where TEntity : class { IEnumerable<TEntity> Get(Expression<Func<TEntity, bool>> filter, Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy, string includeProperties); IEnumerable<TEntity> GetAll(); TEntity GetById(object id); void Insert(TEntity entity); void Delete(object id); void Delete(TEntity entityToDelete); void Update(TEntity entityToUpdate); int SaveChanges(); bool SuspendUser(string id); void ReinstateUser(string id); }

Figure18SusependUserandReinstateUsermethodsaddedtotherepositoryinterface

Theseconditerationofdevelopmentwaseasiertoimplementbecauselotsofthe

dependencieshadalreadybeenbrokeninthefirstiteration.However,aproblemwas

encounteredwhentryingtoaddnewfunctionalitytotheGuidesController,where

45

somemethodsrelieduponthe.NetframeworkclassIIdenity.Themethodschecked

theuser’sidentitybeforeproceedinganddidthisviaacalltoGetUserId(),whichis

anextensionmethodtoIIdentityclass,andtheproblemoccurredbecausethischecks

thedatasourcedirectlyandcannotbemockedasitisanextensionmethod.Aseam

wasfoundbysearchingthroughthedocumentationtofindthatGetUserId()relied

upontheClaimclasssoitwaspossibletocreateastubbedclaimandthenmockthe

HttpContextBaseclasstoreturnthiswhenGetUserId()wascalledinthecontroller.

private void SetFakeHttpContext(string id) { var identity = new GenericIdentity("test id");

identity.AddClaim(new Claim("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameident

ifier", id)); var principal = new GenericPrincipal(identity, new[] { "user" }); var context = new Mock<HttpContextBase>(); context.Setup(s => s.User).Returns(principal); var controllerContext = new Mock<ControllerContext>(); controllerContext.Setup(t => t.HttpContext).Returns(context.Object); _controller.ControllerContext = controllerContext.Object; }

Figure19methodformockingthecontrollercontextsoGetUserId()returnsavalidvaluewithoutcallingthedatabase

Duringthethirditeration,itwasdecidedtoutiliseadependencyinjectionframework

andNinjectwasusedforthis.Thisallowedfordependenciestobeboundanddefault

constructorstoberemovedfromcontrollers.AnuanceoftheMVCframeworkmeans

thatdefaultconstructorsarealwaysusedwithintheframeworkbutNinjectremoves

thisconventionbyautoinjectingthedependencies.AlthoughbeforethisanInversion

of Control patternwas being used to inject dependencies the full benefit of this

regarding couplingof objects couldnot be seen as thedefault constructorwhich

instantiatedtheseobjectswasstillpresentinthecontroller.

Thelastmajorissuetoariseduringdevelopmentwasaproblemwiththerepository

interfacethatwasimplementedearlierintheproject.Asmoreclasseswerebeing

movedovertotheunitofworkpatterntheclasswasbecomingunwieldywithmany

differentrepositorieshavingtobecreatedasproperties,andthiswasleadingtohigh

couplingandlowcohesionoftheclass.Itwasdecidedtorefactorthisclassbymaking

itmore generic, however this lead to hundreds of unit tests failing due to them

mocking the IUnitOfWork interface. This is one of themajor downsides to using

46

mocksandtoTDDingeneral,ifinterfacesarechangedlotsoftestsfail.Despitethis

beingalong-windedprocesstorefactor,itwasrelativelystraightforwarddueinthe

mostparttotheteststhemselves,andanautomatedrefactoringtoolhelpedgreatly

hereduetoitpointingouterrorsandidentifyingalltheplacesthatchangesneeded

tobemade.

public interface IUnitOfWork : IDisposable {

int Save(); IGenericRepository<T> Repository<T>() where T : class; }

Figure20newIUnitOfWorkclassthatusesgenerics

Overallmany difficulties are encounteredwhenworkingwith TDD in .Net legacy

code,butnothingthatcannotbesolved.However, lotsofhoopsmustbe jumped

through to get tests working for what seem to be the most basic of controller

methods.Itisoftenhardtodeterminewhattotestwhenworkingwithaframework

thatautogeneratescodeandthere isafine linebetweentestingtheapplications

owncodeandthatoftheframework,itisnotoftenobvioushowtogoabouttesting

codethatcallframeworkmethods.Anotherdownsideisthatlotsoftestcodeneeds

tobewritten, insome instancesmanymore lines than theproductioncode fora

method.Findingseamsintheframeworktobreakdependenciesisadifficultjoband

takes time,butoneof themajorupsides todoing this is theability to refactor in

confidencewhentestsareinplace.

47

7.RESULTS

7.1SoftwareMetrics

Thissection looksattheresultsforeachofthesoftwaremetricsrecordedforthe

project.Wherepossibleanaveragevaluetakenfromallthetoolsusedtorecordthe

dataispresented.

7.1.1MaintainabilityIndex(MI)

OnlytheVSCodeMetrictoolrecordedMIandthismetricshowsaslight increase

overthecourseofdevelopment.MIinitiallydroppedduringthefirsttwoiterations

butshowedamarkedincreaseaftertheseconditeration.AT-testbetweentheset

ofdatarecordedatthestartandthatafterthefifthiterationrevealednosignificant

differenceintheresults.

Figure21showsthatMIinitiallydecreasesbeforeimproving

7.1.2CyclomaticComplexity(CC)

TheresultsforCCareinconclusive,withtwotoolsrecordingamarginalincreaseand

theothertwotoolsrecordingamarginaldecrease.Theaveragevaluesacrossalltools

showamarginal increase inCCof0.0075.OverallCChasbeenmaintainedat the

samelevelthroughoutdevelopment.ThevaluesrecordedforalltoolsarelowforCC

86.4

86.6

86.8

87

87.2

87.4

87.6

87.8

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

Maintainability Index

VSCodeMetrics

48

meaning thatapplicationstartedwitha lowcomplexity formethodsandthiswas

maintainedduringthedevelopmentwithTDD.

7.1.3LinesofCode(LOC)

AlltoolsusedrecordedaLOCmetricandasexpectedthisvalueincreasedasmore

featureswereadded.However,whatwassurprising is that theaverageLOCfora

classalsoincreasedduringdevelopmentandalltoolsshowedagreateraverageLOC

valueat theend.Thisgoesagainst theconsensus thatTDDproduces smallerand

morecohesiveclasses.Nosignificantdifferenceintheresultsbetweenfirstandlast

readingswereseen.

1.41.451.51.551.61.651.71.751.81.851.91.95

2

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

CyclomaticComplexity

VSCodeMetrics Ndepend SourceMonitor

Designite AverageForAllTools

49

Figure22LOCshowsanincreaseduringdevelopmentaswouldbeexpectedasnewfeaturesareadded

Figure23averageLOCperclassalsoincreaseswhichisunexpected

7.1.4LackofCohesionofMethods(LCOM)

TheresultsforLCOMdifferbetweentoolsandarethereforeinconclusive.Onlytwo

ofthetoolsusedrecordedLCOM.NDependshowedadecreaseof0.11andDesignite

showedanincreaseof0.03betweenfirstandlastreadings.Theaveragevaluefor

bothtoolsshowedanoveralldecreaseacrossdevelopment.Thevaluesfromboth

toolsareconsideredlowforLCOMandthereforetheclassesintheapplicationhave

0

1000

2000

3000

4000

5000

6000

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

LinesofCode

VSCodeMetrics Ndepend SourceMonitor

Designite AverageForAllTools

102030405060708090

100

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

AverageLinesofCodeperClass

VSCodeMetrics Ndepend SourceMonitor

Designite AverageForAllTools

50

goodcohesion.Thereisasignificantdifferencebetweenthefirstandlastreadings

fortheNDependtool,withaPvalueof0.000453foratwosampleT-Testofthedata,

however,thereadingsfortheDesignitetooldonotshowasignificantdifference.

7.1.5CouplingBetweenObjects(CBO)

All the tools that recorded a CBOmetric showed an increase over the course of

development.CBOincreasedwitheveryreading,however,noneofthetoolsshowa

significantdifferencebetweenthefirstandlastiterations.Twoofthetools(VSCode

Metrics and Designite) recorded a small increase in CBO over the course of

developmentbutNDependshowedanincreaseof8.85withthelargestleapcoming

attheendofthefirstiterationwhentherepositoryandunitofworkdesignpattern

wasintroduced.

0.050.060.070.080.090.10.110.120.130.140.150.160.170.18

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

LackofCohesionofMethods

VSCodeMetrics Ndepend SourceMonitor

Designite AverageForAllTools

51

Figure24alltoolsshowanincreaseinCBOthroughoutdevelopment

7.1.6Overall

Theresultsfromthesoftwaremetricsreadingsareconsistentwiththemostofthe

previousworkintheareainthefactthattheyshowthatTDDhasaminimalimpact

uponinternalcodequality.Mostofthemetricsrecordeddidnotshowasignificant

differencebetweenthereadingstakenatthestartandattheendofdevelopment.

OnlytheLCOMmetricshowedasignificantimprovement,however,thiswasonlyon

oneofthetoolsthatrecordedthismetric,theothertoolshowedanon-significant

declineincohesionsotheoverallresultforLCOMisinconclusive.

PreviousstudieshaverecordedasmallpositiveincreaseinCCbutthisstudyshows

thatCClargelyremainedthesamethroughoutdevelopment.TheresultsforMIdiffer

to those found by Kollanus (2011) in his systematic review that concludedmost

studiesfoundTDDproducedcodethatisdifficulttomaintain.

Overall the results show that TDD had aminimal impact upon the internal code

quality of the product, with all the average values across all code metric tools

showingnosignificantdifferenceafterintroducingTDDintotheproject.

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

Start 1stIteration 2ndIteration 3rdIteration 4thIteration 5thIteration

CouplingBetweenObjects

VSCodeMetrics Ndepend SourceMonitor

Designite AverageForAllTools

52

Start 1st

Iteration

2nd

Iteration

3rd

Iteration

4th

Iteration

5th

Iteration

MI 87.12 86.98 86.87 87.51 87.36 87.58

LOC 2846.50 3269.00 3155.50 3565.75 3576.50 3557.50

Avg.LOC 41.96 41.90 49.66 50.71 50.71 52.00

CBO 5.24 6.61 7.33 7.92 8.15 8.97

LCOM 0.11 0.12 0.09 0.09 0.08 0.07

CC 1.59 1.56 1.62 1.58 1.60 1.60

Table21averagevaluesforallmetricsrecordedduringdevelopment

7.2CodeTestCoverage

Test coverage was recorded using JetBrain DotCover tool which integrates with

Visual Studio and provides and analysis of the amount of lines of code in your

application thatare coveredbyunit tests. The legacyapplication startedwith0%

code coverage as it had no formof testing in place. After the fifth iteration, the

applicationhad46%codecoveragefromunittests.

Thepercentageofcodecoveredbyunittestsincreasedduringeachiteration,with

thelargestincreasecomingduringthefirsttwoiterationsofdevelopment.Thiswas

whenmostof the legacy code thatneeded tobeusedwasbroughtunder a test

harness.Duringsubsequentiterations,asmallerbutmoreconsistentincreasecanbe

seen.Thiscanbeputdowntoonlynewfeaturesbeingplacedundertestandnotas

muchlegacycode.

53

Figure25showstheincreaseincodecoverageduringthedevelopmentlifecycle

Newnamespacesthatwereaddedtotheprojectshowa100%coverageratebythe

endofthe5thiteration(exceptWSFinal.App_Start,whichwasautogeneratedbythe

Ninjectframework).ThisshowsthatwhenintroducingnewcodeTDDpromoteshigh

testabilityfromthestartandleadstoanincreaseindeveloperconfidencebecause

codeiscompletelytested.

Start

1st

Iteration

2nd

Iteration

3rd

Iteration

4th

Iteration

5th

Iteration

WSFinal.App_Start - - 0% 0% 0% 0%

WSFinal 0% 0% 0% 0% 0% 0%

WSFinal.Migrations 0% 0% 0% 0% 0% 0%

WSFinal.Controllers 0% 0% 18% 25% 28% 30%

WSFinal.Models 0% 49% 59% 69% 69% 71%

WSFinal.Controllers.Admi

n

- 94% 96% 99% 99% 100%

WSFinal.Helpers - - 100% 100% 100% 100%

Total 0% 11% 30% 37% 41% 46%

Table22showsthepercentageofcodecoveredbyunittestsforeachnamespaceintheapplication

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

1 2 3 4 5 6

UnitTestCoverage

54

ThesefindingsagreewithmostofthepreviousworkonTDDandsoftwarequality,in

that TDD introduces a significant improvement in code coverage from testing.

However,thismetricdoesnotshowthequalityoftheunittestsitonlyrecordsifa

testhascoveredalineofcodebutnothowwellithasbeencoveredandwhetherall

edgecaseshavebeentested.

55

8.CONCLUSION

8.1SummaryandEvaluationThe project set out to determine if introducing TDD into a legacy application

improvedtheinternalcodequalityanditshowedbyusingsoftwaremetricsthatit

hadaminimalimpact.Alegacyapplicationdevelopedinthe.NetMVCframework

wasextendedand refactoredwherenecessaryusingTest-DrivenDevelopment to

implement new features. Softwaremetricmeasurementswere taken throughout

developmentandtheresultsshowedthatTDDhadaminimalimpactuponthequality

ofthecodeintheproject,notallofwhichwerepositive,withmostofthemetrics

displayingnon-significantchangesfromthestartofdevelopmenttothelastiteration.

The results inpart agreewithmostpreviouswork in theareabut goagainst the

consensusofTDDadvocatesandpioneerssuchasBeck(2002)whoclaimthatTDD

producessmallcohesiveclassesthatarenotcomplex.However,itcouldbeargued

thatthelegacycodeinthisinstancewasalreadyofahighqualityandTDDhelpedto

maintainthis.

ThereportidentifiedthechallengesthatintroducingTDDtoa.Netapplicationcan

bring and discovered that most problems encountered are related to breaking

dependenciesandfindingseampointsinthecodetobeabletowriteunitteststhat

donot interactwith thedata layer.The report identifiedsomesolutions to these

problemsbypinpointingways to separateconcernsof theapplication layers. It is

possibletoovercomeallthechallengeswhenworkingwithlegacy.Netcodeinthe

MVCframeworkbutsometimesunnecessaryworkneedstobecarriedouttodoso.

The report also examined the tools available for implementing TDD. From

experiments taken it was discovered that there was no significant difference

betweenunit test frameworksand test runnerswhenanalysing the speedof test

runs.

It is inconclusive from this report if it isworthwhile toextenda legacy .NetMVC

applicationwithTDD.Ononehandtestcoveragesignificantly increaseswhichcan

lead to increased confidence when working with the legacy code regarding

refactoringbut on theother hand TDD leads to complex challenges having to be

56

overcome and does not show a significant improvement in the quality and

maintainabilityofthecode.

8.2FutureWorkDuringthisreport,itwasbroughttolightthatfurtherworkisneededtoinvestigate

theimpactofTDDoninternalcodequalityinlegacyapplicationsbeforeanyconcrete

conclusionscanbedrawn.Theworkcarriedoutinthisreportcouldbeextrapolated

intoalargerscaleprojectwithagreatertimescaletoseeiftrendscanbefoundwhen

moredevelopmentworkiscarriedout.Itwouldalsobedesirabletoinvestigatethe

impactupondifferentquality legacyapplications,for instancea legacyapplication

thathaspoorqualitysoftwaremetricsfromtheoutsettoseeifTDDimprovesthe

qualityupthelevelsseeninthisreport.

There is also scope forwork comparing different development approacheswhen

workingwithlegacycodetoseewhichhasthegreaterimpactupontheinternalcode

quality.AcomparisonbetweenaTest-LastDevelopmentandTDDusingtwodifferent

teamstodevelopthesamelegacyapplicationwiththesameuserstorieswouldbea

worthwhileexperiment.

Furtherworkisalsoneededoncomparingunittestframeworkswithamuchlarger

testsuitethanusedinthisprojectandacrossmultiple.Netapplicationtypestosee

ifanydifferencesintheframeworksandrunnerscanbeestablishedandwhythisis

thecase.

57

REFERENCESBSISO/IEC25023(2016)Systemsandsoftwareengineering.SystemsandsoftwareQuality

Requirements and Evaluation (SQuaRE). Measurement of system and software product

quality2016.BritishStandardsInstitute.

Ambler, S. (2013a) How Agile Are You? 2013 Survey Results. Available at:

http://www.ambysoft.com/surveys/howAgileAreYou2013.html[Accessed:5June2016].

Ambler, S. (2013b) Introduction to Test Driven Development (TDD). Available at:

http://agiledata.org/essays/tdd.html[Accessed:5June2016].

Andersson,M.andVestergren,P.(2004)Object-OrientedDesignQualityMetrics.Masters

thesis.UppsalaUniversity

Aniche,M.andGerosa,M.A.(2015)‘Doestest-drivendevelopmentimproveclassdesign?A

qualitative study on developers’ perceptions’. Journal of the Brazilian Computer Society.

21(1).pp.1-11.

Bakar,N. S. A. A. andBoughton, C.V. (2012) ‘Validation ofmeasurement tools to extract

metrics fromopensourceprojects’.2012 IEEEConferenceonOpenSystems(ICOS).21-24

October2012.KualaLumpur,Malayasia.IEEEXplore.

Basili,V.R.,Briand,L.C.andMelo,W.L.(1996)‘Avalidationofobject-orienteddesignmetrics

asqualityindicators’.IEEETransactionsonSoftwareEngineering.22(10).pp.751-761.

Beck,K.(2002)TestDrivenDevelopment:ByExample.Addison-WesleyProfessional.

Bissi,W.,SerraSecaNeto,AdolfoGustavoandEmer,MariaClaudiaFigueiredoPereira(2016)

‘Theeffectsoftestdrivendevelopmentoninternalquality,externalqualityandproductivity:

Asystematicreview’.InformationandSoftwareTechnology.74.pp.45-54.

C. Klammer and A. Kern (2015) ‘Writing unit tests: It's now or never!’. IEEE Eighth

International Conference on Software Testing, Verification and Validation Workshops

(ICSTW).13-17April2015.Graz,Poland.IEEEXplore.

Chidamber, S.R. and Kemerer, C.F. (1994) ‘Metrics suite for object oriented design’. IEEE

TransactionsonSoftwareEngineering.20(6).pp.476-493.

Chidamber,S.R.&Kemerer,C.F.(1991)Towardsametricssuiteforobjectorienteddesign.

OOPSLA1991ConferenceproceedingsonObject-orientedprogrammingsystems,languages,

andapplications.6-11October1991.Phoenix,USA.ACM.

58

Choudhari,J.andSuman,U.(2015) ‘AnEmpiricalEvaluationof IterativeMaintenanceLife

CycleUsingXP’.SIGSOFTSoftw.Eng.Notes.40(2).pp.1–14.

Choudhari, J. and Suman, U. (2014) ‘Extended Iterative Maintenance Life Cycle Using

eXtremeProgramming’.SIGSOFTSoftw.Eng.Notes.39(1).pp.1–12.

Coleman,D.,Ash,D.,Lowther,B.andOman,P.(1994)‘Usingmetricstoevaluatesoftware

systemmaintainability’.Computer.27(8).pp.44-49.

conorm (2007) Maintainability Index Range and Meaning. Available at:

https://blogs.msdn.microsoft.com/codeanalysis/2007/11/20/maintainability-index-range-

and-atmeaning/[Accessed:21November,2016].

Craparo, Robert M. Significance Level. (2007). In: Encyclopaedia of Measurement and

Statistics,1sted.ThousandOaks,CA:SAGEPublications.pp.889-891.

Crispin,L.(2006)‘DrivingSoftwareQuality:HowTest-DrivenDevelopmentImpactsSoftware

Quality’.IEEESoftware.23(6).pp.70-71.

Desai,C.,Janzen,D.andSavage,K.(2008)‘Asurveyofevidencefortest-drivendevelopment

inacademia’.ACMSIGCSEBulletin.pp.97–101.

Dykstra,T.(2013)ImplementingtheRepositoryandUnitofWorkPatternsinanASP.NETMVC

Application. Available at: https://www.asp.net/mvc/overview/older-versions/getting-

started-with-ef-5-using-mvc-4/implementing-the-repository-and-unit-of-work-patterns-in-

an-asp-net-mvc-application[Accessed:26September,2016].

E.Shihab,Z.M.Jiang,B.Adams,A.E.HassanandR.Bowerman(2010)‘PrioritizingUnitTest

Creation for Test-Driven Maintenance of Legacy Systems’. 2010 10th International

Conference on Quality Software. 14-15 July 2010. Washington DC, USA. IEEE Computer

Society.

Feathers,M.(2004)WorkingEffectivelywithLegacyCode.PrenticeHall,UnitedStates.

Fenton,N.&Bieman, J. (2014)SoftwareMetrics:ARigorousandPracticalApproach.CRC

Press.

Fowler, M. (2007) Mocks Aren't Stubs. Available at:

http://martinfowler.com/articles/mocksArentStubs.html[Accessed:August12,2016].

Guerra,E.(2014)‘DesigningaFrameworkwithTest-DrivenDevelopment:AJourney’. IEEE

Software.31(1).pp.9-14.

59

Halstead,M.H.(1977)Elementsofsoftwarescience.ElsevierNewYork.

Henderson-Sellers, B. (1996)Object-orientatedmetrics:measures of complexity. Prentice

Hall,HemelHempstead.

ITJobsWatch (2016)TDD Jobs. Available at: http://www.itjobswatch.co.uk/jobs/uk/tdd.do

[Accessed:4June,2016].

Janzen,D.S.andSaiedian,H.(2008)‘DoesTest-DrivenDevelopmentReallyImproveSoftware

DesignQuality?’.IEEESoftware.25(2).pp.77-84.

Janzen, D.S. and Saiedian, H. (2006) ‘On the Influence of Test-Driven Development on

SoftwareDesign’.19thConferenceonSoftwareEngineeringandTraining2006.19-21April

2006.TurtleBay,Hawaii.IEEEXplore.

Jeremiah, J. (2015)Agile vs.waterfall: Survey shows agile is now the norm. Available at:

http://techbeacon.com/survey-agile-new-norm[Accessed:4June,2016].

Jiau, H.C. and Chen, J.C. (2009) ‘Test code differencing for test-driven refactoring

automation’ACMSigsoftSoftwareEngineeringNotes.34(1).pp.1–10.

Kaufmann,R.andJanzen,D.(2003)‘Implicationsoftest-drivendevelopment:apilotstudy’.

OOPSLA '03Companionof the18thannualACMSIGPLANconferenceonObject-oriented

programming, systems, languages, andapplications26-30October 2003.NewYork,USA.

ACM.

Kollanus, S. (2011) ‘Critical Issues on Test-Driven Development’. 12th International

ConferenceonProduct-focusedsoftwareprocessimprovement(PROFES)2011.20-22June

2011.TorreCanne,Italy.Springer.

Latorre,R.(2014)‘EffectsofDeveloperExperienceonLearningandApplyingUnitTest-Driven

Development’IEEETransactionsonSoftwareEngineering.40(4).pp.381-395.

Li,W.andHenry,S.(1993)‘Object-orientedmetricsthatpredictmaintainability’.Journalof

SystemsandSoftware.23(2).pp.111-122.

Lincke,R.,Lundberg,J.andLöwe,W.(2008)‘ComparingSoftwareMetricsTools’.ISSTA'08

Proceedingsofthe2008internationalsymposiumonSoftwaretestingandanalysis.20-24

July.Seattle,USA.ACM.

60

M.Mortensen,S.GhoshandJ.M.Bieman(2006)‘TestingDuringRefactoring:AddingAspects

toLegacySystems’.200617thInternationalSymposiumonSoftwareReliabilityEngineering.

7-10November2006.Raleigh,USA.IEEEXplore.

Mancl, D., Fraser, S.D. and Opdyke, B. (2011) ‘Workshop: Beyond Green-field Software

Development:Reuse,Recycle,Refactor’.ProceedingsoftheACMInternationalConference

Companion on Object Oriented Programming Systems Languages and Applications

Companion.22-27October2011.Portland,USA.ACM.

Marinescu, R. (2005) ‘Measurement and quality in object-oriented design’. 21st IEEE

InternationalConferenceonSoftwareMaintenance(ICSM'05).25-30September2005.Los

Alamitos,USA.IEEEComputerSociety.

Martin,M.andMartin,R.C.(2006)AgilePrinciples,Patterns,andPractices inC#.Prentice

Hall,UnitedStates.

Martin,R. (1994) ‘OODesignQualityMetrics -AnAnalysisofDependencies’.Availableat:

https://linux.ime.usp.br/~joaomm/mac499/arquivos/referencias/oodmetrics.pdf

[Accessed:10October,2016].

Martin,R.C.(2003)Agilesoftwaredevelopment:principles,patterns,andpractices.Prentice

HallPTR.

McCabe, T.J. (1976) ‘A ComplexityMeasure’. IEEE Transactions on Software Engineering.

2(4).pp.308-320.

Mens, T. and Tourwe, T. (2004) ‘A survey of software refactoring’. IEEE Transactions on

SoftwareEngineering.30(2).pp.126-139.

Meszaros, G. (2007) xUnit Test Patterns: Refactoring Test Code. Addison-Wesley

Professional.

Mitchell, A. and Power, J. (2005) ‘Using object-level run-time metrics to study coupling

betweenobjects’.Proceedingsof the2005ACMsymposiumonAppliedcomputing.13-17

March2005.SantaFe,USA.ACM.

Mortensen,M.,Ghosh,S.andBieman,J.M.(2008)‘Atestdrivenapproachforaspectualizing

legacysoftwareusingmocksystems’,InformationandSoftwareTechnology.50(7).pp.640.

61

Mü,M.M.(2006)‘Theeffectoftest-drivendevelopmentonprogramcode’.Proceedingsof

the7thinternationalconferenceonExtremeProgrammingandAgileProcessesinSoftware

Engineering.17-22June2006.Oulu,Finland.Springer-Verlag.

NDepend(2016)CodeMetricsDefinitions.Availableat:http://ndepend.com/docs/getting-

started-with-ndepend[Accessed:23November,2016].

Novak,J.andRakić,G.(2010)‘Comparisonofsoftwaremetricstoolsfor:net’.Proceedingsof

13th International Multiconference Information Society. 11-15 October 2010. Ljubljana,

Slovenia.

Oman, P. and Hagemeister, J. (1992) ‘Metrics for assessing a software system's

maintainability’.ConferenceonSoftwareMaintenance1992.9-12November1992.Orlando,

USA.IEEEComputerSociety.

Osherove,R.(2009)TheArtofUnitTesting:withExamplesin.NET.ManningPublications.

Pančur,M.andCiglarič,M.(2011)‘Impactoftest-drivendevelopmentonproductivity,code

andtests:Acontrolledexperiment’.InformationandSoftwareTechnology.53(6).pp.557-

573.

Pressman,R.S.andInce,D.(1997)Softwareengineering:apractitioner'sapproach.McGraw-

Hill,London.

Rosenberg, L.H. and Hyatt, L.E. (1997) ‘Software quality metrics for object-oriented

environments’.CrosstalkJournal.10(4).pp.1-6.

S.NairandP.Ramnath(2005)‘Teachingagoliathtofly[PrimaveraSystemsadoptionofagile

methodologies]’. AgileDevelopment Conference (ADC'05). 24-29 July 2009.Denver,USA.

IEEEComputerSociety.

Sanchez,J.C.,Williams,L.andMaximilien,E.M.(2007)‘OntheSustainedUseofaTest-Driven

DevelopmentPracticeatIBM’.AgileConference2007.13-17August2007.WashingtonDC,

USA.IEEEComputerSociety.

Seemann,M.(2011)DependencyInjectionin.NET.ManningPublications.

Shull,F.,Melnik,G.,Turhan,B.,Layman,L.,Diep,M.andErdogmus,H.(2010)‘WhatDoWe

KnowaboutTest-DrivenDevelopment?’.IEEESoftware.27(6).pp.16-19.

Siniaalto,M.andAbrahamsson,P.(2007)‘AComparativeCaseStudyontheImpactofTest-

Driven Development on Program Design and Test Coverage’. Proceedings of the First

62

International Symposium on Empirical Software Engineering and Measurement. 20-21

September2007.Madrid,Spain.IEEEComputerSociety.

Siniaalto, M. and Abrahamsson, P. (2008) ‘Does Test-Driven Development Improve the

Program Code? Alarming Results from a Comparative Case Study’. Balancing Agility and

FormalisminSoftwareEngineering:SecondIFIPTC2CentralandEastEuropeanConference

onSoftwareEngineeringTechniques,CEE-SET2007.10-12October2007.Poznan,Poland.

SpringerBerlinHeidelberg,Berlin,Heidelberg.

Smacchia,P. (2007)HowdoyoucountyournumberofLinesOfCode(LOC)?.Availableat:

http://codebetter.com/patricksmacchia/2007/10/03/how-do-you-count-your-number-of-

lines-of-code-loc/[Accessed:5November,2016].

Vu, J.H., Frojd, N., Shenkel-Therolf, C. and Janzen, D.S. (2009) ‘Evaluating Test-Driven

DevelopmentinanIndustry-SponsoredCapstoneProject’.SixthInternationalConferenceon

InformationTechnology:NewGenerations,2009.27-29April2009. LasVegas,USA. IEEE

ComputerSociety.

Welker, K.D. (2001) ‘SoftwareMaintainability Index Revisited’.CrossTalk - The Journal of

DefenseSoftwareEngineering.pp.18-21.

Winkler,D.,Schmidt,M.,Ramler,R.andBiffl,S.(2012)‘ImprovingUnfamiliarCodewithUnit

Tests: An Empirical Investigation on Tool-Supported and Human-Based Testing’. 13th

InternationalConferenceonProduct-FocusedSoftwareProcessImprovement,PROFES2012.

13-15June2012.Madrid,Spain.SpringerBerlinHeidelberg.

Y.L.Traon,T.Mouelhi,A.PretschnerandB.Baudry(2008)‘Test-DrivenAssessmentofAccess

Control in Legacy Applications’. 2008 1st International Conference on Software Testing,

Verification,andValidation.9-11April2008.Lillehammer,Norway.IEEEComputerSociety.

zainnab (2011) Code Metrics – Maintainability Index. Available at:

https://blogs.msdn.microsoft.com/zainnab/2011/05/26/code-metrics-maintainability-

index/[Accessed:21November,2016].

63

APPENDIXAFullresultsoftheT-Testsforcomparingsoftwaremetricstoolsandreadingsforeachmetric.

VSMetricsvDesignite VSMetrics Designite

Mean 331.2525 1092.7025

Variance 406916.7697 4629573.679

Observations 4 4

HypothesizedMeanDifference 0

df 4

tStat -0.678589881

P(T<=t)one-tail 0.267314435

tCriticalone-tail 2.131846786

P(T<=t)two-tail 0.53462887

tCriticaltwo-tail 2.776445105

VSMetricsvSourceMonitor

VSMetricsSourceMonitor

Mean 437.92 1469.203333

Variance 542107.4212 6139512.078

Observations 3 3

HypothesizedMeanDifference 0

df 2

tStat -0.69103136

P(T<=t)one-tail 0.280487733

tCriticalone-tail 2.91998558

P(T<=t)two-tail 0.560975465

tCriticaltwo-tail 4.30265273

VSMetricsvNDepend VSMetrics NDepend

Mean 331.2525 368.12

64

Variance 406916.7697 518347.8713

Observations 4 4

HypothesizedMeanDifference 0

df 6

tStat -0.076655038

P(T<=t)one-tail 0.470695081

tCriticalone-tail 1.943180281

P(T<=t)two-tail 0.941390163

tCriticaltwo-tail 2.446911851

DesignitevSourceMonitor

DesigniteSourceMonitor

Mean 1456.67 1469.203333

Variance 6149526.472 6139512.078

Observations 3 3

HypothesizedMeanDifference 0

df 4

tStat -0.006192532

P(T<=t)one-tail 0.497677819

tCriticalone-tail 2.131846786

P(T<=t)two-tail 0.995355638

tCriticaltwo-tail 2.776445105

DesignitevNDepend Designite NDepend

Mean 874.174 294.528

Variance 3710953.786 415839.8158

Observations 5 5

HypothesizedMeanDifference 0

df 5

tStat 0.638030546

P(T<=t)one-tail 0.275762439

65

tCriticalone-tail 2.015048373

P(T<=t)two-tail 0.551524878

tCriticaltwo-tail 2.570581836

SourceMonitorvNDepend

SourceMonitor NDepend

Mean 1469.203333 489.6033333

Variance 6139512.078 688972.6052

Observations 3 3

HypothesizedMeanDifference 0

df 2

tStat 0.649302738

P(T<=t)one-tail 0.291374895

tCriticalone-tail 2.91998558

P(T<=t)two-tail 0.58274979

tCriticaltwo-tail 4.30265273

ReadingforEachMetricandToolMetric VS

MetricsDesignite SourceMonitor NDepend

CC 1.46 1.93 1.47 1.49LOC 1288 4320 4330 1448Avg.LOC 24.3 48.08 76.14 19.32CBO 11.25 0.8 - 3.67LCOM - 0.06 - 0.16MI 87.12 - - -

66

APPENDIXBLinktovideodemonstratingthepreviousversion(1.0)ofthelegacyapplication

https://youtu.be/fVz7LmMcMBo

67

APPENDIXCFullresultsforeachmetricforalliterationsandtools.

MaintainabilityIndexTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

87.12 86.98 86.87 87.51 87.36 87.58

Ndepend - - - - - -

SourceMonitor - - - - - -

Designite - - - - - -

AverageforAllTools

87.12 86.98 86.87 87.51 87.36 87.58

LinesofCodeTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

1288 1468 1560 1856 1952 2055

Ndepend 1448 1629 1523 1729 1743 1728SourceMonitor 4330 5001 4807 5418 5386 5265Designite 4320 4978 4732 5260 5225 5182AverageforAllTools

2846.5 3269 3155.5 3565.75 3576.5 3557.5

AverageLinesofCodeTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

24.3 24.47 26 27.7 28.29 28.15

Ndepend 19.32 19.98 23.08 24.01 24.04 24.69SourceMonitor 76.14 74.64 92.44 91.83 91.29 94.02Designite 48.08 48.5 57.1 59.29 59.22 61.14AverageforAllTools

41.96 41.8975 49.655 50.7075 50.71 52

CouplingBetweenObjectsTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

11.25 11.55 11.75 12.36 12.99 13.27

Ndepend 3.67 7.38 9.21 10.33 10.46 12.52

SourceMonitor - - - - - -

68

Designite 0.80 0.89 1.03 1.07 1.01 1.13AverageforAllTools

5.24 6.61 7.33 7.92 8.15 8.97

LackofCohesionofMethodsTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

Ndepend 0.16 0.17 0.1 0.09 0.07 0.05SourceMonitor - - - - - -

Designite 0.06 0.07 0.08 0.08 0.08 0.09AverageforAllTools

0.11 0.12 0.09 0.085 0.075 0.07

CyclomaticComplexityTool Start 1st

Iteration2ndIteration

3rdIteration

4thIteration

5thIteration

VSCodeMetrics

1.46 1.45 1.51 1.48 1.5 1.49

Ndepend 1.49 1.48 1.55 1.5 1.5 1.48SourceMonitor 1.47 1.46 1.49 1.43 1.42 1.42Designite 1.93 1.83 1.94 1.91 1.98 1.99AverageforAllTools

1.59 1.56 1.62 1.58 1.60 1.60

69

APPENDIXDFullresultsforcodecoveragemetricforeachiteration.

StartNamespace %

CoveredCovered Uncovered Total

StatementsWSFinal 0% 0 71 71WSFinal.Migrations 0% 0 571 571WSFinal.Controllers 0% 0 1435 1435WSFinal.Models 0% 0 281 281WSFinal.Controllers.Admin 0% 0 0 0Total 0% 0 2358 2358

1stIterationNamespace

%Covered

Covered Uncovered TotalStatements

WSFinal 0% 0 71 71WSFinal.Migrations 0% 0 571 571WSFinal.Controllers 0% 0 1435 1435WSFinal.Models 49% 213 224 437WSFinal.Controllers.Admin 94% 82 5 87Total 11% 295 2306 2601

2ndIterationNamespace %

CoveredCovered Uncovered Total

StatementsWSFinal 0% 0 73 73WSFinal.Migrations 0% 0 225 225

WSFinal.Controllers 18% 256 1146 1402

WSFinal.Models 59% 283 195 478WSFinal.Controllers.Admin 96% 173 7 180WSFinal.Helpers 100% 10 0 10Total 30% 722 1646 2368

3rdIteration %

CoveredCovered Uncovered Total

StatementsWSFinal.App_Start 0% 0 33 33WSFinal 0% 0 73 73WSFinal.Migrations 0% 0 243 243WSFinal.Controllers 25% 379 1138 1517

70

WSFinal.Models 69% 409 180 589WSFinal.Controllers.Admin 99% 173 2 175WSFinal.Helpers 100% 10 0 10Total 37% 971 1669 2640

4thIterationNamespace %

CoveredCovered Uncovered Total

StatementsWSFinal.App_Start 0% 0 34 34WSFinal 0% 0 73 73WSFinal.Migrations 0% 0 162 162WSFinal.Controllers 28% 426 1122 1548

WSFinal.Models 69% 407 180 587WSFinal.Controllers.Admin 99% 230 2 232WSFinal.Helpers 100% 10 0 10Total 41% 1073 1573 2646

5thIterationNamespace %

CoveredCovered Uncovered Total

StatementsWSFinal.App_Start 0% 0 34 34WSFinal 0% 0 73 73WSFinal.Migrations 0% 0 6 6WSFinal.Controllers 30% 472 1122 1594WSFinal.Models 71% 447 180 627WSFinal.Controllers.Admin 100% 252 0 252WSFinal.Helpers 100% 12 0 12Total 46% 1183 1415 2598