www.synerzip.com

Test Driven DevelopmentWhat Works & What Doesn’t

November 5, 2008

Confidential

Today’s Discussion• Synerzip Introduction• Agile Development Lifecycle• Test Driven Development

– Case for TDD & Continuous Integration– User Acceptance Testing– Functional Testing– Unit Testing– Regression Testing– Common Tools & Useful Resources

• Q & A

Confidential

Synerzip in a Nut-shell1. Software development partner for small/mid-sized

technology companies• Focus: small/mid-sized technology companies• Deep experience in product development, testing, & deployment• Handles full software development life cycle• Technology and industry domain agnostic

2. Actually reduces risk of development/delivery• Experienced software management team• Brings in appropriate level of engineering discipline• Practices Agile development – responsive & disciplined

3. Reduces cost – dual-shore team, 50% cost advantage

4. Offers long term flexibility – allows (facilitates) taking offshore team captive

Confidential

Our Clients

Confidential

Today’s Discussion• Synerzip Introduction• Agile Development Lifecycle• Test Driven Development

– Case for TDD & Continuous Integration– User Acceptance Testing– Functional Testing– Unit Testing– Regression Testing– Common Tools & Useful Resources

• Q & A

Confidential

Agile Development• Why agile development?

– Due to a dynamic , ever changing business environment; requirements change.

– Innovation happens when requirements change.– Agile not only allows but encourages requirements to change.– Agile is an iterative show and build process. The customer gets to

see some new functionality at the end of each short iteration. Ideas evolve and new requirements get generated when the customer sees something working.

– Shorter the iteration closer the compliance with requirements.

• Challenges of agile development– In a short iteration of 3 weeks if 1 week is required for integration,

testing and deployment then only 2 weeks are left for actual coding or development work.

– There is an interdependence of TDD and automation.

Confidential

Common Misconception

Confidential

Agile Lifecycle – Iterations & QA

Small Release

Unit and Automated F & R/ UA

Testing

Iteration

Release Planning

User Stories

Requirements

User Acceptance

Testing

Test Scenarios

Iteration Plan

Confident Estimates

Feedback

/Functional Regression

Testing

Customer Approval

Confidential

Testing Terminology• Unit testing – testing a code component or class.

– Coverage is measured against lines of code

• Functional testing - testing against required functionality by going through the same steps that an end-user will execute.– Coverage is measured against requirements

• Regression testing – when functional testing is carried out again after fixing defects to ensure that no new bugs get introduced.

• User acceptance testing – a subset of functional test cases that form the basis on which the release is accepted by the customer.

Confidential

Agile Lifecycle - RolesIteration zero Iteration n Final Iteration

Product Owner

Requirements analysis

Review of product demo and changes

User acceptance testing and sign off.

Project Manager

Estimation and planning

Monitoring and re-estimation

Monitoring and release notes

Architect Prototype or POC design

Designing and code review

Design documents

Developer Estimation, understanding requirements and coding POC

Writing unit tests and code. Writing build automation script

Fixing bugs and documentation.

QA Writing test cases Testing and some automation

Filing bugs and regression testing

Confidential

Today’s Discussion• Synerzip Introduction• Agile Development Lifecycle• Test Driven Development

– Case for TDD & Continuous Integration– User Acceptance Testing– Functional Testing– Unit Testing– Regression Testing– Common Tools & Useful Resources

• Q & A

Confidential

What is TDD?

• Test Driven Development (TDD) means capturing the intent of requirements in the form of a test case before starting to code

• Test case could be a manual test case or an automated test script

• Test case could be a functional or a user-acceptance or a unit test

Confidential

TDD Means More Code?

Use your judgement, e.g. don’t write tests for every getter and setter

Confidential

TDD in PracticeValue of TDD – especially in Agile• Tests are more explicit expression of requirements • Earlier defect detection by frequent testing• Ensures better coverage - don’t end up having

superfluous code, without tests

Challenges – team resistance & reluctance?• ROI is realized in the long term – in maint phase• Extra coding effort seems unjustified in the

beginning• Easier said than done- difficult to implement• Developer resistance

Confidential

Role of Continuous Integration• Agile development has its roots in lean

manufacturing. “Stop the line” principle at Toyota is implemented in software development by making automated tests a part of the build process.

• The build fails if even one test fails. This reduces the time-lag between detection of defect and fixing of defect. Ensures timely corrective action.

• This also ensures that tests are updated to stay in sync with changing requirements.

Confidential

User Acceptance Testing (FIT)• What Works

– Wiki based tables are used to express and share requirements or user stories

– Test fixtures are written only for significant and complex requirements. Ready made test fixtures are used for simple requirements

– Subwikis are used to build test suites• What Doesn’t

– Customer/ product owner doesn’t spend enough time reviewing the test cases

– Developers spend too much time writing test fixtures– FIT tables don’t cover unhappy path. A use case can go

wrong in many ways.

Confidential

Manual Functional Testing• What Works

– Better expression of requirements by insisting on writing the test cases up front (Note: 70% of defects arise due to faulty requirements)

– Better understanding of the requirements by the QA staff– Examples or test data– QA should be a part of the Product Management/BA team

• What Doesn’t– No time allocated by the customer and product owner to review the

test cases– Test cases aren’t updated as new requirements evolve. Need to

make sure QA team and Product Mgmt/BAs are always in sync– Test cases are at a high level – but difficult to be useful for

generating test data

Confidential

Unit Testing• What Works

– Write tests first – Write tests only for the complex methods/units– No significant or complex code is written unless there is

a unit test already written for it.

• What Doesn’t– Writing tests later after the code is written.– Completing the formality of writing unit tests for each

and every method including accessor methods.– Writing unit tests for legacy code

• Case Study: A customer started TDD practice by writing unit tests for all the legacy code and ended up testing the tests with the code.

Confidential

Mechanics of Unit TDD

Add some code

Confidential

Functional/ Regression Testing• What Works

– Automating only that functionality which has stable manual test cases

– Test suites that run as a part of the build process– Test reports automatically published on the wiki

• What Doesn’t– Trying to automate everything. Coverage can’t go

beyond 70%– Tests scripts that are never integrated into a test suite– Tests are data specific and start breaking when the data

changes

Confidential

Continuous Integration• What Works

– Separate build tasks for unit, regression and UAT– Quick builds facilitate frequent (continuous) integration – Reports are a part of the build

• What Doesn’t– No distinction in dev, prod and test builds.– Long and tedious builds. Developer’s attention gets

diverted if it takes any longer than 30 secs.– Absence of teardowns to cancel all setup data– Tasks are dropped if the build fails– Untested dependencies

Confidential

Common Tools

Tools

Manual Jira, Bugzilla, Test Director

Unit Junit, Nunit, HttpUnit, DbUnit, SpringUnit etc.

Functional and regression

Watir, Selenium, QTP, AutoIt, Winrunner, RSpec, Tkl/Tk

Performance Loadrunner, OpenSta, jMeter,

User Acceptance Fitnesse

Continuous Integration

Maven,Cruise Control, Luntbuild, PMD, FindBugs, Jalopy (automated code review), Clover (code coverage)

Confidential

Useful Resources• http://www.agiledata.org/essays/tdd.html• http://www.methodsandtools.com/dynpoll/oldpoll.php?UnitTest2• http://www.testdriven.com/modules/news/• http://www.kevinwilliampang.com/post/Is-Code-Coverage-Really-All-

That-Useful.aspx• http://www.fitnesse.org• http://www.rubyforge.org• http://selenium.openqa.org• http://maven.apache.org/• http://cruisecontrol.sourceforge.net/• http://luntbuild.javaforge.com/• http://pmd.sourceforge.net/• http://jalopy.sourceforge.net/• http://findbugs.sourceforge.net/• http://www.atlassian.com/software/clover/• http://www.developer.com/design/article.php/3700651

Confidential

Today’s Discussion• Synerzip Introduction• Agile Development Lifecycle• Test Driven Development

– Case for TDD & Continuous Integration– User Acceptance Testing– Functional Testing– Unit Testing– Regression Testing– Common Tools & Useful Resources

• Q & A

Confidential

Contact Information

• Hemant Elhence (Dallas based)

– hemant@synerzip.com– Cell Phone: 214.762.4873

• www.synerzip.com

• HQ in Dallas, TX– 14228 Midway Rd, #130, Dallas, TX 75244– Office Tel: 469.322.0349– Office Fax: 469.322.0490

Thanks!