269 rosenblum -agile-it-org--ssp2008

Agile Development:The Importance of Testing

Presented by

Bruce D. Rosenblum

CEO

Inera Incorporated

Society for Scholarly Publishers

30 May 2008

© 2008, Inera Inc. All Rights Reserved

Real World Software Development

Credit: http://www.eos-india.net/abhisek/blog/softwaredevcycle-2006-06-14-20-38.html


Software Development Goal Develop Applications

• Deliver value to the user

• Easy to use

• Provide reliable process


The Old Days: Waterfall Development

Scope ChangeFeature Creep

Credit: http://www.csse.monash.edu.au/~jonmc/CSE2305/Topics/07.13.SWEng1/html/text.html


Why Waterfall No Longer Works Computers are more powerful Therefore projects are bigger

• And take longer to develop

No longer possible to anticipate all requirements• And cannot anticipate all changes during project

By delivery time• Customer interests changed• Requirements changed

“What I asked for, but not what I want”


Web Flexible World Old distribution model

• Disk shipments• Waterfall development• Slow, late deliveries

New distribution model• Internet delivery• Automatic updates of client applications• Continual update of ASP applications


New Development Methods Customer-centric Incremental changes Rapid Deliveries Continuous Testing


Agile Model


Agile Goals “Agile” means responsive to customer needs The goal is the customer, not the “project” Simplicity and simple steps


Getting Started Design simple task that does “something useful”

• Even if that task won’t be delivered to “customer”

Start testing immediately Test early and often


Team Test Types Unit tests

• Test assumptions; vital for maintenance

Integration tests• Does it all work together correctly?

Code Reviews• Developers can learn from each other


Developers Can’t Test Developers are lousy testers Developers lie

• Is it done yet?

• When will it be done?

Great developers test 2/3 of their time• Write a little, test a lot


Customer Test Types Formal Quality Assurance

• Scripted test plans Ad hoc testing

• Sometimes called “monkey testing” Usability testing

• One-way mirrors User testing

• The importance of watching users “Run it yourself” testing


Test Administration Bug database

• TestTrack, Bugzilla, etc.

• FileMaker, Excel

Short Cycles• Daily bug review

• Fix bugs immediately− New features must wait


Automated Testing

Use of software to control testingAutomatically run application featuresResults compared to previous “good”

result


Use of Automated Testing Inera relies heavily on automated testing Three months of development over 6 years Today: 30+ hours of automated testing per night Manual validation performed on all releases


Example: CrossRef Testing Inera implemented CrossRef query module Automated testing of CrossRef module

• Queried ~1000 references per night• Compared DOIs with previous “good” results

Sometimes links go away• Usually incorrect deposit (e.g. metadata conflict)

Sometimes new links appear• Replaces previous “good” results


Impact of Automated Testing New problems caught early

• Problems found quickly when code is changed

Stability of code maintained Fixes are never “lost” Short release cycles


Cost of Automated Testing Sustainable and scalable model

• Up-front investment yields large dividends

Lower costs• QA cost ~ $60,000/person/year

• Versus 30+ hours of PC time per night− 4 PCs + electricity + 3 months engineering time

No fulltime QA personnel• Manual testing performed by multiple team members


Time for Testing Testing does NOT take extra time

• Even manual testing

Testing reallocates time• Less time spent tracking latent bugs

• Less time reworking code to fix bugs

Great testing allows faster releases


“True” Cost of Automated Testing Faster Release cycles

• Can release latest code daily if stable

Fewer support calls Happier customers


Who Tests? Developers Quality Assurance Specialists Customer Actual Users Machines (automated testing) So who tests? The Agile Team


Conclusions System stability requires constant testing Great testing lowers costs Great testing delivers software faster


Questions?

Bruce RosenblumInera Incorporated+1 (617) 969 - 3053

[email protected]

Documents

269 rosenblum -agile-it-org--ssp2008