Upload
soaring-eagle-llc
View
2.597
Download
1
Tags:
Embed Size (px)
DESCRIPTION
A presentation on a Developers viewpoint of available testing tools and strategies for testing Java applications. The presentation was given to the Denver Open Source User\'s Group www.denveropensource.org
Citation preview
Testing Tools and Strategies
Scott RyanMarch 2007
Agenda
• Business Drivers• Unit Testing• Coverage• Code Quality Management (CQM)• Integration Testing• Load Testing• Test Data• Automated Test Generation• Demo
Business Drivers
• Errors found early in the development lifecycle cost less to fix
• Cost varies by Industry and Software usage• Cost is not linear• Testing improves quality and reduces cost• Testing costs money too and there is a
tradeoff
Business Drivers
• Cost of running integration and system tests again. • Cost of recertification (if you're in that kind of
environment). • Cost of deploying the software again. • Support costs when only half your customers deploy
the new version. • Indirect costs caused by using resources to fix bugs
rather than implement revenue-generating features. • Liability for damages caused by the bug.
Testing Levels
• Unit Testing• Integration Testing• System Testing
Test Design
• Black Box• White Box
Testing Techniques
• Manual Testing• Automated Testing• Regression Testing• Environment Testing• Stress Testing• Load Testing• Performance Testing
Unit Testing Frameworks
• Junito HttpUnito StrutsUnito XMLUnito DBUnit
• TestNG
JUnit
• Version 3 and 4• Extend TestCase Class• Validate through Asssertions• Setup and TearDown Methods• Testing Suites• Lots of Automation
Test NG
• Java 5 (Primarily)• No need to extend base
class• Good use of Java 5
Annotations• DataDriven Tests• Embedded Beanshell• Embedded Logging, JDK
• Parameter Support• Supports Test
Distribution• New Execution Model
o No Suiteso Groupso Relationships
• Application Server Testing support
TestNG Annotations
• @Test• @Parameters• @DataProvider• @BeforeSuite• @AfterSuite• @BeforeTest• @AfterTest
• @BeforeMethod• @AfterMethod• @BeforeClass• @AfterClass• @BeforeGroup• @AfterGroup
Unit Testing Tips
• Keep tests in same package as tested code• Separate tests for packaging• Start small and work up• Give useful names and assertions• Run often and review the results• Build tests around bugs you are fixing
Coverage
• Cobertura• Clover• Emma• Jtest (Parasoft)• Jester• GroboUtils
• Djunit• Hansel• Gretel
Coverage Types
• Statement Coverage• Branch Coverage (if and implied else)• Condition Coverage• Coverage Percentage
Coverage Hints and Tips
• Use at the beginning of test development• Strive for branch over line coverage• Build tests to increase coverage• Hard to tell what is important so the
developer is key• Review reports often as code base grows
Code Quality Management
• Used to raise the quality of code base• Range from trivial to high impact• All improvements are important• Use a wide variation of tools and tune
around your team’s best practice• Use to drive refactoring
CQM Tools
• Checkstyle• Findbugs• PMD• CPD• Jtest• JDepend
• Jalopy• Eclipse• NetBeans• JavaDoc
CQM Hints and Tips
• Automate and run often• Make incremental improvements over time• Quality is important• Use tools to improve quality at the source• Use as a guide to refactoring
Integration Testing
• Tests the assembled deployed unit• Usually requires automation• Developed after Unit tests• Least used technique but most effective• Fairly complex with many moving parts• Require mature development process
Integration Testing Frameworks
• Selenium• Cactus• Canoo• JWebUnit• HttpUnit• Jmeter• Cargo• Eclipse• Maven
• Dumbster (SMTP)• WinRunner (HP)• SilkTest (Borland)• TeamTest (Rational)
Integration Testing Hints and Tips
• Start small • Automate as soon as practical• Strive for breadth over depth• Work with QA and UAT groups• Tie tests to use cases• Use as smoke test to insure proper deployments• Understand data needs and system impact
Test Data Hints and Tips
• Understand the data needs of your tests• Start the system in a known state• Return system to known state when done• Watch out for protected data• Need to understand the impact of your tests• Keep data current• Understand system impact (i.e. database keys)• Leverage Automation
Test Data Tools
• Setup and teardown methods• Database Scripts• DBUnit• XMLUnit• DDSteps• JTestcase
Mock Testing
• Objects act as mediators• Simulates complex or unfinished systems• Allows access to “closed” systems• Quicker Testing• Mocks are not stubs they are dynamic• Mocks offer behavior verification• Stubs offer state verification
Mock Toolsets
• Easy Mock• Spring Mock• JMock• Jmockit• MockRunner
Automated Test Generation
• JunitDoclet• TestGen4J• Jtest (Parasoft)• Appfuse/Freemarker Templates
Load Testing
• Grinder• Jmeter• JunitPerf• OpenSTA• Eclipse
Other Testing
• Slim (JSP)• J2ME Unit• JFCUnit (Swing)• JSTester (Javascript)• TagUnit (JSP Tags)• SOATest (Parasoft) (Web Services)• XMLUnit (XML)
Demos
Demos
• Junit• TestNG• Coverage• Selenium• Parasoft Jtest• Eclipse
• TestGen4J• Grinder