Testing

What is Testing?

• Definition: exercising a program under controlled conditions and verifying the results

• Purpose is to detect program defects after all syntax errors have been removed and the program compiles

• No amount of testing can guarantee the absence of defects in sufficiently complex programs

Careful Testing is Needed

• Unless all possible inputs are tested (usually impossible), testing cannot guarantee freedom from logic errors– Errors that always occur are easily found– But, errors that occur sporadically or only

in certain special cases are hard to find– Testers must be careful and clever to test

these obscure conditions!

Structured Walkthroughs

• Designer must explain the algorithm to other team members and simulate its execution with other team members looking on

• When you explain, you often catch your own errors

• Teammates might see errors or misconceptions

Levels and Types of Testing

• Unit testing: checking the smallest testable piece of the software (a method or class)

• Integration testing: testing the interactions among units

• System testing: testing the program in context

Levels and Types of Testing (continued)

• Acceptance testing: system testing designed to show that the program meets its functional requirements

• Black-box testing: tests the item based on its interfaces and functional requirements

• White-box testing: tests the software with the knowledge of its internal structure

Preparations for Testing

• A test plan should be developed early in the design phase

• Testing should take place concurrently with the design and coding

• A good programmer practices defensive programming– Include code to detect invalid or

unexpected data (e.g. exception code)

Test Plan

• How will the software be tested?

• When will the tests occur?

• Who will do the testing?

• What is the test data (and expected results)?

Start with Documentation

• Method documentation: include parameters and expected results

• Carefully document method parameters and class attributes in comments as you write the code

• Include pre- and post-conditions

Include Helpful Output

• Leave a trace of execution by displaying the method name as you enter it

• Display values of all input parameters upon entry to a method

• Display the values of any class attributes that are accessed by this method

• Display the values of all method outputs after returning from a method

Developing the Test Data

• Specify test data during the analysis and design phases for the different levels of testing: unit, integration, and system

• Data includes inputs & expected outputs

Black Box Testing

• Test relationship between inputs & outputs

• Check for all (types of) expected inputs

• Check for unanticipated data– If a monkey was sitting at a keyboard…

White Box Testing

• Goal is to test every path through the code

• For every if statement, there should be at least one test case where the condition is false, and one where the condition is true

• All methods of all classes should be exercised

Boundary Conditions

• These are cases where errors often happen– The first or last execution of a loop

• (Off-by-one error)

– The largest or smallest possible value– The first or last location in an array

Who Does Testing?

• The programmer– Risk: blind to his/her own oversights

• Other members of the software team who did not code the module being tested– Quality Assurance organization within company– Team member not responsible for the code

• Final users of the software product– Alpha testing– Beta testing– After delivery (unfortunately)

Unit Testing (Drivers & Stubs)

• To test a method, some other method or main needs to call it– A driver is written specifically to run test

cases on one or more methods

• To test a method, methods that it uses must be implemented– A stub is written to allow its parent to be

tested

Drivers

• A driver program declares any necessary object instances and variables, assigns values to any of the method’s inputs, calls the method, and displays the values of any outputs returned by the method

• You can put a main method in a class to serve as the test driver for that class’s methods

Stubs

• The replacement of a method that has not yet been implemented or tested is called a stub

• A stub has the same header as the method it replaces, but its body only displays a message indicating that the stub was called (and returns a default result if a return is needed)

Using a Test Framework

• A test framework is a software product that facilitates writing test cases, organizing the test cases into test suites, running the test suites, and reporting the results

• A test framework often used for Java products is JUnit, an open-source product that can be used in a stand-alone mode and is available from junit.org

• Junit is also included within Eclipse and BlueJ (and other Java IDE’s)

Junit in BlueJ

• Enable testing:– Preferences : Miscellaneous – check “Unit

Testing Tools”

• Right click a class, ‘Create Test Class’

• Creates a (green) test class

Edit text or right-click for options

• Click ‘run tests’ to run all tests at once

Junit in Eclipse

• With the class you want to test open, select File / New / Junit TestCase

• Click ‘setup’ and ‘teardown’ for items to create (you can delete later if you don’t need them)

• Create tests (all begin with the word test)• Run / Run As / Junit test (to test)