CSE403 Software Engineering Autumn 2000

More Project Management and Testing Overview

Gary Kimura

Lecture #13

October 23, 2000

Today’s quote

“I have a pop quiz and I’m not afraid to use it!”

Today

• Two purposes for information hiding– The academic and engineering arena

– The competitive environment

• A little more on group management

• Testing overview

Group management

• Good group management is a big factor in doing a large multi-person project

• Management works both ways (i.e., managing subordinates and managing your manager)

• When you give someone the responsibility over a project then you’ve also given the authority to make decisions for that project. Another way of saying this is that “Responsibility implies authority”

• Pay versus other rewards

Group rivalries

• Rivalries both inter- and intra-company (for a large company) are a fact of life. This doesn’t mean they are necessarily always good or bad– Good because

• Hopefully the best engineered design wins

• Some people really enjoy and perform well under competition (e.g., Win98 versus WinNT)

– Bad because• Waste of resources

• Hard feelings and undercutting (this happened a lot at DEC)

• Other reasons?

More design issues

• Having a well understood and well communicated architecture is vital – Consensus on a design is not always practical

– Architects at various levels of the system are needed to formulate and communicate the design to others on the team

• A well designed API set helps define the division of modules, labor, and necessary information hiding

Modularization and information hiding are abstractly good; but they do have a cost

• Performance might be sacrificed by a clean design. For example, – “Heavy weight” procedure cost time and space, and identifying

these cost is not always easy. Even the number of parameters has a cost. Using procedures cannot to be avoided, but must be respected.

– Internal data structures are often conceptually different allocations but each allocation has a small overhead cost

• Using well designed submodules can make debugging conceptually easier; however experience shows that programmers often need to know fairly minute subcomponent details to debug their new code

Functional module layout and temporal module organization go hand-in-hand

• A clean design also needs a clean workspace

• Modules can be organized separating exported from local routines

• Global header files need to be well organized

• Smart editors make locating places where functions and structures are defined and used easier

Often we need to sleep on the design• As with other engineering aspects the group must not

become paralyzed trying to settle design issues. An educated decision is better than no decision.

• Experience and education helps us all make better educated decisions

• Try and anticipate changes. Here are some NT design issues that have cropped up through the years. Not all of them are changeable or fixable.– 32 bit to 64 bit expansion– Big and little endian support– Time format without time zone information – String format is a counted string limited to 64KB in length and with

Unicode the string length no longer equals character count.– Memory allocator changes

A few things to keep in mind

• Turnover (via desertion or job growth)

• Not everyone can work at 100% throughout a long (or even short) project

• Training new hires

• Buy-in and communication is always vital

Now onto testing

• Educational objective– Understand why testing is the primary vehicle for quality

assurance

– Understand the inherent flaw with testing

– Understand that there are many different levels and styles of testing, each with different objectives

Verification and validation

• Validation: “Did we build the right system?”– Primarily a requirements-level (upper lifecycle) issue

• Verification: “Did we build the system right?”– Primarily a lower lifecycle issue (design, implementation, testing)

Approaches to verifying software systems

• Testing – A dynamic approach

• Program verification – Use math to show an equivalence between a specification and a

program

• Process – Improving the likelihood that code is correct

Process

• Process includes a broad set of ideas and approaches – Software inspections, walkthroughs, reviews

– Capability maturity model, ISO 9000

– Etc.

Testing vs. proving

• Dynamic – Builds confidence

• Can only show the presence of bugs, not their absence

– Used widely in practice

– Costly

• Static – It’s a proof

• Proofs are human processes that aren’t foolproof

– Applicability is practically limited

– Extremely costly

An aside

• Dynamic techniques are unattractive because they are unsound—you can believe something is true when it’s not

• Static techniques are unattractive because they are often very costly (and there are other reasons)

• The truth is that they should be considered to be complementary, not competitive

Testing

• In any case, testing is by far the dominant approach to demonstrating that code does what it supposed to (whatever that means!)

• Testing is a lot like the weather: everybody complains about it, but nobody seems to do much about it

Two kinds of improvements

• One goal is to improve testing to increase the quality of the software that is produced

• Another goal is to reduce the costs of testing while maintaining the current quality of the software that is produced Even if people don't admit it, this is a common focus

Terminology

• A failure occurs when a program doesn't satisfy its specification

• A fault occurs when a program's internal state is inconsistent with what is expected (usually an informal notion)

• A defect is the code that leads to a fault (and perhaps to a failure)

• An error is the mistake the programmer made in creating the defect

More terminology

• A test case is a specific set of data that exercises the program

• A test suite is a set of test cases

• Old terminology – A test case (suite) fails if it demonstrates a problem

• New terminology – A test case (suite) succeeds if it demonstrates a problem

Root cause analysis

• Tries to track a failure to an error

• Identifying errors is important because it can – help identify and remove other related defects

– help a programmer (and perhaps a team) avoid making the same or a similar error again

Kinds of testing

• Unit

• White-box

• Black-box

• Gray-box

• Bottom-up

• Top-down

• Boundary condition

• Syntax-driven

• Big bang

• Integration

• Acceptance

• Stress

• Regression

• Alpha

• Beta

A simple problem

• The program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is isosceles, equilateral, or scalene.

• Write a set of test cases that you feel would adequately test this program

In practice

• 13 kinds of errors were found in actual programs

• When highly experienced programmers are given this example, on the average they figure out about half of the kinds of errors

The lucky thirteen...

• Valid scalene triangle

• Valid equilateral triangle

• Valid isosceles triangle

• Three cases that represent valid isosceles triangles in all permutations

• One side is zero

• One side is negative

• 3 positive integers where two sum to the third

• All permutations of the previous case

The remaining ones

• 3 positive integers where two sum to less than the third

• 3 permutations of the previous case

• All sides are zero

• A non-integer side

• An incorrect number of inputs

• Machine overflow (my own)