Economics of SoftwareTesting..2

Testing Testing Perangkat LunakPerangkat Lunak

Oleh : Oerip S. Santoso

12/17/03 Bayu Hendradjaya - http://www.if.itb.ac.id/~bayu 2

Common Computer ProblemCommon Computer Problem• Software Problem

– Incomplete Design or Erroneous decision-making criteria• actions have been incorrect• inappropriate decision making criteria

– Fail to meet customer requirement• logic error or programming error

– Ommitting needed edit checks for completeness of output data

• Data Problems– Incomplete Data– Incorrect Data– Obsolete Data

CodingErrors36 %

Analysis andDesign Errors

64 %


Economics of SDLC TestingEconomics of SDLC Testing

Requirements20 Errors

Design20 Errors

Code20 Errors

Test80% Error Reduction

Production"0" Defects

cost detect = 1

cost detect = 1

cost detect = 1

cost detect = 10

cost detect = 100

SDLC TestingAccumulated

Test Cost

AccumulatedErros/1000

loc10 10

25 15

42 18

182 4

582 0

Normal SDLCAccumulated

Test CostAccumulated

Erros/1000 loc0 20

0 40

0 60

480 12

1680 0


The Structured Approach To TestingThe Structured Approach To TestingLife CyclePhase

Verification Activities

Requirements • Determine verification approach• Determine adequacy of requirements• Generate functional Test Data• Determine consistency of design with requirements

Design • Determine adequacy of design• Generate structural and functional test data• Determine consistency with design

Program(Build/construction)

• Determine adequacy of implementation• Generate structural and functional test data for

programsTest • Test application systemInstallation • Place Tested system into productionMaintenance • Modify and retest


Testing in Life CycleTesting in Life Cycle

UNIT

INTEGRATION

SYSTEM ACCEPTANCE

REGRESSIOIN

VERIFICATION

VALIDATION

SYSTEM TESTING/QUALITY CONTROL

Analyze Design Build Test Install Maintain

• Verification is the process of evaluating a system/component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase (IEEE/ANSI)

• Validation is the process of evaluating a system/component during or at the end of the development process to determine whether it satisfies specified requirements (IEEE/ANSI)


Basic Forms of TestingBasic Forms of Testing• Full Testing

– starts no later than the requirement phase and continues through acceptance testing

• Partial Testing– begins any time after functional design has been completed, with less than optimal

influence on requirements and functional design

• Endgame Testing– is highly validation oriented, with no influence on requirements or functional design

• Audit-Level Testing– is a barebones audit plans, procedures, and products for adequacy, correctness, and

compliance to standards.

cost

of

test

ing

Error-detection effectiveness

Audit Level

Endgame

Partial

Full


Computer System Strategic RiskComputer System Strategic Risk• A risk is a condition that can result in a loss• A risk is related to the probability of a loss• The risk is always exists, although the loss may not

occur• Risk can not be eliminated, but the impact of the loss

can be reduced• The most effective methods to reduce the impact of

the loss is testing• Types of strategic risk

– Incorrect result, unauthorized transaction, lost of integrity offile, processing can not be reconstructed, lost of continuity of processing, degradation of services for user, security, unreliable result, difficulties to use and operate, unmaintainable program, not portable, not be able to interconnect, unacceptable performance level


Test StrategyTest Strategy• A strategy must address the risks and present a process that

can reduce those risks• Two component of Testing Strategy

–– Test FactorTest Factor - The risk or issue that needs to be addressed as part of the test strategy.

• Correctness, authorization, file integrity, audit trail, continuity of process, service levels, access control, compliance, reliability, ease of use, maintainability, portable, coupling, performance, ease of operation

–– Test PhaseTest Phase - The phase of the SDLC in which testing will occur

• Note:– The risk associated with testing will be called “Test Factors”– Not all test factors will be applicable– The test phase will vary based on the testing methodology used


Developing Test StrategyDeveloping Test Strategy• Select and rank test factor• Identify the system development phase• Identify the business risk associated with the system under

development• Place risks in the matrix

Req

uire

men

ts

Des

ign

Build

Test

Inte

grat

e

Mai

ntai

n

TestPhase

TestFactorsFA

CTORS

RISK


StrategicalStrategical/Tactical Testing Cube/Tactical Testing Cube

Requ

irem

ents

Des

ign

Build

Test

Inte

grat

e

Mai

ntai

n

TestPhase

TestFactors

FACTO

RS

RISK

Test

Tactics

Test Strategy

PLANS

• The testing methodology cube represents a detailed work program for testing application system

• The tactics add the test plans, test criteria, testing techniques and testing tools used in validating and verifying the SW systemunder development

• A detailed testing work program is important to ensure that the test factors have been adequately addressed at each phase.

• The first and most important dimensions are the test factors


Developing Testing TacticsDeveloping Testing Tactics• Acquire and study the test strategy• Determine the type of development project• Determine the type of software system• Determine the project Scope• Identify the tactical risks• Determine when testing should occur• Build the system test plan• Build the unit test plans


Type of Development ProjectType of Development Project

Type Characteristic Test TacticTraditional • uses a system development

methodology• User knows requirements• Development determines

structure

• Test at end of each task/step/phase• Verify that specs match need• Test function and structure

Iterative/Protyping/CASE

• Requirement unknown• Structure predefined

• Verify that CASE tools are usedproperly

• Verify that prototypes match need• Test functionality

SystemMaintenance

• Modify Structure • Test Structure• Works best with release methods• Requires regression testing

Purchased/Contracted SW

• Structure unknown• May contain defects• Functionality defined in user

documentation• Documentation may vary

from Software

• Verify that functionality matchesneeded

• Test functionality• Test fit into environment


Type of Software SystemType of Software System• Batch (general)• Event Control• Process Control• Procedure Control• Advanced Mathematical

Models• Message processing• Diagnostic software• Sensor and Signal Processing

• Simulation• Database Management• Data acquisition• Data presentation• Decision and planning aids• Pattern and Image

processing• Computer System Software• Software Development Tools

Documents

Economics of SoftwareTesting..2