Part I Chapter 1 - glib.hcmuns.edu.vn · Part I Chapter 1 Part II Chapter 2 ... Step 1: Define a World-Class Software Testing Model ... What Is a Common Body of Knowledge? 125

Introduction

Part I

Chapter1

Part II

Chapter2

Assessing Testing Capabilities and Competencies

AssessingCapabilities, Staff Competency,and UserSatisfactionThe Three-Step Process to Becoming a World-Class Testing

OrganizationStep 1: Define a World-Class Software Testing Model

Customizing the World-Class Model for Your OrganizationStep 2: Develop Baselines for Your Organization

Assessment 1:Assessing the Test EnvironmentImplementation ProceduresVerifying the Assessment

Assessment 2: Assessing the Capabilities of Your ExistingTest Processes

Assessment 3: Assessing the Competency of Your TestersImplementation ProceduresVerifying the Assessment

Step 3: Develop an Improvement PlanSummary

Building a Software Testing Environment

Creating an EnvironmentSupportive of Software TestingMinimizing Risks

Risk Appetite for Software QualityRisks Associated with Implementing Specifications

Faulty Software DesignData Problems

xxv

1

3

357889

13

131414161618

35

373838393939

ix

x Contents

Chapter3

Risks Associated with Not Meeting Customer NeedsDeveloping a Role for Software Testers

Writing a Policy for Software TestingCriteria for a Testing PolicyMethods for Establishing a Testing Policy

Economics of TestingTesting-An Organizational IssueManagement Support for Software TestingBuilding a Structured Approach to Software Testing

RequirementsDesignProgramTestInstallationMaintenance

Developing a Test StrategyUse Work Paper 2-1Use Work Paper 2-2

Summary

Buildingthe Software TestingProcessSoftware Testing Guidelines

Guideline #1: Testing Should Reduce Software DevelopmentRisk

Guideline #2: Testing Should Be Performed EffectivelyGuideline #3: Testing Should Uncover Defects

Defects Versus Failures

Why Are Defects Hard to Find?Guideline #4: Testing Should Be Performed Using Business

LogicGuideline #5: Testing Should Occur Throughout the

Development Life CycleGuideline #6: Testing Should Test Both Function and Structure

Why Use Both Testing Methods?Structural and Functional Tests Using Verification and

Validation TechniquesWorkbench Concept

Testing That Parallels the Software Development ProcessCustomizing the Software-Testing Process

Determining the Test Strategy ObjectivesDetermining the Type of Development ProjectDetermining the Type of Software SystemDetermining the Project ScopeIdentifying the Software RisksDetermining When Testing Should OccurDefining the System Test Plan Standard

40434545464750505154545555555556585860

6363

6465656566

67

686969

6971727474757677777979

Contents xi

Defining the Unit Test Plan Standard 83

Converting Testing Strategy to Testing Tactics 83

Process Preparation Checklist 86

Summary 86

Chapter 4 Seleding and Installing Software Testing Tools 103

Integrating Tools into the Tester's Work Processes 103

Tools Available for Testing Software 104

Selecting and Using Test Tools 108

Matching the Tool to Its Use 109

Selecting a Tool Appropriate to Its Life Cycle Phase 109

Matching the Tool to the Tester's Skill Level 111

Selecting an Affordable Tool 114

Training Testers in Tool Usage 116

Appointing Tool Managers 117

Prerequisites to Creating a Tool Manager Position 118

Selecting a Tool Manager 118

Assigning the Tool Manager Duties 119

Limiting the Tool Manager's Tenure 120

Summary 120

Chapter 5 Building Software Tester Competency 125

What Is a Common Body of Knowledge? 125

Who Is Responsible for the Software Tester's Competency? 126

How Is Personal Competency Used in Job Performance? 126

Using the 2006 CSTECBOK 127

Developing a Training Curriculum 128

Using the CBOK to Build an Effective Testing Team 129

Summary 131

Part III The Seven-Step Testing Process 151

Chapter 6 Overview of the Software Testing Process 153

Advantages of Following a Process 153

The Cost of Computer Testing 154

Quantifying the Cost of Removing Defects 155

Reducing the Cost of Testing 156

The Seven-Step Software Testing Process 156

Objectives of the Seven-Step Process 159

Customizing the Seven-Step Process 160

Managing the Seven-Step Process 161

Using the Tester's Workbench with the Seven-Step Process 162Workbench Skills 163

Summary 164

Chapter 7 Step 1: Organizing for Testing 165

Objective 165Workbench 166

Input 167

xii Contents

Chapter8

Do Procedures

Task 1:Appoint the Test ManagerTask 2: Define the Scope of TestingTask 3: Appoint the Test Team

Internal Team ApproachExternal Team ApproachNon-IT Team ApproachCombination Team Approach

Task 4: Verify the Development DocumentationDevelopment PhasesMeasuring Project Documentation NeedsDetermining What Documents Must Be ProducedDetermining the Completeness of Individual DocumentsDetermining Documentation Timeliness

Task 5: Validate the Test Estimate and Project StatusReporting Process

Validating the Test EstimateTesting the Validity of the Software Cost EstimateCalculating the Project Status Using a Point System

Check Procedures

OutputSummary

Step 2: Developing the Test PlanOverview

ObjectiveConcernsWorkbench

InputDo Procedures

Task 1: Profile the Software ProjectConducting a Walkthrough of the Customer fUser AreaDeveloping a Profile of the Software Project

Task 2: Understand the Project RisksTask 3: Select a Testing Technique

Structural System Testing TechniquesFunctional System Testing Techniques

Task 4: Plan Unit Testing and AnalysisFunctional Testing and AnalysisStructural Testing and AnalysisError-Oriented Testing and AnalysisManagerial Aspects of Unit Testing and Analysis

Task 5: Build the Test Plan

Setting Test ObjectivesDeveloping a Test MatrixDefining Test AdministrationWriting the Test Plan

167167168168169170170170171171174175179180

181182185189200200200

209209210210211212212212212213215222223229235236238240243244245245250251

Contents xiii

Chapter 9

Chapter 10

Task 6: Inspect the Test PlanInspection ConcernsProducts/Deliverables to InspectFormal Inspection RolesFormal Inspection Defect ClassificationInspection Procedures

Check ProceduresOutputGuidelines

Summary

Step 3: VerificationTestingOverview


InputThe Requirements PhaseThe Design PhaseThe Programming Phase

Do ProceduresTask 1:Test During the Requirements Phase

Requirements Phase Test FactorsPreparing a Risk MatrixPerforming a Test Factor AnalysisConducting a Requirements WalkthroughPerforming Requirements TracingEnsuring Requirements Are Testable

Task 2: Test During the Design PhaseScoring Success FactorsAnalyzing Test FactorsConducting a Design ReviewInspecting Design Deliverables

Task 3: Test During the Programming PhaseDesk Debugging the ProgramPerforming Programming Phase Test Factor AnalysisConducting a Peer Review


Summary

Step 4: Validation TestingOverview


Input

254255256256258259262262262263

291292293294294296296296297298298299302310312314315316316318320322323325326328330331331332

409409410410410411

xiv Contents

Chapter 11

Do ProceduresTask 1: Build the Test Data

Sources of Test Data/Test ScriptsTesting File DesignDefining Design GoalsEntering Test DataApplying Test Files Against Programs That Update

Master RecordsCreating and Using Test DataPayroll Application ExampleCreating Test Data for Stress/Load TestingCreating Test Scripts

Task 2: Execute TestsTask 3: Record Test Results

Documenting the DeviationDocumenting the EffectDocumenting the Cause


Summary

412412412413414414

414415416430430434436437438438439439439440

Step 5: Analyzing and Reporting Test Results 459Overview 459Concerns 460Workbench 460

Input 461Test Plan and Project Plan 461Expected Processing Results 461Data Collected during Testing 461

Test Results Data 462Test Transactions, Test Suites, and Test Events 462Defects 462

Efficiency 463StoringData CollectedDuring Testing 463

Do Procedures 463Task 1: Report Software Status 464

Establishing a Measurement Team 465Creating an Inventory of Existing Project Measurements 465Developing a Consistent Set of Project Metrics 466Defining Process Requirements 466Developing and Implementing the Process 466Monitoring the Process 466

Task2:ReportInterimTestResults 470Function/Test Matrix 470Functional Testing Status Report 471FunctionsWorkingTimelineReport 472Expected Versus Actual Defects Uncovered Timeline Report 472

Contents xv

Chapter 12

Defects Uncovered Versus Corrected Gap Timeline ReportAverage Age of Uncorrected Defects by Type ReportDefect Distribution ReportNormalized Defect Distribution ReportTesting Action ReportInterim Test Report

Task 3: Report Final Test ResultsIndividual Project Test ReportIntegration Test ReportSystem Test ReportAcceptance Test Report


Summary

Step 6: Acceptance and Operational TestingOverview


Input ProceduresTask 1:Acceptance Testing

Defining the Acceptance CriteriaDeveloping an Acceptance PlanExecuting the Acceptance PlanDeveloping Test Cases (Use Cases) Based on How

Software Will Be UsedTask 2: Pre-Operational Testing

Testing New Software InstallationTesting the Changed Software VersionMonitoring ProductionDocumenting Problems

Task 3: Post-Operational TestingDeveloping and Updating the Test PlanDeveloping and Updating the Test DataTesting the Control Change ProcessConducting TestingDeveloping and Updating Training Material

Check ProceduresOutput

Is the Automated Application Acceptable?Automated Application Segment Failure NotificationIs the Manual Segment Acceptable?Training Failure Notification Form

Guidelines

Summary

473475475476477478478480480480482482482482483

491491492493494495496497498499

500503509509512513513514515517518518522522522523523524524525

xvi Contents

Chapter 13 Step 7: Post-Implementation Analysis 571Overview 571Concerns 572Workbench 572

Input 574Do Procedures 574

Task 1: Establish Assessment Objectives 574

Task 2: Identify What to Measure 575

Task 3: Assign Measurement Responsibility 575

Task 4: Select Evaluation Approach 575

Task 5: Identify Needed Facts 576Task 6: Collect Evaluation Data 577

Task 7: Assess the Effectiveness of Testing 577

Using Testing Metrics 577Check Procedures 580

Output 580Guidelines 581

Summary 581

Part IV Incorporating Specialized Testing Responsibilities 583

Chapter 14 Software Development Methodologies 585How Much Testing Is Enough? 585

Software Development Methodologies 586Overview 586

Methodology Types 587

Software Development Life Cycle 588

Defining Requirements 592

Categories 592Attributes 593

Methodology Maturity 596

Competencies Required 598

Staff Experience 600

Configuration-Management Controls 600

Basic CM Requirements 600

Planning 602Data Distribution and Access 602CM Administration 602

Configuration Identification 603

Configuration Control 605

Measuring the Impact of the Software Development Process 605

Summary 606

Chapter 15 TestingClient/Server Systems 611Overview 611Concerns 612Workbench 613

Input 614

Contents xvii

Chapter 16

Chapter 17

Do ProceduresTask 1:Assess Readiness

Software Development Process Maturity LevelsConducting the Client/Server Readiness AssessmentPreparing a Client/Server Readiness Footprint Chart

Task 2: Assess Key ComponentsTask 3: Assess Client Needs


Summary

Rapid Application Development TestingOverview

ObjectiveConcerns

Testing IterationsTesting ComponentsTesting PerformanceRecording Test Information

Workbench

InputDo Procedures

Testing Within Iterative RADSpiral TestingTask 1: Determine Appropriateness of RADTask 2: Test Planning IterationsTask 3: Test Subsequent Planning IterationsTask 4: Test the Final Planning Iteration


Summary

TestingInternal ControlsOverviewInternal Controls

Control ObjectivesPreventive Controls

Source-Data Authorization

Data InputSource-Data PreparationTurnaround DocumentsPrenumbered Forms

Input ValidationFile Auto-UpdatingProcessing Controls

614614615621621622622624624624624

633633634634634635635635635636636636638639640640642642643643643

655655657657658658659659659659659661661

xviii Contents

Chapter 18

Detective ControlsData TransmissionControl RegisterControl TotalsDocumenting and TestingOutput Checks

Corrective ControlsError Detection and ResubmissionAudit Trails

Cost/Benefit AnalysisAssessing Internal Controls

Task 1: Understand the System Being TestedTask 2: Identify RisksTask 3: Review Application ControlsTask 4: Test Application Controls

Testing Without Computer ProcessingTesting with Computer ProcessingTransaction Flow TestingObjectives of Internal Accounting ControlsResults of Testing

Task 5: Document Control Strengths and WeaknessesQuality Control ChecklistSummary

Testing COTSand Contracted SoftwareOverview

COTS Software Advantages, Disadvantages, and RisksCOTS Versus Contracted Software

COTS AdvantagesCOTS DisadvantagesImplementation RisksTesting COTS SoftwareTesting Contracted Software


InputDo Procedures

Task 1:Test Business Fit

Step 1: Testing Needs SpecificationStep 2: Testing CSFs

Task 2: Test Operational FitStep 1: Test CompatibilityStep 2: Integrate the Software into Existing Work FlowsStep 3: Demonstrate the Software in Action

Task 3: Test People Fit

662663663664664664665665665666666666668668668669669672673677677678678

685686686686687687688689690691691692693693693693695696697698700701

Contents xix

Chapter 19

Chapter 20

Task 4: Acceptance-Test the Software ProcessStep 1: Create Functional Test ConditionsStep 2: Create Structural Test Conditions

Modifying the Testing Process for Contracted SoftwareCheck Procedures

OutputGuidelines

Summary

702702703704705705706706

Testingin a Multiplatform Environment 717Overview 717

O~ective 718Concerns 718

Background on Testing in a Multiplatform Environment 718'~orkbench 719

Input 720Do Procedures 721

Task 1: Define Platform Configuration Concerns 721Task 2: List Needed Platform Configurations 723Task 3: Assess Test Room Configurations 723Task 4: List Structural Components Affected by the Platform(s) 723Task 5: List Interfaces the Platform Affects 725Task 6: Execute the Tests 726

Check Procedures 726

Output 726Guidelines 726

Summary 727

TestingSoftware SystemSecurity 733Overview 733Objective 734Concerns 734VVorkbench 734

Input 735VVhereVulnerabilities Occur 735

Functional Vulnerabilities 736VulnerableAreas 737Accidental Versus Intentional Losses 738

Do Procedures 739

Task 1: Establish a Security Baseline 739~y Baselines Are Necessary 740Creating Baselines 740Using Baselines 749

Task 2: Build a Penetration-Point Matrix 751

Controlling People by Controlling Activities 751Selecting Security Activities 752Controlling Business Transactions 755

xx Contents

Chapter 21

Chapter 22

Characteristics of Security PenetrationBuilding a Penetration-Point Matrix

Task 3: Analyze the Results of Security TestingEvaluating the Adequacy of SecurityCheck ProceduresOutputGuidelines

Summary

Testinga Data WarehouseOverviewConcernsWorkbench

InputDo Procedures

Task 1: Measure the Magnitude of Data Warehouse ConcernsTask 2: Identify Data Warehouse Activity Processes to Test

Organizational ProcessData Documentation Process

System Development ProcessAccess Control Process

Data Integrity ProcessOperations ProcessBackup /Recovery ProcessPerforming Task 2

Task 3: Test the Adequacy of Data Warehouse ActivityProcesses


Summary

TestingWeb-Based SystemsOverviewConcernsWorkbench

InputDo Procedures

Task 1: Select Web-Based Risks to Include in the Test PlanSecurity ConcernsPerformance ConcernsCorrectness ConcernsCompatibility ConcernsReliability ConcernsData Integrity ConcernsUsability ConcernsRecoverability Concerns

756757760761762762762762

765765765766767768768769769769770771771772773774

774780780780780

799799800800801802802803803804804806806806807

Part V

Chapter 23

Chapter 24

Task 2: Select Web-Based Tests

Unit or ComponentIntegrationSystemUser AcceptancePerformanceLoad/StressRegressionUsabilityCompatibility

Task 3: Select Web-Based Test Tools

Task 4: Test Web-Based SystemsCheck ProceduresOutputGuidelines

Summary

Building Agility into the Testing Process

UsingAgile Methods to Improve Software TestingThe Importance of AgilityBuilding an Agile Testing ProcessAgility InhibitorsIs Improvement Necessary?Compressing Time

ChallengesSolutions

Measuring ReadinessThe Seven-Step Process

Summary

BuildingAgility into the TestingProcessStep 1: Measure Software Process Variability

Timelines

Process StepsWorkbenches

Time-Compression WorkbenchesReducing VariabilityDeveloping Timelines

Improvement Shopping ListQuality Control ChecklistConclusion

Step 2: Maximize Best PracticesTester Agility

Software Testing RelationshipsTradeoffsCapability ChartMeasuring Effectiveness and Efficiency

Contents xxi

807807807807808808808808808808809809809810810811

817

819819820821822823824825826826827

831831832833833834835836841841842842842843845847848

xxii Contents

Improvement Shopping ListQuality Control ChecklistConclusion

Step 3: Build on Strength, Minimize WeaknessEffective Testing ProcessesPoor Testing ProcessesImprovement Shopping ListQuality Control ChecklistConclusion

Step 4: Identify and Address Improvement BarriersThe Stakeholder Perspective

Stakeholder Involvement

Performing Stakeholder AnalysisRed-Flag/Hot-Button BarriersStaff-Competency BarriersAdministrative/Organizational BarriersDetermining the Root Cause of Barriers/ObstaclesAddressing the Root Cause of Barriers/ObstaclesQuality Control ChecklistConclusion

Step 5: Identify and Address Cultural and CommunicationBarriers

Management CulturesCulture 1: Manage PeopleCulture 2: Manage by ProcessCulture 3: Manage CompetenciesCulture 4: Manage by FactCulture 5: Manage Business Innovation

Cultural Barriers

Identifying the Current Management CultureIdentifying the Barriers Posed by the CultureDetermining What Can Be Done in the Current CultureDetermining the Desired Culture for Time CompressionDetermining How to Address Culture Barriers

Open and Effective CommunicationLines of CommunicationInformation/Communication BarriersEffective Communication

Quality Control ChecklistConclusion

Step 6: Identify Implementable ImprovementsWhat Is an Implementable?Identifying Implementables via Time CompressionPrioritizing ImplementablesDocumenting ApproachesQuality Control ChecklistConclusion

856856857857857860860860861861861863863864865865866867869869

869870871873874876878879879879879879880880881882882884885885885886888890890890

Contents xxiii

Step 7: Develop and Execute an Implementation PlanPlanningImplementing IdeasRequisite Resources

Quality Control ChecklistConclusion

Summary

Index

891891891893894894895

929