OVERVIEW: FUNDAMENTALS OF SOFTWARE TESTINGBy Dung NguyenDate: Sep. 23, 2009Duration: 4 hoursCourse Code: GCS-C119

Confidential

ContentsIntroductionConcepts: Black Box And White Box TestingPART 1 TESTING LIFECYCLE IN WATERFALL AND RUPThe Lifecycle Of Testing in Software Development Process (SDP)PART 2 OVERVIEW PHASES OF TEST AND TEST TYPESConcepts: Test Phases and Test TypesPART 3 THE RELATED ACTIVITIES IN PER TEST PHASESUnit Test PhaseIntegration Test PhaseSystem Test PhaseAcceptance Test Phase Glossary

Confidential

IntroductionThe test process defines 4 phases (including Test Types) of software testing which occur between product coding and product delivery. It also covers, in brief, pre-test activities on which the test effort depends. This document is intended to provide an overview about Fundamentals of Software Testing.

Confidential

CONCEPT: BLACK BOX TESTINGBlack box testing: Not based on any knowledge of internal design or code.Tests are based on requirements and functionalities.

Confidential

CONCEPT: WHITEBOX TESTINGWhite box testing: Based on knowledge of the internal logic of an application'scode. Tests are based on coverage of code statements,branches, paths, and conditions.

Confidential

PART 1:

TESTING LIFECYCLE IN WATERFALL AND RUP

Confidential

Testing Lifecycle in Waterfall

Confidential

Testing Lifecycle in Iteration

Confidential

PART 2 OVERVIEW PHASES OF TEST AND TEST TYPES

Confidential

CONCEPTS: PHASES OF TESTThere are essentially 4 test phases that have to be completed between coding and product delivery:

Each test phase is structured to detect specific categories of defectsThe types of tests that are performed in each test phase

Confidential

CONCEPTS: PHASES OF TEST (cont.)

Confidential

Unit Test Phase

Confidential

Integration Test Phase

Confidential

1. Feature testing is NOT performed in this phase of testing.

2. Stubs are used to replace components which are not ready for the initial phase of this testing. Stubs are replaced by actual components in the latter phase of this testing.

3. Performance, load, and reliability test types are all subsystem based that could be performed in this test phase.

4. Integration testing is hierarchical. Units are integrated into modules, and modules may be integrated into other modules or subsystems. Full system integration is the final stage of integration.

5. The target platform is used only for full system integration testingwhen ALL units become available Integration Test Phase (cont.)

Confidential

System Test Phase

Confidential

Acceptance Test Phase

Confidential

PART 3

THE RELATED ACTIVITIES IN PER TEST PHASES

Confidential

UNIT TEST PHASEObjective of the Unit Test Phase :Effective unit testing is the foundation of all later phases of testing. No amount of integration or system testing can compensate for an inadequate unit test - Boris BeizerA unit may be regarded as a single callable routine or function (eg. A function in C). Unit test cases require the unit to be defined in this manner.Note:The functionality provided by the unit is most likely NOT visible to the end-user of the system

Confidential

UNIT TEST RELATED ACTIVITIES Write Detailed Design (includes Unit & Module Level Integration Test Plans) Detailed Design Review (includes review of Unit & Module Level Integration Test Plans) Code & Write Unit Test Cases Code Inspection Unit Test Execution Unit Test Report / Unit Handover Design Group (Author or Programmer) How many Unit Test-related activities are ready?

Confidential

UNIT TEST PHASE (cont.)Deliverables to Unit TestA unit which meets the Unit Test Entrance CriteriaUnit test casesUnit test case data (input and expected output)

Note:Any new or modified unit must undergo unit testing.

Confidential

UNIT TEST PHASE (cont.)Unit Test Entrance CriteriaThe following must occur prior to the start of unit testing:Review of all specifications (Product, and Feature)Review of all appropriate design documents (High Level, and Detailed Level)Unit must have been code inspectedReviewed unit test plan available

Confidential

UNIT TEST PHASE (cont.)Deliverables from Unit TestThe following items are delivered to the integration test primes:UnitUnit test dataUnit test resultsTest report

Confidential

UNIT TEST PHASE (cont.)Unit Test Exit CriteriaA unit must successfully pass unit testing (100% unit test case execution on complete inspected code with 100% of all identified defects resolved)

Note:Defect: A defect may be identified in the code, specifications, test cases, or test plans.

Confidential

UNIT TEST PHASE (cont.)Responsibility for the Unit Test PhaseThe author or designer from the author s design group are responsible for ensuring that this test phase is carried out

Confidential

INTEGRATION TEST PHASEObjective of the Integration Test Phase:Integration testing tests modules, subsystems, and finally the full- system to ensure distinct components of the application still work in accordance to customer requirements.

Note:The test item in integration testing is either a module, subassembly,subsystem, or full-system.

Confidential

INTEGRATION TEST RELATED ACTIVITIESWrite High Level DesignHigh Level Design ReviewWrite Integration Test Plan (subsystem and full-system) & Test Cases Integration Test Plan (subsystem and full-system) & Test Case Review Integration Test Execution (SW & HW) Integration Test Report / Integrated System Handover (System Test Acceptance Meeting) Design / System Test Groups How many Integration Test- related activities are ready?

Confidential

INTEGRATION TEST PHASE (cont.)Integration Test CoverageFocus on uncovering interfacing problems (e.g. Interface problems between units within a module)Inter-unit/module/subsystem communicationFunctional testsTiming and synchronization testsReliabilityLoad and Performance Note :Performance, load, and reliability test types may be performed in theintegration test phase if it is appropriate to the primary functioning of thetest item or indicative of full system performance.

Confidential

INTEGRATION TEST PHASE (cont.)Deliverables to Integration TestThe following items must be delivered prior to integration testing:Units which have successfully passed unit testingUnit test reportIntegration test casesIntegration test dataTarget hardware platform (before full system integration testing)

Confidential

INTEGRATION TEST PHASE (cont.)Integration Entrance CriteriaThe following must occur prior to the start of integration testing:Before a unit may enter into integration testing, it must have successfully completed unit testingAll defects identified in unit test must have been resolvedUnit test results must be provided for inspectionThe reviewed integration test plan and test cases must also be available

Confidential

INTEGRATION TEST PHASE (cont.)Deliverables from Integration TestThe following items are delivered to System Test:Release notesIntegration test report

Integration Test Exit CriteriaIntegration testing must be fully integrated on the target hardware platform, and successfully complete before it meets the integration test exit criteria (100% integration test case execution, with 100% of all identified defects resolved )Test results must be made available to System Test for inspection

Note:Defect: A defect may be identified in the code, specifications,testcases, or test plans.

Confidential

INTEGRATION TEST PHASE (cont.)Responsibility for the Integration Test PhaseDesigners and/or design primes are responsible for low level integration testing. Design primes and the System Test Group are responsible for higher level integration testing and full system integration testing. In some instances, the System Test Group may be augmented by a designated set of designers

Confidential

SYSTEM TEST PHASEObjective of the System Test PhaseIn this test phase, product features are fully tested as wellas system-wide capabilities as a production-like. The tests are focused on ensuring that the system operates in conformity with the product and feature specifications.

Confidential

SYSTEM TEST RELATED ACTIVITIESParticipate in reviews of Product and Feature Specifications, and High LevelDesign Documents Write System Test Plan & Test Cases System Test Plan & Test Case ReviewSystem Test Report / System Handover (Verification Acceptance Meeting) - OptionalSystem Test Group How many System Test related activities are ready?

Confidential

SYSTEM TEST PHASE (cont.)System Test CoverageAll system features (feature testing)Feature interactionsSystem performanceCapacity limitsLoad and overloadReliabilityRecoverySecurityInstallationConversion, and expansion.

Confidential

SYSTEM TEST PHASE (cont.)Deliverables to System TestThe following items must be delivered prior to system testing:Release notesIntegration test reportMinimum hardware requiredTest tools as identified in the system test plan.

Confidential

SYSTEM TEST PHASE (cont.)System Test Entrance CriteriaTest results/reports must be provided to System Test for inspection

Confidential

SYSTEM TEST PHASE (cont.)Deliverables from System TestProblem reports (PR s) if and when any problem is discovered during testing which have not been reported beforeAt the conclusion of the system test phase, the distribution media containing the successfully tested software will be delivered to AcceptanceSystem test report and test resultsSystem Test will also issue a report on the post-mortem of the test effort for this test phase

Confidential

SYSTEM TEST PHASE (cont.)System Test Exit CriteriaSystem testing must successfully complete before it meets the system test exit criteria (100% system test case execution with the product quality requirements)The system test report and test results must be made available to the Acceptance Test group for inspection

Responsibility for the System Test PhaseThe System Test Group is responsible for ensuring that this test phase is carried out. In some instances, the System Test Group may be augmented by a designated set of designers

Confidential

ACCEPTANCE TEST PHASEObjective of the Acceptance Test PhaseAcceptance testing is focused upon determining the productsreadiness to enter the marketplace. Acceptance test will simulatetypical customer scenarios in order to ensure that the productoperates acceptably in that environment. Testing in this phase isnot expected to uncover any significant technical problems.

Confidential

ACCEPTANCE TEST RELATED ACTIVITIESAcceptance Test Groups How many Acceptance Test related activities are ready?

Confidential

ACCEPTANCE TEST PHASE (cont.)Acceptance Test CoverageThere are essentially four customer perspectives which providethe direction for Product Verification:End-userAdministratorDistributorSupport (Product Support)

Test types:InstallationConversion Upgrade Modification Network Administration

Confidential

ACCEPTANCE TEST PHASE (cont.)Acceptance Test Coverage(cont.)Compatibility Support Scenario Documentation On-line Help

Confidential

ACCEPTANCE TEST PHASE (cont.)Deliverables to Acceptance testThe following items must be delivered prior to acceptancetesting :Distribution media containing software to be testedRelease notes providing all appropriate information on the software releaseMinimum hardware as required for the product under testAll test tools identified in the acceptance test planSystem test results/report

Confidential

ACCEPTANCE TEST PHASE (cont.)Acceptance Entrance CriteriaThese items must be made available at the acceptancemeeting:Product (hardware, firmware, software, and documentation) quality must meet or exceed criteriaWorkarounds and/or description must be available for all open PR sAll system test cases must have been executed by the System Test Group

Confidential

ACCEPTANCE TEST PHASE (cont.)Acceptance Entrance Criteria (cont.)All system and designer test cases must be available for audit by AcceptanceTest results for each test must be provided by development for inspection. In the event of a fail, a PR number must also be providedReviewed acceptance test plan must be available. All acceptance test activity is provided in the test plan. The final test plan and test cases are made availableReviewed acceptance test cases must be available

Confidential

ACCEPTANCE TEST PHASE (cont.)Deliverables from AcceptanceProblem reports (PR s) if and when any problem is discovered during testing which have not been reported beforeAcceptance test report and test resultsAcceptance will also issue a report on the post-mortem of the test effort for this test phase

Confidential

ACCEPTANCE TEST PHASE (cont.)Acceptance Exit CriteriaAcceptance will recommend a product that it tests if it has quality and meets or exceeds customers requirements and expectation based on the Product specifications

Responsibility for the Acceptance Test PhaseThe Acceptance Group is responsible for ensuring that this test phase is carried out

Confidential

GLOSSARYSee the Glossary - attached file.

Confidential

REFERENCESReferent Documents:Software Testing and Quality Assurance White Papers Pointe Technology Group, Inc.

Toronto Lab Software Test ProcessNorthern Telecom

Rational Unified ProcessRational Software Corp.

Confidential

Question/ answer

Confidential

THANK YOU

Confidential

An interesting but somewhat subtle difference between Test and the other disciplines in RUP is that testing is essentially tasked with finding and exposing weaknesses in the software product. For this effort to be successful, this necessitates a somewhat negative and destructive rather than constructive approach. The challenge is to avoid both an approach that does not suitably and effectively challenge the software and expose it's inherent problems and weaknesses, and an approach that is so destructive that it will likely never find it possible to consider the quality of the software product acceptable.

Insufficient use is made of productivity tools, making the laborious aspects of testing manageable: in addition to the lack of automated test execution, many test efforts are conducted without tools that allow the effect management of extensive Test Data and Test Results. While the flexibility of use and complexity of software makes complete testing an impossible goal, a well-conceived methodology and use of state-of-the-art tools, can help to improve the productivity and effectiveness of the software testing.A continuous approach to quality, initiated early in the software lifecycle, can significantly lower the cost of completing and maintaining the software. This greatly reduces the risk associated with deploying poor quality software.

Acceptance Testing Exploratory Testing Performance Testing Structure Testing Usability Testing