CSE Senior Design IICSE Senior Design II

Test PlanningTest Planning

Mike O’DellMike O’Dell

Based on an earlier presentation by Based on an earlier presentation by Mike O’Dell, UTAMike O’Dell, UTA

1

CSE 4317 2

Test Planning … when?Test Planning … when?

Occurs throughout the product life cycle Requirement completeness Architectural integrity Component design precision Implementation correctness

But all too often we wait until the end to “validate” what we have already done.

1

CSE 4317 3

System VerificationSystem Verification

SRD: System Requirements

ADS: ArchitectureSpecification

DDS: Detailed Design Specification

Implementation

Integration Test

Component Test (a.k.a.

Function Test)Unit Test

System Validation

Test

System Verification

System Definition

MAP:All SpecifiedRequirements

MAP:All ModuleInterfaces &Interactions

MAP:All Modules

MAP:All Subsystem & LayerInterfaces &Interactions

1 System VerificationSystem Verification

Unit Test: verifies that EVERY module (Hardware & Software) specified in the DDS operates as specified.

Component/Function Test: verifies integrity of ALL inter-module interfaces and interactions.

Integration Test: verifies integrity of ALL inter-subsystem/layer interfaces and interactions.

System Verification Test: verifies ALL requirements are met.

CSE 4317 4

1 System Test Plan ContentsSystem Test Plan Contents

IntroductionExecutive Summary (purpose, scope, etc.)References

SRD (requirements drive the system-level tests you do)

ADS (architecture drives the subsystem and integration testing)

Etc. (detailed design, standards, sample plans, etc. that are test drivers)

Test Item(s) – What will be tested

CSE 4317 5

1 System Test Plan ContentsSystem Test Plan Contents

Design decomposition (relationship of specific modules to test plan, test cases)

Limitations of product under test (restrictions, assumptions, caveats, etc.)

Other product-level restraints on testingRisks (in test phase)

Specific risks that may affect testing/test outcome Impact assessment and management plan as it

relates to these risks

CSE 4317 6

1 System Test Plan ContentsSystem Test Plan Contents

Features to be tested List of specific features (from the USERS’

viewpoint) that you will test

Features NOT to be tested List of those features (USERS’ perspective) that

cannot/will not be tested Describe rationale for not testing each item (not in

current release, insufficient test capability, low risk feature that is hard to test, not a user-visible feature)

CSE 4317 7

1 System Test Plan ContentsSystem Test Plan Contents

Approach (Strategy) Overall test strategy (what, why, and how of your

test plan) Number of hardware and software configurations

tested/not tested Plan to deal with defects identified (re-integration,

regression) Identify metrics for overall success (number of

bugs found/corrected/open, number of critical features passed, etc.)

Identify any special requirements for testing

CSE 4317 8

1 System Test Plan ContentsSystem Test Plan Contents

Item Pass/Fail criteria List specific criteria for each unit/module/feature

to be tested (number and severity of defects, any specific tests that indicate system failure, etc.)

Test Deliverables List what is to be delivered by this test plan (plan

document, test cases, error logs, etc.) Incident/Bug/Problem Reports (IRs)

Test Schedule (should be consistent with your published project plan)

CSE 4317 9

1 System Test Plan - ResourcesSystem Test Plan - Resources

IEEE Std 829-1998, IEEE Standard for Software Test Documentation

Test Plan Outline for IEEE Format (from Gerrard Consulting)

CSE 4317 10