The role of QA in software development

Guido d’AmicoQA leadBEA Systems Inc.

Bio

QA lead for the WebLogic Sip Server(WL Communication Platform, 3 releases).

Prior to that: QA manager for wireless option of Oracle’s iAS, wireless portal and development tools.

Over 9 years in QA.

Before that: training.

BS in applied mathematics

Agenda

Problems posed by quality.

What is it?

How we measure it?

Quality during the sw dev cycle.

What tests need to be there.

What to look for in a test.

When to plan and when to run tests.

Q & A

Problems posed by quality

What is it?

A key measure for the success of a product.

Inherently internal to the product (cannot be modified by policy), best assessed by the final user.

How is it measured?

One of the best way is to measure the value of some known predictive attributes of the code in the making. Thee measures on these attributes can be internal (measured on the code statically and in beta versions) or external (measured running the code).

Typical measures are: number of features (e), of bugs (i/e), benchmark results (e), integration with other products(e), resource consumption (i/e), test coverage(i), etc...

These measures will have an essential role in influencing the other two key success factors: time and cost.

Quality in the SW cycle

What tests?

It really depends on the attributes defined. It’s fundamental to define the attributes before the cycle begins. It cannot always happens then, but that should be the goal.

Standard tests (easy to define):

Unit tests

Feature tests

System tests

Complex tests (hard to define):

Performance tests

Stability tests

Define the hard ones first!

Quality in the SW cycle

What to look for in a test

Characteristics of a good test:

Is reproducible and consistent in its strategy--not necessarily in its results (will make you colleagues happy: QA, developers, support),

Is focused: goes deep into the feature, capture all the measures available for a specific attribute (will make your manager happy),

Is as small as possible (in steps, technologies and features involved),

Yields to reportable results (will make your VP happy).

Quality in the SW cycle

When to plan for tests:

Performance and regression tests: at the beginning of the cycle. Be sure to have two regression suites that can be run quickly at check-in (less than one hour) and nightly (for general regressions).

Feature tests: when the feature is defined.

Unit test: as soon as there is some interface definition available.

When to run them:

Regression: right away.

Feature: as soon as the feature’s first cut code is delivered.

Unit: as soon as there is some code available.

System: once the code is feature-complete.

Performance: once the code for that feature is complete.