CHAPTER 4 Life-Cycle Components. 2 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser 2 Learning Objectives: To discuss: Integrating quality activities in the

ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser 1

CHAPTER 4

Life-Cycle Components

2ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser

2

Learning Objectives:

To discuss:Integrating quality activities in the

project life cycleReviewsSoftware testing Integrating quality in software maintenance


Integrating Quality Activities In The Project Life Cycle

Classic and other software development methodologies

Factors affecting intensity of quality assurance activities in the development process

Model for SQA defect removal effectiveness and cost


Integrating Quality Activities In The Project Life Cycle

SQA can be planned according to the software development process and activities.

Therefore, SQA professionals should be acquainted with the various software engineering models in order to be able to prepare a quality plane that is properly integrated into the project plan.


Classic and other software development methodologies

SDLCPrototypeSpiralObject-oriented


REQUIREMENTSDETERMINATION

BY CUSTOMER

PROTOTYPE DESIGM

PROTOTYPEIMPLEMENTATION

SYSTEM CONVERSION

PROTOTYPEEVALUATION

BY CUSTOMER

SYSTEM OPERATIONAND MAINTENANCE

REQUIREMENTS FORCORRECTIONS, CHANGES

AND ADDITIONSREQUIREMENTS

FULFILLED ?

SYSTEM TESTS ANDACCEPTANCE TESTS

NO

YES

Prototype model

7ESGD5125 SEM II 2009/2010 Dr. Samy Abu NaserSource: After Boehm 1988 (© 1988 IEEE)

Spiral model


Object-oriented model



Project factorsTeam factors


Project factorsMagnitude of the projectTechnical complexity and difficultyExtent of reusable software componentsSeverity of failure outcomes if the project

fails



Team factorsProfessional qualification of the team

membersTeam acquaintance with the project and its

experience in the areaAvailability of staff members who can

professionally support the teamFamiliarity with the team members, in other

words the percentage of new staff members in the team.



Verification – The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase

Validation - The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

Qualification - The process used to determine whether a system or component is suitable for operational use

IEEE Std 610.12-1990 (IEEE 1990)

Model for SQA defect removal effectiveness and cost


A model for SQA defect removal effectiveness and cost

The model deals with two quantitative aspects of an SQA plan consisting of several defect detection activities:The plan’s total effectiveness in removing

project defects.The total costs of removal of project defects.

The plan itself is to be integrated within a project’s development process.


Defect origin distribution – from Boehm (1981) and Jones (1996)

Software development phase

Average % of defects originating

in phase, PODRequirement specification

15%

Design 35%

Unit coding 30%

Integration coding 10%

Documentation 10%

System testing -----

Operation -----


Defect removal effectiveness – from Boehm (1981) and Jones (1996)

Quality assurance activity

Defects removal effectiveness rate for standard SQA

plan

Defects removal effectiveness rate for comprehensive SQA

plan

Specification requirement review

50% 60%

Design inspection ----- 70%

Design review 50% 60%

Code inspection ----- 70%

Unit test 50% 40%

Integration tests 50% 60%

Documentation review 50% 60%

System test 50% 60%

Operation phase detection 100% 100%


Cost of defect removal – based on Boehm (1981) and Pressman (2000)

Software development phase

Average % of defects originating

in phase

Average relative defect removal

costRequirement specification

15% 1 day

Design 35% 2.5

Unit coding 30% 6.5

Integration coding 10% 16

Documentation 10% 40

System testing ----- 40

Operation ----- 110


The model

Based on the following assumptions:The development process is linear and

sequential (waterfall model).A number of “new” defects are introduced in

each development phase.Various review and test software assurance

activities serve as filters, removing a percentage of the entering defects while allowing the rest to pass to the next software assurance activity.


The model (cont)

Incoming defects are the sum of defects passed from the former quality assurance activity together with “new” defects created in the current development phase.

The cost of defect removal is calculated by multiplying the number of defects removed by the relative cost of removing a defect.

Defects passed to the customer will be detected by him or her; their full removal at this phase will incur heavy costs.

Defect removal phase Defect removal effectiveness

Average relative defect removal cost {cost unit}

Defect origination phase

Req Des Uni Int Doc

Requirement specification (Req)

50% 1 --- --- --- ---

Design (Des) 50% 2.5 1 --- --- ---

Unit coding (Uni) 50% 6.5 2.6 1 --- ---

Integration (Int)System documentation (Doc)

50%50%

1616

6.46.4

2.52.5

11

System testing / Acceptance testing (Sys)

50% 40 16 6.2 2.5 2.5

Opertion by customer (after release)

100% 110 44 17 6.9 6.9

POD = Phase Originated Defects PD = Passed Defects (from former

phase or former quality assurance activity)

%FE = % of Filtering Effectiveness (also termed % screening effectiveness)

RD = Removed Defects CDR = Cost of Defect Removal TRC = Total Removal Cost. TRC =

RD x CDR.

Ddoc Dint Duni Ddes Dreq Total

ID 15 15PD 7.5 7.5 %FE=50RD 7.5 7.5RDRC 1 TRC > 7.5 cuR

eq,

spec

ific

atio

n(P

OD

=15

)


ID 35 7.5 42.5PD 17.5 3.8 21.3 %FE=50RD 17.5 3.7 21.2RDRC 1 2.5 TRC > 26.8 cu


ID 30 17.5 3.8 51.3PD 15 8.8 1.9 25.7 %FE=50RD 15 8.7 1.9 25.6RDRC 1 2.6 6.5 TRC > 50 cu


ID 10 15 8.8 1.9 35.7PD 5 7.5 4.4 1.0 17.9 %FE=50RD 5 7.5 4.4 0.9 17.8RDRC 1 2.5 6.4 16 TRC > 66.3 cu


ID 10 5 7.5 4.4 1.0 27.9PD 5 2.5 3.8 2.2 0.5 14.0 %FE=50RD 5 2.5 3.7 2.2 0.5 13.9RDRC 1 1 2.5 6.4 16 TRC > 38.9 cu


ID 5 2.5 3.8 2.2 0.5 14.0PD 2.5 1.2 1.9 1.1 0.3 7.0 %FE=50RD 2.5 1.3 1.9 1.1 0.2 7.0RDRC 2.5 2.5 6.2 16 40 TRC > 50.9 cu


ID 2.5 1.2 1.9 1.1 0.3 7.0PD 0 0 0 0 0 0 %FE=50RD 2.5 1.2 1.9 1.1 0.3 7.0RDRC 6.9 6.9 17 44 110 TRC > 139.2 cu

Des

ign

(PO

D=

35)

Un

it t

ests

(PO

D=

30)

Inte

grat

ion

te

sts

(PO

D=

10)

Op

erat

ion

by

cust

omer

(P

OD

=0)

Sys

tem

te

sts

(PO

D=

0)

Doc

um

enta

tion

revi

ews

(PO

D=

10)21.3

25.7

17.9

7.5

14.0

7.0

Slide 7.12a – relates to updated section 7.4

Reviews


Reviews

Review objectivesFormal design reviews (DRs)Peer reviewsA comparison of the team review

methodsExpert opinion


Review objectives

Direct objectivesIndirect objectives


Direct review objectives

To detect analysis and design errors as well as subjects where corrections, changes and completions are required with respect to the original specifications and approved changes.

To identify new risks likely to affect completion of the project.

To locate deviations from templates and style procedures and conventions. Corrections of these deviations is expected to contribute to improved communication and coordination resulting from greater uniformity of methods and documentation style.

To approve the analysis or design product. Approval allows the team to continue to the next development phase.


Indirect review objectives

To provide an informal meeting place for exchange of professional knowledge about development methods, tools and techniques.

To record analysis and design errors that will serve as a basis for future corrective actions. The corrective actions are expected to improve development methods by increasing effectiveness and quality, among other product features.


Formal design reviews (DRs)

Variously called “design reviews” / ”DRs” / ”formal technical reviews (FTR)”.

Without this approval, the development team cannot continue to the next phase of the software development project.

May be conducted at any development milestone requiring completion of an analysis or design document, whether that document is a requirement specification or an installation plan.


Some common formal design reviews

DPR – Development Plan ReviewSRSR – Software Requirement Specification ReviewPDR – Preliminary Design ReviewDDR – Detailed Design ReviewDBDR – Data Base Design ReviewTPR – Test Plan ReviewSTPR – Software Test Procedure ReviewVDR – Version Description ReviewOMR – Operator Manual ReviewSMR – Support Manual ReviewTRR – Test Readiness ReviewPRR – Product Release ReviewIPR – Installation Plan Review


Participants in a DR

The review leaderThe review team


Characteristics of a DR leader

Knowledge and experience in development of projects of the type reviewed. Preliminary acquaintance with the current project is not necessary.

Seniority at a level similar if not higher than that of the project leader.

A good relationship with the project leader and his team.

A position external to the project team


The review team

Member: Senior members of the project team,

together with senior professionals assigned to other projects/departments.

Customer-user representativesSoftware development consultants.

Size: efficient team – 3 to 5 members.Large sizeproblems?


Preparation for a DR

Main tasks of the review leader in preparation stage are:To appoint the team membersTo schedule the review sessionsTo distribute the design document among

the team members.


Preparation for a DR

Review team members preparations:Review the design and list their comments

prior to the review session.Prepare checklist.

Department team preparation:Short presentation of the design document

– focus on the main professional issues awaiting approval rather than description of the project in general.


The DR session

A short presentation of the design document. Comments made by members of the review

team. Verification and validation of comments is

discussed to determine the required action items (corrections, changes and additions).

Decisions about the design product (document), which determines the project's progress: Full approval Partial approval Denial of approval


Post-review activities

The DR report DR leader is responsible to issue the DR report

immediately after the DR session. Content:

A summary of the review discussions. The decision about continuation of the project. A full list of the required action items — corrections,

changes and additions. For each action item, completion date and project team member responsible are listed.

The name(s) of the review team member(s) assigned to follow up.

Follow-up performance of the corrections and to examine the corrected sections.

Design Review Infrastructure

· Develop checklists for common types of design documents. Train senior professionals serve as a reservoir for DR teams. Periodically analyze past DR effectiveness. Schedule the DRs as part of the project plan. The Design Review Team . Review teams size should be limited, with 3–5 members being the optimum.

The Design Review Session• Discuss professional issues in a constructive way refraining

from personalizing the issues. • Keep to the review agenda. • Focus on detection of defects by verifying and validating the

participants' comments. Refrain from discussing possible solutions. • In cases of disagreement about an error - end the debate by

noting the issue and shifting its discussion to another forum. • Properly document the discussed comments, and the results of

their verification and validation. • The duration of a review session should not exceed two hours.

Post-Review Activities• Prepare the review report, including the action items• Establish follow-up to ensure the satisfactory performance of

all the list of action items


Peer reviews

Major difference between formal design reviews and peer reviews is in their participants: DR – superior positions Peer reviews – project leaders equals, member of

his/her department and other units.

Degree of authority and objectives: DR – approve the design document Peer reviews – detect errors and deviations from

standards (abnormality).


Peer review method

InspectionMore formal, corrective action, effort to

improve development method considered to contribute more significantly to the general level of SQA.

WalkthroughFindings are limited to comments on the

document reviewed.


Review leader The author Specialized

professionals: Designer Coder or

implementer Tester

Review leader The author Specialized

professionals: Standards enforcer

– expert in standards and procedures

Maintenance expert User representative

Participants of peer review


Peer review leader

Be well versed in development of projects of the current type and familiar with its technologies. Preliminary acquaintance with the current project is not necessary.

Maintain good relationships with the author and the development team.

Come from outside the project team. Display proven experience in coordination and

leadership of professional meetings. For inspections, training as a moderator is also

required.


Team assignments

The presenter Chosen by moderator (review leader). Inspection: Not the document’s author, must

understand the technical part of the reviewed document.

Walkthrough: The document’s author.

The scribe Usually team leader, record the noted defects. More procedural: requires thorough professional

understanding of the issues discussed.


Preparation of a peer review session

Peer review leader To determine (together with the author, which

sections of the design document are to be reviewed – the most difficult and complex sections, critical sections, where any defect can cause severe damage, sections prone to defects.

To select team members. To schedule the peer review sessions. To distribute the document to team members prior

to the review session.


Preparation of a peer review session

Peer review team Inspection

Thorough Team members are expected to read the document

sections to be reviewed and list comments before the inspection session begins.

Checklist. Walkthrough

Brief Team member briefly read the material to get general

overview of the section to be reviewed, the project and its environment.

Not required to prepare comments.


Peer review session

Presenter reads a section of the document and adds if needed, a brief explanation of the issues involved in his/her own words.

The participant either deliver their comments to the document. Identification of errors – not deal with tentative solutions. Walkthrough – start with the author’s short presentation or

overview of the project and the design sections to be reviewed. During the session, the scribe should document each error

recognized by location and description, type and character (incorrect, missing parts or extra parts).

Inspection – will add the estimated severity of each defect, a factor to be used in the statistical analysis of defects found and for the formulation of preventive and corrective actions.

Duration for peer review session – not more than 2 hours each session, not more than twice daily.


Session documentation

Documentation produced after the inspection session is more comprehensive than a walkthrough session.

Two documents to be produced: Inspection session findings report

Produced by the scribe, should be completed and distributed immediately after the session’s closing.

Main purpose – documentation of identified errors for correction. Inspection summary report

To be compiled by the inspection leader shortly after the session. Summarizes the inspection findings and the resources invested

in the inspection – basic quality and efficiency metrics. Input for analysis aimed at inspection process improvement and

corrective actions. Copies of error documentation will be handed to the development

team and the session participants.


The efficiency of peer review activities

Common metrics applied to estimate the efficiency of peer reviews: Peer review detection efficiency (average hours

worked per detect detected). Peer review defect detection density (average

number of defects detected per page of the design document).

Internal peer review effectiveness (% of defects detected by peer review as a percentage of total defects by the developer).

Sections recommended for inclusion

• Sections of complicated logic• Critical sections, where

defects severely damage essential system capability

• Sections dealing with new environments

• Sections designed by new or inexperienced team members

Sections recommended for omission

“Straightforward” sections (no complications)

Sections of a type already reviewed by the team in similar past projects

Sections that, if faulty, are not expected to effect functionality

Reused design and code Repeated parts of the design

and code

Properties Design review Inspection WalkthroughOverview meeting No Yes No

Participant’s preparations

Yes - thorough Yes - thorough Yes - brief

Review session Yes Yes Yes

Follow-up of corrections

Yes Yes No

Formal training of participants

No Yes No

Participant’s use of checklists

No Yes No

Error-related data collection

Not formally required

Formally required Not formally required

Review documentation

Formal design review report

1) Inspection session findings report2) Inspection session summary report


Expert opinion

Prepared by outside experts, support quality evaluation by introducing additional capabilities to the internal review staff.

Outside experts transmit their expertise by either: Preparing an expert’s judgement about a document

or a code section. Participating as a member of an internal design

review, inspection or walkthrough team.

· Insufficient in-house professional capabilities in a specialized area.

· Temporary lack of in-house professionals for review team.

· Indecisiveness caused by major disagreements among the organization’s senior professionals.

· In small organizations, where the number of suitable candidates for a review team is insufficient.

Documents

CHAPTER 4 Life-Cycle Components. 2 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser 2 Learning Objectives: To discuss: Integrating quality activities in the