Software Quality Software Quality AssuranceAssuranceBy Eng. Hany KadryBy Eng. Hany Kadry

What Is a Software ?What Is a Software ?

IEEE DefinitionIEEE Definition :- :-

Software is : computer programs, procedures, and possibly Software is : computer programs, procedures, and possibly associated documentation and data pertaining to the associated documentation and data pertaining to the operation of a computer systemoperation of a computer system

Almost identical to ISO definitionAlmost identical to ISO definition Computer programs ……… (“the code”)Computer programs ……… (“the code”) Procedures define the order and scheduleProcedures define the order and schedule Data necessary for operating the software system …… Data necessary for operating the software system ……

(“parameters, codes, …..”)(“parameters, codes, …..”) Documentations are needed for developers, users, Documentations are needed for developers, users,

maintenance personnelmaintenance personnel

The Nine Causes of Software errorsThe Nine Causes of Software errors

Faulty requirements definitionFaulty requirements definition Client-developer communication failuresClient-developer communication failures Deliberate deviation from software requirementsDeliberate deviation from software requirements Logical design errorsLogical design errors Coding errorsCoding errors Non-compliance with documentation and coding Non-compliance with documentation and coding

instructionsinstructions Shortcomings of the testing processShortcomings of the testing process Procedure errorProcedure error Documentation errors Documentation errors

Software QualitySoftware Quality

Software quality is defined asSoftware quality is defined as : conformance : conformance to explicitly stated functional and performance to explicitly stated functional and performance requirements, explicitly documented requirements, explicitly documented development standards, and development standards, and implicitimplicit characteristics that are expected of all characteristics that are expected of all professionally developed software professionally developed software

Good software engineering practices (GSEP), Good software engineering practices (GSEP), reflecting state-of-the-art professional practices, reflecting state-of-the-art professional practices, to be met by the developer even though not to be met by the developer even though not explicitly mentioned in the contractexplicitly mentioned in the contract

Difference Between Quality Control Difference Between Quality Control and Quality Assuranceand Quality Assurance

Quality control is defined as “a set of activities designed to evaluate Quality control is defined as “a set of activities designed to evaluate the quality of a developed or manufactured product” (IEEE), in other the quality of a developed or manufactured product” (IEEE), in other words activities whose main objective is the withholding of any words activities whose main objective is the withholding of any product that does not qualifyproduct that does not qualify

The main objective of quality assurance is to minimize the cost of The main objective of quality assurance is to minimize the cost of guaranteeing quality by a variety of activities performed throughout guaranteeing quality by a variety of activities performed throughout the development and manufacturing. These activities prevent the the development and manufacturing. These activities prevent the cause of errors, and detect and correct them early in the cause of errors, and detect and correct them early in the development processdevelopment process

As a result, QA activities substantially reduce the rate of products As a result, QA activities substantially reduce the rate of products that do not qualify for shipment, and at the same time, reduce the that do not qualify for shipment, and at the same time, reduce the cost of guaranteeing quality in most casescost of guaranteeing quality in most cases

QC activities are only a part of the total range of QA activities QC activities are only a part of the total range of QA activities

SQASQA

Software quality assurance is :-Software quality assurance is :-

a systematic , planned set of actions necessary to a systematic , planned set of actions necessary to provide adequate confidence that the software provide adequate confidence that the software development process or the maintenance development process or the maintenance process of a software system product conforms process of a software system product conforms to established functional technical requirements to established functional technical requirements as well as with the managerial requirement of as well as with the managerial requirement of keeping the schedule and operating costs with keeping the schedule and operating costs with the budgetary confines (ISO, CMM)the budgetary confines (ISO, CMM)

SQA System OverviewSQA System Overview

SQA System ArchitectureSQA System Architecture

An SQA system combines a wide range of SQA An SQA system combines a wide range of SQA componentscomponents

The components are employed to challenge the The components are employed to challenge the multitude of sources of SW errors and to achieve an multitude of sources of SW errors and to achieve an acceptable level of SW qualityacceptable level of SW quality

Two considerationsTwo considerations :- :- Task of SQA is unique due to the specific characteristics of swTask of SQA is unique due to the specific characteristics of sw The environment in which sw development and maintenance is The environment in which sw development and maintenance is

undertaken directly influence of the SQA componentsundertaken directly influence of the SQA components

SQA System Components SQA System Components ClassesClasses

Components can be classified into six Components can be classified into six classesclasses :- :- Pre-project quality componentsPre-project quality components Project lifecycle quality componentsProject lifecycle quality components Infrastructure error preventive and improvement Infrastructure error preventive and improvement

componentscomponents Software quality management components Software quality management components Standardization, Certification and SQA assessment Standardization, Certification and SQA assessment

componentscomponents Organizing for SQA – the human componentsOrganizing for SQA – the human components

1- Pre-Project 1- Pre-Project ComponentsComponents

1- Pre-Project Components1- Pre-Project Components

Objective is to assure that :-Objective is to assure that :-

(a) the project commitments have been (a) the project commitments have been adequately defined considering :-adequately defined considering :- The resources requiredThe resources required The scheduleThe schedule The budgetThe budget

(b) the development and quality plans have (b) the development and quality plans have been correctly determinedbeen correctly determined

……. Cont. Cont

These SQA components are meant to These SQA components are meant to improve the preparatory steps prior to improve the preparatory steps prior to initiating work on the project itself :-initiating work on the project itself :-

Contract reviewContract review

Development and quality plansDevelopment and quality plans

Contract reviewContract review

SW may be developed within a framework SW may be developed within a framework of :-of :-

A contract negotiated with a customerA contract negotiated with a customer

Internal order originating in another Internal order originating in another department :-department :- Order of software product to be sold as a packageOrder of software product to be sold as a package An internal software application to be used by the An internal software application to be used by the

companycompany

Contract review …. contContract review …. cont

In all instances, the development unit is In all instances, the development unit is committed to an agreed-upon :-committed to an agreed-upon :-

Functional specificationsFunctional specifications BudgetBudget scheduleschedule

…….. cont.. cont

Accordingly, contract review activities Accordingly, contract review activities must include a detailed examination of :-must include a detailed examination of :-

(a)- The project proposal draft(a)- The project proposal draft (b)- the contract draft(b)- the contract draft

……. cont. cont

Contract review activities specifically include :-Contract review activities specifically include :-

Clarification of the customer’s requirementsClarification of the customer’s requirements Review of the project’s schedule and resource Review of the project’s schedule and resource

requirement estimatesrequirement estimates Evaluation of the professional staff’s capacity to carry Evaluation of the professional staff’s capacity to carry

out the proposed projectout the proposed project Evaluation of the customer’s capacity to fulfill his Evaluation of the customer’s capacity to fulfill his

obligationsobligations Evaluation of development risks Evaluation of development risks

Development and Quality PlansDevelopment and Quality Plans

OverviewOverview Once the contract has been signed or a commitment has Once the contract has been signed or a commitment has

been made to undertake an internal project, a plan is been made to undertake an internal project, a plan is prepared of the project (“development plan”) and its prepared of the project (“development plan”) and its integrated quality assurance activities (“quality plan”)integrated quality assurance activities (“quality plan”)

These plans include additional details and needed These plans include additional details and needed revisions based on prior plans that provided the basis for revisions based on prior plans that provided the basis for the current proposal and contractthe current proposal and contract

NoteNote : it is quite common for several months to pass between : it is quite common for several months to pass between the tender submission and the signing of the contractthe tender submission and the signing of the contract

……. cont. cont

The main issues treated in the development The main issues treated in the development plan are :-plan are :- SchedulesSchedules Required manpower and hardware resourcesRequired manpower and hardware resources Risk evaluationsRisk evaluations Organizational issues: team members, subcontractors Organizational issues: team members, subcontractors

and partnershipand partnership Project methodology , development tools, ..etcProject methodology , development tools, ..etc Software reuse plansSoftware reuse plans

……. cont. cont

The main issues treated in the project’s The main issues treated in the project’s quality plan are :-quality plan are :-

Quality goals, expressed in the appropriate Quality goals, expressed in the appropriate measurable termsmeasurable terms

Criteria for starting , and ending each project Criteria for starting , and ending each project stagestage

List of reviews , tests, and other scheduled List of reviews , tests, and other scheduled verification and validation activitiesverification and validation activities

2- Software Project 2- Software Project Lifecycle Components Lifecycle Components

2- Software Project Lifecycle 2- Software Project Lifecycle ComponentsComponents

Project lifecycle is composed of two Project lifecycle is composed of two stages :-stages :-

The development lifecycle stageThe development lifecycle stage

The operation-maintenance stage The operation-maintenance stage

……. cont. cont

Several SQA components enter the lifecycle at Several SQA components enter the lifecycle at different pointsdifferent points

They should be planned prior to project’s They should be planned prior to project’s initiation. initiation. They areThey are :- :- ReviewsReviews Expert opinionExpert opinion Software testingSoftware testing Software maintenanceSoftware maintenance Assurance of the quality of the subcontractor’s work Assurance of the quality of the subcontractor’s work

and the customer-supplied partsand the customer-supplied parts

ReviewsReviews

The design phase produces a variety of The design phase produces a variety of documents :-documents :- Design reportsDesign reports Software test documentsSoftware test documents Software installation plansSoftware installation plans Software manualsSoftware manuals OthersOthers

Reviews can be categorized into :-Reviews can be categorized into :- Formal design reviews (DRs)Formal design reviews (DRs) Peer reviewsPeer reviews

Formal Design Reviews (DRs)Formal Design Reviews (DRs)

The committee:- senior professionals The committee:- senior professionals including :including : The project leaderThe project leader The department managerThe department manager The chief software engineerThe chief software engineer Heads of other related departmentsHeads of other related departments

The majority of participants hold The majority of participants hold professional and administrative ranks professional and administrative ranks higher than the project leaderhigher than the project leader

The DRsThe DRs

The DR report includes a list of required The DR report includes a list of required corrections (“action items”)corrections (“action items”)

The committee has several options :-The committee has several options :-

Immediate approval to continue to the next Immediate approval to continue to the next development phasedevelopment phase

Approval to proceed after all action items have been Approval to proceed after all action items have been completed and inspected by the committee completed and inspected by the committee representativerepresentative

An additional DR is required and scheduled after all An additional DR is required and scheduled after all action items have been completed and inspected by action items have been completed and inspected by the committee representativethe committee representative

Peer ReviewsPeer Reviews

Are directed at reviewing short, parts of a Are directed at reviewing short, parts of a report, a coded printout of a software report, a coded printout of a software modulemodule

The main objective is to detect as many The main objective is to detect as many design and programming faults as design and programming faults as possible possible

Expert OpinionsExpert Opinions

Outside expertsOutside experts Useful in the following situations :-Useful in the following situations :-

Insufficient in-house capabilities in a given Insufficient in-house capabilities in a given areaarea

In small organization, the expert can join the In small organization, the expert can join the DRDR

Extreme work pressureExtreme work pressure In case of major disagreementIn case of major disagreement

Software TestingSoftware Testing

Test cases that represent a variety of Test cases that represent a variety of expected scenariosexpected scenarios

All tests should be designed and planned All tests should be designed and planned and approvedand approved

Highly recommended to test by Highly recommended to test by independent test unit rather than the independent test unit rather than the project teamproject team

Tests can be manual or automated Tests can be manual or automated

Software Maintenance Software Maintenance ComponentsComponents

Maintenance services fall into :-Maintenance services fall into :- Corrective maintenanceCorrective maintenance Adaptive maintenance (ex: HW change)Adaptive maintenance (ex: HW change) Functionality improvement maintenance (functionality, Functionality improvement maintenance (functionality,

performance)performance)

Pre-maintenance componentsPre-maintenance components Maintenance contract reviewMaintenance contract review Maintenance planMaintenance plan

Software development lifecycle componentsSoftware development lifecycle components

3- Infrastructure 3- Infrastructure ComponentsComponents

3- Infrastructure components For 3- Infrastructure components For Error Prevention and ImprovementError Prevention and Improvement

The goals of SQA infrastructure are the The goals of SQA infrastructure are the prevention or lowering the software faults prevention or lowering the software faults ratesrates

This class of components includeThis class of components include :- :- Procedures and work instructionsProcedures and work instructions Templates and checklistsTemplates and checklists Staff training, retraining and certificationStaff training, retraining and certification Configuration managementConfiguration management Documentation controlDocumentation control

Procedures and Work InstructionsProcedures and Work Instructions

QA procedures usually provide detailed QA procedures usually provide detailed definitions for the performance of specific definitions for the performance of specific types of development activities in a way types of development activities in a way that assures effective achieving quality that assures effective achieving quality resultsresults

The collection of all SQA procedures is The collection of all SQA procedures is usually referred as the SQA procedures usually referred as the SQA procedures manual manual

Work InstructionsWork Instructions

Work instructions deal with the application Work instructions deal with the application of procedures adapted to the requirements of procedures adapted to the requirements of a specific project team, customer, or of a specific project team, customer, or other relevant partyother relevant party

While general methodology is defined in a While general methodology is defined in a procedure, the specific details that allow procedure, the specific details that allow its application its application to a specific projectto a specific project or unit or unit are often laid out in a work procedureare often laid out in a work procedure

Templates & ChecklistsTemplates & Checklists

One way to combine higher quality with One way to combine higher quality with higher efficiency is to use supporting higher efficiency is to use supporting quality devices , such as templates and quality devices , such as templates and checklistschecklists

checklistschecklists

Checklist refers to the list of items Checklist refers to the list of items specially constructed for each type of specially constructed for each type of document, or menu of preparations to be document, or menu of preparations to be completed prior to perform an activitycompleted prior to perform an activity

Staff Training, Instruction and Staff Training, Instruction and CertificationCertification

Trained and well-instructed professional staff is the key Trained and well-instructed professional staff is the key to efficient , quality performanceto efficient , quality performance

Within the frame of SQA , keeping an organization’s Within the frame of SQA , keeping an organization’s human resources knowledgeable and updated at the human resources knowledgeable and updated at the level required is achieved by :-level required is achieved by :- Training new employees and retraining those who have changed Training new employees and retraining those who have changed

assignmentsassignments Continuously updating staff with respect to professional Continuously updating staff with respect to professional

developmentsdevelopments Certifying employees after their knowledge and ability have been Certifying employees after their knowledge and ability have been

demonstrated demonstrated

Prevention & Corrective ActionsPrevention & Corrective Actions

Systematic study of the data collected regarding Systematic study of the data collected regarding instances of failure and success contribute to instances of failure and success contribute to the QA :-the QA :- Implementation of changes that prevent similar Implementation of changes that prevent similar

failures in the futurefailures in the future Correction of similar faults founds in other projects Correction of similar faults founds in other projects Implementing proven successful methodologies to Implementing proven successful methodologies to

enhance the probability of repeat successesenhance the probability of repeat successes

The source of data are design review reports, The source of data are design review reports, software test reports, and customer software test reports, and customer complaints…complaints…

Configuration ManagementConfiguration Management

The regular software development and The regular software development and maintenance operations involve intensive maintenance operations involve intensive activities that modify software to create new activities that modify software to create new versions and releasesversions and releases

Different team members carry out these Different team members carry out these activities simultaneouslyactivities simultaneously

As a result serious dangers arise …!!As a result serious dangers arise …!! CM deals with these hazards by introducing CM deals with these hazards by introducing

procedures to control the change process procedures to control the change process

…….. cont.. cont

CM procedures relate to :-CM procedures relate to :- The approval of changeThe approval of change The recording of the changesThe recording of the changes The issuing of new SW versions and releasesThe issuing of new SW versions and releases The recording of the versions and releases The recording of the versions and releases

specifications of SW installed in each sitespecifications of SW installed in each site The prevention of any changes in approved versions The prevention of any changes in approved versions

and releases once they are issuedand releases once they are issued

Most CM systems implement computerized toolsMost CM systems implement computerized tools

Documentation ControlDocumentation Control

SQA requires the application of measures to SQA requires the application of measures to ensure the efficient long-term availability of ensure the efficient long-term availability of major documents related to software major documents related to software development (“controlled documents”)development (“controlled documents”)

Controlled documents contain information Controlled documents contain information important to the long-term development and important to the long-term development and maintenance of the software system such as :-maintenance of the software system such as :- SW test resultsSW test results Design review (DR) reportsDesign review (DR) reports Problem reportsProblem reports

4- Management SQA 4- Management SQA ComponentsComponents

4-Management SQA Components4-Management SQA Components

Supports the managerial control of Supports the managerial control of software development projects and software development projects and maintenance services. maintenance services. They includeThey include :- :-

Project progress controlProject progress control SW quality metricsSW quality metrics SW quality costsSW quality costs

Project Progress ControlProject Progress Control

The main objective of project progress control is The main objective of project progress control is to detect the appearance of any situation that to detect the appearance of any situation that may induce deviations from the project’s plan may induce deviations from the project’s plan and maintenance service performance. Project and maintenance service performance. Project progress control activities focus on :-progress control activities focus on :-

Resource usageResource usage SchedulesSchedules Risk management activitiesRisk management activities The budgetThe budget

Software Quality MetricsSoftware Quality Metrics

The measurements apply to the functional The measurements apply to the functional quality, productivity, and organizational aspects quality, productivity, and organizational aspects of the projectof the project

SW quality metrics areSW quality metrics are :- :-

Quality of SW development & maintenance activitiesQuality of SW development & maintenance activities Development team’s productivityDevelopment team’s productivity Helpdesk and maintenance team’s productivityHelpdesk and maintenance team’s productivity SW faults densitySW faults density

5- SQA Standards, System 5- SQA Standards, System Certification, and Assessment Certification, and Assessment

ComponentsComponents

5- SQA Standards, System Certification, 5- SQA Standards, System Certification, Assessment ComponentsAssessment Components

External tools offer another avenue to achieving External tools offer another avenue to achieving the goal of SQAthe goal of SQA

The main objectives of this class of The main objectives of this class of components are :-components are :-

Utilization of international professional knowledgeUtilization of international professional knowledge Improvement of coordination with other organizations’ Improvement of coordination with other organizations’

quality systemsquality systems Objective professional evaluation and measurement Objective professional evaluation and measurement

of the achievements of the organization’s quality of the achievements of the organization’s quality systemssystems

Standards availableStandards available

The standards available may be classified into two main The standards available may be classified into two main sub-classes :-sub-classes :-

Quality management standardsQuality management standards :- focus on the “what” is :- focus on the “what” is required and leave the “how” to achieve it to the organization :-required and leave the “how” to achieve it to the organization :- SEI CMM assessment standardSEI CMM assessment standard ISO 9001 and ISO 9000-3 standardsISO 9001 and ISO 9000-3 standards

Project process StandardsProject process Standards :- methodological guidelines :- methodological guidelines (dealing with the “how”) for the development team :-(dealing with the “how”) for the development team :- IEEE 1012 standardIEEE 1012 standard ISO/IEC 12207 standard ISO/IEC 12207 standard

6- Organizing for SQA6- Organizing for SQAThe Human Components The Human Components

6- Organizing For SQA6- Organizing For SQAThe Human ComponentsThe Human Components

SQA cannot be applied in an SQA cannot be applied in an organizational vacuum: they require an organizational vacuum: they require an organizational baseorganizational base

The base (organizational SW quality The base (organizational SW quality framework) includes :-framework) includes :- Organization’s managementOrganization’s management SQA Unit and SW testing personnelSQA Unit and SW testing personnel

Mission of The Organizational BaseMission of The Organizational Base

Develop and support implementation of SQA Develop and support implementation of SQA componentscomponents

Detect deviations from SQA procedures and Detect deviations from SQA procedures and methodologymethodology

Suggest improvements to the SQA componentsSuggest improvements to the SQA components

Note : although the entire organizational base shares these Note : although the entire organizational base shares these objectives, each segment of the organizational base objectives, each segment of the organizational base concentrates on specific tasks concentrates on specific tasks

Management’s Role in SQAManagement’s Role in SQA

The responsibilities of top management (through The responsibilities of top management (through the executive in charge of SW quality), the executive in charge of SW quality), departmental management and project departmental management and project management include the following:management include the following: Definition of the quality policyDefinition of the quality policy Effective follow-up of quality policy implementationEffective follow-up of quality policy implementation Allocation of sufficient resources to the Allocation of sufficient resources to the

implementation of the quality policyimplementation of the quality policy Assignment of adequate staffAssignment of adequate staff Follow-up complianceFollow-up compliance

The SQA UnitThe SQA Unit

This unit and SW testers devote their full This unit and SW testers devote their full time to SQA matters :-time to SQA matters :- Preparation of annual quality programsPreparation of annual quality programs Consultation with in-house staff and outside Consultation with in-house staff and outside

experts on SW quality issuesexperts on SW quality issues Conduct of internal QA auditsConduct of internal QA audits Support of existing quality assurance Support of existing quality assurance

infrastructure components and their updates, infrastructure components and their updates, and development of new components and development of new components

Software Quality MetricsSoftware Quality Metrics

Metrics ClassificationsMetrics Classifications

First classification distinguishes between First classification distinguishes between the development lifecycle and other the development lifecycle and other phases of a software system :-phases of a software system :-

Process metrics ……….. DevelopmentProcess metrics ……….. Development

Product metrics ……….. MaintenanceProduct metrics ……….. Maintenance

……. cont. cont

The second classification category refers The second classification category refers to the subject of the measurement :-to the subject of the measurement :-

QualityQuality TimetableTimetable Effectiveness (of error removal)Effectiveness (of error removal) productivityproductivity

Estimating the Software SizeEstimating the Software Size

KLOC (thousands of code lines)KLOC (thousands of code lines) Depends on the programming languageDepends on the programming language

FP (function points)FP (function points) Does not depend on the programming Does not depend on the programming

languagelanguage

Sizing … contSizing … cont

Process MetricsProcess Metrics

Process metrics categories :-Process metrics categories :-

SW process quality metricsSW process quality metrics SW process timetable metricsSW process timetable metrics Error removal effectiveness metricsError removal effectiveness metrics SW process productivity metricsSW process productivity metrics

The Function Point MethodThe Function Point Method

It measures the project/program size by It measures the project/program size by functionality not by KLOC (the number of functionality not by KLOC (the number of code lines can only be counted after code lines can only be counted after programming completion which occurs at programming completion which occurs at a very late stage of the project)a very late stage of the project)

Function Point CalculationFunction Point Calculation

FP = CFP X (0.65 + 0.01 X RCAF)FP = CFP X (0.65 + 0.01 X RCAF)

CFP = Crude Function PointCFP = Crude Function Point

RCAF = Relative Complexity Adjustment FactorRCAF = Relative Complexity Adjustment Factor

Thank YouThank You