10.Process Product Quality Assurance

8/13/2019 10.Process Product Quality Assurance

1/54

Software Quality Assurancefor

Process and Product QualityAssurance

Magister Teknologi Informasi


2/54

Staged Representation

Provides a defined

roadmap for organizationalimprovement Lower level processes are

more critical to repeatable

success

Higher level processes build onthe lower level processes

Level4Quantitatively

Managed

Level1Performed

Level2Managed

Level5Optimizing

Level3Defined

CausalAnalysisandResolution

OrganizationalInnovationandDeployment

QuantitativeProjectManagementOrganizationalProcessPerformance

RequirementsDevelopmentTechnicalSolutionProductIntegrationVerificationValidationOrganizationalProcessFocusOrganizationalProcessDefinitionOrganizationalTrainingRiskManagementIntegratedProjectManagement(forIPPD*)IntegratedTeaming*IntegratedSupplierManagement**DecisionAnalysisandResolutionOrganizationalEnvironmentforIntegration*

RequirementsManagementProjectPlanningProjectMonitoringandControlSupplierAgreement

ManagementMeasurementandAnalysisProcessandProductQuality

AssuranceConfigurationManagement


3/54

Topics

SQA Software Quality Assurance

Soft Inspection & Reviews (Verification)

SQM Software Quality Management

Defect Prevention


4/54

CMMI-Process and ProductQuality Assurance (PPQA)

The purpose of PPQA is to provide staff andmanagement with objective insight into

processes and associated work products

The Process and Product Quality Assurance

process area supports the delivery of high-

quality products by providing project staff and

managers at all levels with appropriate visibility

into, and feedback on, processes andassociated work products throughout the life of

the project.


5/54

CMMI-Process and ProductQuality Assurance (PPQA)..

Activities: Objectively evaluating performed processes andwork products against applicable process

descriptions, standards, and procedures

Identifying and documenting noncomplianceissues

Providing feedback to project staff and managerson the results of quality assurance activities

Ensuring that noncompliance issues areaddressed


6/54

Specific Goal and Practice

SG 1 Objectively Evaluate Processesand Work Products

SP 1.1 Objectively Evaluate Processes

SP 1.2 Objectively Evaluate Work Products SG 2 Provide Objective Insight

SP 2.1 Communicate and ResolveNoncompliance Issues

SP 2.2 Establish Records


7/54

PPQA vs. Verification

The practices in PPQA ensure that plannedprocesses are implemented, while the practices in

the Verification process area ensure that specified

requirements are satisfied.

These two process areas can on occasion addressthe same work product but from different

perspectives.

Projects should take advantage of the overlap to

minimize duplication of effort while taking care to

maintain separate perspectives


8/54


9/54

Software failures from time to time

RMI (random memory initiation) Boeing 737-300 crashed in M1 England, April 1989

Denver International Airport, Feb 1994

Air-Traffic Control System in LA Airport, Sept

04 Southwest Airline crash in Chicago, Dec 05

Software failure at Atlanta's Hartsfield-JacksonAirport, Apr 06

MS Excell 2007 multiplication bugs Volvo Cars: software engine failure, Sept 09


10/54

Volvo Recall Sept 2009

Volvo Cars of North America is recallingabout 12,000 vehicles, made between

March 2007 and August 2009, because

of problems with electronic software thatcould keep the fuel pump from starting.

The defect could cause the engine to

stall, potentially resulting in a crash, saidthe National Highway Traffic Safety

Administration.


11/54

MS-Excel 2007 Multiplication bugs


12/54

Impact of software failure

Not just out there Space shuttleAriane 5

But also at home Telephone bill

How about ICR used by the KPU ?


13/54

What is Quality ?

Quality isthe totality of features and characteristics of a

product that bear on its ability to meet stated

or implied needs

Quality is spoken of in terms of

Product quality (software quality) Process quality


14/54

Software Quality

Product quality

conformance to requirements

fitness for use freedom from errors and failures

customer satisfaction SW quality is normally spoken of in terms

of several different dimensions often

called parameters technical quality parameters user quality parameters


15/54

SW Quality parameters

Technical parameters

correctness, reliability, capability,performance, maintainability

these are open to objective measures andtechnical solutions

User parameters usability, installability, documentation,

availability

these often require subjective analysis andnon-technical solutions


16/54

Technical parameters

Correctness: measured in terms of defect rate

Reliability measured in terms of failure rate

Capability

measured in terms of requirement coverage Maintainability

measured in terms of change log Performance

measured in terms of speed and resourceusage


17/54


18/54

Quality characteristics & features(ISO/IEC 9126)

Functionality Reliability

Usability

Efficiency

Maintainability

PortabilityCan be objective or subjective


19/54

Basic principles

Achieving quality

it does not happen by accident, and is notsomething that can be added on after the fact

so we must plan for it from the beginning andcontinuously monitor it day by day

Three general principles know what you are doing know what you should be doing know how to measure the difference


20/54

Principle-1: know what you are doing

It means continuously understanding what itis you are building, how you are building it,and what it currently does

this requires organization, including havinga management structure, reporting policies,regular meeting and reviews, and so on

normally addressed by following a SW

process with regular milestones, planning,scheduling, and tracking procedures


21/54

This means having explicit requirements andspecifications

these must be continuously updated and

tracked as part of the development cycle

normally addressed by requirements and use-

case analysis, explicit acceptance tests with

expected results, frequent user feedback

particular procedures and methods are usually

part of process

Principle-2: know what you shouldbe doing


22/54

This means having standards and bestpractices

achieved using four complementary methods

formal methods testing inspection metric

Principle-3: know how to measurethe difference


23/54

Software Eng vs traditional eng

Lack of maturity Number of combinations of inputs almost

infinite, exhaustive testing impossible

Software is discontinuous

Exhibits only systematic failures

Changes seem to be easy, software isseen as "the easy fix


24/54

Software Quality Assurance

Definition a planned and systematic pattern of actions toprovide adequate confidence that the productconforms to established technical

requirements Purpose

to ensure that the project standard and

procedures are adequate and are adhered tothroughout the project


25/54

Assuring Quality

Assure that each of the software qualities ismet Goals set in requirements specification Goals realized in implementation

Sometimes easy, sometimes difficult Portability versus safety

Sometimes immediate, sometimes delayed

Understandability versus evolvability

Sometimes provable, sometimes doubtful Size versus correctness


26/54

Complexity of QA

No matter how sophisticated the QA process,the problem of creating the initial specificationremains

Sometimes, the software system is extremely

complicated making it tremendously difficult toperform QA

It is difficult to divide the particularresponsibilities involved when performingquality assurance

Quality assurance has a negative connotation


27/54

Story of Software Development


28/54

SQ Management Tasks

Improve software quality!!!

Know applicable standards and introduce them intocompany

Define company internal software development process

Support Software Developers

document templates (Design Templates,) work instructions (Coding Standards) tools (e.g. for quality metrics) support with the conduct of requirements and code reviews

Provide quality related training for developers For Safety Related Systems: be able to "prove" software

safety


29/54

Standards

A standard is

"an authoritative or recognised exemplar ofcorrectness, perfection or some definite degree of anyquality"

Software development for safety critical

systems guided by a huge amount ofstandards! "Standards are wonderful things if you don't like one

there are so many more to choose from" (Anonymous)

In practice, little consistency between sectors between countries


30/54

SWQM Standards Quagmire


31/54

For software safety


32/54

SQ Factors & QA-SComponents


33/54

Software quality factors

Product operation factors

Product revision factors

Product transition factors


34/54

Correctness

Reliability Efficiency

Integrity

Usability


35/54

Maintainability

Flexibility

Testability


36/54

Portability

Reusability

Interoperability


37/54


38/54

No. Software qualityfactor

McCalls classicmodel

Alternative factor models

Evans andMarciniak model

Deutsch andWillis model

1 Correctness + + +2 Reliability + + +

3 Efficiency + + +

4 Integrity + + +

5 Usability + + +

6 Maintainability + + +

7 Flexibility + + +

8 Testability +

9 Portability + + +

10 Reusability + + +

11 Interoperability + + +

12 Verifiability + +

13 Expandability + +

14 Safety +

15 Manageability +

16 Survivability +


39/54

SW-Dev: A complete picture

Sales Prepare Develop Roll-out Closure

PrePost

QA-Coverage


40/54

Components of SQA-S

Pre-project components Project life cycle components

Infrastructure for error prevention and

improvement Quality management

Standard, certification and assessment Human resources


41/54

The Software Quality Shrine

ProjectDevelopment planand Quality Plan

Ch.6

Project Life Cycle SQA components

FormalD

esignReviews

S

ec.

8.2

SQA

ofExte

rnalParticipants

Ch12

Quality Infrastructure components

ProceduresCh. 14

Supporting

Devices

Ch. 15

Training

Instruction

Ch. 16

Preventive

Act ion s

Ch.17

Configuration

Management

Ch. 18

Document-ation

ControlCh. 19

Quality Management

ProjectProgress

ControlCh. 20

SoftwareQuality

MetricsCh. 21

SoftwareQuality

CostsCh. 22

QualityManagement

StandardsCh. 23

Standards

ProjectProcessStandards

Ch.24

Organizational Base Human components

Management - Ch. 25 SQA Unit - Sec. 26.1 SQA Committees Sec. 26.2SQA Trustees Sec. 26.2 SQA Forums Sec 26.4

Contract review

Ch.5

OHT 4.41


42/54

Pre-Project Components

Objective: to improve the preparatory steps

taken prior to initiating work on the project Elements:

Contract Review Development and quality plan

Contract Review: Clarification of customer requirements Project schedule and resources requirement Staffing Evaluate customers capacity to fulfill his obligation Evaluation of development risks


43/54

Common problems in Contract

Vaguely defined requirements

The rights as well as obligations of each parties arenot clearly stated, or

Perceived differently by both parties

Unrealistic budget and schedules Inaccurate cost estimation Shallow and quick resource estimate

Excessive pressure from management

To save time and resources To increase revenue


44/54

Participation in a tender Proposal submission according to

customers RFP

Receipt of an order from a companyscustomer

Internal request from anotherdepartment in the organization


45/54

Proposal draft review

+ Contract draft review---------------------------

Contract review


46/54

To make sure that the following activities have been satisfactorily

carried out:1. Customer requirements clarified and documented

2. Alternative approaches for carrying out the project examined

3. Formal aspects of the relationship between the customer and

the software firm specified4. Development risks identified

5. Project resources and timetable adequately estimated

6. The firms capacity with respect to the project examined

7. The customers capacity to fulfill his commitments examined8. Partner and subcontractors participation conditions defined

9. Protection of proprietary rights defined


47/54

To make sure that the following activities have beensatisfactorily carried out:

1. No unclarified issues remain in the contract

draft

2. All understandings reached subsequent to the

proposal are correctly documented

3. No new changes, additions, or omissions

have entered the contract draft


48/54

(1) Administrative or operativesoftware to be applied internally

(2) Software packages originally intendedto be sold to the public as off-the-shelf

packages

(3) Firmware to be embedded in the

companys products


49/54

Subject Disadvantages to the internal customer

(1) Inadequate definitionof project requirements

* Implementation deviates from the neededapplications

* Low satisfaction

(2) Poor estimate of therequired resources

* Unrealistic expectations about projectfeasibility

(3) Poor timetable * Missing scheduled dates for beginningthe distribution of new products

(4) Inadequate awarenessof development risks

* Customer unprepared for project risksand their consequences


50/54

Subject Disadvantages to the internal customer(1) Inadequate definitionof project requirements

* Higher change requirements* Wasted resources due to introducing

avoidable changes

(2) Poor estimate of therequired resources

* Substantial deviations from budget* Friction between units induced by

requirements for budget additions

(3) Poor timetable * Development activities are under timepressures and suffer from low quality

* Delays in freeing staff for their next

project(4) Inadequateawareness ofdevelopment risks

* Tardy initiation of efforts to overcomedifficulties


51/54

Development & Quality Plans

The DQP document will be used by thedevelopment team as the main referenceto build the software

It is developed based on the approvedproposal + schedules, resourceestimates, development risk evaluation

Should be available before kick off

Depending upon the size of the project;Simpler plans for smaller project


52/54

The DQP is meant for

Scheduling activities & estimating

resources and budget

Recruiting and allocating resources

Resolving risks Implementing SQA activities

Providing management with information

for project control


53/54

Elements of the Dev. Plan

Project products (doc, sw, training)

Project interfaces (with sw/hw) Project methodologies and dev tools

SW standard and procedures

Mapping of the dev process (inc estimate duration for each

phase, logical sequence of activities, resource implication) Project milestones

Project staff organization

Required dev facilities

Dev risks and risks management actions Control methods

Project cost estimates


54/54

Elements of the Q Plan

Quality goals (e.g. higher profit,

efficiency)

Review activities

(scope, types, schedule, procedures, PiC) SW tests

Acceptance test

Configuration management tools and

procedures

Documents

10.Process Product Quality Assurance