8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 1/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Introduction to software quality evaluation
Master on Free Software
Felipe Ortega, Daniel Izquierdo, Pedro Coca
[email protected]/Libresoft
19 de noviembre de 2010
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 2/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
(cc) 2010 Felipe Ortega, Daniel Izquierdo, Pedro Coca.Some rights reserved. This document is distributed under the Creative
Commons Attribution-ShareAlike 3.0 licence, available inhttp://creativecommons.org/licenses/by-sa/3.0/
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 3/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Index
1 Introduction
2 Quality Assurance (QA): standards
3 Quality Planning
4 Quality Control
5 Product quality
6 Process quality
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 4/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
What is software quality?
Complex concept, different from quality in manufacturing.
Product developed to meet its specification.But it is difficult to write good software specification.
Maintainability, reusability, modularity, security, etc. areusually not included (but required).
Quality is linked to meeting end-user expectations(subjective).
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
I d i
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 5/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Factors that may improve quality
OOP, CASE support.
Standards (development and organizational). But bureaucracyis not enough.
Quality culture: experience, best practices, management.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
I t d ti
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 6/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Main areas of software quality in large projects
Quality assurance: Defining framework for organizationalquality.
Quality planning: Selecting elements from the framework,adapted to a particular project.
Quality control: Checking that development teams adhere tothe plan.
For many community FLOSS projects, none of the above aredefined.
Problems for companies with this methods in place tointegrate community FLOSS.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 7/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Process and product quality
In traditional manufacturing, process quality is tightly linkedto final product quality.
But software involves design, creativity. Subject to pastexperience and team skills.
Difficult to measure some aspects: maintainability, reusability,etc.
Obscure links between changes in process and softwareproduct quality.
Blind application of process quaility is not good for allsituations.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 8/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Index
1 Introduction
2 Quality Assurance (QA): standards
3 Quality Planning
4 Quality Control
5 Product quality
6 Process quality
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 9/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Quality assurance overview
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 10/36
IntroductionQuality Assurance (QA): standards
Quality PlanningQuality ControlProduct qualityProcess quality
Role of software development standards
Try to convey best practices in software development.Offer required framework for QA.
Many examples: US DoD, ANSI, IEEE, BSI, NATO, etc.
Engineers usually reluctant to adopt standards (bureaucracy).
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 11/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
ISO 9000
International body of standards for quality in any industry.
ISO 9001 is the most extended version.
Quality in process, development and maintenance of anyproduct.Mainly concerned about high level procedures.Forces definition of specific metrics to measure quality.Involves periodic reviews to maintain ISO 9001 label award.
Involve definition of documentation templates, anddocumenting all processes.
Does not ensure that standarized process reflects bestpractices in the field.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 12/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Documentation standards
3 different categories:
Documentation process standards: Establish good processfor document production.
Documentation standards: Define structure andpresentation of documents.
Document interoperability standards: Compatibilitybetween document formats.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
Introduction
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 13/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Documentation standards
Some examples
Document identification: Unique names, easy tracking andidentification.
Document structure: Homogeneous structure for docs of similar character.
Document presentation: Customized layout and “look and
feel” (design and presentation).
Document update: Tracking changes/updates/reviews.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQ (Q )
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 14/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Index
1 Introduction
2 Quality Assurance (QA): standards
3 Quality Planning
4 Quality Control
5 Product quality
6 Process quality
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQ li A (QA) d d
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 15/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Quality planning overview
5 different aspects:
Product overview: Description, target market, user
expectations.Product plan: Releases, updates, support, distribution, etc.
Process description: Product development and management.
Quality goals: Including identification of quality attributes to
be tracked.Risks and risk management: Risks affecting quality andcorrective actions.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQ lit Ass (QA) st d ds
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 16/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Software quality attributes
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 17/36
IntroductionQuality Assurance (QA): standards
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 18/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Documentation standards
3 components:
Quality reviews: Periodic checks of conformity with qualitystandards define in the project.
Automated assessment: Collecting quality informationautomatically (product and process).
Quality metrics: Measure different quality aspects as aninput for assessment.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 19/36
IntroductionQuality Assurance (QA): standards
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 20/36
Quality Assurance (QA): standardsQuality Planning
Quality ControlProduct qualityProcess quality
Software measurement and metrics
Sofware metric : Measurement linked to a software system,process or related documentation.
Examples: SLOC, complexity, coupling, connascence, etc.
Usually, the main problem is how to define meaningfulmetrics.
We can measure many different aspects and elements.But do they describe the quality aspect we want to track?
2 types: Control metrics (process) and predictor metrics(product).
Key application: predicting possible problems for proactivemanagement and control.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 21/36
Q y (Q )Quality Planning
Quality ControlProduct qualityProcess quality
Measurement procedure
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 22/36
( )Quality Planning
Quality ControlProduct qualityProcess quality
Index
1 Introduction
2 Quality Assurance (QA): standards
3 Quality Planning
4 Quality Control
5 Product quality
6 Process quality
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 23/36
IntroductionQuality Assurance (QA): standards
Q li Pl i
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 24/36
Quality PlanningQuality ControlProduct qualityProcess quality
Product metrics
Static metrics:
Fan-in/Fan-out : Number of functions or methods that callsome other function or method (complexity, connascence).
Length of code : Size of program. SLOC (LOC or LLOC).Cyclomatic complexity : Metric for control complexity of software.Length of identifiers : Average length of identifiers used in aprogram. Supposedly, the longer the ID, the better forreadability and maintenance of code.Depth of conditional nesting : Deeply nested statements areharder to be grasped.Fog index : Average length of words and sentences indocuments.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
Q lit Pl i
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 25/36
Quality PlanningQuality ControlProduct qualityProcess quality
Product metrics
Dynamic metrics:
Depth of inheritance tree : Number of discrete levels in the treeof classes (OOP). The deeper the tree, the more complex the
desgin.Method fan-in/fan-out : Distinction between origin of calls toother methods (from object or from external methods).Weighted methods per class : Number of methods included in aclass, weighted complexity of every method. Too complex
classes are difficult as for understanding and maintenance.Number of overriding operations : Operations overridden insub-classes. The higher this number, the less appropriate maybe the super-class.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
Quality Planning
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 26/36
Quality PlanningQuality ControlProduct qualityProcess quality
Metrics assessment
It is not enough to collect different metrics. We need to inferassessments and conclusions from these results.
Most of times, human interpretation of results is needed.
Value ranges drawn from multiple case examples may serve asguidelines.
Usually, supported through integrated assessment frameworks(Business Intelligence).
Organization of measurement areas and lists of metrics.
Automated production of reports summarizing results.Support for elaboration of assessment reports and executivesummaries.Other buit-in functionalities: data mining, data warehouse,statistical tools, etc.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 27/36
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 28/36
IntroductionQuality Assurance (QA): standards
Quality Planning
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 29/36
Quality PlanningQuality ControlProduct qualityProcess quality
Process improvement
Quality frameworks focused on improving the process, ratherthan just describing it.
3 stages:1 Process measurement : Compilation of metrics and indicators
related to software process (development and management).2 Process analysis : Producing meaningful assessment from
results, identifying or anticipating problems that must be
corrected.3 Process change : Alter problematic elements or practices to
align the process to the quality goals of the project.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
Quality Planning
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 30/36
Q y gQuality ControlProduct qualityProcess quality
Link between process and product
Humphrey, in his book about process management (1988),states that the same techniques applied to any kind of industry are valid for sofware processes.
However, a number of caveats are present in sofware projects.
The design phase plays a key role in the whole process andresults.Skills and experience of team leaders and experts affect qualityof results.Different projects may need different methodologies andapproaches.Great influence of 3 variables: cost, time frame and schedule.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
Quality Planning
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 31/36
y gQuality ControlProduct qualityProcess quality
Process metrics
Process metrics should be aimed to facilitate process
improvement.3 types:
Time to complete the process.Resources required to carry on the process.Number of occurrences of a particular event to be monitored.
The key question is to identify what we need to measure.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standards
Quality Planning
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 32/36
Quality ControlProduct qualityProcess quality
The GQM methodology
Frequently applied to develop customized process qualityframeworks.
Goal-oriented, top-down methodology to measure quality of software processes and products.
First developed at University of Maryland (1984) incollaboration with NASA.
Hierarchical structure composed by 3 kinds of elements: goals,questions and metrics.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
IntroductionQuality Assurance (QA): standardsQuality Planning
Q li C l
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 33/36
Quality ControlProduct qualityProcess quality
GQM Hierarchy
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 34/36
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 35/36
IntroductionQuality Assurance (QA): standardsQuality Planning
Quality Control
8/3/2019 General introduction to quality and evaluation of projects
http://slidepdf.com/reader/full/general-introduction-to-quality-and-evaluation-of-projects 36/36
Quality ControlProduct qualityProcess quality
References
Humphrey, W. S. Characterizing the software process . IEEESoftware, 5(2), 73-79. 1988.
Ian Sommerville, Software Engineering , 8th Edition. AddisonWesley, 2006.
Hans van Vliet, Software Engineering: Principles and Practice ,3rd Edition. Wiley, 2008.
V. R. Basili, Quantitative Evaluation of Software Engineering Methodology , Proceedings of the First Pan Pacific ComputerConference, Melbourne, Australia, Vol. 1 pp. 379-398, 1985.
Felipe Ortega, Daniel Izquierdo, Pedro Coca Introduction to software quality evaluation