Upload
buinhan
View
216
Download
0
Embed Size (px)
Citation preview
Evolution of Software/System Quality
Second Asia-Pacific Conference on
QUALITY SOFTWARE
Dr. Raymond A. Paul
December 11, 2001NOV 2000
2
DefinitionDefinition
QUALITY is the characteristic that distinguishes the grade of excellence or superiority of a process, product, or service.
• In general usage, QUALITY means different things.Meaning of QUALITY varies considerably across specific disciplines and applications.
•What is Quality?
3
Historical Aspects Historical Aspects of QUALITY Evolutionof QUALITY Evolution
• Edward Deming– Postulated Statistical QUALITY Control Principles– Famous 14 Points of QUALITY Management– A Subset of these Principles successfully adapted by
Japanese Manufactures
• William Crosby– Emphasized Humanistic Behavioral Aspects of QUALITY
Improvement– Becoming More Important Now
4
JuranJuran
• QUALITY TrilogyA. QUALITY Planning
• Set of QUALITY Goals• Set Plans for Operations Based on these Goals
B. QUALITY Control• Responsible for Meeting QUALITY Goals• Prevent Adverse Changes• Set and Observe
� Performance Measures� Compare with Industry Standards� Benchmarking� Compare with Deming, CMM
C. QUALITY Improvement• Moving from Current Level to the Next Higher Level• Organize Teams, Train Operators to identify and Correct QUALITY Problems
TOTAL STATISTICAL QUALITY MANAGEMENT
5
Six Sigma InitiativesSix Sigma Initiatives
• GE and Motorola• Recognized Widely by Manufacturing Industry
– Adopted in Various Ways by Software Industry• Major Results
– Total QUALITY Management (TQM)– ISO Certification– Malcolm Baldridge Nation of QUALITY Award (Dept. of Commerce)
Six Sigma Means no more that 3.4 defects per million opportunities (six standard deviations away from the average of an assumed normal distribution)
• IBM (late Harlan Mills) Clean Room “follows” this Metric
6
Perspectives on QUALITY (Garvin 1984)Perspectives on QUALITY (Garvin 1984)
• Transcendented View– Something we recognize but cannot define
EXAMPLE: Beauty, Aristotle's Concept of Form
• User View– Fitness of Purpose
• Manufacturing (Design) View– Conformance to Specification
• Product View– Tied to inherent product functions and characteristics
• Value-Based View– Depends on the price the customer is willing to pay
VIEWS
•What is Quality?
7
QUALITY ModelsQUALITY Models
• McCall’s QUALITY Model (1977)– Shows relationships between external QUALITY Factors
such as correctness, reliability, usability, testability, etc. and Product QUALITY Criteria such as traceability, completeness, error tolerance, etc.
• PROBLEM: Difficult to measure these QUALITY Factors– No Standards, No Methods, No Tools
(A LOOSE INTERPRETATION)
•What is Quality?
8
Boehm's QUALITY Model (1986)Boehm's QUALITY Model (1986)
• Asserts that QUALITY software satisfies the needs of the users, designers, testers, and maintainers
• Relates software’s general utility to maintainability, reliability, testability, etc., to device independence, completeness, accessibility, etc.
• Very difficult to apply in practice, but easy to understand and learn
• Parts of it can be supported by tools; therefore, useful in a restricted sense
•What is Quality?
9
ISO 9126 QUALITY Model (1991)ISO 9126 QUALITY Model (1991)
• Consolidates many views of software QUALITY• ISO hierarchy is strict and non-overlapping unlike
McCall’s and Boehm's– EXAMPLE: In the model, maintainability depends ONLY on
analyzability, changeability, and stability. Whereas, in Boehm’s model, maintainability and understandability depend on structuredness, legibility, etc., in an overlapping way.
• Like Boehm’s model, it does not tell us how to measure characteristics such as security, accuracy, and interoperability.
•What is Quality?
10
Software QUALITY Characteristics DesiredSoftware QUALITY Characteristics Desired
• Functional Correctness� Functions and their specified properties
• Reliability� Attributes maintained by software under stated conditions
• Usability� Effort needed for users
• Efficiency� Software performance with a given set of resources
• Maintainability� Effort needed to make specified modifications
• Portability� Effort needed to transfer software from one environment to another
specified environment
(MOST MODELS INCLUDE THESE)
•What is Quality?
11
Comments on the Software Comments on the Software QUALITY ModelsQUALITY Models
• Some important characteristics such as evolvability, safety, and security are left out
• Dromey’s Model (1996) tries to address and serve some of these issues but not satisfactory
•What is Quality?
12
MATURITY Models (MM)MATURITY Models (MM)
• Specification Process Product
• Complete MM emphasizes– Process MATURITY
• What is MATURITY?– Methods of continuously improving a procedure, activity, or
process by planned measurements and feedbacks.– In essence, it emphasizes incremental but positive
improvement of an activity via repeated use and measurement.
(Design andImplementation)
• Measuring Quality
13
Analysis of Current MATURITY ModelsAnalysis of Current MATURITY Models
– One Dimensional– Does Not Include Items such as
• Maintenance• Application• Customer Satisfaction• Correctness with Respect to Specifications• Ease of Use, Ease of Change (maintenance)
– No Common Set of Assumptions– No Common Set of Measurements (no standardization)– No Financial Business Considerations– No Interoperability and Scalability Considerations
The emphasis in all these models INCREMENTAL!Incremental advances based on current experience.
NO LEAP FROGGING!!!
• Measuring Quality
14
Paul’s CallPaul’s Call
MATURITY means growing older and wiser
• Collecting and retrieving experiences
• Measure, observe, gather feedback from the product and process, update lessons learned from the product and process
• Maturity techniques continuously strive to improve processes, procedures, and activities by repeated use, planned measurements, and feedback
In essence, maturity emphasizes incremental but positive improvement of an activity via repeated use, measurement, analysis of results of the product, and feedback refinement
• Measuring Quality
15
QuestionsQuestions
What to measure? When to measure? Where to measure?How to measure? How to analyze and interpret results?
Remember the problems of measurement...– Assumptions– Assessment– Analysis
…have to be standardized
One Method GQM Paradigm ( )Goal, Question, Metric (BASILI, et al.)
Not Very Effective– Have to customize the product design goal from designers that
could be different from users
goal-primary-samegoal-primary-different
• Measuring Quality
16
Hierarchy of Maturity ModelsHierarchy of Maturity Models
• We Propose a Hierarchy of MATURITY Models for Improvement of IT Processes and Products
IT Maturity Model (ITMM)
Computing MM Communications MM
Hardware MM Software MM
• Measuring Quality
17
Assistive Technology to Assistive Technology to Improve QUALITY of Life Improve QUALITY of Life
• These technologies assist users with physical disabilities. Quality of Life Improvement
• 47 million Americans are in some form disabled• 75% of Americans with major disabilities are unemployed (President
Clinton) Major Disability – Vision Impairment “World is so Visual”
• Key: Augmenting Human Productivity• Designers do not have people with disabilities in mind when they
develop their products• Philosophy – Design products that work with more people• Methods of Design
– Universal Designs– Specialized Designs
• Measuring Quality
18
QUALITY ParametersQUALITY Parameters• Usage Based – Utilitarian
– Satisfaction of User’s Needs• Functionality
• Ease of Use and Handling
• Troubleshooting– Restart and Recovery– Maintenance and Analyzability
• Technology-Based Aspects– Design Complexity – Maintainability– Evolutionary Flexibility
• Reconfigurability• Agility• Growth Enhancement
• Measuring Quality
19
QUALITY Parameters QUALITY Parameters (cont.)(cont.)
• Economic Considerations– Cost to the Seller, and Price to the Buyer– Competitiveness with Similar Products– Ease of Manufacture, Reuse, and Maintenance
• Aesthetic Considerations– Form and Function– Appearance and Content– Appropriateness for the Purpose– Color, Packaging, Appearance– Attractiveness to Users– Attention-Grabbing Features– Entertaining, Interesting, Attention Keeping, e.g. TV/Websites
• Mass Appeal• Class/Culture Appeal• Individual Appeal
• Measuring Quality
20
QUALITY Parameters QUALITY Parameters (cont.)(cont.)
• Personality/Individualization– To support and help physical preferences and disabilities– EXAMPLES
• Right vs. Left Handedness• Poor Vision• Hearing Impairment• Web Browsing – Personalization
� My YAHOO� My EXCITE
• Personalization for:� News Junkies� Financial Analysts� Customized Information
• Measuring Quality
21
Another QUALITY Life Cycle ModelAnother QUALITY Life Cycle Model
Builtto Last
DemandGoodies
Make it Work
New Functions/Enhancements
Activities Processes Products Services Enterprises
Another QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life-Cycle ModelAnother QUALITY Life-Cycle Model
Life-Cycle Objectives of a DesignerWorks the First Time
WorksMany Times
Use BecomesHabit Forming
Establishes User’sDependency and
Brand Loyalty (SONY)
• Achieving Quality
22
Designing Methodologies for Embedding Designing Methodologies for Embedding QUALITY-Based QUALITY-Based (QUALITY SUPPORTING)(QUALITY SUPPORTING) Functions Functions
• Major Characteristics of These Designs– Reliability, Fault Tolerance via:
• Redundancy• Highly Reliable Proven Components/Parts, Interconnections,
Networks• Design Methods
– Flexibility - Reconfigurability– Dynamic Introduction of:
• Standby Optional Functions• Universal Designs
• Component-Based, Easy-Assembly Design:– Building Block Based – Lego like– Assembly – Disassembly Principle
• Achieving Quality
23
Designing Methodologies Designing Methodologies (cont.)(cont.)
• The Methods Presented Give Easy-Assembly (Disassembable) Subsystems– Good for Maintenance (Repair, Modification)– Subsystems and their integral components are loosely
coupled. This helps system understandability, analyzability, and reuse
– Helps to add on or enhance functions
• Complex Systems are Synthesized by:– Layered Architectures – each layer assembled from
components– Ring-Based Architectures– Good for Security
• Achieving Quality
24
Designing Methodologies Designing Methodologies (cont.)(cont.)
Components (Building Blocks)
Assemblers
Subsystems
System (made of loosely coupled subsystems)
To Improve Performance Characteristics (in real-time) the System can be Tightly Coupled
A more compact high performance design can begeneralized by coalescing the layers (rings), i.e., tightcoupling of some layers – SYSTEM B
For Better System Performance, Real-Time Applications use Tight-Coupling Principles (SYSTEM B). SYSTEM A is used as a Prototype.
In coalescing several small components into a bigger element, we reduce the inter-component overhead. More importantly, we eliminate or reduce the conflicts among assumptions made during the design of the components.
SYSTEM A
SYSTEM B
• Achieving Quality
25
Distributed System and Distributed System and Reconfiguration DesignReconfiguration Design
• QUALITY implied here includes:– Fault Tolerance– Control (Concurrency and Event Control)– Reconfigurability (Part of Control)– Heterogeneity
EXAMPLES:– CORBA Components and Distributed Control Technologies– Object-Oriented Design Components– NASA Space Shuttle Avionics
• Hardware and Software� Structured (Preferred) Reconfiguration Philosophy� Very Simple Software Reconfigurations
• Achieving Quality
26
Universal DesignsUniversal Designs• Primarily for Mass-Produced, Batch-Processed Hardware Systems• Holds True for Software Systems as well
PRINCIPLEThe System Includes Many Functions – Primary, Auxiliary, and Supporting Functions• User/Supplier triggers as activities those supporting and auxiliary functions needed for
that specific application or circumstance along with the primary functionsEXAMPLEPentium II MMX– Has 57 supporting instructions (functions) to improve multimedia
application processingUnderlying Philosophy
Design products that work for most applications (or people)
INSIGHT: Easier to mask out unwanted functions from a universal design (cheaper) than to build specialized functions from scratch (more expensive, time consuming, and labor intensive)
• Achieving Quality
27
QUALITY Manager ConceptQUALITY Manager Concept• QUALITY embraces many aspects, for example:
– Dependability, Fault Tolerance– Maintainability– Information Hiding
• QUALITY is a dynamic property at anytime. Different aspects of quality become more important depending on the circumstance and needs of the USER at that moment.
• QUALITY manager is a watch dog function that “detects and supplies” the quality aspect needed by the system at that moment.
• A method of implementing the quality manager function– Remember that computer power is like cash. It can be used to buy:
• Fault Tolerance• Multi-Precision Computation• Support or Take Over Failed Computing Functions
Use a high-performance computing function whose “excess” computing power can be used for quality manager functions that can invoke and support real-time quality needs.
• Achieving Quality
28
Projected Improvement in QUALITYProjected Improvement in QUALITY
• Service Industry Growing Very Rapidly (80% of Workers in US)• Service Industry’s Growth can be Traced to Increased Automation and Decreasing Labor-Intensive (manual) Functions• Increased Automation means better QUALITY of Products and Services
EFFORT
Time
Industry GrowthAUTOMATION
•Evolution of Quality to a User Perspective
29
QUALITY PyramidQUALITY PyramidIn
crea
sing
Com
plex
ity
ENTERPRISE
SERVICES
PRODUCTS
PROCESSESS
ACTIVITIES
Qual
ity In
put
•Evolution of Quality to a User Perspective
30
Customer PrioritiesCustomer Priorities
Works the FirstTime –
All the Time
Work Horse Make it Last
Legacy Dependency - Afraid to Change
A B C D EMake it Work
Legacy
Failure – No Option Part of Lifestyle
Reliability isTaken for Granted
Maintenanceand Cost are
No Issue
Afraid of Anything New –
Not Similar to the One Used
A B C D E
Usage History and QUALITY is AssessedCustomer is Reluctant to Change
Customer is Prepared to Change
Usage Life Cycle
DemandC D E
Make it Last Forever
User’sDependency
• Changes in Quality•Evolution of Quality to a User Perspective
31
Evolution of QUALITY Evolution of QUALITY Consciousness and MeasuresConsciousness and Measures
A
B
C
D
E
(HARDWARE, SOFTWARE, INFORMATION TECHNOLGY, AND SERVICE)
Productivity&
Performance
Reliability
SQC
BehavioralConsiderations
Total QualityManagementand Control
INDUSTRIAL REVOLUTION: Assembly Line and Mass Production, Productivity Consciousness - Time studies and ELI Whitney’s Standards and InterchangeableParts, Ford’s Assembly Line (Productivity Emphasis) Early Operations Research
Machines can be produced rapidly, and they are complex -need for improved reliability. EXAMPLE: Steam Engines, Automobiles, Airplanes
Statistical Quality Control- Edward Deming,Shewhert
Man is not a machine. Quality can be improved by focusing on human behavior and industrial psychology. Rise of industrial unions, human factors, and ergonomic emphasis improve quality and productivity by creating better working environments. ( Crosby)
TQM & C – J. Juran – Trilogy of Quality Management, Most Popular and Important
•Evolution of Quality to a User Perspective
32
Evolution of QUALITY Evolution of QUALITY Consciousness and Measures Consciousness and Measures (cont.)(cont.)
F
G
H
I
J
K
GE and Motorola - Moved from mostly mechanical parts to electronic parts and assemblies - Jack Welch of GE
Structured Programming, High-level languages, object-oriented design, and implementation languages
Structured Programming and Statistical Testing– Harlan Mills, Mike Fagan Inspection Procedures
Watts Humphrey, Bill Curtis
Initiative of DARPA for Communications Networks
SIX SIGMAQuality
Initiatives
SoftwareQuality
Initiatives and Methodologies
IBM Clean Room
Methods
CMM of SEI
QoS
IT MM To Be Developed
•Evolution of Quality to a User Perspective
33
QUALITY Concerns Change withQUALITY Concerns Change with
• User Needs• Technology Evolution• Product and Service Sophistication
EXAMPLE:
HARDWARE HARDWARE&
SOFTWARE
HARDWARE&
SOFTWARE& COMMUNICATION
EMBEDDED&
DISTRIBUTEDSYSTEMS
•Evolution of Quality to a User Perspective
34
QUALITY Concerns with ChangeQUALITY Concerns with Change
LOW
HIGH
HIGHComplexity of Change
Degradation of QUALITY
Due to Change
LOW
•Evolution of Quality to a User Perspective
35
Service TripletsService Triplets
PROCESS PRODUCT SERVICE
• Current Software Quality FADs Emphasize Process Aspects Only
• CMM Emphasizes Continuous Improvement in Software Development Processes (Well-Defined Activities)
• Specification Process Software Product
• CMM Emphasizes Process Maturity
•Evolution of Quality to a User Perspective
36
Change of Customer QUALITY Priorities with Change of Customer QUALITY Priorities with Product (Software) Life CycleProduct (Software) Life Cycle
Customer Expectation Increasing Dependency on the Product
The Ultimate Milestone of QUALITY
Make it Work(Works the First
Time and Allthe Time)
PRODUCT USAGE LIFE CYCLE
User’s Dependencyon Product
User’s Expectation
(Product WorkHorse Ethic)
Builtto Last
User Afraidto Change
LegacyMentality
•Evolution of Quality to a User Perspective
37
QUALITY – Customer PrioritiesQUALITY – Customer Priorities
Usage Life Cycle – How QUALITY is assessed by the customer
(EVOLUTION OF QUALITY CONSCIOUSNESS)
= Reliability
= Reliability and Maintainability
= Failure No Option
= Failure No Option
= Refining, Enhancing User Preferred Features
Make it Work-Works the First
Time –EVERYTIME
User’s Dependency
Work HorseEthic
Builtto Last
User Afraidto Change,
LegacyMentality
A B C D E
ABCDE
•Evolution of Quality to a User Perspective
38
Quality of DatabasesQuality of Databases
• Databases are used widely and extensively in all kinds of enterprises and their use will only increase.
• But what is the view of database quality?• From a database manager point of view, database quality is terms of
database size, transaction speed, data reliability, scalability and data integrity.
• However, from a software designer point of view, quality may mean the maintainability, code reliability, and software architecture attributes.
• However, from the enterprise point of view, quality may mean the contribution of a database to the enterprise mission.
•Evolution of Quality to a User Perspective
39
Quality from a Higher Level PerspectiveQuality from a Higher Level Perspective
• We need to think in terms of a high level perspective from the user point of view rather than a design or process points of view– Design view such as redundancy for fault tolerance, buffers
for scalability.– Process views such as following certain techniques, work
flow, or using specific kinds of tools such as configuration control.
• Furthermore, we should not think in terms of system requirements only, we need to think about the overall mission of the enterprise, operation, enterprise capabilities, and enterprise evolution point of view.
•Evolution of Quality to a User Perspective
40
Enterprise EvolutionEnterprise Evolution
• Technology is changing rapidly at the speed of the Internet.• Enterprise mission also changes, but at a much slower speed.• Enterprise capability also needs to change with knowledge and
technology changes .– This is the trade-off between budget growths at X% per year and
Moore’s law [IT technical capability grows at 67% per year].• An orderly transition of enterprise capabilities, while taking
advantage of technology change, is badly needed.• This also assumes that an enterprise has a sound strategy to
know what is needed and how to incorporate new technologies and new knowledge growth plans into the existing infrastructure and superstucture.
•Evolution of Quality to a User Perspective
41
SummarySummary
• QUALITY - particularly software quality - has to be more adaptive
• Products and processes grow with their application and human environments. Quality criteria have to change with the situation, circumstance, and the environment. These are dynamic characteristics. Therefore, quality has to be dynamic and adaptive.
• Quality aspects change with application and product life cycles. Software is the right place to put quality adoption and evolution.
• The new IT environment will be multidisciplinary, fast changing with rapid diversification and globalization.
• THE QUALITY JOURNEY CONTINUES, BE PREPARED
•Summary
LEADERSHIPLEADERSHIP
•The position or office of a leader•Capacity or ability to lead•A continuum of good decisions•Provide direction•To influence
43
HOPE HOPE • Michelangelo lived a few days shy of 89 years, still sculpting,
painting, writing, and designing in an age when 90 was about 60 years beyond normal life expectancy was speaking to this idea of having high hopes and aims in this quotation (next slide) of his.
• I recall being in front of the statue of David in Florence and being transfixed. The size, the majesty, the spirit that seemed to jump right out of the marble was Michelangelo saying to all of us, “Aim high.”
• High aim indeed. Look at the at the Sistine Chapel where Michelangelo painted the ceiling by lying on his back working every day for 4 years. Lesser artists considered the task impossible, yet Michelangelo took it on and many more and aimed high
44
HOPEHOPE
The greater danger for most of us is not that our aim is too highand we miss it,
but that it is too lowand we reach it.
Michelangelo(1475-1564)
45
HOPE cont.HOPE cont.
• No one knows enough to be a pessimist!!!• Michelangelo’s advice is just as applicable today as it
was almost 500 years ago. Never listen to those who try to influence you with their pessimism. Follow these guidelines:– Refuse to listen to or internalize the proclamations of those
who point to your limitations. You must always remember:• when you argue for your limitations, the only thing you get are
limitations• imagine Michelangelo telling you that you can create the quality
you desire, and the great danger is not in having too much hope, but in reaching what you have perceived as hopeless