IEEE Standard for Organizational

IEEE Std 1624™-2008

IEEE Standard for OrganizationalReliability Capability

IEEE3 Park Avenue New York, NY 10016-5997, USA

5 February 2009

IEEE Reliability Standards Committee Sponsored by theIEEE Reliability Society

1624

TM

Authorized licensed use limited to: IHS Stephanie Dejesus. Downloaded on February 16, 2009 at 05:38 from IEEE Xplore. Restrictions apply.

Provided by IHS Licensee=PDVSA - Maracaibo site 3/9986712008, User=Quintero, JoseNot for Resale, 06/25/2014 11:50:52 MDTNo reproduction or networking permitted without license from IHS

--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

IEEE Std 1624TM-2008

IEEE Standard for Organizational Reliability Capability

Sponsor

IEEE Reliability Standards Committee of the IEEE Reliability Society

Approved 26 September 2008

IEEE-SA Standards Board



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

Abstract: Organizational reliability capability and the identification of the criteria for assessing the reliability capability of an organization are defined. Keywords: assessment, organizational reliability capability, reliability capability, reliability management, reliability program, supply chain

•

The Institute of Electrical and Electronics Engineers, Inc. 3 Park Avenue, New York, NY 10016-5997, USA Copyright © 2009 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Published 5 February 2009. Printed in the United States of America. IEEE is a registered trademark in the U.S. Patent & Trademark Office, owned by the Institute of Electrical and Electronics Engineers, Incorporated. PDF: ISBN 978-0-7381-5814-3 STD95828 Print: ISBN 978-0-7381-5815-0 STDPD95828 No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission of the publisher.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its standards through a consensus development process, approved by the American National Standards Institute, which brings together volunteers representing varied viewpoints and interests to achieve the final product. Volunteers are not necessarily members of the Institute and serve without compensation. While the IEEE administers the process and establishes rules to promote fairness in the consensus development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of the information or the soundness of any judgments contained in its standards.

Use of an IEEE Standard is wholly voluntary. The IEEE disclaims liability for any personal injury, property or other damage, of any nature whatsoever, whether special, indirect, consequential, or compensatory, directly or indirectly resulting from the publication, use of, or reliance upon this, or any other IEEE Standard document.

The IEEE does not warrant or represent the accuracy or content of the material contained herein, and expressly disclaims any express or implied warranty, including any implied warranty of merchantability or fitness for a specific purpose, or that the use of the material contained herein is free from patent infringement. IEEE Standards documents are supplied “AS IS.”

The existence of an IEEE Standard does not imply that there are no other ways to produce, test, measure, purchase, market, or provide other goods and services related to the scope of the IEEE Standard. Furthermore, the viewpoint expressed at the time a standard is approved and issued is subject to change brought about through developments in the state of the art and comments received from users of the standard. Every IEEE Standard is subjected to review at least every five years for revision or reaffirmation. When a document is more than five years old and has not been reaffirmed, it is reasonable to conclude that its contents, although still of some value, do not wholly reflect the present state of the art. Users are cautioned to check to determine that they have the latest edition of any IEEE Standard.

In publishing and making this document available, the IEEE is not suggesting or rendering professional or other services for, or on behalf of, any person or entity. Nor is the IEEE undertaking to perform any duty owed by any other person or entity to another. Any person utilizing this, and any other IEEE Standards document, should rely upon his or her independent judgment in the exercise of reasonable care in any given circumstances or, as appropriate, seek the advice of a competent professional in determining the appropriateness of a given IEEE standard. Interpretations: Occasionally questions may arise regarding the meaning of portions of standards as they relate to specific applications. When the need for interpretations is brought to the attention of IEEE, the Institute will initiate action to prepare appropriate responses. Since IEEE Standards represent a consensus of concerned interests, it is important to ensure that any interpretation has also received the concurrence of a balance of interests. For this reason, IEEE and the members of its societies and Standards Coordinating Committees are not able to provide an instant response to interpretation requests except in those cases where the matter has previously received formal consideration. A statement, written or oral, that is not processed in accordance with the IEEE-SA Standards Board Operations Manual shall not be considered the official position of IEEE or any of its committees and shall not be considered to be, nor be relied upon as, a formal interpretation of the IEEE. At lectures, symposia, seminars, or educational courses, an individual presenting information on IEEE standards shall make it clear that his or her views should be considered the personal views of that individual rather than the formal position, explanation, or interpretation of the IEEE. Comments for revision of IEEE Standards are welcome from any interested party, regardless of membership affiliation with IEEE. Suggestions for changes in documents should be in the form of a proposed change of text, together with appropriate supporting comments. Comments on standards and requests for interpretations should be submitted to the following address:

Secretary, IEEE-SA Standards Board 445 Hoes Lane Piscataway, NJ 08854 USA

Authorization to photocopy portions of any individual standard for internal or personal use is granted by The Institute of Electrical and Electronics Engineers, Inc., provided that the appropriate fee is paid to Copyright Clearance Center. To arrange for payment of licensing fee, please contact Copyright Clearance Center, Customer Service, 222 Rosewood Drive, Danvers, MA 01923 USA; +1 978 750 8400. Permission to photocopy portions of any individual standard for educational classroom use can also be obtained through the Copyright Clearance Center.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

iv Copyright © 2009 IEEE. All rights reserved.

Introduction

This introduction is not part of IEEE Std 1624-2008, IEEE Standard for Organizational Reliability Capability.

This standard establishes the criteria for assessing the reliability capability of an organization. Organizational reliability capability is characterized by key processes and associated metrics. This standard can be used for self-assessment by organizations or for supplier/customer relationship development between members of a supply chain. The standard could also be used to assure conformance to existing supplier/customer reliability requirements. This standard does not seek to create or propose creation of certifying bodies to assess reliability capability of a company.

Notice to users

Laws and regulations

Users of these documents should consult all applicable laws and regulations. Compliance with the provisions of this standard does not imply compliance to any applicable regulatory requirements. Implementers of the standard are responsible for observing or referring to the applicable regulatory requirements. IEEE does not, by the publication of its standards, intend to urge action that is not in compliance with applicable laws, and these documents may not be construed as doing so.

Copyrights

This document is copyrighted by the IEEE. It is made available for a wide variety of both public and private uses. These include both use, by reference, in laws and regulations, and use in private self-regulation, standardization, and the promotion of engineering practices and methods. By making this document available for use and adoption by public authorities and private users, the IEEE does not waive any rights in copyright to this document.

Updating of IEEE documents

Users of IEEE standards should be aware that these documents may be superseded at any time by the issuance of new editions or may be amended from time to time through the issuance of amendments, corrigenda, or errata. An official IEEE document at any point in time consists of the current edition of the document together with any amendments, corrigenda, or errata then in effect. In order to determine whether a given document is the current edition and whether it has been amended through the issuance of amendments, corrigenda, or errata, visit the IEEE Standards Association Web site at http://ieeexplore.ieee.org/xpl/standards.jsp, or contact the IEEE at the address listed previously.

For more information about the IEEE Standards Association or the IEEE standards development process, visit the IEEE-SA Web site at http://standards.ieee.org.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

v Copyright © 2009 IEEE. All rights reserved.

Errata

Errata, if any, for this and all other standards can be accessed at the following URL: http://standards.ieee.org/reading/ieee/updates/errata/index.html. Users are encouraged to check this URL for errata periodically.

Interpretations

Current interpretations can be accessed at the following URL: http://standards.ieee.org/reading/ieee/interp/ index.html.

Patents

Attention is called to the possibility that implementation of this standard may require use of subject matter covered by patent rights. By publication of this standard, no position is taken with respect to the existence or validity of any patent rights in connection therewith. The IEEE is not responsible for identifying Essential Patent Claims for which a license may be required, for conducting inquiries into the legal validity or scope of Patents Claims or determining whether any licensing terms or conditions provided in connection with submission of a Letter of Assurance, if any, or in any licensing agreements are reasonable or non-discriminatory. Users of this standard are expressly advised that determination of the validity of any patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Further information may be obtained from the IEEE Standards Association.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

vi Copyright © 2009 IEEE. All rights reserved.

Participants

At the time this standard was submitted to the IEEE-SA Standards Board for approval, the Reliability Standards Working Group had the following membership:

Michael Pecht, Chair

Diganta Das, Vice Chair

Jerry Cartwright, Secretary

Michael H. Azarian, Technical Editor

Alan Wood, Associate Technical Editor

Jon Elerath Louis J. Gullo Harold Hart

Tyrone Jackson Ari Jain

Fred Schenkelberg Jack Sherman Sanjay Tiku

The following members of the individual balloting committee voted on this standard. Balloters may have voted for approval, disapproval, or abstention.

Gary Arntson Ali Al Awazi Michael H. Azarian Bakul Banerjee Hugh Barrass Thomas Basso Juris Borzovs Pieter Botman J. Bruder Jeffrey Burnworth Massimo Cardaci Juan Carreon Norbert Carte Lawrence Catchpole Keith Chow Raul Colcher John Cole Geoffrey Darnton Diganta Das Thomas Dineen Carlo Donati Ernest Duckworth Sourav Dutta Jon Elerath Harriet Feldman David Friscia Gregg Giesler Lewis Gray

Randall Groves Louis J. Gullo John Harauz Werner Hoelzl Gary Hoffman Robert Holibaugh Atsushi Ito Mark Jaeger Piotr Karocki Jim Kulchisky Saumen Kundu David J. Leciston Albert Livshitz Edward Mccall Gary Michel William Milam James Moore Jerry Murphy Rajesh Murthy Prajit Nair Michael S. Newman Satoshi Oyama Donald Parker Kevin Patton Mark Paulk William Petit Ulrich Pohl Edward Rafter Annette Reilly

Michael Roberts Robert Robinson Charles Rogers Terence Rout Michael Rush Robert Schaaf Fred Schenkelberg Richard Schrenker Stephen Schwarm Jack Sherman Hyeong Sim Carl Singer James E. Smith Luca Spotorno Thomas Starai Raymond Strittmatter Walter Struppler Marcy Stutzman Alourdes Sully Thomas Tullia John Vergis Charlene Walrad Mark Walton John Walz Alan Wood Oren Yuen Janusz Zalewski Alexandru Zamfirescu



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

vii Copyright © 2009 IEEE. All rights reserved.

When the IEEE-SA Standards Board approved this standard on 26 September 2008, it had the following membership:

Robert M. Grow, Chair Thomas Prevost, Vice Chair Steve M. Mills, Past Chair Judith Gorman, Secretary

Victor Berman Richard DeBlasio Andy Drozd Mark Epstein Alexander Gelman William Goldbach Arnie Greenspan Ken Hanus

Jim Hughes Richard Hulett Young Kyun Kim Joseph L. Koepfinger* John Kulick David J. Law Glenn Parsons Ron Petersen

Chuck Powers Thomas Prevost Narayanan Ramachandran Jon Walter Rosdahl Anne-Marie Sahazizian Malcolm Thaden Howard Wolfman Don Wright

*Member Emeritus

Also included are the following nonvoting IEEE-SA Standards Board liaisons:

Satish K. Aggarwal, NRC Representative Michael Janezic, NIST Representative

Lorraine Patsco IEEE Standards Program Manager, Document Development

Matthew Ceglia

IEEE Standards Program Manager, Technical Program Development



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

viii Copyright © 2009 IEEE. All rights reserved.

CONTENTS

1. Overview .................................................................................................................................................... 1 1.1 Scope ................................................................................................................................................... 2 1.2 Purpose ................................................................................................................................................ 2 1.3 Glossary............................................................................................................................................... 2 1.4 Contents description ............................................................................................................................ 2

2. Normative references.................................................................................................................................. 3

3. Definitions .................................................................................................................................................. 3

4. Organizational reliability capability ........................................................................................................... 4 4.1 Basic concepts ..................................................................................................................................... 5 4.2 Organizational reliability capability assessment usage........................................................................ 5

5. Key reliability practices.............................................................................................................................. 7 5.1 Reliability requirements and planning................................................................................................. 8 5.2 Training and development ................................................................................................................... 9 5.3 Reliability analysis ............................................................................................................................ 10 5.4 Reliability testing............................................................................................................................... 11 5.5 Supply chain management ................................................................................................................. 12 5.6 Failure data tracking and analysis...................................................................................................... 13 5.7 Verification and validation ................................................................................................................ 14 5.8 Reliability improvements................................................................................................................... 15

6. Assessment criteria................................................................................................................................... 16 6.1 Reliability capability levels ............................................................................................................... 16 6.2 Reliability capability level definitions specific to each key reliability practice................................. 18

Annex A Informative Bibliography.............................................................................................................. 23



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

1 Copyright © 2009 IEEE. All rights reserved.

IEEE Standard for Organizational Reliability Capability

IMPORTANT NOTICE: This standard is not intended to assure safety, security, health, or environmental protection in all circumstances. Implementers of the standard are responsible for determining appropriate safety, security, environmental, and health practices or regulatory requirements.

This IEEE document is made available for use subject to important notices and legal disclaimers. These notices and disclaimers appear in all publications containing this document and may be found under the heading “Important Notice” or “Important Notices and Disclaimers Concerning IEEE Documents.” They can also be obtained on request from IEEE or viewed at http://standards.ieee.org/IPR/ disclaimers.html.

1. Overview

Traditionally, supplier selection is based on cost, logistics, technical capabilities, production volume, quality, support locations, and other contractual factors. One of the reasons reliability does not typically enter into the supplier selection process is the lack of an industry-accepted method to measure the capability of an organization to develop and build reliable products. This standard provides standardized and objective measurement criteria for assessing and quantifying the reliability capability of an organization.

Organizational reliability capability is the measure of the effectiveness of an organization’s reliability program, practices, or activities in meeting the customer’s requirements for product reliability.

Organizational reliability capability reflects the ability of an organization to produce products that meet the reliability needs of its customers. Self-assessment of reliability capability facilitates improvement of an organization’s reliability practices. Assessment of external organizations provides a means to improve the reliability of products obtained from external suppliers. A supplier selection process that takes into account the ability of the supplier’s product to meet reliability requirements can lead to improvements in product reliability.

Conformance with the provisions of this standard is achieved by demonstrating that the criteria in Table 1 have been used in an assessment of organizational reliability capability, and organizational reliability capability levels have been assigned according to Clause 6.1. The word “shall” is used in this standard to denote a requirement that must be fulfilled to conform to this standard.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---

IEEE Std 1624-2008 IEEE Standard for Organizational Reliability Capability


1.1 Scope

This document presents a standard that defines the reliability capability of organizations and identifies the criteria for assessing the reliability capability of an organization. This standard is intended to be usable by all organizations that design, manufacture or procure electrical/electronics components or products. Although the concepts described in this standard could be applied to both hardware and software products, the focus of the standard is on hardware products.

1.2 Purpose

The purpose for assessing the reliability capability of an organization is to facilitate the improvement of product reliability. This document does not define an audit process, but rather an assessment process that is suitable for providing data and results as input into an audit process. Reliability capability is defined by key practices and associated metrics. This standard does not seek to create or propose creation of certifying bodies that assess whether an organization meets the definitions of reliability capability. This standard could be used for self-assessment by organizations or for supplier/customer relationship development between members of supply chain.

1.3 Glossary

Many of the words used to describe organizational reliability capability have multiple meanings. For example, the word reliability has a specific mathematical meaning, but the word is also used to mean an entire field of engineering study. Clause 3 contains definitions of specialized terms that are used in this document. This includes those whose meanings are not provided in IEEE 100™, The Authoritative Dictionary of IEEE Standards Terms [B10], as well as those which have been defined in IEEE 100 but may have more than one definition in that document, or whose usage in this document is based upon a different definition.

1.4 Contents description

Clause 4 provides background information on organizational reliability capability assessment. This background information includes basic concepts, definitions, and appropriate uses for reliability capability assessment. Clause 5 defines the eight key reliability practices and their inputs, activities, and outputs. These key reliability practices are the following:

⎯ reliability requirements and planning

⎯ training and development

⎯ reliability analysis

⎯ reliability testing

⎯ supply chain management

⎯ failure data tracking and analysis

⎯ verification and validation

⎯ reliability improvements

Clause 6 defines the five reliability capability levels and identifies the set of activities that are necessary to achieve the desired level for each key practice.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



2. Normative references

The following referenced document is indispensable for the application of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments and corrigenda) applies.

None. (See Annex A for an informative bibliography.)

3. Definitions

This clause provides definitions of specialized terms which are used in this document that either have not been previously defined in IEEE 100 [B10], or can be found in IEEE 100 but either have multiple definitions in that document or are used differently in this standard. In all cases where a single applicable definition existed within IEEE 100, that definition is used for the purposes of this document.

3.1 activity: A set of tasks that relate to the performance of a specific function.

3.2 assessment: A planned and documented activity performed to determine whether various elements within an organization’s reliability program are effective in achieving stated objectives.

3.3 audit: An independent examination of accounts and records to assess or verify compliance with specifications, standards, contractual agreements, or other criteria.

3.4 capability: One or more processes or activities that describe how reliability is used, treated, or developed within an organization.

3.5 component: A piece of electrical or mechanical equipment, a line or circuit, a section of a line or circuit, or a group of items that is viewed as an entity for the purposes of reliability evaluation. (IEEE 100 [B10])

3.6 failure modes and effects analysis (FMEA): A systematic procedure by which each potential failure mode in a system is identified and analyzed to determine its effects on the system, leading to a classification of potential failure modes according to their severity, occurrence, and ease of detection.

3.7 organizational reliability capability: The ability of an organization’s reliability practices to ensure that product reliability meets or exceeds its customers’ requirements.

3.8 practice: One or more activities that use specified inputs to develop specified work products for achieving specified objectives.

3.9 process: A sequence of tasks, actions, or activities, including the transition criteria for progressing from one to the next, that bring about a result.

3.10 product qualification: A set of procedures and tests performed to demonstrate the ability of a product to operate without failure under anticipated life cycle conditions in order to transfer the product from the engineering into the manufacturing stage.

3.11 program: The managed collection of an organization’s practices that is structured to ensure that the customers’ requirements and product needs are satisfied.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



3.12 reliability: The ability of a system or component to perform its intended functions without failure, within design parameters, under specific operating conditions, and for a specific period of time. (IEEE 100 [B10])

3.13 reliability capability level: A measure of the effectiveness of organizational reliability capability.

3.14 reliability goal: An objective that is desirable to meet but not mandatory to meet. All activities defined for reliability requirements can also be applied to reliability goals.

3.15 requirement: A condition or capability that shall be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document(s). (IEEE 100 [B10])

3.16 supplier qualification: A verification whether the subject vendor has a demonstrated capability to meet the needs of a certain prospective customer for a certain class of product or service

3.17 task: The smallest unit of work subject to management accountability. A task is a well-defined work assignment for one or more project members. Related tasks are usually grouped to form activities.

3.18 validation: The process of determining the degree to which a model or simulation is an accurate representation of the real world from the perspective of the intended uses. (IEEE 100 [B10]) Validation confirms that the product, as provided, will fulfill its intended use. In other words, validation ensures that “you built the right thing.”

3.19 verification: The process of determining that a model or simulation implementation accurately represents the conceptual description and specifications. (IEEE 100 [B10]) Verification confirms that work products properly reflect the requirements specified for them. In other words, verification ensures that “you built it right.”

4. Organizational reliability capability

This standard specifies the necessary elements of an organizational reliability capability assessment, consisting of eight key reliability practices with five levels of reliability capability. The key practices are the foundation for criteria that shall be used by electronics manufacturers to assess their potential suppliers or for suppliers to assess themselves. The five levels represent stages in the evolutionary transition for an organization. To assign a reliability capability level to a key practice, a set of reliability activities have been enumerated. An assessment based on these reliability activities can place organizations at one of the five levels for each key practice.

This standard can be used by designers, suppliers, customers, and independent authorities to help establish agreement on appropriate reliability practices. The standard can also be used to identify shortcomings in the reliability program of an organization, that can be overcome by subsequent improvement actions. These kinds of agreements and improvement actions are expected to produce increased customer satisfaction, provide competitive advantage, and shorten the product development cycle. For the purposes of this standard, an organization may consist of an entire corporation or agency; a specific division, business group, or facility (site); or a team associated with a specific product or group of products.

This clause provides the necessary background for the reliability capability activities and assessment. Clause 4.1 explains basic concepts such as reliability capability, program, process, practice, and activity. Clause 4.2 describes how to use reliability capability assessments, and why different types of assessments are appropriate in different situations.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



4.1 Basic concepts

There are many different terms that are commonly used, often interchangeably, to categorize the hierarchy of reliability effort expended during the product life cycle. In this standard, the terminology described in this clause has been adopted to allow the standard to be internally consistent.

Figure 1 illustrates the hierarchical relationship among the terms activity, reliability practice, and reliability program. An activity is a set of tasks, usually a single specific step in a reliability practice. A reliability practice is one or more activities that use specified inputs to develop specified work products for achieving specified reliability objectives. It is generally the lowest level at which an organization can institutionalize reliability program improvements. A reliability program is the managed collection of an organization’s reliability practices for a specific product(s), structured to assure that the customer’s requirements with regards to product reliability are satisfied.

Figure 1 —Reliability practice hierarchy

Organizational reliability capability is the measure of the effectiveness of an organization’s reliability program, practices, or activities in meeting the customer’s requirements for product reliability. A reliability capability assessment is the act of performing that effectiveness measurement on programs, practices, or activities, and may be performed at any place in the reliability practice hierarchy as indicated in Figure 1.

The scope of this document is organizational reliability capability assessment of programs and practices. IEEE Std 1413™ [B2] is an IEEE reliability standard that focuses on the assessment of certain specific reliability activities.

4.2 Organizational reliability capability assessment usage

The primary purpose of this standard is to provide criteria for assessing organizational reliability capability. However, before performing such an assessment, it is important to address the following: (i) the purpose of the assessment, (ii) the appropriate organizational sub-structure to assess, and (iii) the appropriate time in the product life cycle to perform the assessment. These issues are briefly addressed in this clause.

Reliability capability assessment may be applied at any place in this hierarchy

Reliability Program

Reliability Practice 1

Reliability Practice n

Activity 1 Activity 2 Activity 3 Activity n





--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



4.2.1 Organizational reliability capability assessment uses

An organizational reliability capability assessment can be used as an aid for specifying or planning reliability practices, evaluating reliability practices, and improving reliability practices. Generally, an evaluation of a reliability practice leads to the identification of some shortcomings, whose fixes would lead to an improved reliability practice.

⎯ Specifying or planning reliability practices—Reliability practices would normally be specified if product development is external or if the customer wants the reliability requirements to be achieved in a specific manner. Reliability practices would normally be planned if product development is implemented internally and the customer allows some flexibility in how reliability requirements are achieved. In either case, the reliability requirements/goals of the product must be understood by the organization, along with appropriate activities associated with those requirements/goals.

⎯ Evaluating reliability practices—The primary purpose of evaluating reliability practices is to determine the extent to which they are capable of providing a product that meets the reliability requirements/goals. An ancillary purpose of evaluating reliability practices is to determine the extent that they are capable of providing a value-added contribution to the product development effort. Generally, an evaluation of a reliability practice leads to the identification of some shortcomings, whose correction would lead to an improved reliability practice.

⎯ Improving reliability practices—If the current reliability practices have been evaluated and improvement is desired or required, a reliability capability assessment can help identify specific activities that are needed to achieve the improvement that would be most beneficial to an organization or customer.

An organizational reliability capability assessment can be performed by an external organization; e.g., by another company seeking to establish a partner or supplier relationship, or by an internal organization. An internal assessment could be performed as a normal business practice or as a response to a specific stimulus such as customer complaints, excessive warranty costs, or a perceived opportunity to use reliability for competitive advantage.

The format of organizational reliability capability assessments can be very general or very detailed, and should be determined on a case-by-case basis. It is possible to tailor the organizational reliability capability assessment to fit the needs of the contract, program, or product. The capability assessment scoring procedure used by the supplier/organization can be quantitative or qualitative. Examples of capability scoring criteria are provided in Clause 6.

4.2.2 Types of entities assessed for organizational reliability capability

An organizational reliability capability assessment may be performed for various subsets of people and products within an organization, including: (1) an entire corporation, (2) a specific facility (site), (3) specific group of products, or (4) a specific product. An organizational reliability capability assessment is appropriate for the following:

a) An entire corporation when the organization is a small business or a new start-up corporation with a limited product offering, especially when these products are related to an emerging technology that has immature processes and minimal or nonexistent production or field application experience.

b) A specific facility (site) when that facility is or will be responsible for the product(s) or technology(ies) of interest.

c) A specific group of products when there is a well-established technology or product line. The organizational reliability capability assessment could be performed for multiple products, a technology, multiple programs, or a project group.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



d) A specific product when there is a well-established business and product that is being evaluated for suitability. The organizational reliability capability assessment could be performed for a product line, a single program/project, or a single product.

The determination of the appropriate organizational substructure for organizational reliability capability assessment should also consider the product maturity, design similarities with risky products, and the availability or lack of established field reliability performance. Design similarity with risky products refers to various product offerings from a standard product line that uses elements of immature technology or problematic processes.

This standard identifies tasks, activities, and practices for assessing organizational reliability capability. The organization being assessed is not required to have a dedicated reliability engineering organization or individuals titled “reliability engineers,” and it is not implied that they are required. Furthermore, the assessment is not a measurement of product field reliability data.

4.2.3 Organizational reliability capability assessment during the product life cycle

Organizational reliability capability assessment can be performed at any stage in a product life cycle or even independently of any specific product (see IEEE Std 15288™ [B4] for information on product life cycles). In the early stages of a product’s life cycle, it may be useful to perform an internal organizational reliability capability assessment to determine if the organization has the capability to satisfy the product reliability requirements/goals, and it may be useful to perform external organizational reliability capability assessments to determine if there is any technology that can satisfy the product reliability requirements/goals. In the design stage, the organizational reliability capability assessments may focus more on specific products or manufacturing capability. In the production/support stage, reliability field data should be available, so that an assessment can also be used as feedback on the effectiveness of various design and manufacturing changes.

5. Key reliability practices

Defining organizational reliability capability involves the identification of a set of key practices that are important in the development of a reliable electronic product. These key practices should encompass all aspects of operation in the manufacturer’s organization from a product reliability perspective. Eight key reliability practices form the basis of a strategy for reliability and risk management, and are important to electronics manufacturers that aspire to achieve high capability in producing reliable products. An organization need not align its reliability activities into the same practices described in this document, but it is important that the key reliability activities described herein are performed during the system or product life cycle. The key reliability practices are shown in Figure 2 and described in this clause.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



Figure 2 —Key reliability practices

5.1 Reliability requirements and planning

The purpose of this key practice is to understand the customers’ reliability requirements1, to generate a set of reliability requirements for the products, to plan the appropriate reliability activities necessary to ensure that those requirements are met, and to plan the procedures for implementing and controlling the reliability activities in order to execute the reliability program plan. Reliability planning is a continuous process to establish and maintain plans that define reliability activities and manage the defined activities, from preliminary design to product maturity. The reliability program plan identifies and ties together all the reliability activities by producing a schedule, apportioning resources, and negotiating commitments.

The inputs, activities, and outputs from this key practice are provided as follows:

a) Inputs:

⎯ Customer inputs in the form of their requirements and expectations

⎯ Competitive reliability data

⎯ Product or program plans including budget and schedule

⎯ Expected environmental and operating conditions throughout the product lifecycle

⎯ The maturity of technology to be used for the product

⎯ Lessons learned from previous product reliability experience (test results, field information, etc.)

⎯ Applicable best practices

⎯ Applicable reliability/quality standards

b) Activities:

1 Throughout this document, all the reliability activities stated in terms of reliability “requirements” can be applied to reliability “goals.”



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Identification of available resources (materials, human resources, equipment, etc.)

⎯ Creation of reliability requirements and allocation to lower level assemblies/components

⎯ Determination of reliability analysis, reliability testing, and failure data analysis/tracking needed to ensure the product meets its requirements

⎯ Identification of potential suppliers for the product and their reliability capabilities

⎯ Planning for resources (materials, human resources, equipment, etc.)

⎯ Identification of the logistics for getting feedback on results of reliability activities

c) Outputs:

⎯ Reliability requirements and allocation to lower level assemblies/components

⎯ A reliability program plan for the product, including the required activities in the key practices of reliability analysis, reliability testing, supply chain management, failure data analysis/tracking, verification and validation, and reliability improvement

⎯ A schedule and budget for conducting the planned reliability activities, and identification of responsible individuals for the activities

⎯ Contingency planning including specification of decision criteria for altering plans and the triggers for reliability improvement activities

NOTE—The tasks of this practice may be implemented within the following life cycle processes of IEEE Std 15288-2008—Acquisition, Stakeholder Requirements Definition, Requirements Analysis, Architectural Design, Project Planning, Project Assessment and Control, and Measurement [B4].

5.2 Training and development

The purpose of training and development is to enhance the technical, business, and specialized strategic skills and knowledge of people so that they can perform their roles in the design, assessment, and manufacture of a reliable product effectively and efficiently. The aim is to generate a wide understanding of the reliability plans, goals, and methods, and their importance for both individual contributors and management. It also involves monitoring of innovative techniques and technologies.


a) Inputs:

⎯ Skills assessment of employees

⎯ Training records

⎯ List of competence requirements

⎯ List of roles and responsibilities

⎯ Current product technology and planned future technology

⎯ Available reliability training resources

⎯ Budget considerations and product schedules

b) Activities:

⎯ Development of a training plan for both individual contributors and management, including a schedule, budget, and identification of training personnel (may be a combination of internal courses, external seminars, college classes, symposiums, and so forth)



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Implementation of the training plan

⎯ Monitoring new developing technologies, modeling and analysis techniques, trends, etc. that impact reliability, and adjustment of training accordingly

c) Outputs:

⎯ Detailed understanding by reliability practitioners within the organization of reliability concepts and statistical methods and their appropriate usage

⎯ Understanding of reliability concepts and statistical methods by design engineers

⎯ Understanding among reliability practitioners about how specific reliability activities can impact reliability throughout the product lifecycle

⎯ Understanding among business managers and senior management about why reliability is important to the product or business

⎯ Training of reliability practitioners, design engineers, and manufacturing engineers on failure analysis, root cause analysis, and corrective action processes and systems

⎯ Understanding of product technology in sufficient detail to identify product failure modes and mechanisms by reliability practitioners

NOTE—The tasks of this practice may be implemented within the following life cycle process of IEEE Std 15288-2008—Human Resource Management [B4].

5.3 Reliability analysis

Reliability analysis is used to assess a product design or field performance to identify failure modes, mechanisms, and effects, and to make reliability predictions or estimates. Uses of the analysis results include, but are not limited to (a) quantifying and mitigating risk for each component or sub-system; (b) predicting and estimating operational availability; (c) establishing the warranty scheme for the product; and (d) making arrangements for spares provisioning at service sites for expected failures during and after the warranty period.


a) Inputs:

⎯ Product information in the form of product design, specifications, and engineering test results

⎯ Reliability requirements for the product

⎯ Expected environmental and operating conditions throughout the product lifecycle

⎯ Planned analysis activities and use of the analysis as outlined in the reliability program plan

b) Activities:

⎯ Identification of the failure implications of components and products; e.g., creation of reliability logic diagrams

⎯ Identification of potential single points of failure for a system

⎯ Identification of potential failure modes, mechanisms, and effects for a product

⎯ Identification of criticality of failure modes and mechanisms for a product

⎯ Creation of reliability predictions at the component and product/system level

⎯ Creation of detailed component stress and damage models when appropriate



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Assessment of adherence to design rules that impact reliability including derating, electrical, mechanical, and other guidelines

c) Outputs:

⎯ Reliability logic diagrams or equivalent identification of failure implications

⎯ Identification of design risks and weaknesses for a product; i.e., critical failure modes and mechanisms

⎯ Reliability predictions for individual components and the entire product or system

⎯ Potential component failure modes and mechanisms along with their effects and criticality

⎯ Expected field returns and spares provisioning

NOTE—The tasks of this practice may be implemented within the following life cycle processes of IEEE Std 15288-2008—Measurement and Decision Management [B4].

5.4 Reliability testing

The purpose of reliability testing is to identify design weaknesses, explore design limits and environments, and to demonstrate the reliability of products by running tests. The tests may be conducted according to industry standards, organizational standards, or customer specified requirements. The reliability testing procedures may be standard/generic schemes commonly used to test all of an organization’s products, or the tests may be custom designed for specific products using detailed product knowledge. The tests may be used for the verification of the existence and occurrence frequency of expected failure modes and mechanisms identified during reliability analysis. Tests may be conducted by the organization under assessment, suppliers, integrators, customers or third-party entities such as test houses.


a) Inputs:

⎯ Expected environmental and operating conditions throughout the product lifecycle.

⎯ Reliability requirements and reliability predictions for a product/component

⎯ Planned testing activities and use of the test results as outlined in the reliability program plan

⎯ Product information in the form of product design, specifications and engineering test results

⎯ Critical items, failure modes, failure mechanisms, and failure models identified during reliability analysis

b) Activities:

⎯ Creation of detailed reliability test plans for the product/product line including sample size for tests and the confidence level specifications

⎯ Performance of discovery testing – identifying the design margin and destruct limits for the product

⎯ Conducting design verification testing

⎯ Conducting reliability testing including on-going reliability testing, reliability demonstration testing, and accelerated testing

⎯ Modification or creation of supplier test requirements based on test results

c) Outputs:



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Detailed reliability test plans and test results

⎯ Updated reliability predictions, critical items, and failure modes/mechanisms based on test results

⎯ Identification of potential design weaknesses

⎯ Guidelines for manufacturing tests including burn-in and environmental stress screening parameters

⎯ Modified supplier test plans

⎯ Test data analysis to update the component stress and damage failure models and model parameters

NOTE—The tasks of this practice may be implemented within the following life cycle process of IEEE Std 15288-2008—Verification [B4].

5.5 Supply chain management

The purpose of supply chain management is to: (1) proactively identify sources of components that may be used to satisfy reliability requirements for a product, (2) create a list of components and suppliers that the manufacturer maintains for long-term business associations, and (3) manage the suppliers that are on contract. This usually involves supplier assessments and audits and may lead to joint customer-supplier reliability exploration. In addition, activities like tracking supplier product change notices, including end-of manufacture notices, are essential for sustaining product reliability through its life cycle and for effecting changes in product specifications or making design changes at appropriate points in time.

The inputs, activities, and outputs from this key practice are provided as follows: a) Inputs:

⎯ Parts list, bill of materials, or specifications based on functional requirements (engineering information)

⎯ Current and previous approved supplier lists and supplier contracts

⎯ Cost and schedule requirements for a product

⎯ Information on product volume and throughput

⎯ Reliability requirements and reliability predictions for a product

b) Activities:

⎯ Creation of list of potential suppliers

⎯ Vendor or supplier selection

⎯ Supplier assessment or audits

⎯ Component qualification, including identification and evaluation of key parameters

⎯ Review of product change notices including end-of-manufacture notices

⎯ Review of component monitoring data from suppliers including process, quality, reliability testing, accelerated test data, and field failure data

c) Outputs:

⎯ A list of preferred/qualified/approved parts and suppliers

⎯ Second source plans and contingency plans to deal with future part obsolescence



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Component and supplier qualification reports and supplier audit reports

⎯ Acceptance or rejection of in-coming lots based on the supplier's historical quality data and current reliability data

⎯ Reliability commitments in supplier agreements

⎯ Impact of product change notices on product reliability

NOTE—Some tasks of this practice may be implemented within the following life cycle process of IEEE Std 15288-2008—Acquisition [B4].

5.6 Failure data tracking and analysis

The purpose of failure data tracking and analysis is to collect manufacturing, functional testing, reliability testing, and field failure data to analyze the failures, to identify the root causes of failures, to generate failure analysis reports for all classes of failures, and to perform statistical analysis of failure data. In order to guide corrective action and reliability improvement activities, failure analysis reports may include the manufacture date or lot code of the failed product/component, the failure site, the failure mode and mechanism, time to failure, and recommendations for avoiding the failure mode in existing and future products. For each product, Pareto charts of failure causes and other statistical analyses are created and continually updated.


a) Inputs:

⎯ Failure reporting, analysis and corrective action system (FRACAS) plan for each project, including a failure review process2

⎯ Failures from functional testing, reliability testing, and factory testing

⎯ Field failures, including failures from alpha/beta sites and post-warranty failures

⎯ Failure dates and operating hours (or a proxy such as shipment dates)

⎯ Customer complaints

b) Activities:

⎯ Conducting statistical analyses of functional test failure data, manufacturing test failure data, reliability test failure data, and field return failure data

⎯ Confirmation of failure and conducting various levels of failure cause-analysis down to the root cause of failure

⎯ Tracking history of the failed components from production through failure

⎯ Preparation of failure analysis reports identifying failure modes and mechanisms traced to specific materials or processes

⎯ Preparation of Pareto charts and other statistical reports based on failure modes and mechanisms

⎯ Determination of appropriate corrective actions

⎯ Analysis of data from prognostic monitoring sensors

c) Outputs:

2 This may be performed during either reliability planning or failure data tracking and analysis.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Pareto charts, failure rate or time-to-failure trends, and other statistical analyses of failure modes and mechanisms

⎯ Failure analysis and corrective action reports, appropriately documented

⎯ Updated reliability predictions based on failure data

⎯ Updated failure distribution models and parameters based on failure data

⎯ Traceability of failed components

NOTE—The tasks of this practice may be implemented within the following life cycle processes of IEEE Std 15288-2008—Quality Management, Measurement, and Decision Management [B4].

5.7 Verification and validation

The purpose of verification and validation is to verify that planned reliability activities are implemented, to validate that outcomes of reliability activities are consistent with outcomes from other reliability activities, and to validate that the product meets its specified reliability requirements. This key practice is used to validate the reliability assertions made at the design stage by comparing them with the tracked and observed reliability during testing and usage. Product field reliability data can be compared with reliability estimates, reliability test conditions, and warranty return estimates. Verification involves an internal review of reliability planning, testing, and analysis activities.


a) Inputs:

⎯ Reliability requirements and planned reliability activities

⎯ Predicted and observed failure modes and failure mechanisms, from both factory testing and field operation

⎯ Identified failure effects, failure causes, and corrective actions

⎯ Reliability predictions and observed field reliability

⎯ Pareto charts and other statistical analyses of failure modes and mechanisms

b) Activities:

⎯ Conducting internal audits to monitor progress and improve reliability plans and activities

⎯ Verification of completion of root cause analysis, corrective action, and prevention of recurrence for appropriate factory test and field return failures

⎯ Verification that reliability commitments in supplier agreements have been satisfied

⎯ Comparison of potential failure modes/mechanisms identified in analysis or test with failure modes/mechanisms from field returns

⎯ Comparison of predicted product reliability and failure distributions with actual field reliability and failure distributions

⎯ Comparison of actual field reliability with reliability requirements

c) Outputs:

⎯ Results from internal audit or review of reliability activities, including opportunities for improvement and additional training

⎯ Updates to the potential failure modes/mechanisms documentation for future products



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Modification of reliability analysis methodologies and failure models used for reliability predictions

⎯ Modification of current factory testing or future reliability testing based on product field failures

⎯ Modification of spares provisioning

NOTE—The tasks of this practice may be implemented within the following life cycle processes of IEEE Std 15288-2008—Verification and Validation [B4].

5.8 Reliability improvements

Reliability improvement is the identification and implementation of product changes based on lessons learned from testing, reported field failures, technological improvements, changing product operating conditions, or other considerations. It involves implementation of corrective actions based on failure analysis, determining the effectiveness of those corrective actions, and preventing the occurrence of identified failure modes and mechanisms in future products. It also involves reacting appropriately to changes in reliability requirements for products or their lifecycle application conditions.


a) Inputs:

⎯ Failure analysis and corrective action reports

⎯ Pareto charts, failure rate or time-to-failure trends, and other statistical analyses of failure modes and mechanisms

⎯ Updates to the potential failure modes/mechanisms documentation for future products

⎯ Change in environmental and operating conditions

⎯ Change in reliability requirements due to customer complaints, competition, changing business climate, technological improvements, or other considerations

⎯ Lessons learned from reliability activities performed in earlier stages of product development or on previous products

b) Activities:

⎯ Implementation of corrective actions identified through failure analysis

⎯ Monitoring the effectiveness of the corrective actions in improving reliability

⎯ Identification and implementation of changes that will prevent recurrence of identified failure modes and mechanisms in future products

⎯ Identification of changes in product reliability requirements or their lifecycle application conditions that may require subsequent action

⎯ Evaluation of developing technologies, modeling and analysis techniques, trends, etc. that could be used to improve reliability of products

⎯ Documentation, review, and communication of lessons learned

⎯ Identification and implementation of corrective actions that will improve reliability in response to product reliability requirement changes

c) Outputs:



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



⎯ Implementation of corrective actions for identified failures, including selection of appropriate implementation scope (e.g., spares swap-out, field swap-out)

⎯ Tracking of the effectiveness of corrective actions

⎯ Modified bill of materials or supplier selection based on field reliability data

⎯ Updated product reliability requirements and changes required to meet the updated requirements

⎯ Documented record of lessons learned

⎯ Enforcement of recurrence prevention for identified failure modes and mechanisms

⎯ Inclusion of new field failure modes and mechanisms in future reliability analysis

⎯ Updated design and manufacturing guidelines for future products based on field data

NOTE—The tasks of this practice may be implemented within the following life cycle process of IEEE Std 15288-2008—Quality Management [B4].

6. Assessment criteria

The performance of reliability activities is expected to vary widely by organization and perhaps by products developed within an organization. Some organizations may be very consistent in performing the key reliability practice activities while others may be more ad hoc in some or all of the practices. The assessment provides an evaluation of organizational capability from a product reliability perspective. The assessment criteria shall be used to evaluate the completeness and consistency of the reliability activity performance. The results of the process will help an organization select process improvement strategies by determining their current reliability capability and identifying the most critical issues regarding product reliability improvements. The assessment results can also be used to steadily improve organization-wide reliability practices to enable continuous and lasting gains in organizational reliability capability.

As described in Clause 4, an organizational reliability capability assessment measures the effectiveness of a reliability program, the practices, and the activities used by an organization in meeting customer requirements. A set of five reliability capability levels is used for this assessment. Each of the eight key reliability practices shall be individually assessed with reference to a specified set of activities required to obtain a specific capability level. There is no intent to combine the key reliability practice capability levels into a single overall measure of organizational reliability capability.

6.1 Reliability capability levels

From a reliability program perspective, a high capability level implies that a particular reliability practice within an organization is well understood, is supported by documentation and training, is being continually monitored and improved by the users, and is effective and efficient.

A key reliability practice of an organization can be assigned five levels of capability that represent stages in the evolutionary transition of that practice. These five levels represent the metrics or measures of the organizational reliability capability. Associated with each capability level is a set of activities for each key reliability practice. For an organization to achieve a specific level of reliability capability, all the activities listed at that level and all lower levels that are not superseded by activities at higher levels shall be fulfilled for all eight key reliability practices.3 The definitions of reliability capability levels are provided below.

3 An exception arises when a reliability activity conducted at a lower level is made redundant by an activity at a higher level.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



6.1.1 Level 1

Level 1 is defined by the absence of qualities linked to the higher levels. Companies at this level are essentially ad hoc in their approach to reliability. These organizations are characterized by a lack of consistent procedures and an ad hoc, or sometimes chaotic nature of design, manufacturing, and reliability practices. The reliability practices, if any, are often changed or modified as a reaction to crisis situations. Reliability performance depends primarily on the capabilities and motivation of individuals, in the absence of any effort at the organizational level. As a consequence, these organizations generally produce products with inconsistent levels of reliability.

6.1.2 Level 2

Level 2 is characterized by consistent and repeatable design, manufacturing, and reliability practices. At this level, reliability activities are minimal but disciplined enough so that successes can be repeated. Reliability activities such as testing are based on generic organizational standards or prior experience with similar products rather than taking into account unique characteristics of the product. Practices that satisfy established organizational standards or that have become accepted by industry are repeated. These organizations are able to deliver products that can show conformance to codes, standards, and requirements. However, they make only minimal use of reliability data, and reliability improvement is based mainly on significant field reliability issues or customer complaints.

6.1.3 Level 3

At level 3, organizations understand and define the reliability requirements for their products. There is a standardized and consistent documentation for reliability activities, and a common understanding among employees about their roles and responsibilities. At this level, reliability training is provided for most engineers and managers to ensure that the employees have the knowledge and expertise to fulfill their assigned roles. They consider product lifecycle environmental and operating conditions in reliability activities. These organizations are responsive to test and field failures and conduct analysis of all failures. They make some design or manufacturing test modifications based on test and field failure modes. They invest resources in supplier management and reliability improvement activities that include using lessons learned for existing products, but are not as proactive at using reliability lessons learned on future products.

6.1.4 Level 4

At level 4, organizations change product designs based on reliability consideration. Reliability plans are detailed, and the defined reliability activities are specifically tailored to the product being developed. All employees are trained to appreciate the impact of reliability on the business. These organizations evaluate reliability very early in the product lifecycle and can change product designs to achieve desired reliability targets. These organizations enlist their supply chain members to help them ensure product reliability. They make some design or manufacturing test modifications based on test and field failure mechanisms as well as failure modes. Reliability is expressed as a distribution rather than a point estimate, and detailed stress and damage models may be included as part of reliability prediction. All the failure mechanisms affecting the reliability of the products are investigated and documented. These organizations are proactive in preventing the recurrence of failure modes and mechanisms in future products. They promote the establishment of closed-loop processes to utilize existing reliability lessons learned and formally identify, approve, and distribute new reliability lessons learned.

6.1.5 Level 5

At level 5, reliability is considered an integral part of strategic business planning, and organizational reliability capability is aligned with business goals. Organizations are characterized as being responsive, adaptive, and proactively focused on continuous reliability improvement across product lines. The feedback



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



from different stages of a product life cycle, such as predictions, simulations, testing, analysis, and field performance, is disseminated widely throughout the organization and incorporated in future products. These organizations use experience not only to correct problems but also to change the nature of the reliability practices that they use. They are proactive in assessing the effects of technology or business climate changes on product reliability and reliability requirements. They proactively plan to accommodate changes in supply chains, manufacturing processes, and technology. They treat reliability as an attribute to be optimized in the lifecycle cost of a product. They are proactive at sharing reliability lessons learned with other organizations and using them on future products.

6.2 Reliability capability level definitions specific to each key reliability practice

Table 1 defines the reliability capability level definitions specific to each of the key practices described in Clause 5 by delineating a set of activities that shall be performed to obtain a specific capability level. The activities are cumulative, meaning that all activities for a lower capability level, except those which are superseded by higher level activities, shall be completed in addition to the activities for the specified level. For example, to obtain a capability level 3 for a key practice, the activities listed in level 2 shall also be completed. All activities at level 1 are superseded by better practices at higher levels.

Table 1 is a set of criteria that must be considered in an organizational reliability capability assessment. Organizations may want to tailor the criteria to better suit their particular product or industry, but the eight key practices must be evaluated. In particular, an organization may want to specify how the key reliability activities are intended to be performed to a much greater level of detail, i.e., specifying a set of tasks that make up each activity.

Notes on the activities in Table 1:

⎯ Activities at lower capability levels tend to be reactive while activities at higher capability levels tend to be more proactive.

⎯ Activities involving failure modes tend to be at a lower capability level than activities involving failure mechanisms. The term failure mode refers to the effect or symptom by which the failure mode is observed. The term failure mechanism refers to the physical, chemical, or other processes that result in failure. The identification of the circumstance that induces or activates the process, often called the root cause, is considered to be included in reliability activities involving failure mechanisms.

⎯ Reliability program planning is considered part of the reliability requirements and planning key practice while detailed plans, e.g., test sample sizes, are considered part of other key practices, e.g., reliability testing.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



Table 1 —Reliability capability level definitions specific to the key reliability practices

Reliability requirements and planning Training and development

Level 1

• Reliability plans and requirements are informal or nonexistent.

• Informal quality or reliability training may be provided to some employees, but no sustainable program is evident.

Level 2 • Reliability requirements exist, based on customer inputs or requirements, existing products, and/or competitive information.

• A reliability program plan with a list of scheduled activities exists.

• A list of reliability training opportunities is provided.

• Reliability practitioners are trained in reliability concepts and statistical methods for reliability prediction and data analysis.

• Reliability training is made available to the design organization.

Level 3 • Reliability requirements take into account environmental and operating conditions.

• Reliability requirements are allocated to lower level assemblies/components.

• The reliability program plan includes a list of required resources such as materials, personnel, and equipment.

• Program plans include planned reliability activities with responsible individuals identified.

• There is evaluation of potential suppliers for new technology to identify reliability risks.

• Reliability practitioners, design engineers, and manufacturing engineers are trained in root cause analysis and corrective action.

• Training records are maintained. • Reliability management is trained on how

specific reliability activities can impact reliability throughout the product lifecycle.

• Business managers appreciate how reliability impacts business.

Level 4 • The reliability program plan is tailored for the specific product and adapted to knowledge gained throughout the product lifecycle.

• Product reliability requirements are expressed as a distribution instead of a point estimate when appropriate.

• Reliability requirements include utilization of lessons learned.

• Reliability and statistics courses are specifically tailored for design and manufacturing engineers.

• Reliability practitioners are trained to identify failure modes and mechanisms relevant to the technology used in a specific design.

• Training is provided to senior managers to appreciate how reliability impacts business.

Level 5

• Contingency planning is used. • Decision criteria, such as business or

marketing considerations, for altering the reliability requirements are identified.

• Reliability is considered an integral part of strategic business planning.

• New technologies, trends, and reliability modeling or analysis techniques that impact product reliability are constantly tracked, and training is adjusted accordingly.

• Proactive support for reliability training across the organization is provided by top management.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



Table 1–Reliability capability level definitions specific to the key reliability practices (cont.)

Reliability analysis Reliability testing

Level 1

• Reliability analysis is non-existent or inconsistent and mainly based on manufacturing issues.

• Testing is primarily functional.

Level 2 • Point estimates of product reliability predictions are created based on modeling or reliability prediction handbooks.

• Potential critical failure modes are identified and informally documented.

• Reliability logic diagrams or equivalent identification of failure implications are created.

• A generic reliability test plan exists. • Reliability testing is based on

customer specifications or industry standards.

• Products are subjected to functional testing before shipping.

• Design verification and product qualification tests are conducted.

Level 3 • Potential failure modes and their effects, especially single points of failure, are identified and documented.

• Potential critical failure mechanisms are identified and informally documented.

• Field reliability data and reliability test data for the product of interest or similar products are used to make or adjust reliability predictions.

• Reliability predictions are used for warranty return estimation and spares provisioning.

• Adherence to design rules that impact reliability, such as derating, is confirmed.

• Design changes initiate a re-evaluation of product reliability.

• A detailed reliability test plan exists, including test sample size and confidence limit specifications.

• Reliability testing is based on potential failure modes and environmental and operating conditions.

• Reliability test requirements for parts supplied by vendors are modified if appropriate.

• Reliability test results are used to make design changes in products prior to production.

• Tests to identify design limits are conducted for all products.

Level 4 • Reliability predictions are expressed as a distribution instead of a point estimate when appropriate.

• Reliability predictions include confidence limits. • Models for specific failure mechanisms are used to

make reliability predictions for products when appropriate.

• Reliability analysis is used to design specific reliability tests for products.

• Changes in environmental or operating conditions initiate a re-evaluation of product reliability.

• Accelerated tests are tailored for expected failure mechanisms throughout the product lifecycle.

• Manufacturing burn-in/screening is designed to address known early life failure mechanisms.

• Tests to identify destruct limits are conducted.

Level 5

• Lifecycle costs of a product are optimized based on reliability vs. cost trade-offs.

• Detailed component stress and damage models are created when appropriate.

• Reliability analysis to determine reliability risk is included in initial evaluation of new technologies.

• Reliability test data is used for component stress and damage model parameters.

• Reliability testing is performed during evaluation of new technologies.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---




Supply chain management Failure data tracking and analysis

Level 1

• Part and supplier selection is mainly based on part functional characteristics and price.

• Failures during functional testing may be recorded.

Level 2 • Approved parts and supplier lists are created. • Supplier assessments or audits are conducted for all

new suppliers of critical components. • Components are qualified for use in their intended

applications based on supplier data or limited functional testing.

• Second source plans are created as appropriate.

• Factory test failure data and field reliability data are collected and regularly reported.

• Pareto charts or other decision aids based on failure data are created.

• Failure analysis and corrective action is conducted for major failures.

Level 3 • Design specifications contain reliability requirements. • Periodic supplier assessments or audits are conducted

for all suppliers of critical components. • Approved parts and supplier lists are created and

maintained based on supplier assessments or audits. • Selection of critical component suppliers includes

evaluation of component field reliability, if available, in the current or similar products.

• Suppliers of critical components are required to conduct self-assessments, qualify their parts and suppliers, and perform failure analysis and corrective action on field failures.

• Product change notices, including end-of-manufacture notices, are evaluated for their effect on product reliability.

• Failure data are reported and used to guide selection of products and failure modes for root cause analysis.

• Failure confirmation and root cause analysis is conducted on critical failures from all sources including development test, manufacturing, and field failures (possibly on a sample basis).

• Cross-functional team meetings are conducted for all major failures with the results recorded and disseminated, including corrective actions.

• Time-to-failure data is collected and analyzed.

• Reliability predictions are updated based on failure data.

Level 4 • Acceptance of critical component shipments is based on evaluation of supplier reliability data for that shipment.

• Technology maturity is considered during the selection of components.

• Suppliers conduct on-going assessments of their suppliers, flow down customer requirements to suppliers, and demonstrate continuous improvement.

• Failure analysis reports include details on underlying failure mechanisms.

• Failure mechanisms are traced to specific materials or processes.

• Failure distribution models and parameters are updated based on failure data.

• Part traceability exists such that failures can be associated with specific manufacturing dates/lots/processes.

Level 5

• Changes in product environmental and operating conditions, reliability requirements, and/or technology trigger a reassessment of parts and suppliers.

• Key component parameters are periodically re-evaluated to identify shifts.

• Contingency plans to deal with future part obsolescence are developed.

• Key suppliers are engaged in a joint exploration of the effect of processes, materials, and technology on product reliability.

• Links between customer satisfaction and failure analysis/corrective action is understood.

• Data from prognostic monitoring sensors is analyzed and used to predict/prevent future failures.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---




Verification and validation Reliability improvements

Level 1

• Verification and validation activities are informal, based on individuals rather than process.

• Process and product reliability improvements are non-existent or undocumented.

Level 2 • Management verifies that the reliability program plan is being followed.

• Field reliability data is compared to reliability requirements and predictions.

• Predicted field returns are compared to actual values to adjust spares provisioning.

• Corrective action implementation is verified.

• Design and process changes that may affect reliability are documented, e.g., using engineering change notices.

• Corrective actions in response to significant field failures and customer complaints are implemented.

• Bills of materials are modified to exclude parts or suppliers with poor field reliability performance.

Level 3 • Supplier management verifies that reliability commitments in supplier agreements have been satisfied.

• Manufacturing test results such as stress screening and on-going reliability testing are compared to product field failures, and tests are modified appropriately.

• Observed field failure mode data is compared to predicted failure modes and used to appropriately modify reliability analysis and reliability test methodologies.

• Corrective actions in response to significant internally-identified failures are implemented.

• Effectiveness of corrective actions is tracked.

• Identified failure modes are used to prevent recurrence in future products.

• Reliability improvements are identified and implemented in response to changing environmental or operating conditions.

Level 4 • Internal reviews of reliability activities are performed and used to identify opportunities for improvement and additional training.

• There is verification that potential failure modes/mechanism documentation is updated and used to prevent recurrence in future products.

• Observed field failure mechanism data is compared to predicted failure mechanisms and used to appropriately modify reliability analysis and reliability test methodologies.

• Identified failure mechanisms are used for preventing recurrence in future products.

• New modeling and analysis techniques are evaluated and implemented to improve product reliability evaluation.

• Lessons learned from reliability activities performed previously are documented, reviewed, and communicated.

• Reliability improvements are identified and implemented in response to changing business conditions and lessons learned.

Level 5

• Observed field failure mechanism data is compared to predicted failure mechanisms and used to update component stress and damage models.

• Reliability predictions match observed field reliability.

• New technologies are evaluated and implemented to improve product reliability.

• Design rules are updated based on field failure information.



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



Annex A

Informative

Bibliography

The following documents provide useful information that can help a user during the implementation of this standard:

[B1] IEEE Std 1332™-1998, IEEE Standard Reliability Program for the Development and Production of Electronics Systems and Equipment.4

[B2] IEEE Std 1413™-1998 (Reaff. 2003), IEEE Standard Methodology for Reliability Prediction and Assessment for Electronic Systems and Equipment.

[B3] IEEE Std 1413.1™-2002, IEEE Guide for Selecting and Using Reliability Predictions Based on IEEE 1413.

[B4] IEEE Std 15288™-2008, IEEE Systems and Software Engineering—System Life Cycle Processes.

The following references were used as source material during the development of the standard and provide useful information on background and previous work that contributed to the conceptualization and formulation of different portions of this document:

[B5] Bamberger, J. “Essence of the Capability Maturity Model,” Computer (June 1997): pp. 112–114.

[B6] Brombacher, A.C. “Maturity Index on Reliability: Covering Non-technical Aspects of IEC61508 Reliability Certification,” Reliability Engineering & System Safety vol. 66, no.2 (Nov. 1999): pp. 109–120.

[B7] Conti, T. Organizational Self-Assessment. London: Chapman and Hall, 1997.

[B8] Cushing, M.J., D.E. Mortin, T.J. Stadterman, and A. Malhotra. “Comparison of Electronics—Reliability Assessment Approaches,” IEEE Transactions on Reliability vol. 42, no. 4 (December 1993): pp. 542–546.

[B9] Fraser, P, J. Moultrie, and M. Gregory. “The Use of Maturity Models/Grids as a Tool in Assessing Product Development Capability,” IEEE International Engineering Management Conference IEMC ’02, vol. 1 (2002): pp. 244–249.

[B10] IEEE 100™, The Authoritative Dictionary of IEEE Standards Terms, Seventh Edition, New York, Institute of Electrical and Electronics Engineers, Inc.

[B11] IEEE Std 1332™-1998, IEEE Standard Reliability Program for the Development and Production of Electronics Systems and Equipment.

[B12] IEEE Std 1413™-1998 (Reaff. 2003), IEEE Standard Methodology for Reliability Prediction and Assessment for Electronic Systems and Equipment.

[B13] IEEE Std 1413.1™-2002, IEEE Guide for Selecting and Using Reliability Predictions Based on IEEE Std 1413.

[B14] IEEE Std C37.10™-1995, IEEE Guide for Diagnostics and Failure Investigation of Power Circuit Breakers.

4 IEEE publications are available from the Institute of Electrical and Electronics Engineers, 445 Hoes Lane, Piscataway, NJ 08854, USA (http://standards.ieee.org/).



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---



[B15] Ke, H-Y., and C-P. Hwang. “Reliability Programme Management Based on ISO 9000,” International Journal of Quality & Reliability Management vol. 14, no. 3 (1997): pp. 309–318.

[B16] Pecht, M.G., and F.R. Nash. “Predicting the Reliability of Electronic Equipment,” Proceedings of the IEEE vol. 82, no. 7 ( July 1994): pp. 992–1004.

[B17] Sander, P.C., and A.C. Brombacher. “MIR: the Use of Reliability Information Flows as a Maturity Index for Quality Management,” Quality and Reliability Engineering International vol.15, no.6 (Nov.–Dec. 1999): pp. 439–447.

[B18] Tiku, S., M. Azarian, and M. Pecht. “Using a Reliability Capability Maturity Model to Benchmark Electronics Companies,” International Journal of Quality & Reliability Management vol. 24, no. 5 (2007): pp. 547–563.

[B19] Tiku, S., and M. Pecht. “Auditing The Reliability Capability Of Electronics Manufacturers,” Proceedings of ASME InterPak’03, the International Electronic Packaging Technical Conference and Exhibition (Maui, Hawaii, USA.) (July 6–11, 2003).

[B20] _______. “Reliability Capability Assessment Methodology,” Proceedings of IMAPS Brazil 2003, the International Technical Symposium on Packaging, Assembling and Testing & Exhibition, Campinas (SP, Brazil) (August 6–8, 2003).

[B21] Tiku, S., M. Pecht, and J. Strutt. “Organizational Reliability Capability,” Proceedings of Canadian Reliability and Maintainability Symposium (Ottawa, Canada) (October 16–17, 2003).

[B22] United States Department of Defense. “Reliability Program for Systems and Equipment Development and Production,” MIL-STD-785B (September 15 1980).

[B23] Van Donk, P.K., and G. Sanders. “Organizational Culture as a Missing Link in Quality Management,” International Journal of Quality & Reliability Management vol. 10, no. 5 (1993).

[B24] Williams, K., N. Robertson, C.R. Haritonov, and J. Strutt. “Reliability Capability Evaluation and Improvement Strategies for Subsea Equipment Suppliers,” Journal of the Society for Underwater Technology vol. 25, no. 4 (2003).



--`,`,`,,`,```,,``,`,``,`,,```,`-`-`,,`,,`,`,,`---