Dealing with nonfunctional requirements in large software systems

Annals of Software Engineering 3 (1997) 367–395 367

Dealing with nonfunctional requirements in largesoftware systems

Christof Ebert

Alcatel Telecom, Switching Systems Division, Antwerp, Belgium

Quality improvement such as increased reliability and maintainability are of utmostimportance in software development. In this field, which was previously ad hoc andunpredictable rather than customer-oriented, increasing competition and focus on cus-tomer satisfaction have motivated management to put more emphasis on quality issues.This paper provides insight in techniques for dealing with what customers call genericquality attributes and what software engineers call nonfunctional requirements. Since re-quirements management more than many other disciplines in software engineering needpractical insight, examples are provided for dealing with four nonfunctional requirementsin large telecommunication systems, namely performance, usability, reliability, and main-tainability. Guidelines are presented for specifying, constructing, analyzing, measuringand tracing nonfunctional requirements. Many examples from telecommunication sys-tem development show how to specifically and pragmatically deal with nonfunctionalrequirements. Extracts from three common standards (SEI CMM, ISO 9000–3, BELL-CORE suite) are added in the form of a checklist for underlining the importance of properrequirements management from a customer viewpoint.

1. Introduction

Failures in telecommunication systems, such as a switching circuit failure in1992, recently showed that low software quality in such a sensible field has muchmore effect than defects located to a distinct application. Compared to other productswith high software complexity (e.g., scientific computing, real-time systems) telecom-munication systems provide manifold risks (based on the product of probability andeffect) due to high coupling which lets failures being distributed rapidly nationwide,and numerous other systems being dependent on correct functionality of real-timeinformation exchange.

Competition, along with the customer’s willingness to change providers when-ever he is dissatisfied has resulted in huge efforts to provide switching software ontime and with exactly the quality the customer has specified and expects to pay for.Formal approaches to guarantee quality products have lead to international guidelines(e.g., ISO 9000–3) and currently established methods to assess the development pro-cesses of providers (e.g., SEI CMM). Our goal that is presented in this paper is toprovide an integrated approach for dealing with all kinds of requirements when theyare first visible and therefore reduce late and expensive rework.

J.C. Baltzer AG, Science Publishers

368 C. Ebert, Dealing with nonfunctional requirements

Due to the limited size of this paper we focused on an area that is too oftenneglected in system development and even in requirements management [Davis 1993;Gause and Weinberg 1989], namely dealing with nonfunctional requirements (here:NFR). This paper provides insight in techniques for dealing with what customerscall generic quality attributes and what software engineers call NFR. We emphasizeexactly this quality viewpoint of NFR, because this is exactly what affects customers,both internal (e.g., testing or maintenance engineers) or external (telecommunicationoperators or service suppliers).

There are two key messages we want to emphasize in this paper:

1. NFR are requirements too. Collection, specification, analysis, tender, allocation,design, and testing can be seen as similar to dealing with functional require-ments.

2. Experience shows that special emphasis must be given to each of the selectedNFR simultaneously, and to assigning priorities in case of conflicting goals.NFR thus can only be treated in an integrated and comprehensive fashion,instead of dealing this year with performance and next year with maintenance.

Dealing with NFR is negotiating and following rather medium-term targets.Firefighting is in many cases contradictive to achieving NFR. Experiences so far aregood which can be seen in the increasing use of the provided checklists and thereported nonfunctional major errors during inspections. It is on the other hand inindustrial projects not feasible to prove the effects of such techniques completelybecause they are never applied in a comparative study. Rather obvious effects areincreasing awareness of the importance of NFR during design and their measurementin late test phases.

Dealing in detail with NFR intends to support the following objectives:

– provide guidelines of specifying and tracing NFR;

– provide guidelines for selecting those NFR that are of the highest customer ormarket priority;

– provide guidelines for resolving conflicts due to different NFR (to be achievedin the same or different related projects);

– provide visibility of achieved product quality in terms of NFR;

– improving product quality in terms of NFR;

– provide early indication of not fulfilled NFR;

– provide visibility of compliance with NFR;

– provide a platform for benchmarking.

For clarity and unambiguous use of phrases the following definitions are used(most of them according to standards terminology [Davis 1993; Dorfman and Thayer1990]): procedures define what is to be done, how it is to be performed, who is theresponsible, and when results are due either on a relative scale or with a time schedule.

C. Ebert, Dealing with nonfunctional requirements 369

Procedures are part of processes. Contracted implies a signed agreement of involvedparties. Established means documented and put into practice. Technical requirementsinclude all requirements covering both functional (e.g., protocols) or nonfunctional(e.g., maintainability) aspects. Nontechnical requirements include requirements cov-ering commercial and planning issues (e.g., delivery dates for technical requirements).A functional requirement is a requirement that specifies each function that a systemmust be capable of performing. Examples for functional requirements include proto-col specifications or the number of lines necessary for a distinct local switch. Non-functional requirements or general quality attributes emphasize that quality meanscompliance to requirements. A requirement that describes not what the software willdo, but how the software will do it is called a nonfunctional requirement. Examplesfor NFR include maintainability or reliability (see ISO 9126 for precise definitions ofsuch nonfunctional requirements; there might be overlappings, for instance reliabilitycan be defined as time between malfunctions, and is as such closely related to thewhole set of functional requirements). Both groups can be classified in being eitheruser-oriented (i.e., specified from a user point of view) or being development-oriented(i.e., specified from an internal viewpoint). The classification of requirements from acontents viewpoint and target-audience viewpoint is presented in Fig. 1.

Tracing requirements is the constructive approach of identifying, designing andtesting according to given requirements. It includes identifying requirements in givenwork products. While the CMM distinguishes between tracing and tracking (to clo-sure) we treat both terms as the same in the context of managing requirements. Tracingimproves all kinds of impact analysis (e.g., changing requirements, corrective mainte-nance and reduced ripple effects). Acceptance criteria are agreed measurable projecttargets. From a purchaser’s point of view they are typically contracted. Internal ac-

Figure 1. Classification of requirements with examples for different technical requirements.


ceptance criteria might be defined to guarantee adherence to defined processes andtheir interfaces in terms of work product deliveries. Operational profiles (operationalusages) are user- (purchaser-) oriented system test cases that are selected together withthe customer to ensure that functionality is tested according to typical usage and re-sulting risks (frequency of usage, combination of functions). They are often realizedwith automatic test tools.

The paper is organized as follows. The introductory section 2 gives an overviewof the environment and development of the Alcatel 1000 S12 digital switching system.Section 3 outlines the objectives of NFR in general with examples from telecom-munication. Sections 4 and 5 describe requirements management for user-orientedNFR and development-oriented NFR. They cover specification, tracing, and measur-ing. Section 6 then gives practical guidelines for these activities including extractsfrom quality plans. Section 7 outlines very detailed how to apply results from sec-tions 4 to 6. Since requirements management more than many other disciplines insoftware engineering need practical insight, examples are provided for dealing withfour nonfunctional requirements in large telecommunication systems, namely perfor-mance (traffic load requirements), usability (documentation), reliability (failure ratesduring field operation), and maintainability (software changes). Guidelines are pre-sented for specifying, constructing, analyzing, measuring and tracing nonfunctionalrequirements.

Extracts from three common standards suites (SEI CMM, ISO 9000–3, BELL-CORE) are added as a synopsis in the form of a checklist for underlining the im-portance of proper requirements management from a customer viewpoint. Althougha tutorial is available that covers several standards dealing with requirements engi-neering [Dorfman and Thayer 1990], this has the disadvantage focusing too muchon US standards and military applications. Knowing the need from own experience(e.g., to check new procedures or process definitions for accordance to standards thatare already around) we tried to cover standards that are now often requested fromcustomers in different locations and in different application domains.

2. Requirements engineering in telecommunication systems development

Telecommunication switching systems are among the biggest challenges in cur-rent software development because they are distributed both during development timeand during runtime. Due to their considerable size (several million LOC), such systemsare developed within locally distributed development units within globally operatingcompanies. The Alcatel 1000 S12 is a digital switching system that is currently usedin over 40 countries worldwide. It provides a wide range of functionality (small lo-cal exchanges, transit exchanges, international exchanges, network service centers, orintelligent networks) and scaleability (from small remote exchanges to large local orTOLL and international exchanges). Its typical size is several MLOC of which a bigportion is customized for local network operators. The organization responsible for


development and integration is registered to the ISO 9001 standard. Typically totalstaff for development, testing, integration, and field delivery of such a system exceeds500 software engineers and provides several local releases for different customers peryear. Even core development staff at Alcatel (not responsible for integration and fielddelivery) is distributed over several sites in different European countries.

At runtime, the switching software is distributed on many interacting processorsin switching systems of various sizes and engaged in various customer interactions.Runtime environments usually differ from the development environment (e.g., main-frames, workstations). The code used for Alcatel 1000 S12 is realized in CHILL, Cand Assembler. In terms of functionality, Alcatel 1000 S12 covers almost all areasof importance in software and computer engineering simultaneously. This includesoperating systems, database management and distributed real-time software. In termsof effort or cost, the share of software is increasing continuously and is currentlyin the range of 80%. Due to manifold customer requirements (e.g., service featuresand correlations of such features), international compatibility (e.g., protocol changes)and efficient maintainability from the service provider point of view (e.g., installa-tion, portability), the problem specific complexity of switching systems is at the upperend of what can be handled by current practices [McDonald 1994; Karmarkar 1994;Glossbrenner 1993]. It is therefore obvious that any additional complexity producedby design and implementation should be minimized.

Requirements management is centered around a complete product lifecycle fromcontracting to delivery and maintenance (Fig. 2). Beyond common approaches fortreating technical requirements this approach covers customer negotiations, tender,analysis, allocation, design for NFR, testing, and maintenance.

Figure 2. Lifecycle from the entry–exit–work product viewpoint.


3. Objectives of nonfunctional requirements

Quality is the degree to which the sum of all features of a system reflect thecustomer’s requirements. From a commercial point of view there is basically no sensein achieving quality without underlying requirements. In order to enhance the overallquality of a product it is not enough only to fulfill functional requirements. Theresulting switching system might work, while being difficult to use (e.g., operationand maintenance activities) or while showing too many failures in a distinct timeframe.

This is when nonfunctional requirements show up. They can be separated inuser-oriented NFR and development-oriented NFR. Typically user-oriented NFR arecontracted based on measurable (performance) aspects. Development-oriented NFRare often stated only in qualitative terms without providing insight in how to achieveor measure them.

User-oriented NFR include:

– performance (e.g., call set up time);

– reliability (e.g., critical failures per month);

– availability (e.g., supplier dependent total downtime per year);

– failure tolerance (e.g., failure recovery upon restart);

– usability (e.g., reaction time upon performing changes of ISDN features);

– correctness (e.g., dropped cell rate per second).

Development-oriented NFR include:

– extendibility (e.g., free memory space of modules);

– maintainability (e.g., few ripple effects of changes);

– readability (e.g., consistency of code and comments);

– reusability (e.g., clearly defined and separated functionality of distinct compo-nents);

– fault tolerance (e.g., distributed functionality);

Unfortunately most of these NFR are not measurable during design. They are evendifficult to test; therefore they are usually evaluated subjectively. It is hence difficultto trace them similarly to functional requirements. Although the difference betweenfunctional requirements and NFR has been known for a while [Davis 1993; Myopouloset al. 1992] it is difficult to find comprehensive coverage for all processes that includerequirements management. Some recent approaches include designing according torequirements [Kazman and Bass 1994] although the mentioned technique goes notfurther then user interface design and common production toolchains. Even establishedcomprehensive lifecycle process models such as the Cleanroom process only considerfunctional requirements and usage profiles, while typical NFR are not treated [Linger1994]. And since they can typically not be specified formally they are not even


verified in this approach. Quality models on the other hand cover NFR with sufficientpreciseness, however typically fail to indicate how to design and trace them. Onlyrecently comprehensive quality models are discussed that follow a similar approachas this article [Dromey 1996]. So far they focus on setting measurable targets and onappropriate design techniques for selected quality factors, such as maintainability. Theonly established technique that bridges NFR in general to design seems to be QualityFunction Deployment (QFD) [Ohmori 1993; Brown 1991]. Analysis techniques werediscussed recently, namely a comprehensive insight in requirements review techniqueswith lots of examples [Porter et al. 1995] and metrics for requirements management[Costello and Liu]. Since this paper tries to combine and integrate different techniquesfor dealing with NFR it refers to and learns from what is available instead of discussingthese approaches.

4. User-oriented NFR

The objective of this section is to support requirements engineering and productdefinition with guidelines on how to achieve specifications and measurements for NFRthat are of highest customer interest. This includes criteria for selecting appropriateNFR. In general user-oriented NFR are treated similar to functional requirements. Bothare relevant for the contract as it is established between the customer and the supplier.Consequences of not covering a NFR are often more severe than leaving out a distinctfunctionality. Examples include reduced reliability in terms of contracted failuresper month or loss of stability due to overheating under extreme climatic conditions[McDonald 1994; Karmarkar 1994; Glossbrenner 1993].

4.1. Specifying user-oriented NFR

User-oriented NFR are typically specified within the technical requirements ofthe user that are the subject of a contract. To get an idea of the spectrum of user-oriented NFR the first part of sections 7.1–7.4 includes headers of typical user-orientedNFR. Unfortunately tradition of telecommunication systems is heavily oriented towardstraffic, load, and performance objectives. Only recently customers’ organizations (EU-RESCOM, BELLCORE) start dealing with other user-oriented NFR, such as reliabilitycriteria [Glossbrenner 1993]. Some even go as far as defining the development processand in process quality checks to be performed or at least audited by the customers’experts [Brown 1991].

As an example some typical definitions for user-oriented NFR are provided asthey are used in common telecommunication standards. A fault is a defect whichmay be described by a state, while a failure is an event and thus dynamic by nature.Failures are caused by faults. Reliability is the probability of failure-free operation ina specified environment for a specified time. Availability is the ratio of uptime (timeof failure-free operation) divided by the total time that the system is intended to beavailable.


Of course NFR should not be separated from functional requirements in differentsections of a spec. While this is reasonable in few cases (e.g., development-orientedNFR), it can lead to serious fragmentation which reduces readability, particularly whenfunctions and performance are split. NFR without clear relations to functions mightbe grouped according to the categories design constraints, external interfaces, qualityattributes, and performance.

4.2. Tracing user-oriented NFR

The basis of all tracing activities of user-oriented requirements, both functionalor nonfunctional is a requirements database. For easiness of tracing requirements fromanalysis through design and coding to testing and delivery and also tracing them back,this database should have close links to similar databases for features (design database)and functional testcases (Figs. 2 and 3). Further links to the project managementdatabase ensures that requirements-oriented metrics can not only be generated easilybut also distributed to wherever they are needed.

Tracing user-oriented NFR starts with the technical and commercial analysisof all requirements. The input to both activities is the requirement specification (i.e.,description with sufficient details and relations to other documents), a descriptionof the proposed project, product, or platform, and the handover date. The outputof both activities is the basis for allocation of requirements to products, projects,or platforms. This includes the statement of clarity, tables with interactions amongrequirements, identification of related requirements, clear description of originalityand differences to existing features, a feasibility and risk assessment, statements of

Figure 3. Requirements tracing.


affected items in terms of documents and components, test specifications related totesting this requirement, and effort estimation.

Technical analysis provides necessary effort for realizing distinct NFR (e.g.,new processors for increased performance or enhanced review effort for improved re-liability) as well as impacts of realizing NFR (e.g., improved usability implies marketresearch and user-oriented prototyping) and potential interactions with other require-ments (e.g., better maintainability by normalizing databases typically reduce failuresdue to data inconsistencies and might even improve overall performance). Based onthese outputs commercial analysis must deal with cost, benefit and conflicts of NFRand assign priorities to each NFR. Techniques for technical analysis include QFD, orroot cause analysis of reported failures [Ohmori 1993; Brown 1991].

In QFD a matrix is assembled, using a graphic display called the house of qual-ity, which contains: information about the quality requirements; the importance to beattached to each of the quality requirements; the mechanisms to address these require-ments in the product; and targets for these mechanisms. QFD forces the developersto be explicit about the software quality requirements and their relative importance.It acts as a vehicle for communication even for tracing NFR. QFD can be applied ina phased fashion with a sequence of matrices in which traceability is preserved bylinking the mechanisms in each phase to the requirements in the succeeding phase.

Designing for usability for example may be achieved by accurately modelingthe users’ tasks and ensuring a direct mapping between these task models and softwaredesign. This is the first step of a – typically hierarchically and recursively applied –QFD approach. Since usability implies intensive and early user interaction, prototypingmight be the appropriate solution. However, prototyping without an awareness of usersand tasks leads to post hoc usability design.

Designing for coverage of NFR implies broad know-how of the system, its func-tionality, and state of the art methodologies in system design and software engineering.As an example take availability. In order to achieve high availability, one of the ba-sic approaches is to use redundancy to build a fault tolerant system. A hierarchicalsystem architecture is best suited to achieve system availability. Formal techniquesmight be applied to avoid failures in components that are critical for functionality(such as basic I/O functions). Most NFR can be achieved by well-known approaches,such as reuse of standard software components, applying structured techniques thatprotect modularity and the original architecture, and using CASE environments forclear design specifications that can be traced back and forth. Sometimes a NFR is sodominant that it can reject large numbers of possible solutions and lead a designer tothe kind of solution that is later incomprehensible.

Testing user-oriented NFR is typically not achieved by simple testcases, suchas observing protocols on network lines and compare them with accurate specifica-tions. Testing an implementation against NFR may require modeling (e.g., for testingreliability), mathematical induction (e.g., for load limits or setup times), or gatheringsuitable information during the qualification process in an environment as close to theoperational environment as possible (e.g., for usability aspects). Deductive approaches


might help in refining a NFR towards simple test-cases. Take subscriber interferenceas an example. The basic NFR states that “a subscriber shall not disturb anothersubscriber or the switching system whatever she does”. To test this NFR, it is brokendown with a root cause analysis. Disturbances may depend on the architecture (e.g.,subscribers lined to the same I/O board) or they might result from faulty terminals(e.g., high voltage, incorrect protocols). According to resulting basic situations, a setof testable requirements can be derived from the untestable NFR.

4.3. Measuring user-oriented NFR

If NFR are not measured they might be well out of control. If there are noclear goals specified for tracing specific metrics through the development processesengineers will automatically stick to functional requirements because those are easierto identify and trace.

After having identified quality goals and measurable targets together with re-spective priorities, the related quality and release criteria are defined (Fig. 4). At leastthe latter must be measurable during the development process or during main reviews.An example for a checklist that can be used in a quality plan is provided in section 6.

An example is provided for measuring usability [Davis 1993; Kazman and Bass1994]. There are two approaches to measure usability which of course both involvecustomers, specifically potential users. The subjective approach would be to let usersassign ratings to the system as a whole and to specific aspects of the system, e.g.,frequency and effects of down-time, psychological perception of performing distincttasks, or time to get familiar with the system when learning new functions. Anobjective setting that even allows for comparing competing products include askingusers to perform precisely defined tasks on both systems and measure time or overalluser performance (e.g., in terms of failing to achieve the goal, number of errors) inoperational use. Goals representing NFR can rarely be said to “accomplish” or “fullysatisfy” these requirements as it is typical for functional requirements.

Figure 4. Quality factors, quality criteria and metrics.


5. Development-oriented NFR

Form a pure design viewpoint it might seem overly accurate to distinguishbetween the treatment of development-oriented and user-oriented NFR. Unfortunatelyour experience especially with large legacy systems tells us that too often the few con-tracted (user-oriented) NFR are followed because they are measured at delivery time,while most development-oriented NFR are neglected when it comes to integration ofthe system. In the long run these tributes to firefighting are those that are most ex-pensive to change. Therefore specification and measurement of development-orientedNFR are worthwhile being considered separately in detail, while the tracing can becombined.

5.1. Specifying development-oriented NFR

Development-oriented NFR must be provided early and consistent. Too oftenexactly these NFR are stated informally and nobody knows what is meant and how itis to be achieved.

What might be even more important from a commercial viewpoint is that thisclass of NFR is clearly oriented towards assuring that the system has an overall future.

Since most of the development-oriented NFR have been repeated for long bycustomers they are typically not restated in a requirements specification. Besides,they are typically the underlying goal for applying design guidelines [Davis 1993;Kazman and Bass 1994]. It is much more important to enforce following these fewdevelopment-oriented NFR through the complete development and maintenance pro-cess, no matter what short-term targets might be emphasized.

Improving an architecture once it is not maintainable any more is impossible.Postponing several technical requirements on the other hand might cost penalties,however never endanger a product’s future.

5.2. Tracing development-oriented NFR

Design decisions may affect positively or negatively particular NFR. Thereforenot only the NFR have to be checked in terms of potential conflicts, but also thefollowing design decisions. QFD and Pareto analysis to weight conflicting factors aretypically best suited.

Tracing fault-tolerance for example starts with system design and architecture(e.g., design diversity, global fault tolerance functions, and redundance). For example,diversity can be specified precisely with design languages and methods, test strategiesand algorithms. Top-level design reviews already allow for checking what architecturalmeans could contribute to fault tolerance. During subsystem design algorithms for faultdetection and fault recovery procedures during runtime can be included dependingon specific functionality and design constraints, while design fault removal is moreof analytic relevance. After detailed design reviews include models, experiments,and simulation to identify potential fault reasons and to evaluate or validate fault


detection and recovery procedures. Knowing about typical (external faults) suggestsanother efficient technique, namely addressing and removing reasons for faults, suchas physical (environmental) or human (interface) faults.

A design for extendibility on the other hand has to provide means to implementnew features and to load it while the switching system is continuously operative.

5.3. Measuring development-oriented NFR

Since it is often impossible to measure coverage of development-oriented NFRbased on primitive metrics, most approaches include independent evaluation or val-idation personnel that neither belongs to the same team nor belongs to a directlycompeting team. One of the few measurable attributes is fault tolerance that has beensubject of hardware design for long. A typical measure would be the probability offailures due to seeded faults in the system (which includes the system’s reaction tosuch faults, e.g., recovery, etc.).

There are many indirect metrics related to development-oriented NFR. Typicalexamples include complexity metrics that are applied for maintainability predictionin terms of identifying such components that are difficult to maintain [Ebert andLiedtke 1995]. An example is a maintenance- or reuse-oriented goal which statesthat control flow should not be overly nested. Measuring the amount of branchingand its hierarchies would be an indirect metric related to the two NFR. Related designdirectives could state that there is a limit of decisions per KLOC, or that only a distinctpercentage of all modules are allowed to violate this first criterion. However, neitherproduct-oriented nor process-oriented indirect metrics have a monopoly on properlytreating NFR.

An example is provided for measuring maintainability. After implementation amaintenance task is performed by several teams and individual effort is measured. Toreduce costs for such an activity the maintenance task could involve finding seededfaults or extend the functionality once a minor change request happens. Anotherapproach would be asking maintainability related questions in order to get subjectiveratings during code inspection activities.

6. Guidelines for specifying, tracing, and measuring NFR

6.1. Guidelines for specifying and tracing NFR

Specifying NFR is basically identical to specifying all other requirements. Mostimportant is to get insight in customers’ priorities and let them specify NFR in moredetail. Current standards activities try to enforce clear specification of NFR includingmeasurable acceptance criteria. EURESCOM, ITU-T or IEC for example recommendthe purchaser to specify requirements for continuous system operation and operationalavailability performance including metrics for measuring fulfillment of such require-ments. The examples in section 7 however show that typically NFR are not evenly


Table 1Example specification of NFR.

Quality goals Priority Measurable targets Related quality and re-lease criteria

Metrics applicable fortracing

Correctness 2 E.g., 95% completelyfulfilled customers’functionalrequirements

E.g., traceability ofallocatedrequirements;consistency;completeness

E.g., percentage of re-quirements that havebeen satisfied duringdesign

Reliability 1 E.g., < 30 highpriority failures/monthafter handover

E.g., fault tolerance;consistency; accuracy;simplicity

E.g., 5 faults/KStmt atbegin system test; faulttolerance implemented

Maintainability 1 E.g., no module splitafter end of coding

E.g., consistency;readability; simplicity;modularity

E.g., depth of nesting< 5; module size < 10KStmt

...

distributed (i.e., focus on performance and reliability) and often fuzzy by nature (i.e.,customers request ‘maintainability’ or ‘low complexity’).

The following template (Table 1) is helpful for documentation of project qualitygoals and therefore including NFR in the overall requirements list. Some examples areprovided for clarity. Of course each single project needs a tailored list before technicalanalysis can start. Unfortunately most NFR are not measurable during design. Theyare even difficult to test; therefore they are usually evaluated subjectively. It is hencedifficult to trace them similarly to functional requirements. Therefore, measurabletargets and related quality and release criteria that are measurable must be providedfor each selected NFR.

We recommend the same approach for dealing with NFR as for all functionalrequirements during system development:

1. Collect and extract all user-oriented NFR according to contracts, proposals,required standards (e.g., EURESCOM, BELLCORE, ITU-T), and marketingstrategies.

2. Collect and extract all development-oriented NFR according to internal productor platform strategies.

3. Perform technical analysis of the given set of NFR in terms of impact, effort,and interactions. Make architectural trade-off explicit.

4. Perform commercial analysis of the NFR based on the technical analysis

5. Assign priorities to allow for conflict resolution.

6. Select those NFR with highest priority and agree on roadmap how to achievethese NFR realistically.

7. Define specific measures for in-process quality checks and risk assessment.


8. Establish continuous risk assessment for each milestone of the NFR roadmapand of the project’s masterplan.

9. Agree on a design scenario for covering these NFR. Due to the broad relation-ships with other requirements this starts already during architectural design.

10. Measure and verify intermediate goals according to the NFR roadmap.

Table 2 provides a list of general quality attributes (i.e., NFR) that might be rel-evant in a project. Some are typically clearly specified in the customer’s contract (e.g.,reliability, performance), while others are related to the development or maintenanceeffort (e.g., testability, maintainability).

6.2. Conflict resolution for interacting NFR

After having specified all NFR and knowing their relations with other NFR orfunctional requirements (e.g., by applying QFD), the next step is to analyze potential

Table 2Quality goals, definitions and examples.

Goals/NFR Definition ExampleCorrectness Extend to which the software satisfies its

specifications and the final product satisfiesits requirements

Interworking of ISDN servicefeatures

Reliability Extend to which the software can be ex-pected to perform its intended function withrequired precision

Critical failures per month

Performance Amount of resources needed by the soft-ware or the final product to provide fullfunctionality under all possible environmen-tal conditions

Call set up time

Availability Ratio of operation time or line access whenthe final product or its components providefull functionality

Supplier dependent total downtimeper year

Maintainability Effort required to locate and correct a faultor make a change in the software

Few ripple effects of changes

Testability Effort required to test the software, its com-ponents, or the final product to ensure theyperform their intended function

Functional tests are related to re-quirements and not to architecture

Reusability Extend to which software components canbe used in other applications without per-forming too many changes

Clearly defined and separated func-tionality of distinct components

Usability Effort required to learn, operate, prepare in-put and interpret output of the final product

Effort to set up new lines with spe-cific ISDN feature combination

Failuretolerance

Extend to which software components pro-vide continuous operation under nonnomi-nal conditions

Failure recovery upon restart

Extendibility Effort required to expand the software orthe final product to allow for enhanced datarequirements or additional features

Free memory space of modules


conflicts. For example, performance might interfere with maintainability. Considerfor instance the use of embedded Assembler commands. It is often impossible toachieve both goals since they are contradictory. The more low-level database accesses,the shorter are database access times, while at the same time understandability ofcode is reduced. It is thus necessary to prioritize such conflicting goals after havingthem identified. Conflicts typically escalate due to different goals of managementresponsibles that decide. Positions might even be unclear between the commerciallyresponsible program manager, his engineering counterpart and the quality deputy, ascan easily be imagined for the given example. The example finally shows that the twocontradictive NFR cannot be achieved by simply changing one module and adaptingtwo messages. Conflict resolution requests changing overall design guidelines!

Obviously coverage of NFR is costly. Hence the technical and commercialanalysis of given NFR, either user-oriented or not, must be taken seriously. Influenc-ing factors might include the architecture, coverage of the NFR by the current basearchitectures or preceding projects, and overall product goals. Redesign of an outdatedbase architecture might be out of question at the sake of reduced maintainability inorder to cover user-oriented NFR that are subject to contracts.

6.3. Guidelines for measuring NFR

Due to the lack of clearly defined metrics for requirements management, mea-suring NFR is primarily achieved by reviews along the development process (Fig. 3).This includes:

– the plan and effort estimation is periodically reviewed on the base of allocatedrequirements;

– the achievement of milestones and related work products is periodically re-viewed (including the question whether distinct efforts can be traced back toformally stated and agreed both internal or external customer requirements);

– work products are inspected to find out internal relationships, ensure consis-tency, and validate coverage of requirements.

Having a common set of measurable quality indicators is essential to gain the necessarymanagerial visibility of potential problems as the basis for effective decision taking andcorrective action planning. Quite obviously, measurement of the quality of the processof developing NFR conformance depends on the underlying processes. Controlling thedesign process in terms of achieving coverage of NFR includes instituting checklists,together with a program for completing them and an agreed procedure for acting onthe results. Metrics for NFR therefore are based on a goal-oriented approach. Thefirst step (that we already performed while condensing and specifying NFR) is to findout goals to be achieved in the requirement management process. Based on goalsthat need prioritization (which we cover with QFD and conflict resolution techniques)questions are derived that lead to metrics. Metrics that do not answer questions related


to achieving efficiency or quality targets are waste of effort and typically result in adata cemetery.

A checklist for measuring NFR is thus a list of properties of the software thattogether determine whether or how far the criteria have been met. An indicator isa number that shows changes in magnitude, usually of some aspect of a situationthat cannot be directly measured. Table 3 results from such a goal-oriented approachthat first concentrated on finding out process improvement goals (typically based onprocess assessments) for the requirements management process. The next step thenwas to break these goals down according to the specific environment. This stepwas separated into finding subgoals of the original goals (second column) and thento ask questions related to achieving these goals. Finally metrics had been selectedthat clearly answer these questions (third column). Obviously such metrics need notnecessarily be numbers with units. Often requirement metrics include checks and arethus on a nominal scale.

Collecting quality data over a long term period helps to provide historical datato improve estimating and planning of new projects in the future. For measuring theimpact of process improvement activities it is recommended to determine some keymetrics for a project that is currently (almost) finished and later compare those to thesame metrics gained from a project after improvement was started (benchmarking).

Individual metrics may also be compiled to form an overall quality index inorder to have a quick indication of the actual quality level.

7. Practical application

This section gives insight in how to deal with NFR, both customer-oriented anddevelopment-oriented. Customer-oriented NFR are based on current contracts. Mostdetailed customer requirements are in the area of performance. However, today eventypical development-related NFR are given interest by the customer due to overallquality goals. Exactly this development from performance requirements (time/spaceperformance) that are today taken for granted by the customer towards maintainabilityor extendibility indicate the mentioned paradigm change.

Due to size restrictions four applications in the areas of performance, usability,reliability, and maintainability are treated. The selection is based on those NFR withhighest overall priority (based on field failures and project design aspects). Severalpractical obstacles are included in order to shift focus towards conflicting NFR (e.g.,use of Assembler language).

7.1. Performance – traffic load requirements

Specification. Several customer requirements touch traffic load management andavailability under distinct traffic conditions. Examples are load-dependent traffic andsignaling routing, traffic capacity, traffic overload control actions, general delay times,call delay times, transmission time, switching delay times, and signaling duration.


Table 3Goal-oriented approach for selecting metrics for the requirements management process.

Goal Subgoal Questions/metricsDefine require-ment manage-ment process

Institutionalize CMM compli-ant requirements management

Does a policy exist?Is the policy deployed?Are all activities performed?

Monitor the cost/benefit ofthe requirements managementprocess

What is the effort expended in requirementsmanagement for new/changed requirements?What is the rework related to requirements?What is the effort spent in preparing and con-ducting reviews of new/changed requirements?Percentage of faults that could have been foundwith requirements reviews

Improvecurrent require-ments manage-ment process

Reduce late changes of require-ments during the developmentprocess

Percentage of requirements that changed duringdesign/laterAmount of rework due to requirements changesNumber of new requirements after designNumber of requirements that are not yet pre-cisely defined (tracing chart over developmentprocess)

Specify all outputs of the re-quirements definition process

Percentage of requirements allocated to a dis-tinct subsystem before architectural designPercentage of requirements with completedatabase entries (e.g., status, risk, effort, releaseallocation, work product allocation)Percentage of requirements allocated to features

Improve con-trol and trac-ing of softwarework products

Improve requirements tracingthrough the complete develop-ment process

Percentage of requirements that were subject ofdelivery and were incorrect/missingNumber of open requirementsTracing plot of closed requirements (percentagefulfilled) over life cyclePercentage of work product items that can betraced to the requirements

Constructive means:

– QFD (impact of performance requirements on design decisions; e.g., messages);

– insert tailored algorithm to handle congestion control;

– insert traffic reduction mechanisms at the originating location (e.g., locationpoint call rejections);

– insert mechanism to avoid that the same message with the same destination isrepeatedly sent on different circuits;

– reduce internal message broadcasts because their acknowledge messages mayblock networks;

– avoid unnecessary internally generated traffic (e.g., messages or database ac-cesses);


– clear modularity (i.e., functional separation and reduced distribution of relatedcommunicating modules over several processors);

– code inspections (e.g., reaction to overload, hard-coded limits for load, loadmanagement).

Analyses and metrics for tracing:

– design reviews according to QFD impacts;

– ensure that all reviews contain checks for overload control mechanisms (e.g.,no overflow in buffers, message discarding mechanisms, priority rules for dis-carding messages);

– apply performance models and simulation.

Qualification test and acceptance criteria:

– system testing according to operational profiles with typical load/usage profiles;

– system testing with automatic load machines.

7.2. Usability – documentation

Specification. Although documentation standards exist, customer requirements typ-ically do not cover them in depth. Such requirements include all documentation interms of contents (e.g., man-machine interface; technical and user documentation forall types of hardware and software components to be delivered; reliability specs; loadspecs; maintenance guidelines), formats (paper, software) and their distribution (e.g.,disks, CD-ROM, file transfer). Style guidelines and layout rules are typically specifiedby customers, especially in the case of on-line documentation.

Constructive means:

– user surveys to clarify imprecise customer requirements;

– providing style guidelines and layout rules that are followed for all types ofuser interfaces;

– training of man machine interface designers in standards, guidelines, engineer-ing tools, and appropriate prototyping techniques;

– prototyping of man machine interface and intensive user interaction for testingand improving;

– installation of usage lab for prototyping and early user interaction.


– ensure that requirements reviews concerning documentation check consistencyand clarity of original customer requirements (e.g., general rules for man ma-chine communication might be insufficient for preparing documents);


– during preparation of any documentation reviews should check against statedcustomer requirements (e.g., ergonomic issues, conformance to standards), read-ability (FOG index, length, number of links to other documents, depth of suchlinks), and internal design rules (e.g., length, layout rules);

– internal checks for clarity, consistency, correctness (in terms of spelling), andcompleteness (in terms of customer requirements) by independent readers;

– documentation quality, both on-line and on paper, can be tested best based ondistinct user functions to be performed (e.g., time to perform a distinct functionby a typical user; time to perform a new function by an untrained user).


– coverage of all available functions by on-line documentation;

– completeness of documents according to customer specs (contents, form, layout,media);

– consistency checks in terms of layout rules.

7.3. Reliability – failure rates during field operation

Specification. Failure rates during field operation are among those contracted re-quirements that are easy to measure, however difficult to design. Typically failurerates are specified in terms of priorities, overall failure numbers, system downtime,and correction time. Since customers realized that field reliability metrics are notenough, recent activities focus on standards on how to achieve improved reliability.This includes metrics, reliability growth models, monitoring of the development pro-cess, and reliability-oriented design approaches. Even a discipline called reliabilityengineering evolved based on designing reliable systems.

Constructive means:

– application of formal methods and cleanroom engineering;

– application of common design principles to ensure good designs in terms offault numbers (modularity, understandability, low complexity, no Assemblerlanguage and direct access to databases);

– redundancy of software components;

– distributed functions;

– existence of reliability-oriented test strategy (including black box functions andwhite box test coverage);

– root cause analysis of known faults in order to detect similar occasions andreasons for faults;

– impact analysis of faults in order to avoid that corrections are incomplete.



– design reviews;

– code inspections with checklists based on typical fault situations or critical areasin the software;

– metrics for code inspections according to efficiency and effectiveness;

– enforced reviews and testing of critical areas (in terms of complexity, formerfailures, expected fault density, individual change history, customer’s risk andoccurrence probability);

– measure the effort spent for analyses, reviews, and inspections and separateaccording to requirements to find out areas not sufficiently covered;

– test coverage metrics (e.g., C0 and C1);

– dynamic execution already applied during integration test;

– application of protocol testing machines to increase level of automatic testing;

– application of operational profiles/usage specifications from start of system test;

– proper fault reporting from start of code reading to detect fault-prone areas andcommon reasons of faults;

– reliability growth models for predictions of fault and failure rates.


– achieved test coverage according to defined release criteria for each phase start-ing with module test;

– predicted failure rates are below contracted rates (application of reliabilitygrowth models);

– no open blocking problems.

7.4. Maintainability – internal software changes

Specification. Recently customers focus on development related issues, such asmaintainability of software. In this context maintainability questions how easy andstraightforward it is to perform changes to existing software (i.e., corrective, cosmetic,and additive maintenance). Performing such changes requests good documentation ondifferent levels (design, interfaces, functional description, message sequences, databaseaccess, data usage, code), understandable code, and governed complexity (size, controlflow, data flow).

Constructive means:

– traceability of allocated requirements and features to software items;

– functional separation with clear interfaces between realized features (modular-ity);


– hierarchical structure in design architecture;

– structured techniques and modeling during design (design changes are includedin design descriptions; automatic code generation from design specification,CASE environment with connection to configuration management system);

– data modeling and normalization of relational databases;

– abstraction (interfaces etc.) to allow for interface changes without affectingother components;

– conformance to design guidelines and training in applying design guidelines;

– increased readability and understandability (no Assembler code, reduced nestingdepth, proper documentation);

– small interfaces and information hiding (limited number of messages per mod-ule);

– proper change management (i.e., no changes without documentation, traceabilityof changes through all related documents).


– completeness of design documents;

– number of traceable requirements;

– complexity metrics related to maintainability (e.g., depth of nesting, messagecomplexity);

– volume (e.g., module size);

– module interactions;

– subjective ratings of maintainability through design staff (clearly indicates po-tential problem areas);

– code density (relationship between code and documentation).


– availability of complete and consistent design documentation;

– controlled experiment to measure time to process change requests;

– controlled experiment to measure effort needed for installation and registrationof patches.

8. Lessons learned and future work

In order to enhance the overall quality of a product it is not enough only tofulfill functional requirements. The resulting switching system might work, whilebeing difficult to use (e.g., operation and maintenance activities) or while showingtoo many failures in a distinct time frame. Therefore, additional effort spent during


design for nonfunctional requirements is necessary to provide a software system thatis internally and externally fit for the future.

The key message of our work is that NFR are requirements too. Collection,specification, analysis, tender, allocation, design, and testing can be seen as similarto dealing with functional requirements. However, experience shows that special em-phasis must be given to each of them, and to assigning priorities in case of conflictinggoals (e.g., performance vs. maintainability). Achieving full completion of such gen-eral quality attributes is not the task of one single designer or a small team. It is thetask of all involved staff. Thus, project management must trace such requirementswith much more effort than functional requirements.

Dealing with NFR in a satisfactory way is not as easy to achieve as switching toanother test strategy. It involves all development processes. The following approachseems to provide both a stepwise and measurable action plan for building up thenecessary procedures:

1. Specify all NFR with sufficient detail covering customer priority, the marketevolution and internal importance to be future-safe. The specification shouldbe on the same level of detail as other requirements specs and must includemeasurable targets, potential ripple effects and constraints.

2. Analyze these NFR both from a commercial and technical viewpoint to beaware of costs for realization and costs of non-realization. The analysis mustinclude the set-up of risk management techniques because often the realizationsuccess of a distinct NFR cannot be determined based on past experiences.In several cases solutions are determined as early as during the analysis (e.g.,increased reliability can be achieved by fault tolerance or by improved testingstrategies). The analysis includes the selection of appropriate products wherethe techniques are introduced (while inspections for instance can be pilotedrather locally, redesign affects the complete product).

3. Select those NFR that have highest priority (see section 3). The decision shouldbe based on customer priorities (such as reliability or performance) and onproduct improvement goals (e.g., maintainability). Be aware of the trade-offbetween the selected NFR and those with lower priority. In many cases thisdecision is a business decision with market-impact.

4. Identify the necessary technical groups that must be involved in the realizationof the selected NFR. Unlike with functional requirements the work breakdownand allocation to distinct components comes rather late and after lengthy dis-cussions with interdisciplinary engineering teams. An outcome must be a list ofactivities to be followed and respective milestones that must be aligned with theprojects’ individual milestones. Often a medium-term roadmap for achievingdistinct NFR is related to several project plans in a row. The NFR milestonesmust be accompanied with relevant metrics of expected effects because other-wise the approach is too diffuse and designers end up improving locally withoutfollowing the overall improvement targets.


5. Identify the relevant techniques and procedure changes to trace the selectedNFR to closure. Often new techniques must be established (e.g., QFD, de-sign techniques, simulation tools), existing procedures must be changed (e.g.,new checklists for inspections), existing components must be redesigned (e.g.,unreadable modules) or even architectures are impacted (e.g., reducing commu-nication overhead).

6. Measure the evolution of the selected NFR immediately, even if most metricsreflect only the process of dealing with NFR and not the achievement per se.This helps in finding out when senior management must be involved to decide.If effort metrics are included it must be clear that NFR do not allow a short-termROI calculation.

7. Regular management reviews of the achievements related to one NFR ensureattention and contribute to risk mitigation. Milestone reviews of the NFRroadmap should determine the effects of what has been done so far. Ratherobvious effects are increasing awareness of the importance of NFR duringdesign and their measurement in late test phases.

Further research in the area of NFR should focus on the areas:

– Tools for management of nonfunctional requirements beyond only stating thatthe customer wants maintainability or to achieve a distinct reliability limit.Such tools must allow for breaking down especially NFR to clear requirementsfor each single designer. This implies in many cases design guidelines andchecklists as mentioned in section 7 that could be included in a QFD-like tool.

– Architectures that are tailored to fit for specific NFR. In the age of highspeedcomputing and distributed real-time databases this focus should be easier toachieve than in the past when most concerns were related to achieving perfor-mance. Similar to tools requests these architectures must emphasize on showingtrade-off of design decisions in terms of NFR, show how design decisions haveimpact on each other and on NFR, and should be scaleable in a sense thatmarket requirements and market volumes increasingly tend to vary.

Acknowledgements

We want to thank the requirements management group of Alcatel Telecom’sSwitching Systems Division for many inputs for the underlying guidelines that arepartly reused for this article. The SQA group receives special thanks for condensingthe different customer standards to the synopsis that is partly reused in the appendix.


Appendix. A checklist for requirements management

Standard statement References ChecksContractingThe supplier has a complete,unambiguous set of func-tional requirements.

CMM: SPE-Ac-2 ISO9000–3: 5.3.1

Is a complete requirements listwith purchaser’s descriptions ofthe requirements available? Thislist should cover functional re-quirements, nonfunctional require-ments, and nontechnical require-ments, such as delivery date.Check for a product strategy re-sponsible to assign requirements toproducts.

The requirements includeall aspects to satisfy the pur-chaser’s needs together withacceptance criteria (i.e., de-sign constraints, such asmodule size or complexitylimits; and nonfunctional re-quirements, such as safety,security, maintainability, re-liability, or operational effi-ciency).

CMM: SQM-Ac-3 ISO9000–3: 5.2.2, 5.3.1 seealso ISO 9126

Check whether requirements list in-cludes nonfunctional requirementswith given acceptance criteria (ifcustomer specifies such). Checkwhether requirements are givenwith acceptance criteria or mea-surements.

Requirements specificationsshould include accep-tance criteria, standards tobe used, and requirementschange handling. This in-cludes subcontractors.

CMM: RM-Ac-1 ISO9000–3: 5.2.2, 5.8.1,6.7 BELLCORE: GR-929-CORE (3.1.3 fea-ture conformance), TR-NWT-179 (3.8 Accep-tance, 4.7 Purchasing)

Check whether requirements in-cluding those for subcontractorsare available together with accep-tance criteria. Check that a proce-dure for requirements change han-dling is available.

Each requirement is definedso that its achievement canbe verified (i.e., require-ments should be measurableand should include qualifi-cation requirements and ac-ceptance criteria).

CMM: RM-Ac-1,SPE-Ac-2 ISO 9000–3:5.2.2, 5.4.4, 5.8.1BELLCORE:GR-929-CORE (3.1.3feature conformance)

Do requirements specs reflect pre-cise and measurable contents? seealso section ‘metrics’ in table.

Each contract is reviewedby the supplier to ensurethat the scope of the con-tract and the requirementsare defined.

ISO 9000–3: 5.2.1,5.2.2 BELLCORE: GR-929-CORE (3.1.3 fea-ture conformance), TR-NWT-179 (3.8 Accep-tance)

Check planning whether these re-views are defined and performed.


Joint reviews between pur-chaser and supplier are es-tablished in order to con-form to agreed require-ments.

CMM: SPE-Ac-2 ISO9000–3: 4.1.3, 5.3.2

Check whether these reviews areplanned and performed before con-tract is signed.

Points in the life cycleare defined at which for-mal agreements for require-ments must be available tothe purchaser. A proce-dure for not feasible re-quirements is established.

ISO 9000–3: 4.1.3,5.3.2

Check planning whether milestonesand responsibles are defined andthe procedure for not feasible re-quirements is documented and putto practice.

Activities for specifying andchange management of pur-chaser’s requirements arecontracted.

ISO 9000–3: 5.2.2 Check for an activity to formallyagree about requirements changes.

Requirements allocationFor each project, responsi-bility is established for ana-lyzing system requirementsand allocating them to hard-ware, software, and othersystem components.

CMM: RM-Ab-1 Check for a systems engineeringresponsible that analyzes and allo-cates requirements; check projectplan for activity.

All allocated requirementsare documented (this in-cludes acceptance criteriaand nontechnical require-ments).

CMM: RM-Co-1, RM-Ab-2

Look for a requirements list withdescription of requirements and re-lationships to components. Cross-check requirements with accep-tance criteria.

Software Requirements aredeveloped, maintained, doc-umented, and verified bysystematically analyzing theallocated requirements ac-cording to the project’s de-fined software process.

CMM: SPE-Ac-2, SPE-V-3

Look for a defined and plannedprocess for requirements analysis.

Allocated requirements arereviewed by SW managersand other affected groups(e.g., system test, SW en-gineering, SQA). Reviewsinclude feasibility, consis-tency, clarity, and testability.

CMM: RM-Co-1, RM-Ac-1, SPP-Ac-6, SPE-Ac-2

Look for a defined and plannedprocedure that describes reviewof allocated requirements so thatrequirements are understood andcommitted by mentioned parties.


All requirements review re-sults are recorded, analyzed,and tracked to completion.

CMM: RM-Ac-3.2,ISO 9000–3: 5.2.1,5.4.6 BELLCORE:TR-NWT-179 (3.3Purchaser’sRequirementsSpecification)

Look for a procedure to track de-viations to completion.

Technical specifications areavailable for each version ofa software item.

CMM: RM-Ac-2 ISO9000–3: 5.3.1, 6.1.3.1

Look for documentation or changerequests for each single softwarecomponents’ versions.

Requirements tracingThe output of each devel-opment phase is verified tomeet all relevant require-ments (incl. nonfunctionalor regulatory requirements).This includes requirementsoriented testing proceduresat every design level.

CMM: SPE-Ac-2...7ISO 9000–3: 5.4.5,5.4.6 BELLCORE:TR-NWT-179 (3.7Testing and Validation)

Check for documented review pro-cedures that are part of projectplan; check especially the aspectof requirements tracing in the re-quirements list. Check if traceabil-ity procedures are applied to assureeach development. phase verifiesall the requirements.

Consistency is maintainedacross software work prod-ucts, including (the softwareplan, process descriptions,allocated requirements, soft-ware requirements, soft-ware design, code), testplans, and test procedures.The supplier shall demon-strate the tools or doc-umented procedures usedto assure the traceabilityof all applicable purchaserand supplier requirements toplanned tests.

CMM: SPE-Ac-10BELLCORE:TR-NWT-179 (3.7.2.7)

Check if traceability procedureassures the consistency of testspecs versus allocated require-ments; check if tools used to sup-port traceability procedure assuresthe consistency of test specs versusallocated requirements.

System and acceptance test-ing of the software areplanned and performed todemonstrate that the soft-ware satisfies its require-ments.

CMM: SPE-Ac-7 ISO9000–3: 5.2.2BELLCORE:TR-NWT-179 (3.8Acceptance)

Do system test cases focus on re-quirements and their correlations?Is regression testing planned thatcovers selection of critical require-ments?


Change managementThe impact of require-ments changes to existingcommitments is assessedand changes are negotiatedwithin the suppliers organi-zation.

CMM: RM-Ac-3 ISO9000–3: 4.1.3, 5.2.2

Check for responsible for require-ments change management.

Changes to allocated re-quirements are reviewedand incorporated into theSW project. This includesnegotiations with all af-fected groups and seniormanagement reviews.

CMM: RM-Ac-3,SPT-Ac-3, SPT-Ac-4,SPT-Ac-5...8,SPP-Ac-4BELLCORE:TR-NWT-179 (3.3Purchaser’sRequirementsSpecification)

Check for responsibles for commu-nicating requirements changes, fortriggering internal reviews of ef-fects on products and planning, forachieving top management commit-ment, and for tracking them toclosure; check for procedures tonegotiate and review requirementschanges.

If allocated requirementschange, all related SWplans, work products, andactivities will be changed tobe consistent.

CMM: RM-Co-1, SPT-Ac-5...8 ISO 9000–3:5.4.1

Check for procedure to trace re-quirements changes through theproject.

Requirements managementThe SW development planis based on and conforms tothe allocated requirements.This includes work productsand activities.

CMM: SPP-Ac-6,CMM: RM-Ac-2

Check for requirements orientedplanning, e.g., based on require-ments list with schedules and effort.

The allocated requirementsare managed and controlled(i.e., configuration manage-ment).

CMM: RM-Ac-2 ISO9000–3: 5.3.1, 6.1.3.1BELLCORE: TR-NWT-179 (4.1.3 Con-figuration ManagementActivities)

Check for configuration manage-ment system for requirements.

Activities for managing al-located requirements are re-viewed with senior man-agement on a periodic ba-sis (i.e., technical, sched-ule, cost and staffing perfor-mance).

CMM: RM-V-1,SPT-V-1 ISO 9000–3:4.1.1.3

Check for procedures to regularlyreview processes related to re-quirements management with se-nior management.

Activities for managing al-located requirements are re-viewed with the projectmanager on a periodic andevent-driven basis.

CMM: RM-V-2,SPT-V-2 ISO 9000–3:5.4.2.2

Check for procedures to review re-quirements tracing, schedule andperformance with project manage-ment.


Adequate resources andfunding are provided formanaging the allocatedrequirements (i.e., staff,tools).

CMM: RM-Ab-3,SPE-Ab-1 ISO 9000–3:4.1.1.2.1, 5.3.2, 5.4.2.3

Does the project plan reflect re-source requests necessary for re-quirements reviews and tracing?Are there suitable tools in use forrequirements management?

SQAThe SQA group reviewsand audits both work prod-ucts (e.g., allocated require-ments) and activities formanaging allocated require-ments (e.g., negotiation pro-cesses).

CMM: RM-V-3, SQA-Ac-4..5, SQA-Ac-8,SQM-Ac-1 ISO 9000–3: 4.3 BELLCORE:TA-NWT-1315 (5.2Quality Audits Results)TR-NWT-179 (3.3 Pur-chaser’s RequirementsSpecification)

Is there a responsible with suffi-cient resources in the SQA groupto perform reviews and audits?

MetricsMeasurements are made andused to determine the sta-tus of the activities for man-aging the allocated require-ments.

CMM: RM-M-1, SPE-M-1..2 ISO 9000–3:5.4.3 BELLCORE: TA-NWT-1315 (3.7 IPQMDeliverable Complete-ness Metrics)

Check for metrics related to re-quirements status? Are these met-rics used within project manage-ment? Are these metrics used forsenior management review?

Process effectiveness ismeasured by requirementstraceability.

BELLCORE:TA-NWT-1315 (4.2RequirementsTraceability)

Check for management reviewsthat question requirements trace-ability.

TrainingSW engineering staff istrained to perform their re-quirements management ac-tivities.

CMM: RM-Ab-4, SPE-Ab-2

Check for specific training forrequirements management, e.g.,methods for requirements analysis,tools for requirements tracing, orrequirements oriented tests. Checkif sales and engineering have beentrained in estimation and planningtechniques.

References

Brown, P. (1991), “QFD: Echoing the Voice of the Customer,” AT&T Technical Journal 70, 2, 18–32.Costello, R.J. and D.-B. Liu (1995), “Metrics for Requirements Engineering,” Journal of Systems and

Software 29, 1, 39–63.Davis, A.M. (1993), Software Requirements – Objects, Functions & States, Prentice Hall, Englewood

Cliffs, NJ.


Dorfman, M. and R.H. Thayer (1990), Standards, Guidelines, and Examples on System and SoftwareRequirements Engineering, IEEE Computer Society Press, Los Alamitos, CA.

Dromey, R.G. (1996), “Concerning the Chimera,” IEEE Software 13, 1, 33–43.Ebert, C. and T. Liedtke (1995), “An Integrated Approach for Criticality Prediction,” In Proceedings of

the. 6th International Symposium on Software Reliability Engineering (ISSRE’95), IEEE ComputerSociety Press, Los Alamitos, CA, pp. 14–23.

Gause, D.C. and G.M. Weinberg (1989), Exploring Requirements: Quality before Design, Dorset House,New York, NY.

Glossbrenner, K.C. (1993), “Availability and Reliability of Switched Services,” IEEE CommunicationsMagazine 31, 6, 28–32.

Karmarkar, V.V. (1994), “Assuring SS7 Dependability: A Robustness Characterization of Signaling Net-work Elements,” IEEE Journal on Selected Areas in Communications 12, 3, 475–489.

Kazman, R. and L. Bass (1994), Toward Deriving Software Architectures from Quality Attributes,CMU/SEI-94-TR-10, Software Engineering Institute, Pittsburgh, PA.

Linger, R.C. (1994), “Cleanroom Process Model,” IEEE Software 11, 2, 50–58.McDonald, J.C. (1994), “Public Network Integrity – Avoiding a Crisis in Trust,” IEEE Journal on Selected

Areas in Communications 12, 1, 5–12.Myopoulos, J., L. Chung, and B. Nixon (1992), “Representing and Using Nonfunctional Requirements:

A Process-Oriented Approach,” IEEE Transactions on Software Engineering 18, 6, 483–497.Ohmori, A. (1993), “Software Quality Deployment Approach: Framework Design, Methodology and

Example,” Software Quality Journal 3, 209–240.Porter, A.A., L.G. Votta, and V.R. Basili (1995), “Comparing Detection Methods for Software Require-

ments Inspections: A Replicated Experiment,” IEEE Transactions on Software Engineering 21, 6,563–575.

Documents

Dealing with nonfunctional requirements in large software systems