Week #7

Analysis of Requirements Documents

What Is Requirements Analysis?

● Validation– Stakeholders agree that the requirements

document matches their needs/wants● Consistency

– Different parts of the document 'line up' with one another; a program can be written that implements every part completely

● Traceability– Backward: any part of the document can be

traced back to its source– Forward: any part can be matched to design

elements

Plan for the Lecture

● 'Classical' view of analysis– We have a complete document and no code– Survey of both informal and formal approaches

to analysis● 'Agile' view

– Document and design are constructed together● Simple prototypes test for validity● More complete prototypes check consistency● Traceability is a map into the code-base

Validation

● Inspection of document● Facilitated by standard, hierarchical

document structure– No 'surprises'– No large chunks of text to digest and try to place

in context● For example: IEEE requirements

documentation standard

Desiderata for Specifications

● Valid (or “correct”)– Expresses only the real needs of the

stakeholders (customers, users,…)– Doesn’t contain anything that isn’t “required”

● Unambiguous– Every statement can be read in exactly one

way

● Complete– Specifies all the things the system must do– ...and all the things it must not do!– Conceptual Completeness

E.g. responses to all classes of input– Structural Completeness

E.g. no TBDs!!!

● Understandable (Clear)– E.g. by non-computer specialists

● Consistent– Doesn’t contradict itself I.e. is satisfiable– Uses all terms consistently

● Ranked – Must indicate the importance and/or stability

of each requirement● Verifiable

– A process exists to test satisfaction of each requirement

– “every requirement is specified behaviorally”● Modifiable

– Can be changed without difficulty Good structure and cross-referencing● Traceable

– Origin of each requirement must be clear– Facilitates referencing of requirements in

future documentation

Source: Adapted from IEEE-STD-830-1998

Source: Adapted from Kovitz, 1999

Typical mistakes

Noise the presence of text that carries no relevant information

to any feature of the problem. Silence

a feature that is not covered by any text. Over-specification

text that describes a feature of the solution, rather than the problem.

Contradiction text that defines a single feature in a number of

incompatible ways. Ambiguity

text that can be interpreted in at least two different ways.

Forward reference text that refers to a feature yet to be defined.

Wishful thinking text that defines a feature that cannot possibly be

validated.

Jigsaw puzzles e.g. distributing requirements across a document and

then cross-referencing Duckspeak requirements

Requirements that are only there to conform to standards

Unnecessary invention of terminology E.g., ‘the user input presentation function’, ‘airplane

reservation data validation function’ Inconsistent terminology

Inventing and then changing terminology Putting the onus on the development staff

i.e. making the reader work hard to decipher the intent Writing for the hostile reader

There are fewer of these than friendly readers

Ambiguity Test● Natural Language

– 'The system shall report to the operator all faults that originate in critical functions or that occur during execution of a critical sequence and for which there is no fault recovery response.'(adapted from the specifications for the international

space station)● Or a decision table?

Originate in critical functions F T F T F T F T

Occur during critical seqeunce F F T T F F T T

No fault recovery response F F F F T T T T

Report to operator?

Source: Adapted from Easterbrook & Callahan, 1997.

Avoiding ambiguity

● Review natural language specs for ambiguity– use people with different backgrounds– include software people, domain specialists and user

communities– Must be an independent review (I.e. not by the

authors!)● Use a specification language

– E.g. a restricted subset or stylized English– E.g. a semi-formal notation (graphical, tabular, etc)– E.g. a formal specification language (e.g. Z, VDM,

SCR, …)

Avoiding Ambiguity, II

● Exploit redundancy– Restate a requirement to help the reader confirm

her understanding– ...but clearly indicate the redundancy– May want to use a more formal notation for the

re-statement

Organizing the Requirements

● Need a logical organization for the document– IEEE standard offers different templates

● Example Structures - organize by…– …External stimulus or external situation

● e.g., for an aircraft landing system, each different type of landing situation: wind gusts, no fuel, short runway, etc

– …System feature● e.g., for a telephone system: call forwarding, call blocking,

conference call, etc– …Subsystem

● e.g. for spacecraft: command&control, data handling, comms, instruments, etc.

Organizing Requirements, II

– …System response● e.g., for a payroll system: generate pay-cheques,

report costs, print tax info;– …User type

● e.g. for a project support system: manager, technical staff, administrator, etc.

– …External object● e.g. for a library information system, organize by book

type– …Mode

● e.g. for word processor: page layout mode, outline mode, text editing mode, etc

IEEE Standard for SRS1 Introduction

PurposeScopeDefinitions, acronyms, abbreviationsReference documentsOverview

2 Overall DescriptionProduct perspectiveProduct functionsUser characteristicsConstraintsAssumptions and Dependencies

3 Specific Requirements

Appendices

Index

1 IntroductionPurposeScopeDefinitions, acronyms, abbreviationsReference documentsOverview

2 Overall DescriptionProduct perspectiveProduct functionsUser characteristicsConstraintsAssumptions and Dependencies

3 Specific Requirements

Appendices

Index

Identifies the product, & application domain

Describes contents and structure of the remainder of

the SRS

Describes all external interfaces: system, user, hardware, software;

also operations and site adaptation, and hardware

constraints

Summary of major functions

Anything that will limit the developer’s options (e.g.

regulations, reliability, criticality, hardware limitations, parallelism,

etc)

All the requirements go in here (i.e. this is the body of the document).

IEEE STD provides 8 different templates for this section

Source: Adapted from IEEE-STD-830-1993 See also, Blum 1992, p160

IEEE STD Section 3 (example)

3.1 External Interface Requirements3.1.1 User Interfaces3.1.2 Hardware Interfaces3.1.3 Software Interfaces3.1.4 Communication Interfaces

3.2 Functional Requirementsthis section organized by mode, user class,

feature, etc. For example:3.2.1 Mode 1

3.2.1.1 Functional Requirement 1.1…

3.2.2 Mode 23.2.1.1 Functional Requirement 1.1…

...3.2.2 Mode n

...

3.3 Performance RequirementsRemember to state this in measurable terms!

3.4 Design Constraints3.4.1 Standards compliance3.4.2 Hardware limitationsetc.

3.5 Software System Attributes3.5.1 Reliability3.5.2 Availability3.5.3 Security3.5.4 Maintainability3.5.5 Portability

3.6 Other Requirements

Source: Adapted from IEEE-STD-830-1993. See also, Blum 1992, p160

Source: Adapted from Blum, 1992, pp369-373

Reviews, Inspections, Walkthroughs…

● Note: these terms are not widely agreed– formality

● informal: from meetings over coffee, to team get-togethers● formal: scheduled meetings, prepared participants, defined

agenda, specific format, documented output– Management reviews

● E.g. preliminary design review (PDR), critical design review (CDR)

● Used to provide confidence that the design is sound● Attended by management and sponsors (customers)● Usually a 'dog-and-pony show'

Reviews, Inspections, Walkthroughs II

● Walkthroughs● developer technique (usually informal)● used by development teams to improve quality of

product● focus is on finding defects

– (Fagan) Inspections● a process management tool (always formal)● used to improve quality of the development process● collect defect data to analyze the quality of the

process● written output is important● major role in training junior staff and transferring

expertise

Source: Adapted from Blum, 1992, pp369-373 & Freedman and Weinberg, 1990.

Benefits of formal inspection

● Formal inspection works well for programming:– For applications programming:

● more effective than testing● most reviewed programs run correctly first time● compare: 10-50 attempts for test/debug approach

– Data from large projects● error reduction by a factor of 5; (10 in some reported

cases)● improvement in productivity: 14% to 25%● percentage of errors found by inspection: 58% to 82%● cost reduction of 50%-80% for V&V (even including cost of

inspection)

Inspection Benefits, II

– Effects on staff competence:● increased morale, reduced turnover● better estimation and scheduling (more knowledge

about defect profiles)● better management recognition of staff ability

● These benefits also apply to requirements inspections– E.g. See studies by Porter et. al.; Regnell et. al.

Source: Adapted from Blum, 1992, pp369-373 & Freedman and Weinberg, 1990.

Inspection Constraints

● Size– “enough people so that all the relevant

expertise is available”– min: 3 (4 if author is present)– max: 7 (less if leader is inexperienced)

● Duration– never more than 2 hours

concentration will flag if longer

● Outputs– all reviewers must agree on the result

accept; re-work; re-inspect; – all findings should be documented

summary report (for management)detailed list of issues

● Scope– focus on small part of a design, not the

whole thing

● Timing– Examines a product once its author has

finished it– not too soon

product not ready - find problems the author is already aware of

– not too lateproduct in use - errors are now very costly to fix

● Purpose– Remember the biggest gains come from

fixing the processcollect data to help you not to make the same errors next time

Source: Adapted from Freedman and Weinberg, 1990.

Inspection Guidelines

● Prior to the review– schedule Formal Reviews into the project planning– train all reviewers– ensure all attendees prepare in advance

● During the review– review the product, not its author

● keep comments constructive, professional and task-focussed

– stick to the agenda● leader must prevent drift

– limit debate and rebuttal● record issues for later discussion/resolution

Inspection Guidelines, II

– identify problems but don't try to solve them– take written notes

● After the review– review the review process

Choosing Reviewers

● Possibilities– specialists in reviewing (e.g. QA people)– people from the same team as the author– people invited for specialist expertise– people with an interest in the product– visitors who have something to contribute– people from other parts of the organization

Source: Adapted from Freedman and Weinberg, 1990.

Choosing Reviewers, II

● Exclude– anyone responsible for reviewing the author

● i.e. line manager, appraiser, etc.– anyone with known personality clashes with

other reviewers– anyone who is not qualified to contribute– all management– anyone whose presence creates a conflict of

interest

Source: Adapted from Porter, Votta and Basili, 1995

Structuring the inspection

● Can structure the review in different ways– Ad-hoc

● Rely on expertise of the reviewers– Checklist

● uses a checklist of questions/issues● checklists tailored to the kind of document (Porter et. al.

have examples)– active reviews (perspective based reading)

● each reviewer reads for a specific purpose, using specialized questionnaires

● effectively different reviewers take different perspectives

Structuring the Inspection, II

● The differences may matter– E.g. Porter et. al. study indicates that:

● active reviews find more faults than ad hoc or checklist methods

● no effective different between ad hoc and checklist methods

● the inspection meeting might be superfluous!

Agile Validation

● Use-case-wise– Elicit a use-case (including exceptional

behaviour)– Prototype it– Validate the prototype– Increment the prototype with the next use-case

● Validate the new use-case● Check for interactions with already-validated ones

– Check NFRs at each cycle● Aspect-wise

Kinds of Consistency

● Between parts of the same kind of representation (in-viewpoint)– Say constraints in the object model..– All pairs of constraints must be mutually

satisfiable● Between different kinds of representation

(inter-viewpoint)– Statechart and object model– Use-case and statechart– ...

In-Viewpoint Consistency

● How prone is a representation to introduction of accidental inconsistency?– Natural language: extremely so– Single simple diagram: mostly much less– Set of simple diagrams, hierarchically structured

(like DFDs, Statecharts): more than a single diagram

– Set of non-hierarchically structured diagrams (such as MSCs): more than hierarchical diagrams

– Logical theory (such as OCL): fairly prone; but automated detection is facilitated

Inter-Viewpoint Consistency (informally)

● Consider 3D models as an example● How to represent a cube in 2D:

Top ViewPerspective View

Inter-Viewpoint Consistency (formally)

● Rely on semantics, not intuition● Consider views of the cube again

– Each has a semantic domain: P( )– A complete description picks out a single

element of the semantic domain– A viewpoint is not complete (in general): many

elements of the domain may be consistent with it● The perspective could mean a complete cube● .. or just the three faces visible with nothing behind!

3

UML Examples

● Statechart/Object model– Domain: states of each object, events (source,

target, parameters)● Each state in an object's statechart must

– Be describable by a predicate over the object's state-space– If it produces events, each must be an operation of either

that object, or one it is associated with● If s is an OR-substate of t, then the predicate

describing s must imply the predicate describing t● ...

UML Examples, II

● Use-case/message sequence chart– Domain: finite sequences of messages and

responses● Types must match● Creation events must match● Step in UC must represent fragment of MSC● If either the UC or MSC is more detailed, then they

must agree on the common events

Elements of Formal Consistency

● What do we need?– A clearly-defined semantic domain

● Does not need to be the most detailed one; the richer, the harder to check consistency

● Should also be an abstraction of the concrete (program) domain

– A mapping from each notation into the semantic domain

● May be imprecise (one diagram = many elements of semantic domain) and probably will be

– Consistency rules● That guarantee (or at least strongly suggest) that the

diagrams are consistent, without building big state-machines

Semantic Domains (From Concrete to Abstract)

● Concrete program state-space– All possible assignments to global variables– All possible heap configurations– All possible program locations (one location per

thread)● Behaviours are possible sequences of program states● If multi-threaded, thread scheduling must be modelled

explicitly● Statechart state-space

– All possible activations– All possible triggered events

● Behaviours are possible sequences of statechart states

Semantic Domains, II

● Object model state-space (constraints)– All possible instantiations of objects– All possible assignments to attributes of each

object (possible = allowed by constraints)– All possible values of associations– Events being sent between compatible objects

● Behaviours are all sequences of states allowed by pre/postcondition constraints

● Object model state-space (no constraints)– Same but less restricted– Behaviours are all type-correct sequences of

attributes and messages

Handling Inconsistency

● Most commonly inconsistency is just a memory lapse– Event in object model returns type A– Statechart assumes type B (not a

superclass/subclass of A)– Analyst forgot the type of event, picked one that

seemed right● ..but it can be a sign of requirements needing

further capture– Need a common superclass of A and B; they

have something in common in the domain model

Backward Traceability

● Each requirement must be justified– Belongs to at least one stakeholder– Can be explained in terms of goals– Does not conflict with any others

Forward Traceability

● Each requirement must map into the program– What is a 'single requirement'?

● Paragraph of text?● Diagram

– Functional requirements map to modules of the program

– Non-functional requirements do not map so well..● Aspect-oriented programming attempts to mitigate

this● Good commenting, and use of relationships between

quality factors and design decisions helps too

Forward Traceability Questions

● Does every requirement correspond to some element of the program?

● (More difficult..) Does every design decision/module/line of code trace back to one or more requirements?

Conflicts In General

● Requirements R, S are inconsistent when satisfying R makes S impossible to satisfy

● Assuming satisfaction is binary● If satisfaction is measurable (such as with

performance criteria), other relationships exist– Positive correlation (++, +)– Negative correlation (--, -)– Unspecified correlation (relationship

undetermined)– Definitely no correlation

Example of Conflict

● Requirement: Quick server response– “File server shall respond to requests within

0.05s”● Requirement: Serve 200 connections

– “File server shall serve up to 200 connections simultaneously”

● Consistent, but under some circumstances may be impossible to satisfy both– These circumstances are called obstacles

Correlation Types

● Structural– R is similar to S

● Resource– Both depend on the same resource

● Task– R describes a dependent task of S

● Causality– R describes a consequence of S

Semantic Conflicts

● Perception– High level goals of agents conflict– Agents' assumptions conflict– Agents have different views of domain boundary

● Communication– Agents use different levels of abstraction– Agents' positions have different levels of

precision– One agent's justification appears circular

● Resource/behaviour

Frameworks for Negotiation

● What is required for negotiation?– Documentation on some level of initial

approximation to requirements, structured to allow browsing and analysis

– Ways for stakeholders to identify issues with a given requirement of another stakeholder

– Ways to generate possible resolution of issues (options)

– Documentation of agreement– Historical view: agreement on an option does not

make the issue disappear, simply resolves it

WinWin: an Example Framework

● Developed by Barry Boehm out of decades of research and practice

● Basic ideas:– Pragmatic approach: use what works and is

readily comprehensible– Cast negotiation as a non-zero-sum game: the

goal is to find conditions under which every stakeholder wins

– The process cannot completely enforce this, but should encourage it as much as possible

● Empirical studies support this hypothesis

WinWin Activities

● Review/expand negotiation topics– Develop a structure and taxonomy of

requirements– IEEE templates can be a guide here

● Brainstorm stakeholder interests– Brainstorming is structured via software as

multiple discussion threads, started by facilitator– Users are periodically switched to another

discussion thread to keep from spending too much time on one topic

– Threads are keyword-indexed and linked in real time

WinWin Activities, II

● Converge on win conditions– Each stakeholder prompted to fix one (or a small

number of) win conditions as a basis for negotiation

– Each win condition must be accepted as such by all stakeholders: this is just agreeing that they understand it and believe it makes sense, not that they accept it as a requirement!

– No contesting or endorsement of win conditions at this stage

WinWin Activities, III

● Capture a glossary of terms– Familiar activity from elicitation..

● Prioritize win conditions– Evaluate each for business importance and

feasibility– Results in 4 broad categories: 'low hanging fruit',

'important, with hurdles', 'maybe later', 'forget it'● Determine conflicts

– Stakeholders identify and explain disagreements– Explanation may uncover ambiguity: not a 'real'

disagreement

WinWin Activities IV

● Identify issues and options– Prioritize issues by contentiousness

● How many different requirements are involved?● How severe is the conflict, and of what type?

– Identifying options is the hardest task!– Choosing the best option is second hardest..

● Reach agreements on options– If option T resolves an issue between R and S,

then the agreement between stakeholders that it replaces the old R and S is recorded

– Record of this agreement is part of traceability

Searching for Resolution

● Relaxation– Stakeholders loosen their requirements to find

mutually-satisfactory option● Refinement

– Conflicting requirements are split into sub-goals, only some of which are in conflict

● Restructuring– Instead of changing the requirements, can some

change to the problem domain be specified?– Or, can the requirements be limited to a specific

context where no obstacles exist?

More Resolutions

● Abandonment● Postponement● Creative compromise by facilitator

Decision Theory

● Alternatives are ordered by a preference relation

● Preference relation translates to a normalized utility function assigning a value between 0 and 1 to each alternative

● If there are multiple criteria for evaluation, each has a preference relation (and corresponding utility function)

● A social choice or preference aggregation function combines the criteria

Decision Theory: Pro

● In small domains, enables automated search for options

● Clearly impartial and authoritative● Requires clear prioritization and

foregrounding of decision criteria

Decision Theory: Con

● Eliciting preferences from stakeholders is time-consuming

● Constraint satisfaction in the software domain is not, in general, computationally feasible

● Arrow's Theorem: finding a preference aggregation that is simultaneously fair and democratic is impossible