Department of Telecommunication SystemsSoftware Engineering Research Group

Master’s ThesisJune 2004

AuthorsCarl EkdahlAndreas Ekvall

SupervisorsLena KarlssonBjörn Regnell

Guidelines for Acquisition of RE-tools

ii

Acknowledgements

We would like to thank all the people involved in making this thesis.

The people in the interviewed organizations, who have contributed with their valuable time to answer our questions about requirements engineering and RE-tools.

The RE-tool vendors, Telelogic (DOORS), IBM (RequisitePro), Nohau (CaliberRM) and Focal Point AB (Focal Point) who also have taken time to answer our questions.

Raimundas Matulevicius for his advice and knowledge.

Anders Bruce for his technical support and maintenance.

And finally, our supervisors, Björn Regnell and especially Lena Karlsson for all support.

iii

iv

Abstract

Requirements engineering is a major part of the software process and it is important that organizations do not neglect it. As more organizations realize why it is so important to write the requirements in an understandable and unambiguous way, the better chance the project has to succeed. A number of requirements engineering tools (RE-tools) exist on the market that could be beneficial to an organization when managing the requirements. It could be difficult for an organization to choose a suitable RE-tool because there are many tools with different functions and the focus of this thesis is therefore to help organizations with guidelines of what to think about when acquiring a RE-tool.

In order to suggest these guidelines, we would like to see this from the organizations’ point of view. Therefore we decided to base the method on interviews with a number of organizations of different size, age and business areas and the results are summarized in the GRTS-method, which in a number of steps will try to help organizations in choosing a suitable RE-tool.

The GRTS-method states that it is important to evaluate the current requirements process and organization before deciding to buy an RE-tool. If an organization has decided to buy an RE-tool, it should first define the criteria the RE-tool should support. After that information should be gathered about the RE-tools and a screening process should be conducted to narrow down the selection. Finally a more thorough evaluation of these RE-tools should be conducted to make the choice.

Some conclusions are that organizations want the tool vendors to focus on usability rather than functionality, integration between the tool and other software is important and that organizations want more comprehensive tools that focus on “the big picture”, i.e. all parts of the project. This is where future work should be.

v

vi

Table of contents

1 INTRODUCTION.........................................................................................................................................1

1.1 PURPOSE OF THE THESIS..............................................................................................11.2 STRUCTURE OF THE REPORT........................................................................................1

2 SOFTWARE ENGINEERING....................................................................................................................3

2.1 THE SOFTWARE ENGINEERING PROCESS......................................................................32.2 EXAMPLES OF SOFTWARE PROCESSES.........................................................................32.3 DIFFERENT PROJECT TYPES.........................................................................................4

3 REQUIREMENTS ENGINEERING AND RE-TOOLS...........................................................................7

3.1 REQUIREMENTS...........................................................................................................73.1.1 What is a requirement?.......................................................................................83.1.2 Different kinds of requirements..........................................................................9

3.1.2.1 Functional and non-functional requirements..................................................93.1.2.2 Other ways to divide requirements...............................................................10

3.2 REQUIREMENTS ENGINEERING...................................................................................123.2.1 The requirements engineering process.............................................................12

3.2.1.1 Feasibility study............................................................................................133.2.1.2 Requirements elicitation and analysis...........................................................14

3.2.1.2.1 Elicitation techniques..............................................................................153.2.1.3 Requirements specification...........................................................................173.2.1.4 Requirements validation...............................................................................183.2.1.5 Requirements prioritization..........................................................................193.2.1.6 Requirements management...........................................................................20

3.2.2 Requirements engineering maturity model.......................................................213.3 RE-TOOLS..................................................................................................................22

3.3.1 History...............................................................................................................233.3.2 Why use an RE-tool?.........................................................................................233.3.3 Alternative tools (Examples: Word, Excel, Access)..........................................25

4 RESULTS FROM INTERVIEWS.............................................................................................................27

4.1 INTERVIEW QUESTIONS..............................................................................................274.2 INTERVIEWED ORGANIZATIONS AND A SUMMARY OF THE RESULTS.........................27

4.2.1 Analysis of conducted interviews......................................................................304.3 SUMMARIZED RESULTS FROM INTERVIEWS WITH RE-TOOL VENDORS......................33

4.3.1 Analysis of conducted interviews......................................................................344.4 A CROSS-ANALYSIS BETWEEN CONDUCTED INTERVIEWS..........................................34

5 HOW TO CHOOSE A SUITABLE RE-TOOL.......................................................................................37

5.1 SOME EXISTING COTS SELECTION METHODS...........................................................375.2 THE GRTS-METHOD FOR RE-TOOLS SELECTION: PRINCIPLES AND FRAMEWORK. . .38

5.2.1 Principles..........................................................................................................385.2.2 The GRTS Framework......................................................................................39

6 EVALUATION OF THE GRTS-METHOD.............................................................................................47

6.1 EVALUATION SCENARIO AND QUESTIONS..................................................................47

vii

6.2 RESPONSE TO EVALUATION.......................................................................................477 CONCLUSIONS AND FUTURE WORK.................................................................................................49

8 REFERENCES............................................................................................................................................53

APPENDIX A. RE-TOOL SURVEY..................................................................................................................57

APPENDIX B. CRITERIA TABLE AND SUMMARY OF THE RE-TOOLS..............................................63

APPENDIX C. INTERVIEWS WITH ORGANIZATIONS............................................................................65

APPENDIX D. INTERVIEWS WITH RE-TOOL VENDORS........................................................................89

Appendix E: Response to evaluation questions......................................................................................................95

viii

1 Introduction

It is now well known that problems with the requirements are one of the key issues in successful product development. If organizations do not define their requirements in a structured and unambiguous way it could cost the organization much both in time and money when failures occur. Research by Boehm (1980), shows that it will be more expensive to fix requirement failures the later in the process they occur.

As more organizations begin to realize that poorly defined requirements will affect the outcome of the product that is developed (since the requirements describe the product), the topic of requirements engineering gets very important.

There can of course be quite a serious amount of requirements that will be defined in some projects and therefore the need for requirements engineering tools (RE-tools) can be valuable (Lauesen, 2002). The RE-tool have many advantages which will be described later in chapter 3.3.2 and appendix A, but some of the very important functions of the RE-tools on the market today are support for traceability of requirements and change management. These functions can be difficult to manage without a specific RE-tool.

From an interview made with a vendor of an RE-tool, it is learned that organizations have become more concerned with managing their requirements and that the sale of RE-tool licenses has increased significantly over the past two or three years.

In this chapter the purpose of this thesis is presented together with a structure of the report.

1.1 Purpose of the thesis

The main purpose of this thesis is to suggest guidelines for the acquisition of an RE-tool. The purpose is however not to recommend a specific RE-tool. These guidelines are described to help organizations, which are in the process of evaluating and choosing RE-tools. The guidelines provide some important things to think about before and when choosing an RE-tool and therefore an overview of some of the different RE-tools on the market today is included. Different functions and structure of the RE-tools together with pros and cons and other facts are described. The overview is based both on our own analysis and the analysis of independent sources, both academics and representatives from organizations.

Interviews have been conducted both with developing organizations and RE-tool vendors to find information about when RE-tools should or should not be used and which benefits they have.

The results of the analysis together with interviews conducted at organizations lead to the guidelines mentioned above.

1.2 Structure of the report

This report has the following structure: chapter 2 describes the topic of software engineering and its different sub-processes, process models and project types; chapter 3 gives an overview of requirements engineering and different types of requirements, RE-tools are also included; chapter 4 presents the most important results gathered from the interviews with the organizations and RE-tool vendors; chapter 5 presents the GRTS (Guidelines for RE-Tools Selection)-method for selection and evaluation of RE-tools and how to use it; chapter 6 presents the response from the evaluation of the GRTS-method; chapter 7 provides

1

conclusions of the things we have learned from this study and chapter 8 contains an extensive reference list for those who want to learn more about the important topics of requirements engineering and RE-tools.

Finally we have appendix A, in which a useful survey of some RE-tools is presented and the benefits and drawbacks they have, appendix B, in which we have created a table of the criteria for the selection of RE-tools and how the examined RE-tools have responded, appendix C, in which we present the interviews with the organizations to their full length, appendix D, in which we present the interviews conducted with the RE-tool vendors and appendix E where the evaluation answers are presented.

2

2 Software engineering

Software is for many people a computer program. It is not only a computer program but also associated documentation and data that is essential for making the computer program run correctly. It is an engineering discipline that involves all aspects of software development, beginning with elicitation of requirements and thereafter making a specification, design, implementation and validation to maintaining and upgrading the system (Sommerville, 2001).

Sommerville (2001) uses two key phrases, “engineering discipline” and “all aspects of software engineering”.

“Engineering discipline”, means solution finding and problem solving for engineers working under constraints. These constraints can be both time and/or money. Engineers use methods and theories when they try to discover solutions to problems; however, when no methods or theories are applicable they must try finding possible solutions anyway.

“All aspects of software engineering”, relates to all parts of the software development process. It is not only a technical process. Other subjects, such as management, methods and theories that are involved in production and tool development that helps the process, are also software engineering concerns.

In this chapter the topic of software engineering and its different processes will be studied. Different project types will also be presented and analysed.

2.1 The software engineering process

A software process is all the stages involved in developing software. It is the engineer’s job to perform most of the stages. Sommerville (2001) points out the four most important parts of the software process.

1. Software specification 2. Software development3. Software validation4. Software evolution

Different processes arrange the stages very differently. Two organizations developing the same type of system can use two different types of processes, which organize, and schedule the activities differently. If an organization uses an inappropriate software process, it could lead to a reduction in the product quality and make it less useful. Some general processes are described in chapter 2.2.

2.2 Examples of software processes

There are a number of different software processes and below we have listed a few of them. For more thorough information see Sommerville (2001).

3

The waterfall model. This model separates the phases of the process in a very structured way. Only when engineers and others involved in the development are finished with one phase they move on to the next.

Evolutionary development. An initial system based on vague requirements is quickly developed. The process interleaves many activities, like specification development and validation. The system is then refined until it satisfies all the needs of the customers.

Formal transformation. The specification in this model is formally mathematical. It is then transformed using mathematical methods to a program. The process is mathematically correct so the developers know they are producing a system to meet the requirements.

System assembly from reusable components. It is often cheaper to buy and not develop all parts of a system if there are any appropriate parts on the market. They are then extended and modified to satisfy the customers’ needs and meet the specification. This method can be more effective but the program is not tailor-made for the organization that is buying, so it can be a bit less suitable.

Another process, described by (Manzoni and Price, 2003) is the Rational Unified Process (RUP). It describes a set of activities that is needed to transform requirements to a software system. RUP is more than a single process; it is a generic process framework that can be specialized for a large class of software systems, different application areas, different types of organizations and projects.

2.3 Different project types

When an organization will develop a system there are a number of different project types for the development process depending, for instance, on whom the customer is and which product is to be developed. Some are: In-house development, Product development, Time and material based development, COTS purchase, Tender, Contract development and Sub contracting (Lauesen, 2002).

In-house development: In this project the system is developed for the organization itself inside the organization. Both the customer and the supplier are departments of the same organization. This often occurs in larger organizations and without well-specified requirements.

Product development: In product development the organization have to develop a product with the right features and release it on the market it is destined for at the right time. One difficult thing is to understand the needs of the market and thereafter develop a product. When you develop for an open market you cannot focus on the needs of one customer. You have the option to include a lot of features but it is likely that it will take more time and be more costly, and therefore you can lose market shares. Karlsson (2003) uses the term “featuritis”, which means that a product contains too many features and which also can make the product more difficult to understand.

4

Contract development: The system is developed by an organization according to a contract with the buying organization. The requirements specification and the contract specify what system should be delivered. Both parties work together to form the contract.

COTS purchase: An organization does not always have to produce every part of the software itself. COTS (“Commercial Off The Shelf”) products are executable software that you can buy directly in the “store”. This makes them often cheaper than developing them in the organization but not always really adapted to the demand of the organization because it is not developed directly from the organization’s requirements.

Tender: The customer requests proposals of solutions. He then receives proposals from different organizations. The organization with the best proposal is the one that gets the job. The system can be tailor-made or based on COTS products, which are extended to the requirements agreed.

Time and material based project: Often time and money is a limitation for a project. The cost of the project depends on the time it takes to produce the software or the material needed. The customer can then pay by month and decide if there is to be more work on the system. The requirements are not written and develop during the project.

Sub contracting. A sub-contractor can develop and deliver a part of the system. He delivers it to the main contractor who puts the system parts together and sells it to the customer.

5

6

3 Requirements engineering and RE-tools

This chapter focuses on the requirements and requirements engineering. First we look at what a requirement is and describe some different types of requirements. After that we will go in to the requirements engineering process in more detail with its different parts. Next we look at a special model made for requirements, called the requirements maturity model (Sommerville and Sawyer, 1997) and finally we take a look at the RE-tools.

3.1 Requirements

“Lack of user input, incomplete requirements, and changing requirements are the major reasons why information technology projects do not deliver all of their planned functionality on schedule and within budget.”

–The CHAOS Report, The Standish Group International, Inc., 1995.

If the project cannot manage the basic requirements, the project will certainly fail (The Standish Research Group, 2003). A good requirements model is one of the main reasons for a successful development project (Wiktorin, 2003). There are a number of common problems that can arise, if requirements are not well defined from the beginning (Sommerville and Sawyer, 1997):

The requirements do not reflect the real needs of the customer for the system. Requirements are inconsistent and/or incomplete. It is expensive to make changes to requirements after they have been agreed. There are misunderstandings between customers, those developing the system

requirements and software engineers developing or maintaining the system.

Research done by Boehm (1980) shows that the cost to fix failures in the requirements, increase almost exponentially during the different phases of the project, see figure 3.1.

7

Figure 3.1. Cost to fix errors in different phases of the project (Karlsson, 1998).

One important point we can make from figure 3.1, is that it takes a lot of time and money to correct failures later in the project and that it is very important to clearly specify the requirements from start. The reason why it will cost more to fix failures later is that you could have started to code and have to rework. In this chapter we will look at why it is important to specify requirements correctly, different ways of defining what a requirement is and different kinds of requirements.

3.1.1 What is a requirement?

A system should function in a certain way and it can be very difficult for a software engineer to establish exactly what a system should do, especially if the system is new. The services and limits or constraints of the system are usually referred to as requirements (Sommerville, 2001).

The term requirement is not consistently used in the software industry. One problem in the software industry is the lack of common definitions. It is now almost everywhere accepted that the term requirement should mean, “what the system should do” and not, “how the system should do it” (Karlsson, 1998). This is also stated by Lauesen (2002) who says that if requirements tell us how the system should do something then we have entered the “design state” of the development process.

A very common definition of a requirement is that it is a desirable attribute or function in a system.

A requirement is according to the IEEE Standard Glossary of Software Engineering Terminology (IEEE Std 610.12-1990) defined as:

1. A condition or capability needed by the user to solve a problem or achieve an objective.

2. A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification or other formally imposed documents.

Requirements Design Code System test Acceptance test Operation

Cos

t to

corr

ect e

rror

s

8

3. A documented representation of a condition or capability as in (1) or (2).

This definition covers however both the user’s and the developer’s view of the requirements. All of these different definitions of what a requirement is, show how difficult it is to get a clear understanding of the term. Other definitions used in the literature are, for instance:

“A requirement is a user need of a necessary feature, function, or attribute of a system that can be sensed from a position external to that system” (Davis, 1993).

“Requirements are the statement of needs by a user that triggers the development of a program or system” (Jones, 1994).

“Requirements are defined during the early stages of a system development as a specification of what should be implemented. They are descriptions of how the system should behave, or of a system property or attribute. They may be a constraint on the development process of the system” (Sommerville and Sawyer, 1997).

“A requirement is a portrait of a user’s needs” (Singer, 1992). “In one way, a requirement could be seen as a high-level, abstract statement of a

service that the system should provide or a constraint on a system. It could also be seen as a detailed, mathematically formal definition of a system function” (Sommerville, 2001).

As we can see, there are very different definitions of what a requirement is, depending on the author’s point of view.

3.1.2 Different kinds of requirements

Requirements could be divided in many ways. Software requirements are often divided and classified as functional or non-functional requirements. These requirements are described in chapter 3.1.2.1. There are also other ways to divide the requirements, e.g. by their capability of satisfying the customer. These are described more thoroughly in chapter 3.1.2.2.

3.1.2.1 Functional and non-functional requirements

It is very common that software requirements are divided and classified as functional or non-functional requirements.

Functional requirements according to Sommerville (2001) are statements of services the system should provide, how the system should react to particular inputs and how the system should behave in particular situations. In some cases, the functional requirements may also explicitly state what the system should not do.

Functional requirements according to Wiegers (1999) define the software functionality the developers must build into the product to enable users to accomplish their tasks, thereby satisfying the business requirements.

Functional requirements according to Lauesen (2002) specify, for the system, the functions, how it transmits, computes, records and transforms data. The different interfaces have its own functions, for instance, the user interface is the most important interface, since most of the data are recorded and shown through it.

9

Non-functional requirements according to Sommerville (2001) are constraints on the services or functions offered by the system. They include timing constraints, constraints on the development process, standards, etc.

Non-functional requirements according to Wiegers (1999); these requirements might include standards, regulations and contracts to which the product must conform; descriptions of external interfaces; performance requirements; design and implementation constraints; and quality attributes.

Non-functional requirements according to Lauesen (he calls them quality requirements); the quality requirements are a description of how well the system performs its intended functions. Lauesen gives many examples of quality requirements, and three of them are, performance, usability and maintenance. These are described very shortly below:

Performance. These requirements describe how efficiently the system should work with the hardware, how fast it should respond, how many computer resources it should use, how accurate the results should be and how much data it should be able to store.

Usability. These requirements describe how efficiently the system should work with the users, how easy it should be to learn and how efficient it should be in daily use.

Maintenance. These requirements describe how easy it should be to repair defects and how easy it should be to add new functionality.

There are of course numerous other quality factors, like for instance, security, reliability and many more.

3.1.2.2 Other ways to divide requirements

Functional and non-functional requirements are very common but there are also other ways to divide requirements. Besides using the terms described above, Wiegers (1999), chooses to use the terms business requirements and user requirements:

Business requirements. These represent high-level objectives of the organization or customer requesting the system or product.

User requirements. These describe tasks the users must be able to accomplish with the product. The user requirements are often captured in use cases or scenario descriptions.

Lauesen (2002) chooses to, besides functional and non-functional requirements, divide the requirements into data requirements and managerial requirements:

Data requirements. These are the requirements that describe input and output dataflow and what data the system should store within itself. Lauesen, here describes four key words: Database, Input/output formats, Communication state and System state. It could be debated if this is internal computer matter and whether it should be specified in requirements. However, the relationship is tight between information in the surrounding domain and data stored in the system. Most software engineers mean that data requirements are a part of the functional requirements but Lauesen means that from a practical point of view it could be convenient to separate them.

Managerial requirements. These lie between requirements and contractual issues. The managerial requirements are, for instance, when the deliverables are going to be delivered, the price and when to pay it, verification, what happens if something goes

10

wrong (legal responsibilities), to specify who owns the software and other things. It may also include a specification of the development process to be used.

Sommerville (2001), however chooses to, besides functional and non-functional requirements, divide the requirements into user requirements and system requirements:

User requirements. These are statements, most often in natural language and diagrams, of what services the system is expected to provide and the constraints under which it must operate. A user requirement may be expanded into several system requirements and should be written for client and contractor managers who do not have a detailed technical knowledge of the system.

System requirements. These explain the system services and constraints in detail. The system requirements document, which is sometimes called a functional specification, should be precise. It could serve as a contract between the system buyer and software developer. The system requirements should be written for project managers and more experienced technical staff.

A completely different way to divide your requirements is described by (Cohen, 1995). He classifies the requirements by their capability to satisfy the different people who are interested in the requirements e.g. customers, users, developers and so on. The classification can be made in three types of requirements, namely, sensational, normal and expected. See figure 3.2.

Figure 3.2. Three types of requirements: sensational, normal and expected (Karlsson, 1998).

These are defined as follows:

1. Sensational requirements. These are the requirements that are not spoken for by the users or customers but will lead to great satisfaction if the system could fulfil them. These could for instance be technical possibilities that the customer or user is not

Satisfaction

Fulfillment

Sensational requirements

Normal requirements

Expected requirements

11

prepared for. These requirements will not lead to less satisfaction if they are not fulfilled, because they were not expected in the first place.

2. Normal requirements. These are the requirements that usually are spoken for by the users or customers and could therefore be collected by the deliverer of the system. The degree of satisfaction is directly dependent on how well these requirements are fulfilled. The better fulfilled the happier the customers and consumers.

3. Expected requirements. These are the requirements that are not spoken for but will undoubtedly lead to dissatisfaction if they are not fulfilled. Examples of these are the basic needs of the customers and users, which often are so trivial they will not be spoken for. The deliverer will need a great knowledge of the domain to fulfil these. It will lead to dissatisfaction if these requirements are not fulfilled but will not lead to greater satisfaction if they are fulfilled since they were taken for granted from the beginning.

3.2 Requirements engineering

Requirements engineering is an iterative and incremental process, which purpose is to define the requirements that an IT-system should fulfil (Karlsson, 1998).

It is a well-known fact that it can be very hard both finding the requirements and to formulate them in an understandable and unambiguous way. It is most likely not enough to find and formulate the requirements, they also have to be prioritized, traced and change managed. The process of doing all this and to create and maintain a system requirements document is called requirements engineering. During the last few years, experiences from the Capability Maturity Model (CMM) have shown, that requirements engineering is one of the areas that is of greatest need of improvement when developing a product (Karlsson, 1998).

Requirements engineering can be seen as the process of applying a structured method, such as object-oriented analysis (Rumbaugh et al., 1991; Booch, 1994). In this chapter we will look at the requirements engineering process and its different parts. We will also look at the requirements maturity model and its impact in different situations.

3.2.1 The requirements engineering process

When making products, requirements engineering is a big part of the process (both when it comes to time and importance). The requirements engineering process (figure 3.3) consists of all activities of a project that leads to a requirements document. This document will become the specification for the project.

12

Figure 3.3 The requirements process (Sommerville, 2001).

In many books and papers in the area, the same activities are used in the requirements engineering process and are described below.

In almost all systems, the requirements change, and therefore is the need for requirements management also an important part of the requirements engineering process. Below there is a short description of the phases.

3.2.1.1 Feasibility study

The feasibility study is an estimation of, if it will be sufficient for an organization to continue using its software and hardware to cover its user needs, or further products must be obtained or the existing added with new needed features (Sommerville, 2001). For a new system proposal, the outline of the system and how it will be part of the organization will be studied, so the result determines if it recommends the organization, that it will be cost efficient to carry on with the requirement engineering process or not.

The aim of the feasibility study is to answer a couple of questions:

1. Does the system contribute to the overall objective of the organization?2. Can the system be implemented using current technology and with the given cost and

schedule constraints?3. Can the system be integrated with other systems which are already in place?

If the answer is no to the first question, the organization will not have any use of the system and further study will be a waste of money. Then the system does not support the organization’s objective and therefore does not contribute to the effort of the organization.

Information assessment, information collection and report writing are brought together to create the feasibility report. It can propose budget and schedule changes or more high-level requirements. The feasibility study should also determine if the system could be developed under existing budget constraints. The feasibility study should be swift and cheap.

13

3.2.1.2 Requirements elicitation and analysis

This phase is used for eliciting the requirements by observing existing systems and interviewing users, developers and analysts.

The issues, which then would be examined and analysed, are the application domain, what service the system should provide and constraints on the system such as hardware and performance requirements.

Below some problems are described that can arise, so called, elicitation barriers (Lauesen, 2002, Sommerville, 2001):

Often stakeholders cannot express what they need. Users have difficulty explaining the task they perform and why they do the tasks. Stakeholders often specify solutions instead of demands. Often different stakeholders have conflicting views. Stakeholders will often reject proposals due to a general resistance to changes. It can be difficult for the stakeholders to agree on which requirements are luxurious

and which are essential. Demands change over time. Stakeholders may not exactly know what they want from the system. Many stakeholders have unrealistic demands. The economic and business environment may change.

There are a couple of activities involved in elicitation and analysis (figure 3.4).

Figure 3.4 The requirements elicitation and analysis process (Sommerville, 2001).

The activities are:

1. Domain understanding. The application domain must be examined and understood. This is because many constraints and requirements will differ because of the environment and domain in which the system is going to be used.

14

2. Requirements collection (elicitation). The help of stakeholders will discover the requirements.

3. Classification. In this activity the unstructured requirements are grouped and organized into coherent clusters.

4. Conflict resolution. If many stakeholders are involved, the requirements may conflict. This activity resolves the possible conflicts.

5. Prioritization. Stakeholders discover how important each requirement is. Some will be a lot more important than others.

6. Requirements checking. To see that requirements are consistent and complete and comply with the goal of developing the system, stakeholders check the requirements.

When these activities are completed, the requirements are specified and a final requirements document is created.

3.2.1.2.1 Elicitation techniques

It is important to have the users participate in the project. The users can have different roles in the project; they can be members of the design and workshops, knowledge sources, and participants in brainstorming. When the system should be tested they can do the acceptance test. The users can review the user interface and do usability tests with the testable system. The elicitation techniques described below are taken from Sommerville (2001) and Lauesen (2002).

Use casesUse cases are a scenario-based elicitation type of technique for discovering requirements. It is an object-oriented method. It identifies an actor and is interacted with the system. A use-case scenario covers all actors and all their interactions with the system.

EthnographyEthnography is used to understand social and organizational issues. It is often critical for the success of a system to be satisfying these factors. It uses an observational technique. The daily work at the organization, that shall use the system is observed and richly documented. Ethnography helps discover the actual rather than formal processes and coherent demands. The importance of the social system of an organization is not always clear to all stakeholders.Ethnography is not a complete elicitation technique and should be used together with prototyping or use-cases elicitation.

Stakeholder analysisIt is important to interact with stakeholders to succeed with the project. In this method some questions need to be analyzed by the developer:

Who are the stakeholders? What goals do they see for the system? Why would they contribute? What risks and cost do they see?

15

What kind of solutions and suppliers do they see?

To get the answer to these questions the developer can have large meetings with stakeholders, or many smaller meetings. If that is not enough, interviews can be made with all stakeholders one by one.

InterviewingMany analysts see interviewing as the head elicitation method. Interviewing shall be conducted with different kinds of stakeholders. The interviewer should ask questions about day-to-day work, problems and critical tasks. Data volumes and task times are questions that can later be asked. First, questions of more importance must be asked, like how tasks are carried out, etc. Make a list of questions so you know how to structure the interview. Group interviews can also be made and not only with just one stakeholder.

BrainstormingIn brainstorming a group of people is brought together in a stimulating atmosphere where they can come up with ideas without being laughed at. The facilitator collects the ideas and notes them on the whiteboard. New ideas are then collected. Focus is laid on goals and requirements for the system. After the brainstorming, prioritizing the collected requirements is very important.

WorkshopsThere are a lot of different workshops. In domain workshops the result is a task description, dataflow diagrams or activity diagrams which describe what goes on in the domain. Later the descriptions are turned into requirements. As an important side effect the teams may specify system goals and critical issues. Expert users are essential for the workshops. They know most details of the domain. To get an overview several experts must be consulted. Managers can participate but they may know little of the details. They may anyway be of use in defining goals. In the design workshop users and developers co-operate to design something, usually the user interface. Another name for this is co-operative design. Users can be too involved in design and technical issues and should help elicit the problems but not come up with solutions. They can also get far from the business goal.

PrototypingA prototype is a kind of simplified version of some part of the final system. The developers experiment with the prototype to get more insight of how the system will work in reality. The result of the experiment can be two types of requirements:

Product-level requirements: The prototype shows what it can achieve in a use case or task.

Design-level requirements: This prototype is a simplified version of the systems user interface.

Prototypes can be useful also in other tasks. One example is test communication with an existing product about something in the domain, which can reveal a lot.

Pilot experimentIn many cases the new system will be COTS-based with perhaps some more functionality. The cost of the system may be high, but the main risk is whether the organization can adapt to

16

the system and use it to improve performance. The changes in the organization is very often more expensive than the new system installed.

In this and other situations the risks can be minimized by pilot experiments. A small part of the organization tries the system on a trial basis but the product data is sharp. They will at the same time experiment with changed work procedures. The project team observes, analyses and evaluates the cost/benefits of the system and suggests ways to use it.

This can help in improving the identification of the final requirements and their priorities. The main problem is that the system must be operational to a large extent.

Risk analysisThe purpose of risk analysis is to identify the risks with the new project. Some ways to reduce risks must be discovered. The analysers can look at work procedures, client relations and the customers’ IT-staff. Questions to the stakeholders can be:

What kinds of changes are needed? Which potential conflicts do they see with other stakeholders? What is the risk that the changes are blocked?

Later the analysers will try to reduce the risks in these problems. The future users of the system must try to imagine working with it.

Cost/Benefit analysisThis method looks at the project and compares the cost of developing it to the benefit gained. Often it is presented in money and many analysts claim that it is impossible to measure all of this in money terms. A broader meaning of cost/benefits can be expressed both in money and qualities. An example of money terms is change cost and an example of quality terms is customer satisfaction.

We must consider the time the methods take, how many people must be involved, is the customer available for observation, and how often can we meet with him.

Scenario analysis and prototyping are not a bad choice if time in the methods is the limitation but can be less applicable if the preparing time is relatively short. The main difference is that prototyping demands more availability for technical apparatus.

3.2.1.3 Requirements specification

In this stage, information on requirements gathered in the elicitation and analysis phase is used to make a document that will define the requirements. Heninger (1980) suggested that software requirements documents should at least satisfy the following:

It should only specify external system behaviour. It should specify constraints on the implementation. It should be easy to change. It should serve as a reference tool for system maintainers. It should record forethought about the lifecycle of the system. It should characterize acceptable responses to undesired events.

To avoid problems, IEEE/ANSI 830-1993 has proposed a standard for the structure of the requirements specification, however, this is just a suggestion and it is certainly not ideal for all projects (Sommerville and Sawyer, 1997).

17

3.2.1.4 Requirements validation

Requirements validation is a process that will determine if the requirements are representing the system that the customer wants (Sommerville, 2001). The process is very important because an error in requirements engineering will later lead to a more difficult error correction of the system. It tends to cost more money the later in the process an error is corrected.

A requirements document error is far easier to correct earlier than later when you must remake the design (see also chapter 3.1). The validation process consists of a number of different checks that should be conducted on the requirements. They are as follow:

1. Validity checks. Different stakeholders check if the system performs how they intended it to perform.

2. Consistency checks. The requirements in the document should not be in conflict with one another.

3. Completeness checks. All functions and constraints of the system should be covered by the requirements in the requirements document.

4. Realism checks. All requirements should be realistic to implement.

5. Verifiability checks. The testing team must be able to verify the requirements. A requirement that is not possible to verify can lead to a disagreement with the customer.

There are some techniques that can be used to conduct checks:

1. Requirement reviews. A team reviewer uses a systematically analysing method. The team reads the report to check it for anomalies and faults. A review can be formal or informal. In the formal reviews the developing team should look at each requirement with the client and check the requirement for a number of different attributes:

Verifiability, the requirements must be written so it is possible to test the requirement. Comprehensibility, so both end-users and developers properly understand the

requirement. Requirements as a whole, for completeness. Consistency, the different requirements should not conflict. Traceability, so a requirement later can be traced to the source of the requirement. Adaptability, which tells us if the requirement can be remade without major changes.

2. Prototyping. A real working model of the system is made and could be experimented with to check requirements with customers. As we can see, prototyping can also be used for validation of a system.

3. Test-case generation. If a requirement is hard to test it is usually hard to implement, and should therefore be reconsidered.

4. Automated consistency analysis. If requirements are expressed as a system model in formal or structured notation, case tools can be used to check requirements.

18

It is almost impossible to make the validation so that no faults remain but the better it is made; less rework has to be done.

3.2.1.5 Requirements prioritization

During the requirements analysis and negotiation process, priorities should be assigned to individual requirements, which reflect their importance to stakeholders and to the overall success of the system. (Sommerville and Sawyer, 1997)

Because of the time and money an organization has to spend developing a product, some sort of choice has to be made of which requirements are more important and less important than others. In practice, some sort of prioritization has to be done. Prioritization should be made in discussions between stakeholders and analysts, but a problem is that it may be difficult to prioritize requirements from the beginning. Therefore, an amount of the requirements have usually already been set when the prioritization of the requirements begin. Two of the benefits of using prioritization are (Sommerville and Sawyer, 1997):

Helping the stakeholders to decide on the core requirements for the system, which is being developed.

Helping the designers to decide on the system architecture and to resolve design conflicts, which may arise during the project.

The prioritization could be done in several ways of which the easiest is perhaps to give the requirements a priority on a scale. There are some difficulties with this approach however, most customers tend to find all the requirements absolutely necessary and give all the requirements the highest degree on the scale (Karlsson, 1998). The other way around is that a customer could argue that a requirement is necessary for the product but when the developer explains and points out the cost, difficulty, technical risk, or other trade-offs associated with a specific requirement, the customer might decide that the requirement was not as essential as he thought. Two relatively easy ways of prioritizing requirements collected from the literature are:

IEEE (1998)

Essential. Software not acceptable unless these requirements are provided in an agreed manner.

Conditional. Would enhance the product, but would not make it unacceptable if absent.

Optional. A class of functions that might or might not be worthwhile.

Kovitz (1999)

3. Must be implemented perfectly. 2. Needs to work, but not spectacularly well. 1. Can contain bugs.

Another way to prioritize requirements can be based on value, cost and risk (Wiegers, 1999). Two methods that can be used are:

19

Quality Function Deployment (QFD), which provides a comprehensive method for relating customer value to the features for a proposed product (Cohen, 1995).

Each requirement is rated against several weighted project success criteria and computes a score to rank the priority of the requirements (Total Quality Management (TQM)).

3.2.1.6 Requirements management

There is nothing permanent but change - Heraclitus

“Requirements Management involves establishing and maintaining an agreement with the customer on the requirements for the software project. The agreement forms the basis for estimating, planning, performing, and tracking the software project’s activities throughout the software life cycle.” - CMM Practice Manual, 1993

When requirements change, and they undoubtedly will, the need for requirements management is an essential part of the requirements engineering process. Requirements can change during the entire project life cycle and also during maintenance. Some reasons for why requirements will change during a project are explained by Lauesen (2002) and Sommerville (2001):

Requirements turn out to be wrong during development. New demands are discovered. Some requirements turn out to be too expensive to meet. Sub-contractors do not deliver as promised. The parties agree to implement something simpler than originally described. Large systems usually have a diverse user community. The people who order a system and the users of a system are rarely the same people. The business and technical environment of the system change and these must be

reflected in the system itself.

Requirements management is described by numerous authors in the literature but to make it easy to understand, the following five points can be used; reporting, analysis, decision, reply and carry out the decision (Lauesen, 2002).

1. Reporting. A requirements issue is reported to the change control board. They record it with date, source, and a short description. If necessary, the source person who reported the issue gets a confirmation.

2. Analysis. The issue is analysed together with other issues. Is it a new demand, a request to change a requirement, or a misunderstanding? What is the real need behind the request? What are the consequences for the customer? What would it cost to fulfil the request? Will we have to investigate it further? This analysis should be made at regular intervals, for instance weekly or bi-weekly.

3. Decision. Should we reject the request because the advantages do not match the costs? Should we include it as a new or changed requirement – with which priority or as an option? Should we include it in the next release or postpone it further? Can we find another way to deal with it? Can we trade it with the supplier for some other changes?

20

Can we delay the decision and for how long? The decisions can be made at the same regular meetings, but not necessarily the same day as the analysis is made.

4. Reply. Report the decision to the source and other people impacted by it, for instance developers who have to carry out the change.

5. Carry out the decision. Revise the specification, negotiate with the supplier, put it on the list for the next release, or whatever has been decided.

When it comes to requirements management an RE-tool could be useful. The RE-tool often has a traceability function (see also chapter 3.3.2) where you can see what happens if a requirement is changed. Which requirements to change or not change is always a question of benefits and costs. These two important factors have to be weighed to one another. What is important to the customer is not always important to the developer, and if the change has to be made it may cost a lot both in time and money.

Some sort of specification of what changes has been made could be done in order to be able to look at the changes later on or after the project. The information has to be stored somewhere. A regular list is perhaps not that good because the list could be very long after a while and difficult to understand. Other ways are to store the information in a database or to use some sort of RE-tools (described more thoroughly in chapter 3.3), which enables information storage. The most important thing is however, that the information is stored.

3.2.2 Requirements engineering maturity model

In the 1980s SEI (The Software Engineering Institute) at Carnegie-Mellon University initialised a study. The goal was to study ways of assessing the capabilities of contractors and the special interest lay in contractors bidding for software projects funded by the US Department of Defence.

The CMM (Capability Maturity Model) was the result of this study. The CMM is a widely used framework for guiding process improvement in a software development organization.

The requirements engineering maturity model is not fully covered in the CMM, which mostly focuses on development and management. Therefore Sommerville and Sawyer (1997) define a requirements engineering maturity model, which consists of three levels. The requirements engineering maturity is based on how well defined the requirements engineering process is, in the organization. In the first level in the model the organization is not very mature but it increases in the higher levels.

Level 1: Initial level. The requirements process on this level is not defined in the organizations. Problems are common and they often do not produce good requirements documents. The organizations frequently use ad hoc requirements engineering. No advanced methods are applied in the process to support it.

Level 2: Repeatable level. Defined processes and standards are used by the organization. The process use advanced tools and techniques to help. Documents have a higher quality.

Level 3: Defined level. Explicitly defined requirements engineering processes are used by the organization. They also have a program for process improvements. It is used for analysing the values of new techniques and methods that are considered to help processes.

21

Sommerville and Sawyer (1997), provide guidelines and suggestions on how to improve the organization’s requirement engineering process with help of this requirements maturity model.

When an organization with an uncontrolled requirements engineering process wants to implement new methods or techniques, it can take additional money and time and the benefit from it may be reduced. Therefore organizations benefit from defining better processes. Some organizations may find it more effective to refine their processes on one level while others try to advance up to the next level. The most cost effective way is decided by the size and complexity of the produced systems.

The model affects the documents produced by the organization. Immature organizations can also produce good documents but have a high probability to fail. Organizations with high maturity will probably produce higher quality documents. This does not mean that they do not have requirement problems when dealing with new systems but they have a better chance of solving the problems.

3.3 RE-tools

When developing the large, complex systems of today it can be difficult for system engineers and other stakeholders to manage a large amount of requirements. The RE-tool can be useful when managing these requirements and to achieve a quality of the product that can compete with other organizations. It is however difficult to weigh quality against quick production.

When not using an RE-tool to manage requirements, a lot of time will almost certainly be spent on tracking requirements changes. These changes will always occur during a project due to customers changing their requests or new software, which has to be developed to compete with the products from other organizations.

This chapter explains why the use of an RE-tool can be beneficial for a requirements process and also give some examples of RE-tools and some benefits and drawbacks with them. Finally we also take a look at some alternative tools, which are still being used by organizations, such as, Word, Excel and Access.

The RE-tools that are being analysed are: DOORS, by Telelogic, CaliberRM, by Borland & Starbase, RequisitePro, by IBM & Rational, RTM Workshop, by Integrated Chipware and Focal Point, by Focal Point AB. These RE-tools are chosen because they represent a large portion of the market and they have been mentioned in our interviews with the organizations. They are also mentioned in several articles, e.g. Yphise (2002) and Wiegers (1999), which have helped us gather information about these RE-tools. Figure 3.5 presents market shares of RE-tool vendors in 2001. The figure is three years old but we have learned from the interviews with the RE-tool vendors that the market shares have not changed much since then. We can see that DOORS and RequisitePro are the two biggest RE-tools with about 30 % each of the market, RTM Workshop has about 13 % of the market, CaliberRM about 8 % and other RE-tools about 17 %.

22

Figure 3.5. RE-tool vendors 2001 (Standish Group, 2002).

3.3.1 History

About two decades ago people were using paper and pencils when they documented their requirements. Even large, complex systems were handled this way, such as, design and development systems for communication, transportation, space programs, military and medical usage. The system engineers had to keep track of hundreds or thousands of requirements by hand and of course this took a lot of time.

In the mid-80’s the engineers began to use word processors, databases and graphical packages as their tools when managing requirements. This was of course a great improvement but had still limitations.

When beginning to use RE-tools as their way of managing requirements, systems engineers could all of a sudden transfer data from one tool to another or from one environment to another. These tools were designed to support a single user instead of the large, teamed system development environment.

Today, many RE-tools are specialized in automated systems engineering and the RE-tools have a greater storage capacity and faster processing speeds. This means that they now have the ability to support the large, complex systems, which are being built today. Also the RE-tools of today have limitations, such as, relating the information stored within one RE-tool to information stored in another. This would require that the RE-tool vendors cooperate, and that is perhaps not realistic (INCOSE, 2004).

3.3.2 Why use an RE-tool?

In some cases it could be enough to use e.g. Word or Excel, but there are many cases when an RE-tool could be useful. For small projects perhaps the use of an RE-tool is not necessary, and the organization could use e.g. Word, to specify and document the requirements.

23

Below we have listed many of the apparent advantages of using RE-tools. The points below describe areas where RE-tools can be useful to simplify or improve the work. They are taken from Wiegers (1999), INCOSE (2004) and other RE-tool surveys.

Configuration managementTo get a working and powerful requirement engineering process the integration of requirements management and configuration management could be helpful. The RE-tool can help in highlighting baselines or controlling the different versions better i.e. control over which requirements are in which releases. This is done to save time making duplicate changes.

TraceabilityA survey made by INCOSE members show that requirements traceability is the most critical need to be addressed by RE-tools. In many cases you have to be able to trace a requirement back to a prior requirement (could be a requirement on a higher abstraction level), which it is connected to, or to the requirements specification. The purpose of traceability can be useful, for instance, if a conflict with the customer should arise, you should be able to trace back the requirement and check why the requirement was created in the first place. Traceability is also useful to show dependencies between requirements. Traceability also exists between requirements and design, code, tests etc. Dependencies between requirements may be viewed in a traceability matrix.

Change management analysisRequirements will almost always change over time and new requirements will be added. Then it is useful to have change management analysis to help find which requirements this will affect. Change management analysis will also help you make a cost estimate of the change.

Levels of accessDifferent people in a project group should perhaps have different levels of access when working on a project. The RE-tool will be useful in making the requirements easier to access to the project team and customers. The project team need this because they should always be up-to-date and customers need this so they may be able to review the requirements and ensure that their needs were understood and documented correctly. The RE-tool can have different access levels so information can be just viewed by the users in the organization and not accessible for change.

PrioritizationWhen developing a larger project you can often have problems with the time limit set to finish the project. You will sometimes have to prioritize the requirements (see also chapter 3.2.1.5). By using prioritization we can answer the question of which requirements should be in the first version of the system and which requirements that have to wait. There are RE-tools that support prioritization with AHP or by adding a number of prioritization attributes.

Cross project linkingIf you have written project specifications with the help of a specific RE-tool earlier which satisfies cross project linking this could be valuable when you are managing another project. Cross project linking could be used to get standards or shared documents from your whole database and you could use it to regain knowledge from prior projects.

24

Graphical representationWhen writing your requirements documents you may need some form of graphical representation e.g. graphical charts and diagrams. Many of the RE-tools have these features, which will make the structure of the requirements document easier to understand and follow.

Assigning and storing requirements attributesTo easier classify and prioritize requirements, analysts could with the help of an RE-tool, assign attributes to requirements. People working on the project must be able to look at these attributes and some people should also be allowed to change them. Examples of attributes are: date of creation, version number, author, status, priority, cost, release number and risk.

Track statusTracking the status of the project is very important to a project manager. With the help of an RE-tool he or she could easily see if the requirements are “implemented and verified”, “implemented but not verified”, “designed”, “tested”, and so on. This gives the project manager a good insight into the project status.

CommunicationWith the help of an RE-tool, discussions could be issued electronically. The RE-tool could provide a group discussion to allow requirements to be refined and prioritized correctly. E-mail messages could notify the stakeholders and other people affected when a requirement is changed.

ReuseAn RE-tool could support reuse of requirements, which have already been used in some sub-system. These requirements could be stored in the database and then linked to when needed to avoid duplicate versions of requirements.

3.3.3 Alternative tools (Examples: Word, Excel, Access)

It is not a sure thing that you need an advanced RE-tool; it can make things more complicated. Below we have listed a number of reasons for not buying an advanced RE-tool:

For smaller organizations the cost of introducing licenses for an RE-tool could turn out to be too great, so the organization has not got the resources to buy one.

The organization has perhaps not the need for an RE-tool if the number of requirements in the requirements specification is small or they do not need the advanced functions of an RE-tool.

The introduction of an RE-tool could make things more difficult because the way people work could change and that could lead to that the work takes more time which could lead to that the organization lose money.

If an organization hires a consultant for writing the requirements specification the need for an RE-tool is not necessary since the consultant takes care of this.

It is quite easy to list requirements in Word or Excel and the bigger part of the people involved with requirement management at the organization already know how to do this and therefore there is no need for education and learning a new RE-tool. With Excel you can also make models or formats, which you can use when managing your requirements.

25

In Access it can be relatively simple to build your own requirements database; there is however the drawback that the organization must maintain and support it them self.

Some of the RE-tools could demand that a person works full time with the RE-tool, which an organization cannot afford.

26

4 Results from interviews

One of the larger tasks of this thesis was to make contact with a number of organizations to find out what (if any) RE-tools they were using and what overall thoughts they had about requirements engineering and RE-tools. Therefore we have interviewed organizations of different business areas, size, age, etc. The reason that we used semi-structured interviews was that we did not know exactly where the interviews would lead. Many of the persons we have interviewed at the organizations have had interesting things to tell us and the results have been beyond our expectations because we did not think that they would be so open with their opinions. On the other hand, all the interviewed persons have worked with requirements engineering for a long time and have seen and lived with the difficulties thereof. Many of them have also worked with RE-tools for a time.

Since we also wanted the vendor’s point of view, some RE-tool vendors have been interviewed. We decided to send a number of questions by mail to the RE-tool vendors we have chosen to evaluate. One RE-tool vendor has been visited, where we asked the same questions.

The chapter is divided into a number of sections. First the interview questions are described. Thereafter characteristics of the different organizations and RE-tool vendors are presented with a summary of the results from the conducted interviews. Next a deeper analysis of the interviews is presented and finally a cross-analysis of the different interviews. The cross-analysis is conducted to see e.g. if the things the organizations wants from the RE-tools are the same as the RE-tool vendors are focusing on.

4.1 Interview questions

There are of course a number of interview questions to be answered by each organization to help us categorize the organizations. These questions have been chosen to give an overview of each organization and their thoughts about requirements engineering and RE-tools. First there are a number of overall questions about the organization, then some questions about their RE-tool/RE-tools and finally some questions about their thoughts about requirements engineering and RE-tools in the future. These questions and the answers are presented in appendix C.

The RE-tool vendors have also been interviewed. The questions have been asked to get an idea of what the RE-tool vendors themselves think is important and what their main focus is in the development of the RE-tool. There are also some questions about price and courses. The questions that have been asked to the vendors and their answers are presented in appendix D.

4.2 Interviewed organizations and a summary of the results

An overall presentation of the organizations interviewed is presented in table 4.1 below. The organizations are called A-I. Other things presented in table 4.1 are the organization’s business area, employee size and which RE-tool/RE-tools (if any) they currently use.

Also presented in table 4.1 is an overview of the answers that the organizations have provided to our interview questions. This is done because we think that it will be the easiest way to get an overview without being forced to look up the answers in appendix C. After that, we will in chapter 4.2.1 conduct a deeper analysis of the summarized results.

27

Answers/Organization A B C D E F G H I

Business area Defenceindustry

Instruments fortemperature

measurements

Information technologyconsulting

Instruments forpharmaceutical measurements

Packagingindustry

Mobilecommunications

Climatemeasurements

control

Surveillanceand network equipment

Medicinedevelopment

Size of employees 2000 persons 50 persons 90 persons 21000 persons 21000 persons 800 persons 2000 persons 300 persons 60000 personsProject size 10-15 persons 5-10 persons 2-10 persons 20-40 persons 5-10 persons 200 persons 10-15 persons 6-10 persons 1000 personsProject time 5-15 years 6-12 months 3-10 months 2-4 years 6 months Short 6 months 2 months 15 years

Software process Á kind of incremental evolution model

Strive to follow the waterfall

modelRUP

A V-process model according

to a standard

The innovation model and different IT

models

“Props”, a self-developed

process model

Ideally follows the waterfall model but is influenced by evolution and

prototypes

A kind of waterfall model but

incremental iterative models and prototypes are also used

Incremental development

RE-processDiscuss theirway to which

requirements that will be used

Strategy is thatmarket controls

their RE-process

RUP,ISO-certification

No, but anumber of supporting processes

No, just partsof different

process models

Yes, try toimprove it all the

time

Yes, a newlydeveloped

process that has not been

used yet

Structure of thereq. spec,

methods for eliciting

requirements

A structure,where the reqs.

must respect authorities and

standards

RE-toolsDOORS,

RequisitePro, CaliberRM

/ RequisitePro DOORS In the processof developing DOORS CaliberRM / Self-developed

RE-tool timein the organization 10-12 years / A few years 5 years / 5-6 years 1.5 years / 2 years

Selection methodsused/will be used

Continuousevaluation,consultants

Study otherorganizations

Selectionthrough

RE-process

Pilot project,consultants,focus groups

Consultants,evaluation

Consultants,evaluation

Evaluationprocess with

different criteria

Possibly build our own /

Improvements in theselection process

Look from anintegration

point of view/ We have no real

selection process

Better insight in own organization’s

needs/ Usability for

all users / / /

Benefits of RE-tool Many usefulfunctions / Flexibility Links, views

and attributes /Different levels of

access for different users

Object orientation,

e-mail notification,

configuration management

/ Global integration

Drawbacks with RE-tool Lack of security,administration issues / No specific

Lack of configuration

management and integration

/ Low usability, toomany functions

Not have to use external

documents,illustration

needs

/ Low usability

Opinions from employees

Need more integrated

tools/ The RE-tool

works well Quite satisfied /Hard to

understandall functionality

The employees are satisfied

with the RE-tool/ Can be a little

unwieldy

Are you spending a lot of money when

selecting a RE-tool?No, just a little / Not much No, but time / Yes, quite much No, not much / /

Usage (%) of RE-toolfunctions

Just therudimentary / Not all

functions are used 60% / Not all functions are used 50-60 % / Most functions

Was it a good investment

to buy the RE-tool?Absolutely /

Yes, wellinvestedmoney

Absolutely / Yes Absolutely / Yes

Ease of use Quite easy / Fits ourprocess

Yes, if you understand

the RE-process/ Yes Yes / Can be a little

unwieldy

Want from the RE-tool /Want more from the RE-

toolBetter integration /

Betterintegrationwith Excel

Configurationmanagement,

integration

Integrationbetween

different tools

Integration, noexternal

documents should have to be used

A straight-forward web-

interface/ Electronic

signature

Main reason for not invested in a RE-tool /

Small organization,

no need, price/ / Should be

tailor-made / /Price and

adaptation toour needs

/

Vision of the futureof RE-tools

Will increase andmust be used

Interestingarea

Importantarea A must Complex area

that will mature

Will be used more if RE-tools are

intuitiveIt will increase A little sceptical

Important with correct

specification

Vision of the future of the company

Stable,same focus

Stable,same focus

Stable,same focus

Stable,same focus

Stable,same focus

New needs to adjust to

Possible expansion

Hard competitionin the future

Stable,same focus

Figure 4.6. Interviewed organizations and summarized results from conducted interviews

4.2.1 Analysis of conducted interviews

The analysis has been performed in this manner; patterns and/or connections from table 4.1 and from the rest of the answers in appendix C has been presented, which could help us draw some conclusions. We have also tried to tie a connection between the answers from the organizations and the theory from books on this subject.

Project sizeThe organizations A, D and I, with project times of over a year all have RE-tools. A reason for this can be that these projects include much people who need to cooperate and use the same sources. A long project time could lead to many changes in the requirements, which can make the RE-tool useful. However, among the organizations with a shorter project time, some of the organizations (C, F and G) have RE-tools and some organizations (B, E and H) have not.

Business areaThe organizations in the medical and defence industry (A, D and I) all have RE-tools. A reason for this may be that their focus is security. They have different standards and conventions that must be fulfilled. The technique-developing organizations (F and G) have RE-tools but not organization H, which is the black sheep. The reason for this is difficult to see. Consulting organizations (A and C) have RE-tools. A reason for this may be that the adjustment to the customers leads to a more advanced requirements process and RE-tools.

BenefitsThe benefits of the RE-tools can be grouped into two: Functional and non-functional benefits. Organizations A, D and G mention different functions of the RE-tools but organizations C, F and I mention different non-functionality of the RE-tools. This can depend much on the interviewed person’s point of view. A point is that some benefits mentioned by the organizations are the same as the benefits described in chapter 3.3.2. Examples are traceability and access levels.

DrawbacksOrganizations B and H, which do not have RE-tools mention the price as a drawback and the main reason for not investing in an RE-tool. However, among the organizations with RE-tools, the drawbacks are split between functional and non-functional drawbacks. Organizations D and G mention functional drawbacks and organizations A, F and I mention non-functional drawbacks. This is also dependent of the interviewed person’s point of view.

Want more from the RE-toolEspecially two things have been mentioned by the interviewed organizations: better integration between the RE-tool and other software (organizations A, C, D, E, F and G) and better usability (F and I). A reason for better integration is that organizations do not want to convert files for import and export. High usability is also important to the organizations so people in the organization can use the RE-tool.

30

Percentage of used functionsAll organizations (A, C, D, F and G) except one (H) have stated that they do not use that many functions of the RE-tool, only the basic functions. This is not that surprising since this is the case with most other tools.

UsabilityThe organizations (in general) seem to think that the RE-tools are quite easy to use for the people who work with them daily. However, it can be difficult for the people who do not use them daily to understand all the special functionality and the user interface. It is however stated (organization D) that you should have the current requirements process clear in mind and you have to understand the meaning of it to be able to use the RE-tool in a satisfying way. Organization I thinks that their RE-tool can be a little unwieldy sometimes.

RE-tool integrationIntegration between tools must also be improved according to results from the interviews (organizations A, C-G). Customers wants to be able to buy tools from different vendors and therefore not only have some integration but an extensive one so they with ease can exchange data between tools without file converting or data adaptation efforts (organization F). One organization even wanted integration between RE-tools (organization E). This can be a good thing to improve. The reason for this is that some organizations use different tools in different projects. This may mostly occur in consultant organizations. Some organizations also want larger tools that focus more on “the whole picture” (organizations E, F and H). Project guidance, test tools and change management tools should all be integrated into the same tool.

Selection of RE-toolsSome interviewed organizations (organization D and F) think that it is of great importance that you first look at the current requirements process and try to think how this can be developed. Perhaps there is not a need for a new RE-tool; the organization could just as well cope with your old one or with alternative tools (see chapter 3.3.3). If the need for the advanced functions of the RE-tools is not there, the introduction of an RE-tool could make things more complicated; for instance, it could take more time to create the requirements specification. Before the organization has started using an advanced requirements process the organization will probably not need an RE-tool. Therefore; first refine the process, then buy an RE-tool if needed. Probably you should let the process settle first before looking for an RE-tool to solve the problems.

However introducing an RE-tool can also help in the process especially if the right tool is chosen and there is strong connection between the RE-tool and the requirements process (an example is RequisitePro-RUP, see chapter 2.2). This is the case for organization C.

One thing we can see from the interviews is that organizations have a lot of different methods although many seem to rely on just one or a few people’s personal experience or opinions in the matter (organizations A, E and F). A reason for this may be that they are not too eager to spend much time and money on conducting an evaluation, which can take some time.

Money spent on selection methodsAll organizations except one (organization F) have stated that they do not spend much money on evaluations of RE-tools before buying a new. If there exists a method that can be useful when selecting an RE-tool instead of buying one ad hoc, this can lead to a more suitable RE-tool without taking much longer time.

31

Software processesSome of the processes used by the interviewed organizations are connected to the processes described in chapter 2.2 (e.g. organization A- Evolutionary development, organization B – Waterfall model, organization C – RUP) but the interviewed organizations have often made changes to these processes to make them fit their developing environment.

Advanced requirements processesAccording to the interviews, advanced well-structured requirements processes have not been used more than five years in any of the organizations. However, more advanced processes are being developed (organizations C, F and G). If the requirements process described in chapter 3.2.1 is followed it is our opinion that an organization has an advanced requirements process.

It is also important to mention here that despite the fact that RE-tools have been on the market for about 15 years now, they have, among the organizations we interviewed (with one exception, organization A), not been used more than 5 years in any organization.

Requirements changeAll of the organizations have stated that the requirements almost always change during the project. The reasons for this are however many. A number of reasons collected from the interviews are: if a project runs for a long time, the software on the market could improve quite a bit, which can lead to that requirements change (organization B). The market can change the conditions of the requirements, which can also lead to a change (organization B and H). Other reasons for requirements to change can be that they can turn out to be “impossible” to implement or that the customers change their minds about the requirements during the project (organization F). In chapter 3.2.1.6 we have described a number of reasons to why requirements change during a project according to literature. As we can see, some of the reasons from chapter 3.2.1.6 and the conducted interviews of why requirements change, agree with each other.

Organizations with a requirements specification where the requirements do not change could have less use of an RE-tool in that aspect but this does not exclude the need for an RE-tool to solve problems in other problem areas.

Requirements engineering and RE-tools in the futureFrom the interviews it is learned that all organizations think that requirements engineering is an interesting and sometimes complex area that will mature and increase in importance in the future. Much in the area of information technology will be developed and therefore requirements engineering will also grow. To get customers satisfied, you must be careful about how you manage your requirements (organization E). Most organizations also think that an RE-tool should be an integrated part of the requirements process.

The usage of RE-tools will probably increase in the nearest future. This increase in the usage of RE-tools could also lead to that usability of the RE-tool will be more important in the future because more people in different parts of the organization will get in contact with the RE-tool.

Change of RE-toolsFrom the interviews we have learned that there could be many reasons to why an organization chooses to change the RE-tool. It could for instance depend on the fact that an objective from the leading board of the organization states that the same RE-tool or server should be used in the whole organization (organization D), many organizations conduct continuous search for improvements in the requirements process which lead to the change (organization A). It seems to be outer factors and not the functionality that lead to a change of the RE-tool. The

32

organizations are reluctant to change, which is not that strange since it can be expensive to change RE-tools.

A conclusion from the interviews is that an RE-tool could be useful when managing the requirements. Therefore we have chosen to look more thoroughly at five major RE-tools on the market today, namely, DOORS, CaliberRM, RequisitePro, RTM Workshop and Focal Point. These are the RE-tools that have been mentioned during the interviews and which we therefore have decided to evaluate.

4.3 Summarized results from interviews with RE-tool vendors

We have chosen to summarize the results in table 4.2 to get a good overview of the answers provided by the RE-tool vendors. The table contains some basic information about the RE-tools plus five questions that we believe are the most important to be answered and analyzed. In chapter 4.3.1 we provide a deeper analysis of the answers. The vendor of RTM Workshop has not answered our questions despite many tries and is therefore not represented in the table below.

Questions/RE-tool DOORS CaliberRM RequisitePro Focal PointVendor Telelogic Starbase/Borland Rational/IBM Focal Point AB

Main environment Windows, Unix Windows Windows Web-based

Cost and support No information14000

SEK/licence excluding support

40200 SEK/licence including support

Tool rented by customers including support and updates

What is the tool especially good at?

Traceability, Scalability,

Robustness, Graphs

Good integration capability, a

complete tool

Combines the best of two worlds: Word for

documentation and database for

filtering/sorting/linking requirements

Not only traditional requirements

administration but prioritization,

planning and decision making

What is the main difference between your

tool and others?

Managing views and a self-made

database

Modern architecture based on OO database,

easy to learn, client-server

solution for multi-users

Easy to learn, importing existing Word documents and continue working with

requirements in Word, synchronization between

Word and database

The only completely web-based tool without local installations,

configuration while running program

Why would you recommend your tool

instead of the competitor’s?

Large potential, new technical improvements,

good integration capability, multi-user and security.

Support and education,

authorization, electronic

signatures, an open API

Easy to use and continue working with requirements

in Word

Only tool with built-in tools for review,

prioritization, release-planning, road

mapping, follow-up and intelligent search

How much time and money do you spend on integration with other

software?

Improvements are made all the time

Caliber’s architecture

allows all info-change, much time on new integrations

No information

A built-in platform which makes it

possible to integrate with other tools, now mostly maintenance

Table 4.1 RE-tool vendor interview summary

For more info about each RE-tool, see appendix A, where a survey of the five evaluated RE-tools is presented or appendix B, where a summary of the RE-tools is presented.

33

4.3.1 Analysis of conducted interviews

In this chapter we provide a deeper analysis of the answers presented in table 4.2 above, which hopefully leads to some conclusions about the main focus of the different RE-tool vendors. As in chapter 4.2 we have made some connections between the answers of the RE-tool vendors and the theory to see how they match.

It could be more difficult to draw conclusions from the interviews with the RE-tool vendors since there are only four participating. The answers given by the RE-tool vendors are of course not complete and there is much more which could be added to these answers. The answers could also be subjective. These interviews were conducted however to get an overview of the main focus of the vendors.

The focus on functionality or usabilityThe RE-tools are quite similar in the aspect that they have the same main functions that the organizations can use but the structure and the vendor’s main focus could change a bit. The different RE-tool vendors state however that they focus much both on functionality and usability. Especially Focal Point have focused on prioritization where the area of decision making also is included while the other RE-tools tend to focus more on traditional requirements management.

RE-tool integrationAll of the RE-tool vendors tend to focus much on integration with other software both in their own product suite and to other RE-tools’ product suites. The different RE-tools integrate with other software in different ways e.g. through an open API.

Requirements engineering and RE-tools in the futureFrom the interviews, the RE-tool vendors have stated that requirements engineering and RE-tools are important areas for the developing organizations and that these will grow in importance in the future. One RE-tool vendor have stated that the sale of RE-tools have increased much the last few years and that it will probably continue to increase the next years also.

4.4 A cross-analysis between conducted interviews

In this chapter a cross-analysis between the conducted interviews with the organizations and the conducted interviews with the RE-tool vendors is presented to see if any patterns and/or similarities are seen and what conclusions that can be drawn from this.

PriceThe price seems to be an important issue to why some developing organizations not are interested in buying RE-tools. The main reason that organizations B and H not have bought an RE-tool is the price and all of the interviewed organizations except one (organization E) mention price as an important factor when buying an RE-tool (see chapter 3.3.3). If RE-tool vendors have noticed this and perhaps lower their price, smaller organizations could afford it and will also be interested in buying the RE-tool.

34

Satisfaction of the RE-toolsWe would like to point out that since the developing organizations have answered that they are satisfied with the RE-tool they have bought, the communication between the RE-tool vendors and the developing organization could not be that bad.

RE-tool integrationWhile RE-tool vendors clearly state that they focus much on integration between the RE-tools and other software, the organizations tend to think that more and better integration is needed between different RE-tools and between RE-tools and other software (see chapter 4.2.1: Re-tool integration). This thing is clearly important to the developing organizations and the RE-tool vendors must focus even more in this area.

Functionality vs. usabilityRE-tool vendors state that they focus much both on functionality and usability, but the interviewed organizations seem to think that they do not focus enough on usability. If RE-tool vendors focus even more on usability perhaps more organizations realize that the RE-tools are easy to use and then perhaps more organizations will start using the RE-tools.

The RE-tools are quite similar in their main focus and which functions they have, so maybe it is not that important to choose an RE-tool based on the functionality, instead you could choose the RE-tool more based on the non-functional characteristics (Wiegers, 1999).

More comprehensive tools?Some interviewed developing organizations (organization E, F and H) mention that more comprehensive tools, that focus more on the “the big picture”, are preferable (see chapter 4.2.1: RE-tool integration). They want to have all the tools in one tool e.g. requirements management, configuration management, change management, project planning and more. This tool should therefore support all the parts of the project. This is however not that simple to introduce for the RE-tool vendors without recreating the existing RE-tool.

There exist whole product suites e.g. Rational, but the developing organizations seem not to be that positive to those either since they can be a bit wieldy and expensive (organization H).

Requirements engineering and RE-tools in the futureThe developing organizations mention that the usage of requirements engineering and RE-tools will increase in the future. This is also rooted in most of the literature e.g. Karlsson (1998). In that case more people will come in contact with the RE-tool and demand better usability of the RE-tool, which may lead to even better usability and more customers. The RE-tool vendors should therefore focus more on usability.

It is also important to mention here that the developing organizations have answered that they are probably not using more than 50-60 % of the functions of the RE-tool. This is another reason for the RE-tool vendors to focus more on usability than detailed functionality. As mentioned in chapter 2.3, we can see that developing organizations seem to invest much money on features, which probably not are used by the greater part of the organizations anyway.

35

36

5 How to choose a suitable RE-tool

This chapter will focus on the selection of a suitable RE-tool for organizations, which are in the process of selecting RE-tools. First we take a look at some of the existing COTS selection methods. After that, we introduce the GRTS-method for selection of RE-tools, which includes guidelines for the selection and important things to think about before and during RE-tool acquisition.

5.1 Some existing COTS selection methods

RE-tools are a kind of COTS-tools. When constructing a method to select these RE-tools you of course have to look at the existing selection methods. There are a number of existing COTS selection methods available today which describe different aspects of selecting tools “off the shelf”. We will describe a number of them including the one on which our method has been influenced, namely SLATE (Social-Technical Approach to COTS Evaluation).

Information about different existing COTS selection methods has also been collected from Matulevicius (2003 and 2004). Matulevicius is also developing a specification exemplar for selection of RE-tools, which focuses on an analysis of the evaluation of the RE-tool before the acquisition to the organizational settings.

The OTSO (Off-The-Shelf Option) method (Kontio, 1996) was originally a way to make the COTS selection process easier. It consists of the following principles: The definitions of tasks in the selection process, the definition of the evaluation criteria, comparing the costs and value at each alternative and decision making methods to analyze the evaluation results. There are two main aspects of this method, namely, evaluation criteria definition and analysis of evaluation data. The first aspect will decompose the requirements for the COTS into a hierarchical criteria set and the second aspect will consolidate the evaluation data for decision-making purposes using the AHP (Analytic Hierarchy Process) technique, which was originally created by Thomas Saaty (1990).

The PORE (Procurement-Oriented Requirements Engineering) method (Maiden and Ncube, 1998) was originally created to propose a model of product-requirement compliance to provide a theoretical basis for technique selection and use. The PORE model integrates a number of techniques for requirements acquisition and product selection: Knowledge engineering techniques, feature analysis techniques, multi-criteria decision techniques and design rationale techniques. These techniques should iteratively be carried out until one or more products comply with a sufficient number of customer requirements. The stages of this process are presented in three template which provides guidance when acquiring the customer requirements and product information necessary to select and reject products, (1) as a result of supplier-given information, (2) from supplier-led demonstrations using test cases for individual requirements, (3) as a result of customer-led product exploration.

The study generic metrics to support if you are going to develop a product in-house or buy COTS components are suggested by (Sedigh-Ali, Ghafoor and Paul, 2001). In this way the organizations can with respect to the CMM (Capability Maturity Model) study the cost of software quality according to their own development environment. The authors of the article suggest these different metrics: Management (e.g. cost, time to market and software engineering environment), Quality (e.g. adaptability, complexity of interfaces and integration

37

and integration test coverage) and Requirements (e.g. requirements conformance and requirements stability).

The STACE (Social-Technical Approach to COTS Evaluation) method (Kunda and Brooks, 1999) was originally created to facilitate a simple, quick and inexpensive social-technical approach to COTS selection process. This method is mainly built on the work of (Kontio, 1996) and (Maiden and Ncube, 1998) described above and has the following principles: Support for a systematic approach to COTS evaluation and selection, support for evaluation of both COTS products and the underlying technology and use of social-technical techniques to improve the COTS software selection process.

The STACE framework itself consists of four interrelated processes, namely, requirements elicitation (include market studies, system documents, domain knowledge and stakeholders), social-technical criteria definition (include technology factors, functionality characteristics, quality characteristics and social-economic factors), alternatives identification (include customer participation, available alternatives in market and techniques such as market research, Internet research and fairs) and evaluation or assessment (include customer participation, evaluation strategy, data collection techniques and data analysis techniques).

5.2 The GRTS-method for RE-tools selection: Principles and Framework

There is a purpose for making a method to help organizations in their way of selecting RE-tools. Our method is developed to help organizations select an RE-tool that is more suitable than selecting one ad hoc and to save the time the organizations would spend on developing a new selection method (see chapter 4.2.1: Money spent on selection methods).

We have therefore developed the GRTS (Guidelines for RE-Tools Selection) – method, which is built mainly on the results from the conducted interviews with the developing organizations and RE-tool vendors, but also influenced by the STACE-method (Kunda and Brooks, 1999) for selection of COTS-tools. We have narrowed it down to make it focus on the selection of RE-tools, which is of importance here.

5.2.1 Principles

The GRTS-method is built on the results and conclusions that we can draw from the analysis of the conducted interviews, both organizations and RE-tool vendors. We have chosen to build this method on social and ethnographical aspects since the interviews have shown that these factors are more important than the functional aspects.

The social aspects are the main similarity to the STACE-method (Kunda and Brooks, 1999) in the way that this method also focuses more on the soft issues e.g. social aspects than the other methods described above. Differences are e.g. that this method focuses more on guidelines to help organizations and different techniques are presented that can be helpful.

The method does in a number of steps remind the organization of important issues concerning the acquisition of the RE-tool. We have decided to keep the method relatively short since the organizations have stated in the interviews that they do not want to spend much time on evaluating the RE-tools. The method is divided into four steps:

1. Evaluation of the current requirements process and the organization’s situation and maturity.

2. Criteria for RE-tool selection.

38

3. Gathering of information and selecting RE-tools for evaluation. 4. Evaluation and final selection of an RE-tool.

There are many important things to think about in these steps that can be difficult, which will be discussed in the different steps below.

The method does not choose a specific RE-tool for the organization but helps the organization with guidelines of the selection. This could shed some light over the questions the organization should ask them to get them in the right direction and narrow down the selection of available RE-tools on the market.

5.2.2 The GRTS Framework

The different steps of the GRTS-method are presented below. Where it is possible, connections to the interview results are made.

Step1: Evaluation of the current requirements process as well as the organization’s situation and maturity.

Some problems may occur if you start by choosing an RE-tool in an ad hoc manner. According to the interviews (see chapter 4.2.1: Selection of RE-tools) you should start with evaluating your current requirements process and the focus shall lie on how mature the organization is.

Evaluating the current requirements process and the organization:

What is the organization’s current situation regarding the requirements process? The organization will have to identify the problems and how they can be solved. It is necessary that the organization realizes the problems with the current situation and that it does not rely on the RE-tool to solve them. The RE-tool is there to help get rid of some of the problems and if the organization is not aware of the current process problems it will be difficult to improve the situation. A problem with introducing an RE-tool into the ongoing requirements process can be that the way some people work may be changed. The RE-tool is a central part of a requirements engineering process and it may demand a certain way of working, which is important to think about before choosing an RE-tool. If you cannot effectively elicit your problems, you might not see all the ways that the RE-tool might help in the work.

What is the organization’s level of maturity? It is possible to use the Capability Maturity Model, (SEI, 2004) or the Requirements engineering process maturity model (Sommerville and Sawyer, 1998), which focus on the requirements part, to evaluate the level of maturity of the organization. However, these models are extensive and time consuming. It is important though, that some sort of evaluation is conducted to see if the organization really has the need for an RE-tool. The level of maturity itself is not that important, it is the evaluation itself that is important and which will give an understanding and insight in the organization and its needs. In this method it is of high importance that the evaluation is made in an objective way. A problem that we have learned from the interviews is that organizations tend to focus on where they want to be instead of where they really are, which can lead to that they do not see all problems.

39

What will it cost to introduce an RE-tool? There are also the economic factors to think about before choosing to introduce an RE-tool (see chapter 4.4: Price). Of course, it will cost the organization to buy the necessary licenses to work with the RE-tool. There will however be other charges, such as, the cost of training the personnel, installation, support and upgrades. These things could be a part of the contract but if they are not, it is important to consider them. If an organization decides to introduce an RE-tool, the requirements process will perhaps have to be changed and it will cost time to introduce a new process adapted to the RE-tool. A cost/benefit analysis could be made to see if the investment in a new RE-tool is profitable.

What ethnographic factors are there to consider? The current working situation almost certainly changes when introducing an RE-tool, so the ethnographic aspects are important to consider (see chapter 3.2.1.2.1: Ethnography). The personnel could be opposed to this introduction because they are used to a certain way of doing things and with the introduction they could be forced to learn the new RE-tool. This might change the work assignments of some persons. This means that the RE-tool introduction can help people with work but also make them less important and even threaten them by making their tasks obsolete. This may also threat some people’s influence in decision-making, thereby making them reluctant of the RE-tool.

Step2: Criteria for RE-tool selection.

According to the interviews with organizations D and F (see appendix C), it is stated that if you know what problems exist in the current requirements process and the organization it is easier to see what the RE-tool can do to help. In this step these problems are a part of the criteria that should be defined. It is also important to think about what the RE-tool should be able to do to function in the domain where it is going to operate. This is also going to produce a number of criteria that should be defined in this step.

It is preferable if the organization does not look at the RE-tools before setting up the criteria because then there is the possibility that the criteria will be influenced by the RE-tool/RE-tools that have been evaluated. You should instead define both the “should have” criteria and the “nice to have” criteria before you conduct the initial screening of the existing RE-tools.

“Should have” criteria are the things that should be fulfilled by the RE-tool so the organization can use it. One example could be that the organization policy dictates that a specific server should be used in the whole organization. If the RE-tool does not support this it will disappear in the initial screening. Another example could be legal issues, such as, that the RE-tool must comply with security policies, for instance, support different levels of access to data files. An example of what the RE-tool can help us with could be that if the evaluation shows that change management is a big problem in the current requirements process then the criteria is that the RE-tool should support change management.

“Nice to have” criteria are the things that we think it is nice if the RE-tool could help us with and could be that the RE-tool should support different traceability graphs or different text formats etc. There is however a thin line between “should have” criteria and “nice to have” criteria. What are “should have” criteria for one organization could be “nice to have” criteria for another. Performance is an example that could be very important for one organization while another organization does not care much about performance since their biggest area of interest is security.

40

This method chooses to divide the requirements in nine criteria groups, selected from the literature e.g. Lauesen, (2002), Briand (1998) and Dean (1999). These criteria groups are chosen to get a broad variety of characteristics and most part of them have been mentioned during the interviews with the organizations. One example is that organizations A, D and I are in the medical and defence industries and have stated during the interviews that they must follow certain standards and conventions when developing the products. Therefore it is important for these organizations that these criteria are supported by the RE-tool. Another example is that organizations C and H have during the interviews mentioned that the RE-tool should support different environments, such as, Windows, Unix and Linux. A third example is that organization I mentioned that the RE-tool should be able to function for a long time without problems and that the RE-tool should be effective. Therefore the performance criteria are important here.

Functionality characteristics: The requirements that the technology of the RE-tool should support, for instance, managing objects, status of requirements, traceability, managing requirements.

Standards: The requirements on the standards, for instance, conventions, laws and other organization policies.

Integration characteristics: The requirements on the integration i.e. which other tools the RE-tool should be able to integrate with.

Interface/platform characteristics: The requirements on the interface and platform i.e. they describe how the interface should be structured and which platforms the RE-tool should support to work with.

Security factors: The requirements on the security. There are a number of different types; personal access to software, protection against computer intrusion and viruses, responsibility of disasters (disk crashes, etc).

Vendor characteristics: The requirements on the vendor of the RE-tool, for instance, maintenance agreements, support training of operators, etc.

Information factors: The requirements on the information provided by the developer, for instance, documentation on the developer’s website, tutorials and downloadable trial versions.

Performance characteristics: The requirements on the performance, for instance, capacity to store data, correctness of program, efficiency (speed, etc).

Usability factors: The requirements on the usability of the RE-tool; for instance, a large number of the persons in an organization shall be able to use the RE-tool.

If organizations come up with other criteria depending on their business, they can of course add these to the list we have created. Depending on the business of the organizations, the focus on these criteria groups will vary much.

If prioritization of the criteria groups or the criteria within the groups is necessary, the AHP (The Analytic Hierarchy Process) could be used for the pair-wise comparisons if the number of comparisons is small. The comparisons should be prioritized according to the value it has for the organization. The AHP is described thoroughly in Saaty (1990) and is built on a pair wise comparison of alternatives with respect to each selection “criterion” and a pair wise weighting of selection “criteria” (Briand, 1998). This method is also suggested for pair-wise comparisons in the OTSO-method (Kontio, 1996) for selection of COTS-products.

It is also possible to keep the prioritization of the criteria groups simple. You just put the most important criteria group at the top, the next below and so on. However, in each criteria group it can be useful to grade each criterion. An example is presented below:

41

A medical organization is going to buy an RE-tool. The criteria groups that are going to be prioritized are security, usability and information factors. To keep this simple, only two criteria in each criteria group are presented. Table 5.1 shows the criteria groups with the criteria and how they are prioritized. The grades are from 1-5, where 5 is the highest. The grades 4 and 5 are set as “should have” criteria and 1-3 are set as “nice to have” criteria.

Criteria groups and criteria GradeSecurityPersonal access 5Protection against occasional shutdown 3UsabilityEasy to operate for the personnel 4Easy to overview 3Information factorsGood documentation 3Downloadable trial version 2Table 5.2. Example of criteria prioritization

We have created a table in appendix B as an example of which criteria that can be used. The criteria are not prioritized however. This can hopefully be helpful to the organizations if they cannot come up with these themselves.

Step3: Gathering of information and selecting RE-tools for further evaluation.

In this step we will describe some different techniques that can be used to get information of RE-tools, both which RE-tools that exist and valuable information about these RE-tools. First of all, which RE-tools exist on the market today? There are a number of different techniques, which can be used to get this information, some have been mentioned during the interviews e.g. organization B has stated that they would study other organization in the same business and organization G has stated that they have downloaded trial versions from the RE-tool vendors’ websites. Below we have listed examples of techniques for gathering information.

Gather information:

Search the Internet: With the possibilities of the Internet of today, perhaps this the obvious way. You can just type some word and search your way forward. It is very easy to get information this way and it is free but the Internet also has false information and the articles are not always objective. An example of an independent analysis organization is http://www.incose.org. See also the reference list in chapter 8.

Known tool developers: Perhaps the organization already has some different tools and the vendors of these tools can be helpful to come up with suggestions. You already have contact with this vendor and you know the quality of their work but they can turn out to be subjective since they perhaps only come up with the RE-tools that are compatible with their tools.

Organization personnel: Some of the personnel of the organization could already know of RE-tools from old jobs or of interest. These persons could have worked with RE-tools before and have knowledge of the RE-tools. These persons could also know about other RE-tools on the market.

Technology journals: From technology journals valuable information could be provided. The articles presented could provide much information about which RE-tools exist on the

42

market and the articles should be correct since they have been reviewed before printed. A problem can be however that it could be difficult to find the most suitable journals.

Other organizations: The organization could get objective opinions from other organizations, such as universities, research institutes, etc. The universities could have the information you are looking for or may know where to find it.

Study other organizations: You can also study other organizations in your business area and look at which RE-tools they are using. A problem may be that other organizations do not always want to share this information.

When you have found out, which RE-tools exist, you should try to gather the information you want about these RE-tools. The techniques of getting information about which RE-tools exist can also be used when you want to gather information about the functionality of the RE-tools. Most of these techniques have been used by the authors when gathering information of the RE-tools. This information is summarized in the RE-tool survey in appendix A and in the table in appendix B. There are also more specific techniques to gather this information:

Study documentation: Much information is no doubt written about these RE-tools and you have to be careful when gathering this information. Facts could be found both on the developer’s web page and in independent research articles published in journals or in conference proceedings. The information is always built on the author’s subjective opinion about the RE-tools. A drawback is also that you do not get to see the product. However, this will get a good overview of the RE-tool, which will be very helpful in the continuing search.

Trial versions: A good way to find objective information about some RE-tool is to download a trial version from the developer’s web page and try the RE-tool for yourself. This is an excellent way to try the criteria you have defined in step 2 and gather the information required. However, not all functions may be included in these trial versions.

RE-tool demonstrations: Another way to gather information is to let the vendor conduct an RE-tool demonstration and show the RE-tool. This is also a good way to get an overview of the RE-tool and you can also pose questions to the vendor. The RE-tool vendor may know a lot about the RE-tool but his opinions could be subjective.

The screening process:

The RE-tools that shall be evaluated should now be selected according to the “should have” criteria described above. If this screening turns out that you get too many RE-tools you have to narrow it down further. This method recommends a maximum of 5 RE-tools. You could choose the RE-tools that you have most information about or you could add some more “should have” criteria from the most important “nice to have” criteria to narrow down the selection. If no RE-tools match the “should have” criteria, you have to look at the “should have” criteria and see if there is anything that is not absolutely necessary and could be converted to “nice to have” criteria. Another choice is to look for alternative solutions, such as developing your own RE-tool or order a tailor-made solution from a vendor.

The example described in step 2 continues. The medical organization has found 3 RE-tools: A, B and C and gathered information about the RE-tools. The screening process is now conducted. The screening shows that all three RE-tools are easy to use but RE-tool C does not support personal access. This RE-tool is taken out and is not going to be further evaluated. The RE-tools A and B are now going to be further evaluated in step 4.

43

Step4: The evaluation and final selection of an RE-tool.

In this step the evaluation of the selected RE-tools should be conducted. The evaluation should be built on the “nice to have” criteria defined in step 2, since the “should have” criteria are already fulfilled by the RE-tools. As in step 2 it is possible to use the AHP to get an opinion of how much better one RE-tool fulfils a certain criteria than another, but only if the number of comparisons is small. In this way it is easier to compare the RE-tools than if you just put a marking that the RE-tool supports a specific feature. Another possibility is to put grades on how well the RE-tools support each criterion. This takes less time than the AHP. In the table in appendix B we have created an example of how criteria could be chosen but the RE-tools are not compared with each other to see which is better.

This method suggests that the final selection of an RE-tool should be performed depending on this evaluation, where both the “should have” criteria and the “nice to have” criteria are influenced.

If we continue the example described above it is now time to evaluate the RE-tools according to the information gathered in step 3 and put grades on how well each criterion is supported by the RE-tool. We have chosen to use the quicker prioritization method because some organizations have stated during the interviews that they do not want to spend much time on evaluation. The grades are 1-5, where 5 is the highest. This grade is weighed against the importance of each criterion and is summarized together with the rest of the grades. This is done for each RE-tool and the RE-tool with the highest score should be chosen. In table 5.2 an example of how the grades could be set for RE-tools A and B is presented.

Criteria groups and criteria RE-tool A RE-tool BSecurityPersonal access (5) (5*) 4 = 20 (5*) 4 = 20Protection against occasional shutdown (3) (3*) 5 = 15 (3*) 3 = 9UsabilityEasy to operate for the personnel (4) (4*) 4 = 16 (4*) 5 = 20Easy to overview (3) (3*) 4 = 12 (3*) 3 = 9Information factorsGood documentation (3) (3*) 4 = 12 (3*) 3 = 9Downloadable trial version (2) (2*) 2 = 4 (2*) 3 = 6Score 79 73Table 5.3. The evaluation of the RE-tools

The numbers in parenthesis are the grades of importance and the number after the parenthesis is the grade of how well the criteria is supported by the RE-tool. The total score for RE-tool A is 79 and RE-tool B, 73. RE-tool A has the highest score and should be chosen according to the method.

To make this method more understandable, we have created figure 5.1, where we try to explain the GRTS-method as a flow diagram divided in the four steps.

44

process Untitled 1(1)

Start

Evaluate current process and organization

Define "should have" criteria

Define "nice to have" criteria

Gather information about the RE-tools

Screening process according to "should have" criteria

Candidates exist?

Loosen the "should have" criteria

More than five candidates?

Add "should have" criteria

Evaluate RE-tools according to "should have" and "nice to have" criteria

Choose RE-tool

End

Yes

No

Figure 5.7. The GRTS process

45

Step1

Step2

Step3

Step4

46

6 Evaluation of the GRTS-method

To evaluate the GRTS-method, we have decided to send the method to some of the interviewed organizations to get some response. The ideal would of course be to let an organization, which is the process of choosing an RE-tool, select according to the method. This is however outside the scope of this thesis, so the next best thing is to let the interviewed organizations evaluate it. We sent the method along with a number of questions to be answered. First we present the scenario and the questions that we have sent some of the organizations and then the response to these questions.

6.1 Evaluation scenario and questions

This is the scenario:

The management of the organization has decided that an RE-tool will be introduced/changed. You have no new candidate (RE-tool) at this moment. A screening and an evaluation of a number of RE-tools shall be conducted. Can this method be used for your organization?

After reading through the method, a number of questions will be answered:

1. How is this method compared with choosing an RE-tool ad hoc?2. How is this method compared with the way you have chosen an RE-tool in the past?3. Do you think that the method would work to select a suitable RE-tool for you?4. What flaws do you see in the method?5. What positive aspects are there with the method?6. Are there aspects that the method does not cover?7. What similarities exist between the method and the way/ways you have selected tools?8. What differences exist between the method and the way/ways you have selected tools?9. Is something unclear with the method?10. Would you recommend someone else to use the method, and why/why not?

6.2 Response to evaluation

The questions of this evaluation have been sent to a number of the interviewed organizations. The response to the evaluation questions is summarized in a number of points where we describe the answers of the organizations. The full answers are presented in appendix E.

Impression and thoughts about the methodAll of the organizations think that the method would work well in comparison with selecting an RE-tool ad hoc. One organization states that with this method some of the pitfalls probably could be avoided which lead to that the selection must be remade. The organizations seem also to think that the method probably would work to select an RE-tool for their organization too and that they would recommend this method to be used by other organizations.

This method in comparison with current selection methodsThe organizations seem to think that this method agrees somewhat to the method/methods used by their organizations when selecting an RE-tool. This method is however a little more extensive, which probably makes it a little better. The main similarities are that some

47

organizations also set criteria of what the RE-tool should support and be compared against and that the examination principles also seem to agree.

Shortages and indistinctnessThe organizations seem to think that the criteria table in appendix B is not entirely complete and that some comparisons between performance and price are not included. They also think that if examples of criteria are included then it is possible that organizations follow these examples instead of thinking for themselves. Another thing is that some sort of manual, i.e. how to think when setting up the criteria for selection of RE-tools, should be included. The administration of the RE-tool, such as release upgrades, which can lead to a new workflow, is perhaps not fully covered in the method.

Positive aspectsA positive aspect is that this method is concise and that it is a larger probability that you choose correctly from the beginning. If this method is used it is a larger probability that more thoroughly work is done and that the people who really need a structured method can use it. Another important aspect is that this method is documented, which makes it possible to use the next and next time an RE-tool should be selected.

48

7 Conclusions and future work

In this chapter we provide the conclusions of this thesis. First the purpose for conducting this thesis is discussed. After that a short analysis of the interview results, which led to the GRTS-method, are presented. The principles and the four steps of the GRTS-method are presented in a shorter version (fully covered in chapter 5.2.2) and finally we present the evaluation of the GRTS-method (conducted by some of the interviewed organizations) and some future work in this area.

PurposeThe purpose of this thesis is to provide guidelines, which can help organizations to select a suitable RE-tool. We choose to give the organizations’ point of view and therefore a number of interviews are conducted with organizations of different size, age and business areas. To compare the wishes of the organizations with the thoughts of the RE-tool vendors, a number of RE-tool vendors are also interviewed.

ResultsA cross-analysis of these interviews shows some interesting things. The RE-tool vendors state that they focus much on RE-tool integration but the developing organizations are not satisfied. They want integration that is not depending on converting files for import and export. There exists a gap that the vendors must fill by making better integration.

Both the developing organizations and the RE-tool vendors state that requirements engineering and RE-tools are going to increase in the future, and therefore more people come in contact with the RE-tool, also the people who use the RE-tool seldom. This leads to that usability is going to get more important in the future. The developing organizations also state that the special advanced functions of the RE-tool do not matter that much; they want an RE-tool that is easy to use. RE-tools could turn out to be more expensive with the many advanced functions and if the organizations do not care about these functions anyway, maybe it will be a good idea for the RE-tool vendors to focus more on usability instead. This way the price of the RE-tool will probably decrease.

The developing organizations state that they want more comprehensive tools that focus more on “the big picture”. These tools should support all activities of the project, such as, project planning, configuration management, change management, requirement management and more. They also want these tools to be easy to use, cheap and that they do not require a large administration staff. This sound like a utopia but is an interesting thought.

The final thing is that the developing organizations state that they are satisfied with the RE-tools although they have some deficiencies.

The results from these interviews together with a study of some existing COTS selection methods were used as input to the GRTS-method. The GRTS-method is divided into four steps and focuses more on the social and ethnographic aspects than functionality. Since we have learned from the interviews that the developing organizations do not seem too eager to spend much money on selecting RE-tools either, we have constructed the method to be adjustable to how much time the organizations want to spend.

The GRTS-method is divided into the following steps:

An evaluation of the current requirements process and organization: We believe that the first thing that should be done is to study the own organization and its requirements process to see if the organization is ready for an RE-tool with respect to a number of factors, such as, the

49

organization’s level of maturity and price of the RE-tool. Perhaps a new RE-tool is not needed, it is sufficient to reorganize the requirements process or you can use some already existing tools.

Define criteria for the selection:In this step the definition of the criteria should be conducted. The criteria should be defined as “should have” criteria and “nice to have” criteria. The “should have” criteria are the criteria that should be fulfilled by the RE-tool in order to be able to use it e.g. security factors or standards and the “nice to have” criteria are the criteria that it is nice if the RE-tool could fulfil e.g. functionality (status tracking, traceability). The criteria can be divided into criteria groups as described in chapter 5.2.2: Step 2. This is just an example and the organization can also come up with other criteria groups that can suit them better. It is also possible to prioritize the criteria groups and the criteria within the groups for importance.

Gather information about the RE-tools and the screening process To gather information about which RE-tools exist and information about the RE-tools there are a number of techniques (see chapter 5.2.2: Step 3) that can be used, e.g. search the Internet, study technology journals or download trial versions of the RE-tools. When you feel you have sufficient information, the screening process should be conducted according to the “should have” criteria defined in step 2. This method recommends that 1-5 RE-tools should be evaluated in step 4. If you have too many RE-tools after the screening, you can e.g. add some more “should have” criteria from the most important “nice to have” criteria. If you get no RE-tools at all, you can see if there are any “should have” criteria that are not absolutely necessary and that can be changed to “nice to have” criteria or alternative solutions can be used. These can be e.g. developing your own RE-tool or buy a tailor-made RE-tool from some vendor.

An evaluation of the candidates and a final selectionIn the fourth step, the evaluation of the RE-tools is conducted. This evaluation should be conducted according to the “nice to have” criteria since the “should have” criteria are already fulfilled by the RE-tools. As in step 2, it is possible and preferable if the criteria are prioritized of importance and the RE-tools are prioritized based on how well the RE-tool supports these criteria (see example in chapter 5.2.2: Step 4). If the importance of the criterion is multiplied with how well the RE-tool supports this criterion, you get a result for each criterion. If these results are summarized for all criteria, the RE-tool gets a total score. This method suggests that the RE-tool with the highest score should be selected.

EvaluationThe evaluation of the GRTS-method has shown that this method is preferable to using ad hoc methods when selecting RE-tools and that the method covers most aspects of the selection of RE-tools.

We have chosen to introduce an example table of the criteria that can be set when choosing RE-tools and this could have both positive and negative influence. Perhaps this criteria table is not fully covered and some aspects are missing but as we have clearly stated, it is just an example and if organizations want other criteria, they can add these too. One important thing is that this method is documented, which means that it can be used over and over again.

50

Future workIt had been better if the GRTS-method could be used in an organization that actually was going to select an RE-tool but that was outside the scope of this thesis. In the future, such an evaluation of the GRTS-method can provide a better judgement and the things that must be improved are shown.

The more comprehensive tools mentioned by the organizations are not evaluated in this thesis and future work in this area can perhaps also provide better answers to if these more comprehensive tools also could be used in a satisfying way for managing requirements. This can be done by evaluating different tools and compare e.g. integration, usability and administration work. Other future work is to make a larger evaluation of the RE-tools (more RE-tools and more evaluation criteria).

51

52

8 References

Books

Boehm, B.W., (1981), Software Engineering Economics, Englewood Cliffs, NJ: Prentice-Hall.

Booch, G., (1994), Object-oriented Analysis and Design with Applications, Redwood City,CA: Benjamin Cummings.

Cohen, L., (1995), Quality Function Deployment: How to Make QFD Work for You, Reading,MA: Addison-Wesley Publishing Organization.

Davis, A. M., (1993), Software requirements: Objects, Functions, and States, EnglewoodCliffs, NJ: Prentice-Hall.

Dawson, C. W., (2000), The Essence of Computing Projects: A Student’s Guide, Essex,England: Pearson Education Limited.

Jones, C., (1994), Assessment and Control of Software Risks, Englewood Cliffs, NJ: PTRPrentice-Hall.

Kotonya, G., Sommerville, I., (1998), Requirements Engineering: Processes andTechniques, Chichester, UK: John Wiley and Sons.

Kovitz, B. L., (1999), Practical Software Requirements: A Manual of Content and Style,Greenwich, CT: Manning Publications Co.

Lauesen, S., (2002), Software requirements: Styles and techniques, Harlow, England:Pearson Education Limited.

Rumbaugh, J., Blaha, M., et al, (1991), Object-oriented Modelling and Design, EnglewoodCliffs, NJ: Prentice-Hall.

Saaty, T.L., (1990), The Analytic Hierarchy Process, McGraw-Hill, New York.Sommerville, I., (2001), Software engineering, 6th edition, Essex, England: Pearson

Education Limited.Sommerville, I., Sawyer, P., (1997), Requirements engineering: A good practice guide,

Chichester: John Wiley and Sons.Thayer, R. H., Dorfman, M., (1997), Software requirements engineering, 2nd Ed, Los

Alamitos, CA: IEEE Computer Society Press.Wiegers, K. E., (1999), Software requirements, Redmond, Washington: Microsoft Press.

Articles and other literature

Alexander, I., Taborda, L.J., Göhlman, D., (2002), Ta kontroll över kraven, Dagens IndustriLedarskapshandböcker, IT Management 3.6 pp. 1-9, November.

Briand, L., (1998), COTS Evaluation and Selection, Software Maintenance, Proceedings,International Conference, pp. 222-223.

Dean, J.C., (1999), Ensuring the Capability of COTS products, Computer Software andApplications Conference, COMPSAC '99, Proceedings, The 23rd AnnualInternational, pp. 96-97.  

Helldén, K., Stenborg, B., (1999), Två kravhanteringsverktyg, A course in CASE (TDDB15)at The university of Linköping, http://www.ida.liu.se/~andha/datalogi-utv-

slutrapport.pdf.Last visited: 2004-05-26.

Heninger, K. L., (1980), Specifying Software Requirements for Complex Systems, NewTechniques and their Applications, IEEE Trans. on Software Engineering, SE-6(1), pp. 2-13.

53

IEEE, (1990), IEEE Std 610.12-1990, IEEE Standard Glossary of Software EngineeringTechnology.

IEEE, (1998), IEEE Std 830-1998: IEEE Recommended Practice for Software RequirementsSpecifications, http://users.snip.net/~gbooker/INFO627/IEEE-830-1998.pdf. Last visited:2004-05-26.

INCOSE Tools Database Working Group (DBWG), Requirements Management ToolsSurvey, http://www.incose.org. Last visited: 2004-05-03.

INCOSE Requirements Management Technology Overview,http://www.incose.org/tools/reqsmgmt.html. Last visited: 2004-05-03.

Integrated Chipware Inc, (2001), RTM Product Datasheet Overview,http://www.chipware.com. Last visited: 2004-05-03.

Johnson, M., (2002), Towards Web-based Requirements Management: A Case Study in aDistributed Software Development Organization.http://www.soberit.hut.fi/~johnson/Johnson_HUT2002.pdf. Last visited: 2004-05-26.

Jung, H. W., (1998), Optimizing Value and Cost in Requirements Analysis, IEEE Software15 (4).

Karlsson, J., Ryan, K., (1997), A Cost-Value Approach for Prioritizing Requirements, IEEESoftware, pp. 67-74, September/October 1997.

Karlsson, J., (1998), Framgångsrik kravhantering – vid utveckling av programvarusystem,Sveriges Verkstadsindustrier, Rapport Informationsteknologi.

Karlsson, J., (2003), Marknadsdriven produktledning – från kundbehov och krav tilllönsamma produkter, Sveriges Verkstadsindustrier, Rapport Informationsteknologi.

Kontio, J., (1996), A Case Study in Applying a Systematic Method for COTS Selection, Inproceedings of the 18th International Conference on Software Engineering (ICSE’96),IEEE.

Kotonya, G., Sommerville, I., (1996), Requirements Engineering with Viewpoints,BCS/IEE Software Engineering J., pp. 5-18.

Kunda, D., Brooks, L. (1999), Applying Social-Technical Approach for COTS Selection, Inproceedings of the 4th UKAIS conference.

Maiden, N., Ncube, C., (1998), Acquiring COTS Software Selection Requirements, IEEESoftware, pp. 46-56.

Manzoni, L.V., Price, R.T., (2003), Identifying Extensions Required by RUP (RationalUnified Process) to Comply with CMM (Capability Maturity Model) Levels 2 and 3, IEEETransactions on Software Engineering vol. 29, no. 2, Feb. 2003.

Matulevicius, R., (2003), Validating an Evaluation Framework for Requirement EngineeringTools, In proceedings of the 8th CaiSE/IFIP8.1 International Workshop on Evaluation of Modelling Methods in Systems Analysis and Design (EMMSAD’03), pp. 84-93.

Matulevicius, R., (2004), How Requirements Specification Quality Depends on Tools: A CaseStudy, Accepted for publication in proceedings of the 16th International Conference on Advanced Information System Engineering (CaiSE’2004), Riga, June.

Robinson, M. S., (2001), Requirements Management Tools: A Trade Study.http://www.ess.stsci.edu/fset/projects/Sabbatical/MSRReport.pdf. Last visited: 2004-

05-26.

SEI (Software Engineering Institute), http://www.sei.cmu.edu/cmm/cmm.html. Last visited:2004-05-03.

Sedigh-Ali, S., Ghafoor, A., Paul, R., (2001), Software Engineering Metrics for COTS-basedSystems, IEEE Computer, 34 (5), pp. 44-50.

Singer, C.A., (1992), A Requirements Tutorial. Quality Systems and Software Requirements,Special Report SR-NWT-002159, Bellcore.

54

Standish Group, The, (1995), The CHAOS Report, Dennis, MA: The Standish GroupInternational, Inc. http://www.projectsmart.co.uk/docs/chaos_report.pdf. Last visited:2004-05-26.

Standish Group, The, (2002), What are your requirements? A Standish Group Research Note.http://www.standishgroup.com. Last visited: 2004-05-26.

Volere, (2003), http://www.volere.co.uk/tools.htm. Last visited: 2004-05-03.Wiegers, K. E., (1999), Automating Requirements Management, Software Development, vol.

7, no. 7.Wiktorin, L., (2003), Kravhantering, http://www.itplan.se/ITA%20kravhantering.pdf. Last

visited: 2004-05-26. Yphise, (2002), Requirements Management Tools: A Software Assessment Report.

http://www.yphise.com. Last visited: 2004-05-03.

55

56

Appendix A. RE-tool survey

Information about the tools has been collected from various research papers, independent tool reviewers, vendor’s web pages, books, interviews, tutorials and analysis of the RE-tools. Some of the RE-tools have been tested and evaluated with licences provided to us by the department. The structure and functions of the different RE-tools are presented with pros and cons and an analysis.

DOORS

DOORS (Dynamic Object Oriented Requirements System) is made by Telelogic and is a multi-platform, enterprise-wide requirements management system used by systems engineers, software developers and product managers to manage and control complex systems development projects (Standish Group, 2002). It has a large user base (50000+ at 1200 locations).

The RE-tool supports environments as Windows 95, 98, NT, Solaris, Hewlett Packard-UX, IBM AIX and Digital UNIX (Wiegers, 1999).

Requirements are managed within DOORS as discrete objects, which can be managed separately. Wiegers (1999) find it a little bit hard to edit a requirement’s attributes, you must select the requirement, then select the attribute to edit and finally edit the values in another dialog box. DOORS uses a folder structure of requirements, quite similar to Windows Explorer. DOORS includes an on-line change proposal and review system that let users submit proposed changes to requirements, including a justification.

The DOORS Extension Language (DXL) is a high level C-like language that provides access to virtually all DOORS functions for user extensions and customization.

The program is the only one that provides an X-Windows interface. This lets the user choose his work interface (Web and Windows platforms can also be used) (Yphise, 2002).

Users can define their own attributes and choose which should be viewed. The RE-tool supports that attributes can be both created and deleted. These updated attributes are used to visualize the requirements status through various views, e.g. business views. This can make tracking fulfilment of the requirements easier for the different users in the project.

The program ensures the completeness of the requirements by listing any mismatch between the original document and the requirements list. Thereby it ensures that all of the original requirements have been analysed.

The layout of the requirements can be controlled by character formatting (such as underline and colours). Since DOORS can make use of external requirements documents in formats, such as, WordPerfect and PostScript it can thereby provide the creation and integration of already defined requirements.

Access can be controlled on user basis on attribute level. Impact and traceability reports as well as reports identifying missing links, are all available across all levels or phases of a project life cycle.

57

DOORS has a change validation process that ensures that major exchanges are not made before all users involved has been notified and seen the proposed changes. However, the RE-tool does not support e-mail notification of changes to affected users.

Tests can be created as database objects, and requirements linked to them. Documents can be produced in a Word format.

Some of the tools it integrates with are Allfusion Component Modeller and Tau UML as modelling tools and as managing tools, TestExpert and Tplan. DOORS also integrates with IBM tools, such as, Rational Rose but a third party tool may do the integration.

We think that this RE-tool is easy to overview with different views, in which you can look at attributes and much more. Different users can choose what to look at in these views. The program can easily be changed to fit your needs. Traceability is a useful feature and you can create traceability matrices where dependencies between different requirements can be viewed. The RE-tool does not support the use of external servers, but it will however be able to in the next version.

CaliberRM

CaliberRM is made by Borland, which is a major producer (although newer than Rational and RTM) of RE-tools. The RE-tool is a web-based system that is able to support collaborative development via a group discussion feature (Standish Group, 2002). This ensures durability and integration into the vendors’ development environment.

The RE-tool support Windows 95, 98, NT client and NT server (Wiegers, 1999). The RE-tool does not support the integration with UNIX so it has to be used through a web access tool or through a Windows emulator (Robinson, 2001).

CaliberRM has a folder structure like Windows Explorer, which enables the users to organize the project, as they like. The product is suitable for users who need to personalize their work interface (Yphise, 2001). This is one of the strengths of the RE-tool.

Changes made to requirement data, such as, traceability, document references, status, user responsibility and more are recorded in CaliberRM's central repository (Volere, 2003). CaliberRM is not able to manage repositories installed on multiple servers and searches on various projects. Only on project level is full text search permitted.

Users can define new requirements attributes or delete unused attributes. Requirements can be imported from Word. CaliberRM allows you to create traceability links to see relationships between requirements and other related development and testing information. Requirements can link from and to any other requirement within the same project and across projects.

CaliberRM supports multiple requirements types for each project. A requirement has two numbers, namely a hierarchical and a serial. The hierarchical number changes when a requirement is changed, moved or deleted, however, the serial number will not be changed even if a requirement is moved, since it is a unique identifier of the requirement.

58

CaliberRM provides dictionaries in various languages and it eases the analysis of the contents of the requirements. Extra dictionaries in medical and legal fields are also a good addition. CaliberRM also enables team members to quickly identify potential requirement problems by highlighting ambiguous and commonly used terms defined in a shared glossary (Volere, 2003).

CaliberRM has a central communication Framework Administrator, which is called SQM, Software Quality Management. The Framework Administrator controls administrative functions, including creating and modifying users, groups and projects, administering security and setting up automatic e-mail notification.

E-mail exchanges can be automatically captured and e-mail notification is supported. CaliberRM does not have ability to manage requirements change request. This makes it unsuitable for projects where changes must be validated before they are effective without the need to rely on an external tool.

Security features reach down to the requirements attributes level, so a user cannot modify or delete information if he does not have access permission. Test objects, which can be linked to, are a standard feature in the RE-tool. CaliberRM allows project teams to provide input on requirements via standard browsers and remote clients can access the system through an Internet connection.

CaliberRM integrates well with Mercury Test Director and Rational Rose. CaliberRM supports reference linking between different document types, such as, Word, Excel, graphical files, HTML files and audio and video files to get additional information. This is so developers and users can better understand requirements.

CaliberRM has a downloadable trial version on their website which makes it possible to examine the RE-tool for yourself. Changing requirements is easy in CaliberRM. You can just change the information you want and save it. Whenever you change something a comment dialog box appears on the screen where you can write the changes and why and when you made them.

RequisitePro

RequisitePro is made by Rational, which is now owned by IBM. RequisitePro is an RE-tool designed for multi-user environments.

The RE-tool supports platforms Windows 98/NT/XP/2000. If the user wants to run the RE-tool in for instance Unix, this is a major drawback. It relies on Microsoft Access, which limits its scalability.

The RE-tool has a close connection to Word, which can be an advantage since people of an organization are usually well oriented in working with Word. Requisite Pro has a web interface with the same features as Windows. This is one of the strengths of the program. However one cannot move requirements in the RE-tool.

The RE-tool features integration of Microsoft Word and a requirements database (Oracle, Microsoft SQL Server, and Microsoft Access), which allow users to organize, prioritize, and trace relationships between requirements. Software project teams can gather, enter and

59

manage requirements within the documents or in a database. RequisitePro includes templates to simplify production of requirements documents.

The web interface can, by editing a CSS file, be customized. Thereby it can be used in projects with members spread out all over the world without installing a large client (Yphise 2002). It can be hard to share projects among more than one manager. Then the directory on the disk containing the project database must be shared and problems can arise when more than one user is writing to a single database file.

Each project can be configured to fit the user’s needs. It is the structure and interface that makes this possible (Robinson, 2001). Requisite Pro lacks the ability to add requirements in the word processor document after they are added in the RE-tool interface. The program is therefore not suited for projects where numerous requirements can be added through the RE-tool interface (Yphise, 2002).

Traceability is displayed both textually and graphically. Related requirements can be linked together, so that as changes occur to one requirement, users can easily see its impact on other related requirements. Colours and symbols indicate the status of the requirement (Volere, 2003). A major drawback is the lack of linking to external documents, graphs, and spreadsheets (Robinson, 2001).

RequisitePro keeps track of the history of changes in a document. However users must search by themselves in the history. The RE-tool can highlight requirements that are new or have been changed. It allows the user to filter those requirements, and show all requirements that are fulfilled and changed. Security is made so that user access can be limited to attributes level (Robinson, 2001).

RequisitePro is an RE-tool that is designed to be both easy to use and easy to learn. It has two main segments, a floating tool palette and a view workplace. The floating palette is used for creating new requirements and modifying the requirements. The view workplace can be accessed from the palette. The workplace is used for viewing the requirements in tabled matrices or trees.

RequisitePro integrates of course with Rational Rose and other Rational software. However, all other integrations must happen through API (Robinson, 2001).

We do not think that the RE-tool was easy to overview. There are many different views and windows. You have to know which buttons to use to know your way around to RE-tool interface. The tutorial was fairly easy to follow and provides good overall information about the RE-tool. There is a downloadable trial version on the vendor’s web site and it is quite simple to install and start. This is a good way to examine and evaluate the RE-tool.

RTM Workshop

RTM (Requirements Traceability and Management) Workshop is made by Integrated Chipware, Inc.

This RE-tool supports Unix and Windows and some Oracle database platforms.

RTM can define both user groups and sub-groups. This is very practical when strict hierarchal structured organizations are using it. Projects are placed in a folder structure. Interaction with

60

database occurs through Word or Freemaker or other user defined forms (Robinson, 2001). The program can add new requirements or replace existing ones. The program has a weakness in its inability to control the layout of its requirements (Yphise, 2002).

RTM is able to import existing requirements documents and supports e-mails and files to be attached to e-mails as documents associated with the requirements. RTM has great flexibility in importing requirements from different types of documents, and it can export requirements in many file formats (Wiegers, 1999).

When a change is made, all of the affected users are informed and the workflow also provides for that the management of a confirmation is sent to the requester, so he can see that the right persons have been notified of the change in the project. Change request capability allows users to propose and review changes to the current baseline requirements from within RTM. An information modelling capability allows users to design change records or problem reports and associate them with specific requirements data (Volere, 2003).

RTM supports distributing the requirements projects among various repositories. It also synchronizes the multiple repositories and enables users to control the index scope. This provides access to the requirements information of various projects. When requirements management is spread over various sites the change can be validated by a list of users. The program uses a specific tool icManage to administer the repositories. The tool makes indexing the information more accurately.

RTM’s toolset supports the ability to capture graphical information as traceable requirements objects (Volere, 2003). You can quickly design forms to manipulate the objects in a specific class using a screen painter tool (Wiegers, 1999).

The program is well suited for projects with many users and where some of the access privileges must be restricted. User access can be controlled down to an attribute on a requirement.

RTM has a useful feature, a process manager, that can prevent links from being constructed that would violate the process. Tests can be set up as entries in the database to which requirements are linked (Volere, 2003).

RTM integrates well with other software because of its interface with reporting tools, such as, Crystal Reports and MS-Query. This provides the customization of reports on the requirements fulfilment. The program can integrate with Rational’s software and Test Director. RTM is responsible for yearly upgrades and maintaining integrations. In the creation of tools RTM supports C++, API and SQL for database access.

The RE-tool does not include hardcopy user manuals (Wiegers, 1999).

We have not been able to download a test version of this RE-tool by ourselves since there is no test version on the vendor’s web site and therefore an analysis has not been conducted. Contact via telephone has been established but no answer by mail and the vendor has not answered our interview questions either.

Focal Point

61

The RE-tool is constructed by Focal Point AB, which is a Swedish organization. It is a web-based platform for market-driven product development, a decision-support system for the core business of product developing organizations (Volere, 2003). The main function of the RE-tool is to help the user validate different requirements according to a number of defined criteria and in that way help in choosing between the different requirements, which are being implemented.

The RE-tool’s main function is to support the prioritization of requirements. This is done by making pair-wise comparisons, according to a number of criteria, for instance, cost of implementation, which of the two requirements are more expensive to implement and how much more expensive it is. The program will summarize the results to get a consistent picture of the prioritization. It is the only RE-tool that is designed to evaluate and weigh different requirements to one another.

Focal Point has a structure that works with four kinds of data elements, which are managed in a similar way: information, objects, criteria and diagrams (Helldén and Stenborg, 1999). Attributes can be set for the different data elements. In the information object you can put information about the project, writers etc. The objects can for instance be requirements but also other types of objects that should be prioritized. The criteria are the defined aspects of which the requirements should be prioritized for instance, cost, effort and value. Diagrams can be constructed according to the validation of the requirements to one another. You can also create tables and hyperboles.

Other functions are changing requirements, managing dependencies between requirements, look at changes made with the requirements and define attributes for the requirements. The RE-tool is also constructed so that customers should be directly involved in suggesting and prioritizing the requirements. They can also follow their requirements through the process. Different access levels can be set according to the role they play in the process.

The RE-tool can define and generate reports based on the organizational choice (Focal Point, website).

The RE-tool can help with integrating your requirements engineering with portfolios, market segmentation, prioritization and release planning. An integration platform is built-in from the beginning, which makes it possible to integrate Focal Point with other tools. There is also a function for importing requirements from rtf-files.

The program seems quite easy to use. The buttons were easy to find and an explanation text appeared when you held the mouse arrow over it. Perhaps the text was too small on the buttons, sometimes we had some problem to see what it said and a person with poor eyesight could have difficult reading them.

The interface is easy to overview and the different functions also. There are many graphs and diagrams to show the results of the comparisons, which is definitely a positive thing. The diagrams can be altered and weighted and new diagrams will appear. In conclusion, the RE-tool is easy to learn and functions well in its area of evaluating requirements.

62

Appendix B. Criteria table and summary of the RE-tools

In table B.1 below, we have used the terms “Yes”, “No” and “No information” to describe if the different RE-tools support a certain task. The criteria below are examples of what could be looked at when selecting an RE-tool. The term “Yes” means that the RE-tool on some level supports the task but not how good or bad it supports it. We have also chosen (because of lack of time and information) not to compare how much better one RE-tool supports a certain task than another. The term “No” means that the RE-tool does not support this task and “No information” means that we have not been able to find any information whether an RE-tool supports a certain task. The information is collected from various research papers, e.g. INCOSE and Yphise (2003), and our own analysis. We have chosen not to include Focal Point in this summary because very little information has been written about this RE-tool.

Product functionality DOORS Caliber RM Requisite Pro RTM Workshop

Functionality characteristicsDoes the tool support configuration management? No information. Yes. Yes. Yes.

Is it possible to prioritize requirements? Yes. Yes. Yes. Yes.Is it possible to assign different types of attributes? Yes. Yes. Yes. Yes.

Does the tool support traceability? Yes. Yes. Yes. Yes.Does the tool support linking requirements to one

or a multiple of requirements? Yes. Yes. Yes. Yes.

Can requirements be linked to other project data? Yes. Yes. Yes. Yes.Can the links be deleted? Yes. Yes. Yes. Yes.

Does the tool support E-mail notifications of changesto affected users? No. Yes. Yes. Yes.

Does the tool support justification of proposals fromaffected users? Yes. No information. No information. Yes.

Can users comment on proposed changes? Yes. No. No. Yes.Does the tool define requirements base lining? Yes. Yes. No. Yes.

Is changes documented by the tool to be reviewed later? Yes. Yes. No. Yes.Can the tool produce quality documents? Yes. Yes. Yes. Yes.

Can requirements be viewed in document context? Yes. No. Yes. No information.

Does the tool support editing requirements? Yes. Yes. Yes. Yes.Does the tool support creating parent-child requirements? Yes. Yes. No. Yes.Can requirements be viewed in a hierarchical structure? Yes. Yes. Yes. Yes.

Does the tool support status tracking? Yes. Yes. Yes. No information.

Can you review requirement as an object? Yes. Yes. No. Yes.Can you define requirement as an object? Yes. Yes. No. Yes.

Does the tool support user groups and role assignment? Yes. Yes. Yes. Yes.Does the tool support scalability in a sufficient way? Yes. No. No. Yes.

Does the tool support multi-user environment? Yes. No information. Yes. Yes.Does the tool support group discussions about

requirements? No information. Yes. Yes. Yes.

Does the tool support creating test object? Yes. Yes. Yes. Yes.Integration characteristics

Does the tool support import of Word? Yes. Yes. Yes. No information.

Does the tool support import of Excel? Yes. Yes. No. Yes.Is the tool compatible with other software? Yes. Yes. Yes. Yes.

Interface/platform characteristics

63

Does the tool support integration with Unix? Yes. No. No. Yes.Can the tool integrate with Windows NT/XP/2000? Yes. Yes. Yes. Yes.

Can the tool access the Internet? Yes. Yes. Yes. Yes.Has the tool a known structure? Yes. Yes. Yes. Yes.

Is it possible to customize the user interface accordingto personal settings? Yes. Yes. Yes. Yes.

Is the tool documents-based? No. No. Yes. No.Is the tool data-based? Yes. Yes. No. Yes.

Does the tool support external servers? No, next version. Yes. Yes. No

information.Does the tool use an internal server? Yes. No. No. Yes.

Security factorsDoes the tool support the setting of access levels? Yes. Yes. Yes. Yes.

Vendor characteristicsDoes the vendor support training-courses? Yes. Yes. Yes. Yes.

Does the developer provide maintenance support? Yes. Yes. Yes. Yes.Information factors

Easy to overlook website with adequate information? Yes. Yes. Yes. Yes.Is there a downloadable trial version on the website? No. Yes. Yes. No.

Does a tool tutorial exist? Yes. Yes. Yes. No information.

Usability factors (personal opinions)

Do we think the tool was easy to overview? Yes. Yes. No. No information.

Do we think the tool was easy to learn? Yes. Yes. No. No information.

Table B. 1 Example of criteria and criteria groups and a summary of the evaluated RE-tools

64

Appendix C. Interviews with organizations

Organization A

1. A brief description of the organization (number of employees, product area, start of organization).

The organization has about 2000 employees, where almost everybody in one way or another works with requirements engineering. Their business is to deliver material (product specification to the military according to their wishes. The organization has done this for about 300 years.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

The organization uses a kind of incremental evolution in which they start to think of the next step in the process while the prior is being delivered. Their main model is to prolong the lifetime of the products. The changes that are made must fit the original system so new errors do not occur.

3. Which operative systems do the organization use?

For requirements engineering, Windows 2000 is used.

4. How long does an average project take?

The projects the organization is working on are mostly larger, with a lifetime of 5-15 years.

5. How many people do normally participate in a project?

From 1 to 100 persons are involved in the project at the same time. An average number is probably 10-15.

6. Do you have a requirements engineering process?

They have a vague process where they discuss their way to which requirements that will be used. They have their processes in a standard, ISO/IEC 15288, but it is not used as much as he would have liked.

7. For how long time have you used requirements engineering?

Requirements engineering have principally always been used.

8. Do you think you have clear requirements when you create the product specification?

No, they do not get clear requirements from the headquarters, and then the industry thinks the requirements from the organization are unclear also.

65

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

Yes, some requirements always change during the projects.

10. Are you using RE-tool/RE-tools?

Yes.

11. If you are, which?

Word (70 %), Excel, Access DOORS (>50 licences), RequisitePro (>10 licences), CaliberRM (5 licences), Focal Point (2 licences).

12. Have you used the RE-tool/RE-tools recently in any project?

Yes, all the time.

13. For how long time have you used RE-tools?

About 10-12 years.

14. How did you choose which RE-tool/RE-tools to use?

An evaluation of DOORS and RTM among others led to that RTM was rejected because DOORS focused more on the future and because it took a whole administration to manage RTM. Then RequisitePro and CaliberRM came and some licences were chosen to a number of projects. A continuous evaluation of RE-tools is conducted all the time. Consultants and the project group agree on which RE-tool that is to be used, and it has to fit the toolbox also (together with other tools).

15. Is it easy to use (good interface)?

DOORS: Large RE-tool with many functions. Overall good, some security flaws perhaps. When persons from two different projects should divide the access to their accounts, secrecy between different projects cannot be attained if several projects share the same store capacity for the requirements.In Access, it is relatively easy to build your own database but the problem is that that database must be maintained. In Word and Excel, it is fairly easy to list the requirements but useful functions, such as, traceability and change management lack.

16. Which are the benefits of the RE-tool/RE-tools?

See question 15.

17. Are there any drawbacks of the RE-tool/RE-tools?

66

See question 15.

18. What do the other employees think about the RE-tool/RE-tools?

The employees want more tools that are used more in the process.

19. What can be improved in the RE-tool selection process?

To go for the tools that focus more on ”the big picture”, such as, RequisitePro-Rose and DOORS-System Architect.

20. Do you spend much money on the selection part when buying a new RE-tool?

No, just a little.

21. If not...how do you choose other kinds of tools (testing, CM, programming)?

CM-tools have been used.

22. How many are trained in the RE-tool/RE-tools?

Not so many as the organization would want.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

No, just the most rudimentary.

24. What other things do you want from the RE-tool/RE-tools?

Better exchange f information between both different kinds of RE-tools and between RE-tools and other tools, such as, CM-tools. Better use of the standard AP233 for exchange of information between tools.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes, absolutely.

26. Have you used other RE-tools earlier (Word and Excel included)?

Yes, RTM Workshop.

27. Why change?

Evaluation of among others, RTM and DOORS led to that RTM was rejected because DOORS focused more on the future and because it took an entire administration to support RTM.

28. Do you think the organization will keep its size and product area?

67

The organization just wants to focus on their bit. The optimal would be if they did not need to get involved in the military’s part.

29. What do you see in the future of requirements engineering and RE-tools?

They think that the tools should be an integrated part of the process, i.e. how to use tools depending on the project. Every project builds its own working environment.

30. Will you keep your present RE-tool/RE-tools?

Yes, the RE-tools will be kept.

31. Other things.

If the developed product does not do exactly as the specification say, it is common with supplementary orders instead of changing the requirements specification. It is cheaper to buy new licences than to upgrade the existing.

Organization B

1. A brief description of the organization (number of employees, product area, start of organization).

The organization has about 50 employees. Their product areas are temperature transmitters (to control the temperature in machines) and digital converters. The organization started 1940.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

The organization has a kind of informal process model where they strive to follow the waterfall-model.

3. Which operative systems do the organization use?

Windows.

4. How long does an average project take?

Very different, but between 6-12 months.

5. How many people do normally participate in a project?

In general, most people in the organization are involved in the project. If you focus on the technical bit, about 5-10 persons.

6. Do you have a requirements engineering process?

The organization works according the strategy that the market controls their requirements process. They follow market specifications.

68

7. For how long time have you used requirements engineering?

Requirements engineering has existed as long as the organization has existed.

8. Do you think you have clear requirements when you create the product specification?

The organization put demands on themselves when developing new products, so the requirements on the specification are quite clear.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

The requirements always change mostly depending on the surrounding demands on the product. Software improvements also make changes in the requirements.

10. Are you using RE-tool/RE-tools?

No.

Since the organization has answered that they do not have a specific RE-tool the questions 11-28 have been deleted and the following questions have been asked instead:

11. Have you thought about requirements engineering and RE-tools?

The organization has thought about using specific RE-tools.

12. How would you select one?

In that case, they probably would see what RE-tools other organizations in the same area use and adjust to that.

13. How do you manage your requirements?

We use alternative tools, such as, Word, to specify the requirements.

14. Word, Excel?

See above.

15. Do you think you have the need for an RE-tool?

No, not right now.

16. What is the reason for not getting one (price, no need, etc.)?

To small organization and we do not see the need for a specific RE-tool.

17. What do you know about RE-tools?

69

Interesting subject that is too little in focus.

18. Do you think the organization will keep its size and product area?

The organization will keep its present working area, since we are established here.

19. Other things. -

Organization C

1. A brief description of the organization (number of employees, product area, start of organization).

The organization is a part of a larger concern and has 90 employees. The organization was established in 1987 and works with consulting in the areas software development and information technology. Other areas are project guidance, architecture, testing configuration management and system development. Mostly tailor-made systems, such as, information portals, calculating and web sites.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

The organization works according to the RUP (Rational Unified Process) that is an iterative process; each iteration is a process of its own. The different processes focus on different things e.g. architecture, design and programming. After each iteration something (part systems, documentation) is delivered. In each iteration one can say that a waterfall-model is used.

3. Which operative systems do the organization use?

Windows, different kinds of Unix, Linux, VMS and Mac.

4. How long does an average project take?

The projects usually last 3-10 months.

5. How many people do normally participate in a project?

Between 2-10 persons usually participate in a project.

6. Do you have a requirements engineering process?

Yes, RUP, where the requirements enter and ISO-certification of the requirements.

7. For how long time have you used requirements engineering?

Requirements engineering has always been used.

8. Do you think you have clear requirements when you create the product specification?

70

Sometimes, clear requirements pictures come from the customer, but often they are vague and then the organization can help with the requirements specification. Then a division in different phases is made where the customer can bid after each phase.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

The requirements always change during the project.

10. Are you using RE-tool/RE-tools?

Yes.

11. If you are, which?

RequisitePro is used as an RE-tool, ClearCase for configuration management and Rose for design. Excel and Word are also used. An XML-based tool has been developed in the organization.

12. Have you used the RE-tool/RE-tools recently in any project?

Yes, all the time.

13. For how long time have you used RE-tools?

RequisitePro has been used for a number of years.

14. How did you choose which RE-tool/RE-tools to use?

We started with RUP. Rational support the process and then RequisitePro was used since it was based on Word. The market situation has also influenced the choice.

15. Is it easy to use (good interface)?

If fits our process.

16. Which are the benefits of the RE-tool/RE-tools?

Excel is flexible because of the models; RequisitePro fits the process as said earlier.

17. Are there any drawbacks of the RE-tool/RE-tools?

Excel lacks traceability, which the specific RE-tool has.

18. What do the other employees think about the RE-tool/RE-tools?

They find them ok.

19. What can be improved in the RE-tool selection process?

71

The organization does not have a selection process for tools but would like to have one.

20. Do you spend much money on the selection part when buying a new RE-tool?

The only tool that has been bought is RequisitePro and that was selected because of the market situation, so not much money was spent.

21. If not...how do you choose other kinds of tools (testing, CM, programming)? –

22. How many are trained in the RE-tool/RE-tools?

Only 3-4 persons are trained in the RE-tool.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

No, I do not think so.

24. What other things do you want from the RE-tool/RE-tools?

We want to have RE-tools that are built on Excel.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes, well invested money.

26. Have you used other RE-tools earlier (Word and Excel included)?

See above. Word and Excel are still used together with something else.

27. Why change?

RequisitePro has many advantages compared with Word and Excel, such as, prioritization, effective documentation and so on.

28. Do you think the organization will keep its size and product area?

Yes, absolutely. We have a well-established organization in this business.

29. What do you see in the future of requirements engineering and RE-tools?

Requirements engineering and RE-tools are important to us and selecting a suitable RE-tool in the future will also be important.

30. Will you keep your present RE-tool/RE-tools?

Yes, the way things are right now.

31. Other things.

72

The Boston-matrix is used by the organization.

Organization D

1. A brief description of the organization (number of employees, product area, start of organization).

The organization has three business areas: Renal Products, Healthcare and BCT. Renal Products is involved in cleaning the blood by developing dialysis machines and has about 32 % of the organization’s occupation, Healthcare has clinics to care the ill and has 61 % of the occupation, BCT (Blood Component Technology) is involved in separating the blood into different parts by developing separation machines and has 7 % of the occupation. The organization has about 21000 employees and started in the end of 1960.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

The organization has developed a kind of V-process model according to the software standard 12207. The standard focuses on that the organization’s process model should be tailor-made according to own concept.

3. Which operative systems do the organization use?

MTOS, among others.

4. How long does an average project take?

Upgrades can take 2 years and other projects about 4 years.

5. How many people do normally participate in a project?

Usually 20-40 persons participate in a normal project.

6. Do you have a requirements engineering process?

No, not a specific process. There is a built-in process with a number of support-processes: Market specification, Assignment specification, Conceptual documents, Risk management summary and Standards. The existing process is that these support-processes and he system engineer together have the responsibility of them and for creating a requirements specification. However, we have the process for prioritization and attributes. This is up to the projects.

7. For how long time have you used requirements engineering?

Requirements engineering has always existed but has been taken more seriously the last 5 years.

8. Do you think you have clear requirements when you create the product specification?

73

No, particularly in the market specification. Besides, the crossover from the support-processes to the system engineer is vague. The requirements specifications are not that clear and they have to be worked with during the project to be good.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

See above.

10. Are you using RE-tool/RE-tools?

Yes.

11. If you are, which?

We have used DOORS for 5 years.

12. Have you used the RE-tool/RE-tools recently in any project?

Yes, all the time.

13. For how long time have you used RE-tools?

For 5 years.

14. How did you choose which RE-tool/RE-tools to use?

Different focus group shave studied the future in different areas. One of the groups would look on new technology. An independent person from another organization came and helped with a pilot project where DOORS was used.

15. Is it easy to use (good interface)?

For requirements engineering is the RE-tool easy to use, but you should not do Power presentations there. You have to have the requirements process clear in mind to be able to use the RE-tool in a satisfying way.

16. Which are the benefits of the RE-tool/RE-tools?

Three things in particularly: Links between objects, views and attributes. Have you worked with requirements engineering on paper earlier, it is easier to see the benefits with the RE-tool than if you do not have worked with requirements engineering earlier.

17. Are there any drawbacks of the RE-tool/RE-tools?

Configuration management and better integration between the vendors’ product suites.

18. What do the other employees think about the RE-tool/RE-tools?

74

They, who have understood the importance of the RE-tool, think it is good.

19. What can be improved in the RE-tool selection process?

Start with looking over your present requirements process. After that the RE-tool could be chosen according to this evaluation, maturity (CMM) and so on. If this is done in a good way, the choice of an RE-tool usually takes care of itself.

20. Do you spend much money on the selection part when buying a new RE-tool?

Not money, but time.

21. If not...how do you choose other kinds of tools (testing, CM, programming)? –

22. How many are trained in the RE-tool/RE-tools?

Everyone who works with developing in the organization is trained on the RE-tool.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

We use perhaps 60 % of the functions.

24. What other things do you want from the RE-tool/RE-tools?

See question 17.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes, absolutely.

26. Have you used other RE-tools earlier (Word and Excel included)?

Yes, Word and Excel was used until 5 years ago, when DOORS was bought.

27. Why change?

Mainly because we wanted traceability that is not provided in Word or Excel.

28. Do you think the organization will keep its size and product area?

Yes, absolutely. In this area the development goes relatively slowly.

29. What do you see in the future of requirements engineering and RE-tools?

It is a must.

30. Will you keep your present RE-tool/RE-tools?

We are satisfied with DOORS but because of a close co-operation with the Italian section (the have recently bought RequisitePro) perhaps we have to change.

75

31. Other things.

The most important thing to think about when discussing a new RE-tool is not to ask the question: Where do we want to be? The answer to this question is almost always: Where the competitors are. The question you should answer yourself is: Where are we now (in maturity)? If this question can be answered correctly, much s won.

Organization E

1. A brief description of the organization (number of employees, product area, start of organization).

The organization works in the packaging industry and has 21000 employees in the whole world. The organization develops cartons for different beverages. It started in the 1940:s.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

TPIN is process support for process orientation and follows the Innovation Process. We also use IT-models and the organization’s own process models.

3. Which operative systems do the organization use?

Standard Microsoft-environments for web-applications.

4. How long does an average project take?

The projects last about 6 months.

5. How many people do normally participate in a project?

In average 5-10 persons.

6. Do you have a requirements engineering process?

No, just parts of different process models.

7. For how long time have you used requirements engineering?

In some way, at least 2 years.

8. Do you think you have clear requirements when you create the product specification?

Not clear enough.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

76

Yes, they are often changed. Requirements have earlier just been changed because of active decisions (time, money). Discussions are conducted but not that many decisions.

10. Are you using RE-tool/RE-tools?

No, no concrete RE-tools.

Since the organization has answered that they do not have a specific RE-tool the questions 11-28 have been deleted and the following questions have been asked instead:

11. Have you thought about requirements engineering and RE-tools?

Yes, absolutely.

12. How would you select one?

External person with recommendations, compare different RE-tools on the market. See what is used in the organization today. See how well the different steps in the software process fit. Integration between different kinds of tools. Ideas, that are good, should be noticed. Different kinds of requirements that support the development and the decision-making.

13. How do you manage your requirements?

”Decision body”, that is the consulting and demanding organ, which exists of the local system- and process owners. The requirements are taken in structured or non-structured form and are anchored in Best Practice form. The requirements that are not in this version should be gathered for the next release.

14. Word, Excel?

Word and Excel will be continued to work with. The modelling of the requirements could use some support. It is also used for bids.

15. Do you think you have the need for an RE-tool?

Absolutely, to increase the detailed part of the requirements. Traceability is also needed.

16. What is the reason for not getting one (price, no need, etc.)?

Not the price. The way we work demands flexibility.

17. What do you know about RE-tools?

No, the common knowledge. There are experts in the organization if needed. RE-tools exist but they cannot be applied to IT-support yet.

77

18. Do you think the organization will keep its size and product area?

Yes, in the near future, the Innovation Process will still be monumental. The main principles of TPIN will remain to visualize the process. Different parts of the process, such as, strategy planning, etc, where discussions with the different tool vendors are conducted.

19. Other things.

Requirements engineering as a complex area, will mature. Also testing will mature. New knowledge and new methods will come. Much in IT and requirements engineering will be developed further. Maturity in the area, makes that much can be improved in requirements engineering. To get customer satisfaction, the requirements must be managed correctly.

Organization F

1. A brief description of the organization (number of employees, product area, start of organization).

The organization started in the mid-80: s and is developing cell phones (UMTS). We have 800 employees in the department here in Lund. The development is also partly conducted abroad.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

”Props”, which is a self-made development model which works like evolution and incremental-functional growth.

3. Which operative systems do the organization use?

Enea-OSE, Symbien.

4. How long does an average project take?

The projects are relatively short.

5. How many people do normally participate in a project?

In software projects, about 200 persons.

6. Do you have a requirements engineering process?

Yes, we try to improve it all the time.

7. For how long time have you used requirements engineering?

About 5-6 years. Requirements engineering changes all the time while the organization changes.

78

8. Do you think you have clear requirements when you create the product specification?

It can always be better.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

Yes, because of operators as large customers, the adjustment to the customers must exist.

10. Are you using RE-tool/RE-tools?

Yes.

11. If you are, which?

DOORS.

12. Have you used the RE-tool/RE-tools recently in any project?

Yes, all the time.

13. For how long time have you used RE-tools?

We have used it for 5-6 years.

14. How did you choose which RE-tool/RE-tools to use?

Through references. The organization had earlier used DOORS programs. Studies were made. DOORS could adjust to their needs. Consultants.

15. Is it easy to use (good interface)?

Yes.

16. Which are the benefits of the RE-tool/RE-tools?

Access levels.

17. Are there any drawbacks of the RE-tool/RE-tools?

Difficult to understand all functionality. The user interface could be hard to understand. DOORS cannot support code on different multi-sites.

18. What do the other employees think about the RE-tool/RE-tools?

They who use the RE-tool only periodically could have difficulties to understand all functionality.

79

19. What can be improved in the RE-tool selection process?

Usability for both super users and they who use it less.

20. Do you spend much money on the selection part when buying a new RE-tool?

Yes, relatively much.

21. If not...how do you choose other kinds of tools (testing, CM, programming)? – 22. How many are trained in the RE-tool/RE-tools?

About 200-300 persons. 20-25 persons use it continuously.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

Difficult to find the basic functions.

24. What other things do you want from the RE-tool/RE-tools?

Import information from Word, Excel, etc. When exporting, external documents are needed, which DOORS not support. Revision on object level.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes.

26. Have you used other RE-tools earlier (Word and Excel included)?

Word and Excel are still used. Operator requirements in Excel lists.

27. Why change?

All information in the same place, no revision management.

28. Do you think the organization will keep its size and product area?

New needs to adapt to.

29. What do you see in the future of requirements engineering and RE-tools?

Will be used more if the RE-tools are intuitive and easy to understand. Requirements prioritization important.

30. Will you keep your present RE-tool/RE-tools?

This is tested all the time.

31. Other things.

80

It should work like this: Study the information flow, then the process, and then the working organization, the RE-tool. A global organization can cope with Word and Excel. Traceability is over-estimated. It costs a lot of money to support traceability. It is a good idea but I have never seen an organization that has succeeded completely. Project guidance, test- and change management tools should be integrated, perhaps a PDM-system.

Organization G

1. A brief description of the organization (number of employees, product area, start of organization).

The organization is involved with developing real estate systems all over the world. They control pumps, light, water and heating. It was established in 1925 and has about 2000 employees.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

We have a self-made process that ideally follows the waterfall-model but also is influenced by prototype and evolution. We also have models for project guidance and CRT (Customer Response Team).

3. Which operative systems do the organization use?

Windows 2000 servers.

4. How long does an average project take?

The projects last about 6 months.

5. How many people do normally participate in a project?

About 10-15 persons are involved in the projects.

6. Do you have a requirements engineering process?

Yes, a newly developed process that has not been used in any project yet.

7. For how long time have you used requirements engineering?

At least since the requirements process started.

8. Do you think you have clear requirements when you create the product specification?

I believe that they are clear. They are probably not clear enough for the development department.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

81

They can change. Requirements can be added from different directions during the project.

10. Are you using RE-tool/RE-tools?

Yes.

11. If you are, which?

We use CaliberRM.

12. Have you used the RE-tool/RE-tools recently in any project?

Yes.

13. For how long time have you used RE-tools?

We have used it for one and a half year.

14. How did you choose which RE-tool/RE-tools to use?

An evaluation process. A number of criteria were set: Usability, overview of the requirements, traceability, and adjustment for requirements in later versions, splitting requirements. Support for the product life cycle, no document-based system. Our goal is not to have to produce documents. We have also tested DOORS, RequisitePro and CaliberRM’s demo versions and had them introduced.

15. Is it easy to use (good interface)?

Yes, I believe so.

16. Which are the benefits of the RE-tool/RE-tools?

Object orientation, requirements in focus instead of documents. E-mail notification, prescription. Feedback through discussion groups, configuration management.

17. Are there any drawbacks of the RE-tool/RE-tools?

Limited options to illustrate, especially in the functionality specification. External documents could have to be created with references.

18. What do the other employees think about the RE-tool/RE-tools?

We in Technology Product Management are satisfied. The developing department has recently bought licences.

19. What can be improved in the RE-tool selection process?

82

We followed our guts and looked especially much on what other organizations had chosen.

20. Do you spend much money on the selection part when buying a new RE-tool?

Not too much. We needed something quickly.

21. If not...how do you choose other kinds of tools (testing, CM, programming)? – 22. How many are trained in the RE-tool/RE-tools?

About 8-10 persons are trained.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

We use quite a lot. Perhaps 50-60 % of the functions. Traceability will grow in importance to us when the integration between the development and TPM occur.

24. What other things do you want from the RE-tool/RE-tools?

Easier illustrations, easier descriptions of requirements (will come in version 6). A clean web-interface, now there are two: application client through Java and web view for discussion. A clean HTML is preferable. It is not possible to ingratiate different kinds of requirements. To be able to trace requirements as it is now, you have to actively go in and change. Be able to create maps, as it is now, empty maps can exist.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes, absolutely.

26. Have you used other RE-tools earlier (Word and Excel included)?

Word was used earlier and is still used for the design of documents, but not after the integration with the design department.

27. Why change?

Configuration management and traceability.

28. Do you think the organization will keep its size and product area?

The board of the organization thinks that it can increase drastically.

29. What do you see in the future of requirements engineering and RE-tools?

It will definitely increase.

30. Will you keep your present RE-tool/RE-tools?

There are no plans of changing for the time being.

83

31. Other things. –

Organization H

1. A brief description of the organization (number of employees, product area, start of organization).

The organization was started 1984 and has 300 employees. Their main business areas are the development of cameras (40 %)and print servers. Also developing network products to increase the value of the network of an organization.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

A kind of waterfall-model is still used but incremental iterative process models are being used more, also prototypes are used.

3. Which operative systems do the organization use?

Several, but they must support both Windows and Linux.

4. How long does an average project take?

Silicon projects last longer but in average the projects last 2 months.

5. How many people do normally participate in a project?

The chip-projects involve more persons than the product projects but in average 6-10 persons.

6. Do you have a requirements engineering process?

We have a structure of the requirements specification that decides. We also have methods for eliciting requirements. A change management process that works well. Reviews are used and we are getting better with using use cases. Requirements are listed names are put.

7. For how long time have you used requirements engineering?

This method has been used for 4-5 years.

8. Do you think you have clear requirements when you create the product specification?

The project order can be unclear. We think the requirements are clear in the end.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

84

The market and the dynamic surroundings can make requirements change.

10. Are you using RE-tool/RE-tools?

No, we use a HTML-template, where we list the requirements.

Since the organization has answered that they do not have a specific RE-tool the questions 11-28 have been deleted and the following questions have been asked instead:

11. Have you thought about requirements engineering and RE-tools?

Focal Point has been viewed as a task-changing tool. It is more important with project management tools. A little sceptic to the large CASE-tools, they probably function more on the management level.

12. How would you select one?

Probably we would build our own since the RE-tool must be adjusted to fit our needs.

13. How do you manage your requirements?

We divide the requirements specification in different groups, such as, functional and non-functional requirements, which are linked to use cases.

14. Word, Excel?

We use HTML-templates and Excel.

15. Do you think you have the need for an RE-tool?

See question 11.

16. What is the reason for not getting one (price, no need, etc.)?

The price and that the RE-tool must be adjusted to fit our needs.

17. What do you know about RE-tools?

We know some.

18. Do you think the organization will keep its size and product area?

Yes, in the nearest future.

19. Other things.

The Checkbox-mentality could do that fewer requirements must be in the specification. He, who is best at choosing the right project and the right requirements, will win.

85

Organization I

1. A brief description of the organization (number of employees, product area, start of organization).

The organization is involved in research and development of medicine. The organization started in 1913. All together the organization has 60000 employees in countries all around the world.

2. Which process models do you use in the organization (ex. waterfall, prototype, evolution, formal)?

The organization uses a kind of incremental development where a substance is made that is later refined all the time.

3. Which operative systems do the organization use?

A kind of document management system.

4. How long does an average project take?

The projects last long, perhaps 15 years, but the goal is to shorten this to about 6 years.

5. How many people do normally participate in a project?

About 1000 persons from different departments.

6. Do you have a requirements engineering process?

Rather a kind of structure, where the requirements must respect authorities and standards.

7. For how long time have you used requirements engineering?

This structure has existed for 2 years. Requirements engineering has always existed.

8. Do you think you have clear requirements when you create the product specification?

We think that they are clear.

9. Do the requirements often change during the project or do the requirements specified from the beginning last?

The requirements always change of different reasons, most often because the authorities have high standards.

10. Are you using RE-tool/RE-tools?

Yes.

86

11. If you are, which?

We use a self-made document management system that is called GEL (Global Electronic Library). It is constructed in the way that our different organizations around the world can enter the database. Word is used to write product specifications.

12. Have you used the RE-tool/RE-tools recently in any project?

It is used all the time.

13. For how long time have you used RE-tools?

It has been used for 2 years.

14. How did you choose which RE-tool/RE-tools to use?

It was chosen because the organization needed a global tool. Also to guarantee that affected authorities would get the correct documents.

15. Is it easy to use (good interface)?

Can be a bit wieldy sometimes, but mainly it works well.

16. Which are the benefits of the RE-tool/RE-tools?

The Review-process has been improved since the tool was introduced; configuration management is also a great benefit.

17. Are there any drawbacks of the RE-tool/RE-tools?

The education material is not that good. Sometimes the availability can be bad i.e. you could have to wait for an hour because of high traffic.

18. What do the other employees think about the RE-tool/RE-tools?

See question 15.

19. What can be improved in the RE-tool selection process?

I do not know.

20. Do you spend much money on the selection part when buying a new RE-tool?

I do not know.

21. If not...how do you choose other kinds of tools (testing, CM, programming)? –

22. How many are trained in the RE-tool/RE-tools?

87

Principally everybody in the development department.

23. Do you use all the functions of the RE-tool/RE-tools (bigger, not details)?

We use most of the functions of the tool.

24. What other things do you want from the RE-tool/RE-tools?

We also need electronic signature, which will come in the next version.

25. Do you think it was a good investment to buy the RE-tool/RE-tools in the first place?

Yes, it was.

26. Have you used other RE-tools earlier (Word and Excel included)?

Word was used earlier.

27. Why change?

See question 14.

28. Do you think the organization will keep its size and product area?

Our business area focus will exist in the future.

29. What do you see in the future of requirements engineering and RE-tools?

It will grow in importance to use the correct method, specification, etc.

30. Will you keep your present RE-tool/RE-tools?

Yes, to my knowledge, but new versions will surely come.

31. Other things. -

88

Appendix D. Interviews with RE-tool vendors

Since these questions have been answered by mail, we have decided to provide the answers in Swedish because we do not want to change the meaning of the answers. A summation of the answers is however presented in chapter 4.4.

DOORS

Sammanfattning av intervju med Telelogic, Malmö. DOORS första version kom 1991 av dåvarande QSS. När Telelogic köpte QSS och Continuous 1999, tog därmed Telelogic över DOORS och Synergy vilka de vidareutvecklade.

1. Vad tycker ni att ert verktyg speciellt är bra på?

Verktyget har en robusthet och skalbarhet som gör att det inte blir så många programfel utan verktyget klarar av en stor användarpotential och en stor kravmängd utan att falera. Verktyget kan lika väl användas för 1 licens som för 2500 licenser. Traceability är en annan funktion som DOORS har arbetat mycket på för att förbättra. Verktyget är integrerbart med Analyst och UML vilket gör att modellering av grafer är möjlig.

2. Vad är det som särskilt skiljer ert verktyg från andra på marknaden?

Att hantera kravvyer är också en funktion som DOORS är ensam om. Här kan man välja att titta på t.ex. endast användarkrav eller tekniska krav och dessutom se hur dessa är länkade till varandra direkt i kravvyn utan att behöva använda ”proporties”. Även olika attribut kan ses i denna kravvy. DOORS har en egendefinierad databas vilket både är en fördel och en nackdel. Fördelen är att ett företag inte behöver köpa in extra programvara i form av en ny databas, medan nackdelen är att om ett företag har bestämt från ledningen att de ska använda t.ex. Oracle så måste Telelogic lägga ner pengar på att göra DOORS integrerbart med Oracle.

3. Varför skulle ni rekommendera oss just ert verktyg istället för konkurrenternas?

Verktyget har en stor potential och är etablerat på marknaden. Verktyget har de sista sex åren utsetts till det bästa kravhanteringsverktyget av oberoende utvärderingar. Det har dessutom utsetts till det verktyg som kommer med mest nya tekniska förbättringar. Att verktyget är integrerbart med ca 50 andra program inom bl.a change management, testning och configuration management är också en stor anledning till att köpa DOORS. Verktyget stöder multi-user hantering och är långt framme inom t.ex. ett område som säkerhet vilket gör att de har många licenser inom försvarsindustrin och läkemedelsbranschen.

4. Hur är tillväxtpotentialen för ert kravhanteringsverktyg?

Svårt att säga. Just nu har vi 32 % av marknaden. Marknaden för kravhanteringsverktyg har ökat och mognaden på företagen har gjort att behovet för kravhanteringsverktyg med alla dess funktioner har ökat.

89

5. Hur mycket tid och pengar lägger ni ner på att få verktyget integrerbart med andra program (management tools, kravhanteringsverktyg, olika operativsystem)?

Förbättringar sker hela tiden. DOORS är byggt kring en egenproducerad databas, medan vissa företag kan ha krav på sig från ledningen att de ska använda t.ex. Oracle, då måste DOORS hela tiden förbättras så det kan integreras med andra verktyg, både inom requirements management och change management.

6. Var kan vi hitta information om vad som skiljer era olika releaser åt (hemsida, nyhetsbrev)?

Fyra olika nyhetsbrev släpps som information, ca var tredje månad, Automate, DOORS news, UML och CM. Prenumeration på dessa kan beställas.

7. Hur stora är era kurser för utbildning (tid, pengar)?

Fullständiga kursutbud finns tillängliga. Vid ”kick start” ordnas en två dagars kurs för inlärning av verktyget. För de som är mer vana vid verktyget och använt det tidigare ordnas administrationskurser.

8. Vad är priset/licens för verktyget och ingår eventuella uppdateringar i detta pris?

Finns på hemsidan. Olika priser per licens. Rabatt ges då många licenser köps. Uppdateringar ingår oftast i kontraktet som ”maintenance” och behövs då inte betalas mer för.

CaliberRM

Svar på frågor som skickats till Nohau i Malmö.

1. Vad tycker ni att ert verktyg speciellt är bra på?

Detta är upp till kunden att bedöma, utifrån de krav som kunden ställer på sitt verktyg...Ett verktyg måste idag bedömmas dels på dess egna funktioner och dels på hur väl det hänger samman i företagets hela verktygssvit. Båda dessa faktorer bidrar till att höja produktivitet och kvalitet på det utförda arbetet.

CaliberRM har goda integrationer med olika verktyg:

- projektplanering: MS Project, EstimatePro...- modellering: Rose, Rhapsody, Together...- Test: Mercury Interactive/Test Director- CM: Starteam och en mängd andra CM verktyg- olika SW utvecklingsmiljöer- .....

Alla dessa kopplingar tillför något mer än den "rena kravhanteringen" och måste genomlysas i detalj för att man ska få en full bild. En organization behöver ju inte "ett kravverktyg" utan ett bra sätt att dels samla , dels distibuera, dels iterativt arbeta med, dels dokumentera, dels skapa spårbarhet etc, information av alla möjliga slag där krav

90

är en delmängd (men en mycket viktig sådan). Caliber är bra på att hantera även sådant som inte är rena krav.

I funktionalitet på "rena kravhanteringen" är CaliberRM mycket komplett. Se vidare nedan.

2. Vad är det som särskilt skiljer ert verktyg från andra på marknaden?

Jag kan inte alla verktyg på marknaden, men i Sverige är de tre stora Borland/Caliber, Rational/ReqPro och Telelogic/DOORS. I jämförelse med dessa så har Caliber den modernaste arkitekturen baserad på en objekt-orienterad databas. Det gör att det är enkelt (on-the fly) att configurera om inställningar i programmet, lägga till attribut mm.

Kunderna brukar tycka att Caliber är enklast att lära och att använda, det gör att även icke-tekniker tar det till sig.

Caliber har ett mycket öppet API, vilket har gjort det enkelt för oss att bygga integrationer med andra verktyg.

Caliber har en Client-server lösning och databas som gör att det fungerar lika enkelt och snabbt i organizationer med en användare eller tusen, hundra krav eller hundratusen krav.

Goda funktioner för auktorisation, användarstyrning, elektroniska signaturer.

3. Varför skulle ni rekommendera oss just ert verktyg istället för konkurrenternas?

Förutom det ovan nämnda, så är Borland en dokumenterat innovativ och finansiellt stark partner. Vi har en god supportorganization och goda utbildningsmöjligheter i Skandinavien.

4. Hur är tillväxtpotentialen för ert kravhanteringsverktyg?

Vår bedömning är att potentialen är mycket stor. Medvetandet ökar om att dålig kravhantering är roten till de flesta problem som ett projekt löper in i. Därför kommer allt fler företag att vara intresserade av att hantera sina krav på ett professionellt sätt, och det kräver i sin tur bra stödverktyg. Än så länge har största intresset kommit från organizationer som har haft hårda myndighetskrav och/eller arbetat med säkerhetskritiska applikationer. Efter hand som fler organizationer inser att det finns både kvalitet och effektivitet att vinna på kravsidan ökar marknaden. Vår bedömning är att inom fem år är ett kravhanteringsverktyg lika självklart som ett projektplaneringsverktyg (om de inte är integrerade då).

5. Hur mycket tid och pengar lägger ni ner på att få verktyget integrerbart med andraprogram (management tools, kravhanteringsverktyg, olika operativsystem)?

Det läggs inte tid på detta efter som grundarkitekturen i CaliberRM redan tillåter att all information kan utbytas med omgivningen. Däremot läggs ofta tid på nya

91

integrationer. Det är sällan eller aldrig som två organizationer har samma verktygssviter.

6. Var kan vi hitta information om vad som skiljer era olika releaser åt (hemsida,nyhetsbrev)?

Det finns bara en variant av CaliberRM. Däremot finns det olika integrationer med andra verktyg. Vissa ingår, andra kostar en slant. Information och release notes finner man på http://www.borland.se/caliber/index.html

7. Hur stora är era kurser för utbildning (tid, pengar)?

De mesta utbildningar körs som företagsinterna kurser. Det brukar räcka med en knapp dag för användare och ytterligare en dag för installation och utbildning av verktygsadministatör. Vi har även schemalagda kurser: se http://www.nohau.se under "Utbildning".

8. Vad är priset/licens för verktyget och ingår eventuella uppdateringar i detta pris?

Ca 14.000/licens. Support och Underhållsavtal inkl uppdateringar kostar 20% /år.

RequisitePro

Svaren på frågorna kommer från IBM Sverige.

1. Vad tycker ni att ert verktyg speciellt är bra på?

Rational RequisitePro integrerar det bästa av två världar: Word för att skriva dokument med, krav och databas för att filtrera/sortera/länka krav.

2. Vad är det som särskilt skiljer ert verktyg från andra på marknaden?

RequisitePro är enkelt att lära sig och att använda. Möjligheten att importera redan befintliga worddokument och att fortsätta att arbeta med kraven i Word. Kravtexten i Word är alltid synkroniserad med kravtexten i databasen.

3. Varför skulle ni rekommendera oss just ert verktyg istället för konkurrenternas?

Därför att RequisitePro är så lätt att använda och att man kan fortsätta att arbeta i befintliga Wordokument.

4. Hur är tillväxtpotentialen för ert kravhanteringsverktyg?

Mycket stor i och med att företagen blir mer medvetna om hur viktigt det är att hålla ordning på sina krav så att man vet vad man har levererat till kund.

5. Hur mycket tid och pengar lägger ni ner på att få verktyget integrerbart med andra program (management tools, kravhanteringsverktyg, olika operativsystem)?

92

Vet ej.

6. Var kan vi hitta information om vad som skiljer era olika releaser åt (hemsida, nyhetsbrev)?

http://www-306.ibm.com/software/awdtools/reqpro/support/index.html

7. Hur stora är era kurser för utbildning (tid, pengar)?

”Essentials of Rational RequisitePro”, 1 dag, 6 800 kr exkl moms, “Mastering Requirements Management with Use Cases”, 3 dagar, 15 800 kr exkl moms.

8. Vad är priset/licens för verktyget och ingår eventuella uppdateringar i detta pris?

40200 kr exkl moms per flytande licens inkl 1 års teknisk support och underhåll. En flytande licens innebär 1 samtidig användare i ett nätverk.

RTM Workshop

RTM Workshop’s återförsäljare i Sverige har inte svarat på våra mail och därför kan inga svar på frågorna presenteras.

Focal Point

Svaren på frågorna kommer från Focal Point AB.

1. Vad tycker ni att ert verktyg speciellt är bra på?

Focal Point tar inte bara hand om traditionell kravadministration utan även prioritering, planering och beslutsfattande.

2. Vad är det som särskilt skiljer ert verktyg från andra på marknaden?

Focal Point är det enda som är helt webbaserat utan lokala installationer. Det går att konfigurera under drift och det krävs ingen programmering för konfigurationen. Focal Point har som enda produkt inbyggda vertyg för granskning, prioritering, release-planering, roadmapping, uppföljning, intelligent sökning och roadmapping.

3. Varför skulle ni rekommendera oss just ert verktyg istället för konkurrenternas?

Se ovan.

4. Hur är tillväxtpotentialen för ert kravhanteringsverktyg?

Vi har växt med 40 % per år och är nog den leverantör av kravhanteringsverktyg somväxer snabbast sett till tillväxt i licensintäkter.

5. Hur mycket tid och pengar lägger ni ner på att få verktyget integrerbart med andraprogram (management tools, kravhanteringsverktyg, olika operativsystem)?

93

Vi har byggt en integrationsplattform som gör att man kan integrera Focal Point med andra verktyg. Så investeringen är redan gjord. Nu är det mest lite underhåll.

6. Var kan vi hitta information om vad som skiljer era olika releaser åt (hemsida,nyhetsbrev)?

Vi publicerar bara information om den senaste releasen. Vi har en hyresmodell som innebär att alla kunderna alltid kör den senast releasen. Det finns ingen anledning att presentera information om releaser som inte används eller går att köpa.

7. Hur stora är era kurser för utbildning (tid, pengar)?

En intro på fyra timmar som räcker för att komma igång, en tvådagarskurs för att blinormalanvändare och ytterligare en tvådagarskurs för att bli administratör.

8. Vad är priset/licens för verktyget och ingår eventuella uppdateringar i detta pris?

Vi hyr ut verktyget till kunderna. All support och alla uppdateringar ingår. Priset varierar kraftigt eftersom vi har allt från 2-personersföretag som kunder till 250000.

94

Appendix E: Response to evaluation questions

The response to these evaluation questions is presented in Swedish because we do not want to alter any answers. There is however a summary of the response in chapter 6.2.

Organization A

1. Hur står sig metoden mot att välja ett verktyg ad hoc?

BRA!

2. Hur står sig metoden mot hur ni har valt verktyg?

Något bättre, vår metod finns ej dokumenterad, vilket denna metod gör!

3. Tror ni metoden skulle fungera för att få fram ett lämpligt verktyg för er?

Absolut.

4. Vilka brister ser ni med metoden?

Appendix B är inte heltäckande, exempelvis säger man att DOORS inte är dokumentorienterad, vilket den är, men detta är ju inte uppenbart, eller kan sägas vara en definitionsfråga. Likaså att man ej kan använda externa servrar, vilket även det är ett sådant diskussionsämne. (vid en kontroll tror jag att ni menar "external database server", i vilket fall påståendet i appendix är korrekt, men då är det felskrivet istället.

5. Vilka positiva aspekter finns med metoden?

Att man följer en dokumenterad metod, metoden är med andra ord spårbar.

6. Finns det aspekter metoden inte täcker?

Säkerligen, men jag kommer inte på någon just nu.

7. Vilka likheter finns mellan metoden och hur ni valt verktyg?

Att vi skrev ner krav på verktyget och vilka egenskaper vi skulle jämföra.

8. Vilka skillnader finns mellan metoden och hur ni valt verktyg?

Att metoden är dokumenterad och efter flera år kan upprepas.

9. Finns det något som är oklart i metoden?

Inte vid snabb genomläsning, däremot kan man säkert finna något vid testkörning.

10. Skulle ni rekommendera någon annan att använda metoden, och i så fall varför/varför

inte?

95

Jomenvisst, bra jobbat!

Kör metoden på prov i något projekt, dvs haka på något som pågår någonstans i industrin och deltag med utvärderingshjälp, jag tror att det skulle uppskattas och ge overderlig valideringshjälp för metoden.

Organization B

1. Hur står sig metoden mot att välja ett verktyg ad hoc?

Bra mycket bättre än ad hoc.

2. Hur står sig metoden mot hur ni har valt verktyg?

Bra.

3. Tror ni metoden skulle fungera för att få fram ett lämpligt verktyg för er?

Kanske, inte helt säkert.

4. Vilka brister ser ni med metoden?

För en liten organisation som oss kan det lätt bli att man fokuserar för mycket på att hitta rätt verktyg istället för att jobba med de egna produkterna och kärnverksamheten som skall ge inkomst. Även om det är ett engångsarbete verkar det ganska tidsödande.

5. Vilka positiva aspekter finns med metoden?

Man gör säkert mer rätt från början.

6. Finns det aspekter metoden inte täcker?

Saknar någon slags grov prestanda - pris jämförelse.

7. Vilka likheter finns mellan metoden och hur ni valt verktyg?

Inga.

8. Vilka skillnader finns mellan metoden och hur ni valt verktyg?

Inga.

9. Finns det något som är oklart i metoden?

Svårt att säga på så kort tid. Man måste nog genomföra ett case för att uttala sig om det.

10. Skulle ni rekommendera någon annan att använda metoden, och i så fall varför/varförinte?

96

Jag skulle definitivt rekommendera att använda denna eller någon annan genomtänkt metod istället för ad hoc metoden.

Organization E

1. Hur står sig metoden mot att välja ett verktyg ad hoc?

Metoden täcker väl upp de generella krav som finns och utgör på ett snabbt och enkelt sätt en checklista på bra saker att ta hänsyn till och planera för i valet av ett verktygsstöd. Fördelar mot att välja ad hoc är att de största fallgroparna kan undvikas som skulle kunna göra att valet måste göras om - tidsödande - därför sparar den föreslagna metoden också tid, vilket ju torde vara anledningen till att man väljer ett verktyg ad hoc.

2. Hur står sig metoden mot hur ni har valt verktyg?

Metoden är mer omfattande än den "metod" som först var tänkt att användas för att välja ett verktyg ad hoc. I slutändan valdes ett verktyg som redan fanns inom organisationen enligt metodens föreslagna moment.

3. Tror ni metoden skulle fungera för att få fram ett lämpligt verktyg för er?

Ja.

4. Vilka brister ser ni med metoden?

Eftersom föreslagna kriterier finns bifogat kan dessa lätt användas som mall för krav. Risken som finns med mallar är att man inte "tänker till" och enbart går efter checklistan.

5. Vilka positiva aspekter finns med metoden?

Risken att inte ha mallar vore att arbetet inte görs alls, eftersom det inte blir handgripligen tydligt vad som skall göras. Det mest positiva är att metoden sannolikt kommer att användas för den som är övertygade att en urvalsmetod skall användas - sannolikheten ökar för dessa intressenter ju mer instruerande metoden är.

6. Finns det aspekter metoden inte täcker?

Om metoden kunde föreskriva moment som indikerar var man skall "tänka till" hur man skall välja, listar och utvärderar kriterier instruerar metoden urvalsprocessens viktigaste moment mer handgripligen. Det är dock en balansakt vad som är sundast att göra.

7. Vilka likheter finns mellan metoden och hur ni valt verktyg?

Metodens första stegen liknar mycket de steg som vidtogs eftersom dessa åtgärder nog bäst kan beskrivas som sunda undersökningsprinciper - och dessa första steg räckte till

97

för att hitta ett befintligt verktyg för oss. Skulle det visa sig att vi någon gång kommer att välja om kommer vi sannolikt att följa den häri beskrivna metoden.

8. Vilka skillnader finns mellan metoden och hur ni valt verktyg?

Vi hade ett befintligt verktyg inom koncernen som redan användes och var betalt. Det räckte för oss vid det tillfället. Dock gick vi inte vidare med att försöka hitta alternativ, vilket vi nog hade spenderat tid på om metoden hade varit känd innan vi satte igång.

9. Finns det något som är oklart i metoden?

Nja, manual/instruktioner till mallarna metoden mm. och när samt hur dessa skall användas bör tydliggöras om metoden skall kunna användas av lekmän. Det krävs idag att man är tämligen insatt i problematik, terminologi samt med fördel i utvecklingsproblematik för att inse vad kravhantering ämnar åstadkomma och vilka fördelar det innebär att arbeta ordentligt fram till specifikation för att få ut allt tänkt värde i metoden och att kunna göra "rätt" val i slutänden.

10. Skulle ni rekommendera någon annan att använda metoden, och i så fall varför/varför

inte?

Ja, absolut. Framförallt är metoden i dess presenterade skick fullt ut användbar som tillvägagångs-manual för oss som tror att vi redan vet hur vi "borde" göra. Jag kommer spara den till nästa gång det blir dags.

Organization F

1. Hur står sig metoden mot att välja ett verktyg ad hoc?

Mycket, Bra att ni har försökt fånga det viktigaste i ett antal steg. När vi valde DOORS verktyget, gjorde vi step 1 – 3. Kanske borde vi mera undersökt step 4. Men vi hade goda referenser för DOORS av andra bolag i koncernen.

2. Hur står sig metoden mot hur ni har valt verktyg?

Metoden stämmer mycket bra överens.

3. Tror ni metoden skulle fungera för att få fram ett lämpligt verktyg för er?

Definitivt, JA.

4. Vilka brister ser ni med metoden?

Inga direkta, men det är som ni själva påtalar, kan vara politik som styr toolvalet. Det kanske finns en viss policy inom företaget som styr beslutet, som inte har att göra med stegen i Er definierade metod.

5. Vilka positiva aspekter finns med metoden?

98

Kortfattad och kärnfull.

6. Finns det aspekter metoden inte täcker?

Administrationen av toolet. Skilj mellan innehållet content i toolet och administrationenen av toolet, typ uppgradering av nya releaser etc. Införandet av ett nytt tool i kombination av ett nytt Workflow, kräver stora utbildningsinsatser i en organisation.

7. Vilka likheter finns mellan metoden och hur ni valt verktyg?

Stora.

8. Vilka skillnader finns mellan metoden och hur ni valt verktyg?

Step 4, som är en resurs – och tids- aspekt.

9. Finns det något som är oklart i metoden?

Inte vad jag kan se.

10. Skulle ni rekommendera någon annan att använda metoden, och i så fall varför/varför

inte?

Definitivt Ja.

99