Upload
lykiet
View
217
Download
0
Embed Size (px)
Citation preview
Analysis of Software Requirements
Negotiation Behavior Patterns
Alexander EgyedUniversity of Southern CaliforniaCenter for Software Engineering
Henry Salvatori Computer Science BuildingLos Angeles, CA 90089-0781 USA
+ 1 213 740 5703
[email protected]:// sunset. usc.edu/
ABSTRACT
Roughly 35 three-person teams played the roles of user,customer, and developer in negotiating the requirements ofa library information system. Each team was provided witha suggested set of stakeholder goals and implementationoptions, but were encouraged to exercise creativity in expanding the stakeholder goals and in creating options fornegotiating an eventually satisfactory set of requirements.
The teams consisted of students in a first-year graduatecourse in software engineering at USC. They were provided with training in the Theory W (win-win) [3] ap
proach to requirements determination and the associatedUSC WinWin groupware support system [5][8]. They wererequired to complete the assignment in two weeks.
Data was collected on the negotiation process and results,
with 23 projects providing sufficiently complete and comparable data for analysis. A number of hypotheses wereformulated about the results, e.g. that the uniform set ofinitial conditions would lead to uniform results. This papersummarizes the data analysis, which shows that expectations of uniform group behavior were generally not realized.
KeywordsWinWin, Negotiation, Theory W, People Behavior
Barry BoehmUniversity of Southern CaliforniaCenter for Software Engineering
Henry Salvatori Computer Science BuildingLos Angeles, CA 90089-0781 USA
+1 213 7405703
[email protected]://sunset. usc.edu/
INTRODUCTION
A major objective in several software process initiatives isto achieve repeatable processes, to which techniques suchas statistical process control may be applied. For example,Level 2 of the SEI Capability Maturity Model [12] is calledthe Repeatable level. An early description of the model [9]states:
"Dr. w.E. Deming, in his work with the Japaneseafter World WarII, applied the concepts of statistical process control to many of their industries.While there are important differences, these concepts are just as applicable to software as they areto producing consumer goods like cameras, television sets, or automobiles. "
How repeatable are, or should be, such people-intensivesoftware processes as software requirements determination? It is very difficult to obtain comparable observationalor experimental project data to explore this and relatedissues.
An opportunity arose to experiment on such issues in thecontext of a first-year graduate course on software engineering. In this course, student teams were trained in aparticular software requirements approach (win-win).They then used the approach to negotiate a set of requirements for a Library Information System, for which a common baseline problem description and candidate set ofsystem functions were provided.
Given the prepackaged nature of this activity, one mightexpect that the teams would execute repeatable processesand produce repeatable results. Hypotheses were formulated and tested on several dimensions of this issue; the
results are provided below.
Although the student-and-coursework context of this requirements activity is only moderately representative ofactual practice, it can be argued that the results of such apre-structured activity would serve as a lower bound on the
covers
3. Reconcile winconditions. Establishnext level of
objectives, constraints,alternatives
2. Identify Stakeholders'win conditions
I. Identify Nextlevel Stakeholders
Figure t - Win Win Spiral Model [5]
4. Evaluate productand process
6. Validate Prozuc '\... alternatives, resolveand process ~ risksdefinitions 5. Define next level 0
product and process including partitions
Figure 2 - Inter-Artifact Relationship [4]
Agreement
Option
Agreements. Win Conditions capture the stakeholders'goals and concerns with respect to a new system. If a WinCondition is non-controversial, it is adopted as an Agreement (see Figure 2). Otherwise, an Issue artifact is createdto record the resulting conflict among Win Conditions.Options allow stakeholders to suggest alternative solutionswhich can resolve Issues. Finally, Agreements may be usedto accept these solutions.
Each of the four artifact types contains information whichis helpful for negotiations. This information includes:
Name: One-line title.
Body: Multi-line description text field to describe an artifact in more detail (nature, rationale, etc.).
Comments: Allow other stakeholders to make statements
or to request information about a specific artifact.Comments are the primary medium of communicationbetween stakeholders. Using Comments is also theonly way of adding information to somebody else'sartifacts, because only the owner has the right tomodify the contents of his or her artifacts.
repeatability of software requirements engineering processes and results in general project practice. A primaryobjective of the experiment and analysis is to explore thenature of such lower bounds.
Context
Software development is largely a group activity withgroup sizes ranging up to several hundred people. McCue'sstudy showed that about half of a developer's time is spenton interaction with other people [11]. Thus, being able tointeract efficiently and unambiguously is a major first stepin developing a successful product. However, interactionalone, how efficient it may be, is not enough if participants(stakeholders) do not solve the right problem. One approach to focusing on the right problem, Theory W, statesthat your project will succeed if and only if you make winners of all the critical stakeholders [3], thus, ensuring thatthe right product is being built.
Stakeholders in a software development process are usuallydevelopers, customers, users, maintainers, and others. Inorder to make everybody a winner you have to ensure thatall critical stakeholders are able to participate in the development process. Therefore, efficient collaboration of allimportant stakeholders is a significant pre-condition forachieving the common goal of building a successful software product.
The Theory- W concepts have been incorporated into theWin Win spiral process model [5] shown in Figure 1. Thespiral model approach [2] incorporates an evolutionarysoftware development process which allows, and even encourages, multiple iterations of software developmentstages until the final goal is achieved. The Theory- W extension modifies the spiral model in such a way that eachiteration identifies all critical stakeholders and their win
conditions in advance. To complement the process model,
a groupware tool support system has been developed toensure that the right amount of communication and collaboration is guaranteed during all cycles of the softwaredevelopment process. This tool is Win Win which is described below.
The Win Win Groupware Support SystemThe software tool Win Win is a groupware system whichwas primarily designed to be applicable for requirementsengineering. Stakeholders use the tool to identifY win conditions, resulting conflicts, possible options, and finallysolutions. The goal is to work towards agreements whichincorporate all win conditions and resolve all conflicts.Additional negotiation aids such as taxonomy, glossary,and risk-resolution tools support the stakeholders in theirefforts [8].
The Win Win negotiation model is primarily based on fourartifact types: Win Conditions, Issues, Options, and
Attachments: Allow additional data files to be attached to
an artifact. For instance, a stakeholder may attach acost estimation file from COCOMO (a software costestimation tool [1]) to an artifact in order to documenta cost or schedule Issue, or a proposed resolution Option.
Relationships: Allow an artifact to be associated withother artifacts (see Figure 2). Relationships are used tobuild an artifact chain starting with Win Conditions,followed by Issues, Options, and finally Agreements.The artifact chain describes the sequence in whichnegotiations are conducted if Win Win is being used.Note that the Issue and Option artifacts are optionaland not needed if an agreement can be made directlyfrom a Win Condition.
Creation and Revision Date
Domain Taxonomy: Allows artifacts to be referenced to ahierarchical taxonomy for the one or more terms inthe domain being addresses. This taxonomy serves asa means of terminology control and as a means of querying or analyzing related artifacts.
The Win Win model has been formally specified and analyzed for consistency [10] but only little is known about thecorrectness and usefulness of assumptions made duringthis process. Many questions have been raised such asthese:
How are people and negotiation results affected by using
a negotiation tool such as Win Win?
How similar are the negotiation results if stakeholders for
all groups have a similar win conditions to start with anda pre-defined negotiation model to follow?
How do people factors, like work experience or age, effect
the process and the outcome of the negotiation?
Do people use the tool as it was anticipated by the model?
And there are many more unanswered questions. Some ofthese questions are general; others are more related to thespecific Win Win methodology. However, knowing theanswers to these questions is vital in providing more usefuland powerful negotiation aids for stakeholders.
THE PROJECT
The Win Win usage analysis was based on student projects.The goal of these projects was to negotiate functionality,budget, and schedule for a proposed library system, calledLSDI (Library Selective Dissemination of Information), tobe used at the hypothetical SCU university (SouthernCalifornia University). The university has three major
campuses, each of which has a main library which willprovide LSDI services. Each of the three campuses willoperate a server running a COTS client-server library
service package.
In addition, the SCU Library, Computer Science Department, and Computing Services Operation have beenfunded to develop an experimental Selective Dissemination of Information (SDI) system to provide SCU userswith information about new library acquisitions of interest.It will do this by comparing attributes of new library acquisitions with interest profiles provided by the libraryusers. The funding grant provided initially $1,350K fordevelopment of the system and additional money formaintenance.
The basic components for the system were given prior tothe negotiations. However, the students had the freedom toagree on different levels of detail for each component, oron whether it should be implemented or not. They couldalso add new components if desired. The following components and their levels of detail were offered:
User Interest Profile: User interest profile creation, deletion, update, query, etc. (mandatory)
Access Control:
basic level: basic password access control.
extended level: basic level plus authentication,authorization, intrusion detection
rigorous level: extended plus formal specification andverification
Data Acquisition Handling and Profile Checking:
Acquisition record and file access, validity checking,logging, storage, etc. Determination of matches between acquisitions and interest profiles.
basic level: simple matching
extended level: conditional matching
User Services:
basic level: Acquisition-match query, browse, and request functions. Basic help and profile managementfunctions
extended level: basic level plus tutorial functions, interoperability with COTS user interface.
Usage Analysis: Recording and summarizing system usage by time of day/week/year, by user attributes, byacquisition attributes.
Trend Analysis: Extrapolation of rates of increase, decrease, or cyclical trends in services.
Library Network Access: Extension of acquisition notification to cover acquisitions from a regional network often additional universities.
COTS integration: Integrate SDI functions with COTSlibrary information functions (mandatory).
Each level of each capability was characterized by itsnumber of lines of code and other COCOMO cost drivers,
enabling the stakeholders to use COCOMO for analysis oftradeoffs among cost, schedule, functionality, performance,and reliability.
Besides the software components, the students had tochoose between two types of processors which would beused for the central library server. Processor X was aslower but reliable processor whereas processor Y was afaster but less mature processor. Additionally the size ofthe COTS integration was affected by the choice of theprocessor. The use of processor X required less lines-ofcode integration for COTS than processor Y.
The budget was initially $1,3 50K but was changed later to$1,IOOK. No constraints were defined for the schedule butthe different stakeholders had different requirements onhow fast the system should be available.
Schedule and PeopleThe project did not include the actual coding of the libraryinformation system by the students. They only had to workon the initial stages of its development. The goal of eachproject was to come up with a project plan, specificationfor requirements, user interface, and architecture within 8weeks.
The tool Win Win was used at the beginning to negotiatethe requirements of the library system between the stakeholders. The results of the Win Win negotiation could thenbe mapped to the requirements document which completedthe first milestone of the project. For the negotiation, eachgroup member was assigned a specific role. Thus, negotiating the requirements was held in a cooperative environment between equal participants rather than in a performerteam with a team leader.
After completion of this milestone the teams were changedinto developer teams in order to complete their plans andarchitectures for satisfYing the negotiated requirements.However, this paper concentrates on the events and activities going on during the first stage of the project - the requirements negotiation.
As described earlier, three stakeholder parties were included in these negotiations: a customer, a developer, anda user. Each team member had to choose one of these
stakeholder roles for the negotiation part. Furthermore,each stakeholder role was described in terms of its primarywin conditions and a brief biographical sketch of the
stakeholder's background. This means that all teamsstarted with the same suggested win conditions for all oftheir stakeholders. Here are the stakeholder profiles:
User: The LSDI system user representative (URep) is thegraduate student representative of the SCU Library
Users' Group. The URep is a computer science major,and would like a terse, powerful user interface language; but is aware that most library users would findsuch an interface too complex. He or she would likethe system to be available right away, with very fastresponse time, and as many functions and amenities aspossible.
Customer: The LSDI system customer representative(CRep) is the Director of Library Operations for SCU.The CRep has a library background, and has a gooddeal of experience as a library computing user, but noexperience in programming work. The CRep wants astable, low-risk, highly reliable system, with good security protection, usage monitoring, and with as manyuser amenities as possible within the allocated budgetof$I,350K (later $1,IOOK).
Developer: The LSDI system developer representative(DRep) is SCUs Manager of Application SoftwareDevelopment, within the Computing Services Operation. The DRep has a computer science background,and would like challenging, high-tech assignments forthe software development staff, but would not want ahigh risk of not meeting the budget, schedule, performance, security, or reliability objectives for theLSDI system.
Out of 36 project negotiations conducted by students only23 where used for this evaluation. Many negotiation samples could not be evaluated because data was incomplete ornot available. Some samples were dismissed because ofincommensurate team sizes (the project team consisted ofone or two students only) and these samples could not beseen as negotiations between different stakeholders sinceone person played two or more stakeholder roles.
Furthermore, students were given a questionnaire at theend of the term to acquire individual information such asage, work experience, facility in English (most studentswere international students) and other information. Theidea was to compare analysis results ftom the test sampleswith individual student data in order to find similarities.
Table 1 gives an overview of the average team characteristics, based on the data samples returned.
The table shows that only very few students spoke Englishas their mother language. However, most students ratedtheir facility in English as good (B) or excellent (A). Theaverage work experience includes programming, analyst,and other computer science related experiences.
Each team was given the fteedom to organize their groupsin whatever way they found most appropriate. This included the decision which member had to play whichstakeholder role during the negotiation.
Table 1 - Team Characteristics (3 members in each team)
j ! 1
Facility in English iB+! A 1 A I AI B B B B- A B+!B+J B-1 A B B B A A A A B iB+i A-
0!2 0 0 0 0 0 0101010 0001200
A~erag~ Work 4.711 O.5i 12 1 0.7 0 2 4.5 2 i 81313 8 I 5 4 1.3 10 2.510.7[2.7xpenence' ! i ! 'j j! 1 ! I ~ i
Average Age 31:27 25137 29132 25 26 25 25;29!30/26 42 27 33 35 26 29 36 24:25131
1/213/010/3:3/013/011/2fl/2 3/0 3/0 1/2i3/013/012/1i1/2,2/1:3/0:3/0:1/2i3/012/113/0'3/011/2: : ! : ! ~ : : 1 : ! ! : i ! i :. : :
Native English
MalelFemale
0:
I 2 3 4 5' 6 7 8 9 10' II 12 13 14 15' 16 17 18 19 20' 21 22 23
User Interest ProfileDDDDDDDDDDDDDDDDDDDD
Access Control
8:~:B:D:~H:~:~rH:r~~JB:~rB:H~:B:HHBtr~r0::
Acquisition Data,
D-D'O-D'~o-D'~'B'B~'8'D'B'8D'D'B-D'~on -~--Handl ing
User Services
,B·B·Bn'D
tJ,B·D·H-BB'o·B,B·eD'~'8'~'Bo,B·B..
Usage Analysis
DDDDDDDDDDDDDDDDDDDDDDD
Trend Analysis
DDDDDDDD DDDDD DDDD D
Library Network Access
000000000000000DODDo 0
Limitations and Constraints
The analysis had to focus primarily on the end results, andonly partially on the way people reached these results. Thiswas caused because only the final team results were captured (the negotiation exercise was followed by a storageintensive architecture exercise which caused the students
to delete their negotiation files before we had capturedthem; we have since developed a more robust instrumentation capability).
Another limitation was that it could not be assured that all
teams worked independently. Interaction and informationexchange might, therefore, have caused students to changetheir minds. However, based on observations during theshort time ftame in which the students had to do the re
quirements negotiation (only 2 weeks), it can be assumedthat information exchange was not very extensive.
PROJECT ANALYSIS
A number of hypotheses were formulated about the uni-
Figure 3 - Functionality Results
formity or repeatability of the negotiation results across the23 teams. The data analysis resulted in rejection of severalof these hypotheses, indication that the results were notrepeatable even in this prestructured situation. For someother hypotheses, however, the uniformity hypothesis wassustained by the data.
Functional and Non-Functional RequirementsSince functionality of the new system was the single mostimportant negotiation factor, the outcome of it is especiallyinteresting.
Because of the fact, that all students had a similar educa
tional background, the same negotiation goals, and a predefined process model to follow, it was assumed that mostteams would come up with uniform functional results forthe LSD! system. Figure 3 lists the functionality results ofall teams. No box means that this function was not built. A
light gray colored box means basic level, gray means extended level, and dark gray means rigorous level. The 23
groups produced 12 different results in functionality. The mostcommon result (see e.g. group 1 inFigure 3) was used seven times.The next common ones were used
three times, three times, and twotimes. All other teams negotiateddifferent results.
A significant size was the relativepriority of non-functional requirements, such as performance andreliability. It is ftequently hazarded that computer science students are more enamored with
performance than reliability. Thus,another hypothesis was that mostteams would choose the fasterprocessor Y instead of the slower,but more reliable processor X.This turned out to be false because
all teams chose to use processor X. Reliability was therefore rated far more important than performance, indicatingthat the student teams were doing a reasonable job of customer and developer role-playing.
Negotiation CardinalityAdditional hypotheses, based on the previous ones, werecreated which reflected assumptions about the negotiation
processes. For all teams to come up with uniform solutions, it was assumed that the solution paths (processes)must have been uniform as well. A uniform solution pathdoes not only imply a similar number of artifacts but also asimilar way of connecting and commenting them.
One set of hypotheses was therefore that the number ofartifacts, connections, comments, etc. created by thestakeholders would be similar for all teams (±10%).Thesehypotheses were clearly rejected as Tables 2 and 3 indicate. As indicated in line 3 of Table 2, the hypothesis ofuniform number of artifacts was rejected at the 19.2% level(a significance level of less than 5% is generally considered necessary for acceptance of a hypothesis). Lines 4-6 ofTable 2 indicate that the uniformity hypotheses for numbers of connections, comments, and attachments were re
jected between 57.5% and 75.3%.
Table 2 summarizes the experiment hypotheses and theirlevel of significance. The smaller the number the strongerthe support for this hypothesis (a level of significance below 5% indicates strong support). Table 2 highlights accepted hypotheses through gray shaded areas. Dark grayshaded areas indicate hypotheses with a level of significance below I % (very strong support). These significancelevels were used to accept or reject the hypotheses in thispaper.
The diversity in outcome is not only reflected in the functionality but also in the way the students used the Win Wintool. The average, minimal, and maximal number of artifacts, connections, comments, and attachments show highratios between the evaluated projects (I to 4 for artifacts, 1to 8 for connections). This is a clear indication that theproject groups came up with very different ways of solvingtheir problems .
To get more insight into the usage of artifacts, some hypotheses were formulated about the cardinality of the different artifact types (Win Conditions, Issues, Options, andAgreements). Two further hypotheses were that Win Conditions would be the most common artifact type becausethey represent the knowledge base and that there would bemore Options than Issues. These hypotheses were accepted(see the level of significance of less than I% for lines 21and 22 in Table 2). If artifacts are viewed in more detail(see Figure 4 left) it can be seen that the most common
artifact types were indeed Win Conditions, followed byAgreements and Options. Issues were used the least of allartifact types. This is reasonable because not every stakeholder Win Condition may cause conflicts and an identified conflict (Issue) may have more than one Option whichmight resolve it.
However, the ratios between the number of artifacts foreach stakeholder were somewhat unusual. It was assumed
that all stakeholders within all teams would participateequally during all 'phases' (see artifact chain in Figure 2)of the negotiation, thus, producing similar number of artifacts regardless of artifact types. On the right half ofFigure 4 the absolute number of artifacts for all stakeholders are presented. There, all users together had only a littlebit more than half as many artifacts as the customers.
Table 2 - Level of Significance for Hypotheses
No. Hypothesis Level ofSignificance1
Budgets are within 10% of $1, lOOK «1%
2
Schedules are within 10% of mean «1%
3
Number of artifacts are within 10% 19.2%...........................................................................................................................................
4 Number of connections are within 10% 57.5%................................................................................................... .............................
5 Number of comments are within 10% 69.4%.............
..............................................................................................................................6 Number of attachments are within 10% 75.3%
7
Customers have more artifacts than users <1%•••••••••••••••••••••••••••••••••••••••••••••••••••••.••• ,;, ••••• :'I:'I •• :'I •••• j,.:i. .••••• ~.•••••• _•••
............................8 ..Q~X:.~2P.~~~.~~X:..~2~~.~~.iX~~.~~.!M.l},,~f,~................. <1%.............................9 Customers have more artifacts than developers 13.8%
10
More comments on Issues than on Win Conditions 42.7%................................................................................................... ...........................
II ..~.?~~.~~~~~~~~.~~.!.~~~:.~.~.~.~~.~~..~:~E~:.~:.~~.........2.1%...........................12 More comments on Options than on Win Condi- 28.8%tions
.................................................................................................................................
13 ..~.?~~. ~.~~~~~~~.~~.~P'!.i.?l}.~.~~~~.~.l}..~.9!.:.:.~~.~~~......35%...........................14 More comments on win condo than on Agreements 2.1%
15..'!:!.~~.s:~~~! !.i.?l}.~.~~f.: ..~.~~~.~~~~~f..t.~.~..~~.~~~~........... 1.8%
............................16 ..'!:!.~~..~~~~!~.i.??~.."::~~7..~.~~~.~~~~~~.~~~l}.gp'~! ~~~........ 1.3%...........................17 ..'!:!.!~.s:~~~!~.i.?l}.~.~~f.:..~~~~.~~~~~f..t.~~.l}..~.~~~~~~~~ .. 2.4%...........................18 ..~.~~.~.7.~.~~f~.~~:.~.~~~~~~..t.~.7.l}.g.~.t.~?~~......................... 47.4%...........................19 ..~~f.:~~~~~~.~~~~.~~:.~.I.?~~~~.~~.~~..I.~~~~~.................. 30.4%...........................20 Agreements were used longer than Options 27.6%
21More Win Conditions were used than Issues, Op- <1%
...........
..H~!?~:.~f..~.~~~:.~:.~~~............................... i••••••••••••••••••••...........................22 More Options were used than Issues <1%
Table 3 - Summary of Win Win Entries
AverageMinimumMaximum
Artifacts
46.12180
Connections
65.523188
Comments
19.50101
Attachments
2.3016
450
400
350
300
250
200
150
100
50
o
winconditions
issues options
450
400
350
300
250
200
150
100
50
o
agreerrents customer developer user
Figure 4 - Artifacts Distribution
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Figure 5 shows even more surprisinglythat almost throughout all projects theusers tend to have less artifacts re
gardless of artifact types (Win Conditions, Issues, etc.) compared to those ofthe customers. Level of significancefor lines 7 to 9 in Table 2 rejects theuniformity hypothesis for users, butnot for customer vs. developer.
To clarify, Figure 5 presents the artifact type ratios of each individualstakeholder for all projects. The uppermost figure represents the artifactdistribution for the customers of all
teams. The middle and lowermost figures represent the same view for developers and users.
It can also be seen in Figure 5 that notall stakeholder participated during allphases of the artifact chain. Only approximately, two thirds of all stakeholders produced artifacts of all types.Interestingly again, the user was lesslikely to participate during all phasesthan the customer.
Figure 6 shows also that even the relative number of connections (relativemeaning the absolute number of connections of each stakeholder divided bythe number of artifacts of each stake
holder) used to associate artifacts mayvary significantly throughout all projects. For instance, many stakeholdersidentified the same or similar issues
but they connected them differently toother artifacts (e.g. Win Conditions
o Agreementso Options.IssueE1Win conditions
Team numbers
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 192021 2223
Figure 5 - Artifact Distribution for each Team and Stakeholder
5
o
30
25(f)
20
~ 0 15(f),e
::) '';:<10
30
25•..
20Q) (f) c.-o (.)_ co
15Q):t: > t:~tCt
10
50
30
25~~
20
E (.)o co
15U;~
8tCt
10
50
--------------------
A hypothesis was that the teams' negotiated budget wouldbe within 10% of available budget. The budget of the library system was restricted to $1,1 OOK for the final system. It turned out that all teams used up most of theirmoney saving in average less than $50K (or $IOOK at
and Options).
Furthermore, Figure 5 sheds some new light on a previoushypothesis as well. It was said above that the artifact distribution was very unequal for all projects and stakeholders. However, Figure 5 shows also that the number of artifacts created by the developer tend to be less variable thanthose of the other stakeholders.
It can also be seen trom this figure that the number of WinConditions (knowledge base) were very different. Thisshows that the negotiation processes started to be differentfor most teams right trom the beginning. Also, the factthat some win conditions were given prior to the negotiations did not prevent the students from finding new ones.
Cost and ScheduleThe cost and schedule for the software development werecalculated with the software cost estimation tool
COCOMO 81 [1]. This tool derives the cost and scheduletrom the effort required to build the system. The effort isbased on the estimated number of lines-per-code and a setof adjustable cost drivers, which allow the incorporation ofsoftware, project, and people factors. As discussed above,the teams were provided with size and cost driver ratingsfor each of the candidate Library SDI component choicelevels. The cost estimations yielded by COCOMO are typically accurate within 20% of the actuals around 70% of thetime.
Similar to the cost results, it was assumedthat the development schedules would bevery similar for all projects (±10%). Thisturned out to be true for almost all projects.(see also level of significance of line 2 inTable 2). Only one team decided to buildthe system faster by applying a lowCOCOMO schedule cost driver in theircost/schedule estimation. This cost driver
incorporates the effects of building the system faster by using more developer andsacrificing, for instance, functionality toafford it. Most teams came up with a sched-ule of 15 months. Two teams needed morethan 16 months. The team with the low
schedule cost driver was able to develop thesystem within 13 months. Despite the desire of the userstakeholders to build the system as fast as possible, onlyone team finally agreed to do so.
Negotiation InteractionThe main interaction method in Win Win is via Comments.
Comments can be attached to all types of artifacts (e.g.Win Conditions, Issues, Options) by all stakeholders. Additionally, Comments give stakeholders the ability to addtheir ideas to somebody else's artifact because only theowner of an artifact is allowed to change it's contents.
It was therefore interesting to know, whether students usedthe Comments extensively to do their negotiation orwhether they performed face-ta-face communication. Itwas assumed that stakeholders would use primarily Comments and only secondary face-to-face communication.Many groups chose to use Comments for their negotiation.However, face-to-face negotiation must have been used as
well because only few teams had a complete coverage of allnecessary information (e.g. critics, rationale, etc.). It israther unlikely that an average of one or two Commentsper artifact was enough to come up with decisions.
Furthermore, it was assumed that different artifact typeshave different needs for communication. The hypothesiswas that Issues and Option would be more controversialthan the other artifact types. Lines 10 though 14 in Table2 show that different groups had indeed different views onwhere to use Comments. It appears that Comments wereused far more trequently for Win Conditions, Issues, andOptions rather than for Agreements. It is, however, notvery clear which artifact type required the most negotiation. Nevertheless, Agreements seem to be the least controversial of all artifact types.
most). One team used more money thanactually allowed. Thus, no team saved morethan 10% of its budget (see also level ofsignificance for line 1 in Table 2).
EJrelations of customer• relations of developero relations of user
1.1 1
1[ 4 [
.I.I 1. ~
.~1
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Figure 6 - Relative Usage of Artifact Connections for each Stakeholder
4
3
2
a
3.5
2.5
0.5
1.5
Lines 10-14 in Table 2 does not, however, reflect face-to
face communication. Frequent face-to-face communicationwould imply that the tool would have been used synchronously most of the time. This kind of usage is of majorimportance for a groupware system because it should provide capabilities for both asynchronous and synchronoususage by multiple users. Synchronous use implies negotiation going on at the same time at any place (same or different place). Asynchronous use implies different time atany place (see also [7]). To distinguish between same anddifferent places was not feasible in this particular case because students had to use SUN workstations to do their
work and there was only one lab they could go to.
It was therefore assumed that the WinWin tool would beused mainly synchronously for most of the time (manystakeholder working at the same time). To answer thishypothesis consider Figure 7. This figure shows the creation and last revision time of artifacts (the message tablecontaining a list of all changes with a brief description ofthe nature of changes was not available for this analysis).The figure shows all project members mapped to a timetable which spans up to 13 days. Each project's time tableis encapsulated between two gridlines with the customeron the left, the developer in the middle, and the user on the
right.
The figure shows that most of the time two or more stakeholders were using the system at the same time. Nevertheless, the tool seems to have been used asynchronously aswell; often with gaps of one or two days in between. Thiscannot, however, clearly be derived from this figure because most revision time stamps were unavailable.
The figure gives also answer to another hypothesis. It wasassumed that the negotiation problem was rather simpleand would require only one or two sessions to solve it. Theteams' negotiation processes were, however, mostly complex. Figure 7 shows very clearly that most teams were notable to come up with a solution immediately (in the firstsession) but had to conduct multiple sessions over up to 13days. This might give an indication of the complexity ofthe task the students had to perform.
Lines 15 to 20 in Table 2 gives another view of the creation/revision figure. It compares the number of days aproject group spent for each artifact type (creation of thefirst artifact until revision of the last one). This may helpin answering the hypothesis that stakeholders would spentmost of their time working on Issues, Options, and Agreements. In fact, the time invested were very variable for allartifact types. However, the hypothesis was rejected be-
cause most time duration wasassociated with Win Conditions.
Issues, Options, and Agreementshad shorter time durations, partially because they were startedlater and partially because WinConditions continued to change.
12
10
8
6
4
2
o
- customer--
• - developer
.r- user
--. :--1-.
•• ••• •r ••-
•
--.
---•• 1--••••
- IE. - .-In
--.- - -
III!
••I •••• •••
•••••••••..- -
.-- -._!!lI_- ___ ••--- I •• =.
~-. .-,,
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Figure 7 - Artifact Creation and Revision over Time
RELATED OBSERVATIONS
The following observations weremade from additional data not
presented here due to limitationin space .
Stakeholder experience, age,etc. do not seem to have had a
strong influence on negotiationtime, number of conflicts identified, number of goals formulated, and so on.This statement is, however,blurred by the fact that the stakeholders had very different cultural backgrounds. Furthermore,the ages of stakeholders in thegroups were very variable whichmakes it difficult to compare.
A fundamental change in a stakeholder's goal can easily nullify all progress made prior to this point.This was the case, when the customers got the new constraint of having a budget of only $1,1 OOK instead of$1,350K to build the system. This change caused majorchanges in almost all teams. Many teams even restartedthe whole negotiation from the scratch.
People specified most of their goals prior to their negotiations. However, many people modified and even introduced new Win Conditions until the end of their ne
gotiations. Thus, the requirements negotiations werecyclic processes rather than sequential ones.This point basically follows the general trends in softwareengineering. Similar observations were made regardingwaterfall model vs. spiral model, specification vs. prototyping, coding vs. testing, etc. Nevertheless, one mighthave expected a rather sequential processing of the artifactchain in this particular case because the project was rathersimple and the students were supported with much moreinformation than they would have had in the 'real' world.
The spiral like usage re-enforces the initial motivations forcoming up with software requirements negotiations basedon the win-win spiral model as it is described in [4]. Further, the change in budget, as it was described above,shows that a cyclic approach is needed to accommodatesuch changes, but that it may require re-initialization formajor changes.
SUMMARY
Even though, the teams had a similar educationalbackground and basically the same Win Conditions,they came up with very different negotiation approaches and solutions.The 23 teams produced 12 different results in functionality. Repeatability was not achieved despite simplified project assumptions and a predefined negotiation process.Even those projects which came up with the same resultsachieved them through different negotiation paths.
Reliability and better functionality of the software systems were far more important for the stakeholders
than performance.All teams chose to use the more mature and reliable processor X instead of the newer, faster, but less reliable processor Y. In an environment which grows and changes asrapidly as does information processing, many, if not most,information system practitioners tend to play it safe. Itwas, however, surprising that this also turned out to betrue for young computer scientists, who are probably themost enthusiastic people when it comes to new computerrelated technology, when they played the roles of information system practitioners.
Tool supported interaction provided through Comments could not replace face-to-face communicationbetween the stakeholders.
The Comments method supported by Win Win may not bethe best method to support group communication and collaboration but it is an adequate one. Many students usedthem, however, only very few teams used them extensively.Providing a simple, e-mail like, communication methodseems, therefore, not to be enough to support full collaboration. Groupware systems may need to offer additionalcommunication methods, such as videoconferencing andothers. However, in many situations, a mix of automatedand manual methods is likely to be the best way to get thejob done.
Most people tried to solve their conflicts on an equallevel with similar participation. Only very few groupsseemed to have had a strong leader dominate the creation of artifacts.
When it comes to negotiation, it is very important that allcritical stakeholders have an equal opportunity to participate in the negotiation(s) and thus ensuring that everybodywill become a winner. A rather democratic way of solvingproblems on equal terms is therefore clearly most appropriate. Since students tend to be more democratic amongthemselves it is unclear whether this interaction behavior
was also encouraged by Win Win in any way (note: participation was 'measured' in number of artifacts and Comments used by stakeholders). This equal-participation pattern is probably characteristic of peer-to-peer negotiations(e.g. independent user, customer, developer), but might nothold across asymmetric power structures (e.g. bosssubordinate). Also, participation was not exactly equal, asdiscussed next.
Even though most stakeholders participated extensivelyduring the negotiation, the users produced clearlyfewer artifacts of all types than the other stakeholders.The users had fewer artifacts than the customers or devel
opers in almost all project groups regardless of artifacttypes. Playing the role of an user must not have been difficult because the role characteristics clearly described theuser as a computer science student who will use the librarysystem. It should not be assumed that this phenomenonwas caused because the user had fewer goals than the developer and customer. In reality the number of Win Conditions were almost as high as the equivalent ones of thecustomer and the developer. The user produced far lessIssues and Options, and these are activities were the userswere equally qualified to participate since all stakeholderwere computer science students in real life. Maybe theanswer is that the most significant Issue driver involvedwas meeting a limited budget, which affected the customers and developers more immediately than the users.
Most teams did not finish their negotiations in one session but required multiple sessions, negotiating for aslong as 13 days.Even though the problems the students had to solve wererather simple ones (many real life risks could be ignored orwere simplified in the task statements), it seems that it wasstill very difficult to come up with agreements. This indicates that collaboration is a very difficult task which requires some training. This was borne out by comments inthe students' post-project critiques, which indicated theyfelt they had learned a great deal about how to negotiaterequirements.
People tend to accept the results of analytic tools (suchas COCOMO) without questioning them.The cost estimation tool COCOMO was used not only todecide on cost but also to come up with schedule andfunctionality. However, people tend to accept its results as'given facts' even though its "accuracy within 20%, 70%of the time" was presented in class and in the model description. All teams used up most of their money($1, lOOK) saving in average less than $50K (less than 5%)for risk contingencies.
CONCLUSIONSThe results based on these facts may not necessarily beapplicable in all areas of group behavior because the teamsizes were rather small and people factors, such as experience and age, were diverse (see also [6] for other observations). Conclusions derived from this analysis must,therefore, considered suggestive rather than definitive withrespect to other situations.
Nevertheless, this experiment shows how different thesame problems may be solved by different people. A list ofgoals together with a general process model on how toproceed does not achieve repeatability. If we wish that thefollowing of a process will lead to repeatable results, thisraises the question on how detailed this process must be. Inthis context it seems that the quest for strong software repeatability may require so much proceduralization that thesoftware process becomes bureaucratized and dysfunctional.
ACKNOWLEDGMENTSThis research is sponsored by DARPA through RomeLaboratory under contract F30602-94-C-0 195 and by theAffiliates of the USC Center for Software Engineering:
Aerospace Corp., Air Force Cost Analysis Agency, AT&T,Bellcore, DISA, Electronic Data Systems, E-Systems,Hughes Aircraft, Interactive Development Environments,Institute for Defense Analysis, Jet Propulsion Laboratory,Litton Data Systems, Lockheed Martin, Loral FederalSystems, Motorola, Northrop Grumman, Rational Software, Rockwell International, Science Applications International, Software Engineering Institute, Software Rome
Laboratory, US Army Research Laboratory, and Xerox.
Special thanks to Ming June Lee, Brad Clark, CristinaGacek, and other students, faculty, and staff at the Computer Science Department, University of Southern California.
REFERENCES1. Boehm, B.W. "Software Engineering Economics,"
Prentice Hall, 1981
2. Boehm, B.W. "A Spiral Model of Software Develcpment and Enhancement," IEEE Computer, May 1988,pp.61-72
3. Boehm, B.W. and Ross, R. "Theory W Software Prqiect Management: Principles and Examples," IEEETransactions on Software Engineering, July 1989,pp.902-916
4. Boehm, B.W., Bose, P., Horowitz, E., Lee, M.J.
"Software Requirements As Negotiated Win Conditions", Proceedings oflCRE, April 1994, pp.74-83
5. Boehm, B.W., Bose, P., Horowitz, E., Lee, M.J.
"Software Requirements Negotiation and Renegotiation Aids: A Theory- W Based Spiral Approach", Proceedings ofICSE-17, April 1995, pp.243-253
6. Bullen, C.V., Bennet, J.L., "Learning from User E xperience with Groupware", Conference on ComputerSupported Cooperative Work, October 1990, pp.291302
7. Ellis, C.A., Gibbs, S.J., Rein, G.L., "Some Issues andExperiences", Communications of the ACM, Vol. 34,No.1, January 1991, pp.38-58
8. Horowitz, E. "Win Win Reference Manual: A Systemfor Collaboration and Negotiation of Requirements",Center for Software Engineering, University of Southern California Technical Report, March 1996
9. Humphrey W.S., "Managing the Software Process,"Addison-Wesley, 1989
10. Lee, M.J. "Foundations of the Win Win RequirementsNegotiation System," Ph.D. Dissertation, Center forSoftware Engineering, University of Southern California Technical Report, May 1996
II. McCue, C.M., "IBM's Santa Teresa Laboratory: Architectural Design for Program Development," IBMSystem Journal 17(1), ppA-25
12. Paulk, M.C., Weber, C.V., Curtis, B., Chrissis, M.B.,
"The Capability Maturity Model - Guidelines for Improving the Software Process", Addison-Wesley, 1995