THE PROCESS OF DERIVING REQUIREMENTS : LEARNINGFROM PRACTICE

Jennie M. Carroll1

Paul A. Swatman2

1School of Information TechnologySwinburne University of Technology

2School of Management Information SystemsDeakin University

Abstract

The process by which systems analysts, or requirements engineers, deriverequirements is poorly understood. The research reported in this paper involvesstudying professional requirements engineers at work in their everyday environments.The findings include the identification of a number of influences on requirementsengineers’ practice and elucidation of several roles that they play during the analysisprocess. Further areas for research are also highlighted.

Keywords

Requirements Engineering, Case Study

INTRODUCTION

The research reported in this paper is part of a research program aimed at improvingthe process of deriving requirements, often called systems analysis or requirementsengineering (RE). We believe that it is necessary to develop understanding of the REprocess before concentrating on tools, techniques and methods to be used to improvethis process. The case outlined here is the first in a series of case studies directed atbuilding knowledge and theory about the practice of real-life requirements engineersworking within their everyday environments. The findings constitute some learningabout how RE is performed, and point to areas of RE practice into which furtherresearch is needed.

A high-quality process for deriving requirements is critical for developing successfulinformation systems (IS). Constructing a usable, effective and efficient businesssolution necessitates clear understanding of what is required, by which stakeholders,within the given constraints. This understanding needs to be communicated to thestakeholders - to check whether their needs have been accurately and completelycomprehended - as well as the technical staff involved in developing systems - whoneed clear, concise and unambiguous specifications in order to deliver what thestakeholders require.

Successful requirements engineers need a range of personal, social and technical skills.They deal with a variety of stakeholders having different views of the proposed system,

investigate the domain in which the system will operate, evaluate present and futureinfluences on the success of the system and model possible problem solutions in arange of formats. Exactly how they use these skills to produce a requirementsspecification is not clearly understood. This is partly because practising requirementsengineers appear unable to describe how they transform stakeholders’ needs into aformal specification document (Lubars, Potts and Richter 1993), partly because theprocess is creative and cognitive and so has few external markers, and partly becauseof the nature of existing research.

Much of the literature on the RE process has shortcomings, most acutely in itsrelationship with the practices of professional requirements engineers working in thereal world. Much of the published research has some or all of the followingcharacteristics - it examines students rather than practitioners, studies the solution ofcontrived problems (where the scenario is already derived rather than dealing withconflicting viewpoints held by a range of stakeholders), and presents cases performedin a laboratory rather than in the field (see, for example, Adelson and Soloway 1985,Guindon 1990, Sutcliffe and Maiden 1992, Vessey and Conger 1994). This researchproject aims to study requirements engineers operating in context to increase ourunderstanding of RE.

THE RESEARCH METHOD

This research project aims to build knowledge and theory about requirementsengineering. A case study approach is appropriate because the research emphasis is onhow requirements engineers operate in their real-life environment. The case selectedinvolves the RE phase in the development of an Information Broker, a Web-basedapplication to locate and supply heavy machinery (see Hunt and Swatman 1997). Theidea for the application originated with a consultant having extensive experience indealing with large mining and engineering companies. He saw that the purchasing andsupply of heavy equipment could be facilitated by the Internet, and conceived a mentalmodel of how such an application could operate. The consultant was seeking totransform these ideas into a formal set of requirements that could be given to asoftware house to build the application. He worked with an assistant who hadpreviously worked in the Defence industry. They are the clients in the case study, andwere backed financially by an organisation having experience in providing businessapplications, but without experience in the Internet.

The case of the Information Broker was selected for convenience and itscharacteristics include:• it involves developing a new and relatively simple application• the application is market-driven rather than customised for any one group of users• only one set of stakeholders is involved in the RE process, so there is no viewpoint

analysis or conflict resolution between stakeholders• there are no identified users; rather, one client has a view of the type of

organisations which would be interested in the new application• some hardware and software had already been purchased, and• the client expressed interest in using object oriented methods.

The application is now being built by a software house, with project management ofthe development being performed by the clients’ backing company.

A weakness of case studies, and qualitative research generally, is a perceived lack ofrigour. Suggestions for overcoming this centre on the use of additional structure(Checkland 1991, Dick 1992). A variant of case study research, called structured-case(Carroll, Dawson and Swatman 1998), is used in this research. It assists researchers infocusing and ordering the research process as well as evaluating and justifying theoutcomes of this process as follows:• a conceptual framework is specified prior to entering the research siteThe conceptual framework determines what is to be examined and guides thecollection, analysis and interpretation of data (Miles and Huberman 1994). Theconceptual framework for this project was developed as a result of four influences: theresearch themes, the researcher’s theoretical foundations, the existing literature andinsights from a requirements engineer.• it has an iterative research cycleOnce the conceptual framework is defined, a cycle of planning, data collection, analysisand reflection provides structure to ensure rigour as well as the flexibility to includeunexpected outcomes. The research cycle, adapted from action research (see Susmanand Evered 1978), is used to expand, enrich and validate the conceptual framework,laying the foundations for theory building.• it focuses on theory buildingIn this iterative, cyclical method, the research is planned, research propositions refined,data collected, analysed and interpreted and the outcomes evaluated. Reflection on theimplications of the outcomes of data analysis for the overall research project may resultin changes in the conceptual framework before another cycle is started. This is howtheory is built: the conceptual framework represents the current state of theory andchanges in it represent the theory built to date.

The Initial Conceptual Framework

The conceptual framework (see Figure 1) defines the areas of interest of the research,in this case the stakeholders of the system within context and the transformation ofdata about the problem domain by the requirements engineer. The stakeholders (allthose with a stake or interest in the new system) are represented within the problemdomain, as their needs can only be understood in context. Influences within theproblem domain include the environment, existing domain knowledge, technology,organisation and theories about computing. Data about the problem domain istransformed by the elicitation, representation and validation sub-processes (for furtherdetails, see Carroll and Swatman 1997).

Data Collection and Analysis

The requirements engineer was employed to assist with specifying the requirements,so that system development could be contracted out to a software house. Theresearcher was a participant observer: welcome to participate in the process, make

Figure 1 The initial conceptual framework, CF1

comments and suggestions as well as ask questions. For the first two days, therequirements engineer examined documentation developed by the clients, givingbackground of the system and their views of the requirements. Data recorded includednotes from observation, representations sketched by the requirements engineer and theresearcher’s reflections on the process.

During the next two days, the requirements engineer worked with the two clients todefine the requirements for the new system. Data was collected by observation andinterview, comprising detailed notes of conversations and the representations used bythe requirements engineer; the researcher’s interpretations noted during the sessions,as a result of reviewing the notes at the end of each day, as well as at the end of theprocess; a videotape of one session; and notes made of an interview with therequirements engineer after the final session.

Initial analysis of the data was performed using coding (see Miles and Huberman1994). Each concept in the conceptual framework was used as a code, plus an “anyother” code in each section to allow for unexpected findings. The initial coding wasused to retrieve and organise chunks of transcripts. Analysis is an ongoing, iterativeprocess: at first the codes were used to match concepts and sub-processes from theproblem domain with sections of the text. On re-reading the text, further analysis andinterpretation led to adding new codes. Therefore, the process of reading, re-readingand interpreting the data led to reviewing and revising the codes.

FINDINGS

The findings are presented in relation to the main aspects of the conceptual framework:the problem domain, the analyst’s or requirements engineer’s domain, and theinteraction between the two.It should be noted that, due to the characteristics of the application being developed,not all influences or concepts in the initial conceptual model could be observed. Some

Stakeholders

Organisation

Environment

Technology Existing domain knowledge

Theories

Elicitation Representation Validation

Domain knowledgeand requirements

Models and requirements

Problem domain

Analyst’s domain

Knowledge Informal & formal

models

of the findings may be related to the characteristics of the Information Brokerapplication or the relative simplicity and brevity of the RE process.

The Problem Domain

Four of the influences depicted in the problem domain in the initial conceptualframework were observed:• environment

The influence of government and legislative requirements was observed. A far strongerinfluence, which pervaded the entire RE process, was that of “commercialimperative”. Commercial pressures - in particular, the need to gain competitive andstrategic advantage - meant that time and speed to market were of great importance tothe clients.• organisation

The importance of the business rules of the backing organisation to the clients’ view ofthe system were critical. The client reiterated the aims and rules he believes areimportant to the operation of the Information Broker; often discussing the businesscontext gave rise to further requirements.• technology

The software and hardware purchased for the Information Broker had some influenceon the RE process, particularly later in the process when the requirements becameclearer. During discussions about whether the purchased software could handledrawings, the requirements engineer stated that “[It is] Not a requirements analysisissue - but a detailed design issue”. However, the requirements engineer could notignore the constraints that existing technology placed on the system.• theories

One client expressed a preference for using object oriented (OO) methods for theproject, citing advantages of reusability and better engineering. This influenced thedirection of the RE process for three and a half days; it was only during the last halfday that OO was rejected in favour of conventional data base technology. Therequirements engineer argued that the system would be “made much more difficult bychoosing OO. A basic database is all we need....Tried and trusted technology.”

An additional influence on the clients was observed: the stakeholders’ existingknowledge of computing. One of the clients is a specialist in electronic commerce, andso has extensive knowledge of the standards, protocols and technology available forWeb-based electronic commerce. This meant that the client had superior technicalknowledge to the requirements engineer in a very narrow area of the problem domain,which was valuable when discussing constraints to possible solutions.

The Analyst’s Domain

Influences on the Requirements Engineer

It is clear that, just as the stakeholders’ requirements need to be understood in context,so too the requirements engineer’s actions can only be understood within context.Aspects of the analyst’s context include:• theories about IS and RE; the world-view of the analyst

It is expected that requirements engineers’ views of their role will influence theiractions. In the Information Broker case, while the requirements engineer worked co-operatively with the clients, his view is that of the problem-solver (and he resistedefforts of the client to “solve the problem” or look at design issues until therequirements were clearly understood).• familiarity with the domain

The requirements engineer is not an expert in the problem domain, Web-basedelectronic commerce; consequently, time was spent gathering information about it.Domain knowledge is seen as important for developing software (Curtis, Krasner andIscoe 1988) as well as problem solving generally (Mayer 1992:390). If analysts anddesigners lack such knowledge, there may be significant costs in learning about it(Walz, Elam and Curtis 1993). In the case of the Information Broker, the process ofderiving requirements was similar to the sub-processes performed when developingoff-the-shelf software, although a range of additional issues such as providing servicesfor the marketing function, the tight time constraints and adding value to an existingbusiness process meant that the requirements engineer was working outside his domainof specialised knowledge and experience. The requirements engineer overcame this byusing similarities with previous situations - seeking commonality not difference - andwhen faced with an unfamiliar situation, being very careful to discover exactly whatneeded before attempting to provide a solution.• familiarity with the type of problemProblems may vary in whether they are structured or unstructured, soft or hard (seeCheckland and Scholes 1991); the specific nature of problems means that differentapproaches are needed. Mayer (1992:413) argues that problem solvers discriminatebetween problem types to categorise problems based on solution plans. In the case ofthe Information Broker, the requirements engineer “solved” the problem when hecategorised the Web-based commerce application as a data base problem. As such,the requirements engineer’s unfamiliarity with the problem domain was not a criticalfactor in the RE process; seeking similarities in the type of problem caused thebreakthrough in the solution of the problem.• experience in RE

A requirement engineer’s experience is a vital influence on practice. This experienceneeds to cover a range of areas: business contexts, types of systems, representationsand problem types as “Breadth of experience is a better predictor of individualperformance than years of experience.” (Walz, Elam and Curtis 1993:74). Therequirements engineer’s broad experience in systems analysis enabled him to overcomehis lack of familiarity with Web-based electronic commerce; similarities with previousproblems encountered helped him work towards resolution of this problem. This wasevident when discussing project management of the Information Broker developmentwhich he emphasised that it could not be done as “quick and dirty”. He argued thatthe system would evolve over time, so its foundations must be sound, with clearstandards to guide current and future development.• learning and problem-solving approachIn practice, learning and problem solving are not distinct, but rather learning is anessential part of problem solving. Learning involves collecting data and then analysingit to develop the information and knowledge which is the basis of problem-solving.There was a clear point during the Information Broker case when a breakthroughoccurred and the problem was solved; this happened during the second session of thefinal day. The requirements engineer realised that the Information Broker was

basically a simple system; a data base solution was chosen, presented and thenevaluated. The requirements engineer sketched the outlines of the application whichwould solve the problem, and reiterated the solution many times (to educate the clientabout this new view of the system). The clients subjected the solution to a SWOTanalysis, then eventually moved on to discuss project management of the systemdevelopment (in its new form): the solution was evaluated and accepted.• existing domain knowledge

Many problem domains are well-understood, with an existing body of knowledgewhich a requirements engineer can access. It was noticeable in this case that - as theWeb is relatively new with scant existing knowledge - the requirements engineer wasreliant on the clients for domain knowledge and had no recourse to any larger, moregeneral body of domain knowledge.

How the RE sub-processes are performed

The initial conceptual framework depicts the requirements engineer as drivinginteraction with the stakeholders through the elicitation, representation and validationsub-processes. The performance of these sub-processes was seen as “unstructured, adhoc and possibly chaotic, rather than systematic” (Carroll and Swatman 1997:461).Observations of the sub-processes in the Information Broker case occurred on twodifferent levels:• moving from sub-process to sub-process (the pattern of elicitation, representation

and validation), and• performing the sub-processes on different parts of the problem (movement

between the areas of the problem being analysed).

The pattern of elicitation, representation and validationInitially, much attention was placed on determining the system’s constraints: what wasneeded to be delivered and by when. This was performed in question-and-answermode. Once the deliverables and their deadlines were established, the requirementsengineer focused on the functionality needed for the application. A fairly orderly,sequential pattern of eliciting then validating chunks of information was observed:elicitation, elicitation, elicitation, perhaps some representation, then validation, to giveunderstanding of a fragment of the problem. Not much representation was seen -occasional sketches and diagrams were used, and verbal representations of sections ofthe problem were presented for validation. It should be noted that this pattern ofperforming the sub-processes, and the lack of diagrammatic representations, may bedue to the simplicity of the system being studied.

Movement between the areas of the problem being analysedExamining the areas of the problem being tackled through the elicitation,representation and validation sub-processes gives a different view of the RE process.The requirements engineer spends some time on one area, fills in some detail, moveson to another area then back to the first area. A common theme throughout the REprocess (noted by plotting this movement diagrammatically, as shown in Figure 2) isthat one problem area is not “solved” before the requirements engineer moves on toanother area. The requirements engineer commented in the post-process interview thathe may leave a difficult area for a while and move on - to approach it later from adifferent angle or with a new insight.

A__________ A = eliciting what is needed, by what__________ date.__________ B

__________ B = giving advice about management of __________ the project __________ C __________ C = eliciting information about the __________ interface with client software____________________

Figure 2 A sample of movement between the areas of the problem being analysed

In the section of the RE process diagrammed in Figure 2, the requirements engineerstarts at A (eliciting what is needed, by what date). After three chunks of informationare gathered, the focus moves to B (giving advice about management of the project).After three areas of discussion about project management, the focus moves to C(eliciting information about the interface with client software). More discussion aboutproject management precedes further elicitation of what is needed, by when. Atintervals, the requirements engineer’s understanding of the information gathered isvalidated, by presenting his views of the requirements to the clients; in this way, heconstructs fragments of understanding of the problem.

Consolidation of these validated fragments to create understanding of the problem as awhole appears unstructured and opportunistic, rather than incremental andevolutionary. Sometimes increased understanding of A leads to development ofunderstanding of a large piece of the problem. Insights into the nature of the problemand its context lead to resolution of problem pieces and fitting them into the whole.The solution to the Information Broker case is an example of such insights: at onepoint the requirements engineer stated “There’s not much there. It’s quite a simplelittle system ” and changed the focus of all subsequent discussions.

The observed performance of the RE process is similar to opportunistic problem-solving, which is “characterised by frequent discovery and/or adaptation of goals andactivities, is response to changing circumstances” (Khushalani, Smith and Howard1994:13). Insights from working on parts of the problem lead to changes in theapproaches to the problem. Thus, derivation of requirements is an ever-changing, ever-adaptive, non-structured and intuitive process, driven by what is best described asinsights, opportunism and breakthroughs rather than incremental, evolutionary or eveniterative methods (as insight may resolve a situation first up, or after repeated tries).These characteristics of the RE process can be understood in relation to two basicaspects of RE - the nature of the problems encountered (soft or unstructured, whichcannot be defined and have no clear solution) and the nature of human problem solvingwhich is “dynamic and evolutionary, requiring continuous adaptation ... to deal witha broad spectrum of ideas” (Khushalani, Smith and Howard 1994:14).

Interaction Between the Domains

The initial conceptual framework modelled the interaction between the requirementsengineer and the stakeholders in a limited way, showing the requirements engineereliciting and representing information which is then validated with stakeholders. Thissuggests that the process is driven and controlled by the requirements engineer whoperforms a narrow range of activities. Observing the RE process for the InformationBroker shows that the interaction is much more complex than this. The dominantimpression of this RE process is that the requirements engineer is providing a service.This service takes many forms and, like all quality services, is responsive to the needsof the customer as well as fulfilling the professional responsibilities of a requirementsengineer. It includes solving a business problem and deriving a requirementsspecification. This is not achieved solely as a result of the requirements engineer’sactions. At times, the requirements engineer directed the process to a certain area, thenthe main client would take control and move to another area that he thought wasimportant. The clients actively educated the RE team about aspects of the system andbusiness which they felt were important and compelled the RE team to consider theseissues. Thus, the interaction was more negotiated and co-operative than driven by therequirements engineer. The clients were not passive, supplying information andvalidation on request; rather they actively shared in control of the process.

IMPLICATIONS FOR THE CONCEPTUAL FRAMEWORK

These findings led to reviewing and revising the conceptual framework. Changes to thecodes used in the analysis (derived from the concepts contained in the initialconceptual framework) were incorporated into the updated conceptual framework..The concepts in the problem domain were revised - one was added and another movedto the analyst’s domain. Influences in the analyst’s domain were added, so that thecontext in which the requirements engineer operates is explicit. The representation ofthe interaction between the two domains was changed to reflect the dual input andcontrol of the RE process. The first revision of the conceptual framework arising fromthe Information Broker case (CF2a) is shown in Figure 3.

Figure 3 The first revision of the conceptual framework, CF2a.

Additional analysis of the data led to the emergence of a number of themes related tothe interaction between the requirements engineer and the clients including conflictresolution, education, technical and business advice and management of the REprocess also occur.• conflict resolutionThe positive role played by the clients in the RE process means that conflict resolution,between the clients and the requirements engineer rather than between differentstakeholders, is important. At times, this is a complex and delicate task; therequirements engineer needs well-developed social skills to manage this interactionwithout offending the clients but still focusing on what was needed to derive therequirements. The requirements engineer avoided confrontation on contentious issues,but returned later to such issues, taking a slightly different tack. Walz, Elam and Curtis(1993) observe that conflict may facilitate learning in RE and design; conflict needs tobe effectively managed to maximise benefits for the process.• mutual education

Requirement engineers have imperfect knowledge of the problem domain (and of thedifferent stakeholders’ views of the critical issues). In the Information Broker case, thisknowledge was added to by eliciting information; at other times, the clients activelyeducated the RE team, for example about the business, the domain in which the systemwill operate and the main client’s conception of how the system will operate. Theclients actively worked to fill in areas that seem to be lacking. The requirementsengineer educated the clients about possible business, technical and process choices,drawing on his past experience and business and technical knowledge. For example,the requirements engineer spent considerable time discussing project management ofthe Information Broker project, the feasible options and their implications so that theclients could make an informed choice for managing the system development.• technical and business advice

Elicitation Representation Validation

Stakeholders

Method used

Theories - IS, RE and world view

Existing domain knowledge

Familiarity with domain

Experiencein RE

OrganisationEnvironment

Technology

Familiaritywith type ofproblem

Problem-solving approach

Analyst

Theories

Knowledge of computing

The requirements engineer offered a range of technical advice, especially after thedecision to use data base technology was made. Advice on how to set up the data baseand interface it with the Web reflected his expertise. Because the requirementsengineer was not expert in the Web commerce domain, and the main client hadconsiderable expertise in Web technology, the client offered technical advice in his areaof expertise. The requirements engineer perceived various business opportunities ofwhich the clients were unaware, and offered advice about maximising the financialbenefits of the system. One example involved the requirements engineer’s idea thatthere ‘must be money in the order of the list”, and suggesting that suppliers could payto be on the top of the list of Information Broker suppliers.• management of the RE processThe requirements engineer sought to bound and focus the interaction, so that he couldfollow through a line of thought to its end. Also, he was concerned that the mainclient was trying to “solve” the problem prematurely rather than fully understand therequirements first. Developing the Information Broker was subject to time pressuresbeyond those common to all commercial systems development, and the concept of anInformation Broker was already in the public domain (see Hunt and Swatman 1997).The requirements engineer warned against trying to go too fast and oversimplifyingthe requirements; he dealt with these pressures by keeping the interaction focused andbounded.

The variety of the themes emerging from the data analysis indicates that severaladditional roles are performed by requirements engineers beyond eliciting, representingand validating information. These roles involve human and social skills, as well astechnical skills: the requirements engineer uses a range of socio-technical skills whenoperating in a socio-technical environment. Depicting the richness of this interactionmeans that revision of the conceptual framework is needed. The full range of themesevident in the interaction between the problem and analyst domains has not beenincluded, as they are still poorly understood; including them in an inclusive model ofthe RE process requires significant additional reflection as well as more detailedresearch and knowledge. However, management of the RE process is included in thesecond revision of the conceptual framework; the three sub-processes and otheraspects of the interaction between requirements engineers and clients (conflictresolution, education and providing technical and business advice) must all bemanaged. The second revision of the conceptual framework, CF2b, is presented inFigure 4.

SUMMARY AND CONCLUSION

There has been significant learning from the Information Broker case. The process ofevaluating the implications of the data analysis and reflecting on the observationshighlighted omissions, errors and ambiguities in the initial conceptual model. Insightswere gained as a result of observing practising requirements engineers and additionalreading was undertaken to clarify these observations. As a result, the conceptualframework was twice modified, reflecting the incremental construction of knowledge.

Figure 4 The second revision of the conceptual framework, CF2b.

The latest version of the conceptual framework of the RE Process (CF2b)demonstrates this learning: the concepts in the problem domain have been refined andvalidated; a range of influences on the analyst has been included; the interactionbetween the domains now reflects its co-operative nature; and the importance ofmanaging the RE process is acknowledged. In addition to confirming, adding and re-placing concepts is the process of noting emerging themes, where deeper interpretationand reflection on the findings has led to ideas about the interaction between theproblem and analyst’s domains. A number of themes have been noted for furtherresearch; these include conflict resolution, education and technical and business advice.

Requirement engineers bring theoretical and technical knowledge, along with breadthof experience, to the RE process. The stakeholders bring detailed knowledge of theirdomain and experience of the problem situation. Neither has superior knowledge,rather they are both experts in their own domains; it is the interaction between thesetwo groups of experts that is the basis for the problem solving required from the REprocess. It is in this area that further research and reflection will bring additionalunderstanding of the RE process.

REFERENCESAdelson, B. and Soloway, E. (1985) The Role of Domain Experience in Software Design,

IEEE Transactions on Software Engineering, 11, 1351-1360 (Nov).Carroll, J.M. and Swatman, P.A. (1997) “How Can the Requirements Engineering Process be

Improved?” in D.J. Sutton (ed.) Proceedings of the 8th Australasian Conference onInformation Systems, Darlinghurst: The Australian Computer Society, 458-469.

Carroll, J.M., Dawson, L.L. and Swatman, P.A. (1998) Using Case Studies To Build Theory:Structure and Rigour.

Elicitation Representation Validation

Stakeholders

Method used

Theories - IS, RE and world view

Existing domain knowledge

Familiarity with domain

Experiencein RE

TheoriesOrganisation

Environment

Technology

Familiaritywith type ofproblem

Problem-solving approach

Analyst

MANAGEMENT OF THE RE PROCESS

Knowledge of computing

Checkland, P. (1991) “From Framework through Experience to Learning: the Essential Natureof Action Research” in H.-E. Nissen, H.K.Klein and R.Hirschheim (eds),Contemporary Approaches and Emergent Traditions, North-Holland, 397-403.

Checkland, P. and Scholes, J. (1991) Soft Systems Methodology in Action. Chichester: Wiley.Curtis, B., Krasner, H. and Iscoe, N. (1988) A Field Sudy of the Software Design Process for

Large Systems, Communications of the ACM, 31:11, 1268-1287 (Nov).Dick, B. (1992) So You Want To Do An Action Research Thesis? Internet document,

University of Queensland.Guindon, R. (1990) Knowledge Exploited by Experts During Software System

Design,International Journal of Man-Machine Studies, 33:3, 279-304 (Sept).Hunt, B.W. and Swatman, P.A. (1997) “The Information Broker as an Alternative to Web

Search Engines in Sourcing Potential Suppliers of Goods and Services” in D.R. Vogel,J. Grizar and J. Novak (eds), Proceedings of the Tenth International Bled ElectronicCommerce Conference: Global Business in Practice. Vol 1: Business. Kranj,Slovenia: Moderna Organizacija, 166-182.

Khushalani, A., Smith, R. and Howard, S. (1994) What Happens When Designers Don’t Playby the Rules: Towards a Model of Opportunistic Behaviour in Design, AustralianJournal of Information Systems, 13-31 (May).

Lubars, M., Potts, C. and Richter,C. (1993) A Review of the State of the Practice inRequirements Modeling’, RE’93: Proceedings of the IEEE International Sdyposiumon Requirements Engineering, Los Alamitos, CA: IEEE Computer Society Press, 2-14.

Mayer, R.E. (1992) Thinking, Problem Solving, Cognition, 2nd ed. New York: W.H. Freemanand Company.

Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis. 2nd ed. ThousandOaks,CA: Sage.

Susman, G.I. and Evered, R.D. (1978) “An Assessment of the Scientific Merits of ActionResearch’, Administrative Science Quarterly, 23:4, 582-603 (Dec).

Sutcliffe, A.G. and Maiden, N.A.M. (1992) Analysing the Nove Analyst: Cognitive Models inSoftware Engineering, International Journal of Man-Machine Studies, 36, 719-740.

Vessey, I. and Conger, S.A. (1994) Requirements Specification: Learning Object, Process andData Methodologies, Communications of the ACM, 37:5, 102-113 (May).

Walz, D.B., Elam, J.J. and Curtis, B. (1993) Inside a Software Design Team: KnowledgeAcquisition, Sharing, and Integration’, Communications of the ACM, 36:10, 63-76(Oct).

Yin, R.K. (1984) Case Study Research: Design and Methods, Beverly Hills,CA: Sage.

COPYRIGHTJennie M. Carroll and Paul A. Swatman © 1998. The authors assign to ACIS and educational andnon-profit institutions a non-exclusive licence to use this document for personal use and in courses ofinstruction provided that the article is used in full and this copyright statement is reproduced. Theauthors also grant a non-exclusive licence to ACIS to publish this document in full in the ConferencePapers and Proceedings. Those documents may be published on the World Wide Web, CD-ROM, inprinted form, and on mirror sites on the World Wide Web. Any other usage is prohibited without theexpress permission of the authors.