[email protected] Daniel M. Berrydberry/ATRE/Slides/HistoryOfREandSE.pdf · PSL/PSA: A Computer -Aided Technique for Structured Documentation and Analysis of Information Processing

The Prehistory andHistory of RE (+ SE)as Seen by MeDaniel M. BerryUniversity of Waterloo, [email protected]

2017 Daniel M. Berry Requirements Engineering at 40 My View of the Prehistory & History Pg. 1

Outline (Pictorial) RE

JHS Adder

HS FORTRAN time (notProg- to scale)

BS ram-ing contemporaneity

PhD PLs PLs

UCLA FMs begatSecure

SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

VocabularyCS = Computer Science

CBS = Computer-Based System

SW = Software

PL = Programming Language

FM = Formal Method

SE = Software Engineering

EP = Electronic Publishing

RE = Requirements Engineering


Overall Focus

We will see that my focus has always been onwriting correct and good SW, even while Ihave been in many different, SW-related fields.

My progression through PLs, FMs, Security,SE, and finally RE, has been to follow what Ithought would help most to achieve thatfocus.

That is, when I specialized or shifted fields, itwas because I thought the field I was in wasnot getting to the root of the problem.


Origin of These Slides RE

These slides are an enhancement of slidesprepared for a keynote at a 2017 workshopcelebrating the 40th anniversary of the birth ofRE in 1977.

The workshop organizers had identified theJanuary 1977 issue of IEEE TSE as markingthe birth of RE.


,; IEEE TRAN8SACTIFNlN7; ON

SOFTWAREENGINEERINGJANUARY 1977 VOLUJME SE-3 NUMBER I

A PUBt 1CATION OF THE IEEE COMPUTER SOCIETY

:il i < i. (C)LiIFTIO\ ON Rl 1)1Q( 0 i NT -\ \ . iiS

(G uest Editorial-- Rcflcctio(ns on RequirelCnts ...... 1). 1. R,,I sStructured Analysis for Rcquircinients Definiaioll n.J). T. R'. aold K. I" S& iomim. Jr.Strluclturcd Analysis (SA): A Luan!uagc for C( nlluil, ,,ti........i.....d .. ............... ). 7' Ro.vsAutoinated Softfware Engincering Through Strucured \Ianwl.rcalt . ...... C. <1 Irm-iniem and .1 11. fIra1cltiPSI. I'1SA: A Computer-Aided Tecniquiielc for Siructioie(l t)ocueial nand An i lv>is ol 1nlormaion Processing

Systeins .1). Tichr,ow and Ii. -1. lershev 111A\a EIxtendable Approach to Computer-Aided Sofma-ta Requiremmint,s 5nginccrinog

....................... 7 Bliel, ) C. 8ix lr, 1,,'81. L'. JierA Requirements Engincering Methodology for RclA-Timc ProccsC'.ing Reqcuireimnlit ,1. lU'. AlfOard[he1C Software Development System .(.CG. 1)ix (i, e Kl.I 'i

PAPIpRS

I1nall/sis oJ Al/goriih;n.s

iluhoit roccssor Scheduminlg with the Aid of Netwxork Flow \l-orithnis IS..StoneIv o Mcthods for the Efficient Analysis of Meniory Address Trace D)ata .1. J. .S11mith

ACKNO W LED.G MI)(i\IINT 01 RIF It ................I. .....................I...l... ). T7Yeh

I

1 634

4 1

49)606)9

94

FM ] OR 'SW T I( 1: ..... P T )"ch

I ()

SPECIAL COLLECTION ON REQUIREMENTS ANALYSIS

Structured Analysis for Requirements DefinitionStructured Analysis (SA): A Language for Comunicating IdeasAutomated Software Engineering Through Structured ManagementPSL/PSA: A Computer -Aided Technique for Structured Documentation and Analysis of Information Processing

SystemsAn Externdable Approach in Computer-Aided Software Requirements Engineering

A Requirements Engineering Methodology for Real_time Processing RequirementsThe Software Development System

Guest Editorial --- Reflection on Requirements

,; IEEE TRAN8SACTIFNlN7; ON

SOFTWAREENGINEERINGJANUARY 1977 VOLUJME SE-3 NUMBER I

A PUBt 1CATION OF THE IEEE COMPUTER SOCIETY

:il i < i. (C)LiIFTIO\ ON Rl 1)1Q( 0 i NT -\ \ . iiS

(G uest Editorial-- Rcflcctio(ns on RequirelCnts ...... 1). 1. R,,I sStructured Analysis for Rcquircinients Definiaioll n.J). T. R'. aold K. I" S& iomim. Jr.Strluclturcd Analysis (SA): A Luan!uagc for C( nlluil, ,,ti........i.....d .. ............... ). 7' Ro.vsAutoinated Softfware Engincering Through Strucured \Ianwl.rcalt . ...... C. <1 Irm-iniem and .1 11. fIra1cltiPSI. I'1SA: A Computer-Aided Tecniquiielc for Siructioie(l t)ocueial nand An i lv>is ol 1nlormaion Processing

Systeins .1). Tichr,ow and Ii. -1. lershev 111A\a EIxtendable Approach to Computer-Aided Sofma-ta Requiremmint,s 5nginccrinog

....................... 7 Bliel, ) C. 8ix lr, 1,,'81. L'. JierA Requirements Engincering Methodology for RclA-Timc ProccsC'.ing Reqcuireimnlit ,1. lU'. AlfOard[he1C Software Development System .(.CG. 1)ix (i, e Kl.I 'i

PAPIpRS

I1nall/sis oJ Al/goriih;n.s

iluhoit roccssor Scheduminlg with the Aid of Netwxork Flow \l-orithnis IS..StoneIv o Mcthods for the Efficient Analysis of Meniory Address Trace D)ata .1. J. .S11mith

ACKNO W LED.G MI)(i\IINT 01 RIF It ................I. .....................I...l... ). T7Yeh

I

1 634

4 1

49)606)9

94

FM ] OR 'SW T I( 1: ..... P T )"ch

I ()

SPECIAL COLLECTION ON REQUIREMENTS ANALYSIS

X

Structured Analysis for Requirements DefinitionStructured Analysis (SA): A Language for Comunicating IdeasAutomated Software Engineering Through Structured ManagementPSL/PSA: A Computer -Aided Technique for Structured Documentation and Analysis of Information Processing

SystemsAn Extendable Approach in Computer-Aided Software Requirements Engineering

A Requirements Engineering Methodology for Real-Time Processing RequirementsThe Software Development System

Guest Editorial --- Reflection on Requirements

We

In the following, at any time, …

“We” = all the people in whatever field I was inat the time.

So it is context dependent.

I use hats, e.g., RE , in the upper right handcorner of a slide to name the current context.


1960s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

My 1960s Start in Computing HS

In the beginning, I

g built a relay computer, an adder, in 1962,age 14, for a junior high school sciencefair,

g learned to program in FORTRAN in thesummer of 1965, age 17, at an NSF SSTP atIIT in Chicago (Ed Reingold was my dormcounselor!),


My Start, Con’dHS/Un

g wrote my first real-life application,Operation Shadchan, a party 1-1 matchingprogram based on the questionnaire ofOperation Match, a 1-n dating program, inthe Spring of 1966, age 17, for mysynagogue’s youth group’s annual party,

g studied pure math from 1966–1969, at RPI,an engineering school, to get a B.S., not aB.E. as most of my class mates,


My Start, Con’d Un

g programmed statistical and curve-fittingSW for the Chemistry Dept. at RPI, to makespending money (I wrote FORTRAN fromformulae they gave me.),

g joined ACM in 1967 (member # 10*****), and

g programmed payroll applications in RPGfor a service bureau in Troy, NY (home ofRPI) in the Summer of 1969, to makemoney to go to grad school.


SOTP BIAFIUIW Un

Through all this, I did seat-of-the-pants build-it-and-fix-it-until-it-works (SOTP BIAFIUIW) SWdevelopment, …

simultaneous RE, design, and coding, …

not really understanding the distinctionbetween RE, design, and coding, …


SOTP BIAFIUIW, Cont’d Un

thinking that all of it were just parts ofprogramming, …

probably like a whole lot of programmers,even professionals, did.


Grad School Gr

Later, I

g started grad school at Brown in 1969 as apure Math PhD student (Never mind an MS;that’s for people who want to work for aliving.),

g took Measure Theory from Herbert Federer,who literally wrote the textbook, anddiscovered that I had promoted myself tomy level of incompetence (the PeterPrinciple) in math,


Grad School, Cont’d Gr

g did a lateral transformation to takecomputer science courses in the AppliedMath department down the street,

g fell in love with PLs when I took PeterWegner’s course, PLs, InformationStructures, and Machine Organization(PLISMO), from the book he wrote from hisPhD thesis, and


Grad School, Cont’d Gr

g ended up getting my PhD in 1973 fromPeter on

the design of and the formal specificationof Oregano, an improvement over Algol 68and over Basel; …

it was designed to be more orthogonal thaneither by keeping the architecture of itsimplementation firmly in mind; …

that architecture became the basis for itsoperational VDL formal specification.


CS Journals in Early 1970s

At that time, there were only 3 journals in CS,CACM, monthly,JACM, quarterly, andCR, quarterly.

So, I read at least the abstract of every paperpublished in CS journals for a few years.


CS People in Early 1970s

Also, the number of people in CS in the early1970s was small enough that any personcould know just about everybody in his or herfield and many in other fields.

And most of the pioneers were still alive.

So, I met just about everybody, …

including the authors of IEEE TSE January1977, at conferences or even in LA while atUCLA.


1970s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

Assistant Professor at UCLA PL

I started as an assistant professor in 1972 atUCLA, where the ARPAnet that later becamethe Internet, was happening.

I started off in the field of PLs.

SIGPLAN was the biggest SIG of the ACM atthe time.

We all knew how difficult it was to writecorrect SW that does what its client wants.


PL Research in Early 1970s PL

The overarching concern of PL research in theearly 1970s was:

g to design a PL in which people would writecorrect and good SW, and

g to try to design a PL in which it wasdifficult, even impossible, to write bad SW


Mission Impossible PL

But of course, that is impossible

We realized that you could easily write reallyatrocious SW in even the most structured PL…

At one meeting, someone (I forgot whom)came up to the blackboard & showed us thefollowing goto-free structured program:


Atrocious SW PL

for i from 1 to 4 docase i in

1: S1,2: S2,3: S3out S4

esacod

which, of course, is equivalent to

S1; S2; S3; S4


My PL Research PL

My own PL research was in

g making PLs more orthogonal,

g adding features to PLs in an orthogonalway

g operational formal semantics of PLs andtheir features.


My PL Research, Cont’d PL

I ended up being involved with the Algol 68committee from 1972 through the early 80s.


My PL Research, Cont’d PL

I supervised research

g on new PL features integrated into existingorthogonal PLs, e.g., Algol 68, in thecleanest, orthogonal way, with few or noleaky abstractions,

g finding optimal implementations for thesefeatures, e.g., for garbage collection, and

g formal semantics of the features or of PLs,e.g. of Algol 68.


Early Signs of RE Thinking RE

Note my own RE orientation of trying to fit anew feature into the existing language in thecleanest way, exploring it thoroughly beforebeginning to implement it.


SARA PL

All this time at UCLA, I was a member of JerryEstrin’s SARA group.

SARA was a multi-notation system designlanguage, a competitor of SA and PSL/PSA,and …

a precursor of UML.


SARA, Cont’d PL

SARA was implemented with textual input butline-printer graphic display of models so thatit could be used over ARPAnet.

SARA provided analysis tools to verify well-formedness and mutual consistency ofmodels, to run simulations, etc., like PSA forPSL.


SARA, Cont’d PL

Several of my PhD students built pieces of,analyzed parts of, or applied SARA for theirtheses.

It was in connection to this research that I metsome of the authors of the papers of thepapers in the January 1977 issue of TSE, …

e.g., Doug Ross, John Brackett, DanTeichroew, and Mack Alford.


SARA, Cont’d RE

The irony of all this SARA work is that …

while other things I did feel to me as havingused what became RE thinking or havingfacilitated my realization of the importance ofRE and its activities, …

this SARA work did nothing of the sort.


SARA, Cont’d RE

In fact, I will admit to being totally surprisedthat the organizers of this 40th anniversaryworkship thought that the collection of papersin the January 1977 TSE marked the birth ofRE.

To me, the work they did is more technical andnotational, than attacking the fundamentals ofRE, but that’s my viewpoint.


SARA, an Aside RE

You see, …

All of this work assumed that therequirements were GIVEN to you by the clienton a silver platter, and the hard part was thespecification and the analysis. It was onlyyears later that we began to realize thatgetting the requirements to start with was theHARD part.


January 1977 TSE RE

Two of the articles have “RE” in their titles:g “An Extendable Approach to Computer-

Aided Software Requirements Engineering”g “A Requirements Engineering Methodology

for Real-Timc Processing Reqcuirements”


January 1977 TSE, cont’d RE

But the articles consider RE to be the processof arriving at consistent, completerequirements specifications from therequirements the client gives to the engineers.

None of the articles deals with the HARD partof RE.


RE: Only Three Journals in CS

Look at the advertisement that appeared in theJanuary 1977 TSE …

about all three IEEE computer journals!


YOURCOMPUTER

ENGINEERINGLIBRARYDOESN'T

SUBSCRIBE TO

Published by the Computer Society of the Institute of Electrical and Electronics Engineers

IEEE COMPUTER SOCIETY + INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS

IEEE COMPUTER SOCIETY PUBLICATIONS OFFICE: 5855 NAPLES PLAZA, LONG BEACH, CALIFORNIA 90803

E-mail

In 1977, I started using e-mail as areplacement for the difficult-to-use telephoneto connect with most of my acquaintances,who were CSers!

In 1980s, I started a campaign to convince mynon-CS friends and my family to do the same.



JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

Mid ’70s Foment in PL Area SE

In the mean time, in the PL field, we realizedthat the key to getting better SW was not toimprove PLs, but to improve the process ofSW development.


1968 NATO Meeting SE

The 1968 NATO meeting had alreadysuggested in response to the SW crisis (badand badder and badderer SW is beingproduced as the need for SW is growing) thatmaybe

g we should be systematic and sciencebased and

g we should be engineering our SW,

just like bridge builders engineer their bridgesbased on the laws of physics.


1968 NATO Meeting Report SE

“SE” was used only in the report title and inother meta-text, …

not in any participant’s article.

The field did not exist yet.


Birth of SE field SE

Thus, was born the field of SE, initiallypopulated with PL people who realized

g that the PL used in programming has littleor no effect on the quality of the SWprogrammed with it, and

g that programmers’ behavior had a farbigger impact on the quality of SW theyproduced than the PLs they used.


Switching to SE SE

So I, like a whole bunch of other PL people,ended up switching in the mid to late 1970s toSE.

We tried during the 1970s and 1980s (whenICSE met only every 18 months) to findmethods, possibly assisted by math, todevelop correct SW meeting its client’s needs.


Morphing of Fields SE

For these switchers, …

g the study of PLs morphed to the study ofSW development methods, and …

g formal semantics for PLs morphed to FMsof SW development.


My Sojourn into Security Sec

In the early 1980s, as a result of superivisingseveral people doing formal methods, and inparticular Richard Kemmerer who did (1) aformal specification of the kernel of the UCLAsecure UNIX and (2) a formal verification ofthat the kernel met the specification ofsecurity, …

I got involved in the security community.


1980s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

Security, Cont’d FM

I consulted for the Formal DevelopmentMethod (FDM) group of SDC that was workingon secure operating systems, in particularBlacker.

I ended up publishing a paper in IEEE TSEshowing how the theorems that the group’sverifer proved about an Ina Jo formalspecification of a system were sufficient toprove that the system, if implemented asspecified, would meet the specified criteria.


Security, Cont’d RE

From all this work and from its communitythat included such people as Peter Neumann, Ilearned a lesson that goes right to theessence of RE:

There is no way to add security to any CBSafter it is built; the desired security must berequired from the beginning so that securityconsiderations permeate the entiredevelopment lifecycle.


My Sojourn into EP EP

While I was doing this SE and FM stuff, I madea parallel diversion in the mid 1980s throughmid 1990s into Electronic Publishing (EP).

I got to design and build SW for multi-lingualand multi-directional word processing.

I tried to find the most orthogonal way tointegrate the new features, using the leastleaky user abstractions.


EP SW EP

It was all based on troff (piped architecturewith a separate program for the feature bundlefor one class of document artifact, e.g., table,formula, line drawing, etc.).

This way, I could add a new feature or artifactby building a relatively independent programfor the feature or artifact and stick it into thepipe in the right place.


RE Orientation Even in EP RE

Note the RE orientation here

g in the concern for orthogonality and

g in finding the least leaky user abstractions.

These make the new features easier to usebecause they suffer no surprising exceptions.


I Left the Field EP

I left the EP field when

g EP’s leaders decreed that all future papersin the area had to be written in LATEX, evenpapers about additions to troff.

(There was no way I could keep the rule ofusing the SW a paper is about, to producethe camera ready copy of the paper in thevenue’s traditional format.)


Left the Field, Cont’d EP

g The Unicode consortium ignored mycommand-heavy, but simple commandsand leak-free abstractions for bidi wordprocessing to …

develop their standard, which usesdefaults to avoid commands in the normalcase, but has invisible commands for theexceptional cases, the commandsrequiring an incredibly complex algorithmthat is still being corrected, and formingvery leaky abstractions.


Quit Unicode Effort over RE EP

I quit the Unicode bidirectional working groupover a requirement issue.

g A majority wanted only one period in thewhole character set, with contextualdetermination of an instance’s writingdirection and override for exceptions.

g I and a few others wanted one period perwriting direction, with explicit specificationof an instance’s writing direction.

Choice has MAJOR impact on users’ actions.



JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

Beginning My Move to RE RE

During this time, in 1981, I published a paperwith Orna Berry about how I managed to dothe best job ever in specifying software thatshe had to write, in a domain that I knewnothing about.

I agreed to do this job only because I wasmarried to her at the time!


Beginning My Move, Cont’d RE

In retrospect, I consider this to be my first REpaper.

It’s certainly one of the very earliest on theelicitation aspect of RE.


Ignorance Hiding SE

She had to write some programs that playedstatistical games with experimental data.

I got my lowest Math grade in the undergradProbability and Statistics class, a B, (it ruinedmy perfect Math GPA.) because I had nointuition for probability.

So, I was ignorant in the statistics domain.


Ignorance Hiding, Cont’d SE

To be able to hide my ignorance so I couldwork effectively with the requirements as sheexpressed them to me, …

I made the experimental data an ADT, witheach magic function that I did not understand,e.g., standard deviation or standard error,being a method of the ADT. I knew that theclient understood what they mean and how toimplement them. So I worked with this ADTwith its methods taken as primitive.


Ignorance Hiding, Cont’d SE

I thought and claimed in this paper that thisignorance hiding technique was the basis ofthe success …

as well as my ability to nudge the client to giveinformation

and to do strong-type checking on naturallanguage sentences.(Using the same verb with different numbersand kinds of direct objects in differentsentences is a type error.)


Importance of Ignorance RE

By 1994, I figured out that the reason for thesuccess was not the ignorance hiding, but thevery ignorance!


Importance of …, Cont’d RE

So in 1994, I published “The Importance ofIgnorance in RE” claiming that every RE teamfor a CBS requires along with domain (of theCBS) experts at least one smart ignoramus ofthe domain, who will

g provide out-of-the-box thinking that leadsto creative ideas, and

g ask questions that expose tacitassumptions.


Empirical Validation RE

In 2013–2015, my PhD student, Ali Niknafs,conducted controlled experiments toempirically validate that

for the task of brainstorming for requirementideas, …

among 3-person teams consisting of onlycomputer scientists or software engineers, …


Empirical Validation, Cont’d RE

the teams with one or two members ignorantin the domain …

generated more and better requirement ideas…

than teams consisting of …

only ignorants of the domain or …

only awares of the domain.



JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

The Birth of the RE Field RE

After a while, in the mid 1980s, a subset of theSE people began to notice that SE methodsand FMs do not really solve the problem ofensuring the production of quality SW.

g They don’t scale well, particularly FMs: Forsome funny reason, FM people did not useFMs when building tools to help do FMs.


Birth of RE Field, Cont’d RE

g A method works well only the first time onany CBS. After that, when the CBS must beupdated, e.g., for requirements changes,the artifacts produced by the method mustbe updated to be consistent with thechanges.


Birth of RE Field, Cont’d RE

f This updating is difficult because itis akin to lying perfectlyconsistently, which is very hard todo.

f The lie is making all artifacts appearas if they were produced during anapplication of the method to producethe current version from scratch!

g Change is relentless, and therefore,lying is perennial!


Change is Relentless RE

Why is change in a CBS relentless? Becauseof changes in the CBS’s requirements:

g We did not understand the CBS’srequirements to begin with.

g We made mistakes in expressing what weunderstood.

g We deployed the CBS into the real world,giving rise to the Lehman feedback loopthat changes the CBS’s own requirements!


A Realization RE

Then, a subset of the SE field came to therealization that the real problem plaguing CBSdevelopment was that we did not understandthe requirements of the CBS we are building.


A Realization, Cont’d RE

Brooks, in 1975, had said it well:

“The hardest single part of building a softwaresystem is deciding precisely what to build….No other part of the work so cripples theresulting system if it is done wrong. No otherpart is more difficult to rectify later.”


Even a FMs Person Got it RE

Even an initial-algebras, formal-methodsperson, Joe Goguen, came to this realization.

He ended up being a keynoter at the first REconference in 1993.


1990s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

A Realization, Cont’d RE

This subset of the SE folk formed the RE field,

1. by piggybacking on the nearly annualInternational Workshop on SoftwareSpecification and Design (IWSSD) in themid to late 1980s and early 1990s,

2. from 1993, in two alternating conferences,ISRE and ICRE, that later merged into one(RE),

3. from 1994, in an annual workingconference, REFSQ,

4. from 1996, in a flagship journal, REJ.


IWSSD’s Modus Operandi RE

g Each workshop had its designated CBS,e.g., meeting scheuler, library, elevator.

g An exemplar natural-language specificationof it was distributed prior to the workshop.

g Each paper’s authors should apply itsmethod or tool to the exemplar.


IWSSD Not an RE Conference RE

The workshop, as a whole, was still assumingthat the requirements were given.

But we made a regular special session at theworksop dealing with elicitation!


2000s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

HCI from Graphics

I believe that the same forces that created REout of SE, …

a realization that the hard part of developmentproblem at hand was figuring out what todevelop, …

created HCI out of Computer Graphics.


RE Now RE

Even within RE, there has been a lot ofconcern for …

technology: notation, methods, tools, FMs,etc. …

as well as for …

the human side: elicitation, creativity,emotions, politics, psychology, sociology, etc.


Both Are Important RE

Both technology and the human side areimportant.

Both need to be studied thoroughly andshould be the subject of research.


A Continuous Struggle RE

However, I find a continuous struggle withinRE that mirrors the decades-long struggle thatcreated RE from PLs via SE.

The struggle is that:

g As CSers, we love technology. We like tothink that technology can solve allproblems.

g But, we discover that it doesn’t, sometimesto our surprise.


Struggle Within PL Field PL

For example, we thought that designing theperfect PL would improve SW development.

They did, …

but not anywhere nearly enough.


PL Field Struggle, Cont’d PL

The problem was that the process of makingthe SW has a bigger impact than the PL on theeventual quality of the SW.

So we invented SE to focus on the actualprocess of developing SW.


Struggle Within SE Field SE

We thought that methods and tools applied tothe actual programming would improve SWdevelopment.

They did, …

but not anywhere nearly enough.


SE Field Struggle, Cont’d SE

The problem was that following the bestmethods was useless if we did not know whatto build, and …

the available methods had no effect on gettingthat knowledge.

So we invented RE to focus on the process ofdeciding what to build.


Struggle Within RE Field RE

It’s always a tension between technology,methods, tools, etc.

and a human thing, e.g. how do we humansdevelop, how do we humans find out how tobuild.

As in SE, both are essential, …

but within the RE field, this tension continues.


Even From the Beginning RE

I remember in the early 90s, when we werepiggy backing on the IWSSD conference in theRequirements Elicitation Track, when many apaper offered a new method, occasionally witha tool, for analyzing requirementsspecifications.

We were trying to accumulate a collection ofexemplar specifications that could be used totest any such method or any such tool in astandard, comparable way.


Exemplar Specs RE

I was going along with this, when all of asudden it hit me:

These examplars are too late!

They represent the polished output of theprocess that we were concerned about,namely requirements elicitation.


Exemplar Specs, Cont’d RE

Each exemplar needs to be a collection of thehorrendously incomplete, inconsistent, sloppyinitial documents that are produced bymembers of a customer’s organization whenthey first decide to build any system.

These are unpolished RFPs, visiondocuments, etc.


Exemplar Specs, Cont’d RE

Not everyone agreed with me, and someagreed only partially.

But about 5 years later, I saw this:


P1: STR/JHR P2: STR/ASH P3: STR/ASH QC:

Automated Software Engineering KL466-01-Feather July 21, 1997 10:49

Automated Software Engineering 4, 419–438 (1997)c© 1997 Kluwer Academic Publishers. Manufactured in The Netherlands.

Requirements and Specification Exemplars

MARTIN S. FEATHER [email protected] Propulsion Laboratory, Pasadena

STEPHEN FICKAS [email protected] Science Department, University of Oregon

ANTHONY FINKELSTEIN [email protected] of Computer Science, City University

AXEL VAN LAMSWEERDE [email protected] d’Ingenierie Informatique, Universite Catholique de Louvain

Abstract. Specification exemplars are familiar to most software engineering researchers. For instance, manywill have encountered the well known library and lift problem statements, and will have seen one or more publishedspecifications. Exemplars may serve several purposes: to drive and communicate individual research advances; toestablish research agendas and to compare and contrast alternative approaches; and, ultimately, to lead to advancesin software development practices.

Because of their prevalence in the literature, exemplars are worth critical study. In this paper we consider thepurposes that exemplars may serve, and explore the incompatibilities inherent in trying to serve several of them atonce. Researchers should therefore be clear about what successfully handling an exemplar demonstrates. We go onto examine the use of exemplars not only for writing specifications (an end product of requirements engineering), butalso for the requirements engineering process itself. In particular, requirements for good requirements exemplarsare suggested and ways of obtaining such exemplars are discussed.

system (Olle, 1982); the package router (London and Feather, 1986); the heating system(Marca and Harandi, 1987); the Swiss tournament system (van Diepen and Partsch, 1991);etc. A representative sample can be found in (Icarus, 1989).

Such exemplars generally amount to a self-contained, informal description of a problem insome application domain; they are proposed as unique input for the specification process.Exemplars thus define, in the broadest sense, model specification tasks. They are to beconsidered immutable; the specifier must do the best she can to produce a specification

....

Acknowledgment

Thanks to Dan Berry, Robert Darimont, Philippe Massonet, Bashar Nuseibeh, David Till

and the anonymous reviewers for their help, guidance and suggestions.

Struggle Over Technology RE

You find RE researchers developingtechniques, methods, and tools, i.e.,technology.

Often this technology is being developed toassist in doing a task that people do not like todo, e.g., tracing.


Struggle Over …, Cont’d RE

The main reason a person doesn’t like to dosuch a task, is that the beneficiary of the taskis someone else down stream, and …

the person who has the knowledge to do thetask gains nothing from doing the task [Arkley& Riddle] other than a pain in the tukhis, …

mainly because he or she already has theknowledge.

I.e., there is no incentive to do the task, even ifthere is assistive technology.



But, if people have no incentive to apply thetechnology,

g in the interest of being more agile,

g because the technology is toocumbersome, or

g they don’t even see the value of the doingwhat the technology helps them do,

then, the technology is not going to beapplied, no matter how good it is.



Unfortunately, many technology developersare failing to consider this human aspect.

(Note that this is all independent of NIH (notinvented here).)


Another Struggle RE

RE in practice involves a lot of talking withpeople and asking them questions.

Yet, you find an attitude that just talking withpeople and asking them questions isn’t sexyenough to be the subject of good RE research.


RE is Very Inclusive RE

To me, RE includes anything, I repeat,anything, that can be shown to …

improve the process by which we determinethe requirements of a CBS and …

that leads, downstream, to a better CBS …

as a result of what is done to determine theCBS’s requirements.


Very Inclusive, Cont’d RE

I don’t care what the anything is —

technology, psychology, sociology,management, role playing, fun and games,and even feeding your client milk and cookiesbefore eliciting requirements

— so long as it has an empiricallydemonstrable positive effect on therequirements gathering and on the eventualCBS!


2010s


JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

RE Everywhere RE

I see RE problems and lessons …

walking down the street, …

everywhere!

(The ambiguity of who is walking is intended.)

E.g., in house building, house remodeling, NYbagels, a synagogue’s kitchen, the atrium ofUW’s Davis Centre, Waterloo Region’s lightrail, U.S. income tax instruction booklet,and even Biblical passages.


In Today’s World! RE

In today’s world, everything, especially SWdevelopment, is multi-disciplinary.

At Google, requirements elicitation teamshave people from multiple disciplines, expertsin the CBS’s domain, engineers, lawyers,psychologists, sociologists, HCI experts, UXexperts, and even SW engineers, …

to gather what is needed for the CBS, andto gather new, out of the box ideas.


RE Struggle 1 RE

It really rankles me when I see people youngerthan I being more conservative and protectiveabout the boundaries of their field.


RE Struggle 1, Cont’d RE

I am talking about reviewers for conferencesand journals who reject papers because

g “It’s too much psychology | sociology |management | games.”

g “It’s really an HCI paper.” (and the HCIreviewer says “It’s really an RE paper.”)



g “It’s too much of a story.” (about a casestudy of the successful application of sometechnology)

g “But the method did not work.” (about acase study showing that a believedtechnology didn’t work in at least onesituation)


RE Struggle 2 RE

Why do we insist that a tools for doing an REtask on natural language documents havehigh precision when the task is one in whichhumans are not good?



If we are not good at the task and need a tool’shelp to do it, then it seems clear enough thatrecall is the key criterion for the tool’ssuccess, not precision, …

particularly when it takes a human an order ofmagnitude longer time to find a good answerthan it does to reject a tool-offered nonsenseanswer.



It seems to me that in borrowing InformationRetrieval’s methods to build these natural-language processing tools, we have adoptedtheir measures without considering therequirements for our tools.

We are failing to do RE for our own RE tools!


RE Struggle 3 RE

I see that many a talk or paper on a cognitiveRE process ends with a promise to build a toolto assist human requirements analysts incarrying out the process.

Yet these tools never get built.

I am not complaining about the fact that theydon’t get built. I don’t think anyone reallyexpected any such tool would be built.



I am complaining about our need to promise tobuild such a tool.

It’s as if the promise is admitting that we donot feel comfortable doing this research aboutsoft cognitive stuff. So, to justify doing thisresearch, we say that we will build a tool.

You see, all this research is not just about softstuff; it’s going to build a good respectabletool!



The reality however is that this cognitive stuffis fundamental to understanding RE and todoing it well; …

So doing research on it is the right thing to do!


RE Struggle 4 Sec

I see a lot of work on RE for security.

Only rarely does this work ever cite the workdone in the early 1980s.

Recall how I learned a fundamental lesson ofRE from this work.


Security, Cont’d Sec

From all this work and from its communitythat included such people as Peter Neumann, Ilearned a lesson that goes right to theessence of RE:

There is no way to add security to any CBSafter it is built; the desired security must berequired from the beginning so that securityconsiderations permeate the entiredevelopment lifecycle.


RE Struggle 4, Cont’d Sec

We in RE need to be looking at that old workfor

g what it has already solved and

g insights that are relevant today.


RE Struggle 4, Cont’d Sec

Probably the best place to start is with theOakland Symposium on Security.

Its current instantiaion has a Web site,https://www.ieee-security.org/TC/SP2017/

Its past proceedings can be found athttp://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000646Security and Privacy, IEEE Symposium onClick on “More History”


I Just Don’t Understand RE

How is self-adaptive SW different from …

ordinary well-designed, robust SW, which isable to field any input and has responses foreach already programmed into the SW, …

especially since adaptations in self-adaptiveSW have to already be programmed in forthem to be invoked automatically by the SW?


Don’t Understand, Cont’d RE

The RE problem for both is the same:

g anticipating all situations that need(adaptation = unusual responses), and

g anticipating the correct (adaptation =responses) for them.



JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

Conclusion RE

I have been in computing in one way oranother since 1963 and have beenprogramming since 1965.

While I have been in a whole gamut of CSfields and have picked up understandings ofother CS fields, …

often, by supervising a graduate student whopicked his or her own topic and taught it tome.


Conclusion, Cont’d RE

I see now that I have always been headingtowards my current field, RE, …

because, in retrospect, no matter what X I wasbuilding, the hardest problem that demandedmost of my attention was “What is reallyrequired of X?”, i.e., “What are X’srequirements?”


Lessons I Have Learned: RE

g importance of talking with customers andusers,

g importance of domain ignorance in RE,g security, robustness, user interfaces, etc.

have to be required into a CBS from thebeginning,

g importance of knowing what the CBS is todo, as much and as early as possible,

g RE is everywhere, andg every RE rule has exceptions.


Take Away RE

My main take away message is very simple:

The RE field includes whatever helps do RE inreal life.

And I intentionally left off “for CBSs” in theprevious sentence.



JHS Adder



PhD PLs PLs


SE SETech. EP begat

RE RE

UW Struggles


dberry

Oval

dberry

Oval

dberry

Line

dberry

Line

dberry

Line

dberry

Line

SE History Appendix SE

Continuing the SE Thread from the point of theRE split in early 1990s:


Losing Faith SE

In the early 1990s, I began to lose faith in thetraditional here’s-a-new-method-or-tool-that-will-solve-all-your-SW-development-problemsSE research, …

and even in the here’s-a-new-method-or-tool-that-will-solve-some-of-your-SW-develop-ment-problems reseach, …

i.e., neat, cool solutions to solve not-very-realor non-existent problems that we thought hadto exist.


Losing Faith, Cont’d SE

Each new method or tool was YAM or YAT thatwas really no better than any of the precedingMs or Ts for the same purpose.

Of course, each such M or T worked betterthan the others for the developers of the M orT!

I was just as guilty as everyone else!


Paying Attention to … SE

I began to pay more attention to thepractitioners that were attending and weresaying, in essence, that we researchers werenot listening to what practitioners were sayingwere the real problems that they faced.

People like Barry Boehm, Fred Brooks, BillCurtis, Tom DeMarco, Tim Lister, etc.


Paying Attention, Cont’d SE

And they were saying that the hard problemswere

g people problems andg just understanding the problems that a

system to be developed was being askedto solve.

Technology, i.e., methods and tools, did notreally address these problems.


To Empirical SE SE

In 1998, Walter Tichy, of RCS fame, asked inIEEE Computer “Should computer scientistsexperiment more?” and answered in thepositive, especially in SE:


To Empirical SE, Cont’d SE

Computer scientists and practitioners defendtheir lack of experimentation with a wide rangeof arguments. Some arguments suggest thatexperimentation is inappropriate, too difficult,useless, and even harmful. This articlediscusses several such arguments to illustratethe importance of experimentation forcomputer science.

He argued that we need to put the science intoCS.


Importance of Empirical SE SE

Experimentation is needed to test the validityof long-held, folklore-, belief-, and eventheory-supported claims of method and tooleffectiveness, e.g.,

“… the famous Knight-and-Levesonexperiment … [about] the failure probabilitiesof multi-version programs. Conventionaltheory [Avizienis et al.] predicted that thefailure probability of a multi-version programwas the product of the failure probabilities ofthe individual versions.


Importance of …, Cont’d SE

“[The experiment showed that] the failureprobabilities of real multi-version programswere significantly higher [than predicted]. …the experiment falsified the basic assumptionof conventional theory, namely that faults inprogram versions are statisticallyindependent.”


Very Satisfying Result SE

Personally, this was a very satisfying result,because I had been the outvoted dissentingexaminer of a PhD thesis of one of Avizienis’sstudents that reported as “promising” theresults of an experiment run in my SE class inwhich most three-version programs weremore incorrect on test data than the leastcorrect individual program.


Conundrum SE

The basic conundrum of experimental SE:

An experiment about an SE method or toolthat is small enough to be well controlled andrepeated enough to be statistically valid(internally valid) …

is too small to be realistic and generalizable toreal-life SW development (externally valid).


Conundrum, Cont’d SE

How the hell do you conduct a realistic, butcontrolled experiment to compare theeffectiveness of waterfall and agiledevelopment approaches?



How realistic is a two-hour fully controlledexperiment comparing 20 3-person teamsdoing waterfall to 20 3-person teams doingagile on the same programming problem withthe same programming language with randomassignment of people to teams, etc.

(to make sure that the only difference betweenthe teams are the approaches used)?



How generalizable is a comparison betweentwo multi-year industrial developments ofsystems, one using waterfall and one usingagile methods?

(The languages, the systems developed, thepeople, the team sizes, etc. may all bedifferent in the two developments.)


Experiments on Inspections SE

In the mid to late 1990s, there were lots ofexperiments to assess the costs and benefits,e.g., compared to testing, of many variationsof Mike Fagan’s code inspections.

These experiments, conducted by Vic Basili,John Knight, Dewayne Perry, Aadm Porter,Harvey Siy, Larry Votta, etc. systematicallytested all sorts of variations, e.g., team sizes,durations of steps, checklists or not, real orvirtual meeting, synchronous or not, etc.


Doubly Valid Experiments SE

These experiments were internally andexternally valid because of the lucky accidentthat …

a real-life inspection meeting lasts about twohours, …

and is thus experiment sized!

Essentially, no other real-life SE process isexperiment sized.


Acknowledgements

Thanks to Jo Atlee, Nancy Day, ShahramEsmaeilsabzali, Mike Godfrey, DanaMohaplova, Derek Rayside, and VickySakhnini for their comments on dry runs ofthis talk.


Appendix


My PhD StudentsP Meyers, Barbara F. “Design of a Language

for an Associative Processor” 1975 (UCLA)P Schwartz, Richard L. “An Axiomatic

Definition of ALGOL 68” 1978 (UCLA)P Erlinger, Michael “Analysis of Retention

Storage Management for Generalized BlockStructure Languages” 1979 (UCLA)

R Kemmerer, Richard A. “Verification of theUCLA Security Kernel: Abstract Model,Mapping, Theorem Generation, and Proof”Co-advised, 1979 (UCLA)


R Leveson, Nancy G. “Applying Abstract DataType Methodology to Data Base Systems”Co-advised, 1980 (UCLA)

P Misherghi, Shakib H. “An Investigation ofThe Architectural Requirements of SIMULA67” 1980 (UCLA)

S Penedo, Maria Heloisa “The Use of a ModuleInterconnection Specification Capability inthe synthesis of Reliable SoftwareSystems” Co-advised, 1980 (UCLA)

P Yemini, Shaula “The Replacement Model forModular, Verifiable Exception Handling”1980 (UCLA)


R Paolini, Paolo “Abstract Data Types andData Bases” 1981 (UCLA)

R Schwabe, Daniel “Formal Techniques for theSpecification and Verification of Protocols”1981 (UCLA)

P Mazaher, Shahrzade “An Approach toCompiler Correctness” 1981 (UCLA)

R Burstin, Meir D. “Requirements Analysis ofLarge Software Systems” Co-advised, 1985(Tel Aviv Univ., IL)


S Worley, Duane R. “A Methodology,Specification Language, and AutomatedSupport Environment for Computer AidedDesign Systems” 1986 (UCLA)

S Krell, Eduardo A. “A SARA-based AdaProgramming Support Environment” 1986(UCLA)

S Lor, Edward Kar-Wing “A Requirement-Driven System Design Environment” 1988(UCLA)

R Maarek, Yoelle “Using StructuralInformation for Managing Very LargeSoftware Systems” 1989 (Technion)


Schwarz, Avner “Representing and Solvingthe Automated Building Design (ABD)Problem” Co-advised, 1992 (Technion)

R Goldin, Leah “A Method for AidingRequirements Analysts in RequirementsElicitation for Large Software Systems”1994 (Technion)

Jehuda, Jair “A Top-Layer Design Approach toComplex Real-Time Software” 1998(Technion)

R Breitman, Karin “Evolucao de Cenarios(Evolution of Scenarios)” Co-advised, 2000(PUC Rio de Janeiro, BR)


R Kamsties, Erik “Surfacing Ambiguity inNatural Language Requirements” Co-advised, 2001 (Universitat Kaiserslautern,DE)

R Ramos, Isabel “Aplicacoes das Tecnologiasde Informacao que Suportam asDimensoes Estrutural, Social, Polıtica,Simbolica do Trabalho” Co-advised, 2001(Universidade do Minho, Guimaraes, PT)

R Svetinovic, Davor “Increasing the SemanticSimilarity of Object-Oriented DomainModels by Performing Behavioral AnalysisFirst” Co-advised, 2006 (Waterloo)


R Tjong, Sri Fatima “Avoiding Ambiguity inRequirements Specifications” ExternallyCo-advised, 2008 (University of NottinghamMalaysia Campus, MY)

R Niknafs, Ali “The Impact of DomainKnowledge on the Effectiveness ofRequirements Engineering Activities” 2014(Waterloo)

R Ribeiro, Cristina “The Prevalence andImpact of Persistent Ambiguity in SoftwareRequirements Specification Documents”2016 (Waterloo)


External PhD StudentsR Gonzales, Regina “Capturing Requirements

by Formalizing and Integrating StakeholderConceptual Models” 2000, New MexicoState Univ., USA

R Sobczak, Andrzej “Techniques for RankingPriorities of Initial Requirements for PublicSector Management Information SystemsBased on Strategic Changes” 2003,Warsaw School of Economy, PL


R Mauger, Cyril “Methode de Definition desExigences d’un Produit Integrant sesServices Appliquee aux Batiments Publics”2015, Arts et Metiers ParisTech, FR


My MS StudentsP Yueh, Betty “Contour Model Display

Processor” 1975 (UCLA)P Burgess, Henry W. “An Abstract Data Type

Extension to a Typeless Language” 1975(UCLA)

P Rempe, Barbara “Structured Programmingof a Compiler for a Structured Language”1975 (UCLA)

P Walters, Linda K. “A Method for theComparative Evaluation of ComputerArchitecture” 1975 (UCLA)


P Riggins, M. Christian “A Definition of Run-Time Error Handling in ALGOL 68” 1975(UCLA)

P Allen, Steven James “The Implementation ofString Manipulation in Strimula 76” 1976(UCLA)

P Kaplan, Ronald E. “The Strimula 76 System”1976 (UCLA)

S Linden, Nancy M. “A Software DevelopmentProcessor: A Tool for Program Design”1976 (UCLA)


P Schwartz, Richard L. “Parallel Compilation:A Design and its Application to SIMULA 67”1976 (UCLA)

F Eggert, Paul R. “A Constructive Definition ofVienna Objects” 1977 (UCLA)

P Hethely, Attila “Structured Documentationfor On-line Digital Systems” 1977 (UCLA)

Kaufman, Lawrence J. “Another CapabilityBased Computer” 1977 (UCLA)

F Takemura, Joan E. “Proof of Correctness ofImplementations of Pointers” 1977 (UCLA)


P Omi, Bert Y. “Implementation Techniquesfor a High Level MicroprogrammingLanguage” 1977 (UCLA)

P Chan, Francis Yiu-Tung “Type Checking andType Breaching” 1978 (UCLA)

P Campbell, Douglas C. “An Almost SLR(1)Grammar with Semantics for STRIMULA’76” 1978 (UCLA)

S Mujica, Sergio T. “Data Flow Languages andInterpreters” 1978 (UCLA)

P Pugh, Eric “Forth: A Redesign andImplementation” 1978 (UCLA)


F Smallberg, David “Automatic VerificationCondition Generation Using IntermittentAssertions” 1978 (UCLA)

F Beyschlag, Ulf “An Euler Style Definition ofSimula 67” 1981 (UCLA)

W Dempsey, John “The Design, Development,and Maintenance System” 1983 (UCLA)

E Buchman, Cary “DITROFF/FFORTID, AnAdaptation of UNIX’s DITROFF forFormatting Bi-Directional Text” 1983(UCLA)

P Krell, Eduardo A. “An ADA Translator for theSyntax Directed Editor” 1983 (UCLA)


P Eterovic, Yadran “Porting a Unix Version 6Algol 60 Interpreter to Unix 4.1 BSD” 1985(UCLA)

E Fuller, David A. “A Ditroff Device Driver forthe Line-Printer and the Diablo” 1984(UCLA)

P Fung, Antony “The Architecture of A ForthMachine” 1985 (UCLA)

Nomicos, Sylvana Garlepp “An Assessment ofa Software Quality Metrics Model forDistributed Systems and its Application tothe Evaluation of the LOCUS DistributedSystem” 1985 (UCLA)


E Ip, Chok-Ho “CWPR, A Chinese-JapaneseWord Processor for Ditroff” 1985 (UCLA)

E Fornaciari, William P. “An Outline Editor”1986 (UCLA)

F Holtsberg, Steven “Ina Jo Axioms for Ada’sData Types” 1986 (UCLA)

E Takata, Kris “Indx, A Semi-AutomaticIndexing Program” 1987 (UCLA)

R Aguilera, Christine “Finding Abstractions inProblem Descriptions Using findphrases”1987 (UCLA)


E Becker, Zeev “ditroff/ffortid/

ditroff

, An Adaptation

of the UNIX ditroff for Formatting Tri-Directional Text” 1988 (Technion)

E Habusha, Uri “vi.iv, a Bidirectional Versionof the vi Full-Screen Editor” 1989(Technion)


E Allon, Gil “Minix.Xinim, a BidirectionalVersion of Minix, a UNIX variant” 1989(Technion)

E Wolfman, Tony “flo—A Language forTypesetting Flowcharts” 1989 (Technion)

E Yanai, Shimon “An Environment forTranslating METAFONT to PostScript” 1989(Technion)

P Erez, Ruth “A Contour Model DisplayingInterpreter” 1990 (Technion)

E Srouji, Johny “Bi-Directional Formattingwith Arabic and Farsi” 1990 (Technion)


E Shpilberg, Faina “WD-pic, a WYSIWYG,Direct-Manipulation pic” 1997 (Technion)

W Hornreich, Harry “A Case Study of SoftwareReengineering” 1997 (Technion)

E Ravid, Alon “A Method for Extracting andStating Software Requirements that a UserInterface Prototype Contains” 1999(Technion)

E Denger, Christian “High QualityRequirements Specifications for EmbeddedSystems through Authoring Rules andLanguage Patterns” Co-advised, 2002(Universitat Kaiserslautern)


R Ou, Lihua “WD-pic, A New Paradigm forPicture Drawing Programs and itsDevelopment as a Case Study of the Use ofits User’s Manual as its Specification” 2002(Waterloo)

R Fainchtein, Igor “RequirementsSpecification for a Large-Scale Telephony-Based Natural Language SpeechRecognition System” 2002 (Waterloo)


R Kwan, Irwin “On the Maintenance Costs ofFormal Software RequirementsSpecifications Written in the Software CostReduction and in the Real-Time UnifiedModeling Language Notations” Co-advised,2005 (Waterloo)

W Chen, Hsing-Yu “Analysis of SoftwareConfiguration Management” 2007(Waterloo)

W Yu, Colin “Field Based Development: CaseStudies of IBM Product Development” 2007(Waterloo)


McDonald, Keith “Combining Processor-Sharing and First-Come-First-ServedQueueing Disciplines Using Estimated JobSize” Co-advised, 2007 (Waterloo)

W So, Joel “Autonomous Dynamic Workflow:Explicating the Problem Space andDesigning a Viable Solution” Co-advised,2007 (Waterloo)

W Chodos, David “Creating a Web-basedStatistical Tool” Co-advised, 2007(Waterloo)


E Mohsen, Shahab “The Problem of Stretchingin Persian Calligraphy and a New Type 3PostScript Nastaliq Font” 2009 (Waterloo)

R Weber, Janna-Lynn “Privacy and SecurityAttitudes, Beliefs and Behaviours:Informing Future Tool Design” Co-advised,2010 (Waterloo)

R Isaacs, Daniel “Developers LikeRequirements, Project Managers Don’t”2010 (Waterloo)

R Mehrotra, Gaurav “Role of DomainIgnorance in Software Development” 2011(Waterloo)


R Mak, Andrew “Agile Requirements: A CaseStudy of the Agile Practices in the OSGiApplications Tools Development Team”2011 (Waterloo)

R Werner, Colin “An Industrial Case Study of aVery Large Organization” 2011 (Waterloo)

R Ellis, Keith “Quantifying the Impact ofRequirements Definition and ManagementProcess Maturity on Project Outcome inBusiness Application Development” Co-advised, 2011 (Lancaster University, UK)


R Dembla, Shivam “The Effect of SeveralTradeoffs in the Implementation of LargeDisplays on the Performance of the Usersof the Displays” 2015 (Waterloo)

R Lan, Xiao Ye “An Experimental Studytowards Achieving 100% Recall ofSynonyms in Software RequirementsDocuments with Selected Methods” 2015(Waterloo)
