A model of the mechanical design process based on ......Jul 14, 2016 · (Al EDAM) (1988) 2(1), 33-52 A MODEL OF THE MECHANICAL DESIGN PROCESS BASED ON EMPIRICAL DATA DAVID G. ULLMAN1,

(Al EDAM) (1988) 2(1), 33-52

A MODEL OF THE MECHANICAL DESIGN PROCESSBASED ON EMPIRICAL DATA

DAVID G. ULLMAN1, THOMAS G. DIETTERICH2 AND LARRY A. STAUFFER3

'Department of Mechanical Engineering, 2Department of Computer Science, Oregon State University and3Department of Mechanical Engineering, University of Idaho, U.S.A.

This paper describes the task/episode accumulation model (TEA model) of non-routine mechanical design, which wasdeveloped after detailed analysis of the audio and video protocols of five mechanical designers. The model is able to explain thebehavior of designers at a much finer level of detail than previous models. The key features of the model are (a) the design isconstructed by incrementally refining and patching an initial conceptual design, (b) design alternatives are not considered outsidethe boundaries of design episodes (which are short stretches of problem solving aimed at specific goals), (c) the design process iscontrolled locally, primarily at the level of individual episodes. Among the implications of the model are the following: (a) CADtools should be extended to represent the state of the design at more abstract levels, (b) CAD tools should help the designermanage constraints, and (c) CAD tools should be designed to give cognitive support to the designer.

1. Introduction

This paper presents a model of the mechanical designprocess. The purpose of a design model is explanationand prediction. A design model should explain howthe design process unfolds—why it succeeds in somecases and why it fails in others. A model should alsobe able to predict future successes and failures andprovide some estimate of the resources needed todevelop good designs.

The model described in this paper, which we callthe TEA model (task-episode accumulation model), isstill under development, and it does not provide asmuch predictive power as we would like. It does,however, explain many aspects of the design process,and it provides significant insight into the waymechanical designs are developed.

The major reason for developing design models isto improve the design process—to raise the quality ofthe designed products and improve the efficiency ofthe designers. There are three major avenues topursue: development of design aids, improvement ofdesign education, and complete automation of somedesign tasks.

Developers of design aids need to know whatproportion of the design process they are currentlyassisting. They can benefit by identifying new aspectsof design where assistance could be provided.

Design educators are interested in understanding

Received 3 February 1988 and in revised form 2 August 1988.

0890-0604/88/010033 + 20$02.00/033

what kinds of knowledge designers need. Is thereknowledge about design, independent of a specifictechnology, that can be taught? What forms ofexpertise do successful designers have, and how canthis expertise be transmitted to students?

Researchers in artificial intelligence would like tounderstand how designers reason about the evolvingdesign. What constraints are brought to bear to guidethe search for solutions? How are potential solutionsevaluated?

The research reported in this paper makes initialcontributions in each of these areas. The paper isorganized as follows. Section 2 describes the researchmethods employed to gather and analyse data on thedesign process. Section 3 gives an overview of themodel. This is followed by sections describing each ofthe major components: the design state (Section 4),the goal structure (Section 5), the operators (Section6), and the constraints (Section 7). There are someshort, example episode traces presented in Section 8.Sections 9 and 10 conclude with a discussion of theimplications of the model and the problems requiringfurther research.

The results presented in this paper are based on atwo-year study of the mechanical design process basedon the evaluation of experienced mechanical designerssolving real design problems. Early results of thisstudy along with details on the methods used havebeen previously published (Stauffer et al., 1987;Ullman and Dietterich, 1987; Ullman etal., 1987a, b).Other detailed results are reported in recent papers:

© 1988 Academic Press Limited

34 D. G. Ullman et al.

details on the development of mechanical designoperators (Stauffer, 1987; Stauffer et al., 1987), detailson the requirements of a state representation formechanical design (Tikerpuu and Ullman, 1988), anda comparison of high level observations made in thisstudy compared to those from other studies (Staufferand Ullman, 1987).

2. Research methods

Five mechanical design engineers of varying back-ground and experience were given the initialspecifications for one of two fairly simple, yet realistic,industrial design problems. The engineers wererequested to think aloud as they solved the problems.Their verbal reports, sketches, and gestures werevideo- and audio-taped for a period of 6—10 hours.The taped data were then transcribed to obtain a'protocol' of the design session. This protocol wasthen analysed in a variety of ways to provide the datafor this paper.

The two problems used in the study were the'battery contacts' problem and the 'flipper-dipper'problem. The battery contacts problem statement, inabbreviated form, is the following:

Design a plastic envelope (dimensions provided)and the electrical contacts to accept three batteriesto power the time clock in a new computer. Thebatteries (detailed dimensions provided) must beconnected in series and to an adjacent printedcircuit board. The external dimensions of theenvelope are provided as are needed contactpressures. The volume is 50,000 units/month forthree years and the assembly will use a robot.

Two subjects, SI and S2, solved this problem.The flipper-dipper problem statement, in abbre-

viated form, is the following:

Design a mechanism that will accept a 10 x 10 x0.063 in. aluminum plate from a worker, lower oneside so that it just touches the surface of a chemicalbath (to receive a chemical coating), lift the plateoff the bath surface, flip it over, lower and coat theother side, and present it to the worker for removal.There were only to be three of these built.

Three subjects, S4, S5, and S6, solved this problem.More details on these problems can be found inStauffer (1987) and Ullman et al. (1987a,b).

Approximately 46 hours of data were taken. All ofthe data was transcribed and analysed to determinethe general problem flow. Several detailed analysistechniques were tried on selected parts of this data in

an attempt to develop an analysis method thatprovided insight into the goal structure of the design.It was found that an analysis based on tasks, episodesand operators (defined below) was most revealing andwas reasonably repeatable by different researchers.Four general types of tasks were identified: conceptualdesign, layout design, detailed design, and catalogselection. For each protocol, all instances of each ofthese tasks were identified. Then one instance of eachwas selected at random for detailed dissection. Thisyielded 19 sections (one subject never performed acatalog selection task) constituting 204 minutes of data(8% of the total data taken). The TEA model hasbeen constructed from the detailed analysis of these19 sections.

3. Overview of the model

The TEA model is a problem-space model in whichthe fundamental components are the design state andthe design operators. The design state contains allinformation about the evolving design includingproblem specifications, additional constraints intro-duced by the designer, proposed designs, drawings,calculations, assembly plans, and so on. Designoperators are primitive information processes thatmodify the design state by performing calculations andsimulations, creating new proposed designs, evaluat-ing proposed designs, and making decisions to acceptor reject proposed designs. The TEA model containsten operators: select, create, simulate, calculate,compare, accept, reject, suspend, patch and refine.These are discussed in detail in Section 6.

The most important parts of the design state are theproposals and constraints. Proposals are designelements created by operators as alternative ways ofachieving some goal. For example, to fasten twopieces of plastic together, the create operator mightcreate three proposals: adhesives, welds and snaps. Inmechanical design, proposals are normally forms,although they can also be functions, plans andconstraints.

Constraints include specifications, requirements,needs, performance measures, and objectives as theseterms are used by other authors. A design problemnormally begins with some rather abstract functionalconstraints. The designer may introduce otherconstraints based on past experience (e.g. 'don't useadhesives with plastics'). Additional constraints enterthe design state whenever a proposal is accepted. Forexample, once snaps are adopted for fastening theplastic pieces, a new space constraint is derived: spacemust be reserved for the snaps. Hence, each proposal

Model of the mechanical design process 35

can also be viewed as a constraint on the remainder ofthe design—once it is accepted into the design. Thisobservation is the rationale for including constraintpropagation as an important component in AI de-sign systems (Steinberg, 1987) and is discussed inSection 7.

To accomplish a design, the design engineer appliesthe primitive operators in meaningful sequences calledepisodes (Stauffer, 1987). An episode is a sequence ofoperator applications that addresses some primitivegoal. The nature and scope of primitive goals changesas the design unfolds. Initially, a primitive goal mightbe 'come up with a central concept for this design' or'select a source of power for this machine'. Later, aprimitive goal might be 'find some way to mount theflipping frame to the arm'. And still later, a primitivegoal could be 'determine the tolerance on the shaftdiameter'. At any level, an episode requires anaverage of 56 sec (maximum 80 sec, minimum 32 sec).The TEA model has six different types of episodes:assimilate, document, plan, repair, specify and verify.These are discussed in Section 5.

An episode ends in one of three ways: (a) a solutionhas been accepted that satisfies the goal; (b) noacceptable solution has been found (all have beenrejected) thus the goal itself has been rejected (usuallyreplaced by a reformulated goal); or (c) the goal issuspended to be reconsidered at a later time. Anepisode may also be temporarily interrupted by asub-episode. The sub-episode addresses some goalwhose solution is required by the original episode.After the sub-episode is completed, the episode isimmediately resumed.

Within an episode, the decision about whatoperator to apply next is guided by a set of heuristicrules. Many of these rules are presented in this paper,but a complete set of such rules has not beendeveloped. In future research, we plan to extend theset of rules and test them in a computer simulation ofthe model.

When an episode is completed, the designer usuallytackles another, closely related primitive goal. Acollection of related primitive goals is called a task.Generally, a task can be described as a goal of largerscope, for example, 'layout the left battery contact' or'dimension the flipping frame'. Within this larger goal,there will be several primitive goals: 'determine thelength', 'determine the width', 'determine the shaftdiameter', and so on. There are four major types oftasks in the TEA model: conceptual design, layoutdesign, detail design, and catalog selection.

This completes the description of the basiccomponents of the TEA model. The model has fourimportant (and interrelated) features that should be

noted. First, the TEA model addresses non-routinedesign tasks. None of the subjects in our studynormally design the kinds of devices and productsrequired by our two problems. However, the subjectsdid all have some understanding of the requiredtechnologies (e.g. simple electrical-mechanical de-vices, injection-molded plastics, adhesives, etc.).Second, in the TEA model, the design is accumulatedgradually by the incremental contributions of eachoperator, hence the name 'task/episodeACCUMULATION model'. Non-routine design,covered in this study, it not conducted by instantiatinga prototype or filling in some skeletal framework.Third, alternative designs are only consideredWITHIN episodes. By the time an episode iscompleted, one of the alternatives will have been

TABLE 1. Mechanical design model terminology

1. Levels of abstraction (see Section 4.2)A. Level 1, AbstractB. Level 2, IntermediateC. Level 3, Concrete

2. Constraints (see Section 4.3)A. GivenB. IntroducedC. Derived

3. Goal structure (see Section 5)A. Tasks

1. Conceptual Design2. Layout Design3. Detail Design4. Catalog Selection

B. Episodes1. Assimilate2. Plan3. Specify4. Repair5. Verify6. Document

4. Operators (see Section 6)A. Generate

1. Selecta. State Selectb. Memory Select

2. CreateB. Evaluate

1. Calculate2. Simulate3. Compare

C. Decide1. Accept2. Reject3. Suspend4. Patch5. Refine


selected and accepted into the design state. Fourth,the design process is controlled locally, at the episodeand task level. The designer does not formulate andthen execute a global design plan. Instead, a centralconcept is developed and gradually extended toaccomplish the design goals.

The terms used in the overview above are refinedthroughout the rest of this paper. In Table 1 thisterminology is listed along with references given to thesections of the paper where each term is discussed.

Now that the general components of the modelhave been reviewed, the sections that follow describeeach of these in more detail.

4. The design state

As mentioned above, the design state is the sum of allinformation about the design that has been processedby the designer. In particular, the state includes thefunctional and geometric constraints on the design,descriptions of the geometry, configuration, andmaterials for all forms involved in the design,functionality of the design, manufacturing informa-tion, and so on. All of this information must berepresented in some way and stored in some location.

4.1 WHERE THE DESIGN STATE IS STORED

Figure 1 depicts the environment in which thedesign takes place. It is based on the models ofinformation processing psychology developed byNewell and Simon (1972) and Stauffer (1987). The figurecan be viewed as a 'map' of the locations in whichinformation may be stored. It is divided into internallocations (i.e. inside the mind of the designer) and

' DESIGNSTATE

Accepted designproposals

ConstraintsStrategic

\

EXTERNALENVIRONMENT

INTERNALENVIRONMENT

FIGURE 1. The design environment

external locations (i.e. outside the mind of thedesigner). Within the designer, there are two locationscorresponding to the two different kinds of memory:short-term memory (STM) and long-term memory(LTM). There is also a 'processor' that is responsiblefor applying operators and controlling the designprocess. It is the goal of this study to identify theoperators and the heuristics that control theirapplication. External to the designer there are many'design state storage locations' including pieces ofpaper, CAD tools, handbooks and colleagues.

The design state can only be represented in threelocations: short-term memory, long-term memory,and the so-called, external memory (which includesdrawings and notes on pieces of paper or stored inCAD tools). During the design process, otherinformation is brought into the design state from theenvironment through a process called assimilation. Itis by assimilation that such information as the problemspecification, handbook information, or generalknowledge about design is brought into the designstate. Information can be assimilated from variousexternal sources or from knowledge and experiencesstored in long-term memory.

Each 'location' has certain properties that affecthow it can be used in design. Short-term memory isvery fast and powerful. Design operators can onlyoperate on information that is brought into short-termmemory. Unfortunately, STM has limited capacity.Studies have shown that it is limited to approximatelyseven 'chunks' of information (Miller, 1956). (Achunk is a meaningful unit of information the size ofwhich varies with experience. Experts generally havelarger chunks than novices.)

Long-term memory has essentially infinite capacity,but access is slow (from 2 to 10 sec per chunk). Accessto long-term memory is also not direct. Instead,memories must be triggered by some cue or retrievalstrategy based on information in short-term memory.During design, parts of the design state are stored inlong-term memory and these are relatively easy tocue, because, at any time, currently important parts ofthe design state are in short-term memory and can actas pointers for the knowledge in the long-termmemory.

Other information is also kept in long-term memory(e.g., knowledge about adhesives), but it does notenter the design state unless it is specifically generatedand assimilated.

All operations on the design and all informationpassing to or from long-term memory must passthrough short-term memory, so short-term memoryforms a critical bottleneck for human designers.


4.2 LEVELS OF ABSTRACTION IN MECHANICALDESIGN

The exact form in which the design state isrepresented in STM and LTM is the subject ofcontinuing research (Tikerpuu and Ullman, 1988).One critical property of this representation that hasbeen identified is its level of abstraction (Sacerdoti,1974; Adelson, 1985). A design problem typicallybegins with a set of functional constraints expressedvery abstractly. During the design process, the level ofabstraction of the design state is progressively reduceduntil it is detailed enough to be manufactured.

The degree or level of abstraction of any element ofthe design state can be characterized by what itexplicitly describes and by what it omits. There is acontinuum of degree of abstraction ranging from themost abstract to the most concrete. For the purposesof describing the TEA model, we have arbitrarilydivided this continuum into three levels: Abstract,Intermediate and Concrete.

Each of these levels is defined below in operationalterms. Separate definitions have been given for thedifferent kinds of 'media' in which the stateinformation may be represented: verbal/texture,visual, and physical. Where necessary, distinctionshave been made between form representations andfunction representations.

Level 1. AbstractVerbal /Textual

Both form and function represented in termsof qualitative descriptors (e.g. in theflipper-dipper problem statement, the ma-chine must 'lift the plate. . . , flip it over,lower and coat the other side')

VisualForms represented as a rough sketch (back-of-the-envelope). These sketches may con-tain the basic geometry and topology of the

FIGURE 2. First sketch of battery contact

object. Major features and interfaces arecontained in the sketch. Functions arerepresented in block diagrams showing theflow of material, energy, and signals asidentified textually (defined above) or withsymbols such as arrows (e.g. one subjectsolving the battery contacts problem drewthe first representation of a contact as inFigure 2).

PhysicalThere are no physical representations at thislevel of abstraction.

Level 2. IntermediateVerbal/Textual

Both form and function are represented interms of specific variable names or values forthe overall measures of the design. Thisincludes information on the manufacturingprocess and the material families for theforms (e.g. one subject solving the flipper-dipper problem defined an 'arm' that had a'width' of '28.25 in.' early in the design).

VisualForms represented as a sketch of drawingthat is near scale or to scale. These sketchesor drawings may contain all the basicdimensions. All major features are complete(e.g. the 'arm' was represented at the time asin Figure 3).Functions represented in block diagramshave the variables identified textually (de-fined above). Results of analysis representedin graphs or plots.

PhysicalBoth form and function can be representedin crude models such as clay, paper, woodblock and rough functional analogs (e.g. onesubject solving the flipper-dipper problemmade a paper model of a part of hismechanism to confirm its operation).

Level 3. ConcreteVerbal/Textual

Both form and function represented in termsof specific variable names and values for allmeasures of the design and their tolerances.Bill of materials complete with specificmaterials and processes (e.g. many subjectsmade bills of materials and all wrotetolerances).

VisualForms represented as detailed drawings withtolerances. Functions represented in anima-


FIGURE 3. Intermediate drawing of arm

FIGURE 4. Final drawing of battery contact

tions or demonstrations of the actual hard-ware or prototypes of it (e.g. the 'batterycontact' contact was finalized as in Figure 4).

Physical

Both form and function can be representedin final hardware or in static and workingmodels of the hardware. (Note: none of thedesigns were reduced to final hardware.)

These 'levels of abstraction' will be incorporatedinto the heuristic rules in Section 5 of this paper.

4.3 CONSTRAINTS

Constraints play a central role in the TEA model.They can usually be viewed as limits on the form or

function of the device. Constraints are developed inone of three ways: (a) they are GIVEN external to thesolution of the design being considered, (b) they areINTRODUCED by the designer based on his/herdomain knowledge, or (c) they are DERIVED duringthe solution of the design problem.

'Given' constraints appear in the initial statement ofthe problem, or they are imposed on the designproblem by the solution of other components notunder the control of the designer responsible for theproblem under consideration. The given constraintsfor the protocol experiments were summarized abovein Section 2.

'Introduced' constraints are those constraintsimposed on the problem by the designer based onspecific domain knowledge. For example, subject S2(working on the battery contacts problem) early in hisconceptual design imposed a constraint that he wouldnot use adhesives, because they are messy and hard todeat with. This is considered an 'introduced'constraint, because it was not imposed by the problemstatement and it was not derived from decisions madeduring the design process. Introduced constraints canbe universal in nature. For example, our subjectsoften verbalized a preference for 'round' numbersover more precise quantities. Introduced constraintscan also be quite domain specific such as the adhesiveknowledge above.

'Derived' constraints arise as a consequence ofdesign decisions. In the battery contacts problem, forexample, the decision to place the contacts on thesides of the batteries resulted in a derived constraintconcerning the amount of space available for thecontacts. This constraint is derived from the constrainton the total available space and the previous designdecision.

These constraint definitions complete the descrip-


tion of the design state. The next section presents thegoal structure of the design process.

5. The goal structure of mechanical design

According to the TEA model, the goal structure ofthe design process has three main levels. At thehighest level is the main goal comprising thesatisfaction of the given constraints. This top-levelgoal is decomposed into finer goals, each of which issolved by a single task. These task-level goals arefurther decomposed into the primitive goals that areeach addressed by an episode. The remainder of thissection describes the structure of task-level andepisode-level goals and design processes. Section 6describes the individual operators that are used tosatisfy the goals.

The top level goal is to design a machine which issubject to many stated restrictions on its function andits form. These given constraints are usuallydominated by some abstract functional requirementson the operation of the device along with some lessabstract, possibly very concrete spatial and manufac-turing demands. Additionally, these given constraintsmay be represented in different ways: verbal, textual,visual or physical. Lastly these given constraints maynot all be satisfiable; they may conflict with each otheror with what is achievable. As an example, in one ofthe problems solved by three of our subjects, a givenconstraint was that the machine needed to 'flip' analuminum plate. In the TEA model this requirementis classified as a textually represented, abstract, givenconstraint on the function. Another part of thedescription for the same problem was a sketch (withdimensions) of the table top on which the device wasto be mounted. These dimensions (textual) and thedrawing (visual) combined to describe spatial,concrete, given form constraints.

Other constraints on this problem were leftunstated, of course. Every design is constrained by thelaws of physics and by the desire to minimize costsand part count. The TEA model considers these to begiven constraints as well.

Below the top-level goal, there are three generalkinds of task-level goals each addressed by a task:conceptual design, layout design, and detail design.Details on these tasks are given in Sections 5.1-5.4. Inthe protocol study, these tasks required from 8 to 20min to be performed with an average duration of10.8 min. Each of these task-level goals focuses on thefunction of the device, some assembly, some specificpart, or a feature of a part. The task of catalogselection (mentioned previously) usually addresses a

subgoal that arises during a layout or detail designtask.

In the protocol data, contrary to many designmodels, the design process is not broken into separateconceptual, layout, and detail design phases (there areconceptual, layout and detail design tasks throughoutthe design). Efforts were made to reduce the protocoldata into separate phases and develop a goal tree forthe design. This was found not to be possible as the'phases' are a management artifice and are notderiveable from the protocol data. For example, therewere several occasions in which problems identifiedduring layout design of some component triggered theconceptual design of a new component or a newfeature of an existing component. Hence, tasks ofdifferent kinds are interleaved in the protocol analysis.It appears that the best level to develop a goal tree isat the episode level as is demonstrated in Section 8.

Each task is composed of a sequence of episodesaddressing primitive goals. Six types of episodes thatmake up the tasks have been identified from theprotocols. These are Assimilate, Document, Plan,Repair, Specify and Verify. They are defined asfollows

'Assimilate' episodes have the goal of bringinginformation from the external environment or thelong term memory into the design state. Usually theinformation being assimilated is constraint informa-tion, but designers also bring into the design spacespecific design proposals or strategies from theirlong-term memory, colleagues and handbooks.

'Specify' episodes have the goal of clarifying ordeveloping a design proposal into a more specificdescription (lowering its level of abstraction). Thisis the workhorse of the episodes. If the constraintswere all known and the plan fully defined, thedesign process would reduce to a sequence ofspecification episodes followed by the need todocument the design.

'Planning' episodes have the goal of developingstrategies for how to proceed. Planning episodes donot help in solving the goal of designing a machine,but are a part of establishing the goal structurenecessary for the solution. In the protocol data,little planning was observed.

'Document' episodes have the goal of recordinginformation textually or graphically in the externalenvironment with the purpose of communicating toothers. In other words the goal of the documentepisodes is not to specify any more informationabout the design but to record previous results.However, it must be noted that often, even though


the goal might be to document information, newdesign decisions must be made because the designerdiscovers that the information to be documented isincomplete.'Repair' episodes have the goal of alteringpreviously specified information to repair a conflictbetween a previously accepted design proposal anda constraint or a conflict between two or moreconstraints. In other words, repair episodes takeinformation accepted into the design state andreconsider it in the light of conflicting constraints.

'Verify' episodes have the goal of repeating anepisode to ensure that its results are still acceptable.In the protocol data there were several instances ofthe designer going back and reperforming theoperations in an episode to verify decisions beforethe investment in them is too great.

In the 204 min of data analysed, the subjectsperformed 208 episodes. The distribution over all thetasks is shown in Table 2. Note that this paperdescribes specify, document, assimilate and planepisodes. Additional information on verify and repairepisodes is in Stauffer (1987).

In analysing the protocol data, it was much easier toidentify episode boundaries than it was to detect taskboundaries. The reason for this is that the subjects didnot solve the problems in a particularly orderlyfashion. A task focuses a series of episodes on ageneral 'region' of the design, but subjects oftendeparted from this region to gather information ormake decisions regarding other, related, parts of thedevice. One good example of this behavior aroseduring the layout task selected from S6's protocol.This task involved finding some method for mountingthe 'flipping frame' (the frame that held the platewhile it was flipped). This was a real sticking point inthe design. S6 would work on it for a while and then,when an acceptable solution was not found, addressanother task, and return to the mounting problemlater.

As a consequence of this interleaving andinterruption of tasks, the flow of the design process isbest mapped by following the individual episodes. In

the remainder of the paper, the tasks are used only asa rough guide and the episodes are the primary focus.

The remainder of this section describes in detail theepisode structure in each of the task types. Theheuristic rules given are numbered in brackets. Thesenumbers are referred to in the example episode tracesin Section 8.

5.1 THE TASK OF CONCEPTUAL DESIGN

Conceptual design is primarily concerned withassimilating the given constraints and specifying formsto meet the functional given constraints. Thisspecification is accomplished to at least an abstractlevel.

In the reduction of the protocol data, fiveconceptual design tasks (totalling 68 min) wereselected at random—one from each subject's protocol.Within these tasks, the episodes are distributed asshown in Table 2. Clearly, the subjects are focused onassimilating information (36%) and developing (speci-fying) design proposals (39%) in this stage of thedesign.

Some of the rules governing the sequence ofepisodes in conceptual design are as follows:

If the design state is empty (Rl)Then internalize the given constraints via

assimilate episodes.

The first task begins with a series of assimilateepisodes. Each episode has the sub-goal of thedesigner reading or otherwise considering (looking atsamples or other hardware, talking to the experimen-ter in the protocol recordings, talking to the client,etc.) a constraint and understanding it. The designstate is said to be empty at the start of the designprocess and these assimilation episodes form the firstconception of what the design is to become.

If assimilating given constraintsThen perform episodes with goals of

breaking the given constraints intoindividual constraints and consideringeach one at a time.

(R2)

TABLE 2. Episode distribution

ConceptualLayoutDetailedCatalog selection

Total

Assimilate

36183

30

(%) Document

06

5211

(%) Plan (

19128

18

Repair (%) Specify (%) Verify (%)

19 21 13

39492326

34

366

16


When the subjects were first given the problemdescription, they usually read through it twice. Thefirst time was a direct word-for-word reading, but thesecond contained many pauses to evaluate constraints,relate the text to the figures, and assimilate otherinformation not in the problem statement. In general,constraints were assimilated in the order theyappeared in the problem statement. Some subjectscompiled a written list of the major constraints. Thislist frequently broke the given constraints intoindividual constraints, which were then assimilatedone by one. This process of individualizing theconstraints must be based on the designer's domainknowledge. Each constraint is evaluated before it isincorporated into the design state (see Section 6.2 formore on evaluation).

Assimilation does not occur only during this earlystage. Any time the designer brings information fromlong-term memory or external sources into the designstate, assimilation is required (see Figure 1). Even theproblem statement is reconsidered as the designunfolds. The subjects frequently went back and rereadthe statement to assimilate details that had beenskipped during the initial task.

The following rule terminates this first period ofassimilation:

If the given constraints have been (R3)assimilated to the point that (1) thedesigner feels the problem is under-stood or (2) the constraints cannot befurther understood without conceptsto consider

Then develop design proposals.

In the protocols, the subjects each spent between 5and 32 min understanding the constraints. They thenpaused and often announced that they were ready tobegin the design. This occurred in spite of the fact thatin many cases there were constraints that wereobviously not well understood or were wronglyinterpreted. In most cases these constraint problemswere resolved later in the problem, but misinterpreta-tions here in establishing the early state of the designwere often built on throughout the remainder of thedesign.

If a design proposal is to be developed (R4)Then select the most important functional

constraint or group of relatedconstraints.

The primary goal of conceptual design is to specifyforms to meet the given constraints. This isaccomplished by selecting a few of the constraints andgenerating a conceptual design to satisfy them. This

initial design is then elaborated to meet the remainingconstraints. This is what is meant by accumulationdesign in the acronym 'TEA'.

It is not clear when or how the subjects determinewhich constraints to pursue first. One hypothesis isthat each constraint is evaluated and ranked for'importance' as it is assimilated. A reason for notbelieving this hypothesis is that separate 'ranking'episodes were not observed in the protocols. Anotherhypothesis is that subjects prefer to begin withfunctional rather than form constraints, since func-tional constraints relate more directly to the purposefor which the device is being designed. Most of theconstraints initially focused on were functionalconstraints. A third hypothesis is that constraints areconsidered 'important' if they appear to be difficult tosatisfy. The less experienced subject in the batterycontact problem (SI), for example, focused on theconstraint that the batteries must be connected inseries. Subsequent episodes showed that SI was notsure exactly how to achieve a series connection, so forher, this constraint was difficult. Subject S2, on theother hand, focused on a constraint that specified thecontact pressure on the batteries. When S2 made aninitial drawing, he immediately drew a configurationfor connecting the batteries in series. Hence, for him,the series connection constraint was considered easyto satisfy. Clearly, the designer's prior experience isimportant in helping him/her determine the degree ofdifficulty of each constraint.

If a functional constraint(s) has been (R5)selected

Then generate, evaluate, and accept a formdesign proposal for that constraint(s).

It is interesting to note that in domains where formand function can be aligned (e.g., so that a functionaldecomposition corresponds directly to a form de-composition), this rule could be applied repeatedlyto carry out the entire design. While it is rare to find adomain where this occurs, software and circuit designapproach this ideal more closely than mechanicaldesign. Problems arise in all engineering fields whenindividual forms are engineered to do many functionssimultaneously (Sussman and Steele, 1980; Ulrich andSeering, 1988) and this can not be avoided in mostmechanical designs. Even static structures not onlycarry load but must function as a stiffness, transferthermal energy, etc. In the flipper-dipper problem, forexample, the same mechanism that allows the plate tobe dipped may also assist in flipping the plate.

This has an important consequence for the designprocess: each design decision can potentially affectevery subsequent decision. The reason is that all


subsequent design tasks must consider the possibilitythat their goal can be achieved by modifying apreviously specified form (i.e., to save space) ratherthan by introducing some new form. Furthermore,even if the previously specified forms are notmodified, new forms must coexist with them. Theconstraints of space, weight, and so on may make thisimpossible. In a functionally-decomposed design, bycontrast, each design decision on a given form onlyaffects subsequent decisions concerning subcom-ponents of that form.

As an example, consider subject S5. He began theflipper-dipper problem by focusing on the functionalconstraint of dipping of the plate. His first form designproposal that was generated, evaluated, and acceptedwas a ferris-wheel-type machine. This decision carriedwith it the derived, functional constraint that themotion of the plate must be tangent to the water. Thisin turn became a. very important constraint on thedevelopment of the method to hold (another function)and flip (yet another) the plate. This constraintbecame so imbedded in the design that it was not laterrelaxable without paying the price of virtually startingall over again.

If forms have been accepted for the (R6)most important functional constraints

Then the next goal is to develop forms forthe next most important con-straints).

After each decision to update the design state byaccepting a design proposal, the constraints (given,derived and introduced) still to be satisfied arereconsidered to find the most important ones to use asthe focus for the next goal. This is not to say theengineer keeps a long agenda of constraints to satisfybut, when verbalized there were often three or four inperceived order of importance. Of course this orderchanges as new, derived constraints are developed.Additionally if an episode was suspended due to alack of information, the new, most important goalmay be to assimilate or in some other way specify theneeded information to satisfy the suspended goal. Thisbehavior often gave rise to sub-episodes (withinepisodes) to develop needed information. Twenty-oneper cent of all episodes were sub-episodes with thenesting sometimes three levels deep.

5.2 THE TASK OF LAYOUT DESIGN

Once the major forms have been conceptualizedto satisfy the 'important' (primarily functional)constraints, the focus changes to specifying thecomponents (assemblies and individual parts) at

decreasing levels of abstraction. There was generally aclear verbal demarcation at this point. The subjectsreported that they now understood what their solutionwould look like and that only the 'details' remained tobe worked out.

In the reduction of the protocol data, five layouttasks (totalling 63 minutes) were selected atrandom—one from each subject's protocol. Withinthese tasks, the episodes were distributed as shown inTable 2. Notice that about half the effort is onspecifying the design (49%). Assimilation is much lesscommon than it was during conceptual design (downfrom 36 to 18%), and documentation is still playingonly a minor role.

Only one rule controls the selection of episodesduring layout tasks:

If performing a layout design taskThen specify the design proposals to

intermediate or concrete abstraction.

Layout is simply concerned with moving the designstate to refined levels of abstraction. Most of theinteresting problem-solving behavior during layoutdesign arises because of the need to repair problemsthat arise during this process (see the discussion ofevaluation and repair operators in Section 6.2 and 6.3)

5.3 THE TASK OF DETAIL DESIGN

As the design becomes more finalized, thedesigner's attention gradually shifts to focus primarilyon documentation and continued refinement asneeded. In the protocol analysis, detailed design taskswere assumed to begin when the subject startedmaking scale drawings rather than sketches. However,long before this point the subject would usually havebegun specifying particular dimensions (and eventolerances) for some components. Unlike the cleardemarcation between the conceptual and layout tasks,where the subjects virtually announced they had aconcept defined, there was no observable transitionbetween layout and detail design in the protocol data.

Based on the definition of the levels of abstractionand with a desire to be consistent with traditionalterminology, the detail design task has been defined inthe TEA model as focusing on the completerefinement and the complete documentation of thedesign. This is captured by the following rule:

If performing a detail design task (R8)Then reduce all the form design proposals

to concrete forms and documentthem.


As with the conceptual and layout tasks, five detaildesign tasks (totalling 61 min) were chosen at random(one per subject). Within these tasks, the episodebreakdown was as shown in Table 2. The breakdownshows the importance of documentation at this point(52%). In contrast to earlier tasks, assimilation isvirtually absent.

5.4 THE TASK OF CATALOG SELECTION

In addition to the three kinds of tasks discussedabove, there was one other task that involvedsignificantly different kinds of problem solving:catalog selection design. The goal of a catalogselection task is to select some part or assembly froma catalog. Four particular catalog selection tasks werechosen for protocol analysis (one subject did notperform any catalog selection). The episode break-down was as shown in Table 2. This breakdown showsthat during catalog selection, the designer interactswith the catalog to assimilate new information as wellas to specify and document particular components.Catalogs play a significant role in suggesting newdesign possibilities to the designer, as well as inproviding the information the designer is seeking.

The fact that catalog selection was different enoughto constitute a new kind of task reveals two importantfacts. First, it shows that the traditional sequentialmodel of design involving three phases (conceptual,layout, and detail) is incomplete. Second, it suggeststhat other kinds of design tools (aside from catalogs)may also have a significant impact on the waydesigners organize their design processes (e.g.intelligent CAD tools such as parametric design toolsand expert systems).

the strategy they intended to use to locate the desiredinformation. Such a plan may provide a kind ofdefense against the possible distracting effects of thecatalog or handbook. There is so much information insuch books that it is important to focus on finding thedesired information in a fairly directed way.

This completes the description of the task andepisode types. The next section describes theindividual operators that make up these episodes andtasks.

6. The mechanical design operators

The building blocks of the design process are theoperators. Once selected by the controller (under theguidance of heuristic rules), the operators are appliedto change the state of the design.

To analyse the protocols, each utterance wasconsidered separately (see Stauffer (1987) for detailsof what constitutes an utterance). These utterancesaveraged 11 sec in duration each. The average episodecontained 5.4 utterances, and the average task 60utterances. Generally, each utterance reflects achange in the state of the design, so each is consideredthe result of the application of an operator. Instudying the protocol data, it was found that bydefining ten kinds of operators, all episodes and taskscould be analyzed to a more detailed and more usefullevel.

The following sections define each of these tenoperators and present several heuristic rules forapplying the operators during design. The operatorsare broken into three groups: generation operators,evaluation operators, and decision operators.

5.5 PLANNING EPISODES

Although control of the design process is primarilycarried out at the operator level, there were occasionsin which the subjects formed plans and verbalizedthem. These plans were usually rather short-range,near-term plans. Their main purpose seems to be toevaluate whether a proposed task is worth performingat the current time. One point of evidence in favor ofthis hypothesis is that the plans, once formulated,were not followed very exactly.

Another hypothesis about planning is that plans areformed prior to tasks in which many distractions arepossible. For example, before picking up a catalog orhandbook, subjects often verbalized their plan for thecatalog selection task. They would explicitly mention

6.1 THE GENERATION OPERATORS

The generation operators bring design proposals,constraints, or strategies into consideration. There aretwo such operators: Select and Create.

The 'Select Operator' causes information to bebrought into consideration in the short termmemory. The information is either old informationobtained from the design state or new informationobtained from longer term memory or someexternal source.

The 'Create Operator' introduces new informationinto short term memory, but the source of thegenerated information is unknown. Many unverbal-ized steps may have occurred to generate theinformation from long term memory.


During protocol analysis, the Select operator is usedwhen it is possible to infer where the informationcame from. Otherwise, the Create operator is used.These two operators are most commonly used tointroduce new constraints and new design proposalsinto the design state. New constraints may be derivedby combining previous design decisions with unver-balized domain knowledge. They may also arise frommental simulations and visualizations, which are alsounlikely to be verbalized (Kant and Newell, 1982;Kant, 1985; Adelson and Soloway, 1984). Similarly,new design concepts, especially during conceptualdesign, may arise from a memory retrieval or via somevisualization process. An important unsolved problemin design research is to develop some technique fordetermining where these new ideas originate.

6.2 THE EVALUATION OPERATORS

The evaluation operators serve to relate or compareinformation (especially a proposal and a set ofconstraints) from the design state in order to make adecision. Three operators are used in evaluation:simulate, calculate and compare.

The 'Simulate Operator' has the role of adjustingthe representation of its arguments to the samelevel of abstraction. This operation can rangeanywhere from 'hand waving' to building papermodels, to performing formal mathematical simula-tions. Whatever the method, the purpose is thesame: to reduce the arguments to be used in acomparison to the same representation and level ofabstraction (Adelson and Soloway, 1984).

In order to compare two items, it is necessary tohave them expressed at the same level of abstractionand represented in the same language. For example, ifthere is a constraint that says a part must fit in a2.00 ±0.005 in. slot and the abstract design proposalfor the part just says that it is 'small', then it isimpossible to compare the proposal to the constraint.The comparison can be performed if either (a) theconstraint dimension is translated into a verballanguage (perhaps 'large' in this case), (b) theproposal dimension is given a precise numerical value(with associated tolerance), or (c) both the constraintand the proposal are transformed to some intermedi-ate common level. The Simulate operator performsthis transformation. In this example, the constraintdimension of 2.00 ±0.005 in. might be translated intoa verbal description by performing a mentalsimulation.

The 'Calculate Operator' combines constraints ordesign proposals to derive new information. Thenew information is in the same representation andat the same level of abstraction as the initialinformation.

The calculate operator's purpose is to derive newinformation at the same level of abstraction. Calculateoperations employ logic, mathematics and inferencerather than search. Often calculation is driven by thefact that there is a proposal for which no constraintsdirectly apply for comparison, but a calculation usingexisting constraints yields a constraint directlycomparable to the proposal. An example of this iswhere subject S5 is trying to evaluate a variable-speed(3.2 to 89rpm) gear motor for use in his design. Heknows that he must dip 40 plates per min, that hisdesign will handle three plates per revolution, andthat he can have a 3:1 chain drive reduction. A quickmental calculation using the plate rate, plates perrevolution, and the ratio gives a new constraint(40/3 x 1/3 = 4.44 rpm) which is directly comparableto the proposed motor. Note that calculate usually isapplied to develop new constraints whereas simulatetransforms constraints and/or proposals to a commonlevel of abstraction prior to comparison.

The 'Compare Operator' has three functions: (a)to relate a set of design proposals to a set ofconstraints to determine whether the proposalssatisfy the constraints, (b) to compare twoconstraints to determine whether they are com-patible, and (c) to compare two strategies todetermine which one is better.

In the 208 analysed episodes, there were 141compare and 75 calculate operators. In most cases,each of these was preceded by a simulate operator sothat evaluation usually appears as a simulate-compareor simulate-calculate pair. This makes sense, sinceinformation must be at the same level of abstractionand in the same representation in order to apply thecompare and calculate operators.

The following rules are some of those identified inthe protocol data used by the subjects to control theapplication of these operators:

If the goal is to compare a design (R9)proposal(s) to a constraint(s), andthe proposal(s) and constraint(s)cannot be directly compared,

Then perform a simulate operation to putall the information in the samerepresentation- at the same level ofabstraction and then do the compareoperation.


In about one-third of the evaluation operations(75/216), the need was for new information ratherthan a comparison of existing information:

If the currently selected information (RIO)does not convey the desiredinformation

Then attempt to calculate new constraintsfrom the existing ones in the samerepresentation and at the same levelof abstraction otherwise select orcreate new information.

If a design proposal satisfies the (Rll)constraints and if there is anotherproposal(s) that was generated atthe same time as the first one and ata similar level of abstraction.

Then compare it to the constraints andagainst the previous proposal(s).

This last rule mentions the comparison of multipleproposals. Previous work (Ullman etal, I987a,b) hasobserved that designers typically pursue only a singledesign proposal. One hypothesis explaining thisobservation is that multiple proposals (especiallydetailed proposals) are too complex to be handledwell by human designers. The detailed analysis of theprotocols supports this hypothesis. Multiple proposalswere compared in only 13% of the comparisonoperations. Of these, the great majority werecomparisons involving abstract proposals duringconceptual design. Only 25% of the multiple-proposalcomparisons involved layout or detail tasks.

To further limit the complexity of multiple-proposalcomparisons, the designers usually (75% of the time)focused the comparison on only a few constraints(e.g., comparing two proposals for their cost, ease ofassembly, etc.). When three proposals were to becompared, one was taken as the focus, and the othertwo proposals were only compared to it rather than toeach other.

Evaluation operators arise particularly often duringcatalog selection tasks. The designer typically has a setof constraints in mind before picking up the catalog.The catalog can be viewed as providing severalalternative design proposals and constraints, all at thesame level of abstraction and in the same repre-sentation. As the designer assimilates each of thesecatalog proposals and constraints, he must compareeach of them to his design constraints.

If searching a catalog or handbookThen every new proposal examined in the

catalog must be compared to theexisting constraints.

(R12)

Another place where compare operators arefrequently applied is during the assimilation ofconstraint information from outside sources.

If assimilating a constraint (R13)Then use the compare operator to evalu-

ate it relative to existing constraints,analogous designs from memory,and other existing design in-formation.

Even during the reading of the initial problemspecification, each constraint is evaluated by thedesigner to see if it 'makes sense'. This includescomparing it to previously assimilated constraints todetermine which constraints conflict and comparing itto analogous designs from memory in order tounderstand what the constraint means and how itmight be achieved. An important example concernedsubject S5 (flipper-dipper problem). As he wasassimilating the constraint concerning the way theplate must touch the chemical bath, he compared it tothe motion of a microtome (a device for cutting thinsamples for microscopic analysis). This helped himunderstand the constraint. Unfortunately, the analogyled him to focus incorrectly on the idea that the platemust move tangent to the surface of the bath, and thisgreatly complicated his design.

6.3 THE DECISION OPERATORS

Once an evaluation is made, the decision operatorscan be invoked. There are five decision operators:Accept, Reject, Suspend, Patch and Refine.

The 'Accept Operator' is used if the comparisonresults are satisfactory. Then the design proposal,constraint, or strategy under consideration isaccepted as a part of the design state. Anacceptance is not permanent, however, as there arerepeated cases of an accepted decision later beingreconsidered and rejected.

The 'Reject Operator' is invoked if the comparisonresults are not satisfactory. As above, a rejectionmay not be permanent, as subsequent recomparisonsare possible (e.g., with new constraints or with aredefinition of what constitutes satisfaction).

It is not evident from the protocol data whatconstitutes satisfaction. One hypothesis is thatdesigners are able to make an estimate, based on thegiven constraints, of how good a solution is attainablefor the problem. Proposals that approach this estimateare considered satisfactory, while proposals that fall


short are not. According to this hypothesis, thedesigner may need to revise this estimate in the lightof repeated failure to find a satisfactory solution.

Not all evaluations result in an accept or rejectdecision. Two other things can occur. First, theevaluation can be suspended while additionalinformation is gathered. Second, the results of theevaluation can be used to guide further developmentof the proposal, and then another evaluation can bemade.

In the first case, the Suspend Operator is applied:

The 'Suspend Operator' discontinues work on thecurrent episode and establishes a new goal togenerate and evaluate the information needed tocomplete the suspended episode.

Twenty-five per cent of all episodes are suspended togather more information or work on some otheraspect of the design.

In the second case, either the Refine or the PatchOperator is applied:

The 'Refine Operator' continues to develop thedesign proposals, constraints, and strategies in theepisode to a more refined state, guided by the resultof the preceding evaluation.

The 'Patch Operator' keeps the design proposal,constraint, or strategy at the same level ofabstraction but alters it in some way guided by theresult of the preceding evaluation.

The distinction between Refine and Patch is thatinformation patched remains at the same level ofabstraction and only changes some aspect of thedesign proposal. Refine does not change the proposal,but elaborates it, to more detailed levels ofabstraction.

There is some question about whether Refine andPatch should be considered decision operators orgeneration operators. Clearly they are both. We haveplaced them here because they are invoked as thedirect result of an evaluation, like other decisionoperators. But they can be viewed as closing the loopbetween decision-making and generation of additionalinformation.

Some of the heuristics that control the applicationof the decision operators are:

If the design proposal compares satis- (R14)factorily with the constraints

Then accept it.

to accept information into the design state. In otherwords, there are approximately 550 changes in thedesign state that led from the given constraints to thecompleted design in the protocol data.

If a design proposal, constraint, or (R15)strategy is common knowledge or isvery abstract

Then accept without evaluation.

In 33% of the uses of the Accept Operator,information is accepted into the design state withoutany observable evaluation. There are three hypoth-eses that explain this observation: (a) an evaluationoccurred but it was not verbalized, (b) the informationbeing considered was so familiar to the designer thatno evaluation was needed, or (c) the informationbeing considered was so abstract that little was riskedby accepting it. The third hypothesis is particularlyapplicable early in the design process.

Another case where there is no evaluation beforean Accept Operator arises with patches andrefinements:

If

With 1 episode per min or an estimated 600episodes per 10 hour protocol and with an average of0.91 acceptances per episode there were 550 decisions

a patch or refinement (performed in (R16)response to a comparison) is guar-anteed to satisfy the relevantconstraints

Then accomplish the patch or refinementand accept without re-executing thecomparison.

In 74% of the patches and refinements, the designerwas evidently confident that the change made wasguaranteed to satisfy the constraint(s) in thecomparison, so no re-comparison was needed. In theremaining 26% of the cases where the comparison wasrepeated, it appears that another constraint wasbrought into the evaluation of the improved proposal.These observations support the hypothesis thatdesigners have a form of 'patching' expertise. Thisexpertise relates different failures or constraintviolations to appropriate patches or refinements.

Of course proposals are sometimes rejected. Thiswas relatively rare: only 12% of the 257 proposalsconsidered were rejected. The rules governingrejection (versus patching) are

If a proposal is unsatisfactory (when (R17)compared to the relevant con-straints) and little has been investedin it

Then reject it or (if a simple fix is evident)patch it.

If a proposal is unsatisfactory (when (R18)compared to the relevant con-


straints) and there has been heavyinvestment in it

Then patch the proposal or reject it if nopatch can be found.

These rules explain another observation reported inUllman et al. (1987a,b): there were many cases wheremajor problems had been identified in a proposal andyet the designer preferred to apply patches rather thanto reject the proposal outright and develop a betterone. One possible explanation is that all designactivity takes place under limited resources of time,money, and personal pride and energy. Oncesignificant resource investment has been made in aproposal (even a moderately unsatisfactory one), it ismore cost-effective to patch the proposal than to startover.

Some additional observations support this hypothe-sis. Most of the rejections (65%) were found duringconceptual design and catalog selection tasks, wheresubstantial resources have not yet been invested inany one proposal. Rejection was much rarer in layoutand detail design tasks. Even there, a proposal wasmore likely to be rejected if it was a proposedrefinement. If a proposal had been previouslyaccepted and incorporated into the design state, it wasvery unlikely to be rejected.

These rules led one designer, S6, who had hisconceptual design finished in less than one-half hour,to spend about 2 hours in layout design patching apoor, early design decision. It would have been to hisadvantage to abandon the original decisions muchearlier. However, the more time he spent patching,the more he had invested and less likely he was toabandon the design.

Here is one of the most important controlling rules:

If there is not enough information to (R19)execute a comparison

Then suspend the comparison and eitherspecify additional proposals or as-similate more constraint informa-tion.

The comparison operator not only needs all theinformation at the same level of abstraction, but it is

also sensitive to the amount of information available.If there is insufficient information to execute acompare operator, then a decision to suspend thecomparison is made and either the control remains inthe same episode (the focus is still on the same goal)or control is given to a new, assimilate episode with agoal of bringing more information into the problem.

7. Constraint management

Because constraints play an important role in thedesign process, this section describes them in moredetail and presents some control rules for constraintmanagement.

In the 208 episodes, 246 constraints were used. Thedistribution of these constraints is shown in Table 3.As can be seen, although the given constraints drivethe conceptual design, the derived and introducedconstraints play major roles. Also, the derivedconstraints, those based on design commitments,dominate the design process after the conceptualdesign.

Another statistic that clarifies the use of constraintsis the ratio of number of constraints to number ofproposals as shown in Table 3. In conceptual designthere are more proposals than constraints but, by thetime the detail tasks are reached, the design is quiteconstrained as would be expected.

Some of the heuristics that govern the use ofconstraints are given below.

If a design proposal(s) is to be (R20)compared to some constraint(s)

Then only include constraint(s) that havea direct influence on the pro-posals).

This rule raises the question of how the designerdetermines that some constraints have a 'directinfluence' on the proposal. One hypothesis, commonto most information-processing psychology models, isthat constraints are recalled through a process of'spreading activation'. This is a technique whereby theproposal itself is 'activated' and it spreads this

ConceptualLayoutDetail

TABLE 3.

Given (%)

398

11

Constraint distribution

Derived (%)

325053

Introduced (%)

284236

Ratio'

0.821.291.39

' Ratio = number of constraints/proposal


activation to items in memory that are 'nearby' insome sense. For proposals that are representedgeometrically, the activation would spread to topolog-ically adjacent objects. This would tend to identifyconstraints relating one object to its geometricalneighbors. Activation could also spread to the'function' or 'purpose' of the proposal, and fromthere, to related functions.

Spreading activation is not guaranteed to find allrelevant constraints. Indeed, part of the exploratorynature of design involves discovering that some of thegiven constraints Interact in ways not immediatelyforeseen. In the protocol data, there were a numberof cases in which relevant constraints were missed.For example, during the layout design of the batterycontacts, S2 forgot to address the constraint that thecontacts must be graspable by a 0.25 in. suction-typerobot end-effector. Eventually, during detail design,he noticed this violated constraint and patched thedesign.

Sometimes the acceptance of a proposal occurred intwo stages within an episode: initial acceptancefollowed by closer scrutiny using the less criticalconstraints (see R6).

Another reason why all constraints are not retrievedand evaluated simultaneously may be the capacitylimits of short-term memory. Since the proposal andthe constraints must all simultaneously reside in STM,it is unlikely that more than three or four constraintscan be processed simultaneously. This is summarizedin the following rule:

If evaluating a design proposalThen simulate/compare the proposal rela-

tive to no more than three or fourconstraints at a time.

(R21)

There were three cases where more than fourconstraints were simultaneously considered. Theseexceptions occurred when the constraints were veryabstract or when the subject referred to a handbookor catalog. It is not clear whether these were instancesin which the constraints were actually consideredsimultaneously or whether that was just the way theywere verbalized.

One way to overcome the limits of short-termmemory is to exploit external memory. This appearsto be one of the many important roles that drawingplays in the design process. Preliminary reduction ofthe protocols shows that over 60% of the episodes haddrawing activity (sketching, note taking, or mechani-cal drawing) in them. One of the purposes of thesedrawings, especially early in the process, was todevelop a visual image of the geometrical constraintsaffecting the design. Other purpose of drawings

include recording of accepted proposals and sum-marizing the state of the design to identify pointswhere further work is needed. Work on understandingthe role of drawing in the protocols is still under way.

8. Example episode traces

To give the reader a feel for the flow of the designprocess and to illustrate the decentralized nature ofthe episode-level control, two example episode tracesare given below. It must be re-emphasized that the 21rules are not complete. However, the TEA modeldoes make explaining the flow of the design apossibility in spite of this limitation. Rule numbers aregiven in traces where they apply.

The first example is taken from the conceptualdesign task for subject SI. The goal is the conceptualdesign of the overall assembly for the battery contactsproblem. The result of this sequence is a sketch of thelocations and orientations of the batteries and contacts(Figure 2):

episode 1: plan how to generate a design proposal.This includes selecting the constraints onthe batteries (R19) and developing a planto 'connect the batteries in series and alsohow to connect them to the printed circuitboard' (RIO).

episode 2: specify the arrangement of the batteries.She only generates one idea with the twoouter batteries 'right side up' and 'themiddle one up side down' and withcontacts from top to bottom (R5). Ineffect she has met the goal of the task butthis configuration has resulted in a derivedconstraint on the space available for thecontacts and the plastic envelope.

episode 3: plan to check out the amount of space.Although the battery arrangement wasaccepted, there is concern that theproposal may violate some of theunassimilated constraints.

episode 4: assimilate information on the space.She has selected the envelope and batteryheight from the problem statement (R2)and calculated (RIO) the remaining spacefor the contacts and the envelopethickness. This space (0.046 in) is a newconstraint on the problem and as it seemssmall 'that's not going to leave me muchroom at all in there' it is the next focus ofattention (R18).

episode 5: specify ideas for the envelope wall.After considering three ideas (R3) (each


of which she has no way to evaluate,because she is not sure of the givenconstraints) she suspends the episode(R19).

episode 6: assimilate more information.Because she is unsure of the constraints,she asks the examiner taking the protocol(an experienced design engineer) toclarify the envelope dimensions (R2).

episode 7: verify the space calculation.With the space constraint clarified, shestill has the same reaction as in episode 4and thus recalculates the space constrainton the contacts and envelope (RIO).

episode 8: specify the location of the contacts.She accepts this space constraint now thatshe is sure of it, and proceeds to refine thecontacts to be on the bottom of thebatteries (RIO). She mentions puttingthem on the side but never verbalizes anypursuit of this idea.

To this point, the subject has taken 9.0 min. It isinteresting to note that subject S2, after looking at thebatteries in his personal calculator, only consideredputting the contacts on the side of the battery. Thisdidn't turn out to be any easier than Si's approach.

In the protocol, another 5 min and 13 sec of datawere analysed for this conceptual design task.However, the example above is sufficient to show howeach episode partially determines the goal for thenext.

The second example is taken from a detail designtask performed by subject S5. In this excerpt, S5 isdetailing part of one component of his flipper-dipperdesign.episode 1: document the hole for the shaft head.

In the design, he has already drawn theoverall outline of the part called the'chair'. He now needs to document thepivot hole (R8), since it is the next mostdominant feature on the part. Unfortun-ately, he has not refined the shaft that fitsinto this hole to any specific diameter.

episode 2: specify the shaft diameter.He proposes a value of 5/8. Then hecompares this to the chair and theinterfacing part and patches it to 3/4(R16). He accepts this and records it onthe drawing. However the interfacing parthas to fit on the shaft below the surfacethus requiring a counterbore (R6).

episode 3: specify the dimensions for the counter-bore.

In specifying the counterbore, he con-siders not only the constraints imposed bythe dimensions of the two parts (derivedconstraints) but also brings in theuniversal constraints of (a) using commonvalues on dimensions and (b) keeping themanufacturing simple. He finishes theepisode (R16) by drawing the results.

To this point the subject has used 2 min and 42 sec.This task continues through the other features of the'chair' as the designer specifies and documents each ofthem.

9. Discussion

The main contribution of the TEA model is its abilityto explain the design process to the level of individualutterances in the protocol data. Beyond thisexplanatory power, the model raises several issues andimplications for future research.

9.1 FORMS OF DESIGN EXPERTISE

The foremost question raised by the model is how thetask/episode accumulation approach to design cansucceed in creating good designs. This generalapproach can be considered a 'greedy' approachbecause it operates by selecting, at each episode, thealternative that seems best. There is no global searchthat constructs whole alternative proposals in detailand evaluates them to select the best. Our designersubjects are satisficers, not optimizers.

It would appear that three things must be true inorder for this overall problem-solving strategy tosucceed.

First, the designer must choose a good conceptualdesign during the early episodes. This is because allsubsequent design effort involves refining andpatching this basic idea. Designers must have someexpertise that enables them to choose a good startingconcept.

Second, designers must be able to generate andselect good refinements throughout the design.Although not as critical as the initial conceptualdesign, it is important that the difficult subproblemsbe resolved in a way that does not make othersubproblems unsolvable.

Third, designers must be able to identify constraintviolations and apply good patches to the design.

Three general forms of expertise are thereforeneeded by our designer subjects: generative expertise,evaluative expertise, and patching expertise.


Generative expertise is the ability to analyse a set ofconstraints and generate design concepts that arelikely to satisfy them. The protocol data, unfortun-ately, did not provide much insight into the conceptgeneration process. This is probably because conceptgeneration is heavily based on long-term memory andvisual thinking, neither of which are revealed in verbalprotocols.

Evaluative expertise is the ability to evaluateproposals for feasibility. This includes not onlycomparing the proposals to the constraints but alsoestimating the likelihood that the proposals can berefined to yield a complete, satisfactory design.

Patching expertise is the ability to analyze aconstraint failure and generate a patch that repairs thefailure and does not introduce many new problemsinto the design. In expert systems for routine designtasks (Marcus et al., 1987), it has been found thatdesigners have this kind of expertise and can articulateit in some situations.

In addition to these three basic forms of expertise,designers must also have some ability to identify themost important constraints governing a design and toretrieve the constraints that directly affect a designproposal.

Further study is needed to investigate these formsof expertise.

9.2 IMPLICATIONS FOR CAD

One of the values of a design model is that itprovides some measure of the degree to whichcomputer-aided design tools support design. The TEAmodel reveals that existing CAD tools are addressingprimarily the detail design tasks, because CAD toolsare only capable of representing the geometric designstate when it is refined to a concrete or near-concretelevel. An important direction for future work in CADis to raise the abstraction level at which computer-tools can provide external memory aids for thedesigner. This is beginning to occur in the parametricdesign tools now appearing on the market (Cognition,Wisdom, ICAD and Parametric Technology).

Another, related, way in which CAD tools might beextended is to provide constraint managementassistance. The protocol analysis reveals that someconstraints are not retrieved in situations where theyare relevant. A constraint management tool might beable to monitor the design as it evolves and signal thedesigner when constraints might be violated.

Lastly, and somewhat embodied in constraintmanagement, there is a general need for CAD tosupport the human designer's cognitive limitations.

With the ability to only deal with seven or so chunksat a time, human designers have had to employ adesign process that fits within their limitations. It ispossible that a human/computer design system will beable to overcome these limitations and greatlyincrease the quality and efficiency of the process.

9.3 IMPLICATIONS FOR DESIGN EDUCATION

It is premature to make strong recommendations,based on the TEA model, for design education. Oneintriguing avenue to explore is whether the variousforms of expertise discussed above can be taught.Most current design education focuses on evaluativeexpertise. Even there, the focus is primarily on theconstruction of mathematical models and techniquesfor their solution. In the protocol problems, thesubjects constructed very few mathematical models.

9.4 IMPLICATIONS FOR ARTIFICIALINTELLIGENCE AND EXPERT SYSTEMS

The TEA model suggests that existing artificialintelligence models of design are still rather simplisticand inflexible. Control of reasoning in the protocolswas extremely flexible and dynamic. It is substantiallymore complex than such methods as top-downrefinement with constraint propagation (Steinberg,1987) and transformational development (Fickas,1985). The model suggests that future Al designsystems incorporate the flexible control ideas de-veloped in architectures such as BB1 (Hayes-Roth,1985) and SOAR (Laird et al., 1987).

Before the TEA model can provide the basis forautomating parts of the design process, (at least) twoareas require further research. First, the expertise ofmechanical designers must be studied in much moredetail so that it can be captured in the computer.Second, formal languages must be developed torepresent and reason about the state of the design atall levels of abstraction.

10. Concluding remarks

The development of the task/episode accumulationmodel of design has shown the usefulness of protocolanalysis as an exploratory tool for studying the designprocess. There is still more that protocol analysis cantell us about the design process, especially in the areaof identifying the ways designers represent and reasonabout proposals and constraints.


However, other research techniques will need to be

applied to explore many of the issues raised in this

paper. Studies of design expertise will probably

require careful comparisons of expert and novice

designers. Similar controlled experiments will be

needed to study detailed hypotheses such as the

hypothesis that the failure to reject inferior proposals

is a cost-benefit decision based on the amount of effort

already invested in the proposal. Further refinement

of the TEA model must await the completion of

studies of this kind.

Acknowledgements

The authors wish to acknowledge the National Science

Foundation for supporting this work under grants

DMC-8514949 and DMC-8712091.

References

Adelson, B. 1985. Comparing natural and abstract categories: acase study from computer science. Cognitive Science 9, 417-430.

Adelson, B. and Soloway, E. 1984. A cognitive model of softwaredesign. Rep. No. 342, Department of Computer Science, YaleUniversity.

Akin, O. 1986. Psychology of Architectural Design. England: Pion.Fickas, S. 1985. Automating the transformational development of

software. IEEE Transactions on Software Engineering SE-11 (11),1268-1277.

Hayes-Roth, B. 1985. A blackboard architecture for control.Artificial Intelligence 26 251-322.

Kant, E. 1985 Understanding and automating algorithm design. InProceedings of UCAI-S5. Los Altos, CA: Morgan-Kaufmann, pp.1243-1253.

Kant, E. and Newell, A. 1982. Problem-solving techniques for thedesign of algorithms. Rep No. CMU-CS-82-145. Department ofComputer Science, Carnegie-Mellon University.

Laird, J. E., Newell, A., and Rosenbloom, P. S. 1987. SOAR: An

architecture for general intelligence. Artificial Intelligence 33,1-64.

Marcus, S., Stout, J. and McDermott, J. 1987. VT. an expertelevator designer. AI Magazine 8(4), 39-56.

Miller, G. A. 1956. The magical number seven, plus or minus two:some limits on our capacity for processing information.Psychological Review 63, 81-97.

Newell, A. and Simon, H. A. 1972 Human Problem ProblemSolving. Englewood Cliffs, NJ: Prentice Hall.

Sacerdoti, E. D. 1974. Planning in a hierarchy of abstraction spaces.Artificial Intelligence 5(2), 11-135.

Stauffer, L. 1987. An empirical study on the process of mechanicaldesign. Thesis, Department of Mechanical Engineering, OregonState University, Corvalhs, OR.

Stauffer, L. and Ullman, D. G. 1987. A comparison of the results ofempirical studies into the mechanical design process. Submittedto: Design Studies.

Stauffer, L , Ullman, D. G. and Dietterich, T. G. 1987. Protocolanalysis of mechanical engineering design. Proceedings 1987International Conference on Engineering Design, WDK 13,68-73.

Steinberg, Louis I. 1987. Design as refinement plus constraintpropagation: the VEXED experience. Proceedings Sixth NationalConference on Artificial Intelligence (AAA1-&7). Los Altos, CA:Morgan-Kaufmann, pp. 830-835.

Sussman, G. J. and Steele, G. L., Jr. 1980. CONSTRAINTS—alanguage for expressing almost hierarchical descriptions. ArtificialIntelligence 14, 1-39.

Tikerpuu, J. and Ullman, D. G. Data representations formechanical design based on empirical data. Submitted to 1988International Computers in Engineering Conference.

Ullman, D. G. and Dietterich, T. G. 1987 Mechanical designmethodology: implications on future developments of computer-aided design and knowledge-based systems Engineering withComputers, No 2, 21-29.

Ullman, D. G., Stauffer, L., and Dietterich, T. G 1987aPreliminary results on an experimental study of mechanicaldesign Proceedings NSF Workshop on Design Theory andMethodology, pp. 145-188. Available as Report Number 86-30-9Department of Computer Science, Oregon State University.

Ullman, D. G., Stauffer, L., and Dietterich, T. G. 1987ft. Towardexpert CAD. ASME, Computers in Mechanical Engineering6(3), 21-29.

Ulrich, K. T. and Seenng, W. P. 1988. Function sharing inmechanical design. Proceedings of AAA1-88.

David G. Ullman is an Associate Professor in the Mechanical Engineering Department atOregon State University with funded research in Design Methodology and theDevelopment of Intelligent CAD Systems. He has prior teaching experience at RensselaerPolytechnic Institute's Center for Interactive Computer Graphics, Union College and theOhio State University; prior research experience at Battelle's Columbus Labs, The AirForce Flight Dynamics Lab and the David Taylor Naval Ship R&D Center; andengineering design management experience with Oxygen Enrichment Co., a small medicalproducts manufacturing company. His PhD is from the Ohio State University in 1978. Hisresearch is focused on the development of computer tools to aid in the early stages of thedesign process. Using AI and Cognitive Psychology he is studying practicing designengineers as a basis for data used in tool development. He is the author of MechanicalDesign Failure Analysis and on the editorial board of Research in Engineering Design andThe International Journal of Engineering Design.

Thomas G. Dietterich has been a faculty member in Computer Science at Oregon State

University since 1985. He received his PhD from Stanford University in 1984, an MS from

the University of Illinois in 1979, and an AB (in mathematics) from Oberlin College in

bpo46

bpo46

bpo46


1977. He is the author of several papers in the area of machine learning, and he served asthe editor and principal author of the 188 page survey of machine learning that appearedas Chapter 14 of the Handbook of Artificial Intelligence. Dr Dietterich's research interestsinclude machine learning theory, intelligent computer-aided design for mechanicalengineering, models of learning by experimentation, and methods for automaticallyimproving the efficiency of algorithms. He is an editor of the journal Machine Learningand recently received a Presidential Young Investigator Award from the National ScienceFoundation.

Documents

A model of the mechanical design process based on ......Jul 14, 2016 · (Al EDAM) (1988) 2(1), 33-52 A MODEL OF THE MECHANICAL DESIGN PROCESS BASED ON EMPIRICAL DATA DAVID G. ULLMAN1,