Unified Program Planning

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-2, NO. 5, NOVEMBER 1972

Unified Program PlanningJ. DOUGLAS HILL, MEMBER, IEEE AND JOHN N. WARFIELD, SENIOR MEMBER, IEEE

Abstract-Program planning begins with problem definition and endswith planning for action. The key products that result from the problemdefinition, value system design, and system synthesis steps are discussedand interrelated through the use of interaction matrices. Particularemphasis is given to defining objectives and to defining a set of measureson the objectives by which to determine their attainment. Interactionmatrices relate objectives measures to objectives and link activities andmeasures of their accomplishment to the attainment of objectives. Amajor consequence of program planning is the choice of a program topursue, and identification of the projects that will be carried out as a partof a selected program. Selecting the set of projects is discussed in termsof consistency with corporate or agency policy, and the economics, risk,and potential benefits associated with each project. A criterion functionthat incorporates the latter three factors is described and proposed as apractical way of evaluating the relative merits of projects.

INTRODUCTIOND EVELOPMENT of a theory of systems engineering

that will be broadly accepted is much to be desired.The process of developing such a theory is iterative betweenform and content. If one has a form, i.e., a broad frameworkfor such a theory, the content can be matched to that form.In the process of developing the content, it may be foundthat the form is deficient and requires change. Then thecontent will have to be reorganized, amended, and aug-mented. This may result in further modification of the form.Two things can be said about the initially chosen form,

with which the content is to be associated:1) The form does not have to be totally correct, but only

reasonably adequate to permit the content to be developedand structured.

2) Without such a form, the iterative process needed todevelop the theory cannot proceed.An initial form that seems quite adequate for develop-

ment is that given by Hall [1]. This form is a three-dimen-sional morphological box. Two dimensions of this box arethe phases and the steps of systems engineering. If one takesthese phases and steps as given for beginning the iterativedevelopment of a theory of systems engineering, a significantconceptual obstacle to the development is overcome. It thenbecomes possible to proceed to relate existing content tothe phases and the steps, and to discover where there is aneed for new ideas to augment that content.

Fig. 1 shows Hall's matrix with seven phases, the sevenlogic steps shown as coordinate indexes. The a entered inthis matrix represent sets of activities associated with eachsquare of the matrix. For example, a1, represents thoseactivities to be carried out in the problem definition stepof the program planning phase. This paper seeks to developcontent associated with phase 1 of Hall's matrix, the pro-gram planning phase.

Manuscript received November 23, 1971.The authors are with Battelle Columbus Laboratories, Columbus,

Ohio 43201.

In presenting the framework, Hall did not attempt tospell out the content in any detail, but he did call attentionto the use of the framework as an "aid to discovering, orseeing more clearly, unique activities." It is in this sense thatthe program planning phase described in his paper is used.It is important to read his paper before proceeding withthis one.

It is intended in this paper to discuss steps in the programplanning phase as a connected set. It is hoped to show thatwhile there is continual iteration and reevaluation amongsteps in this phase, it is possible to unify the process ofprogram planning. This is done by linking the primaryproducts of various steps in a way that allows documentationand display of an overview of what has transpired. Thoughthis paper is limited to the program planning phase, muchof what is presented could be applied to other phases aswell.

Because of the complexity, and the need to be able toperceive a multiplicity of relations, a graphic approachseems essential. An example will show application to plan-ning for the development of short-takeoff and landing(STOL) aircraft as part of an air-transportation system.

PROBLEM DEFINITION LINKAGES

At the outset of a program-planning activity, an organiza-tion is faced with defining a problem or issue that theorganization might wish, after suitable study, to address.The assumed input conditions are a broadly defined set oforganizational goals and a set of available resources. Prob-lem definition is usually a group activity since it requires anoutscoping in thinking to encompass a broad scope ofpotential ideas and candidate problems. Outscoping is adeliberate group attempt to embed the problem or issue inthe next larger problem or issue iteratively in order toexpand the scope until the problem or issue is seen in anencompassing context. This requires a language in whichto develop and portray the product of the group. Thelanguage of graphics appears to fulfill this need. Trees andmatrices can be used to provide a unifying visual picture ofthe program plan as it evolves.Problem definition in the program planning phase is

needed for the value-system design step and the planningfor action step. It is also needed for the problem definitionsteps of later phases. Twelve products of problem definitionare:

1) a well-conceived title for the problem or issue;2) a descriptive scenario, explaining the nature of the

problem, and how it came to be a problem, present-ing as much history and data as can be prepared withavailable resources;

3) an understanding of what disciplines or professionsare relevant to an attack on the problem;

610

HILL AND WARFIELD: UNIFIED PROGRAM PLANNING

Steps of theFine Structure 1 2 3 4 5 6 7

Problem Value System System Systems Optimization Decision Planning fologic Definition Design (de- Synthesis Analysis of each Making Action (to

velop objec- (collect and (deduce con- Alternative (Application implementtives and invent alter- sequences of (iteration of of Value next phase)criterion) natives) alternatives) Steps 1-4 System)

plus modeling)Phases of tthe Coarse i|Structure m

e\

1 Program Planning a 11 a 12 a 16 a 17

Project Planning2 (and preliminary 21

design)

System Development3 (implement project 37

plan

Production (orconstruction)

5 Distribution(and phase in)

6 Operations a(or consumption) 61

7Re irement a

71a

72 a 76 a 77(and phase out) 71 72

Fig. 1. Hall activity matrix.

To Introdtice a Safe, Reliable, and Economical STOL System into the National

Transportation System

To EstimAte Impact of STOL Transportation System on Urban and ltegional Development

To Develop Svstem and Subsystem Technologies to Fuirtlher STOL System Development

To Select Potential STOL System Technology Developments and Allocate Resources

To Establish and Disseminate System Criteria to Indtustry and Government

To Cotndtuct Experimenital Flight Test Program

To Svnthesize STOL Systems Configurations to Satisfy Mission Performance Goals

To Specify Desired Flighet Pathis and Corresponding States

To Define Candidate STOL Svstem, Stubsystem, and Components

To Establish Candidate STOL System Operating Procedures

To Deftine the Total Air Transportation Operational Environment

To Forecast STOL Traffic Demand by Geographic Areas

To Compile anid Alnalyze All Backgrounid Material Relevant to STOL Transportation System

To Define STOL 'Mission Performance Coals

To Define STOL Svstenm FtunctionisStrong Interaction

2 Moderate Interaction

Fig. 2. Example of self-interaction matrix.

4) an assessment of scope;5) a determination of the societal sectors involved;6) an identification of the actors to be involved in the

problem-solving situation;7) an identification of need;8) an identification of alterables (those elements in the

system that are subject to change);9) an identification of major constraints;

10) some partitioning of the problem into relevantelements;

11) some isolation of the subjective elements of theproblem;

12) description of interactions among relevant elementsof the problem.

It is particularly important during the problem definitionstep to congeal the ideas of the planning-group members.

611

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, NOVEMBER 1972

Competing Modes of Short-Haul Transportation

Competition for Development Funds

Environmental Pollution

Real-Estate Availability

Public Acceptance

System Operating Costs

4J

4-4;

to

545..-4 0)

0 $4

am>44

Om

co>54>

to 0

0. OS4 Wa4> 4iO en

H

4

ctU

H

0

.0V

54

0

X

54..o

Z

nm5454

4.4 u

nH0 44:D w

-A0

0.0a u.0 W54 .410=

COn9

0uz

54

IL40

*j C

0 ea.C 4i.04

4to> 0

t4

AH

54

3

.sI0

0Vjjl_

oW_,

514

00a4

HU1

In

C:O

z

LEGEND

E - Severe Constraint

E - Moderate Constraint

Fig. 3. Example of cross-interaction matrix.

Alterables Needs

ConstraintsInteraction _Matrixa

l Needs

AlterablesInteractionMatrix

4

o

C,

.0-

4

C:

Societal Sectors

H

I II ISocietalSectorsInterac-tion7tri

Fig. 4. Interactions of importance in problem definition for program planning.

612

Needs

m44

Cu0

00

4

00.

as<:0.

C,.4:3 S

001la

0 C:>.4$4 O9 4-


The arrows indicate that one objectivecontributes to the attainment of anotherat a higher level

Fig. 5. Form of objectives tree.

A clear picture of the interactions between societal sectorsand needs, between needs and systems alterables, betweenalterables and constraints, and between constraints andneeds is desired. In addition, the self-interactions amongsocietal sectors, needs, system alterables, and constraintsshould be clear. The tool suggested for presenting theseinteractions is an array of interaction matrices of two types.The first type is the self-interaction matrix [2] illustrated

in Fig. 2. This example illustrates two levels of dependencypresented by the interactions of the objectives defined for atechnical development program that could lead to introduc-tion of an air-transport system based upon STOL aircraft.The name self-interaction derives from the fact that thesame set of coordinates appears along both axes of thematrix.The second type is the cross-interaction matrix illustrated

in Fig. 3. The cross-interaction matrix portrays the inter-action between different types of elements; for example,needs and constraints. The degree of impact of the con-straints upon the needs can be shown by using differentsymbols at the intersections. For example, in Fig. 3 we seethat environmental pollution is considered to be a severeconstraint in planning the airports for short-haul aircraft,but only a moderate constraint on the development ofcommercial short-haul aircraft.The overall relationships between constraints, alterables,

needs, and societal sectors as they both self interact andcross interact in a problem-definition activity may beportrayed by linking them together as shown in outlineform in Fig. 4. Such a presentation shows five of the twelvetangible products expected from the problem definitionstep of program planning. These products are:

1) a determination of the societal sectors involved;2) an identification of need;3) an identification of alterables;4) an identification of major constraints;5) description of interactions among relevant elements of

the problem.The use of such a presentation allows the planning group

to keep track of the various elements that are consideredduring the problem definition process. More important, itprovides a formal structure for problem definition, and, aswill be seen, the rationale for defining objectives.

VALUE SYSTEM DESIGN LINKAGES

Value system design activity includes 1) defining objec-tives and ordering them in a hierarchical structure; 2) re-lating the objectives to needs, constraints, and alterables:and 3) defining a set of measures on the objectives by whichto determine the attainment of objectives.To provide some precision to program planning, a

specific syntax has been developed for the form of anobjective. An objective is defined as:

infinitive verb + object word or phrase + constraints.Thus "to teach children the French language" is an exampleof an objective. To provide a structure for graphicallyportraying the relationship among objectives, one constructsan objectives tree. Two simple rules are employed to con-struct this tree: 1) Each objective is written within a rect-angular box to form a vertex of the tree. 2) Two boxescontaining two objectives are connected, if achievement ofone of the objectives contributes directly to achievement ofthe other.

In constructing an objectives tree, one should not beconcerned about where to start. Instead, one will start withany objective that is clearly contributory toward the desiredchanges. As soon as one objective is defined, one thenconsiders lower and higher level objectives related to it.A lower level objective will have to be contributory to the

one that was stated first. A higher level objective will haveto be such that the one stated first is contributory to it. Ifone thinks of at least one lower level objective and at leastone higher level objective, he is on his way to constructingan objectives tree. When one is through, he will probablyhave a structure like that shown in Fig. 5, from which thename "tree" derives. It may turn out that there is more thanone tree, since it is not always true that all the objectivesone would seek to satisfy could be shown on one tree.Usually though, if there are separate trees, one can find ahigher level objective to which all trees can be tied, thuscreating a single (though perhaps rather leafy) tree.A different way of portraying the information contained

in an objectives tree is by way of a corresponding self-interaction matrix as shown in Fig. 6. Fig. 6 contains all ofthe information required to draw the objectives tree shownin Fig. 5 so long as we know that low-numbered objectivescorrespond to high-level objectives and vice versa. The self-

613


Fig. 6. Objectives self-interaction matrix corresponding to objectivestree shown in Fig. 5.

interaction matrix method of portrayal is not as clear as theobjectives tree for viewing the relationships among objec-tives, but it incorporates significant advantages in relatingobjectives to constraints, alterables, and needs. The methodsuggested for portraying these relationships is the use ofcross-interaction matrices as depicted in Fig. 7. This figurerelates objectives to needs; to alterables, which can bemodified to bring about attainment of the objectives; andto constraints, within which the objectives must be attained.In one concise figure, a complete outline of a rationale tosupport the objectives and a great deal of informationneeded to plan a program for attaining them appears.

A particular advantage of the method shown in Fig. 7is the ease with which the interaction of the objectives can

be traced through the needs back to the societal sectorswith which the objectives interact. If the interactions are

categorized as either significant or insignificant (i.e., binary),then a simple Boolean multiplication of the objectives x

needs interaction matrix with the needs x societal sectorsinteraction matrix will result in an objectives x societalsectors interaction matrix.The Boolean multiplication of cross-interaction matrices

can be extended to the mathematical generation of some ofthe matrices shown in Fig. 7. For example, if the four cross-

interaction matrices that lie closest to the self-interactionmatrices were filled in by hand, the three remaining cross-

interaction matrices shown in Fig. 7 could be generatedmathematically. Such a formal procedure has considerablemerit. Without it, one tends to end up with a set of cross-

interaction matrices which are not mutually consistent andit is not always easy to spot the inconsistencies. Checkingof logic is especially useful when the matrices form a loopas will be discussed later in relation to an example.The need for defining a set of measures on the objectives

by which to determine their attainment is an importantconcern in program planning. Too often, people defineobjectives without thought as to how they will measure

their accomplishment. Upon examination of the objectivestree, one usually finds that some of the objectives are

axiological (rooted in value judgments), while others are

not. The axiological objectives usually lie at the top of theobjectives tree. An example of such an objective is: "toimprove the public schools." The word "improve" makesthis objective axiological in nature, since whether thisobjective is attained or not is a matter of subjective opinionor value judgment. A nonaxiological objective is one like"to teach children to distinguish a Mozart compositionfrom a Beethoven composition." The achievement of thisobjective is determinable, and not a matter of opinion as towhether the children can or cannot make the distinction.The axiological objectives serve an inspirational purpose,but the nonaxiological objectives are more useful in plan-ning because they are more readily converted into plannedactivities.One may examine the objectives tree to see which objec-

tives are measurable, and how they may be measured. Forthe musical objective mentioned above, the measure is theagreement between the child's answer and the correctanswer. The determination as to whether the public schoolshave been improved is vastly more difficult to make, andsuch an objective is virtually immeasurable within anyreasonable cost. However, the attainment of lower levelobjectives that are contributory to that one may suggestthat progress is being made.

For example, it should be possible to measure the attain-ment of an objective such as, "To improve the method ofteaching reading to sixth graders to the extent that at least70 percent of the students exceed the fiftieth percentileperformance on a standardized sixth-grade reading achieve-ment test." The corresponding objective measure could be"Percent of sixth-grade students, whose performance ex-ceeds the fiftieth percentile performance on a standardizedsixth-grade reading achievement test."The total process of measurement involves more than just

the selection of the measure or unit by which the attainmentof objectives will be assessed. Often, as in the above example,a threshold for judging acceptable performance can bedefined and built into the objective and objective measure.The balance of the process of measurement includes plan-ning for how the data required for evaluation are to besensed and how they are to be analyzed to generate anindication on the selected measurement scale. In the aboveexample, planning for the sensing function would involveselection of the achievement test, planning when, to whichstudents, and under what conditions the test would beadministered and planning for how the resulting tests wouldbe interfaced with the test-scoring activity. Planning theindicator function would involve selection of procedures foranalyzing the test scores and reducing them to standardachievement scores in a timely and efficient manner.Thus the planning of objectives and objectives measures

is tightly interwoven with the process of determining howthe measures are to be obtained, i.e., how the data are tobe sensed, and how the indication on the scale of measure-ment is to be attained.The measures of objectives may be conveniently related

to the objectives through the use of a cross-interactionmatrix as indicated in Fig. 8. Measures 1 and 2 relate to theattainment of objective 6; measure 3 to the attainment of

614


Constraints

Objectives

Matrix

I I

I I

ConstraintsInteractionMatrix

Alterables Needs

Alterables Needs

_ ____

Needs

I ~ ~ ~Alterables /

Matrix / ;

4 jp

Societal Sectors

I ISocietalSectorsInteraction /Matrix

Fig. 7. Interaction of objectives with constraints, alterables, needs, and societal sectors.

0h

0)

Fig. 8.

J2

Objectives

Objectives

Measure 1

Measure 2

Measure 3 *

Measure 4 _

Measure 5

Measure 6

Measure 7

Measure 8

Measure 9 1 1

Relating objectives measures to objectives.

615

m0)

14J

4

0

ci

0

C,

.0

0

4)z;


Activities Measures Objectives Constraints

Activitiesca Interaction3 Matrix

.4/

I I

I I1. II II II II I

I

I Activities Measures |

I II Objectives

Ccns tratntsInteraction/Matrix/

a)

0

.0

0

4J

Fig. 9. Linkages for system synthesis.

objective 5; measures 4, 5, and 6 to objective 4; measures7 and 8 to objective 3; and measure 9 to objective 2. In theexample, no measure relates to objective 1, the assumptionbeing that the highest level objective, in this case, is notdirectly measurable.

SYSTEM SYNTHESIS LINKAGES

Hall's matrix, Fig. 1, shows that following problemdefinition and value system design comes the system syn-thesis step. System synthesis activities are directed atanswering the following questions. What are the alternativeapproaches for attaining each objective? How is eachalternative approach described? The answers are usually inthe form of a series of activities which form a plan forevaluating alternative approaches for attaining the programobjectives.Three major linkages to the preceding steps must be

given visibility. 1) The relationship between the plannedactivities and the program objectives. 2) The interactionbetween the planned activities and the program constraints.3) The measurement system required for relating the pro-gress on the activities to the attainment of objectives.

Again, when faced with a linkage problem, the self- andcross-interaction matrices are used as illustrated in Fig. 9.The activities x objectives cross-interaction matrix is usedto relate the proposed activities to specific objectives.Similarly, the interactions of the constraints with theactivities are illustrated by the activities x constraintsinteraction matrix. Thus the first two of the major linkages

listed in the preceding can be given visibility throughcross-interaction matrices.

Next, the measurement of progress on activities is relatedto progress on attainment of the program objectives. De-velopment of a set of objectives measures was discussedpreviously. An analogous procedure is followed in measur-ing the accomplishment of activities. One or more measuresof accomplishment are defined for each activity and relatedto it through the activities x activities measures interactionmatrix shown in Fig. 9. Often, the activities measures areof the form "Percent completion of " where the threedots represent one of the products of the activity underconsideration.A question which management is likely to ask is "How

can you relate the attainment of objectives to the accom-plishment of activities?" One method is to examine therelationship between activities measures and objectivesmeasures. The objectives measures x activities measurescross-interaction matrix can be generated mathematicallyby Boolean multiplying the objectives measures x objec-tives, objectives x activities, and activities x activitiesmeasures interaction matrices. The resulting matrix mustthen be carefully examined to ensure that measures ofaccomplishment of the activities do relate to the objectivesmeasures. If such is not the case, a reexamination of allmeasures and of the activities x objectives interactionmatrix must be made. Either the matrix must be changedor the measures redefined so the activities measures relateto the attainment of objectives.

616

co

4)

IIIIIIIIII

_c_4)

ct


.Activities MeasuresI~~~~~~~~~~~~~~iAciite Mesue li

109n

.14

n

Fig. 10. Program planning linkages.

617

Activities

Objectives Constraints

Alterables Needs

co

ucJ

.In

e

4cOv

Societal Sectors

m

Go0

.4>>i

n44)

:


To this point, a series of related linkages for the problemdefinition, value system design, and system synthesis stepsof program planning has been discussed. An overall viewof a program as planned at the end of the system synthesisstep is obtained by combining Figs. 7 and 9 as in Fig. 10.Added to Fig. 10 is an objectives x agencies interactionmatrix to portray which government or industrial groupshave an interest in the defined objectives and an agents xactivities interaction matrix to identify the agents responsiblefor conduct of each activity.One concise figure portrays the major products of initial

program planning efforts and their inter- and intrarelation-ships, and provides a useful tool for keeping track of subse-quent progress as action is taken to implement the activitiesand attain the program objectives. Fig. 11 illustrates anapplication of a chain of binary interaction matrices. Thischain was used to relate a proposed program directed atidentifying technology development areas critical to STOLaircraft to the more comprehensive problem of implement-ing commercial STOL aircraft service as an integral part ofthe U.S. national transportation system. At the right sideof Fig. 11 are sets of needs, alterables, and constraintsassociated primarily with the comprehensive problem withinwhich the problem of identifying needed technology de-velopment is embedded. The activities and all but the twoobjectives at the top of the list shown on the self-interactionmatrix in the center of the figure relate to the proposedprogram for identifying technology developments neededto further STOL system development. The top two objec-tives relate to the comprehensive problem.

In planning this proposed program, the illustrated set ofinteraction matrices was used both to develop a perspectiveof the proposed program in relation to the larger problem,and to assure that the program would support the objectivesof the larger problem taking into account the needs, alter-ables, and constraints associated with implementing acommercial STOL system.

For example, one of the proposed activities shown inFig. 11 is "Synthesize STOL vehicle system configurationusing Monte Carlo avionics evaluation program (AEP) tosatisfy mission performance goals." This activity is shownin a box at about the center of the list of activities in Fig. 1.As indicated by the activity x activities measures crossinteraction labeled "I," the activity measure associatedwith this activity is "percent completion of definition ofvehicle system configurations." The activity was designedto attain the objective "To synthesize STOL systems con-figurations to satisfy mission performance goals" as in-dicated by the cross interaction labeled "G" on the activi-ties x objectives cross-interaction matrix.By following around the loop of interaction matrices at

the left side of Fig. 1, one sees that the interaction labeled"J" in the objectives measures x activities measures cross-interaction matrix relates the activity measure to the objec-tive measure "percent completion of mission performancegoals definition" which, through interaction K, is related

and its measure of accomplishment is tightly connected tothe attainment of a corresponding objective or objectivesand their measures of attainment.One can also -see from Fig. 11 how the proposed activities

relate to the more comprehensive problem. For example,the aforementioned activity must take into account theconstraint "system operating costs" since it is directlyrelated to that constraint by the activities x constraintinteraction labeled "H." In turn, through the cross inter-actions labeled "B" and "A," it is seen that one of themajor alterables to consider in conducting the activity is"aircraft performance characteristics" and that the activityrelates to the need for "commercial short-haul aircraft."The cross interaction labeled "F," "E," and "D" point

out, as would be expected, that the needs, alterables, andconstraints previously discussed also affect the objective"To synthesize STOL system configuration to satisfymission performance goals." The agencies responsible forattaining objectives and the agents responsible for ac-

complishing activities are also contained in Fig. 11. Forthe objective and activity discussed, these responsibilitiesare indicated by cross interactions "M" and "L" respec-

tively.The system analysis and optimization steps shown in

Fig. are generally concerned with reducing the number ofprogram alternatives through the application of a widevariety of analysis procedures that are highly contextual.For that reason, they will not be discussed in this paper;

but those procedures must be planned to produce an outputwhich is consistent with the input requirements of thesubsequent decision-making step.

DECISION MAKING IN PROGRAM PLANNING

During the system synthesis step in program planning,there will have been defined measures for determining theattainment of program objectives. Also, a set of activitiesand activities measures for guiding subsequent activitiestoward the development of a complete program plan willhave been defined. The questions that arise are "Whatcriteria will be used to select projects for development?"and "What information must be obtained in the systemanalysis and optimization steps in order to compare

alternative projects?"Four major factors concern the decision maker in evaluat-

ing alternative projects for possible further development.First, he must determine that the scopes of the projectsunder consideration are consistent with corporate or agency

policy. This determination can be made by evaluating howwell the candidate projects satisfy the program objectiveswhich are assumed to be in consonance with corporate or

agency policy. (A program whose scope is not consistentwith corporate or agency policy would be rejected on thebasis of this alone.) Those projects that pass this initialscreening are then rated in terms of the remaining threefactors discussed below.The second major factor is the comparative economics of

to the objective mentioned above. In this way, the activity

618

the alternative projects. The analysis should look at the


long-range project costs, not just the development costs orthe cost required to get a system into operation. Total life-cycle costs appear to be an appropriate economic measuresince they include all system costs and put the cost analysisfor each alternative project on an equitable basis forcomparison.The third factor, risk associated with projects, has re-

ceived considerable emphasis recently, particularly by theDepartment of Defense. At least two types of risks shouldbe considered in selecting projects. The first is the "riskdue to nature." By this is meant the probability that aproject will not succeed because the technical requirementsare incompatible with basic physical laws. The second riskis the "risk due to technology." This risk is the probabilitythat a project will be unsuccessful because it requirestechnology beyond the current state of the art even thoughno laws of nature appear to be limiting. Other types ofrisks would be appropriate in assessing programs withhigh-social content.The fourth major factor to be considered by the decision

maker is that of benefits which would result from the pursuitof each alternative project. Decision makers are faced withthe problem of evaluating the worth of each project underconsideration. Worth assessment [3] is a formal procedurefor assessing the worth of discrete alternatives. It appearsto be well suited to providing a "benefit" input.

Criterion Function

In comparing alternative projects, it is desirable to com-bine the major evaluation factors into a single, scalar-valuedcriterion function. Such a function must be reasonablygeneral and easy to interpret if it is to have wide applicationThe function suggested here is derived by multiplying twoparts to yield its value. The first part is the probability ofbeing able to successfully carry out a candidate project,calculated by multiplying one minus the risk due to natureby one minus the risk due to technology. The second part ofthe criterion function is a cost-benefit factor composed ofthe weighted sum of inverse normalized project cost andproject worth.The criterion function is calculated for each alternative

project. The criterion function is expressed as

Qi = (1 - Rn)(l - R,)(aCi0 + byV)where

Qi value of criterion function for ith project; for i =1,2, ,q, where q is the total number of projectsunder consideration;

Rn risk of nature to the ith project = 1 - lk(l -rk,i),where rk, in = (k = 1,2, ,m) is the risk due tonature to the kth project characteristic of the ithproject, where m is the total number of projectcharacteristics for which risk due to nature isestimated;

R, risk of technology to ith project = 1 - k(l -rk t),

nology to kth project characteristic of the ithproject, where p is the total number of projectcharacteristics for which risks due to technology isestimated;

Cjo normalized inverse life-cycle cost of ith alternativeproject = (l/Ci) mini Ci, where Ci is the estimatedcost of ith alternative project;

a weighting factor for normalized inverse life-cyclecosts;

b (1 - a) = weighting factor for worth scores;Vi worth score for ith alternative project. (0< V. < 1;

i- 1,2, ,q).

The criterion function combines the risk factors with aweighted average of inverse normalized life-cycle cost andworth assessment score. The ideal configuration/programalternative combination would have zero risk due to nature,zero risk due to technology, the least life-cycle cost of anyalternative project, and a worth score of 1.0. In this idealsituation, the criterion function would have a value of 1.0.

If the risk due to nature or the risk due to technology forany project characteristic is I (that is, the characteristic isjudged to be impossible to meet), then the probability ofsuccessfully developing the configuration under considera-tion is zero and its criterion function has a zero value. Sincethe risk due to nature and risk due to technology will eachbe greater than or equal to the maximum risk of eachcontributing risk factor, careful consideration must begiven to the estimation of each risk factor. The computationof risk draws attention to those factors that would poten-tially prevent project success and helps ensure that a criticalfactor is not ignored.

Weighting factors "a" and "b = (1 - a)" allow changingthe relative importance of the cost and worth factors. Thechoice of values for a and b will be governed by such factorsas confidence in the cost analysis (e.g., low confidence;make "a" small), the magnitude of the costs relative tototal resources, and the significance of the benefits that couldbe used depending upon the sensitivity to cost. For example,

O maxi Ci - Cimaxi Ci - mini Ci

provides a linear weighting to cost variation between themaximum and minimum cost projects. The cost normaliza-tion scheme suggested earlier provides considerable sen-sitivity to cost variations near the minimum cost and muchlower sensitivity and a lower weight to costs much greaterthan the minimum cost. Numerous other ways of normaliz-ing the project cost estimates are available and should beexamined in the context of the particular problem understudy.The numeric value of the criterion function at the pro-

gram planning phase is normally quite small. This observa-tion reflects the uncertainties associated with projects duringan early planning phase. In particular, technically demand-ing projects will have low criterion values due to the risksinvolved. The absolute value of the criterion function is

619

where rk, i' = (k = 1,2, - -,p) is the risk due to tech-


not as important, however, as the relative value for eachalternative project.

Project selections are made on the basis of the programscope being consistent with corporate or agency policy,as mentioned earlier, and the relative values of the criterionfunction for each project. If the criterion values are low, thereasons for this should be considered and action taken inthe subsequent project planning phase to investigate thereasons underlying the low values.

Evaluation of RisksIn the program planning phase, the evaluation of risks

inherent in pursuing various alternative projects is usuallydependent upon expert opinion and subjective judgmentrather than detailed analyses. A typical approach to riskevaluation is to make a detailed breakdown of the func-tional performance factors forecasted for each project andto call in experts in each of the functional areas to assessrisks due to nature and technology associated with attainingthe projected performance. Care should be taken in definingthe functional performance factors to assure that they areall of about the same level of importance and that the risksare due to independent causes. This care is suggested sincethe value of the criterion function is equally sensitive toeach risk associated with each factor and the risk calculationassumes independence of risks.

High-risk performance factors should be flagged so thatsubsequent project planning calls for an early second assess-ment of the high-risk performance factors for each of theprojects selected for further development.

Worth Assessment

Worth assessment is a formal procedure for assessing theworth of discrete alternatives in the decision-making en-vironment. The following is a brief outline of a worthassessment procedure developed by Miller, [3].

A. Define worth criteria.List criteria for worth assessment ensuring list:

1) contains all significant criteria;2) contains only mutually exclusive criteria;3) contains only criteria of major significance;4) contains only worth independent criteria.

B. Develop hierarchical structure of worth assessmentcriteria.Break down high-level worth assessment criteria intoone or more lower level criteria which contribute tothe high-level criteria.

C. Develop performance measures.Select a single physical-performance measure for eachlowest level worth assessment criterion in the hier-archical structure.

D. Develop worth relationships between performancemeasures and lowest level worth assessment criteria.Define a scoring function to assign a unique worthscore in points to every possible value of a physicalperformance measure. A scoring function is defined,either explicitly or implicitly for every lowest levelworth assessment criterion.

E. Develop a single overall index of worth.Define an additive weighting function with constanttrade-off weights to combine the lowest level criteriaworth scores.

The index of worth is devoid of any risk and/or un-certainty. It assumes that the project, activity, or perfor-mance consequence being evaluated will occur for certainand the process of assigning a worth number provides nomechanism for reflecting perceived trade-offs between theworth of an outcome, conditional upon its actual occur-rence, and the variable risk or uncertainty surrounding itsoccurrence. The index of worth appears to complement theCriterion Function which has separate risk factors builtinto it.

Miller's worth-assessment technique relates to programplanning in another way. The objectives measures providea baseline upon which to develop the worth assessmentcriteria and performance measures of each alternativeproject. Also, the objectives interaction matrix and otherinteraction matrices that relate the objectives to constraints,alterables, needs, and societal sectors provide considerablevisibility to the relative importance of the worth assessmentcriteria. The task of weighting the worth assessment criteriacan then be done in relation to their contribution to therelated needs, constraints, etc.As part of the worth assessment procedure, the relation

of the performance measures to the needs should be ex-amined and used to develop the scoring functions thatrelate the performance measure of each lowest level per-formance criterion to a worth score. The relation of per-formance measures to needs should also be consideredwhen developing the adjusting factors to compensate forthe fact that performance measures may not adequatelyrepresent the performance criteria. Worth assessment isrecommended as a formal approach to evaluating thecomparative worths of alternative projects. The resultingworth scores satisfy the requirement of the criterionfunction for a worth score for each alternative project.

SUMMARY

The systems engineering steps required to do programplanning are not related only to their neighboring stepsnor are they carried out in a sequential temporal order.Rather, they form a logical set of operations that arehighly interrelated and consequently must be continuallyreviewed with respect to each other as program planningprogresses. In planning a complex program, the linkagesbetween these operations tend to become buried in thecomplexity of the problem rather than being emphasizedand given visibility. The set of interaction matrices illus-trated in Figs. 10 and 11 provides a visible means of organiz-ing and managing program planning activities.

Decision making will always require the subjective inputof experienced managers. Nevertheless, formal evaluationtechniques are helpful in assessing the relative merits ofalternative projects that comprise a program. The criterionfunction described in this paper combines the factors of

620

t,tgbll,h CC Idgit STOL Sytt.t Optt.tttg PFCi,dtt.g

iltCthtTttClAlFTtpCFtltiCtOpgFgtlCC.lEttttttttttt

ttt STOL Tt.tfl. tt.tttd by C gt.phtt Att.,

ttd Att.ly.t til B kgtt.ttd Mttttigl Rgl. .tt it STOL .ptttgtttt Sy.t*.

Blight Ptyftyggttt Gigi.

Fittitig

Ag.

-II

:1

Objtttltt. Obj.ttlt..

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-2, NO. 5, NOVEMBER 1972

risk, worth, and cost into a single, scalar-valued functionfor comparing alternative projects in a rational, objectivemanner.

ACKNOWLEDGMENT

The authors wish to thank A. D. Hall for permission touse Fig. 1, Hall activity matrix, which also appeared in hispaper [IJ.

REFERENCES[1] A. D. Hall, "Three-dimensional morphology for systems engineer-

ing," IEEE Trans. Systems Sci. Cybern., vol. SSC-5, pp. 156-160,Apr. 1969.

[2] S. F. Love, "Modern design methods for electronics," IEEE Trans.Syst. Sci. Cybern., vol. SSC-5, pp. 91-94, Jan. 1969.

[3] J. R. Miller, III, "A systematic procedure for assessing the worth ofcomplex alternatives," Mitre Corp., Bedford, Mass., underContract AF 19(628)-5165, EDP Equipment Office, ElectronicSystems Div., AF Systems Command, USAF, Tech. Rep. ESD-TR-67-90, AD 662001, Nov. 1967.

The Modeling ProcessG. ARTHUR MIHRAM, MEMBER, IEEE

Abstract-Considerable interest currently exists in the applicationof the systems approach to the solution of societal, political, and environ-mental problems. The essence of this systems approach is modeling, thecapability to describe large-scale complicated interactive systems bysymbolic representations so that inferences regarding the effects ofalternative system configurations can be easily and rapidly structured.The modeling process is itself becoming better understood as a directextension of the scientific method. Furthermore, the applicability ofstatistical methodology to the design and analysis of experiments withcomputerized symbolic models is leading to wider acceptance of theserepresentations as tools of considerably credible scientific stature. Thispaper presents a taxonomy of 24 model categories and, in a discussionof the scientific method and the modeling process, indicates the evaluationspertinent to the selection of a modeling medium appropriate to particularsystems studies. The dynamic stochastic simulation model is shown tobe the most general category of symbolic models which are amenable tofacile organized experimentation. The application of such models to theunderstanding and solution of societal, political, psychological, medical,judicial, environmental, social, economic, and biological problems isindicated and is considered imminently practicable.

INTRODUCTION

A SYSTEM may be defined as a collection of inter-1 dependent elements which act together in a collectiveeffort to achieve some goal. The elements of systems arefrequently referred to as entities, the fundamental com-ponents which interact with one another, positively ornegatively, as the system seeks its goal.

For most systems the goal is self-evident or, at least, canbe described precisely. The maximization of profit, themaximization of productivity, and the optimization ofsystem performance are typical goals of managerial, pro-ductive, industrial, and military systems. However, formany systems goals are not so evident, such as is the casefor biological evolution or ontological development.

Manuscript received June 20, 1971. This work was supported by theNSF under Grant GK-5289.The author is with the Moore School of Electrical Engineering,

University of Pennsylvania, Philadelphia, Pa. 19104.

Nonetheless, whether a system is organized and con-trolled or is self-adaptive and self-regulating, the systemscientist (or operations research specialist, or system en-gineer, or management scientist, or operations analyst, ashe is variously denominated) takes as axiomatic the as-sumptions that system goals can be defined and that systemsare atomistic, capable of being dissociated into their com-ponent entities in such a way that their interactive behaviormechanisms can be described.

A TAXONOMY OF MODELS

In order to describe the phenomena internal to a systemthe systems scientist prepares a model of the system. Thesearch for mathematical laws has been commonplace amongscientists literally for centuries, and the use of replicas byarchitects and engineers has been equally well established.However, only relatively recently has an awareness of theircommon purpose been made manifest.

Rosenblueth and Wiener [27] were apparently among thefirst to note that both the scaled replica and the math-ematical law are examples of models. Their initial categoriza-tion of models prescribed two types, each viewed as an aidto scientific enquiry: 1) material models-transformationsof original physical objects, the representation of a complexsystem by another physical system assumed to be simplerthan, yet similar to, the original system; and 2) formalmodels symbolic assertions in logical terms of an idealized,relatively simple situation, the assertions representing thestructural properties of the original factual system. Thoughno explicit statements of preferability regarding these twomodel categories were forthcoming, it seems apparent that,at the time, greater credibility and a more intricate repre-sentation would be associated with a material model if onewere feasible in a particular instance.Two other categorization schemes are also available in

classifying models. First, a model is said to be dynamic or

621

Documents

Unified Program Planning