IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-6, NO. 11, NOVEMBER 1976

is 2- and is attained only when x = -and y = 2. It followsthat max>x_ ic1(cc,p,d) 0 and the maximum is attainedonly atp1 = p3 =d d3 = 3. Similarly,

max r2(x,p,d) = 09 X Q

and the maximum is attained only when P2 = p4 =d4 = 2. Also, for each k, at Pk-3- and dk = 3, we have

z=2anof 2 1 (i.e., the minimum-supplyconstraint is met). Thus in this example the set 6o ofoptimal points contains exactly one point.Here the regulated multiservice firm is certainly able to

provide just one variant of each service, but only if theprice of each service, as well as the index of quality of eachservice, are chosen to be S, which obviously does notcorrespond to an increased value of the objective function.This example is not inconsistent with H. In fact, assumingthat the multiservice firm provides both variants of eachservice, we have i(p,d) > 0 at Pk 3dk-3 for all k,while 7r(p,d) < 0 when Pk = 1036 for all k and dk = 3 forall k. Therefore, two very direct continuity-type argumentsare all that is needed (for this simple example) to show thatthere is a de 9. such that 3 < dk' < 1.5 for each k,

7(p,d') > 0 when Pk =- for all k, and 7r(p,d') < 0 whenPk = 10-6 for all k, and that there is a p' e ,j) such that10 6 < Pkt < ~-3 for each k, dk'lh(pk1) > -lz(2) for each k,and n(p',d') = 0o16

REFERENCES[1] 1. W. Sandberg, "Some theorems concerning multifirm alternatives

to the single regulated multiservice firm, Part II," ConferenceRecord of the Eighth Asilomar Conference on Circuits, Svstems, andComputers, Pacific Grove, California, pp. 1-17, Dec. 1974.

[2] , "Some theorems concerning multifirm alternatives to thesingle regulated multiservice firm, IEEE Trans. Systems, Man, andCybern., vol. SMC-5, no. 2, pp. 201-208, March 1975.

[3] , "Two theorems on a justification of the multiservice regulatedcompany," The Bell Journal of Economics, vol. 6, no. 1, pp. 346-356, Spring 1975.

[4] 0. L. Mangasarian, Nonlinear Programming. New York:McGraw-Hill, 1969.

[5] G. R. Faulhaber, "'On subsidization: Some observations andtentative conclusions." Proc. of the Conf. Communication PolicyResearch, Office of Telecommunication Policy, Washington, D.C.,Nov. 1972.

[6] E. E. Zajac, "Some preliminary thoughts on subsidization," Proc.of the Conf. on Communication Policy Research, Office of Tele-communication Policy, Washington, D.C., Nov. 1972.

16As a few moments' thought will show, in the example givenabove, the important hypothesis is that the three two-service costfunctions are related as indicated. The specific form chosen for thedemand function, and the fact that (6° contains exactly one point,are not crucial.

Man-Machine Communication: An ExperimentalAnalysis of Heuristic Problem-Solving UnderOn-Line and Batch-Processing Conditions

JAMES V. HANSEN, MEMBER, IEEE

Abstract-Possibly the most revolutionary development to impactupon decisionmaking has been the electronic computer. Its capabilityto process large volumes of data rapidly, to make complex computationsin fractions of seconds, and to provide outputs in a variety of formats hasmarkedly expanded the boundaries of decisionmaking. Notwithstandingsuch progress, the computer's contribution to decisionmaking has thusfar been largely limited to routine and repetitive operations which lendthemselves to the development of standard decisionmaking procedures.This has been an important contribution, yet most decisionmakingsituations, particularly at higher levels of management, are unique andill structured requiring flexible and adaptive responses. Man-machinesystems are thought to offer potential for assisting management inmeeting these requirements. The study reported here examines com-

parative problem-solving performance of individuals communicatingdirectly with the computer (on-line processing) versus individuals com-

municating with the computer in an intermittent fashion (batch processing).The study is marked by an effort to take up the challenge of managementproblems having no set solution procedure. Performance measurementsinclude objective functions, understanding of variable interactions, andcomparative problem-solving patterns.

Manuscript received August 10, 1974; revised June 23, 1976.The author is with the Graduate School of Business, Indiana

University, Bloomington, IN 47401.

INTRODUCTIONA GREAT DEAL has been written about the enhance-

ment of human intelligence and problem-solvingabilities through the use of computers. It is generallyconcluded that the analysis of complex systems and prob-lems can be aided by man-machine systems, whereinman's ability as a heuristic reasoner and his ability to makejudgments based on incomplete data is augmented by thecomputer, which provides the capability needed by man to

ask "what if" types of questions and to see the resultsimmediately. The ability of the computer to make com-

putations rapidly, to provide outputs in a variety of formats,and to provide immediate feedback provides the decision-maker the capability to inquire more deeply into whateverprocesses concern him [3].

Observations of decisionmakers on the job indicate thatthey spend a large portion of their time in verbal com-

munication with others. This is believed to hold true

within the decisionmaking cycle. That is, the process ofdefining and solving problems is often accomplished by

746

HANSEN: MAN-MACHINE COMMUNICATION

"talking" the problem through to solution. The decision-maker has in his mind a set of guidelines, or an informal"model" against which to evaluate the facts he perceivesin a conversation. From this, and other sources, he definesthe problem, i.e., a variance between what his internal"model" says is expected and what he actually perceives.Likewise, he has informal models that he uses to solveproblems as he perceives them. In both instances the problem-finding and problem-solving are intuitive, judgmental typesof processes [9].

It seems desirable to encourage the decisionmaker toexercise this freedom but also to assist him to construct asfirm a base from the facts as possible, and then have himapply his judgment to this base. This may perhaps beaccomplished by providing flexible access to the data andby making available relevant formal models to process andfilter the data for him. Computer technology and modelingtechnology facilitate such developments, and it is likelythat these technologies can be used by normal line managerswith considerable benefit.

However, fundamentals of man-computer communica-tion are only partially understood and are not yet animportant part of research efforts in computer science. Anextensive review of the changing state of the art in man-computer communication was conducted by Ruth Davis[4], yet virtually no experimental studies ofuser performanceare cited in her study. Similar results are reported byBarmack and Sinaiko [2] in their review of man-computercommunications. Their observations led them to call formore experimental evaluation of human performance withcomputer systems. Sackman [11] asserts that this needmay be even more important today as interest grows in theuse of interactive computing systems. This is a clearly aclass of man-computer communication which is attractingan increasing amount of resources and research effortsfrom the technical side; yet there has been little experience,and even less research, into the human-machine systemcombined. This is paradoxical since the essence of inter-active systems is the inclusion of the human element to agreater degree than before. Further, interactive computingfor managers is still somewhat primitive in terms ofmanagerial applications today.'

PRIOR RESEARCHThe first published study of on-line performance versus

batch performance appeared in 1966. Since that time,several other experimental studies of comparative per-formance have appeared in the literature. Erickson [5],Grant and Sackman [7], Smith [13], Schatzoff, Tsao, andWiig [12], Sackman [10], Adams and Cohen [1], andSackman and Gold [6] compared on-line and batch-processing performance with subjects given programmingproblems of varying difficulty to solve. The problems usedrepresented a broad range of problem-solving, but nonerequired large data storage, extensive computation, or long

' While there are some interesting models and applications beingreported, their use by managers is still relatively limited.

involved programs. With the exception of the open-endedproblem utilized in the Sackman and Gold study [6], allproblems were by nature closed, with a single correct answerbeing sought. While variations in research methodologyand computer systems used make comparisons difficult,the aggregate results suggest an advantage for batch-processing with respect to computer time and costs. On-lineprocessing most often demonstrated an advantage in termsof man-hours and also emerged as a "user-preference"leader. While these studies have advanced understandingof comparative man-computer communication, theirresults have largely derived from the analysis of computerprogramming problems.

In managerial situations, problems are often complicatedby the fact that there are not explicit rules for determininga solution. Random search is impractical. Consequently,search techniques can be employed by recognizing patternsof improvements in selections. In addition, as one proceedstoward solving a problem, he learns and becomes moreefficient in his methods. These techniques, referred to asheuristics, are important in understanding man-machineproblem-solving. Vasonyi (14) has suggested that all man-machine problem-solving should be heuristic, as problemsfor which algorithms are available should be solved bymachine alone. It is with the heuristic type of problem-solving under alternative modes of man-machine com-munication that this study is concerned.

HYPOTHESESThere have been relatively few published studies of

comparative on-line/batch-processing problem-solving per-formance. However, what has been reported thus far doescontribute to the development of the hypotheses tested andreported in this paper.For the most part, only crude, rule-of-thumb guidance

exists for the allocation of functions between man and thecomputer, as well as the division of labor between real-timeand non-real-time information processing. Licklider [8]made special note of the fact that in spite of the great volumeof literature on thinking and problem-solving, he couldnot find anything comparable to a time-and-motion studyof human mental processes in a technical task. The increas-ing presence of man-computer communication will continueto generate strong pressures for investigations of the realtime dynamics of work tasks. The focal need in man-computer communication lies in the wider utilization ofexperimental method and in effective dissemination of thefindings [I1].From the genesis of data processing, computers have

been used as task-doers far more often than as problem-solvers. This mode of usage may be of limited help to theperson with a problem which cannot be fully described atthe onset. Moreover, this processing mode is not adaptiveto the problem-solving style of the individual. In responseto the challenge of these limitations, the concept of on-lineproblem-solving has emerged as an alternative to con-ventional off-line computer usage. Under the on-lineapproach, human intelligence and computing machinery

747

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, NOVEMBER 1976

act as elements in a continuous feedback loop. Some

advocates contend that because of its unlimited range of

applicability and its inherent advantages to the users,

on-line problem-solving will in time become the dominant

mode of usage. Sackman [10] has provided a useful synopsis

of the fundamentals of both systems:

The batch or off-line system is the evolutionary pre-

decessor of on-line systems and is the operational workhorseof most contemporary data processing. In batch processing,jobs are stacked and done one at a time on a waiting-linebasis. Programmers typically have only indirect contact

with the computer, and in conventional batch systems

turnaround time (the interval between submission and

completion of a computer job) usually takes hours or days.Proponents of off-line systems argue that throughput time(useful computations per unit time) is at a maximum in

batch processing with minimum waste of computer

resources.

By contrast, under the on-line system the user has directaccess to the central computer; he shares the computer

facility simultaneously with other users who are a more or

less random and changing collection of people at any pointin time; typically, but not necessarily, users work on un-

related tasks with different programs; they enter and leavethe system independently of one another, and use theirterminals for varying and largely unpredictable periods oftime. Such use approaches that of a public utility, roughlyanalogous to the quasi-random pattern of telephone traffic.

Advocates of on-line usage have produced a number ofarguments in favor of that system. Principal among theseare the following.

1) The user has fast and direct access to the computerwhen he wants it.

2) For many varieties ofjobs he can obtain what he wants

in minutes instead of hours or days.3) He can exert continuous control over his program.

4) He has the flexibility to change variables as he interactswith the computer.

5) For many applications, costs are reduced.

Critics of on-line usage are quick to counter with some

perceived disadvantages of such systems. Prominent among

the criticisms are the following.

1) There are higher costs and lower machine efficiencies

under on-line usage, contrasted with impressiveadvances in job shop scheduling, mass servicing of

users, and faster turnaround in the latest batch

systems.2) Users of on-line systems develop poor habits by going

to the computer with poorly designed and untested

programs in an effort to trade computer time for

human time.

This confrontation has important and far-reachingimplications for the future of man-computer problem

solving, especially as the human factor assumes a more

dominant role. Which type of man-computer communica-

tion is better? Why? By how much? In what respects is it

better? The answers to these and many other questionshave major implications for the study of problem-solving,in general, and for the computer industry and those chargedwith decisionmaking responsibility, in particular.An interest in certain of these questions led to the syn-

thesis of the study reported here. Of particular interest wasthe testing of the following three null hypotheses. Givenproblems requiring heuristic methods of analyses:

1) there is no significant difference in the quality ofobjective functions achieved by on-line and batch-processing analysts;

2) there is no significant difference in the ability toaccurately evaluate system interactions by analystsworking under on-line and batch-processingconditions;

3) there are no important differences in the problem-solving patterns of analysts operating in on-line andbatch-processing modes.

RESEARCH METHOD

Problems

Two complex problems were selected for this study.Problem 1 was a stochastic capital-budgeting model.Problem 2 was a product-demand forecasting simulator.Both required heuristic approaches to solution, neitherbeing solvable by mathematical algorithm. Decision rulesand search strategies were implied in order to developimproved solutions and to reduce the scope of search.Feedback from each input-output cycle could suggest newdecision rules, etc. These problems were selected from anumber of problems available based upon their 1) size,2) degree of complexity, and 3) probable nonbias charac-teristics for the participating subjects.

SubjectsAs subjects for the research. ten students were selected

from the introductory computer-based information systemscourse in the Graduate School of Business Administration(GSBA) at the University of Washington. The reasons forselecting this particular group were threefold. First, theirbackgrounds and experience in using computers was roughlyequivalent, as was their knowledge of business operations.Second, there was the opportunity to motivate participationand performance through an arrangement whereby par-ticipation in the study satisfied certain of the course

requirements. Third, this group's knowledge of comnputerstaken toward the end of the term approximated that of theMBA graduate entering into management as a career.

Complete homogeneity of the sample group was, ob-viously, not possible. However, because of the introductorynature of the course, the students' background experiencewith computers was generally equivalent, as was theirknowledge of business operations. It was further assumedthat the admissions process conducted by the GSBAresulted in a graduate student subset of somewhat equalintellectual capacity.

748

HANSEN: MAN-MACHINE COMMUNICATION

ProcedureBoth on-line and batch-processing subjects utilized the

same computer (CDC 6400). Since there were a number ofinterrelated variables whose values influenced the results,an important facet of the study was to test the learningwhich took place over the duration of the experiment asmeasured by demonstrated understanding of those relation-ships. As the number of possible combinations of variablevalues made enumeration impossible, the ability to developimproved answers would depend upon the rapidity andfacility with which efficient decision rules could be developed.Both problems had been used in industry in a similar fashionwith similar objective functions. While the real situation wasnot precisely transferable, it was posited that many of therelevant problem-solving conditions were maintained.The primary differences were lack of familiarity with theparticular industrial setting and less incentive for solvingthe problems.

On-line users interacted directly with the computer usingteletypewriter terminals. Both on-line and batch-processingusers were scheduled to spend a total of six hours on eachproblem over a ten-day period. Batch-processing users

submitted inputs on prepared coding sheets with turn-around being provided within one hour.A Latin-Square, or counterbalanced, research design was

used. This particular approach was chosen as allowing a

reasonably good compromise between the costs andmonitoring difficulties associated with large sample sizesand the need for statistical efficiency. The following criteriawere utilized as measures of comparative on-line/off-lineperformance.

1) Objective Function Values: For Problem 1 the desiredobjective function value was maximum expected value.That value, given the prescribed constraints, was estimatedbased upon a large series of preexperiment test runs. ForProblem 2 the desired objective function value was a

predetermined forecast demand for a specified year.

2) Understanding of Variable Interactions: The correct-ness of subject's rankings of variables in terms of objectivefunction sensitivity was used as a surrogate measure of thedegree to which understanding of variable interactions was

realized.This evaluation was undertaken in recognition of the

fact that, in working with complex problems, decision-makers are concerned not only with objective functionsbut also with improving their understanding of the inter-actions within those systems comprising the problem context.In practice, the latter knowledge can be of critical import-ance. Consider a capital budgeting problem under un-

certainty such as represented by Problem 1. An effectivemanager is going to be interested in the expected presentvalue resulting from a given set on inputs, but he will beeven more interested in learning whether modifications tovariables which are under his control can significantlyeffect the resulting expected value. That is, he may discoverthat changes to system variables, individually or in variouscombinations, may have significant or negligible effects on

expected present value. This information can be of marked

value in identifying areas which merit special examination.For example, it may be found that increasing market shareby a small percentage significantly improves the expectedpresent value. This, in turn, may prompt an evaluation ofvarious alternative strategies of applying resources whichmay be expected to increase market share.

It may be noted that the efficiency of this approach isdependent on the validity of the structure of variablerelationships built into the model of the problem. Here,again, the decisionmaker can apply his judgment and priorknowledge in establishing the proper relationships to beprogrammed for any problem variable. He may learn fromtrying various configurations of relationships that some ofhis notions about problem interactions do not hold. Furtherinvestigations prompted by this type of discovery may bevery useful to the decisionmaker. Continuing with theexample above, the decisionmaker may find that increasingmarket share even a small percentage will require anincremental expenditure of funds exceeding the resultantincrease in expected present value. He might thus decideto forego efforts to increase market share. Alternatively,it may be discovered that increasing market share can beaccomplished through, let us say, increasing advertisingexpenditures, but that the incremental expenditure willrequire funds which had been earmarked for warehouseexpansion. He may then wish to evaluate the cost-effective-ness of such a tradeoff. The benefit is that attention isfocused on the more important system variables as wellas those tradeoffs which merit consideration. The processof rational decisionmaking may be correspondingly en-hanced because of an increased knowledge of the con-sequences of various courses of action and the value whichshould be attached to those consequences.

In an effort to derive a performance measure for thiscriterion, subjects were tested at the end of each problem-solving segment on their abilities to correctly rank problemvariables in order of sensitivity. Concerning the method ofscoring, it was recognized that an erroneous switch of, let ussay, the first and sixth ranking variables could be a moresignificant error than a switch of, let us say, the first andsecond ranking variables and that the method of scoringshould account for this. The method measurement thereforeattempted to account for the distance of each ranking error.Each variable which was erroneously ranked was assigneda penalty of one unit for every position it was removedfrom its proper ranking. Using the above example, theerroneous switching of the first and second ranking variableswould be assigned a penalty of two units. That is, eachvariable is misplaced by one position. Correspondingly,the erroneous switching of the first and sixth rankingvariables would receive a penalty of ten units, both variablesbeing misplaced by five positions. Of course, not all errorswere the result of one-for-one switches, but the logic holdsfor any case.

3) Problem-Solving Patterns: Subjects were required torecord their daily problem-solving activity in terms of inputstrategies, underlying rationale for same, and results. Theobjective of recording this type of information was to

749

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, NOVEMBER 1976

-ANALYSIS OF VARIANCE

Batch On-LineSubject Score Subject Score

Problem I Si 86 S6 95S2 79 S7 86S3 79 S8 92S4 85 S9 84S5 81 S1O 85

On-Line BatchSubject Score Subject Score

Problem 2 Si 85 S6 78S2 95 s7 84s3 97 S8 84S4 92 S9 90S5 84 S5O 87

Analysis of Variance

Source SS DF MS F Value

A (Between Groups) 0.2 1 0.2 0.009B (Between Problems) 28.8 1 28.8 1.3C (Between Computer Modes) 192.2 1 192.2 8.6Residual 0 0 0 0Within Cell 353.6 16 22.1

The F Value statistic must have a value of 4.49 at the 0.5 level and8.53 at the 0.1 level. Residual sum of squares was actually a verysmall number (0.0019). This may have been a true value or the resultof a truncation error by the computer.

enable tracking of the problem-solving patterns demon-strated by on-line and batch-processing subjects. It was

intended that such information should facilitate identifica-tion of location and magnitude of pattern differences.

RESULTS

Hypothesis 1

The first hypothesis was that there is no significantdifference in the quality of the achieved objective functions.Individual objective function results are shown in Table I.

All scores were normalized. Analysis of variance showsscores achieved by on-line subjects as being significantlyhigher (0.01 level) than those achieved by batch-processingsubjects.

Hypothesis 2

The second hypothesis was that there is no significantdifference in the ability to accurately evaluate systeminteractions. The resulting error scores were analyzedusing the Wilcoxen rank-sum test. Table II shows that theerror scores realized by on-line subjects are significantlyless (0.028 level) than those achieved by batch-processingsubjects. It is noted that by using rank-sum analysis it wasnot possible to partition the results by subjects, problems,or computer modes.

Hypothesis 3

The third hypothesis was that there are no importantdifferences in the problem-solving patterns. It was deter-mined that there were at least four types of problem-solvingactivity which could be identified. In all cases there was an

initial search activity (xl). That is, one or more inputs were

devoted to random variable changes in order to observe

TABLE IICOMPARATIVE UNDERSTANDING OF VARIABLE INTERACTIONS-

RANK-SuM ANALYSIS

Batch On-LineRanking RankingError Error

Subject Scores Subject ScoresProblem 1 S1 9 S6 2

S2 8 S7 4S3 4 S8 4S4 6 S9 2S5 6 sIo 4

On-Line BatchRanking RankingError Error

Subject Scores Subject ScoresProblem 2 S1 2 S6 4

S2 4 S7 4S3 2 S8 4S4 2 S9 6S5 2 S10 4

Rank-Sum Test

Problem 1 Problem 2

Scores in ascending order 2, 2, 4, 4, 4, 4, 2, 2, 2, 2, 4, 4, 4,(Entries from batch group 6, 6, 8, 9 4, 4, 6are in italics)Scores replaced by rankings 1, 2, 4.5, 4.5, 4.5, 1, 2, 3, 4, 7, 7,

4.5, 7, 8, 9, 10 7, 7, 7, 10Sum of italicized Sum of italicizedranks = R-38.5 ranks = R = 38.0

The probability of R > 37 for n1 = 5, n2 = 5 is 0.028.

the results and to begin construction of some sort of strategy.A second type of activity involved sensitivity testing (x2).For example, a subject's search activity might suggestthat the objective function is more sensitive to changes inone variable as opposed to another. A specific test mightinvolve increasing the former variable to its maximumallowable value in order to measure the impact of the changeon the objective function. A third category of activity wasidentified as testing of changes in two or more variables atonce (x3). This approach was generally used to test fornonlinearity. Most often a few variables were identifiedwhich, with all others held constant, significantly affectedthe value of the objective function. These "more important"variables were often tested with various combinations ofvalues in order to identify possible nonlinear relationships.Through activities xl, x2, and x3, subjects typically selectedseveral variables on which to concentrate in attempting toimprove the objective function. The final class of problem-solving activity was that of applying a specific strategy toimprove the objective function (X4). A subject might, forexample, search and test several variables, and with theresulting information devise a specific improvementstrategy.A multiple-regression analysis of these activities was

constructed with the expected objective-function scorehypothesized as a function of the number of search,sensitivity-testing, multiple-variable testing, and improve-ment-strategy activities, as follows:

objective function score

= + A1X1 + 2X2 + A 3X3 + / 4X4 + E. (1)

TABLE ICOMPARATIVE OBJECTIVE FUNCTION VALUES-

750

HANSEN: MAN-MACHINE COMMUNICATION

Scores and activities were grouped by on-line and batch-processing results, and regression analysis was performedon each. The regression of on-line scores produced thefollowing model:

objective function score (on-line)

= -25.87 + 9.76x1 + 4.21x2 + 0.54x3 + 8.57x4

R2 = 0.83. (2)

Regression analysis of variance determined that for an arisk of 0.05 the above equation was a reasonably goodpredictor of the objective function score-the calculatedF = 6.26 for the regression being greater than the tabulatedF(4,5,0.95) = 5.19.For batch-processing scores, the model below was

computed:

objective function score (batch)

= -6.19 + 0.75x1 + 4.34x2 + 2.78x3 + 3.81x4

= 2 = 0.94. (3)

Analysis of variance shows that for a = 0.05 this equationis also a reasonably good predictive model-the calculatedF = 8.94 for the regression again being greater than thetabulated F(4,5,0.95) = 5.19.

DISCUSSIONAnalysis of the results of the study indicate significant

differences in the objective function scores which wereachieved on the two problems by on-line and batch-processing subjects. Interviews with participants followingthe exercise tended to support the desired absence ofpeculiarities associated with either problem, which made iteasier or harder to solve or which biased the problem infavor of either computer mode. No subject perceived biasin either problem.

Analysis of comparative understanding of variableinteractions demonstrated significant differences in theability to correctly rank variables according to their ob-jective-function sensitivity. It is likely that in actual practicea decisionmaker would assign differing degrees of im-portance to proper sensitivity identification of the respectivevariables. However, for purposes of this study, the onlyplausible approach was to treat all variables as if theirproper placement was equally important.

Multiple regression analysis of problem-solving patternsproduced the models shown as (2) and (3). It is observedthat over the range of scores produced, search, strategy-improvement, and sensitivity-testing activities provided thelargest contribution to the objective function score for on-line problem-solving, with the highest weights assigned tosearch and strategy improvement activities. Multiple-variabletesting contributed relatively little.For batch-process problem-solving, sensitivity-testing,

multiple-variable testing, and strategy-improvement activitiesmade the largest contribution, with search activity con-tributing only slightly.

SUMMARYThe thrust of this study was to explore on-line and batch-

process problem-solving, utilizing problems which were ofsufficient size and complexity as to require heuristic methodsof solution. This represents a contrast to much of the priorresearch on man-computer problem-solving which has beenprimarily concerned with small problems amenable toalgorithmic solution. There is little question but that themost important and interesting management problems areof the former type. The computer has greatly enhanced ourcapability to examine these types of problems, and it is ofincreasing importance to understand the ways in whichthe computer can be most effectively combined with humanintelligence to improve problem-solving capability.

It has been argued that working off-line forces the user tomore carefully plan his input and to utilize less computertime in superficial search activity. Given the type of problempresented in this study, it appears that some major ad-vantages of working on-line are the ability to quicklycomplete initial search activity and to engage thereafterin a broader range of strategy testing and sensitivityanalysis. With large, complex problems it maybe impossibleto do a thorough job of off-line analysis and strategydevelopment until the problem has been reduced to manage-able size by finding the most sensitive variable combinations,discovering nonlinearities, etc.

While these may seem to be common-sense conclusions,the author has found that in numerous organizations,where models are used by decisionmakers for planning andcontrol purposes, the mode of interface is primarily afunction of what systems are available and the manner inwhich they have traditionally used-not as a function ofwhether or not the model involves heuristic methods ofanalysis. In the past, much of the engineering concernedwith people in systems has been accomplished by trialand error, or by common sense. The decisionmaker has beenleft to adjust to inadequacies as best he could. Indeed, he hastypically adjusted well, but frequently at a cost of poorsystem performance. It has thus become important to takeaccount of the man-machine interaction in the designprocess in such a way that the consequences can be predictedin terms of system performance. This, in turn, requires thatthe designer be able to predict the performance of thedecisionmaker as a component of the man-machine system.Two important limitations on this study should be ad-

dressed. First, how far can we generalize from the perform-ance of a laboratory group? This study could have beenconducted using actual decisionmakers on the job. How-ever, the problem of biases due to differences in 1) companyexperience, 2) education, and 3) exposure to computermodels was overwhelming. The laboratory experimentseems to be the necessary place for developing concepts tobe used as a basis for design testing in the field.

Second, where do costs come into the picture? Thisstudy has only examined, but not measured, possiblebenefits. Ideally, an evaluation of the effectiveness of asystem ought to include some consideration of costs andbenefits. Costs were, in fact, accumulated for both types ofusers and were not found to be significantly different

751

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, NOVEMBER 1976

(0.05 level, applying analysis of variance). However, systemconfigurations and chargeout schemes vary so widely thatsuch information must be regarded very tentatively. Thebenefits in this instance were of the intangible class, themeasurement of which was regarded as being in need of agreat deal of research itself.A concluding observation is made that it is not necessary

to predict accurately and in detail in order to be useful.Man-machine research may be effective if it serves only tohelp the designer to organize his thinking about howdecisionmakers perform, to enable him to distinguishthose variables which are likely to be important, and todesign ad hoc experiments to answer specific questions.

REFERENCES[1] J. Adams and L. Cohen, "Time sharing versus batch processing,"

Computers and Automation, vol. 18, pp. 30-34, March 1969.[21 J. E. Barmack and H. W. Sinaiko, Human Factors Problems in

Computer Generated Graphics Display, Study S-234. Institutefor Defense Analysis, Washington, D.C., 1966.

[3] G. B. Davis, Computer Data Processing. New York: McGraw-Hill, 1970.

[4] R. M. Davis, Annual Review of Information Science and Tech-nology, Vol. 1. New York: John Wiley & Sons, 1966.

[5] W. J. Erickson, "A pilot study of interactive versus non-inter-active debugging," TM-3296, Santa Monica, CA, System Develop-ment Corp., 1966.

[6] M. M. Gold and H. Sackman, "Time sharing versus batch pro-cessing: An experimental inquiry into human problem solving,"Santa Monica, CA, System Development Corp., 1968.

[7] E. E. Grant and H. Sackman, "An exploratory investigation ofprogrammer performance under online and offline conditions,"IEEE Trans. Human Factors in Electronics, vol. HFE-8, pp.33-48, March 1967.

[8] J. C. R. Licklider, "Man-computer symbiosis," IRE Trans.Human Factors in Electronics, vol. HFE-1, pp. 4-11, March 1960.

[9] M. S. S. Morton, Management Decision Systems. Boston:Harvard University, 1971.

[10] H. Sackman, Man-Computer Problem Solving. Princeton:Auerbach, 1970.

ti 1] ~~,Computers, Systems Science and Evolving Society. NewYork: John Wiley & Sons, 1967.

[12] M. Schatzoff, R. Tsao, and R. Wiig, "An experimental com-parison of time sharing and batch processing," Communicationsof the ACM, vol. 10, pp. 261-272, May 1967.

[13] L. B. Smith, "A comparison of batch processing and instantturnaround," Communications of the ACM, vol. 10, pp. 495-500,August 1967.

[14] A. Vasony, "Automated information systems in planning, controland command," Management Science, vol. 11, pp. B-2-B-41,Feb. 1965.

Correspondence-Control of Health-Care Quality

J. H. U. BROWN, SENIOR MEMBER, IEEE

Abstract-Control theory applies to the health-care system in much

the same way that it applies in physical systems. The major problemhas been the failure of the health-care system to determine the "set

point" of desirable health care and to provide the proper feedback to

regulate the system. A feedback system for the health-care operationmay involve a floating "set point" and process control. The determination

of the parameters is a matter of political and consumer input, which to

date has not served to close the control loop.

The theory of control in a social system is in many respectssimilar to the theory of biological control mechanisms in nature.

Control in nature depends upon feedback mechanisms [1].Negative feedback promotes stability. Positive feedback creates

instability and failure to achieve a "set point." If the time delayis incorrect, inadequate control is obtained and wide fluctuations

occur. (Several papers have dealt with specific examples of the

control process [2]-[4] in health-care delivery.)The production of poor batches of polio vaccine a few years

ago is a classic example of poor feedback [5]. A sample control

system may be diagrammed as shown in Fig. 1. Too stringentcontrol on the production would have produced very little

vaccine, while a less stringent control resulted in bad batches

Manuscript received April 8, 1976; revised June 21, 1976.The author is with the Southwest Research Consortium, San Antonio,

TX 78284.

which continued to be made for some time. The "set point"had not been accurately determined, and the time delay was

incorrectly judged. The paralytic doses were finally detected,only after a new feedback system, the clinical response, began to

function. The inaccurate laboratory assay used as feedbackwould have continued to produce more and more vaccine. Theloop in the feedback mechanism had been opened so that theinputs and the responses were decoupled.

In any control system, the operator must receive two inputs,the desired value and the actual value. He must compare thesetwo and determine the correction to make the real value approachthe desired level. The health-care system has no comparison ofwhat constitutes "good" health care, and this has created seriousproblems in health-care delivery [6].

Control through feedback mechanisms can lead to severe

oscillations if the control is too tight. For example, a controlsystem regulating quality of milk might result in high pricesand little available milk, or low prices for milk of low qualitydepending on the "set point" of acceptable bacterial count.

We are faced with a similar problem at the moment in the case of

polio vaccine where relaxation of immunization procedures anddecreased advertising after a very thorough campaign hasresulted in the reoccurrence of the disease.

Control systems in societies have not been highly successfulbecause they have not been developed with functional feedback,and the delays have been either too long or too short. An

understanding of the feedback mechanism is essential in develop-ing a working regulatory process in society as well as in nature.

In a very real sense the consumer of goods or services regulatesthe product by acceptance of a standard quality or a price [9].

752