An Experimental Validation of the Gorry and Scott Morton ... · Introduction As a relatively new field of applied research, in-formation systems (IS) displays a considerable ... 184

Validation of the Gorry-Scott Morton Framework

An ExperimentalValidation of theGorry and ScottMorton Framework

By: Peeter J. KirsDepartment of Decision Sciences

and Information SystemsFlorida International UniversityMiami, Florida 33199

G. Lawrence SandersDepartment of Management

Science and SystemsState University of New York

at BuffaloBuffalo, New York 14260

Robert P. CervenyDepartment of Management

Science and SystemsState University of New York

at BuffaloBuffalo, New York 14260

Daniel RobeyDepartment of Decision Sciences

and Information SystemsFlorida International UniversityMiami, Florida 33199

.AbstractThe role of frameworks in information systemshas recently received a great deal of critical at-tention. One prominent indictment, which hasbeen directed at even commonly acceptedframeworks, is that they lack empirical support,and in fact are not constructed in operationalterminology. This article reports the results ofan experimental lab study using MBA studentsas subjects to investigate the tenets of the Gorryand Scott Morton framework (Gorry and ScottMorton, 1971). While firm support is found for

the assumption that the level of information at-tributes varies across system type in the direc-tion postulated, there is evidence that the abil-ity to differentiate the component attributes isaffected by such factors as field dependencyand mode of presentation.

Keywords: Information systems frameworks, in-formation attributes, system classifi-cation, information requirements,system evaluation

ACM Categories: H.1.1

IntroductionAs a relatively new field of applied research, in-formation systems (IS) displays a considerableamount of self-consciousness about its origins,current status, and future (Culnan, 1986; 1987;Culnan and Swanson, 1986). Lacking an ac-cepted paradigm like management science, or-ganization theory, or computer science, IS re-searchers are guided by frameworks derivedfrom related disciplines and applied to IS prob-lems. These "conceptual frames of reference"represent attempts to differentiate theoretical ele-ments, postulate their interrelationships, and in-tegrate them into descriptive or predictivemodels. In this way, they can provide a struc-ture for the generation of testable hypotheses.Yet, IS frameworks have recently been criticizedon a number of levels, most notably becausethey do not clarify their relationship with the in-tended reference discipline, they poorly definethe dependent variables involved, and they lacka cumulative tradition (Keen, 1980).

Common IS frameworks include those proposedby Blumenthal (1969), Chervany, et al. (1971),Gorry and Scott Morton (1971), Ives, et al.(1980), Lucas (1973), Mason and Mitroff (1973),Nolan and Wetherbe (1980), and Sprague(1980). These frameworks were intended guide research efforts, allow better communica-tion among scholars with common interests, andenable practitioners to make sense of researchresults in the absence of an accepted scientificbody of knowledge (Naumann, 1986). To someextent these purposes have been accomplishedsince a community of scholars has grown uparound these frameworks and uses them in itswork. It appears further that many researchershave accepted them as conclusive; a briefreview of IS citation analyses (see Culnan, 1986;

MIS Quarterly/June 1989 183

Validation of the Gorty-Scott Morton Framework

1987; Hamilton and Ives, 1982; Van Over andNelson, 1986) indicates that the articles in whichthe frameworks first appeared are among themost cited in the field.

One disadvantage of frameworks in IS stemsfrom their relative ease of construction. Notbound to a single theoretical approach, they areboth easily generated and readily discarded andforgotten. Since they are not necessarily requiredto meet any minimum standard of theoretical con-sistency, trivial conceptual modifications areoften heralded as novel and definitive. The con-sequences are failure to build cumulative re-search findings, failure to establish an enduringscientific presence for IS in the academic com-munity, and promotion of disciplinary alienationfor IS researchers.

To guard against the tendency to build dispos-able frameworks, researchers should draw ideasfrom strong areas of theory in related disciplinesand propose classifications that can be opera-tionally ordered and empirically substantiated.Only then does it make sense to pursue re-search on the relationships suggested by themodel. Unfortunately, untested frameworks arefrequently misapplied and used to guide empiri-cal studies, the results of which are often incon-clusive. In the area of user involvement, for ex-ample, such casual approaches to empiricalresearch have led to mixed, or contradictory, find-ings on the relationship between user involve-ment and system success (Ives and OIson,1984).

As a consequence, closer examination of con-textual and structural variables and their impacton system success, as well as organizationalframeworks in general, have been advocated(Cerveny and Sanders, 1986). Construction more durable and applicable frameworks re-quires that the theoretical categories specifiedby the framework be validated empirically.

The purpose of this article is to experimentallyinvestigate the framework proposed by Gorryand Scott Morton (1971), which describes thecharacteristics of the information required forthree different types of managerial activities. Areview of that framework’s components and areordering of the relationships in operational-izable terminology are presented. Then, an ex-periment is described that uses typical problemoutlines (scenarios) associated with the mana-gerial functions in Gorry and Scott Morton’s frame-work. Each scenario incorporates varying de-

grees of the seven informational attributesoriginally postulated by Gorry and Scott Morton.Our approach is to determine the extent to whichsubjects can differentiate the component infor-mational attributes specified by the frameworkacross these scenarios and how readily theseattributes can be associated with accepted IScategories.

The Gorry and Scott MortonFrameworkA common practice in IS research is to treat in-formation systems themselves as either a de-pendent variable or an independent variable(Markus and Robey, 1988). Accordingly, IS frame-works usually attempt to classify information sys-tems in one of two ways. First, systems can beclassified based on technical attributes. For ex-ample, Bakopoulos (1985) characterizes infor-mation technology in terms of its capacity, qual-ity, cost, storage, processing, and communica-tions capabilities. It is also possible to classifycomputing arrangements as interactive versusbatch (Keen, 1976), standalone versus net-worked, and so on. The second approach is tofocus on the functions information systems per-form within their context of use (Alter, 1980) andwhose interests are served by information tech-nology (Dutton, 1982; Jacques and Brown,1976). For example, Markus (1984) identifies fivetypes of information systems, each describinga dominant type of function: operations, com-munication, planning and decision making, moni-toring and control, and interorganizational trans-actions. The Gorry and Scott Morton frameworkalso builds its classification of information sys-tems upon functional differences rather than tech-nical attributes.

The Gorry and Scott Morton framework is per-haps the best known, most durable, and mostfrequently cited in the IS field. It is essentiallya mapping of Anthony’s (1965) hierarchy of man-agement functions across Simon’s (1960) clas-sification of decision making. Anthony identifiesthree managerial functions closely associatedwith different levels of the typical organizationalhierarchy: operational control, management con-trol, and strategic planning. Gorry and ScottMorton propose seven characteristics of infor-mation that vary among Anthony’s types of de-cision categories: source, scope, level of aggre-gation, time horizon, currency, required accu-

184 MIS Quarterly~June 1989


racy, and frequency of use. Recognizing Simon’sconcern about the degree of structure in deci-sion making, Gorry and Scott Morton argue thatunstructured decisions require different informa-tion than structured decisions.

The resultant framework relates functional ac-tivities in an organization with the requirementsof the information needed to support them. Al-though it does not attempt to associate a de-fined set of information with any particular activ-ity or IS type, the framework nonetheless impliesthat such relationships could exist, as Figure 1indicates.

The Gorry and Scott Morton framework hasbeen used as the basis of other classificationschemas. For example, Zmud’s (1983) organi-zationally based IS typology is used to associ-ate IS types with common organizational func-tions. The typology includes:

1. Transaction Processing Systems (TPS).This system supports procedural activitiesthat.do not require choices between alterna-tive actions. The goal is to process routineorganizational transactions in an efficient, cost-effective manner.

2. Information Reporting System (IRS). IRS specifies organizational directions and re-directions, its focus is not on situation-specific activities, but rather on controlling be-haviors by providing cues, directing attention,reviewing past performance, and raisingquestions.

3. Decision Support System (DSS). Thissystem type is characterized as supportingthose decision-making activities that are ill-structured and situation-specific, and involvechoosing from a number of alternatives.

By associating IS types with managerial func-tions, the framework illustrates variations in thestructural complexity of decisions made. Trans-action processing systems (TPS) are used forrecurring, programmable decisions. At the otherextreme, decision support systems (DSS) arebetter suited for non-programmable decisions.Information reporting systems (IRS) fall some-where between the two extremes, focusing onexceptions to routine performance. Each typeof IS implies the need for different levels of eachinformation attribute that corresponds to differ-ences in the nature of decisions made in anorganization.

Despite its popularity, the Gorry and ScottMorton framework has never been carefully op-erationalized (Naumann, 1986). The categoriesof managerial activities serve as "summative"units that draw together several properties of de-cision making and information, giving them a com-posite label (Dubin, 1969). However, reliance composite concepts precludes the precise defi-nition of either the components of managerialactivity or information requirements. Thus, whilethe framework is interesting, enormously popu-lar, and intuitively appealing and acceptable, itscurrent formulation falls short of being a vali-dated empirical framework useful for guiding re-seamh on information systems.

Research ObjectiveOne step toward a more precise definition of com-ponents is to differentiate the summative cate-gories proposed in the original Gorry and ScottMorton framework. Each of the seven informa-tion attributes can be readily delineated alongan ordinal scale, rendering it suitable for inclu-.sion on a questionnaire. In any given problemsetting, either real or simulated, an assessmentof the level of that attribute deemed appropriatefor a given problem setting should be obtain-able. If the framework is valid, the componentproperties of different types of information sys-tems should be distinguishable from one sce-nario to the next.

The objective of this research, therefore, is toinvestigate the following null hypothesis:

H0~: The information attributes Inferred byeach scenario will be the same acrossscenario types.

The rejection of H10 implies that information at-tributes perceived as desirable in an informationsystem vary across the problem situations thatGorry and Scott.Morton define in theoreticalterms. Problems characterized by greater uncer-tainty and higher levels of managerial activityshould require information attributes character-ized by the dght side of the continua in Figure1. Conversely, problems characterized by loweruncertainty and lower levels of managerial ac-tivity should require information attributes char-acterized by the left side of the continua inFigure 1. If statistical tests reject the null hypothe-sis stated above, and if information attributesvary predictably with decision type, the Gorryand Scott Morton framework will be supported.

MIS Quarterly~June 1989 185


Cells contain typical examples as given by Gorry and Scott Morton

System Classes

TPS IRS DSS InformationalInternal ¯ ¯ External Dimensions

Historical ,~ ~ Future I, Input AttributesCurrent ,~ ~, Old

In.

Standard ,, ~, TailoredFrequent ¯ ~ Infreq.

Throughput Attributes

-- "-D’~t~l~-d-- ~, ¯ AggregateNarrow ,, ,, Very Wide

I

~) o)l_ml

~lc~ I~-I I

I-JI

High ¯ ,, LOW IAccounts I I Budget I TankerReceivable I Analysis I Fleet Mix- . - -I ...... I-- -~rg’;r~’~-Production I Variance IScheduling I Analysis 1 Acquisitions

I -tPERT/Cost I Sales & I R & DSystems I Production I Planning

I IOperational Management Strategic

Control Control Planning

Output Attributes

Decision Class

Structured

Semi-Structured

Unstructured

Managerial Function

Adapted with permission from: Gorry, G.A. and Scott Morton, M.S. "A Framework for ManagementInformation Systems," Sloan Management Review, Fall 1971, pp. 55-70.

Figure 1. Information Requirements by Functional Activity

Method

Scenario development and problem-setting presentationThe use of scenarios, or abbreviated stereotypi-cal problem outlines, has been advocated forresearch of this type because scenarios illumi-nate the interaction of multiple variables, sim-plify the model of the organization or system,aid in the consideration of alternative outcomes,and refine the scope of a study’s.research ob-jectives (Kahn and Weiner, 1977). Using sce-narios also reduces the time necessary for datacollection and increases experimental control. An-other advantage in using this medium is that sce-narios can be constructed to describe a com-plex problem in feasible, understandable termswhile still allowing for multiple solutions. Scenar-ios have been used in IS research (Alavi, 1984;Courtney, et alo, 1983; Doktor and Hamilton,

1973; Henderson and Nutt, 1980; Kirs, 1987),but have not specifically been applied to the in-vestigation of the Gorry and Scott Mortonframework.

The scenarios used in this study were purposelydeveloped in a multi-phase process. The objec-tive of the initial stage was to broadly capturethe information attributes of the Gorry and ScottMorton framework and the characteristics ofZmud’s information system types. Initial scenariovalidation consisted of submitting the scenariosto five IS researchers (faculty members) and twoexperienced practitioners (a system analyst anda consultant), all of whom were well acquaintedwith a variety of IS classifications. Through aquestionnaire, they were asked to categorize thescenarios according to one of three IS classes,respond to the information attributes and char-acteristics of each system, and comment on thescenario’s format and content. Their suggestionswere consolidated and used to modify the con-



tent of the outlines that were then given to alarger (N =44) sample of MBA students (whohad limited familiarity with IS classifit:ationschemes) for structured evaluation. Based ontheir responses and comments, the scenarioswere further refined into their final state. The re-sultant scenarios consisted of:

1. A TPS scenario, in which the manager wasresponsible for development of an online in-formation system to monitor inventorytransactions,

2. An IRS scenario, where the manager was re-quired to develop an information system thatcould produce periodic, on-demand, and ex-ception reports to aid in the planning and con-trol of inventory.

3. A DSS scenario, in which the manager wasrequired to develop an information systemthat could be used to forecast demand andprofits for items in inventory.

Although each scenario varies according to de-cision requirements, all are similar in terms ofdecision-maker title (general manager) industryand environment (retail tire sales; mature andstable environment) and general task (the needto develop the information system). The scenar-ios are described in detail in the Appendix.

SampleTwo hundred and twenty three (223) subjectsenrolled in an MBA program were asked to readeither one, two, or three scenarios in order totest for presentation and order effects (Campbelland Stanley, 1963). The subjects were askedto envision themselves as the manager facedwith the given problem(s). In each case it wasemphasized that they were not responsible forthe actual system design but were to ensure thatthe system developed contained the informationrequired for their task.

Although there has been some controversy re-garding the use of MBA students as managerialsurrogates (see Remus and Kotteman, 1987),research contrasting the decisions of the twogroups in experimental settings has generallyfailed to find any significant differences (Remus,1986). Additionally, this study was intended toinvestigate whether the information attributes sug-gested in the Gorry and Scott Morton frameworkcould be recognized by subjects irrespective oftheir level of sophistication. The subjects for this

experiment spanned the spectrum of experience.It was therefore felt that for the purposes of thisexperiment the findings could lead to somegeneralizations.

Of the 223 subjects, 72 were given only onescenario, 54 received two senarios, and 97received all three scenarios. In terms of thescenario types distributed, 153 subjectsevaluated the TPS scenario, 158 evaluated theIRS scenario, and 160 evaluated the DSSscenario.

MeasuresAfter reading each scenario, subjects wereasked to rate the information attributes requiredfor an effective managerial information systemto deal with the problems described in the sce-nario(s). Seven-point Likert-type scales were con-structed, anchored at each end by the bipolaradjectives used to characterize the informationdimensions (see Figure 1). In order to assist thesubjects in understanding the questions,-eachscale was preceded by a shod, written illustra-tion. One final question asked the subject toselect, from a textbook definition of each IS type,the type of system that came closest to describ-ing the system called for by the scenario.

Experimental designIn any experimental research, the conditions po-tentially affecting results should be identified andcontrolled where possible. One question ad-dressed in this experiment was whether directcomparisons between scenarios influenced per-ceptions of the level of information necessaryfor a particular task. In other words, are percep-tions of required information field-dependent? Asecond question relates to the potential effectof order of scenario presentation on require-ments assessment. Will subjects’ ratings differdepending on the sequence of presentationwhen multiple scenarios are presented? Whateffects might increasing decision complexity, de-creasing sequence, or random ordering have?

The hypotheses pertaining to these two ques-tions, stated in null form, are:

H0~Regardless of the number of scenariosgiven (one, two, or three) the informationattributes Inferred by each of the scenar-ios will be the same across scenarios.



H~: For subjects receiving either two or threescenarios, regardless of the order or pres-entation, the information attributes in-ferred by each of the scenarios will bethe same across scenarios.

In order to test these hypotheses and the mainhypothesis, H~), the experimental design con-sisted of three treatment groups and a total of15 different scenario presentation sequences:

1. Single Scenario. Subjects in this group weregiven only one of the scenarios describedabove (TPS, IRS or DSS).

2. Mixed Scenarios. Two scenarios were givento subjects in this group. Since there werethree distinct scenarios, there were a total ofsix different combinations: (1) TPS and IRS,(2) TPS and DSS, (3) IRS and TPS, (4) and DSS, (5) DSS and TPS, or (6) DSS IRS.

3. All Scenarios. Subjects in this group receivedall of the problem outlines, but in one of sixpossible sequences: (1) TPS, IRS and DSS,(2) TPS, DSS and IRS, (3) IRS, TPS DSS, (4) IRS, DSS and TPS, (5) DSS, and IRS, or (6) DSS, IRS and TPS.

Hypotheses H20 and H~ resulted from examin-ing differences across scenario types. It was alsonecessary to examine the impact of treatmentgroups and scenario sequences within the samescenario type. That is, will the number of sce-.narios presented affect perceptions within thesame scenario type? Does the sequence of pre-sentation affect assessments within scenariotype? In null form, hypotheses addressing theseconcerns are:

HI~:The information attributes inferred byeach scenario type will be the sameacross the entire sample regardless ofthe number of scenarios presented.

H05:The information attributes inferred byeach scenario type will be the sameacross treatment groups regardless ofthe order in which the scenarios arepresented.

H~ is concerned with the robustness of the sce-narios and whether evaluation of required infor~mation attributes can be made in isolation (i.e.,whether field dependency and information suffi-ciency affected evaluation). It was assumed, forexample, that subjects who were asked to evalu-

ate the information dimensions required of theTPS scenario would perceive the attributes asbeing the same, regardless of whether they weregiven only the TPS scenario or were given theTPS scenario along with one or both of the otherscenario types.

Hypothesis H50 is intended to examine the issueof order effects. Within the multiple scenariogroups (two or three scenarios evaluated), it wasassumed that subjects would rate the requiredinformation dimensions for each scenario thesame, whether that scenario was the first onein the sequence or one of the later ones. In otherwords, subjects rating the information attributesassociated with a DSS should perceive the samelevel of each attribute whether the scenario waspresented first or last in the sequence.

ResultsIn order to test the first hypothesis -- whetherthe information attributes varied across the sce-narios u a multivariate analysis of variance(MANOVA) was performed to examine whetherthe interrelated dependent variables (responsesto the informational dimension questions) dif-fered among the treatments (scenario type ad-ministered). As shown in Table la, Wilkes’ like-lihood ratio criteria (Wilkes, 1932) stronglysupport the assumption that information attrib-utes inferre,~ by each scenario do vary acrossIS types (H~).

To further examine the effect of scenario typeon each of the seven questionnaire scales, indi-vidual analyses of variance (ANOVA) tests wereperformed. T-tests for independent sampleswere then used to search for differences be-tween any two scenarios (TPS vs. IRS, TPS VSoDSS, and IRS vs. DSS). As shown in Table la,for the entire sample the scenario types wereviewed as different across each of the meas-ures. Although the F-statistics vaded across ques-tions, each was significant at p<0.01o Whenpairwise differences were examined (Table lb),all ratings were in the order postulated by Gorryand Scott Morton. That is, for each informationattribute dimension the mean ratings for the TPSscenario were less than those for the IRS sce-nario, which in turn were less than those for theDSS scenario. With .the exception of the differ-ence between the TPS and IRS scenarios onone information attribute (time horizon), whichwas significant at p<0.05, all were different atp<0.01.

188 MIS Quarter/y/June 1989


Table la. Differences Across Scenarios for Full Sample

F-Statistics’

MANOVA=4.74

TreatJAttrib. df. Source= Time Hor. Currency Expect. Detail Organiz. Accur.All Subjects 470 186.84 146,85 78,61 65.29 52,75 84,53 106,20

Table lb. Differences Between Scenarios for Full Sample

t-StatisticsTreatJAttrib. df. Source Time Hor. Currency Expect. Detail Organiz. AccuracyTPS vs IRS 309 -3.92 - 1.49~ - 9.54 -.4,97 -6,55 - 5,03 - 3,81TPS vs DSS 311 -17,48 -15,61 -12,44 -11.12 -9.87 -12,28 -13,31IRS vs DSS 316 -13,92 -14.16 - 2.21 - 6.51 -3.68 - 7,95 -10,03p<0.01 for all F-statistics.Information Attributes listed correspond to Table 1.F-Statistic for Wilkes’ Lambda Listed (p<0.01).p<0.05; all others significant at p<0.01.

The second hypothesis concerned whether thenumber of scenarios given would influence theindividual ratings across scenario types. Like thefirst hypothesis, MANOVAs were employed tosearch for any overall differences in responsesacross scenarios by the number of scenarios pre-sented; ANOVAs and t-tests were performedacross and between scenario types to identityindividual differences caused by the treatments(see Table 2a and 2b). The results indicate thatsubjects were able to distinguish the informa-tion characteristics across scenario types, regard-less of the number of scenarios (all F-statisticswere significantly different at p<0.01). However,there are indications that the level of differentia-tion may vary with the number of scenariosgiven. When only one scenario is given, con-tiguous system types (TPS and IRS, IRS andDSS) are viewed as having some of the sameinformation attributes even though overall rat-ings are significantly different. Therefore, al-though H~ is rejected for the across-scenariocase, further between-scenario analysis isindicated.

An important question is whether sequence ofpresentation affects the perception of system at-tributes. Similar analyses were applied to inves-tigate the question of whether presentation orderwithin groups influenced ratings across scenariotypes. The t-tests were performed on the groupreceiving only two scenarios (Table 3a); for thegroup that received all three scenarios, MANO-VAs (for overall differences) and ANOVAs wereperformed (Table 3b). As Tables 3a and 3b in-

dicate, the results were mixed. For the two-scenario group, all the relationships were in thedirection postulated, but only nine of the 48 indi-vidual comparisons were significantly different.However, when the sequence in which the sce-narios were presented was considered, a differ-ent picture emerged. For the three individual treat-

.ments where the more structured task scenariowas presented before the less structured taskscenario (TPS/IRS, TPS/DSS, and IRS/DSS)four of the 24 individual comparisons were sig-nificantly different (p<0.05)o For the set of ques-tions directed toward the three individual treat-ments where the scenarios were presented inreverse order of task structure (IRS/TPS, DSS/TPS, DSS/IRS) only two individual question com-parisons were significantly different. Further, bothof these were noted when the DSS scenario (theleast structured task description) was followedby the TPS scenario (the most structured taskdescription). One possible explanation for thesefindings is that when a task description is pre-sented in order of progressive complexity, orwhen the distinctions between task structuresare readily apparent, subjects are better able toperceive the differences between the types ofsystems necessary to deal with them.

For the three-scenario group, overall differenceswere apparent for all combinations. Five of sixsequences were significantly different at p<0.01;the remaining sequence was significant atp<0.05. When the individual information attrib-ute responses were examined, all but seven ofthe 24 comparisons yielded significant F-


Validation of the Gorty-Scott Morton Framework

Table 2a. Differences Across Scenarios Within Treatment Groups

F-Statistics

TreatJAttrib. df. Source Time Hor.Currency Expect. Detail Organiz. Accur. MANOVA

1 Scenario 71 15.59"* 7.72** 8.44** 2.26 1.04 1.84 6.25** 4,76**2Scenarios 107 40.17"* 30.79** 22.21"* 21.51"* 18.11"* 23.45** 23.47** 11.02"*3Scenarios 290 141.12"* 122.18"* 50,24** 50.25** 40.00** 69.95** 81.98"* 27.50**

Table 2b. Differences Between Scenarios Within Treatment Groups

t-Statistics

TreatJAttrib. df. Source Time Hor. Currency Expect. Detail Organiz. Accuracy

1 Scenario:TPS vs IRS 45 0,96 - 0,00 -3,24** -2,16"* -0,11 -0,82 -0,18TPS vs DSS 46 - 3.80** - 3,66** -3.94** -1,66"* -1.25 -1,92" -2,93**IRS vs DSS 47 - 5.36** - 3,14"* -0.66 0.28 -1,32" -1.10. -3,08**

2 Scenarios:TPS vs IRS 68 =- 0,71 1.15 -3,82** -1,07 -4.52** -1,53" -1,19TPS vs DSS 69 - 7,54** - 5.81"* -7.46** -5.99** -6,16"* -6,81"* -6,19"*IRS vs DSS 73 - 7.04** - 7,51"* -2.58** -5.11"* -1,24 -4,69** -5.13"*

3 Scenados:TPS vs IRS 154 - 4,24** - 2,01" -6,58** -2.93** -4.36** -3,64** -2,89**TPS vs DSS 154 -14,05"* -12.32’* -7.43** -7.47** -6,53** -8.92** -9.55**IRS vs DSS 154 - 9.09"* -10,08"* -0.48 -4.57** -2.49** -5,97** -6,91"*

* p<0,05 (one-tailed t-tests),** p<0,01 (one-tailed t-tests),

Table 3a, Differences Across Scenarios Between Treatments

t-Statistics

Seq,/Attrib, df, Source Time Hor, Currency Expect, Detail Organiz, Accuracy

TPS/IRS 7 -0.28 -0.21 -0.91 -0.80 - 1.44" - 1.55" -0.61TPS/DSS 9 -0.99 -0,86 -0.85 -1.32 -1.29 -1.97" -0.98IRS/TPS 9 0.00 0.29 0.96 -0.27 0.52 0.20 0.22IRS/DSS 14 -1.00 -1.69* -0.89 -1,03 -0.31 -0.88 -1.13DSS/TPS 8 1.84" 0.46 1.04 0.75 2.49** 1.22 1.30DSS/IRS 7 0.97 0.79 0.20 0.80 0.22 0.22 0.44

Table 3b. Differences Across Scenarios Across Treatments

F-Statistics

Seq./Attrib. df. Source Time Hor.Currency Expect. Detail Organiz. Accur; MANOVA

TPS/IRS/DSS 56 41.72"* 48.42** 23.69** 39.25** 23.72** 32.58** 38.30** 9.94**TPS/DSS/IRS 62 29.84** 18.51"* 22.28** 12.28"* 14.67"* 16.38"* 13.18"* 7.34**IRS/TPS/DSS 41 58.11"* 23.29** 12.48"* 13.31"* 11.40"* 15.76"* 12.82"* 8.15"*IRS/DSS/TPS 41 19.36"* 29.71"* 1.82 10.72"* 0.36 17.17"* 40.74** 5.55**DSS/TPS/IRS 41 10.16"* 4.58* 2.21 1.58 2.53 7.81"* 4.76* 2.02*

= DSS/IRS/TPS 44 13.68"* 22.23** 3.79* 1.66 4.86* 1.72 6.03** 4.10"*

* p<0.05 (one-tailed t-tests).** p<0.01 (one-tailed t-tests).



statistics. All of the insignificant differences oc-curred whenthe scenarios were presented outof sequence, lending further support to the pos-sibility that sequence of presentation may be anintervening variable.

Becau~se of th~ese indicators, the null hypothe-ses H~ and H~ cannot be rejected outright, butshould be given further attention. Subject per-ceptions of system differences may be influ-enced by the manner of presentation.

As shown in Tables 4a and 4b, the same bat-tery of tests were again~used to examine theremaining hypotheses (H~, and H50), whether thenumber of scenarios or the sequence of scenar-ios impacted ratings within each scenario typeacross the treatments. Although only six of the21 attribute comparisons differed, overall ratingsfor the extreme system types (TPS and DSS)varied depending on whether that scenario wasthe only one to be evaluated or whether con-trasting examples were also given. This findinglends support to the possibility that accurate re-quirements assessment is enhanced by provid-ing examples that illustrate extreme contrasts.Null Hypothesis H40 is therefore rejected.

For subjects receiving multiple scenarios, theorder of presentation appeared to have lessimpact than did the availability of reference point(Table 4b). Whereas in the previous analysesratings for the extreme IS types (TPS and DSS)varied, only four of the 21 information attributecomparisons and one of the three overall sce-nario classes (TPS) varied as a result of the se-quence in which they were presented. Furtherinvestigation is indicated.

ConclusionsThe major focus of this article has been to lendempirical support to a frequently applied but un-substantiated framework.. There appears to bean implicit assumption in much of the IS re-search that pdor validation of generally acceptedmodels is neither necessary nor possible. Thisstudy contradicts those positions.

There are two main outcomes of the study. First,the tenets of the Gorry and Scott Morton frame-work for information systems are generallyupheld, as is the assumption that an IS typologycan be associated with it. The seven informa-tion attributes suggested are identifiable and dif-ferentiable, by even naive subjects, and can beordered along the continua as postulated. How-ever, certain experimental conditions affect therelationships implied by the Gorry and ScottMorton framework, Including the manner of pres-entation (whether opposing examples are in-cluded), and the sequence of presentation(whether the examples are in ascending, descend-ing, or some other order of information attributemagnitude). While this is a methodological con-cem that directly applies only in this experiment,there may be practical issues pertaining to in-formation requirements analysis.

Second, the approach used to investigate thehypotheses has been shown as an effectivedevice for gathering data. A set of scenarios hasbeen demonstrated as representative of thethree information system types. Because thesescenarios can undoubtedly be applied to anumber of different research areas, and can fa-cilitate inter-researcher communication and

Table 4a. Differences Across Sample: Number of Scenarios EvaluatedF-Statistics

Scen./Attrib. df. Source Time Hor.Currency Expect. Detail Organiz. Accur. MANOVA2.92**1.052.50**

TPS 152 8.91"* 7.00"* 1.47 1.49 4.91"* 1.09 1.55Ins 157 0.22 0.34 0.88 1.25 0.75 2.32 1.32DSS 159 1.31 2.65 0.25 7.97** 0.35 13.97"* 6.60**

Table 4b. Multiple Scenario Evaluations: Order EffectsF-Statistics

ScenJAttrib. df. Source Time Hor.Currency Expect. Detail Organiz. Accur. MANOVA

TPS 129 1.83 1.79 1.47 3.48** 2.50** 0.89 0.91 1.60**Ins 133 0.96 0.96 0.86 1.30 1.16 3.03** 2.66** 1.05DSS 134 1.09 1.74 1.22 1.31 0.68 1.74 1.22 1.01

** p<0.01.


Validation of the Go~. -Scott Morton Framework

reduce the expense and time invested in astudy, their employment for these purposes isencouraged. For example, they might be usefulin studies of:

1. the ability of users to differentiate and de-velop system types based on systemdifferences,

2. the selection of evaluation strategies (e.g., eco-nomic versus information satisfaction) basedon system types,

3. comparisons of system development strate-gies (e.g., prototyping versus the SDLC ap-proach), and

4. comparisons of abstraction tools (e.g., E-Rdiagrams, DFDs, CASE tools, 4GLs) bysystem types.

The results and limitations of the study point toareas that need additional investigation. The ex-periment was directed toward a sPecific frame-work using a limited sample with a single setof scenarios. These scenarios were carefully con-structed to clearly suggest the information char-acteristics delineated by the framework for a sug-gested class of information systems. The studywas intended to investigate the structural foun-dations of the framework and does not purportto address the causal logic and theory behindit. The robustness of the framework needs yetto be examined through the use of less directed,more diverse instruments aimed at additional ap-plications using larger heterogeneous subjects.

One area of additional research suggested bythe results concerns the issue of order effectsand field, dependency. There are indications thatthe ability to recognize information characteris-tics, and by implication, information require-ments, is facilitated by the inclusion of differentor contrasting examples. The presentation of at-tributes along a continuum also appears to aidin determining the degree to which an attributeis available or required.

In conclusion, the framework proposed nearly20 years by Gorry and Scott Morton appearsto describe differences among information sys-tems in a valid manner. By operationalizing theinformation attributes in their framework, our re-search has shown experimentally that discrimi-nations among desired information attributes aremade for different types of problems. Since em-pirical validation of the framework has not beenreported previously, this study makes a belated

but fundamental contribution to the literature oninformation systems. It confirms the intuitive ty-pology of information systems that has guidedso much IS research and practice since the ty-pology was originally formulated. As IS contin-ues to grew as an academic field, empirical vali-dation of this and other frameworks will providethe necessary foundation for meaningful re-search and theory.

ReferencesAlavi, M. "An Assessment of the Prototyping Ap-

proach to Information System Development,"Communications of the ACM (27:6), June1984, pp. 556-563.

Alter, S.L. Decision Support Systems, Addison-Wesley, Reading, MA, 1980.

Anthony, R.N. Planning and Control Systems:A Framework for Analysis, Harvard .UniversityPress, Boston, MA, 1965.

Bakopoulos, J.Y. "Toward a More Precise Con-cept of Information Technology," Proceedingsof the Sixth International Conference on In-formation Systems, Indianapolis, IN, Decem-ber 1985, pp. 17-24.

Blumenthal, S. Management Informa~on Sys-tems: A Framework for Planning and Devel-opment, Prentice-Hall, Englewood Cliffs, NJ,1969.

Campbell, D.T. and Stanley, J.C. Experimentaland Quasi-experimental Designs for Re-search, Rand McNally & Co., Chicago,, IL,1963.

Cerveny, R.P. and Sanders, G.L. "Implementa-tion and Structural Variables," Information andManagement (11:4), 1986, pp. 191-198.

Chervany, N.L., Dickson, G.W. and Kozar, K.A."An Experimental Gaming Framework for In-vestigating the Influence of Management In-formation Systems on Decision Effectiveness,"working paper, Management Information Sys-tems Research Center, Minneapolis, MN,1971.

Courtney, J.F., DeSanctis, G.L. and Kasper,G.M. "Continuity in MIS/DSS Laboratory Re-search: The Case for a Common Game Simu-lator," Decision Sciences (14:3), July 1983,pp. 419-439.

Culnan, M.J. "The Intellectual Structure of Man-agement Information Systems, 1972-1982: ACo-Citation Analysis," Management Science(32:2), February 1986, pp. 156-172.



Culnan, M.J. "Mapping the Intellectual Structureof MIS, 1980-1985: A Co-Citation Analysis,"MIS Quarterly (11:3), September 1987, pp.341-355.

Culnan, M.J. and Swanson, E.B. "Research inManagement Information Systems, 1980-1934: Points of Work and Reference," MISQuarterly (10:3), September 1986, pp. 289-302.

Doktor, R.H. and Hamilton, W.F. "Cognitive Styleand the Acceptance of Management ScienceRecommendations," Management Science(19:8), April 1973, pp. 884-894.

Dubin, R. Theory Building, The Free Press, NewYork, NY, 1969.

Dutton, W. Computers and Politics: High Tech-nology in American Local Governments, Co-lumbia University Press, New York, NY, 1982,pp. 14-15.

Gorry, G.A. and Scott Morton, M.S. "A Frame-work for Management Information Systems,"Sloan Management Review (13:1), Fall 1971,pp. 55-70.

Hamilton, S. and Ives, B. "Knowledge UtilizationAmong MIS Researchers," MIS Quarterly(6:4), December 1982, pp. 61-77.

Henderson, J.C. and Nutt, P.C. "The Influenceof Decision Style on Decision Making Behav-ior," Management Science (26:4), April 1980,pp. 371-386.

Ives, B. and Olson, M.H. "User Involvement andMIS Success," Management Science (30:5),May 1984, pp. 586-603.

Ives, B., Hamilton, S. and Davis, G.B. "A Frame-work for Research in Computer-Based Man-agement Information Systems," ManagementScience (26:9), September 1980, pp. 910-933.

Jacques, E. and Brown, W. A General Theoryof Bureaucracy, Heinemann EducationalBooks, London, 1976.

Kahn, H. and Weiner, A.J. The Year2000, MacMil-lan, New York, NY, 1977.

Keen, P.G.W. "Interactive Computer Systems f~)rManagers: A Modest Proposal," Sloan Man-agement Review (18:1), Fall 1976, pp. 1-17.

Keen, P.G.W. "MIS Research: Reference Disci-plines and Cumulative Tradition," Proceedingsof the First International Conference on Infor-mation Systems, Philadelphia, PA, December1980, pp. 220-232.

Kirs, P.J. An Experimental Investigation into

Levels of User Involvement in System Devel-opment Across System Types, unpublishedPh.D. dissertation, State University of NewYork at Buffalo, NY, 1987.

Lucas, H.C. "A Descriptive Model of InformationSystems in the Context of the Organization,"Data Base (5:2,3,4), Winter 1973, pp. 27-39.

Markus, M.L. Systems in Organizations: Bugsand Features, Pitman, Marshfield, MA, 1984.

Markus, M.L. and Robey, D. "Information Tech-nology and Organizational Change: CausalStructure in Theory and Research," Manage-ment Science (34:5), May 1988, pp. 583-598.

Mason, R.O. and Mitroff, I.I. "A Program of Re-search on Management Information Systems,"Management Science (19:5), May 1973, pp.475-487.

Naumann, J.D. "The Role of Frameworks in MISResearch," Proceedings of the 1986 DecisionSciences Institute, Honolulu, Hawaii, Novem-ber 1986, pp. 569-571.

Nolan, R.L. and Wetherbe, J.C. "Toward a Com-prehensive Framework for MIS Research,"Management Science (26:9), 1980, pp. 910-934.

Remus, W.E. "An Empirical Test of the Use ofGraduate Students as Surrogates for Manag-ers in Experiments on Business DecisionMaking," Journal of Business Research(14:1), February 1986, pp. 19-25.

Remus, W.E. and Kotteman, J.E. "Semi-Struc-tured Recurring Decisions: An ExperimentalStudy of Decision Making and Some Sugges-tions for DSS," MIS Quarterly (11:2), June1987, pp. 233-243.

Simon, H.A. The New Science of ManagementDecisions, Harper and Brothers, New York,NY, 1960.

Sprague, R.H. "A Framework for the Develop-ment of Decision Support Systems," MIS Quar-terly (4:4), December 1980, pp. 1-26.

Van Over, L.D. and Nelson, R.R. "Research inManagement Information Systems, 1980-1985: A Profile of Leading Journals, Articlesand Authors," Unpublished paper, Universityof Houston, Houston, TX, 1986.

Wilkes, S. "Certain Generalizations in the Analy-sis of Variance," Biometrika (54), 1932, pp.471-494.

Zmud, R.W. Information Systems in Organiza-tions, Scott, Foresman and Company, PaloAlto, CA, 1983.



About the AuthorsPeeter J. Kirs is an assistant professor of infor-mation systems at Flodda International Univer-sity in Miami. He received his Ph.D. in manage-ment information systems from the StateUniversity of New York at Buffalo in 1987. Hiscurrent research interests include human factorsin systems development, user padicipation in sys-tems design, and the strategic use of informa-tion systems.

G. Lawrence Sanders is assistant professor ofmanagement science and systems in the Schoolof Management at the State University of NewYork at Buffalo. His current research interestsare in the cognitive science approach to sys-tems development, strategic database design,and systems success measurement. He has pub-lished articles in several journals including MISQuarterly, Decision Sciences, and The Journalof Management Information Systems.

Robert P. Cerveny is an associate professorof management science and systems in theSchool of Management, State University of NewYork at Buffalo, where he is the director of MISprograms and a director of The Center for In-dustrial Effectiveness. He holds a Ph.D. degreefrom the University of Texas at Austin and has

published over 25 papers in the information sys-tems areas. His research interests include sys-tems implementation issues, requirements analy-sis, and expert systems. He is a member of sev-eral professional societies, including the As-sociation for Computing Machinery, the DecisionSciences Institute, and The Institute for Manage-ment Sciences.

Daniel Robey is professor and chairman of theDepartment of Decision Sciences and Informa-tion Systems at Florida International University.He is the author of two textbooks and has pub-lished widely in the administrative science andinformation systems literature, including articlesin such journals as Management Science, MISQuarterly, Communications of the ACM, HumanRelaEons, Academy of Management Review,Academy of Management Journal, and DecisionSciences. Dr. Robey currently serves on sev-eral editorial boards and was an associate editorfor MIS Quarter/y. His current research dealswith the consequences of information systemsin organizations and the process of system de-velopment. Dr. Robey earned his doctorate in1973 from Kent State University. He has previ-ously served on the faculties of The Universityof Pittsburgh, Marquette University, and theCopenhagen School of Economics and BusinessAdministration;

Appendix

Scenario Descriptions

TPS ScenarioAssume that you are the distdct manager of a chain of retail stores that sell automobile tires. Thecompany has just purchased small business computers for each of the stores and the company’sheadquarters. In a meeting of company executives, it was decided that the computer should be usedto assist in running day-to-day operations. You were assigned the responsibility of developing a com-puterized inventory tracking system that could be used by each of the stores to keep up-to-the-minuterecords of items in stock.

Each store typically carries five different brands of tires. Each brand generally comes in two types(poly-glas and radials), either black-walls or white-walls, up to three different widths, three differentwheel sizes, and five different styles. Therefore, each store may carry as many as 900 distinct items,although the typical inventory ranges between 200 and 600. The exact levels maintained for eachitem have historically been set based on the best estimate of each general manager.



A loose-leaf notebook is presently used as a record of the number of each item in inventory, theselling and purchase prices, the dates they were last received, withdrawn, and ordered, as well asany outstanding orders or items backorde~’ed (i.e., orders previously accepted were incomplete, anditems shorted are considered backordered). It is the responsibility of an inventory clerk to keep thenotebooks up to date.

.When items are received, they are checked by the inventory clerk, the number received and the pur-chase order number are entered in the notebook, and the number on hand is updated accordingly.If the order is incomplete, the number on backorder is also updated. Invoices are then signed by theclerk and sent to the front office to be recorded in the general ledger. When items are taken fromthe storeroom, the number and date withdrawn are recorded. The number remaining in stock is up:dated (previous number on hand less number withdrawn). If new items are needed, the inventoryclerk fills out a purchase order to the appropriate supplier (there are typically two to four suppliersfor each item). Any order for more than 100 items requires the approval of the general manager.

You have discussed the information needs for the inventory tracking system with each of the generalmanagers, it was agreed that all inventory transactions should be recorded as they occur (online),so that precise, up-to-the-minute information on any item in inventory would be available upon request.Additionally, it was agreed that each of the stores should maintain data in the same format in theirrespective computers, so that if a store needed to borrow items from another store they could calland "talk the same language." However, there was general disagreement about what options shouldbe included in the information system and how much information it should generate. Ultimately, itwas agreed that the decision would be yours and yours alone.

Your concern is to develop an effective information system to assist each manager in running his orher operation. There are a number of considerations. Foremost among these is the selection of optionsto be included in the inventory tracking system. Since the system is to be used by the inventoryclerks, you are concemed that the system will be easy to use and will include all options necessaryfor smooth operations, but will not be too involved and provide unnecessary information (that wouldalso add to its cost). You need notconcern yourself with the technical programming chores since systems analyst has been hired by the company for this purpose.

IRS ScenarioAssume that you are the district manager of a chain of retail stores that sell automobile tires. In arecent meeting of company executives, you were asked to prepare summary reports of inventory tumoover (the time between receipt and employment of items) for each of the stores, and the companyas a whole. The executives particularly wanted to.know about monthly item sales in each of the stores,and if any unusual changes in demand had been noticed. You were assigned the responsibility ofdeveloping an information system that .could be used to more closely monitor inventory usage andserve as a basis in developing managerial control policies for inventory levels.

Each store typically carries five different brands of tires. Each brand generally comes in two types(poly-glas and radials), either black-walls or white-walls, up to three different widths, three differentwheel sizes, and five different styles. Therefore, each store may. carry as many as 900 distinct items,although the typical inventory ranges between 200 and 600.

A computerized inventory tracking system was set up last year, and each store maintains its owninventory records on its own computer. For each item, up-to-date data is available of the numberon hand, number on order and backordered, the date items were last withdrawn, received and ordered,and the purchase and selling prices. All transactions (withdrawals and receipts of inventory) are re-corded as they occur, and each evening this data is transmitted (via telephone line) from each of thestores to company headquarters. All stores maintain data in the same format so they can "talk thesame language."



Your main concern is in developing an effective information system to assist you and each of thestore managers in controlling inventory and developing inventory maintenance policies. Specifically,you have been charged with developing an information system that is capable of:

1. producing monthly detailed and summary reports on inventory usage for each store and the com-pany as a whole,

2. pointing out any exceptionally large or small turnover rates for any item carried, and

3. producing special, "on-demand" reports about individual item or group turnover ratios.

Based on these reports, you could determine which items and item lines contribute the most to profit,which items/item lines show increases in demand, and which items/item lines might best be droppedfrom inventory. The problem appears to be in determining the content of the reports (the categoriesand subcategories included; the range for acceptable turnover rates) and the amount of informationto be included in the reports. If all available information is included, the reports would become volumi-nous (and costly). Since the reports will also be distributed to each of the store managers, you areconcerned with providing information that might not be of use to a particular branch. On the otherhand, you do hot wish the reports to be so condensed that potential problems are not exposed.

Consequently, a number of decisions are required. These include the selection of the individual outputcategories, the cross-referencing of output categories, what turnover rates should be considered ex-ceptional, what and how analysis of the data should be carried out, and what reporting periods shouldbe used. If you wish, information on industry turnover rates, which could be used as a basis of compari-son, is also available. You need not concern yourself with the technical programming chores sincea systems analyst has been hired for this purpose.

DSS ScenarioAssume that you are the district manager of a chain of retail stores that sell automobile tires. In arecent meeting of company executives, it was pointed out that the industry is forecasting a shift indemand patterns from the larger, wide-body tires toward the smaller, narrower radials. You were askedto prepare alternative inventory planning strategies for various demand changes for each of theseitems. Specifically, you were requested to prepare a report on how inventory turnover (length of timethat items remain in inventory) would be affected in any given situation. To this end, you are to developan inventory information system that will assist you in gathering the information and making organiza-tional planning decisions.

Each store typically carries five different brands of tires. Each brand generally comes in two types(poly-glas and radials), either black-walls or white-walls, up to three different widths, three differentwheel sizes, and five different styles. Therefore, each store may carry as many as 900 distinct items,although the typical inventory ranges between 200 and 600.

A computerized inventory tracking system was set up last year, and each store now maintains itsown inventory records in their own computer. For each item, up-to-date data is available of the numberon hand, number on order and backordered, the date items were last withdrawn, received and ordered,and the purchase and selling prices. All transactions (withdrawals and receipts of inventory) are re-corded as they occur, and each evening this data is transmitted (via telephone line) from each of thestores to the company headquarters. All of the stores maintain data in the same format so they can"talk the same language."

Monthly summary reports of individual item and/or category turnover ratios and "exception" reports(i.e., item and group turnover ratios either 25% .above or below expected ratios) are prepared at head-quarters. If desired, "on-demand" reports of individual item or group turnovers can be prepared atany time.

Your main concern is in developing an effective information system to assist you in evaluating howgiven changes in future demand will affect the individual stores and the company as a whole, and



to serve as a basis for decision making about developing inventory purchasing procedures. Sinceyou, and perhaps your assistant, will essentially be the only users of the System, you can be relativelydetailed in specifying your needs.

Nonetheless, you would like a system that is flexible and can evaluate a large number of differentalternatives quickly, but is not so involved and complicated that it becomes cumbersome and difficultto understand. Consequently, a number of decisions are required. These include:

1. selection of areas for analysis to be included in the system,

2. what analyses and analytic techniques should be used,

3. how many factors should be considered in the evaluations, and

4. what output should be included and how it should be presented.

If you wish, industry and area economic data is available. The industry data gives information onturnover rates; the economic data gives information on changes and forecasts of population, CPI,and per capita income. All data already being used in generating the monthly reports can also beincluded. You need not concern yourself with the technical programming chores since a systems analysthas been hired by the company expressly for this purpose.


Documents

An Experimental Validation of the Gorry and Scott Morton ... · Introduction As a relatively new field of applied research, in-formation systems (IS) displays a considerable ... 184