1

The Efficacy of Matching Information Systems

Development Methodologies with Application Characteristics

– An Empirical Study

Present bySaidur RahmanMuayah Angas

2

Presentation Layout (1)

• Part 1– Introduction– Why are we studying this– System Development Methodologies– Cognitive Fit Theory– Terms and definition

3

Presentation Layout (2)

• Part 2– Proposition & Hypothesis– Experiment Detail– Results & Conclusions– Limitations of Study– Other Studies– Group Conclusion

4

Introduction

• Vessey and Glass (1994) characterize systems development as ``fundamentally a problem-solving'' activity.

• System development is a problem solving activity meanwhile methodologies represent different approaches for developing systems solutions.

5

Why we are studying this?

• Nakajo & Kume, 1991 – Examined programs developed with/without the SDM. Programs developed with methodolgy assistance resulted in fewer errors.

• Nosek & Schwartz, 1991 – claims no difference in users’ understanding of requirements from both Data flow diagrams and Straight narrative descriptions, this is supported by Cioch, 1991.

6

SDM divide into 2 categories:• Weak• Strong

Method/1 – is an examples of a Weak SDM

RMM – is an example of a Strong SDM

.

System Development Methodologies

7

Weak Methodology

• Andersen pioneered in the 1960's a step-by-step methodology for handling any computer project called Method/1

• This is an example of a weak system development methodology that claims to be able to guide any kind of system development project, independent of the technology being applied or the nature of the application being developed.

8

Strong Methodology

• RMM ( Relationship Management Methodology) is a systems development and project management technique used mainly for the design and construction of hypermedia applications – (Isakowitz and Thring.)

RMM facilitates the design of WWW sites, and their seamless integration with databases and enterprise-wide information systems

9

Figure 1

10

Cognitive Fit Theory

• The theory proposes that the correspondence between task and information presentation format leads to superior task performance for individual users. In several studies, cognitive fit theory has provided an explanation for performance differences among users across different presentation formats such as tables, graphs, and schematic faces (e.g., Vessey, 1991, 1994; Vessey & Galletta, 1991; Umanath & Vessey, 1994)

11

Cognitive Fit Theory• Source: Shaft, Teresa M. and Iris Vessey, (2006) "The Role of Cognitive Fit in the Relationship

between Software Comprehension and Modification", MIS Quarterly, Volume 30, Issue 1, pp. 29-55.Figure 2

12

Class of Methodologies(1)

• Process-based Methodology – Defining the activities associated with the system.Most scientific and engineering applications, for example, are of this kind. In the information systems area, payroll, inventory, accounts receivable, and accounts payable are often characterized in this way.

13

Class of Methodologies(2)

Data-based Methodology – Defining the contents of the data storage containers and how the contents are organized.Applications that deal with record keeping such as medical records systems are usually this type. The processes for these types of applications may be relatively simple, but the organization and access of the system application may be quite complex.

14

Class of Methodologies(3)

The object-oriented approach to handling complexity treats both data and process as a package. An object is a component of the real world, a cohesive collection of data coupled with the processes that act on that data. The act of systems development using the object-oriented approach interleaves analysis and design of objects with analysis and design of the operations relating to these objects. The rationale for the object-oriented approach is that application problems often evolve around real-world objects and the ways in which they interact.

15

Proposition

Use of a Data-based Methodology to guide development of a data-based application will produce a higher quality system than Process-base Methodology

16

Hypothesis 1

The quality of the system designs applied to a data-centered design problem produced using the data-based methodology will be significantly better than the quality of the system designs produced using the process-based methodology.

17

Hypothesis 2

The quality of the system designs applied to a data-centered design problem produced using a system developmentmethodology will be significantly better than the quality of the system designs produced using no system development methodology.

18

Experiment

30 Students were used for the experiment. They were randomly divided into 3 groups. One for the data-centered methodology, one for the process-centered methodology and one without a methodology (called the control group). They were each given the problem posed in Fig.3Two methodologies were developed for the experiment. The data-centred methodology required subjects to develop a data model before proceeding to the rest of the system design. The process-centered methodology leads with process modeling and makes only limited use of data modeling.

19

Results of Empirical Evaluation

Methodology Number of Evaluations

Score0 = data-based40 = process-based

Data-based 7 6.7

Process-based

7 23.9

20

Context Diagram

Figure 3

Deli Sandwich System

0

Store

a

Production Department

b

Invoice

Order

Daily slicing ReportDaily Production Report

21

Entity Relationship Diagram

Figure 4

ChainStore Std Costbelongs to

is charged

SandwichOrder ItemOrder

Recipe ItemIngredient

exist for

contains

consists of

consists of

places

is used as

StoreID

SandIDOrderID

ChainID

IngredientID

OrderID +

SandID +

SandID

IngredientID

22

Decomposition DiagramFigure 5

Accounting Subsystem

Order Entry Subsystem

Deli Sandwich Context

Deli Sandwich System

Production System

Generate Daily Invoices

Generate Daily Slicing Schedule

Generate Daily Production Schedule

Order Entry

23

System Diagram

Figure 6

Accounting Subsystem

1 Standard D1

StoresD2

standard cost

store address

Storeb

Production Subsystem

2

Order Entry Subsystem

3

Production Department

c

OrdersD3

Standard D4

order

recipeprocessed order

order

Daily Sl icing ReportDaily Production Report

order

invoice

24

Hypothesis tests results

Pair compared Ri/ni – Rj/nj Kruskal-Wallis p=0.05

Sign’n level

(p<0.05, 12 df, N=15)

Results

DCM vs PCM 4.96 5.41 H1 – Not Supported

DCM vs No Methodology

8.30 5.41 H2 - Supported

PCM vs No Methodology

6.70 5.41 H2 - supported

25

Paper Conclusion

• How much Support should a Methodology provide a designer,

• Too much and it hinders.• Too little and the designer lacks support.• Use of methodology better than no methodology.

Human designers need coarse guidance in building systems to assure that the macro steps are all performed and that they are done in the proper order, but the microguidance provided by strongly typed methodologies may be counterproductive

26

Limitations of Study(1)

• Possibility of contamination as a few students had previous industry experience.

• The problem needs to be better validated. Whilst care was taken to ensure the problem posed was data-centered it would be better experiment if a suite of problems were used.

27

Limitations of Study(2)

• Sufficient time needs to be allotted for the subjects to perform systems design.

• More real world tools should be used.

• Extend the experiment to include object orientated methods.

28

Experimental Studies

Agarwal, 1996 –The proposition was that methods and application problems should be matched and would result in better systems. Process-oriented methodology resulted in better system design in process based tasks. but in the case of Object orientated tasks the OO methodology did not result in a better system.

29

Other StudiesIn “Be Flexible with standards” Levin (1997) reported that developers atEriccson opted to reject the constraints of formalmethodologies and still succeeded in building a highlysuccessful engineering and sales system.

In “Methodologies for the future” Griffin(1997) in a study, the Data Warehousing Institute in Bethesda, Md. surveyed 21 data warehouse project managers on their most difficult challenges. Methodology was the third biggest menace, right behind technology and education. They argued a good methodology can ease the burden of development and educating users

30

Group Conclusion

• Flaws in experiment as shown by Limitations of study

• Flaws in reasoning (seem to have already decided what answer was before test disproved it and then were hesitant in supporting it)

• Result of study shows that data-centred methodology are little better than the process centred one, but it doesn’t show it to be counter productive. Why have they drawn this conclusion?

31

Presentation Reference

• Agarwal, R., A. P. Sinha, and M. R. Tanniru (1996) “Cognitive Fit in Requirements Modeling: A Study of Object and Process Methodologies” Journal of Management Information Systems, 13(2), 137–162.

• Vessey, Iris (1991). Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature. Decision Sciences 22,(2), 219-240.

• Vessey, Iris, Galletta, Dennis (1991). Cognitive Fit: An Empirical Study of Information Acquisition. Information Systems Research, 2(1), 63-8

• Levin, R., 1997. Be flexible with standards. Information Week 618 (February), 1A-5A.• Griffin, J., 1997. Methodologies for the future. Software Magazine17(2), S5-S7.• http://www.ccsr.cse.dmu.ac.uk/staff/Srog/teaching/strategy.htm