Design Science and Critical Realism Some Methodological Issue

Greg Hill, Monash UniversityCBIDSR Workshop, 20/10/05

Overview

Historical Development and Rationale

Structure of Design Science Research

Evaluation in Design Science

Guidelines for Assessment

Overview

Philosophy of Design Science Research

Ideas from Critical Realism

Critical Realist Evaluation

Implications for Information Systems

Historical Development

– Natural versus artificial sciences

• truth, necessity, analysis, descriptive, predictive

• utility, contingency, synthesis, prescriptive

– Focus on artefacts, goals and environment

– Simulation and Imitation

– Design distinguishes professions

Historical Development“… the natural sciences have almost driven the

sciences of the artificial from professional school curricula ...”

“ … Engineering schools have become schools of physics and mathematics …”

“… professional schools … hanker after respectability …” (Simon, 1969)

– Engineering, Business and Medicine

– Law, Journalism and Library Science

– (Computer Science and Management Science)

Structure of Research

Process View Vaishnavi, V. and Kuechler, W. (2004/5). “Design Research in Information Systems” URL: http://www.isworld.org/Researchdesign/drisISworld.htm

Structure of Research

Knowledge View (Hevner et al, MISQ, 2004)

Evaluation Methods

– Observational

• case study, field study

– Analytical

• static, architecture, optimisation, dynamic

– Experimental

• controlled experiment, simulation

Evaluation Methods

– Testing

• black box, white box

– Descriptive

• informed argument, scenarios

Guidelines for Assessment

– Design as an Artefact

– Problem Relevance

– Design Evaluation

– Research Contribution

Guidelines for Assessment

– Research Rigour

– Design as Search

– Design as Communication

Philosophy of Design Science

– Essentially instrumentalist and pragmatist

• accepts “natural sciences” as law-generator

• claims to knowledge assessed on usefulness

– But spans socio-technical systems

• aeroplanes, computers, mobile phones ...

• organisations, markets, families … ?

Philosophy of Design Science

– Key question for design of socio-technical

systems (like information systems):

Can the philosophies and methods of

logical positivism work for “social science”

• Do reasons have causal powers?

• Does poverty cause crime?

Ideas from Critical Realism

– Kicked off by Roy Bhaskar (1975)

• “Transcendental Realism” - attempts to unify

positivism and interpretivism

• Naturalism - the idea the social and natural

phenomena can be studied the same way

• Proposes three ontological domains: – real

– actual

– empirical

Ideas from Critical Realism

• Rejects “social atomism” / “methodological

individualism” - a form of social reductionism

• Rejects Durkheim (agency -> structure)

• Rejects Weber (structure -> agency)

• Proposes an interplay (structure <-> agency)

Ideas from Critical Realism

“He found God. Worst of all it was a very down market

god, nothing more than your common or garden New

Age variety, the type readily available at any incense

saturated shop frequented by a Shirley MacLaine or

Nancy Reagan.” Gary MacLennan.

Ideas from Critical Realism

“Put very simply, a central feature of realism is its attempt to preserve a ‘scientific’ attitude towards social analysis at the same time as recognising the importance of actors´ meanings and in some way incorporating them in research.” (Layder, 1993)

Ideas from Critical Realism

– Analogy and metaphor

• hypothesis formulation

– Patterns and triggers

• demi-regularities, enablers/disablers

– C-M-O configurations

• Context - Mechanism - Outcomes

Ideas from Critical Realism

“ … outcomes cannot be explained in isolation; rather, they can only be explained in the sense of a mechanism that is introduced to effect change in constellation of their mechanisms and structure, embedded in the context of pre-existing historical, economic, cultural, social and other conditions. This process of explanation, known as retroduction, enables the realist inquirer to investigate the potential causal mechanisms and the conditions under which certain outcome will or will not be realised.” (Kazi, 2003)

Critical Realist Evaluation in Design Science Research

– Goal of evaluation phase is to understand

the C-M-O configurations (explanatory)

– What works, for whom, in what circumstances?

• Scenarios, case studies, simulations

– “Three boxes” of evaluation

• black, grey and white (clear)

• making the box “greyer”

Implications for IS

– What is the role for IS researchers?

• Business needs?

• Reference disciplines?

• Consultancies?

• Multi-methodology research?

Implications for IS

– What is the role for IS theory?

• CSF -> CMO?

• Design theories?

• Evaluation theories?

Implications for IS

I agree that it works in practice. But how

can we be certain that it will work in

theory? -- Attributed to a professor at

École Normale Supériere.