Management Research Methods: Insights from Critical Realism

Eric W. TsangSchool of Management

University of Texas at Dallas

Why does philosophical perspective matter?

Every researcher subscribes to a certain philosophical perspective, whether explicitly or implicitly.

That perspective affects the formulation of research questions, method of data collection and analysis, as well as development of theory.

Critical realism

A mind-independent reality, which has its own inherent order, exists.

Reality consists of structures and mechanisms.

Structures refer to sets of internally related objects.

E.g., the employer-employee relationship presupposes the existence of employment contract, wage, hiring, firing, and so on.

Critical realism

Mechanisms refer to ways of acting. The combined effects of structures and

mechanisms may generate events that in turn may be observed.

The absence of an observable event does not necessarily mean that the underlying mechanisms do not exist.

Critical realism

Three domains of reality: Empirical: events experienced through direct or

indirect observation. Actual: events, whether or not they are observed. Real: generative mechanisms capable of

producing patterns of events.

Real Domain Actual Domain Empirical Domain

Mechanisms

Events

Experiences

Critical realism

Events E1 E2 E3 E4 E5 E6 E7 E8 Ei Contingent conditions Mechanisms M1 M2 M3 M4 M5 M6 Mj Structures S1 S2 S3 S4 Sk

Critical realism

E.g., if an observed choice of governance structure contradicts the prediction of transaction cost economics, it is not necessarily an incident of falsification.

It may indicate the presence of countervailing mechanisms, such as tax incentives or government restrictions.

Critical realism

In social studies, it is rarely possible to achieve conditions of closure where an event of type a is invariably accompanied by an event of type b.

The impossibility of constructing closed systems in the social sciences implies that the disciplines are primarily explanatory or interpretive rather than predictive.

Critical realism

A causal explanation aims to identify objects, structures and the mechanisms that connect them, causing events to occur.

The goal of scientific inquiry is to provide knowledge about the real structures and mechanisms underlying the phenomenon in question.

Critical realism vs. positivism

Positivists reject the critical realist view that theories may contain theoretical terms that (1) are in principle unobservable and (2) cannot be defined through a set of correspondence rules with observable terms.

Positivists adopt Hume’s view that causation refers to the regular succession of events.

Some insights from critical realism

Need assumptions be realistic?Need empirical studies be replicated?Can case findings be generalized?

Need assumptions be realistic?

Friedman (1953) argued that it does not matter whether the assumptions of an economic theory are realistic as long as the theory yields sufficiently accurate predictions.

Shugan (2007) echoed, “Criticizing assumptions as unrealistic is absurd.” (p. 449)

Need assumptions be realistic?

A core assumption is often a key element of the mechanism that underlies an explanation.

E.g., for transaction cost economics, the assumption of opportunism plays a crucial role in the mechanism that accounts for the choice of hierarchy versus market.

An unrealistic core assumption will lead to an unrealistic explanation and thus a defective theory.

Need assumptions be realistic?

If possible, core assumptions should be included in empirical tests.

Structural model

z = f(x, y) (1a)

where x = g(u), and y = h(v) (1b)

Reduced model

z = φ(u, v) (2)

Need assumptions be realistic?

Equations 1b “explain” the intermediary variables x and y, whereas Equation 2 does not even contain them.

The reduced model can be derived from the structural model but not the other way round because the task of working out the structural model from the reduced model is an inverse problem with an indefinite number of solutions.

Need assumptions be realistic?

A core assumption is often eliminated in the process of converting a structural model into its reduced form.

Most empirical studies of transaction cost economics are based on the reduced model format; thus the related core assumptions and mechanisms are not subject to direct testing.

Need assumptions be realistic?

Williamson claims that transaction cost economics “is an empirical success story” (1996: 55).

Ghoshal and Moran counter, “Even though an impressive number of empirical studies have found a positive relationship between asset specificity and internalization …, correlation does not demonstrate causation” (1996: 40).

Need empirical studies be replicated?

In many natural sciences, the replication of empirical findings is a common practice.

Replication research is seldom conducted, not to mention being published by management journal.

An issue that has to be addressed by management researchers if they claim that management is a science discipline.

Need empirical studies be replicated?

1. Accommodation and prediction

Accommodation: a researcher constructs a theoretical explanation for the data collected from his or her study.

Prediction: another researcher later successfully replicates the results of the original study

The replication provides the theoretical explanation with a quantum leap in credibility.

Need empirical studies be replicated?

When Dmitri Mendeleev constructed a theory of the periodic table to account for all the sixty known elements, the scientific community was only mildly impressed.

However, when he went on to use his theory to predict the existence of two unknown elements that were later independently discovered, the Royal Society awarded him a Davy Medal.

Need empirical studies be replicated?

2. Verification and falsification of theories

When many replications conducted in diverse contexts are repeatedly successful, it is highly likely that the theory concerned has hit upon some real structure or mechanism in the social world.

When a theory repeatedly fails to be replicated, it is more plausible to regard the original findings as the result of chance factors or idiosyncrasies of the context rather than the manifestation of real structures or mechanisms.

Need empirical studies be replicated?

Since management studies are rarely conducted under conditions of closure, replicability does not mean conclusive verification and failure to be replicable does not mean conclusive falsification.

Continuous testing of theories through replicated and innovative studies is needed for theory development.

Can case findings be generalized?

Case studies are often criticized on the grounds that their findings are not generalizable to other settings because of the small-N problem.

“If there is no generalizing beyond the data, no theory. No theory, no insight. And if no insight, why do research?” (Mintzberg, 2005: 361).

Can case findings be generalized?

There is some invariance in the social world, i.e., some structures and mechanisms remain stable across space and time.

If all events and their causes are unique, there could neither be valid generalizations nor theories. In fact, our lives would be virtually impossible.

Can case findings be generalized?

Case study researchers collect detailed data through various sources on events that occur in an open-system context.

Though data are collected in the empirical domain, case studies shed light on the specific contingent conditions under which the postulated mechanisms operate and interact in the real domain.

Can case findings be generalized?

In this respect, case studies are superior to quantitative studies, most of which are based on the reduced model format.

Given the existence of invariance in the social world, the extent of which can be checked by replicated studies, case findings are generalizable, though not in the statistical sense.

Advice for doctoral students

When you have time, it is always good to read Western philosophy, especially philosophy of science or social science.

You’ll be amazed by the quality of arguments presented in these writings. Try Bertrand Russell’s (1912) The Problems of Philosophy as a start.

At the very least, it will help you avoid glaring mistakes of argument.

Management information systems

“The generalizability of sample points to a sample estimate depends on whether certain conditions required by statistical methods are satisfied …” (p. 234)

Lee, A. S., & Baskerville, R. L. 2003. Generalizing Generalizability in Information Systems Research. Information Systems Research, 14(3): 221-243.

Marketing

“without models, every situation at every time on every variable would be unpredictable” (p. 449).

Shugan, S. M. 2007. It’s the findings, stupid, not the assumptions. Marketing Science. 26(4) 449-459.

Organization theory

“The essence is that no matter how hard we might try, the characteristics of complexly interactive and tightly coupled systems will cause a major failure, eventually” (p. 216)

Perrow, C. 1994. The limits of safety: The enhancement of a theory of accidents. Journal of Contingencies and Crisis Management, 2(4): 212-220.