Causal inference in the social sciencesWhy mechanisms matter and how to find out

about them

by Attilia RuzzeneErasmus Institute for Philosophy and Economics, Rotterdam

attilia.ruzzene@gmail.com

Overview

About me:Phd dissertation: “Causality in economics. A

methodological inquiry”◦ (defended Feb 2010, department of economics Cognetti

de Martiis, Torino) Currently, Phd student at Erasmus Institute for

Philosophy and Economics (EIPE, Rotterdam): “Tracing the process: why and how mechanisms matter in the social sciences”

Today presentation:◦Brief summary of my past research◦ Insights to pursue further◦Pending problems in need of a solution

“Causality in economics. A methodological inquiry”

General motivation to my (first) project: Relevance of causal knowledge to the variety of purposes

economists (and social scientists as well) pursue: ◦ (To differ extent) explanation, prediction, intervention.

Difficulties in having that knowledge reliable◦ J.S. Mill (1843), on the difficulties of applying its methods for causal

inference to a science that is not experimental◦ Marshall (1870), on the complexity of causal forces as to their number,

(lack of) regularity, way of combining (chemical rather than mechanical) Hence, the questions:

◦ How do economists attain causal information (the methods in use);◦ What type of causal information this is (do different methods provide

causal knowledge that is diverse?);◦ What purposes it is intended to serve (what type of causal knowledge

serves what type of purposes).

The methods in use

Economists use a variety of methods to draw causal inferences:◦Statistical methods (SEM, Bayes Nets);◦Experiments (thought ex., natural ex., laboratory

ex.);◦Case-based methods (process tracing).

I selected two cases –instances- of scientific practice and analysed them in detail to see:◦How the methods work in the specific context;◦What assumptions they rely on;◦What causal information they provide;◦What the specific causal information is (thought)

useful for.

The sample

Case study research: Regional Advantage. Culture and competition in Silicon Valley and Route 128 by A. Saxenian 1994;

Techniques of causal analysis: small-n method of comparison and process tracing;

Type of evidence: correlation and mechanistic;

Research purpose: explaining socio-economic phenomenon (asymmetry in regional development).

Structural econometric model: Marshallian causal functions by J. Heckman 2000, 2001, 2008;

Technique of causal analysis: controlled (ceteris paribus) variation in thought experiments;

Type of evidence: causal effects

Research purpose: measuring the impact of economic and social policies on outcomes of interest (intervention).

Some conclusions on the role of mechanisms

Mechanisms play a crucial role in the case-study research: they help causal inference. ◦ In particular, the mechanism gives structure (and order) to the causal

backbone (the web of causal relations) around which the historical narrative is articulated.

◦ If the mechanism was not detected the narrative would not be possibly recounted.

◦ Thus, process tracing –and the mechanism it detects- are necessary in the circumstances, though not sufficient by themselves, for the causal inference. The small-n method plays a complementary and equally necessary role.

Mechanisms play no role in Heckman’s proposal for evaluating intervention. However, they should.◦ MCF for causal effects are irrelevant in the range of circumstances in which

interventions affects the causal structure in intended and unintended ways .◦ These alterations need to be taken into account if one wants the model to

evaluate the impact of intervention. ◦ I believe that what is needed in these circumstances is a model for causal

mechanisms.

Mechanisms matter; but, how?

Mechanisms matter in the social sciences. They help us attain epistemic and non epistemic goals by:1. Helping causal inference ;2. Facilitating extrapolation;3. Evaluating interventions.

This formulation is too vague: are mechanisms necessary and sufficient for 1, 2 and 3?

No. Whether and how they matter depends on the context. There is no general demand for mechanisms (Kincaid 1997). ◦ Sensible proposal. By itself, however, it does not tell us when and how

mechanisms matter. It simply relocates the methodological investigation and places it closer to the actual scientific practice.

What are the contextual features that make the search for mechanisms desirable?◦ Daniel Steel argues that it depends on the available information. In the

social sciences it happens sometimes that we can more easily access the mechanisms and their components than the causal relationships at the aggregate level (Steel 2008).

Contextual clues

What types of clues suggest that the search for mechanisms is desirable in the circumstances?

General clues: field of investigation, research purposes, subject of study, applicable methods, background knowledge.

More specifically, consider Heckman’s MCF for measuring intervention. (If my conclusions are correct) causal effects are irrelevant in the circumstances in which:◦ The causal structure is affected by the intervention;◦ The intervention operates by jiggling distant rather than proximate

causes of the effect of interest. These are contextual clues. As such, they

◦ do not require one to have full knowledge of the causal structure in place;◦ point out the irrelevance of a model for causal effects and the need of a

model for mechanisms. The need for clues is consistent with the idea that in the social

sciences reliable knowledge of mechanisms is not easy to obtain...

Mechanisms matter only if valid

Mechanisms help scientists attaining epistemic and non epistemic purposes under conditions:◦ They help causal inference only if they solve the problem of confounders

(spurious causal relations);◦ They help extrapolation only if they bear external validity besides being

internally valid;◦ They help assessing interventions only if they are relevant to the policy

implemented.The conditions above can be fulfilled only if mechanisms are detected correctly

This is a major issue for the social sciences because social mechanisms are not there ready to be picked up: ◦ They are not directly observable;◦ They consist of social practices that have to be interpreted (Steel 2004, 2008);◦ They are bound to uncertainty;◦ They are context-dependent;◦ Various mechanisms can operate simultaneously.

Therefore, scientists need first to find out methods that give reliable evidence of causal mechanisms if they are to use those mechanisms as valid evidence for causal and policy hypotheses.

The troubles of process tracing

Process tracing is the method for detecting mechanisms in the social sciences. ◦Whether process tracing provides valid evidence for

mechanisms and how (at what conditions) is related with how we define mechanisms in the first place;

◦ If we understand social mechanisms as regular sequences of events (Little 1991, Bunge 2004, Mayntz 2004), process tracing should test the stability in the association of events;

◦but then it would fall prey of the same problems that plague statistical inferences (spurious correlation).

◦ If we understand social mechanisms as sets of social practices (Steel 2004 2008), process tracing is to provide evidence for the existence of those practices. What we need to establish is then how social practices are to be interpreted unambiguously.

To conclude

To sum up:Social scientists make extensive use of mechanisms and

mechanistic language,And philosophers and methodologists promote the use of

mechanisms for epistemic and non epistemic purposes.However, how mechanisms actually matter and to what

extent seem to depend on the context. Moreover, they can help solve the aforementioned

problems if correctly detected. Hence, the importance of1. Identifying the relevant contextual features:

Clues that tell the researcher that the search for mechanisms is fruitful in the circumstances;

2. Identifying the conditions under which process tracing secures the validity of the mechanisms it detects.