Research Design: Causal inference and counterfactuals › ... › uploads ›...

IntroductionCounterfactuals

ConfoundingCriteria

Research Design:

Causal inference and counterfactuals

Johan A. Elkink

University College Dublin

8 March 2013

Johan A. Elkink counterfactual causal inference

ConfoundingCriteria

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

Inference

In regression analysis we look at the relationship between (a set of)independent variable(s) and a dependent variable.

Statistical inference is concerned with the question how likely it is toobserve this relationship given the null hypothesis of no relationship(frequentist)

A different question is whether or not we can deduce that the

independent variable is a cause of the dependent one.

(Note: references in these slides can be found in the associated handout.)

ConfoundingCriteria

Inference

In regression analysis we look at the relationship between (a set of)independent variable(s) and a dependent variable.

Statistical inference is concerned with the question how likely it is toobserve this relationship given the null hypothesis of no relationship(frequentist) or how much we should update our beliefs concerning thisrelationship given our new evidence (Bayesian).

A different question is whether or not we can deduce that the

independent variable is a cause of the dependent one.

(Note: references in these slides can be found in the associated handout.)

ConfoundingCriteria

Association

association 6= causation

ConfoundingCriteria

Association

Given that, say, X and Y are correlation (associated), there arestill many possible causal patterns at play.

ConfoundingCriteria

Many possible patterns

ConfoundingCriteria

Inference

Generally, to make causal inferences from your analysis, additionalassumptions need to be made in addition to the ones already madefor associational or predictive inference.

ConfoundingCriteria

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

Fundamental problem

Imagine, there are two kinds of people, one group, T = 1, that has acollege degree, and another group, T = 0, that does not. We want tomeasure where a college degree leads to a higher salary, Y .

ConfoundingCriteria

Fundamental problem

Imagine, there are two kinds of people, one group, T = 1, that has acollege degree, and another group, T = 0, that does not. We want tomeasure where a college degree leads to a higher salary, Y .

What we would like to know is the difference for any individual i whether

they have a college degree or not: Y Ti=1i

− Y Ti=0i

. However, for every

individual i , we either observe Y Ti=1i

, or we observe Y Ti=0i

– they either

have the degree or they don’t.

ConfoundingCriteria

We wish ...

Respondent Degree Y Ti=0i

Y Ti=1i

effect

1 Yes 121 133 +122 Yes 100 109 +93 No 90 92 +24 No 87 88 +15 Yes 143 146 +36 Yes 111 124 +137 No 92 92 08 Yes 95 109 +14

ConfoundingCriteria

We wish ... we have ...

Respondent Degree Y Ti=0i

Y Ti=1i

effect

1 Yes 1332 Yes 1093 No 904 No 875 Yes 1466 Yes 1247 No 928 Yes 109

ConfoundingCriteria

Potential outcomes

Potential outcome =

Y1i if Ti = 1Y0i if Ti = 0

E.g., Y1i is the salary of individual i had (s)he a college degree,irrespective of whether (s)he actually does.

(Angrist & Pischke 2009, 13-14)

ConfoundingCriteria

Potential outcomes

Potential outcome =

Y1i if Ti = 1Y0i if Ti = 0

E.g., Y1i is the salary of individual i had (s)he a college degree,irrespective of whether (s)he actually does.

Yi = Y0i + (Y1i − Y0i )Ti = Y0i + δTi ,

where δ = Y1i − Y0i is the causal effect.

(Angrist & Pischke 2009, 13-14)

ConfoundingCriteria

Average treatment effect

Because it is impossible to observe individual treatment effect, we usuallyturn to average treatment effect:

E [δ] = E [Y1i − Y0i ] = E [Y1i ]− E [Y0i ],

which we could naively estimate with

δ̂ = E [Y1i |Ti = 1]− E [Y0i |Ti = 0].

ConfoundingCriteria

Average treatment effect

Because it is impossible to observe individual treatment effect, we usuallyturn to average treatment effect:

E [δ] = E [Y1i − Y0i ] = E [Y1i ]− E [Y0i ],

which we could naively estimate with

δ̂ = E [Y1i |Ti = 1]− E [Y0i |Ti = 0].

This assumes that E [Y1i ] reflects the salary for people with a college

degree, irrespective of whether they got one or not, and that E [Y0i ]

reflects the salary without a college degree, irrespective of whether they

got one or not.

ConfoundingCriteria

Counterfactual causality

By making such assumptions – by looking at the ATE – we aremaking a counterfactual argument. We are making assumptions ofwhat Y1i would have been, had i had a college degree.

ConfoundingCriteria

This implies that we cannot measure a causal effect, only estimateit.

ConfoundingCriteria

This implies that we cannot measure a causal effect, only estimateit.

To understand when the ATE assumptions are reasonable, we needto look at the effect of covariates – other variables that relate toY , which we will denote by X.

ConfoundingCriteria

Treatment effect: abbreviations

ATE Average Treatment Effect E [δ] = E [Y1i − Y0i ]ATT ATE for the Treated E [δT ] = E [Y1i − Y0i |Ti = 1]ATC ATE for the Control (untreated) E [δC ] = E [Y1i − Y0i |Ti = 0]

PATE Population ATE E [Y1i − Y0i ]SATE Sample ATE En[Y1i − Y0i ]LATE Local ATE E [Y1i − Y0i |Xi = x ]CATE Conditional ATE E [Y1i − Y0i |Xi = x ]

and analogously we have LATT, PATT, SATC, etceta. Note that ATE,ATT, ATC implicitly refer to population values.

En[·] is the sample mean, i.e. En[x ] = x̄ = 1n

i=1, where the En allowsfor the formulation of conditional and counterfactual means.

ConfoundingCriteria

Bias in causal inference

Using shorthand E01 = E [Y0i |Ti = 1], etc., and taking π as thepopulation proportion that received the treatment,

E [δ] = πE [δ|Ti = 1] + (1− π)E [δ|Ti = 0]

= π(E11 − E01) + (1− π)(E10 − E00)

ConfoundingCriteria

E [δ] = πE [δ|Ti = 1] + (1− π)E [δ|Ti = 0]

= π(E11 − E01) + (1− π)(E10 − E00)

can be decomposed into

(E11 − E00) = E [δ] + (E01 − E00) + (1− π){(E11 − E01)− (E10 − E00)}.

ConfoundingCriteria

E [δ] = πE [δ|Ti = 1] + (1− π)E [δ|Ti = 0]

= π(E11 − E01) + (1− π)(E10 − E00)

can be decomposed into

(E11 − E00) = E [δ] + (E01 − E00) + (1− π){(E11 − E01)− (E10 − E00)}.

(E11 − E00) observed difference in effectE [δ] average treatment effect

(E01 − E00) selection bias(1− π){(E11 − E01)− (E10 − E00)} differential treatment effect bias

(Morgan & Winship 2007

ConfoundingCriteria

The stable unit treatment value assumption “SUTVA is simply thea priori assumption that the value of Y for unit i when exposed totreatment t will be the same no matter what mechanism is used toassign treatment t to unit i and no matter what treatments theother units receive.”

(Rubin (1986: 961), as cited in Morgan & Winship (2007: 37))

ConfoundingCriteria

When to control?How to control?

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

Confounding

When studying effect of, say, T on Y , by examining the statisticalassociation between the two variables, we need to ascertain that theobserved effect is not caused by a third variable, say, X.

(Pearl 2000: 182-183)

ConfoundingCriteria

Confounding

When studying effect of, say, T on Y , by examining the statisticalassociation between the two variables, we need to ascertain that theobserved effect is not caused by a third variable, say, X.

“We can say that T and Y are confounded when there is a third variable

X that influences both T and Y ; such a variable is then called a

confounder of T and Y .”

(Pearl 2000: 182-183)

ConfoundingCriteria

Confounding

Another way of saying this is that if

E (Y |T ,X ) 6= E (Y |T )

andE (T |X ) 6= E (T ),

X is a confounder of the effect of T on Y .

(Lee 2005: 44)

ConfoundingCriteria

Confounding

If healthier patients take a drug and sicker patients do not, wecan find an association between drug and recovery even whenthe drug does not work.

If sicker patients take a drug and healthier patients do not, wemight not find an association between drug and recovery evenwhen the drug works.

association 6= causation

The first example is also called a spurious effect (not to beconfused with spurious regression).

ConfoundingCriteria

Endogeneity

The situation where cor(X, ε) 6= 0 is called endogeneity.

Endogeneity has three main causes:

Measurement error in X

Simultaneity or reverse causation

Omitted variables

ConfoundingCriteria

Confounding

Note that confounding is a causal concept, not an associationalone!

(Pearl 2000)

ConfoundingCriteria

Confounding

Note that confounding is a causal concept, not an associationalone!

X has to have a causal effect on T and X has to have a causaleffect on Y for there to be an issue.

(Pearl 2000)

ConfoundingCriteria

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

When to control?

X affects both T and Y =⇒ control

(Lee 2005: 43-48)

ConfoundingCriteria

Do control

This is the typical case of a confounding factor, and hence shouldbe eliminated through controlling.

ConfoundingCriteria

Do control

ConfoundingCriteria

When to control?

T affects Y , which in turn affects X =⇒ do not control

(Lee 2005: 43-48)

ConfoundingCriteria

Don’t control

ConfoundingCriteria

Don’t control

In this case, X is an effect of Y . By controlling for X , you canseverily underestimate the effect of T on Y .

ConfoundingCriteria

Don’t control

In this case, X is an effect of Y . By controlling for X , you canseverily underestimate the effect of T on Y .

Imagine that a college degree leads to a better income leads to anicer car. Controlling for the price of the car in estimating theeffect of having a college degree on income might cancel the effect.

ConfoundingCriteria

When to control?

T affects X , which in turn affects Y =⇒ do not control ...

(Lee 2005: 43-48)

ConfoundingCriteria

Don’t control

ConfoundingCriteria

Don’t control

To get the overall effect of T on Y , you want to include the effectthrough X .

ConfoundingCriteria

Don’t control

To get the overall effect of T on Y , you want to include the effectthrough X .

E.g. if you want to know the effect of changing the policyregarding smoking in pubs on the amount of smoking in general,you do not care through what mechanism this happened (throughpeer pressure, laziness, etc.), but only about the overall effect.

ConfoundingCriteria

When to control?

... unless you explicitly want only the direct effect

(Lee 2005: 43-48)

ConfoundingCriteria

Maybe control

ConfoundingCriteria

Maybe control

Remember the following equation:

β∗ = β + φγ

ConfoundingCriteria

Maybe control

Remember the following equation:

β∗ = β + φγ

Sometimes you are interested in β (so control), sometimes in β∗

(so don’t control).

ConfoundingCriteria

Maybe control

Example: A scholarship for poorer students might help them to get acollege degree, which in turn might help them to earn more money laterin life.

ConfoundingCriteria

Maybe control

Example: A scholarship for poorer students might help them to get acollege degree, which in turn might help them to earn more money laterin life. Having a scholarship on your CV, however, might also further yourcareer, independent of the effect of having a college degree.

ConfoundingCriteria

Maybe control

To see the overall effect of the scholarship, don’t control on having acollege degree.

ConfoundingCriteria

Maybe control

To see the overall effect of the scholarship, don’t control on having acollege degree.

To see the effect of having a scholarship, independent of the effect of

getting a college degree, do control for college degree.

ConfoundingCriteria

When to control?

X affects Y , but not T , nor the effect of T on Y

(Lee 2005: 43-48)

ConfoundingCriteria

Maybe control

ConfoundingCriteria

Maybe control

When X affects Y , but not T , there is no confounding issue andthe estimates for the effect of T on Y should not be affected byinclusion of X . However, including X in the model can still help forefficiency.

(Gelman & Hill 2007: 177)

ConfoundingCriteria

When to control?

X affects Y , but not T , nor the effect of T on Y

X affects Y , not T , but it does affect of effect of T on Y (interaction)

(Lee 2005: 43-48)

ConfoundingCriteria

Maybe control

Here including the interaction in your model can highlight how theeffect is different for different groups.

ConfoundingCriteria

Maybe control

Here including the interaction in your model can highlight how theeffect is different for different groups.

Note that it affects the interpretation, but that the estimation ofthe overall ATE is not affected by controlling for X .

ConfoundingCriteria

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

The ideal experiment

To avoid any effect of covariates the ideal is to randomly selectparticipants for your research from the overal population (enablesinference to the population) and to randomly assign the treatmentto these participants (enables causal inference).

ConfoundingCriteria

How to control?

Experiment

Field experiment

Natural experiment

Blocking

Matching

Regression

ConfoundingCriteria

Kitchen sink

A typical approach in the quantitative social sciences is to collect anumber of different theories / hypotheses, add them all asvariables to a regression, and see “who wins”. This is the kitchensink approach (or garbage can approach).

If anything, the above discussion should have made clear that todraw causal inferences, a clear distinction of treatment fromcovariates is crucial. In other words: focus your research!

ConfoundingCriteria

Kitchen sink

A typical approach in the quantitative social sciences is to collect anumber of different theories / hypotheses, add them all asvariables to a regression, and see “who wins”. This is the kitchensink approach (or garbage can approach).

If anything, the above discussion should have made clear that todraw causal inferences, a clear distinction of treatment fromcovariates is crucial. In other words: focus your research!

(Note that the “garbage can” phrase has also been used to argue against

ignoring nonlinearities (Achen 2005), as opposed to careless specification of the

causal effect.)

ConfoundingCriteria

Kitchen sink

Another way of putting the issue is that the above is all abouttrying to study the effect of a cause (treatment), rather than thecause of an effect. The latter is perhaps ill-defined and runs intothe “infinite regress of causation”.

(See Gerring (2001, 2012) for an extensive discussion of Y -centered and

X -centered research.)

(Gelman & Hill 2007: 187)

ConfoundingCriteria

Causal diagrams

The preceding examples underline how it is important to alwaysdraw out the causal diagram and consider carefully how you selectcases and select controls when making causal inferences.

ConfoundingCriteria

Equifinality

Equifinality refers to the situation where a particular outcomemight come about through different causal paths. No particularpath might show a strong association between independent anddependent variable.

ConfoundingCriteria

Equifinality

Equifinality refers to the situation where a particular outcomemight come about through different causal paths. No particularpath might show a strong association between independent anddependent variable.

Should this be seen as problematic for the counterfactual causalinference framework?

ConfoundingCriteria

Outline

1 Introduction

2 Counterfactuals

3 Confounding

When to control?

How to control?

4 Criteria

ConfoundingCriteria

KKV’s 5 rules

1 Construct falsifiable theories

2 Build theories that are internally consistent3 Select dependent variables carefully

No endogenous relationshipEnsure variation in dependent variable

4 Maximize concreteness

5 State theories as encompassing as feasible

(King, Keohane & Verba 1994: 100-114)

ConfoundingCriteria

Gerring’s criteria

Clarity

Manipulability

Separation

Independence (priority)

Impact

Mechanism

ConfoundingCriteria

Discussion points

How do causal inference and prediction relate? (“The proof ofthe pudding is in the eating?”)

ConfoundingCriteria

Discussion points

How does the theory thus far translate to qualitative research?Does it?

ConfoundingCriteria

Discussion points

Are all causal inferences counterfactual?

ConfoundingCriteria

Discussion points

Are all causal inferences counterfactual?

How does this all relate to causal mechanisms in the sense ofHedstrom & Swedberg?

Research Design: Causal inference and counterfactuals › ... › uploads ›...

Documents

Causal Inference in Education Policy Research · 2020-02-28 · Causal Inference Stapleton, McNeish & Mao Ñ 1 Causal Inference in Education Policy Research MLDS Research Series December

Causal inference using regression on the treatment variablegelman/arm/chap9.pdf · CHAPTER 9 Causal inference using regression on the treatment variable 9.1 Causal inference and predictive

Causality & causal inference - TeachEpi

Causal Inference v16

Causal Inference and Data-Fusion in Econometrics · 2019-03-29 · Causal Inference in Econometrics I Despite a strong interest in causal inference in general, graphical models of

Introducing Counterfactual Causal Inference

Module . s: Causal Inference - edX€¦ · CAUSAL INFERENCE AND IMPACT EVALUATION Causal inference and impact evaluation is all about attributing a change in an outcome of interest

Lecture 1: Causal Inference in Social Science€¦ · Outlines 1 Course Overview 2 Causal Inference in Social Science Causal Inference: The Core of Empirical Studies in Economics

Origo: causal inference by compression · 2018. 6. 29. · Origo: causal inference by compression 287 – a practical framework for causal inference based on MDL, – a causal inference

Causal Inference

Causal Inference Overview - Harvard University · Fundamental Problem of Causal Inference (ctd). I The fundamental problem of causal inference is that at most only one of the two

Representation Learning for Causal Inference AAAI …cobweb.cs.uga.edu/~shengli/Docs/AAAI-20-Causal-Inference...Representation Learning for Causal Inference Sheng Li1, Liuyi Yao2,

Principles of Causal Inference

Causal Inference Woth Observational Data

Introduction to causal inference

Causal Inference and Machine Learning - uni-potsdam.de · 2019-09-09 · Causal Inference Course 1 September 2019 Potsdam Causal Inference and Machine Learning Guido Imbens, imbens@stanford.edu

Introduction to Causal Bayesian Inference Chris Holmes ... · Causal Bayesian Inference 3 Overview of Course – Introduction to causal inference and statistical association – The

Causal Counterfactuals and Impossible Worlds

Statistics and Causal Inference Author(s): Paul W. Holland ...blei/fogm/2020F/readings/Holland...Statistics and Causal Inference PAUL W. HOLLAND* Problems involving causal inference

CompSci 295, Causal Inference