THE MATHEMATICS OF CAUSAL MODELING Judea Pearl Department of Computer Science UCLA

THE MATHEMATICSOF CAUSAL MODELING

Judea PearlDepartment of Computer Science

UCLA

• Modeling: Statistical vs. Causal

• Causal Models and Identifiability

• Inference to three types of claims:

1. Effects of potential interventions

2. Claims about attribution (responsibility)

3. Claims about direct and indirect effects

• Robustness of Causal Claims

OUTLINE

TRADITIONAL STATISTICALINFERENCE PARADIGM

Data

Inference

Q(P)(Aspects of P)

PJoint

Distribution

e.g.,Infer whether customers who bought product Awould also buy product B.Q = P(B|A)

THE CAUSAL INFERENCEPARADIGM

Data

Inference

Q(M)(Aspects of M)

Data Generating

Model

Some Q(M) cannot be inferred from P.e.g.,Infer whether customers who bought product Awould still buy A if we were to double the price.

JointDistribution

FROM STATISTICAL TO CAUSAL ANALYSIS:1. THE DIFFERENCES

Datajoint

distribution

inferencesfrom passiveobservations

Probability and statistics deal with static relations

ProbabilityStatistics

•


Datajoint

distribution




Causal analysis deals with changes (dynamics)i.e. What remains invariant when P changes.

• P does not tell us how it ought to change

e.g. Curing symptoms vs. curing diseases e.g. Analogy: mechanical deformation


Datajoint

distribution




CausalModel

Data

Causalassumptions

1. Effects of interventions

2. Causes of effects

3. Explanations

Causal analysis deals with changes (dynamics)

Experiments

FROM STATISTICAL TO CAUSAL ANALYSIS:1. THE DIFFERENCES (CONT)

CAUSALSpurious correlationRandomizationConfounding / EffectInstrumentHolding constantExplanatory variables

STATISTICALRegressionAssociation / Independence“Controlling for” / ConditioningOdd and risk ratiosCollapsibility

1. Causal and statistical concepts do not mix.

2.

3.

4.




4.

3. Causal assumptions cannot be expressed in the mathematical language of standard statistics.


2. No causes in – no causes out (Cartwright, 1989)

statistical assumptions + datacausal assumptions causal conclusions }

4. Non-standard mathematics:a) Structural equation models (Wright, 1920; Simon, 1960)

b) Counterfactuals (Neyman-Rubin (Yx), Lewis (x Y))




3. Causal assumptions cannot be expressed in the mathematical language of standard statistics.


2. No causes in – no causes out (Cartwright, 1989)

statistical assumptions + datacausal assumptions causal conclusions }

WHAT'S IN A CAUSAL MODEL?

Oracle that assigns truth value to causalsentences:

Action sentences: B if we do A.

Counterfactuals: B would be different ifA were true.

Explanation: B occurred because of A.

Optional: with what probability?

Z

YX

INPUT OUTPUT

FAMILIAR CAUSAL MODELORACLE FOR MANIPILATION

WHY CAUSALITY NEEDS SPECIAL MATHEMATICS

Y = 2XX = 1

X = 1 Y = 2

Process information

Had X been 3, Y would be 6.If we raise X to 3, Y would be 6.Must “wipe out” X = 1.

Static information

SEM Equations are Non-algebraic:

CAUSAL MODELS ANDCAUSAL DIAGRAMS

Definition: A causal model is a 3-tupleM = V,U,F

with a mutilation operator do(x): M Mx where:

(i) V = {V1…,Vn} endogenous variables,(ii) U = {U1,…,Um} background variables(iii) F = set of n functions, fi : V \ Vi U Vi

vi = fi(pai,ui) PAi V \ Vi Ui U•

CAUSAL MODELS ANDCAUSAL DIAGRAMS




vi = fi(pai,ui) PAi V \ Vi Ui U

U1 U2I W

Q P PAQ 222

111uwdqbp

uidpbq




vi = fi(pai,ui) PAi V \ Vi Ui U(iv) Mx= U,V,Fx, X V, x X

where Fx = {fi: Vi X } {X = x}(Replace all functions fi corresponding to X with the constant

functions X=x)•

CAUSAL MODELS ANDMUTILATION





vi = fi(pai,ui) PAi V \ Vi Ui U

U1 U2I W

Q P 222

111uwdqbp

uidpbq

(iv)





vi = fi(pai,ui) PAi V \ Vi Ui U(iv)

U1 U2I W

Q P P = p0

0

222

111

pp

uwdqbp

uidpbq

Mp




vi = fi(pai,ui) PAi V \ Vi Ui U(iv) Mx= U,V,Fx, X V, x X

where Fx = {fi: Vi X } {X = x}(Replace all functions fi corresponding to X with the constant

functions X=x)

Definition (Probabilistic Causal Model): M, P(u)P(u) is a probability assignment to the variables in U.

PROBABILISTIC CAUSAL MODELS

CAUSAL MODELS AND COUNTERFACTUALS

Definition: The sentence: “Y would be y (in situation u), had X been x,” denoted Yx(u) = y, means:

The solution for Y in a mutilated model Mx, (i.e., the equations for X replaced by X = x) and U=u, is equal to y.

)(),()(,)(:

uPzZyYPzuZyuYu

wxwx

Joint probabilities of counterfactuals:•

U

D

B

C

A

S5. If the prisoner is dead, he would still be dead if A were not to have shot. DDA

3-STEPS TO COMPUTING3-STEPS TO COMPUTINGCOUNTERFACTUALSCOUNTERFACTUALS

TRUE

Abduction

TRUE

(Court order)

(Captain)

(Riflemen)

(Prisoner)

U

D

B

C

A

S5. If the prisoner is dead, he would still be dead if A were not to have shot. DDA

3-STEPS TO COMPUTING3-STEPS TO COMPUTINGCOUNTERFACTUALSCOUNTERFACTUALS

TRUE U

D

B

C

A

FALSE

TRUE

Action

TRUE

U

D

B

C

A

FALSE

TRUE

PredictionAbduction

TRUE

U

D

B

C

A

P(S5). The prisoner is dead. How likely is it that he would be dead if A were not to have shot. P(DA|D) = ?

COMPUTING PROBABILITIESCOMPUTING PROBABILITIESOF COUNTERFACTUALSOF COUNTERFACTUALS

Abduction

TRUE

Prediction

U

D

B

C

A

FALSE

P(u|D)

P(DA|D)

P(u)

Action

U

D

B

C

A

FALSE

P(u|D)

P(u|D)

CAUSAL INFERENCEMADE EASY (1985-2000)

1. Inference with Nonparametric Structural Equations made possible through Graphical Analysis.

2. Mathematical underpinning of counterfactualsthrough nonparametric structural equations

3. Graphical-Counterfactuals symbiosis

IDENTIFIABILITYIDENTIFIABILITYDefinition:Let Q(M) be any quantity defined on a causal model M, and let A be a set of assumption.

Q is identifiable relative to A iff

•

for all M1, M2, that satisfy A.

•

P(M1) = P(M2) Q(M1) = Q(M2)



In other words, Q can be determined uniquelyfrom the probability distribution P(v) of the endogenous variables, V, and assumptions A.

P(M1) = P(M2) Q(M1) = Q(M2)


•




P(M1) = P(M2) Q(M1) = Q(M2)

A: Assumptions encoded in the diagramQ1: P(y|do(x)) Causal Effect (= P(Yx=y))Q2: P(Yx=y | x, y) Probability of necessityQ3: Direct Effect)(

'xZxYE

In this talk:

THE FUNDAMENTAL THEOREMOF CAUSAL INFERENCE

Causal Markov Theorem:Any distribution generated by Markovian structural model M (recursive, with independent disturbances) can be factorized as

Where pai are the (values of) the parents of Vi in the causal diagram associated with M.

)|(),...,,( iii

n pavPvvvP 21

•

THE FUNDAMENTAL THEOREMOF CAUSAL INFERENCE

Causal Markov Theorem:Any distribution generated by Markovian structural model M (recursive, with independent disturbances) can be factorized as

Where pai are the (values of) the parents of Vi in the causal diagram associated with M.

)|(),...,,( iii

n pavPvvvP 21

xXXViiin

i

pavPxdovvvP

|)|( ))(|,...,,(|

21

Corollary: (Truncated factorization, Manipulation Theorem)The distribution generated by an intervention do(X=x)(in a Markovian model M) is given by the truncated factorization

RAMIFICATIONS OF THE FUNDAMENTAL THEOREM

U (unobserved)

X = x Z YSmoking Tar in

LungsCancer

U (unobserved)

X Z YSmoking Tar in

LungsCancer

Given P(x,y,z), should we ban smoking?

Pre-intervention Post-interventionu

uzyPxzPuxPuPzyxP ),|()|()|()(),,( u

uzyPxzPuPxdozyP ),|()|()())(|,(

•


U (unobserved)


LungsCancer

U (unobserved)

X Z YSmoking Tar in

LungsCancer


Pre-intervention Post-interventionu

uzyPxzPuxPuPzyxP ),|()|()|()(),,( u

uzyPxzPuPxdozyP ),|()|()())(|,(

To compute P(y,z|do(x)), we must eliminate u. (Graphical problem.)

THE BACK-DOOR CRITERIONGraphical test of identificationP(y | do(x)) is identifiable in G if there is a set Z ofvariables such that Z d-separates X from Y in Gx.

Z6

Z3

Z2

Z5

Z1

X Y

Z4

Z6

Z3

Z2

Z5

Z1

X Y

Z4

Z

•

Gx G

Pre-intervention Post-intervention


U (unobserved)


LungsCancer

U (unobserved)

X Z YSmoking Tar in

LungsCancer


•

THE BACK-DOOR CRITERIONGraphical test of identificationP(y | do(x)) is identifiable in G if there is a set Z ofvariables such that Z d-separates X from Y in Gx.

Z6

Z3

Z2

Z5

Z1

X Y

Z4

Z6

Z3

Z2

Z5

Z1

X Y

Z4

Z

Moreover, P(y | do(x)) = P(y | x,z) P(z)(“adjusting” for Z) z

Gx G

RULES OF CAUSAL CALCULUSRULES OF CAUSAL CALCULUS

Rule 1: Ignoring observations P(y | do{x}, z, w) = P(y | do{x}, w)

Rule 2: Action/observation exchange P(y | do{x}, do{z}, w) = P(y | do{x},z,w)

Rule 3: Ignoring actions P(y | do{x}, do{z}, w) = P(y | do{x}, w)

XG Z|X,WY )( if

Z(W)XGZ|X,WY )( if

ZXGZ|X,WY )( if

DERIVATION IN CAUSAL CALCULUSDERIVATION IN CAUSAL CALCULUS

Smoking Tar Cancer

P (c | do{s}) = t P (c | do{s}, t) P (t | do{s})

= st P (c | do{t}, s) P (s | do{t}) P(t |s)

= t P (c | do{s}, do{t}) P (t | do{s})

= t P (c | do{s}, do{t}) P (t | s)

= t P (c | do{t}) P (t | s)

= s t P (c | t, s) P (s) P(t |s)

= st P (c | t, s) P (s | do{t}) P(t |s)

Probability Axioms

Probability Axioms

Rule 2

Rule 2

Rule 3

Rule 3

Rule 2

Genotype (Unobserved)

RECENT RESULTS ON IDENTIFICATION

Theorem (Tian 2002):

We can identify P(v | do{x}) (x a singleton)

if and only if there is no child Z of X connected

to X by a bi-directed path.

X

Z Z

Z

k

1

• Do-calculus is complete• A complete graphical criterion available

for identifying causal effects of any set on any set

• References: Shpitser and Pearl 2006 (AAAI, UAI)

RECENT RESULTS ON IDENTIFICATION (Cont.)

OUTLINE


• Causal models and identifiability


1. Effects of potential interventions,


3.

DETERMINING THE CAUSES OF EFFECTS(The Attribution Problem)

• Your Honor! My client (Mr. A) died BECAUSE he used that drug.

•

DETERMINING THE CAUSES OF EFFECTS(The Attribution Problem)

• Your Honor! My client (Mr. A) died BECAUSE he used that drug.

• Court to decide if it is MORE PROBABLE THANNOT that A would be alive BUT FOR the drug!

P(? | A is dead, took the drug) > 0.50

THE PROBLEM

Theoretical Problems:

1. What is the meaning of PN(x,y):“Probability that event y would not have occurred if it were not for event x, given that x and y did in fact occur.”

•

THE PROBLEM



Answer:

),(),,'(

),|'(),(

'

'

yYxXPyYxXyYP

yxyYPyxPN

x

x

THE PROBLEM



2. Under what condition can PN(x,y) be learned from statistical data, i.e., observational, experimental and combined.

WHAT IS INFERABLE FROM EXPERIMENTS?

Simple Experiment:Q = P(Yx= y | z)Z nondescendants of X.

Compound Experiment:Q = P(YX(z) = y | z)

Multi-Stage Experiment:etc…

CAUSAL INFERENCEMADE EASY (1985-2000)

1. Inference with Nonparametric Structural Equations made possible through Graphical Analysis.

2. Mathematical underpinning of counterfactualsthrough nonparametric structural equations

3. Graphical-Counterfactuals symbiosis

AXIOMS OF CAUSAL COUNTERFACTUALS

1. Definiteness

2. Uniqueness

3. Effectiveness

4. Composition

5. Reversibility

xuXtsXx y )( ..

')')((&))(( xxxuXxuX yy

xuX xw )(

)()()( uYuYwuW xxwx

yuYwuWyuY xxyxw )())((&)((

:)( yuYx Y would be y, had X been x (in state U = u)

GRAPHICAL – COUNTERFACTUALS SYMBIOSIS

Every causal model implies constraints on counterfactuals

e.g.,

XZY

uYuY

yx

xzx

|

)()(,

consistent, and readable from the graph.

Every theorem in SEM is a theorem in N-R, and conversely.

CAN FREQUENCY DATA DECIDE CAN FREQUENCY DATA DECIDE LEGAL RESPONSIBILITY?LEGAL RESPONSIBILITY?

• Nonexperimental data: drug usage predicts longer life• Experimental data: drug has negligible effect on survival

Experimental Nonexperimental do(x) do(x) x x

Deaths (y) 16 14 2 28Survivals (y) 984 986 998 972

1,000 1,000 1,000 1,000

1. He actually died2. He used the drug by choice

500.),|'( ' yxyYPPN x

• Court to decide (given both data): Is it more probable than not that A would be alive but for the drug?

• Plaintiff: Mr. A is special.

TYPICAL THEOREMS(Tian and Pearl, 2000)

• Bounds given combined nonexperimental and experimental data

)()(

1

min)(

)()(

0

maxx,yPy'P

PN x,yP

yPyP x'x'

)()()(

)()()(

x,yPyPy|x'P

y|xPy|x'Py|xP

PN x'

• Identifiability under monotonicity (Combined data)

corrected Excess-Risk-Ratio

SOLUTION TO THE ATTRIBUTION SOLUTION TO THE ATTRIBUTION PROBLEM (Cont)PROBLEM (Cont)

• WITH PROBABILITY ONE P(yx | x,y) =1

• From population data to individual case• Combined data tell more that each study alone

OUTLINE


• Causal models and identifiability


1. Effects of potential interventions,



•

QUESTIONS ADDRESSED

• What is the semantics of direct and

indirect effects?

• Can we estimate them from data? Experimental data?

1. Direct (or indirect) effect may be more transportable.2. Indirect effects may be prevented or controlled.

3. Direct (or indirect) effect may be forbidden

WHY DECOMPOSEEFFECTS?

Pill

Thrombosis

Pregnancy

+

+

Gender

Hiring

Qualification

z = f (x, 1)y = g (x, z, 2)

X Z

Y

THE OPERATIONAL MEANING OFDIRECT EFFECTS

“Natural” Direct Effect of X on Y:The expected change in Y per unit change of X, when we keep Z constant at whatever value it attains before the change.

In linear models, NDE = Controlled Direct Effect

][001 xZx YYE

x

z = f (x, 1)y = g (x, z, 2)

X Z

Y

THE OPERATIONAL MEANING OFINDIRECT EFFECTS

“Natural” Indirect Effect of X on Y:The expected change in Y when we keep X constant, say at x0, and let Z change to whatever value it would have under a unit change in X.

In linear models, NIE = TE - DE

][010 xZx YYE

x

POLICY IMPLICATIONS(Who cares?)

f

GENDER QUALIFICATION

HIRING

What is the direct effect of X on Y?

The effect of Gender on Hiring if sex discrimination is eliminated.

indirect

X Z

Y

IGNORE

SEMANTICS AND IDENTIFICATION OF NESTED COUNTERFACTUALS

Consider the quantity

Given M, P(u), Q is well defined

Given u, Zx*(u) is the solution for Z in Mx*, call it z

is the solution for Y in Mxz

Can Q be estimated from data?

)]([ )(*uYEQ uxZxu

entalnonexperim

alexperiment

)()(*uY uxZx

Example:

Theorem: If there exists a set W such that

GRAPHICAL CONDITION FOR EXPERIMENTAL IDENTIFICATION

OF AVERAGE NATURAL DIRECT EFFECTS

zw

xzxxz wPwzZPwYEwYEYxxNDE,

** )()|()|()|()*;,(

)()|( ZXNDWWZYXZG and

Corollary 3:The average natural direct effect in Markovian models is identifiable from nonexperimental data, and it is given by

IDENTIFICATION INMARKOVIAN MODELS

X Z

Y

z

xzPzxyEzxYEYxxNDE *)|()*,|(),|()*;,(

z x zZPzxYEzxYEYxxNDE )()]*,|(),|([)*;,( *

THE OVERRIDING THEME

1. Define Q(M) as a counterfactual expression2. Determine conditions for the reduction

3. If reduction is feasible, Q is inferable.

• Demonstrated on three types of queries:

)()()()( exp MPMQMPMQ or

Q1: P(y|do(x)) Causal Effect (= P(Yx=y))Q2: P(Yx = y | x, y) Probability of necessityQ3: Direct Effect)(

'xZxYE

CONCLUSIONS

Structural-model semantics enriched

with logic + graphs leads to formal

interpretation and practical assessments

of wide variety of causal and counterfactual

relationships.


• Causal Models and Identifiability


1. Effects of potential interventions



• Actual Causation and Explanation

• Robustness of Causal Claims

OUTLINE

ROBUSTNESS:MOTIVATION

Smoking

x y

Genetic Factors (unobserved)

Cancer

u

In linear systems: y = x + u cov (x,u) = 0 is identifiable. = Ryx


The claim Ryx is sensitive to the assumption cov (x,u) = 0.

Smoking

x


Cancer

is non-identifiable if cov (x,u) ≠ 0.

y

u


Z – Instrumental variable; cov(z,u) = 0

Smoking

y


Cancer

u

x

ZPrice ofCigarettes

xz

yz

xz

yzR

R

R

R

is identifiable, even if cov (x,u) ≠ 0


Smoking

y


Cancer

u

x

ZPrice ofCigarettes

10

10

Suppose

xz

yzyx R

RR

Claim “ = Ryx” is likely to be true

Smoking


Z1

Price ofCigarettes

Invoking several instruments

If =1 = 2, claim “ = 0” is more likely correct

2

22

1

10

xz

yz

xz

yzyx R

R

R

RR 1

x y


Cancer

u

PeerPressure

Z2


Z1

Price ofCigarettes

x y


Cancer

u

PeerPressure

Z2

Smoking

Greater surprise: 1 = 2 = 3….= n = qClaim = q is highly likely to be correct

Z3

Zn

Anti-smoking Legislation


Assume we have several independent estimands of , and

x y

Given a parameter in a general graph

Find the degree to which is robust to violations of model assumptions

1 = 2 = …n

ROBUSTNESS:ATTEMPTED FORMULATION

Bad attempt: Parameter is robust (over-identified)

f1, f2: Two distinct functions

)()( 21 ff

distinct. are

then constraint induces model if

)]([

)]([)()]([)(

,0)(

21

gt

gtfgtf

g

i

if:


x y


Cancer

u

Smoking

Is robust if 0 = 1?

sxsysy

sx

yx

RRRR

R

,1

0

s

Symptom


x y


Cancer

u

Smoking

Symptoms do not act as instruments

remains non-identifiable if cov (x,u) ≠ 0

s

Symptom

Why? Taking a noisy measurement (s) of an observed variable (y) cannot add new information


x


Cancer

u

Smoking

Adding many symptoms does not help.

remains non-identifiable

ySymptom

S1

S2

Sn

INDEPENDENT:BASED ON DISTINCT SETS OF ASSUMPTION

u

z yx

u

zyx

EstimandEstimand AssumptiomsAssumptioms

xz

yz

yx

R

R

R

1

0

others

0),cov( ux

EstimandEstimand AssumptiomsAssumptioms

zy

zx

yx

RR

R

1

0

others

0),cov(

0),cov(

ux

ux

RELEVANCE:FORMULATION

Definition 8 Let A be an assumption embodied in model M, and p a parameter in M. A is said to be relevant to p if and only if there exists a set of assumptions S in M such that S and A sustain the identification of p but S alone does not sustain such identification.

Theorem 2 An assumption A is relevant to p if and only if A is a member of a minimal set of assumptions sufficient for identifying p.

ROBUSTNESS:FORMULATION

Definition 5 (Degree of over-identification)A parameter p (of model M) is identified to degree k (read: k-identified) if there are k minimal sets of assumptions each yielding a distinct estimand of p.

ROBUSTNESS:FORMULATION

x y

b

z

c

Minimal assumption sets for c.

xy z

c xy z

c

G3G2

xy z

c

G1

Minimal assumption sets for b. xy

bz

FROM MINIMAL ASSUMPTION SETS TO MAXIMAL EDGE SUPERGRAPHS

FROM PARAMETERS TO CLAIMS

DefinitionA claim C is identified to degree k in model M (graph G), if there are k edge supergraphs of G that permit the identification of C, each yielding a distinct estimand.

TE(x,z) = Rzx TE(x,z) = Rzx Rzy ·x

xy zx

y z

e.g., Claim: (Total effect) TE(x,z) = q x y z

FROM MINIMAL ASSUMPTION SETS TO MAXIMAL EDGE SUPERGRAPHS

FROM PARAMETERS TO CLAIMS

DefinitionA claim C is identified to degree k in model M (graph G), if there are k edge supergraphs of G that permit the identification of C, each yielding a distinct estimand.

xy zx

y z

e.g., Claim: (Total effect) TE(x,z) = q x y z

Nonparametric y x

xPyxzPxyPxzTExzPzxTE'

)'(),'|()|(),()|(),(

SUMMARY OF ROBUSTNESS RESULTS

1. Formal definition to ROBUSTNESS of causal claims:“A claim is robust when it is insensitive to

violations of some of the model assumptions relevant to substantiating that claim.”

2. Graphical criteria and algorithms for computing the degree of robustness of a given causal claim.

Documents

THE MATHEMATICS OF CAUSAL MODELING Judea Pearl Department of Computer Science UCLA